3. Describing research data (documentation, metadata)

From the very beginning of a research project a careful documentation and description of research data is essential. It ensures that the data remain understandable both to the researchers themselves and to others, reduces the risk of misinterpretation, and significantly promotes data reusability. Describing research data is therefore key to putting the FAIR principles into practice. Without proper description and documentation, research data may become merely a collection of disconnected files, numbers, and characters, making the data difficult to interpret, use, and reuse.

Describing and documenting research data throughout the entire life cycle is a core part of everyday work in a research project, even if the data are not ultimately made openly available. Investing time in documentation and description during the project saves time later when the data are published. Sharing descriptive information is always recommended to increase the visibility of the research, even when the research data themselves cannot be made openly available for a reason.

Some descriptive metadata can be generated automatically, for example by the instruments used, but in all research, metadata must be planned and created actively. Descriptive information can be recorded in laboratory or research notebooks, systematically organised files and directories, or other structured notes.

Documentation, metadata, paradata?

At the core of data description, we have two basic terms: documentation and metadata. These terms do not necessarily have precise definitions, and they may be used somewhat differently in different guidelines. In addition, the term paradata is occasionally used. Together, these terms refer to information describing the origin, structure, content, and processing data.

Data documentation is typically understood as the detailed recording of the research process during a study, in order to enhance the transparency and reproducibility of the research. It can be produced, for example, using common office software (such as MS Word or LibreOffice Writer) or discipline-specific electronic laboratory notebooks (ELN), in which essential information about the stages of data processing is recorded. On some cases, it may be more practical to make notes by hand; if so, their conversion into digital format should be planned and carried out with risk management in mind (see Chapter 2).

Metadata refers basically to both administrative and content-related descriptions of a dataset. Metadata may include information on the origin and purpose of data collection, temporal coverage, geographical location, equipment and software used, creator, storage location, terms of use, and contact persons for further information. In addition, metadata can describe folder structure, file-naming conventions, variables within the dataset, and the terminology used.

In the context of open science, The Helsinki Term Bank for the Arts and Sciences defines metadata as data that describe data in summarised form, such as their content, structure, and context, and notes that metadata are particularly important for the retrieval and use of data. Metadata also often refers to standardised and structured information, which will be discussed further in the next section.

Paradata refers to observational information related to the data collection process or its context. This may include, for example, the time a respondent takes to answer survey questions, the interviewer’s observations during an interview, or events taking place at the time of data generation or collection. The term paradata was introduced in the context of survey research and statistics in the late 1990s.

Standards

Research disciplines may have their own established practices for describing data, which may not be directly applicable to other fields. However, there are certain key elements of data description that apply across all disciplines. Taking these into account helps to implement the FAIR principles. Above all, they support findability and help ensure that data can be reused in different contexts. In the national Guidelines for describing research data, these key elements are compiled into a clear table that also indicates which FAIR principle each element is particularly related to and who or which service is responsible for providing the information.

In this context, metadata standards are commonly referred to. They are commonly agreed frameworks for organising metadata in a uniform and machine-actionable way. Machine-actionability refers to information that can be readily processed by computers.

The easiest way to apply metadata standards is to complete the structured e-form of a data repository or a description service when the form includes mandatory fields (see, for example, Qvain tool and Etsin search service below). The form guides the user to provide specific information, such as the title, creator, date, and location of a resource, in a standardised way. This ensures that the resulting metadata are in a consistent format. One concrete example is that the creator’s name may be required to follow a specific structure, such as “Last name, First name” or “Last name First name”.

There are several types of metadata standards available, including the generic and widely used Dublin Core (DC), the standard for geographic information ISO 19115, and the standard for the worldwide exchange and communication of date- and time-related data ISO 8601. Metadata standards also vary across research fields and disciplines; researchers can find more information about disciplinary metadata standards on websites hosted by the Digital Curation Centre (DCC) and the Research Data Alliance (RDA).

Data repositories may follow discipline-specific metadata standards. For example, the Finnish Social Science Data Archive uses the DDI metadata standard (Data Documentation Initiative), which was developed especially for social science data.

In addition, equipment used in a project may automatically produce standardised metadata. For this reason, it is important to include information about the equipment and software used in the descriptive information on the research data.

README file: The key to a dataset

A README file is a structured and concise text-based description of a research dataset, including code and software. It is attached to the dataset and/or its metadata in such a way that it stands out clearly, serving as a kind of entry point or key to the dataset itself.

A README file should be saved in a simple, software-independent format, such as a plain text file (.txt) or a Markdown file (.md). Of these, Markdown allows some formatting and works well, for example, in Git environments. A plain text file, in turn, can be opened in different computing environments and is easy to create and transfer. Files created with word processing software are software-dependent, may not remain usable over time, and often contain unnecessary formatting that may change when opened with different programs.

The README file name should be kept concise and should begin with the word README, for example: README_[dataset name, subfolder name, etc. as needed]_[file format, e.g. .txt].

There is no strict content standard for a README file, but a useful rule of thumb is to provide key information at three different levels:

Project level: information about the research project provides the context of the data
File level: details about the files describe the data themselves
Variable or vocabulary level: explanations of the data’s variables or terms make interpretation possible

In practice, a checklist for a README file could include the following:

Basic information: name of the data, persistent identifier, contact details, creator information, including ORCID and organisation/affiliation, brief description of the contents, date, location, funder, collaborators
Data use and citation: restrictions or licence information, terms of use, publications related to the data, such as articles or theses, citation instructions, references to reused datasets, if applicable
Overview of folders and files: list of files and folders with brief descriptions, explanation of the relationships between the files
Data-specific information: variables, units of measurement, terms, explanations of codes, notes
Methodological information: data collection, processing, analysis, instruments, software

Some examples of README-files with templates:

Organising and naming files

High-quality data are well organised, structured, named, and versioned. Clear file names and folder structures make it easier to find data files and keep track of them. At the beginning of a research project, you should design a folder structure that makes information easy to find and develop a consistent and informative file-naming convention.

This naming system should be used consistently throughout the project. It is also important to document the system and save the documentation in a location where it can be easily found. This contributes to the comprehensibility of the data and is an essential part of data documentation

It is recommended to document the following information for each file:

file name
project abbreviation or acronym
file location (file path)
file size
file format
software used to create the file
date of creation
file creator
file version
file access rights

Tips for file naming:

create meaningful but concise names
use file names to indicate broad file types or categories
avoid spaces and special characters
if a date is included in the file name, always write it in the order year, month, day: YYYYMMDD (e.g. 20200901) or YYYY-MM-DD (e.g. 2020-09-01); if needed, also include the time as HHMMSS or HH:MM:SS
include the version number in the file name, for example V02.

Tips for folder structure:

Own folder for each project
Own folders for data, administrative data, methods etc.
Access rights
Folder hierarchy: balance between deep and shallow hierarchy
Tagging (keywords, index terms) links files to several contexts.

This video produced by CSC takes a closer look at file management and file naming (duration 11:41): Manage well and get preserved – 6. Managing files and file naming

To consider

What does documentation mean in your field?
Are there discipline-specific standards related to description in your field?
What kind of information about the data and data processing do you need yourself during the project in order to work smoothly in your research?
What kind of information about the data and data processing is needed after the project in order that you and/or others can understand the data?
How would you describe the basic information of your research data in a README file?

Qvain for describing, Etsin for finding, persistent identifiers (PIDs) for linking

Standardised and structured metadata for research data of all kinds, regardless of the repository used, can be created using the Qvain tool. Qvain allows users to publish dataset descriptions, after which the metadata become available through the Etsin service. Qvain and Etsin are part of the Finnish Fairdata.fi services.

Qvain

The Qvain User Guide contains information, for example, on the minimum requirements for dataset descriptions. Guidance on using Qvain is also available in video format. UEF also provides a quick guide to using Qvain.

The information entered in Qvain includes the following fields:

Organizational maintenance rights to metadata
Datasource
Lisense and Access Type
Title, Desciption and other basic information
Actors
Related publications and other material
Geographical Area (spatial coverage)
Time Period (temporal coverage)
Infrastructure
History and events (provenience)
Project and Funding

Some of these fields, or their subfields, are mandatory. Qvain provides guidance on how to complete the fields.

Etsin

The Etsin search service enables researchers to find metadata for research data created using Qvain, as well as data made available through other services. In other words, Etsin harvests metadata from external sources such as the Finnish Social Science Data Archive and the Language Bank of Finland. Furthermore, metadata from Etsin can be transferred to other services, such as the UEF eRepository (eRepo).

Persistent identifiers (PIDs)

A persistent identifier (PID) is a unique and unambiguous identifier that functions in an online environment. It can be used to identify, for example, a research publication, a research dataset, or a person. A persistent identifier works like a link even if the online location of the object changes.

Common persistent identifiers for publications and datasets include DOI, URN, and Handle. Publishers and data repositories (see Chapter 5) assign persistent identifiers to publications or datasets. The persistent identifier for researchers is ORCID, which the researcher obtains independently.

Watch the video

Metadata is crucial for the findability of data: The Elements of FAIR – Findable, CSC (8:49).

In brief

Good data description includes concise information on:

the context of data collection (e.g. the aims and objectives of the project, and information about the research for which the data were collected)
data collection methods (e.g. sampling, the data collection process, instruments, and the hardware and software used)
the structure of the data files
quality assurance procedures carried out
version control
access and use conditions, including any confidentiality requirements
the names, labels, and descriptions of variables, records, and their values
explanations or definitions of the codes and classification schemes used
definitions of specialist terminology and acronyms used
the codes used for missing values and the reasons for them

Further information

Data description and metadata. Finnish Social Science Data Archive.
Guidelines for describing research data. AVOTT Avoimen tieteen ja tutkimuksen koordinaatio. (2024). Zenodo. (English translation added and links updated to the Finnish document on Jan 7, 2025.).
Siiri Fuchs, & Mari Elisa Kuusniemi. (2018). Making a research project understandable – Guide for data documentation (1.2). Zenodo.

_(2026-07)

_{Previous: 2. Collecting and Using Data}

_{Next: 4. Data storage, backup and sharing during a research project}