6. Add, View, and Delete Metadata Files via UI

6.1. Add, View, and Delete Metadata Files

A C(DCC) Administrator can add metadata files to a study in any status. However, in order to upload a metadata file, they must be associated with a data file. The C(DCC) Administrator may choose not to upload a metadata file at all because data files are not required to have an associated metadata file. Once uploaded, the file will undergo file validation and data loss prevention. First, log in to the Data Hub and select the study name from the table that you wish to upload your metadata files to.

6.1.1. Adding Metadata Files in bulk to a Study

Like data files, navigate to the appropriate file upload page depending the study status (see data file upload link) and click that page’s upload button. Once the pop-up appears and you are browsing for files, you must ensure that each metadata file you wish to upload has a corresponding data file. Metadata files absolutely must have a data file associated with it because a metadata file’s purpose to describe that data file. Order of operations when browsing files does not matter. When associating metadata files to data files, the two requirements are that metadata files must be json and adhere to the file naming convention.

For example, if a data file is named “test_file_origcopy.csv” and the metadata file is named “bob_is_your_uncle_origcopy.json”, you must rename the data file to “test_file_DATA_origcopy.csv” and the metadata file to “test_file_META_origcopy.json”. The key is to ensure that both file names start with “test_file” and end with “”_DATA_origcopy”” for data files “”_META_origcopy”” for metadata files. Metadata files that are uploaded and do not adhere to this naming convention will not be uploaded at all and an error message will appear.

Does not adhere to naming convention:

Adheres to naming convention:

6.1.2. Adding a Metadata File to an existing Data File

On the Study Overview tab, select the Data Files tab. For data files without a metadata file, you will see an “Add Metadata File” button a few columns to the right in the same row.

Click this button and you will notice a pop-up similar to the one for adding data files. However, this one only allows you to attach a single metadata file.

When you attach more, you will see the following.

Now attach a single metadata file.

6.1.3. Delete Metadata Files from a Study

Metadata files can be deleted from a study if it has not yet been shared with a Researcher. This means study data files in the status of Draft, Pending Approval, and Rejected can be deleted at any time.

There are two ways to delete a metadata file. One way is to delete the data file, which will also delete the associated metadata file.

The second way is to delete the file directly from the Metadata Files tab located next to the Data Files tab.

A confirmation pop-up window will appear, prompting the C(DCC) Administrator to confirm their action. Click the “Delete” button.

The metadata file is now deleted.

6.2. Submit Metadata Files

When new metadata files are added to a study, they will be in the “Draft” status. These files will not be visible to the Data Hub Data Administrator for review until the files are submitted.

To submit files, select the “Submit Files” button.

A list of Draft files that will be submitted to the Data Hub Data Administrator will appear in a table. This will include both data and metadata files.

Scroll down to the confirmation statement to agree and sign. Select the “Submit Files” button.

The “Submit Files” button will only be enabled if there are files in the “Draft” status - otherwise, it will be disabled and grayed out.

Note: The data and metadata files go through their own separate file validation and PHI/PII validation processes. If the file passes all validations, the file is automatically moved to the Approved tab.

6.3. PII/PHI and File Validation

The Data Hub will run two types of validations when data files are uploaded to the system: File and PII/PHI validation. Both validation processes will be performed on all metadata files uploaded to the Data Hub.

The File validation will check that the metadata file contains the required fields indicated in the CEDAR metadata specification. File validation will also check that any additional fields present exist in the specification’s predefined list of optional fields. If the metadata file is missing any required fields or if any of the additional fields do not match any of the optional fields, the file will be flagged as containing errors.

The Data Hub will also check that all files uploaded do not contain PII/PHI. If the system detects that the file contains PII/PHI, the data file will be flagged as non-PII/PHI-compliant.

Note: An hourglass icon will appear while the system is running the validation.

If data files are flagged, users can view the File and PII/PHI validation analysis by selecting the “View” buttons under the File Validation or PII/PHI columns.

File validation analysis -

In the errors view, by default, only 300 errors would show up. These will display as a scrollable list of all the fields that were either missing or did not adhere to the CEDAR metadata specification.

PII/PHI validation analysis -

Note: Failed File or PII/PHI validations do not prevent the C(DCC) Administrator from submitting a study. They serve as warnings for the C(DCC) Administrator to review the outputs and decide to submit the existing files or replace them with updated files. The Data Hub Data Administrator will see these same validation outputs and may reject the submission due to what is found in these validations.

6.4. View Metadata Files

Select the study name that you wish to view from the My Studies page.

If you have selected a Draft study, you will land on the Study Details page. Continue to the Add Data Files page by selecting the “Save & Continue” button at the bottom of the page.

If you have selected a Submitted, Approved, or Rejected study, you will land on the Study Overview tab. Select the Metadata Files tab to see a list of metadata files that have been associated with the study’s data files.

Note: You may also view the data file it’s associated with by clicking the link under the “Data File” column.

To view the metadata file, select the file name under the “File Name” column, which will open the metadata file viewer window.

All fields identified within the file will be opened in a pop-up viewer window. These field values will be presented in prettified format.

Note: Based on the CEDAR metadata specification defined for the data hub.