6. Add, View, and Delete Metadata Files via UI

6.1. Add, View, and Delete Metadata Files

A C(DCC) Administrator can add metadata files to a study in any status. However, in order to upload a metadata file, they must be associated with a data file. The C(DCC) Administrator may choose not to upload a metadata file at all because data files are not required to have an associated metadata file. Once uploaded, the file will undergo file validation and data loss prevention. First, log in to the Data Hub and select the study name from the table that you wish to upload your metadata files to.

View Studies Landing

6.1.1. Adding Metadata Files in bulk to a Study

Like data files, navigate to the appropriate file upload page depending the study status (see data file upload link) and click that page’s upload button. Once the pop-up appears and you are browsing for files, you must ensure that each metadata file you wish to upload has a corresponding data file. Metadata files absolutely must have a data file associated with it because a metadata file’s purpose to describe that data file. Order of operations when browsing files does not matter. When associating metadata files to data files, the two requirements are that metadata files must be json and adhere to the file naming convention.

For example, if a data file is named “test_file_origcopy.csv” and the metadata file is named “bob_is_your_uncle_origcopy.json”, you must rename the data file to “test_file_DATA_origcopy.csv” and the metadata file to “test_file_META_origcopy.json”. The key is to ensure that both file names start with “test_file” and end with “”_DATA_origcopy”” for data files “”_META_origcopy”” for metadata files. Metadata files that are uploaded and do not adhere to this naming convention will not be uploaded at all and an error message will appear.

Does not adhere to naming convention:

Bad Example

Adheres to naming convention:

Good Example

6.1.2. Adding a Metadata File to an existing Data File

On the Study Overview tab, select the Data Files tab. For data files without a metadata file, you will see an “Add Metadata File” button a few columns to the right in the same row.

Data Files Tab

Click this button and you will notice a pop-up similar to the one for adding data files. However, this one only allows you to attach a single metadata file.

Upload Metadata File Popup

When you attach more, you will see the following.

Upload Metadata File Popup Too Many Files

Now attach a single metadata file.

Upload Metadata File Popup One File

6.1.3. Delete Metadata Files from a Study

Metadata files can be deleted from a study if it has not yet been shared with a Researcher. This means study data files in the status of Draft, Pending Approval, and Rejected can be deleted at any time.

There are two ways to delete a metadata file. One way is to delete the data file, which will also delete the associated metadata file.

Delete A Metadata File With Data File

The second way is to delete the file directly from the Metadata Files tab located next to the Data Files tab.

Delete A Metadata File Only

A confirmation pop-up window will appear, prompting the C(DCC) Administrator to confirm their action. Click the “Delete” button.

Delete Confirm File

The metadata file is now deleted.

6.2. Submit Metadata Files

When new metadata files are added to a study, they will be in the “Draft” status. These files will not be visible to the Data Hub Data Administrator for review until the files are submitted.

To submit files, select the “Submit Files” button.

Submit Files Button

A list of Draft files that will be submitted to the Data Hub Data Administrator will appear in a table. This will include both data and metadata files.

Submit Files List

Scroll down to the confirmation statement to agree and sign. Select the “Submit Files” button.

Submit Files Button

The “Submit Files” button will only be enabled if there are files in the “Draft” status - otherwise, it will be disabled and grayed out.

Additional Data Files

Additional Data Files

Note: The data and metadata files go through their own separate file validation and PHI/PII validation processes. If the file passes all validations, the file is automatically moved to the Approved tab.

6.3. PII/PHI and File Validation

The Data Hub will run two types of validations when data files are uploaded to the system: File and PII/PHI validation. Both validation processes will be performed on all metadata files uploaded to the Data Hub.

The File validation will check that the metadata file contains the required fields indicated in the CEDAR metadata specification. File validation will also check that any additional fields present exist in the specification’s predefined list of optional fields. If the metadata file is missing any required fields or if any of the additional fields do not match any of the optional fields, the file will be flagged as containing errors.

The Data Hub will also check that all files uploaded do not contain PII/PHI. If the system detects that the file contains PII/PHI, the data file will be flagged as non-PII/PHI-compliant.

Validation Flags

Note: An hourglass icon will appear while the system is running the validation.

Validation Icons

If data files are flagged, users can view the File and PII/PHI validation analysis by selecting the “View” buttons under the File Validation or PII/PHI columns.

File validation analysis -

In the errors view, by default, only 300 errors would show up. These will display as a scrollable list of all the fields that were either missing or did not adhere to the CEDAR metadata specification.

File validation

PII/PHI validation analysis -

PII validation

Note: Failed File or PII/PHI validations do not prevent the C(DCC) Administrator from submitting a study. They serve as warnings for the C(DCC) Administrator to review the outputs and decide to submit the existing files or replace them with updated files. The Data Hub Data Administrator will see these same validation outputs and may reject the submission due to what is found in these validations.

6.4. View Metadata Files

Select the study name that you wish to view from the My Studies page.

View File

If you have selected a Draft study, you will land on the Study Details page. Continue to the Add Data Files page by selecting the “Save & Continue” button at the bottom of the page.

View File

If you have selected a Submitted, Approved, or Rejected study, you will land on the Study Overview tab. Select the Metadata Files tab to see a list of metadata files that have been associated with the study’s data files.

View File

Note: You may also view the data file it’s associated with by clicking the link under the “Data File” column.

To view the metadata file, select the file name under the “File Name” column, which will open the metadata file viewer window.

View File

All fields identified within the file will be opened in a pop-up viewer window. These field values will be presented in prettified format.

Note: Based on the CEDAR metadata specification defined for the data hub.