Transferring Data Between Projects

This guide covers how to transfer DICOM data between XNAT projects, including copying (sharing) and moving (relocating) data.

Overview

Transferring data between projects is useful when:

  • Sharing data with collaborators in different projects

  • Moving data to a new project or consolidating projects

  • Creating backups or archival copies in separate projects

  • Reorganizing data across project structures

Primary Focus: DICOM Data Transfers

This guide focuses on transferring DICOM data (raw imaging data from scanners), which is the most common transfer scenario. DICOM data can be downloaded and re-uploaded using standard XNAT tools.

About Resources (Pipeline Outputs)

Data stored in Resources folders (such as BIDS rawdata, fMRIPrep outputs, MRIQC reports) are typically regenerated by running pipelines in the destination project rather than transferred. However, advanced users can transfer Resources using API methods (Python xnatpy, MATLAB scripts, or REST API) if needed. See Download via Python and Download via MATLAB for API-based transfer methods.

Prerequisites & Planning

Before transferring data, ensure you have the necessary access and have planned the transfer carefully.

Access Requirements

You must have appropriate permissions in both source and destination projects:

  • Source Project: Read access (Member or Owner role)

  • Destination Project: Write access (Member or Owner role) - May also need subject creation permissions if creating new subjects

To check your access level, navigate to each project and verify you can view (source) or upload (destination) data. If you don’t have the required access, contact the project administrators or see Accessing XNAT for requesting project access.

Pre-Transfer Checklist

Before starting a transfer, complete this checklist:

  1. Verify Access: Confirm you have read access to source project and write access to destination project

  2. Check for Conflicts: Check if subjects/sessions with the same labels already exist in the destination project

  3. Plan Label Mapping: If projects use different naming conventions, plan how to map subject/session labels

  4. Review DICOM Metadata: Consider Study Instance UIDs, Patient IDs, and routing strings

  5. Check Project Settings: Verify upload destination (Prearchive vs Archive) and subject creation permissions

  6. Verify Anonymization: Ensure data is properly anonymized before transfer (critical for compliance)

Important Considerations

Before transferring data, be aware of these critical considerations:

Warning

Data Overwriting and Merging

Uploading data to an existing project or session can overwrite existing files or merge new data into an existing session.

  • Merging: If you upload data with the same subject/session labels as existing data, XNAT will merge the new scans into the existing session (not overwrite)

  • File Overwriting: Files with identical names may be replaced

Always check for existing subjects/sessions in the destination project before uploading to avoid unintended merges.

Warning

Data Anonymization

Data from MRI scanners is typically anonymized at acquisition. When transferring data manually, ensure that your data is properly anonymized before transfer. All personally identifiable information (PII) and protected health information (PHI) must be removed or de-identified according to your project’s requirements and institutional policies.

DICOM Metadata Considerations

When transferring DICOM data, consider:

  • Study Instance UIDs: Unique identifiers for each study

  • Patient IDs: May need to be consistent or updated depending on project requirements

  • Routing Strings: If using DICOM receiver for upload, Study Description routing strings (xnat://project_id/subject_label/session_label) need to be updated for the destination project

Project Settings Differences

Different projects may have different upload settings:

  • Upload Destination: Some projects send data directly to Archive, others use Prearchive first

  • Subject Creation: Some projects allow users to create subjects during upload, others require pre-registration

  • Bulk Upload Restrictions: Some projects restrict bulk uploads

Check with the destination project administrator if you encounter restrictions. For more information, see Data Upload Procedures.

Transfer Methods

There are several methods for transferring DICOM data between projects. Choose the method that best fits your needs.

Method 2: Browser Download + Compressed Uploader

This method is best for:

  • Small to medium transfers

  • Single session transfers

  • Quick one-off transfers

It involves downloading sessions via the web browser, then uploading compressed archives using the Compressed Uploader.

For detailed instructions, see: - Download via Browser for browser download procedures - Data Upload Procedures for Compressed Uploader procedures

Method 3: Browser Download + Desktop Client Upload

A hybrid approach where you download via browser but upload via Desktop Client. Useful when you prefer different methods for download and upload.

Step-by-Step Workflows

Workflow A: Copying DICOM Sessions (Desktop Client)

This workflow uses Desktop Client for both download and upload, recommended for large or multiple sessions.

Step 1: Verify Access

  • Log into XNAT and verify you can access both source and destination projects

  • Check that you have read access to source and write access to destination

Step 2: Download from Source Project

  1. Navigate to the source project in XNAT web interface

  2. Select the sessions you want to transfer (can select multiple)

  3. Click “Download Images” from the Actions menu

  4. Choose “Catalog XML” as the download format

  5. Download the XML file to your computer

  6. Open XNAT Desktop Client (see Install XNAT Desktop Client if not installed)

  7. Ensure you’re logged into the correct XNAT server

  8. Select “Download Files” from the main menu

  9. Browse and select your downloaded XML file

  10. Set download location and click “Download”

  11. Wait for download to complete

For detailed download instructions, see Download via Desktop Client.

Step 3: Prepare Data

Before uploading, verify your downloaded data:

  • Check that all expected sessions were downloaded

  • Verify DICOM file structure is intact

  • Ensure data is properly anonymized

  • Remove any non-DICOM files if present

Step 4: Upload to Destination Project

  1. Navigate to the destination project in XNAT web interface

  2. Click “Upload Images” from the Actions menu (this opens Desktop Client automatically)

    OR

    Open Desktop Client directly and click “Upload Images”

  3. Select the destination project from the project dropdown

  4. Select the downloaded DICOM folders (drag and drop or browse)

  5. The client will scan files to identify sessions

  6. Configure Upload Settings: - Subject Selection: Choose existing subject or create new (if allowed) - Subject Label: Ensure it matches destination project naming conventions - Session Label: Auto-generated or override if needed - Check for Conflicts: Verify no existing sessions with same labels - Scan Selection: Select which scans to include (uncheck any you don’t want)

  7. Select Upload Destination: Choose Prearchive (recommended) or Archive

  8. Click “Start Upload”

  9. Monitor progress in the transfer manager

For detailed upload instructions, see Data Upload Procedures.

Step 5: Verify Transfer Success

  1. Navigate to the destination project

  2. If uploaded to Prearchive, review the Prearchive first

  3. Check that subjects and sessions were created correctly

  4. Verify scan counts match between source and destination

  5. Compare file sizes to ensure completeness

  6. Test downloading a sample scan to verify integrity

Step 6: Archive (if using Prearchive)

  1. Review data in Prearchive

  2. Verify all metadata is correct

  3. Archive the sessions to move them to permanent storage

Workflow B: Copying DICOM Sessions (Browser + Compressed Uploader)

This workflow is faster for single sessions or small transfers.

Step 1: Verify Access

  • Verify access to both source and destination projects

Step 2: Download from Source Project

  1. Navigate to the session in the source project

  2. Use the session-level download option

  3. Download the session as a ZIP archive via browser

For detailed instructions, see Download via Browser.

Step 3: Upload to Destination Project

  1. Log into XNAT web interface

  2. Navigate to Upload > Images > Compressed Uploader

  3. Select Project: Choose the destination project from dropdown

  4. Configure Upload Settings: - Importer handler: Select “DICOM Zip” - Custom Labeling: Choose “Extract From DICOM (Default)” or “Customize” to specify labels - Subject ID and Session Label: Enter if using custom labeling - Ignore unparsable files: Check to skip problematic files

  5. Select Destination: Choose Prearchive (recommended for safety)

  6. Choose File: Select your downloaded ZIP file

  7. Click “Begin Upload”

  8. Wait for confirmation message

For detailed instructions, see Data Upload Procedures.

Step 4: Verify and Archive

  1. Navigate to Prearchive in destination project

  2. Review the uploaded session

  3. Verify subject/session labels are correct

  4. Check that all scans are present

  5. Archive the session to move to permanent storage

Workflow C: Moving Data (Copy then Delete)

To move data (rather than copy), follow these additional steps after completing Workflow A or B:

Warning

Permanent Deletion

Deleting data from the source project is permanent. Only proceed after thorough verification that all data has been successfully transferred to the destination.

If you accidentally delete data, contact XNAT administrators immediately. See Data Loss and Recovery for information about data recovery options.

Step 1: Complete Copy Workflow

  • Follow Workflow A or B to copy data to destination project

  • Ensure upload is complete and archived

Step 2: Verify in Destination

  • Thoroughly verify all data transferred correctly: - All subjects and sessions present - All scans present with correct counts - File sizes match - Sample downloads work correctly - No errors in upload logs

Step 3: Delete from Source (if appropriate)

  • Only proceed if you have deletion permissions and have verified the transfer

  • Navigate to source project

  • Delete subjects or sessions as needed

  • Note: Deletion may require Owner-level permissions

Step 4: Final Verification

  • Confirm deletion from source project

  • Verify destination data is complete and accessible

  • Document the transfer for project records

About Resources (Advanced Users)

What are Resources?

Resources are folders in XNAT that store pipeline outputs and processed data:

  • Session-level Resources: BIDS rawdata, fMRIPrep outputs, MRIQC reports, processing logs

  • Subject-level Resources: Subject-specific processed data

  • Project-level Resources: Shared configuration files, project-wide data

Resources are typically stored in the Resources folder at each hierarchy level (Project, Subject, Session).

When to Transfer Resources

Generally, Resources should NOT be transferred because:

  • They are outputs from pipelines that can be regenerated by running the same pipelines in the destination project

  • Transferring Resources can be complex and error-prone

  • Pipeline outputs are often project-specific and may not work correctly in a different project context

Transfer Resources only if:

  • You need to preserve specific pipeline outputs that cannot be easily regenerated

  • You have custom processing results that are not reproducible

  • You need to maintain exact version compatibility

How to Transfer Resources (Advanced)

If you need to transfer Resources, use API-based methods:

  • Python: Use xnatpy library or Python scripts (see Download via Python)

  • MATLAB: Use MATLAB scripts with XNAT API (see Download via MATLAB)

  • REST API: Direct API calls to download and upload Resources

Note: Desktop Client is primarily designed for DICOM transfers, not Resources. For Resource transfers, API methods are required.

Verification Steps

After completing a transfer, verify the data integrity:

  1. Subject/Session Creation: Check that all subjects and sessions were created in the destination project

  2. Scan Counts: Verify the number of scans matches between source and destination

  3. File Sizes: Compare file sizes to ensure no data loss during transfer

  4. DICOM Metadata: If applicable, verify Study Instance UIDs and Patient IDs are correct

  5. Prearchive Review: If using Prearchive, review all data before archiving

  6. Sample Download: Test downloading a sample scan to verify file integrity

  7. Upload Logs: Review upload logs for any warnings or errors

Troubleshooting

Common Issues and Solutions

Permission Errors

  • Symptom: Cannot access source project or cannot upload to destination

  • Solution: Verify you have Member or Owner access to both projects. Contact project administrators if needed.

Duplicate Label Conflicts

  • Symptom: Data merges with existing sessions instead of creating new ones

  • Solution: Check for existing subjects/sessions with the same labels in destination. Use different labels or coordinate with project administrators. If you encounter UID errors (such as “Invalid modification of session UID”) when merging different sessions, see Resolving UID Errors During Archiving for detailed resolution steps.

Upload Failures

  • Symptom: Uploads fail or timeout

  • Solution: - Check file permissions and quota limits - Verify network connectivity - Try smaller batches or use Desktop Client for better reliability - Check upload logs for specific error messages

Missing Data After Transfer

  • Symptom: Not all scans appear in destination

  • Solution: - Verify all scans were selected during upload - Check upload logs for failed scans - Ensure all DICOM files were included in download - Retry failed uploads if needed

DICOM Routing Issues

  • Symptom: Data doesn’t route correctly when using DICOM receiver

  • Solution: Verify Study Description routing strings (xnat://project_id/subject_label/session_label) are correct for destination project

Prearchive Stuck

  • Symptom: Data remains in Prearchive and cannot be archived

  • Solution: Contact XNAT administrators at admin.nyuad.xnat@nyu.edu for assistance

Large Transfer Performance

  • Symptom: Very slow transfers or timeouts

  • Solution: - Use Desktop Client for large transfers (better resume capability) - Transfer during off-peak hours - Break large transfers into smaller batches - Ensure stable network connection

Getting Help

If you encounter issues not covered here:

Best Practices

Follow these best practices for safe and efficient data transfers:

  1. Always Verify Before Deleting: Never delete source data until you’ve thoroughly verified the transfer in the destination project. If data is accidentally deleted, see Data Loss and Recovery for recovery options

  2. Use Prearchive: Upload to Prearchive first to allow review before archiving

  3. Document Transfers: Keep records of what was transferred, when, and why for project documentation

  4. Coordinate with Administrators: For large transfers or complex scenarios, coordinate with project administrators

  5. Test First: Test with a small dataset before attempting bulk transfers

  6. Check Naming Conventions: Verify subject/session label conventions match destination project requirements

  7. Ensure Anonymization: Always verify data anonymization compliance before transfer

  8. Monitor Transfers: Keep an eye on transfer progress and logs to catch issues early

See Also

Related Documentation

Advanced Resource Transfers

Support