Data Transfer

Efficient and reliable data transfer is a critical component of scientific research computing. There are a variety of useful tools available for expedient data transfer, whether you are transferring data from an external site or within different computing environments at UVa.

Use Cases

The data transfer method you choose heavily relies on where the data are currently located and to where you want to transfer the data. Click on a row in the table below to learn more about the data transfer methods available for a specific use case.

Source Destination Example Use Cases


Lab Workstation


Non-Sensitive Data Storage
  • A research scientist wants to copy data from her lab workstation to her lab's /project storage share.

  • A graduate student needs to copy scripts he wrote to his /scratch directory on Rivanna.

Sources

  • Laptop
  • Lab Workstation

Destinations

Data Transfer Methods



Secure Lab Workstation


Sensitive Data Storage
  • A clinician wants to transfer HIPAA data from his Health Systems workstation to Ivy Central Storage.

Sources

  • Laptop
  • Lab Workstation
  • Health Systems Workstation

Destinations

Data Transfer Methods



Non-Sensitive Data Storage


Lab Workstation
  • An undergraduate student needs to move results from his /home directory on Rivanna to his personal laptop.

  • A postdoc wants to copy imaging data from Research Value storage to her lab workstation.

Sources

Destinations

  • Laptop
  • Lab Workstation

Data Transfer Methods



External Institution


University of Virginia
  • A new faculty member wants to transfer data she collected at another university to her Ivy Central Storage share at UVa.

  • A researcher needs to transfer results from the analysis he ran using ORNL's Titan supercomputer.

Sources

Any institution that uses Globus, such as:
  • Other universities
  • Supercomputing facilities

Destinations

Data Transfer Methods

Data Transfer Methods

Globus Connect

Large-scale research data transfer

Transferring large amounts of research data is fast and simple with Globus Connect. Globus gives researchers unified access to their data through an easy-to-use web interface, and can be used to transfer data between your laptop and storage systems mounted on the Rivanna HPC system. Globus can also be used to transfer data from other universities or supercomputing facilities.




For transferring highly sensitive data such as HIPAA or CUI data to the Ivy secure computing environment, researchers must use the secure Globus data transfer node (DTN).

Filezilla & Cyberduck

File transfer protocol (FTP) solutions

FTP clients such as Filezilla and Cyberduck give researchers a graphical user interface to transfer data between their lab computers and a remote storage location. These apps allow for drag-and-drop file manipulation.

Command Line Tools

Small-scale file transfer

Researchers who are comfortable with using the command line can use a variety of command line tools to transfer their data between their laptops and storage systems. Tools such as scp, sftp, and rsync can be used to quickly transfer a small number of files.

More Resources for Data Transfer with the Command Line

Many of the UVa storage options are based on a Linux file system. Users can invoke generic Linux commands to manage files and directories (mv, cp, mkdir), manage permissions (chmod, chown) and navigate the file system (cd, ls, pwd). If you or your collaborators are unfamiliar with some of these commands, we encourage you to take time to review some of the material below:

For more help, please feel free to contact SOMRC to set up a consultation or visit us during office hours. We also provide in-person training opportunities around basic command line skills and more through the CADRE Academy portal.