Transferring Files

Aside from standard tools like scp and rsync, LCRC offers Globus Online for large file transfers. Globus Online is a free online service that simplifies data movement and addresses the challenges faced by researchers in moving, sharing, and archiving large volumes of data among distributed sites. With Globus Online, you hand-off data movement tasks to a hosted service that manages the entire operation: monitoring performance and errors, retrying failed transfers, correcting problems automatically whenever possible, and reporting status to keep you informed so that you can focus on your research. Command line and web-based interfaces are available.

Getting Started

  • Register for a Globus Online account.
  • Sign in to Globus Online (using your Globus Online username and password)
  • Select Transfer Files in the top bar.
  • On the Transfer Files page, you can view the list of available endpoints by clicking the text field labeled Endpoint.
    • You can use the LCRC endpoint “LCRC DTN” as well as other sources or destinations.
    • You can type letters into the box to filter endpoints.
  • Once you select the LCRC endpoint, you will be redirected to an LCRC OAuth authentication portal. Simply use your LCRC username and password to authenticate and you should be redirected back to Globus automatically.
  • You will see a listing of the contents of your home directory in LCRC. Double click on a directory to view its contents.
  • Select a file or directory and click on the highlighted arrow button to initiate the transfer.

Transferring Data Between LCRC and Your Machine

To transfer data between LCRC and your laptop or desktop, you can install Globus Connect Personal on your machine and access it via Globus Online. Globus Connect is available as a one-click install for Mac, Linux, and Windows.

  • Install Globus Connect Personal following the instructions for your operating system.
  • Run Globus Connect Personal. When you run Globus Connect Personal for the first time, it will ask you for a setup key.
  • On the Globus Online web page, select Endpoints on the menu bar. Click on add Globus Connect Personal endpoint, and enter an endpoint name (eg., mylaptop) to identify your machine on Globus Online.
  • Click Generate Setup Key
  • Copy the setup key and paste it into the Globus Connect Personal window to complete the setup process.

You should now see your machine in the list of endpoints on Globus Online. You can select it to view the contents of your machine and transfer files to and from it as above.

Using the Command Line Interface

In order to use the Globus Online Command Line Interface (CLI), you must add your SSH public key (typically found in $HOME/.ssh/id_rsa.pub) to your Globus Online account.

  • Select Account from the top menu bar.
  • Click the manage SSH and X.509 keys link and follow the instructions on the Globus ID site.
  • Paste the contents of your SSH public key into the SSH Public Key field (you can optionally enter any name you like in the Alias field) and click on Add SSH Key. It will take a minute or two for the SSH key to become active for use.

You can now log in to the CLI by typing ssh @cli.globusonline.org. This logs you into a restricted shell where you can execute only Globus Online commands. Type help to see all available commands.

Globus Transfer API

Globus Online also has a REST-style interface that facilitates integration into custom applications. More information on the REST API is available here.