Globus Online

condodtn and cyencedtn are the only two machines we are currently supporting with Globus Online.

 

Login

  • In a browser, navigate to https://www.globus.org/ , click ‘Log In’ in the upper right corner.
  • Select Iowa State University for your organization, click ‘Continue’ then click ‘Log On’ on the next page.
  • Login with your university credentials at the familiar login page if you have not previously logged in.
  • You should now be on the ‘Transfer Files’ page

 

Copying Data

The Transfer Files page has two panes to represent the two system that you want to copy data between. You will need to select an Endpoint for each pane and most likely, authenticate.

  • Click in the ‘Endpoint’ box on one of the unused panes. A selection window will appear.
  • If you see your desired end point, click it and continue. If you do not see your desired end point start typing the name in the box at the top. When your endpoint appears, select it.
  • If authentication is required for the selected endpoint required fields or other instructions will appear. If you selected an endpoint supported by the ISU HPC Group you will be prompted to login to hpc-proxy.its.iastate.edu.
  • Username is your ISU NetID, i.e. the text to the left of ‘@’ in your iastate.edu email
  • Your password is your ISU password PLUS the six digits generated by Google Authenticator for access to the condo cluster. You would use ‘Password123456’ for the password field where ‘123456’ would be the current code from Google Authenticator and ‘Password’ would be your normal password.
  • Once you have logged in on both panes you will need to navigate to the data you want to copy in one pane(source pane) and navigate to the location you want it copied to on the other pane(destination pane).
  • Before you start your copy your look at the transfer settings at the bottom of the page. The question marks will give a short description of each option.
  • To start the transfer click the blue box with the arrow at the top of the page pointing in the direction of the destination pane.
  • You may now click ‘Activity’ at the top of the page to see a list of current and prior transfers.

 

Globus Connect Personal

To transfer files to your personal computer you may use Globus Connect Personal. A link can be found at the bottom right of the Transfer Files window or at the top of the Endpoints window.

 

Command Line

Authentication

When you authenticate to use GridFTP you will be issues a proxy certificate that will be valid for a limited period of time before being renewed. The default time limit is 12 hours. To obtain a proxy certificate you need to use the myproxy-logon command.

    myproxy-logon -s hpc-proxy.its.iastate.edu -l [your username]11

You will be prompted for a password. Your password is your ISU password PLUS the six digits generated by Google Authenticator for access to the condo cluster. You would use ‘Password123456’ for the password field where ‘123456’ would be the current code from Google Authenticator and ‘Password’ would be your normal password. The ‘grid-proxy-info’ command can be used to verify that you have successfully authenticated and show you how much time remains before you will need to authenticate again.

 

globus-url-copy

globus-url-copy is a command line utility that can be used to transfer data between grid endpoints or the local filesystem  if permitted by the system firewall/network configuration. The usual structure of the command is ‘globus-url-copy options source destination`.

To list files:

globus-url-copy -list gsiftp://condodtn.its.iastate.edu/work/somegroup/
To copy a file:
globus-url-copy options gsiftp://condodtn.its.iastate.edu/work/somegroup/somefile gsiftp://stream.its.iastate.edu/work/somegroup/
Options:
-p # - parallelism, good for splitting large files
-cc # - concurrency, number of control connections
-r - recursive
-dcsafe - data integrity protection
-dcpriv - data encryption - benefits from higher cc count
-sync - sync files from src to dest
-sync-level - 0-3, 0 copy non-existent, 3 do checksum, 2(def) copy newer and non-existent files
-v - verbose
-vb - verbose with transfer rates
 

When working with directories you will want to have a / at the end of the source or destination address.

 

Uberftp

Uberftp operates much like a standard command line FTP client and will work behind a firewall with the use of passive FTP(default). Passive mode transfers will not benefit from the performance features of GridFTP. If you can make changes to your firewall you should be able to increase performance by switching to active mode. More details are in the uberftp man page under the sections: DEFAULT TRANSFER MODE, IMPROVING FILE TRANSFER PERFORMANCE, and CONTROLLING EPHEMERAL PORT SELECTION. By default, put and get command will copy data between the local system and the system you mention on the command line or with the open command.

 

    uberftp lightning3.its.iastate.edu

 

If you wish to move data between GridFTP endpoints you may use lopen to connect to a second endpoint and the other ‘l’ commands(lpwd,lcd,lls,...) to navigate. The ‘help’ command will show you a list of command. Enter ‘help’ again followed by one of the shown commands will give you more information on that command.