- Home
- About
- Research
- Education
- News
- Publications
- Guides
- Introduction to HPC clusters
- UNIX Introduction
- Nova
- HPC Class
- SCSLab
- File Transfers
- Cloud Back-up with Rclone
- Globus Connect
- Sample Job Scripts
- Containers
- Using DDT Parallel Debugger, MAP profiler and Performance Reports
- Using Matlab Parallel Server
- JupyterLab
- JupyterHub
- Using ANSYS RSM
- Nova OnDemand
- Python
- Using Julia
- LAS Machine Learning Container
- Support & Contacts
- Systems & Equipment
- FAQ: Frequently Asked Questions
- Contact Us
- Cluster Access Request
Getting data to and from a cluster
To copy files to and from the cluster please use cluster's data transfer node (and not the login node).
The cluster storage is relatively expensive and should not be used as an archival storage. Important data that is not actively used on the cluster should be copied somewhere else. For this purpose we recommend using Large Scale Storage System (LSS) provided by LAS. LSS is mounted on the data transfer nodes on all clusters, so it appears like another directory and can be accessed as /lss. You will need to ssh to cluster's data transfer node, issue "kinit" command, enter your ISU password when prompted, and then use "cp ..." command to copy files to and from /lss . You will need to know the full path to your directory in LSS.
As of May 2019 you can also copy files to and from MyFiles which is mounted on all data transfer nodes as /myfiles . To copy a file test.txt from your home directory to your directory in Myfiles, issue the following command on the cluster's data transfer node:
cp /home/$USER/test.txt /myfiles/Users/$USER/
USER is an environment variable, like PATH that we saw earlier in the CLI page. Its value is your cluster account name, which is the same as your ISU NetID. You can also use your ISU NetID explicitly instead of typing $USER in the command above.
If you need to copy data between a cluster and a remote computer (including your laptop or desktop), you can use an scp client program. "scp" command is available on Linux, MacOS and MS Windows 10 (in powershell). This command is similar to the "cp" command above, but you will also need to specify the remote machine name and provide login credentials for that machine.
If you prefer to use GUI program, there are several options available.
We recommend using Globus Connect which allows to transfer files between Globus endpoints. Users login to their account on https://www.globus.org and use GUI to transfer files between selected endpoints. Many Data Centers have Globus endpoints configured. We also configured endpoints for the data transfer nodes on all clusters listed on this website. To use Globus Connect to transfer data between your computer and another endpoint, you will need to install a Globus Connect Personal on that computer to create personal endpoint. One does NOT need to have administrator rights to install Globus Connect Personal.
Next: List of CLI Commands