Skip to main content

Data Transfers with SFTP

Transfers can be completed via SFTP through your SSH client by connecting to one of Palmetto's dedicated data transfer nodes. These nodes are set up to ensure data transfer activity does not impact other users.

NodeConnection Address
HPC Data Transfer Node 1hpcdtn01.rcd.clemson.edu
HPC Data Transfer Node 2hpcdtn02.rcd.clemson.edu

If one data transfer node is unavailable, try using the other one.

tip

For transfers smaller than 100 MB total, you may transfer them via SFTP to the login nodes.

Instructions

You can use the scp (secure copy) command to perform file transfers. This command is available on most Linux systems and modern versions of Windows 10/11.

The general form of the scp command is:

scp <path_to_source> username@hpcdtn01.rcd.clemson.edu:<path_to_destination>
scp <path_to_source> username@hpcdtn02.rcd.clemson.edu:<path_to_destination>

Example of copying files to Palmetto

For example, here is the scp command to copy a file from the current directory on your local machine to your /home/username directory on Palmetto (this command is entered into a terminal when not logged-in to Palmetto):

scp myfile.txt username@hpcdtn01.rcd.clemson.edu:/home/username
scp myfile.txt username@hpcdtn02.rcd.clemson.edu:/home/username

You can include the full path to a file if it is outside your current working directory.

# Windows
scp "U:\Documents\img.bmp" username@hpcdtn01.rcd.clemson.edu:/home/username

# macOS/Linux
scp "~/Documents/img.bmp" username@hpcdtn01.rcd.clemson.edu:/home/username

Example of copying files from Palmetto

... and to do the same in reverse, i.e., copy from Palmetto to your local machine. (again, from a terminal running on your local machine, not on Palmetto):

scp username@hpcdtn01.rcd.clemson.edu:/home/username/myfile.txt .
scp username@hpcdtn02.rcd.clemson.edu:/home/username/myfile.txt .

The . represents the working directory on the local machine.

Example of copying entire folders

To copy a folder (and all of its contents), you must include the -r switch to enable recursive mode:

scp -r myfolder username@hpcdtn01.rcd.clemson.edu:/home/username
scp -r myfolder username@hpcdtn02.rcd.clemson.edu:/home/username

Tips for Windows Users

If you do not know the full path to a file or folder, you can use the Copy Path button in the ribbon of File Explorer to retrieve it.

Make sure you have highlighted the correct file or folder before pressing the button.

file explorer ribbon with copy path button

You can use this path in your scp command by right-clicking on the PowerShell window.