Skip to content

Latest commit

 

History

History
134 lines (82 loc) · 5.64 KB

filetransfer.md

File metadata and controls

134 lines (82 loc) · 5.64 KB
Raw

Transferring Files to and from Oscar

There are several ways to move files between your machine and Oscar. Which method you choose will depend on how much data you need to move and your personal preference for each method.

  1. SMB
  2. Command line (scp)
  3. GUI application
  4. Globus online (best for large transfers)
  5. LFTP

1. SMB

You can drag and drop files from your machine to the Oscar filesystem via SMB. This is an easy method for a small number of files. Please refer to this page for mounting filesystem via SMB.

2. Command line

Mac and Linux

SCP

You can use scp to transfer files. For example to copy a file from your computer to Oscar:

scp /path/to/source/file <username>@ssh.ccv.brown.edu:/path/to/destination/file

To copy a file from Oscar to your computer:

scp <username>@ssh.ccv.brown.edu:/path/to/source/file /path/to/destination/file

RSYNC

You can use rsync to sync files across your local computer to Oscar:

rsync -azvp --progress path/to/source/directory <username>@ssh.ccv.brown.edu:/path/to/destination/directory

Windows On Windows, if you have PuTTY installed, you can use it's pscp function from the terminal.

3. GUI programs for transferring files using the sftp protocol and transfer.ccv.brown.edu hostname

  • DUO is required if you are not connected to approved networks, e.g., home network
    • There is no interactive terminal message but your Phone will get a prompt automatically
  • DUO is NOT required if you are connected to approved Brown networks
    • A personal Windows computer must have CrowdStrike installed in order to be on approved Brown networks.

In general, you can specify the following for your GUI programs:

  • Protocol: SFTP
  • Host: transfer.ccv.brown.edu
  • User: your Oscar username
  • Password: your Brown password

3.1 WinSCP for Windows

3.1.1 Limit Concurrent Transfer and Change Reconnect Options

Click the Optionsand then Preferences menu in WinsCP. In the poped up window, click Transfer and then Background to (Figure 1)

  • change Maximal number of transfers at the same time to 1
  • uncheck Use multiple connections for single transfer

Figure 1 WinSCP Maximal Transfers

click Endurace to (Figure 2)

  • set Automatically reconnect session to 5 seconds
  • uncheck Automatically reconnect session, if it stalls
  • set Keep reconnection for to 10 seconds

Figure 2 WinSCP Reconnect

3.1.2 Add a New Site

Figure 3 WinSCP Session Creation

3.2 FileZilla

3.2.1. Disable Timeout

Click the Edit menu and then select the Settings submenu, and then change the Timeout in seconds to 0 to disable, as shown in Figure 2

Figure 4 Disable Timeout

3.2.2 Add a New Site

Open the Site Manager as show in Figure 5.

Figure 5 Open Site Manager

Click the 'New Site' button to add a new site, as shown in Figure 4:

Figure 6 New Site

Limit the number of simultaneous connections to 1, as shown in Figure 5.

Figure 7 Limit Simultaneous Connections

Click the 'Connect' button to connect to Oscar and transfer files.

3.3 Cyberduck

Figure 8 Cyberduck Connection

You may see a popup window on 'Unknown Fingerprint'. You just need to check the 'Always' option and click 'Allow'. This is windows should not pop up again unless the transfer server is changed again.

Figure 9 Unknown Fingerprint

4. Globus online

Globus is a secure, reliable research data management service. You can move data directly to Oscar from anothe Globus endpoint. Oscar has one Globus endpoint:

BrownU_CCV_Oscar

If you want to use Globus Online to move data to/from you own machine, you can install Globus Connect Personal. For more instructions on how to use Globus, see the Oscar section in the Globus documentation.

5. LFTP

LFTP is a sophisticated file transfer program supporting a number of network protocols (ftp, http, sftp, fish, torrent). It has bookmarks, a built-in mirror command, can transfer several files in parallel and was designed with reliability in mind. You can use the LFTP module from Oscar to transfer data from any (S)FTP server you have access to directly to Oscar. Below are the main LFTP commands to get you started:

module load lftp  # To load the LFTP module from Oscar
lftp -u login,passwd MyAwesomeUrl  # To connect to your (S)FTP server
ls   # To list files on the (S)FTP server
!ls  # To list files in your directory on Oscar
get MyAwesomeFile  # To download a single file
mirror # To download everything as is from the server
mirror --directory=/name_of_directory/ # To download a specific directory