Remote Access to Instances¶
All of the access methods described below are built on top of ssh
and require completion of the steps in Basic Security Setup to use with cloud.rc
. Some of these steps are referenced in that document.
Command Line via SSH¶
SSH stands for Secure SHell and is a powerful tool for executing terminal commands on remote machines. It is widely used and ubiquitous, and a number of other technologies are built on top of SSH, like sftp
and scp
for transferring files. It is also the primary mode of command line communication with Research Computing technologies like Cheaha and cloud.rc.
Install an SSH Client¶
There are two main steps to working with SSH efficiently. The first is to ensure you have an SSH client installed, which will let your local machine communicate with remote machines. The second is to ensure you have ssh-agent
running in each terminal window to automate management of key files. The ssh-agent
software comes with most SSH clients, but does not always run automatically. How to start the ssh-agent
software automatically varies depending on operating system and shell flavor, which we will describe below.
Terminal Multiplexers¶
Terminal multiplexers are software that aggregate multiple SSH client sessions into one location. They often have the added benefit of keeping sessions alive on the remote machine, so short internet outages won't require you to log in again.
- For Linux, try using
tmux
: https://github.com/tmux/tmux/wiki/Installing - For Windows, try using Windows Terminal: https://apps.microsoft.com/store/detail/windows-terminal/9N0DX20HK701
Install an SSH Client (Linux)¶
Virtually all Linux distributions come with SSH preinstalled and configured appropriately for ease of use, including automatically starting the ssh-agent
.
Install an SSH Client (MacOS)¶
MacOS comes with an SSH client installed.
If you are on version Leopard 10.5.1
or lower, you may want to have the ssh-agent
start automatically using the command sudo touch /var/db/useLS
at a terminal window. Versions newer than Leopard 10.5.1
start the ssh-agent
automatically.
Install an SSH Client (Windows)¶
There are several options for installing an SSH client on Windows, described below. It is highly recommended to install Windows Subsystem for Linux (WSL) as it provides a complete Linux environment within Windows.
Windows Subsystem For Linux (WSL)¶
Follow the instructions starting here to install Windows Subsystem for Linux.
WSL shells do not automatically start or share the ssh-agent
. To fix this we recommend installing keychain
to automatically manage the ssh-agent
. Run the following command depending on your Linux distribution.
- DEB-based (Debian, Ubuntu):
sudo apt install keychain
- RPM-based (CentOS, Fedora, openSUSE):
sudo yum install keychain
Then modify the .*rc
file for your shell, generally .bashrc
or .zshrc
, to automate the ssh-agent
by adding the following line.
- eval `keychain -q --eval --agents ssh`
Tip
You can access WSL files from within Windows in two ways.
In the WSL terminal, enter explorer.exe .
to open a File Explorer window in the current directory.
In Windows, open a File Explorer window, click in the top navigation bar and enter \\wsl$
. Then select your distribution from the file window to access the filesystem of that WSL operating system.
OpenSSH for Windows¶
Follow the instructions here to install the OpenSSH client. Only install the OpenSSH server if you need it, otherwise skip that part.
Once the OpenSSH client is installed, you'll want to enable the OpenSSH Agent service on your local machine to streamline adding and using keys.
- Open the Start Menu and search for "Services", and open the result shown in the image.
- Find the "OpenSSH Authentication Agent" service in the list. Double click it, or right-click it and select "Properties".
- In the dialog box, under the "General" tab, look for "Startup Type". Click the drop-down menu and select "Automatic (Delayed Start)". Click "Apply" at the bottom-right corner. This will cause the
ssh-agent
service to start when Windows starts. - The "Start" button under the horizontal line should become enabled. Click it to start the
ssh-agent
service now.
Git Bash terminal (Git for Windows)¶
The fine folks at Git have worked very hard to package everything needed to use Git on Windows into one installer. This includes a Linux command line interface emulator, Bash and SSH. Visit https://git-scm.com to download and install. Follow the installer instructions. It is recommended to use all of the default installation options. Once installed, locate "Git Bash" on your machine to open the Bash terminal. It should be searchable in the Start Menu.
To automate running ssh-agent
add the following block to the file .bash_profile
in the ~
directory within Git Bash. Then use source .bash_profile
to start the ssh-agent
, or open a new terminal.
Note
If such a file does not exist, please add it using nano .bash_profile
to create a new file in the nano text editor. Copy and paste the block below into the text editor window. The character ^
means Ctrl. Use ^x
(Ctrl+X) to exit, and continue following the prompts to save the file, using Ctrl shortcuts as needed.
env=~/.ssh/agent.env
agent_load_env () { test -f "$env" && . "$env" >| /dev/null ; }
agent_start () {
(umask 077; ssh-agent >| "$env")
. "$env" >| /dev/null ; }
agent_load_env
# agent_run_state: 0=agent running w/ key; 1=agent w/o key; 2= agent not running
agent_run_state=$(ssh-add -l >| /dev/null 2>&1; echo $?)
if [ ! "$SSH_AUTH_SOCK" ] || [ $agent_run_state = 2 ]; then
agent_start
ssh-add
elif [ "$SSH_AUTH_SOCK" ] && [ $agent_run_state = 1 ]; then
ssh-add
fi
Tip
Git Bash can also be used with Windows Terminal using this stackoverflow answer: https://stackoverflow.com/a/57369284.
Generating Key Pairs¶
The instructions for generating key pairs are identical for all operating systems. GitHub maintains excellent documentation on generating key pairs. The gist of those instructions follows.
- Open a terminal window.
- Use the command
ssh-keygen -t ed25519 -C "your_email@example.com"
- You will be prompted to choose a location to store the key, including a file name.
- You will be prompted to enter a passphrase to secure the key. It is highly recommended to secure your key pair with a passphrase to minimize risk.
Managing Keys¶
The instructions below are the same for all operating systems with one small exception noted below.
Important
If at any point you encounter an error like below, please check to be sure your ssh-agent
is running based on how you Installed your SSH Client.
Starting the SSH Agent for a Single Session¶
If ssh-agent
isn't already running and you encounter an error, use the following commands to start the ssh-agent
depending on your environment. It is highly recommended to use the most appropriate method described in Install an SSH Client to have ssh-agent
start automatically.
- Linux, MacOS, Git Bash, WSL:
eval $(ssh-agent -s)
- Windows OpenSSH:
start-ssh-agent
Add a Private Key¶
- Move the key file to the
.ssh
directory under your home directory. - Navigate to the
.ssh
folder in a terminal window. -
Run
ssh-add <private_key_file>
Bug
For Linux users and WSL on Windows users. If you experience a
Warning: Unprotected Private Key File
error when usingssh-add
, yourssh
file and directory permissions may be incorrect. To fix, please use the following commands.Tip
MacOS allows storing passphrases to the builtin Keychain with a special flag. Use
ssh-add -K <path/to/private_key_file>
to permanently store the passphrase that goes with the key file.
Remove a Private Key¶
Run ssh-add -d <path/to/private_key_file>
Push a New Public Key File to a Remote Machine¶
To push a new public key file to a remote machine, please use the ssh-copy-id
command. If your ssh-agent
is running and has a known-good private key added, then the command below will work as expected and add the <new_public_keyfile>.pub
to the remote machine. You must also have the private key counterpart <new_private_keyfile>
with the same name as the public key file, without the .pub
extension.
The value <user>
should be replaced with the remote user you will login as. The value <remote_ip>
should be replaced with the IP address of the remote machine.
To verify, use ssh -i ~/.ssh/<new_private_keyfile>.pub <user>@<remote_ip>
.
Remove an Invalid Host Fingerprint¶
Danger
The following command should only be run when reusing a floating IP for a new instance in a cloud context. Using it arbitrarily for remote machines you do not control can result in a security breach. Be absolutely certain you trust the source of the key change.
A "Remote Host Identification Has Changed" error can be resolved by using the following command. It looks like the image below.
Run ssh-keygen -R <hostname>
where <hostname>
is the URL or IP address of the remote machine.
Setting up a Configuration File¶
SSH configuration files help streamline the process of logging in to remote terminals by storing commonly-used arguments and flags for each host. To create a configuration file, navigate to your .ssh
directory. Create a new plain text file called config
with no extension. Open the file and add content like the following. Note that indent matters. Variable values in <>
will be replaced with appropriate values before saving.
- Be sure to give a meaningful name under
<host>
so you can easily refer back to this config later and for ease of typing when usingssh
with this configuration. Only letters, numbers, dashes and underscores are allowed, and it must start with a letter. - The value
<remote_ip>
can be any remote machine relevant to your work. For cloud.rc it should be whatever IP was assigned in Creating a Floating IP. - The value
<user>
should be whatever user name you will log in as. For cloud.rc,ubuntu
orcentos
are typical, depending on instance operating system. - The value
<path_to_private_key_file>
is the absolute path to the private key file, e.g. the path to your.ssh
folder followed by the<private_key_file>
file name. For cloud.rc this will be whatever private key file was generated in Creating a Key Pair.
Save the config
file. Start a new terminal and use the command ssh <host>
, with no other flags, to test.
SSH Client Usage¶
If you've Set up a Configuration File, simply use ssh <host>
, using the configuration name, to connect.
If you haven't set up a configuration file, use the following.
Where user
is the remote username, remote_ip
is the IP address of the remote machine, and <private_key_file>
is the private key file used for access the remote machine. See Generating Key Pairs for general instructions on creating a key pair, or Creating a Key Pair for cloud.rc specific instructions.
Server Software¶
Remotely accessing server software requires configuration of Security Groups to open ports the server will communicate on. Please see our information on Installing Server Software for details.
Data Transfer¶
SCP¶
SCP stands for Secure CoPy and works like the cp
command, but allows transferring files and directories with remote machines. SCP is built on top of SSH and is installed with most SSH Clients. To install SCP, see Install an SSH Client.
Warning
The OpenSSH developers recommend using SFTP instead of SCP. Future releases of OpenSSH will have SCP use SFTP protocol.
The value <user>
is the user you will login as on the remote machine <hostname>
. Note that if you are using an SSH Configuration File with Host <host>
, then replace all of <user>@<hostname>
with just <host>
, as you would with SSH.
scp <source_file> <user>@<hostname>:<destination_file> # single file
scp <source_file> <user>@<hostname>: # retains the file name
scp -r <source_directory> <user>@<hostname>:<destination_directory> # full directory
scp -r <source_directory> <user>@<hostname>: # retains the directory name
Examples:
# file
scp script.py cheaha:
scp script.py user@cheaha.rc.uab.edu:~/existing/shared_script.py
scp script.py user@<cloud_vm_ip>:
# directory
scp -r my_scripts/ user@cheaha.rc.uab.edu:
scp -r my_scripts/ cheaha:my_shared_scripts/
SFTP¶
SFTP stands for Secure File Transfer Protocol and allows transferring files and directories with remote machines. SFTP is built on top of SSH and is installed with most SSH Clients. To install SFTP, see Install an SSH Client.
SFTP works differently from SCP, as it has an interactive prompt. When connected to a remote, the prompt sftp>
will appears and enable use of SFTP commands. SFTP can also be used in batch mode with the -b <batch_file>
argument. The plaintext <batch_file>
should contain one SFTP command per line.
To connect, use sftp <user>@<hostname>
where <user>
is the user you will login as on the remote machine <hostname>
. If you are using an SSH Configuration File with Host <host>
, you may use sftp <host>
. You may optionally use sftp <host>:/path/to/dir
to start in a specific directory.
Some examples of commands are given below. A complete list is available here
# general commands
sftp> pwd # remote current directory
sftp> lpwd # local current directory
sftp> ls # contents of remote pwd
sftp> lls # contents of local pwd
sftp> mkdir my_dir # create my_dir on remote
sftp> lmkdir my_dir # create my_dir on local
sftp> cd my_dir # change directory on remote
sftp> lcd my_dir # change directory on local
# copy to remote
sftp> put <local_file> <optional_remote_path>
sftp> put -r <local_directory> <optional_remote_path>
# if optional remote path is not supplied, uses pwd
# file
sftp> put script.py # copies file to pwd, same name
sftp> put script.py shared_script.py # copies file to pwd, renames
sftp> put script.py all_scripts/ # copies to pwd existing subdirectory of pwd
# directory, must use trailing '/' character!
sftp> put -r my_scripts/ # copies directory to pwd
sftp> put -r my_scripts/ all_scripts/ # copies to existing subdirectory of pwd
# copy from remote, same syntax as put, reversed direction
sftp> get <remote_file> <optional_local_path>
sftp> get -r <remote_directory> <optional_local_path>
# if optional local path is not supplied, uses lpwd
RClone¶
Please see our RClone page for more information on using RClone with the SFTP remote option.