Ssh Remote

SSH Remote Access

Overview

SSH (Secure Shell) provides encrypted remote access, file transfer, and tunneling over untrusted networks. OpenSSH is the standard implementation on Linux, macOS, and Windows (via built-in client). The client configuration lives at ~/.ssh/config and supports per-host settings, identity management, and connection reuse.

When to use: Remote server management, secure file transfers, port forwarding, jump host traversal, automated deployments, SOCKS proxying.

When NOT to use: High-throughput bulk data transfer across WANs (use Globus or similar), GUI-heavy remote desktop (use VNC/RDP), container orchestration (use kubectl/docker CLI).

Quick Reference

Key Type Recommendations

File Transfer Decision Guide

Common Mistakes

Security Checklist

• Generate Ed25519 keys with strong passphrases
• Set PasswordAuthentication no on servers
• Set PermitRootLogin prohibit-password or no
• Use IdentitiesOnly yes in client config
• Restrict keys with command= and from= in authorized_keys
• Use FIDO2 hardware keys (ed25519-sk) for high-security environments
• Install fail2ban on servers to block brute-force attempts
• Consider SSH certificate authentication for fleet management

Delegation

• Server inventory discovery and connection testing: Use Explore agent
• Multi-host deployment or bulk file transfers: Use Task agent
• Network architecture and bastion host planning: Use Plan agent

References

• Connections, SSH config, and remote commands
• File transfers with rsync and scp
• Port forwarding, SOCKS proxy, and jump hosts
• Key management, FIDO2 keys, agent, and security hardening

Connections

Basic Connection

Connect to a remote server with default settings:

ssh user@hostname

Connect on a non-default port:

ssh -p 2222 user@hostname

Connect with a specific identity (private key) file:

ssh -i ~/.ssh/my_key user@hostname

SSH Config File

The client config file at ~/.ssh/config defines per-host settings that simplify repeated connections. Entries are matched top-down; the first matching Host block wins for each directive.

Host myserver
    HostName 192.168.1.100
    User deploy
    Port 22
    IdentityFile ~/.ssh/myserver_key

After defining a config entry, connect with just the alias:

ssh myserver

Wildcard Defaults

Apply settings to all hosts with a wildcard block. Place this at the end of the config file so host-specific blocks take precedence:

Host *
    ServerAliveInterval 60
    ServerAliveCountMax 3
    AddKeysToAgent yes
    IdentitiesOnly yes

• ServerAliveInterval sends keepalive packets to prevent idle disconnects
• ServerAliveCountMax disconnects after this many missed keepalives
• AddKeysToAgent yes automatically adds keys to the running ssh-agent
• IdentitiesOnly yes prevents offering every key in the agent to every host

Multiple Identity Files

Specify different keys for different services:

Host github.com
    IdentityFile ~/.ssh/github_ed25519

Host gitlab.com
    IdentityFile ~/.ssh/gitlab_ed25519

Host production
    HostName prod.example.com
    User deploy
    IdentityFile ~/.ssh/prod_ed25519

Running Remote Commands

Execute a single command on the remote host and return:

ssh user@host "ls -la /var/log"

Chain multiple commands:

ssh user@host "cd /app && git pull && systemctl restart myapp"

Allocate a pseudo-terminal for interactive commands. Without -t, commands that expect a terminal (like htop, vim, top) fail or produce garbled output:

ssh -t user@host "htop"

Force pseudo-terminal allocation even when stdin is not a terminal (useful in scripts piping commands):

ssh -tt user@host "sudo systemctl restart nginx"

Passing Environment Variables

Send local environment variables to the remote host (requires AcceptEnv on the server):

ssh -o SendEnv=MY_VAR user@host "echo \$MY_VAR"

Known Hosts Management

When connecting to a host for the first time, SSH prompts to verify the host key fingerprint. Accepted keys are stored in ~/.ssh/known_hosts.

Remove an outdated host key (after server rebuild or IP change):

ssh-keygen -R hostname

Pre-scan and add a host key without interactive prompts (useful in automation):

ssh-keyscan -t ed25519 hostname >> ~/.ssh/known_hosts

Scan for a specific key type to avoid adding weaker algorithms:

ssh-keyscan -t ed25519 -p 2222 hostname >> ~/.ssh/known_hosts

Hash Known Hosts

For privacy, hash hostnames in known_hosts so they are not readable if the file is compromised:

Host *
    HashKnownHosts yes

Debugging Connection Issues

SSH provides three verbosity levels. Start with -v and increase if needed:

ssh -v user@host

ssh -vv user@host

ssh -vvv user@host

Key things to look for in verbose output:

• Which config file and Host block matched
• Which identity files were offered
• Authentication methods attempted and their order
• Key exchange and cipher negotiation
• Host key verification status

Common Connection Failures

File Transfer

rsync (Preferred)

rsync is the recommended tool for SSH file transfers. It transfers only changed data blocks (delta sync), supports compression, can resume interrupted transfers, and preserves file metadata. rsync uses SSH as its transport by default.

Basic Patterns

Sync a local directory to a remote host:

rsync -avz ./local/ user@host:/remote/path/

Sync from remote to local:

rsync -avz user@host:/remote/path/ ./local/

The trailing slash on the source path matters:

• ./local/ syncs the contents of local into the destination
• ./local syncs the directory itself, creating local/ inside the destination

Common Flags

rsync -avzP ./local/ user@host:/remote/path/

Dry Run Before Sync

Always preview destructive operations (especially with --delete):

rsync -avzn --delete ./local/ user@host:/remote/path/

Add --itemize-changes for a detailed breakdown of what changed:

rsync -avzn --itemize-changes ./local/ user@host:/remote/path/

Exclude Patterns

Skip files or directories from the sync:

rsync -avz --exclude='node_modules' --exclude='.git' ./project/ user@host:/deploy/

Use an exclude file for complex patterns:

rsync -avz --exclude-from='.rsyncignore' ./project/ user@host:/deploy/

Bandwidth Limiting

Limit bandwidth to avoid saturating the network link:

rsync -avzP --bwlimit=10m ./large-data/ user@host:/backup/

The --bwlimit value accepts suffixes: k (KiB/s), m (MiB/s).

Optimizing Transfer Speed

For high-bandwidth links where CPU is the bottleneck, skip compression and use a faster cipher:

rsync -avP -e "ssh -T -c aes128-gcm@openssh.com -o Compression=no -x" ./data/ user@host:/data/

• -T disables pseudo-terminal allocation
• -c aes128-gcm@openssh.com uses hardware-accelerated AES-GCM cipher
• -o Compression=no skips SSH-level compression (rsync handles its own)
• -x disables X11 forwarding

rsync with SSH Multiplexing

Combine rsync with SSH multiplexing for repeated transfers to the same host:

rsync -avzP -e "ssh -o ControlMaster=auto -o ControlPath=~/.ssh/sockets/%r@%h-%p -o ControlPersist=600" ./data/ user@host:/data/

Or configure multiplexing globally in ~/.ssh/config and rsync will use it automatically.

Mirror with Delete

Create an exact mirror of the source (removes extra files on destination):

rsync -avz --delete ./source/ user@host:/mirror/

scp (Legacy)

The SCP protocol is deprecated by the OpenSSH project due to security design issues. The scp command still exists in modern OpenSSH but uses SFTP internally. For new workflows, prefer rsync or sftp.

Basic scp Patterns

Copy a file to a remote host:

scp local.txt user@host:/remote/path/

Copy a file from a remote host:

scp user@host:/remote/file.txt ./local/

Copy a directory recursively:

scp -r ./local_dir user@host:/remote/path/

When scp Is Acceptable

• Quick one-off file copies where rsync is not installed on the remote host
• Environments where only the scp command is available
• Simple CI/CD pipelines copying single artifacts

For anything recurring, large, or requiring resume capability, use rsync.

sftp

sftp provides an interactive file transfer session over SSH:

sftp user@host

Common sftp commands within the session:

put local-file.txt /remote/path/
get /remote/file.txt ./local/
ls /remote/path/
mkdir /remote/new-dir

Non-interactive sftp for scripting (batch mode):

sftp -b commands.txt user@host

Where commands.txt contains one sftp command per line.

Key Management

Key Generation

Ed25519 (Recommended)

Ed25519 is the recommended algorithm for all new SSH keys. It provides strong security with small key sizes (256-bit), fast operations, and resistance to side-channel attacks:

ssh-keygen -t ed25519 -C "user@machine"

Use a descriptive comment to identify the key's purpose and origin. Use a strong passphrase when prompted.

Specify a custom filename to organize keys by purpose:

ssh-keygen -t ed25519 -f ~/.ssh/prod_ed25519 -C "deploy@production"

RSA (Legacy Compatibility)

Use RSA only when Ed25519 is not supported by the remote system. Always use 4096-bit minimum:

ssh-keygen -t rsa -b 4096 -C "user@machine"

FIDO2 Hardware-Backed Keys

FIDO2 keys (ed25519-sk) bind the private key to a physical security key (YubiKey, SoloKey, etc.). The private key never leaves the hardware device. Requires OpenSSH 8.2+ (8.3+ for verify-required).

Generate a FIDO2 key:

ssh-keygen -t ed25519-sk -C "user@machine (FIDO2)"

Generate a resident FIDO2 key (stored on the security key, portable across machines):

ssh-keygen -t ed25519-sk -O resident -O verify-required -C "user@machine (FIDO2)"

• -O resident stores the credential on the hardware key for portability
• -O verify-required requires physical touch (PIN or biometric) during authentication

Recover resident keys on a new machine:

ssh-keygen -K

This extracts key handles from the security key and writes them to disk. The "private key" file contains only a reference; the actual private key remains on the hardware.

Key Algorithm Comparison

Deploying Public Keys

Copy a public key to a remote server's authorized_keys:

ssh-copy-id user@host

Copy a specific key:

ssh-copy-id -i ~/.ssh/prod_ed25519.pub user@host

Manual deployment when ssh-copy-id is not available:

ssh user@host "mkdir -p ~/.ssh && chmod 700 ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys" < ~/.ssh/id_ed25519.pub

SSH Agent

The SSH agent caches decrypted private keys in memory, eliminating repeated passphrase entry during a session.

Start the agent (if not already running):

eval "$(ssh-agent -s)"

Add a key to the agent:

ssh-add ~/.ssh/id_ed25519

Add with macOS Keychain integration (persists across reboots):

ssh-add --apple-use-keychain ~/.ssh/id_ed25519

List loaded keys:

ssh-add -l

Remove all keys from the agent:

ssh-add -D

Automatic Agent Key Loading

Configure SSH to add keys to the agent automatically on first use. In ~/.ssh/config:

Host *
    AddKeysToAgent yes

On macOS, also add Keychain support:

Host *
    AddKeysToAgent yes
    UseKeychain yes

Connection Multiplexing

Multiplexing reuses a single TCP connection for multiple SSH sessions to the same host. Subsequent connections authenticate instantly and skip the TCP/TLS handshake.

Configuration

Add to ~/.ssh/config:

Host *
    ControlMaster auto
    ControlPath ~/.ssh/sockets/%r@%h-%p
    ControlPersist 600

Create the socket directory:

mkdir -p ~/.ssh/sockets

• ControlMaster auto creates a master connection if none exists, reuses if one does
• ControlPath defines the socket file location; include %r (user), %h (host), %p (port) for unique sockets
• ControlPersist 600 keeps the master alive for 10 minutes after the last session disconnects

Managing Multiplexed Connections

Check if a master connection is active:

ssh -O check myserver

Gracefully stop accepting new sessions (existing sessions continue):

ssh -O stop myserver

Immediately terminate the master and all sessions:

ssh -O exit myserver

Per-Host Multiplexing

Enable multiplexing only for frequently accessed hosts:

Host production-*
    ControlMaster auto
    ControlPath ~/.ssh/sockets/%r@%h-%p
    ControlPersist 30m

Host dev-*
    ControlMaster auto
    ControlPath ~/.ssh/sockets/%r@%h-%p
    ControlPersist 5m

Security Hardening

Server-Side Configuration

Key settings for /etc/ssh/sshd_config:

PasswordAuthentication no
PermitRootLogin prohibit-password
PubkeyAuthentication yes
MaxAuthTries 3
MaxSessions 10
AllowUsers deploy admin

• PasswordAuthentication no forces key-based authentication, eliminating brute-force password attacks
• PermitRootLogin prohibit-password allows root login only with keys (or use no to disable entirely)
• AllowUsers restricts which users can log in via SSH

Client-Side Security Practices

Restrict key usage in ~/.ssh/config:

Host *
    IdentitiesOnly yes

This prevents the SSH client from offering every key in the agent to every server. Without it, a server sees all your loaded keys, which leaks information.

Restricting Keys in authorized_keys

Limit what a key can do on the server:

command="/usr/local/bin/deploy.sh",no-port-forwarding,no-X11-forwarding,no-agent-forwarding ssh-ed25519 AAAA... deploy@ci

File Permissions

SSH enforces strict file permissions. Incorrect permissions cause silent authentication failures:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/config
chmod 600 ~/.ssh/id_ed25519
chmod 644 ~/.ssh/id_ed25519.pub
chmod 600 ~/.ssh/authorized_keys

Brute-Force Protection

Install fail2ban on servers to automatically block IPs after repeated failed login attempts:

sudo apt install -y fail2ban
sudo systemctl enable fail2ban

The default configuration monitors SSH and bans offending IPs. Customize ban thresholds in /etc/fail2ban/jail.local.

SSH Certificate Authentication

For organizations managing many servers, SSH certificates provide centralized trust without distributing individual public keys to every server.

How It Works

1. A Certificate Authority (CA) key pair is created 2. User public keys are signed by the CA, producing a certificate 3. Servers trust the CA public key, automatically accepting any certificate it signed

Signing a User Key

ssh-keygen -s /path/to/ca_key -I "user-identity" -n deploy -V +52w ~/.ssh/id_ed25519.pub

Server Trust Configuration

Add the CA public key to the server's sshd_config:

TrustedUserCAKeys /etc/ssh/ca_user_key.pub

Certificates can be combined with FIDO2 keys for hardware-backed, centrally managed authentication.

Tunneling

Local Port Forwarding

Local forwarding makes a remote service accessible on a local port. Traffic flows: local port -> SSH tunnel -> remote host -> target.

Access a remote web server locally:

ssh -L 8080:localhost:80 user@host

This binds localhost:8080 on your machine. Connections to it are forwarded through the SSH tunnel to port 80 on the remote host.

Forwarding to a Third Host

Forward through the SSH host to a different machine on the remote network:

ssh -L 5432:db-server:5432 user@jumphost

This makes db-server:5432 (accessible from jumphost) available at localhost:5432. The connection from jumphost to db-server is unencrypted unless db-server provides its own TLS.

Background Tunnel

Open a tunnel without an interactive shell, running in the background:

ssh -fNL 8080:localhost:80 user@host

Bind to All Interfaces

By default, local forwards bind to localhost only. To make the tunnel accessible from other machines on your network:

ssh -L 0.0.0.0:8080:localhost:80 user@host

Requires GatewayPorts yes on the SSH server for remote forwards.

Remote Port Forwarding

Remote forwarding exposes a local service to the remote host. Traffic flows: remote port -> SSH tunnel -> your machine -> target.

ssh -R 9000:localhost:3000 user@host

Connections to port 9000 on the remote host are forwarded to localhost:3000 on your machine. Useful for exposing a local development server to a remote environment.

Persistent Remote Tunnel

Combine with background and no-command flags:

ssh -fNR 9000:localhost:3000 user@host

Server-Side Configuration

The SSH server must allow remote forwarding. In /etc/ssh/sshd_config:

GatewayPorts clientspecified

Without this, remote forwards bind only to localhost on the server, preventing external access.

Dynamic Port Forwarding (SOCKS Proxy)

Dynamic forwarding creates a local SOCKS5 proxy. Applications configured to use this proxy route all traffic through the SSH tunnel:

ssh -D 1080 user@host

This creates a SOCKS5 proxy at localhost:1080. Configure browsers or applications to use localhost:1080 as a SOCKS5 proxy to route traffic through the remote host.

Background SOCKS proxy:

ssh -fND 1080 user@host

Use with curl

Test the proxy with curl:

curl --socks5-hostname localhost:1080 https://example.com

The --socks5-hostname flag routes DNS through the proxy as well, preventing DNS leaks.

Jump Hosts (ProxyJump)

ProxyJump (-J flag, available since OpenSSH 7.3) connects to a target through one or more intermediate hosts without agent forwarding. This is more secure than ForwardAgent because the jump host never has access to your SSH keys.

Command-Line Usage

Single jump host:

ssh -J jumphost user@internal-server

Multiple jump hosts (comma-separated):

ssh -J jump1,jump2 user@internal-server

With explicit users and ports:

ssh -J admin@jump1:2222,deploy@jump2:22 user@target

SSH Config

Define jump host relationships in ~/.ssh/config:

Host bastion
    HostName bastion.example.com
    User admin
    IdentityFile ~/.ssh/bastion_ed25519

Host internal
    HostName 10.0.0.50
    User deploy
    ProxyJump bastion
    IdentityFile ~/.ssh/internal_ed25519

Then connect directly:

ssh internal

Chained ProxyJump in Config

Chain multiple hops by referencing config aliases:

Host jump1
    HostName jump1.example.com
    User admin

Host jump2
    HostName jump2.internal
    ProxyJump jump1
    User admin

Host target
    HostName 10.0.0.100
    ProxyJump jump2
    User deploy

File Transfer Through Jump Hosts

rsync and scp work with ProxyJump:

rsync -avzP -e "ssh -J jumphost" ./files/ user@target:/path/

scp -J jumphost local.txt user@target:/path/

Port Forwarding Through Jump Hosts

Combine tunneling with jump hosts:

ssh -L 8080:localhost:80 -J jumphost user@internal-server

ProxyJump vs. Legacy ProxyCommand

ProxyJump replaces the older ProxyCommand directive. Use ProxyJump for all new configurations:

# Legacy (avoid for new setups)
Host internal
    ProxyCommand ssh -W %h:%p bastion

# Current (preferred)
Host internal
    ProxyJump bastion

ProxyJump advantages over ProxyCommand:

• Simpler syntax
• Supports comma-separated chaining
• No shell escaping issues
• Works with -J on the command line