Copy a remote directory with scp—and prove the copy is complete

You can copy a directory from a server with one command. The difficult part is knowing exactly which directory level will appear locally, whether symbolic links will pull in unexpected data, and whether “100%” on a progress line means the usable tree arrived. Treat the transfer and its verification as two separate operations.

Inspect the source before transferring it

ssh -p 2222 ops@files.example.net   'set -eu; du -sh -- /srv/releases/app; find /srv/releases/app -xdev -type f | wc -l'

Run the inventory with an account that can read the intended tree. Record the apparent size, file count and filesystem boundary. A command copied from a ticket may point at a symlink, a mount, or yesterday’s release. Resolve that ambiguity first:

ssh -p 2222 ops@files.example.net   'readlink -f -- /srv/releases/app; find /srv/releases/app -maxdepth 2 -type l -ls'

OpenSSH documents that recursive scp follows symbolic links encountered while walking the tree. A link to a large archive or another sensitive path can therefore expand into copied content. If that is not intended, package an explicit file list or choose a transfer tool whose symlink policy you can control.

Copy the directory from the local machine

mkdir -p -- "$HOME/recovery"
scp -r -P 2222   ops@files.example.net:/srv/releases/app   "$HOME/recovery/"

Run this on the receiving machine. The capital -P selects the SSH port; lowercase -p preserves modification times, access times and mode bits. The final result here is $HOME/recovery/app. Quote local paths that contain spaces, and prefer a configured SSH host alias to repeatedly typing identity and jump-host options.

Remove path ambiguity with an SSH configuration alias

Host release-files
    HostName files.example.net
    User ops
    Port 2222
    IdentityFile ~/.ssh/release_readonly
    IdentitiesOnly yes
    ProxyJump bastion.example.net

ssh release-files 'test -d /srv/releases/app'
scp -rp release-files:/srv/releases/app "$HOME/recovery/"

OpenSSH applies the same authentication security as an SSH login. Verify the host key through an approved channel on first connection; do not work around a mismatch with StrictHostKeyChecking=no. A changed key can indicate a legitimate rebuild, a stale DNS record, or an interception attempt—the correct response is investigation.

Know what modern scp actually speaks

Since OpenSSH 9.0, scp uses SFTP for transfers by default. The familiar host:path interface remains, but old shell-expansion behavior may differ. The -O option forces the legacy SCP protocol only for compatibility with old servers or particular legacy wildcard behavior. Do not add it unless you can name the incompatibility and accept the remote-shell quoting risk.

scp -V 2>&1 || ssh -V
scp -vv -r release-files:/srv/releases/app "$HOME/recovery/"

Use verbose mode for a failed connection, then remove sensitive host and account details before sharing logs. It can distinguish DNS, routing, host-key, authentication, subsystem and permission failures.

Preserve metadata deliberately, not automatically

umask 077
scp -rp release-files:/srv/releases/app "$HOME/recovery/"

-p preserves times and mode bits, but it does not make a byte-for-byte filesystem replica with owners, ACLs, extended attributes, hard links, sparse-file layout or every special file. The local user also needs permission to recreate the requested mode. For an application backup, write down which metadata is part of the recovery contract before selecting the tool.

Requirement	Practical choice
Small one-off directory copy	scp -r plus verification
Interrupted multi-gigabyte transfer	rsync over SSH where approved
Scripted SFTP-only endpoint	sftp batch mode
Exact archive semantics	Create and verify a tar archive, then transfer it
Snapshot-consistent database data	Database-native backup or filesystem snapshot first

Do not copy a changing tree and call it a backup

scp reads files while the application may still be changing them. The resulting directory can contain files from different points in time. For release artifacts, copy an immutable release directory. For mutable uploads, coordinate a snapshot. For databases, use the database’s supported backup mechanism. Pausing an application without understanding queues and timeouts can create a larger incident.

ssh release-files   'test -f /srv/releases/app/RELEASE_ID && cat /srv/releases/app/RELEASE_ID'
cat "$HOME/recovery/app/RELEASE_ID"

A release identifier is a useful consistency marker, but it is not a substitute for a snapshot if files can change independently.

Verify the received tree independently

(
  cd "$HOME/recovery/app"
  find . -xdev -type f -print0 | sort -z | xargs -0 sha256sum
) > local.sha256

ssh release-files   "cd /srv/releases/app && find . -xdev -type f -print0 |
   sort -z | xargs -0 sha256sum" > remote.sha256

diff -u remote.sha256 local.sha256

Both manifests use paths relative to the application root, so diff compares the same names and hashes. Test the pipeline on a small directory first. Hashing reads every file again and can be expensive on busy storage. At minimum, compare file counts, total bytes and critical artifact hashes. Never delete the source merely because scp exited zero.

Handle partial failure without making it worse

destination="$HOME/recovery/app.incoming"
mkdir -p -- "$destination"
scp -rp release-files:/srv/releases/app/. "$destination/"
test -f "$destination/RELEASE_ID"
mv -- "$destination" "$HOME/recovery/app.verified"

Stage into a new destination instead of merging blindly into a previously failed copy. If the connection drops, retain the partial tree for diagnosis or resume with an approved tool. Do not repeatedly overwrite the only known-good recovery copy.

Diagnose the common failures

ssh -G release-files | grep -E '^(hostname|user|port|identityfile|proxyjump) '
ssh release-files 'namei -l /srv/releases/app'
df -h -- "$HOME/recovery"
df -i -- "$HOME/recovery"

• No such file: confirm which host expands the path and whether the account is chrooted.
• Permission denied: separate SSH authentication failure from source read permission.
• Disk full: check both bytes and inodes on the receiver.
• Unexpected nesting: compare copying app with copying app/. into an existing directory.
• Broken automation: use batch authentication and check the process exit code; never parse the progress meter.

Close with a restore test

test -r "$HOME/recovery/app.verified/config/schema.json"
sha256sum -c critical-files.sha256
./restore-smoke-test "$HOME/recovery/app.verified"

A directory that cannot be restored is only a collection of copied bytes. Exercise the consumer, record the source host and release ID, retain the verification manifest, and apply the required encryption and retention controls to the local copy.

The transfer command is the easy line. Confidence comes from controlling the source state, understanding path and symlink behavior, and proving the received tree satisfies the restore contract.

Sources: the solved Stack Overflow question (question by Slasengger; accepted answer by Gryphius, CC BY-SA) and the official OpenBSD/OpenSSH scp(1) manual.