Ubuntu 24.04 Server Hardening: A Production Checklist Built From 1,422 Real Attacks
Published 6 September 2026. Every log figure below comes from a live internet-facing Ubuntu 24.04 server we operate, over the eleven days it has been running. The fail2ban findings were reproduced with fail2ban-regex against the shipped filter.
1. What Actually Attacks a New VPS
Over one seven-day window, our server recorded 1,422 failed SSH authentication attempts from 27 distinct addresses. One address accounted for 1,404 of them — 99% of the traffic from a single host working through a username list. The names it tried were exactly what you would expect: admin, user, pi, support, installer, guest.
Every one of those attempts failed at the protocol level, because password authentication was disabled. That is the entire defence, and it is one line of configuration. Key-only authentication does not slow an attacker down; it removes the attack.
The uncomfortable part of the same dataset: fail2ban was installed on that machine and banned none of those 1,404 attempts. It was not running during that window. Section 4 covers how we found that, and the more subtle failure underneath it, which is that fail2ban’s default configuration does not recognise a failed login on a server where password authentication is switched off. Installed is not the same as working, and the difference is invisible until you check.
Priority Matrix
| Control | Effort | Effect on the traffic we measured | Priority |
|---|---|---|---|
| Key-only SSH authentication | 10 min | Neutralises 100% of the 1,422 attempts | First |
| Firewall denying inbound by default | 5 min | Removes every service you did not intend to expose | First |
| Unattended security upgrades | 2 min | Closes CVEs you will never read about | First |
| Databases and caches on loopback | 5 min | An exposed Redis is compromised in hours | First |
| fail2ban, correctly configured | 20 min | Stops log noise; adds little once keys are enforced | Second |
| Disabling root login entirely | 15 min | Removes the single most targeted account | Second |
| File integrity baseline | 15 min | The only control that answers “what changed” | Second |
| Changing the SSH port | 5 min | Reduces log volume, not risk | Optional |
The ordering matters more than the list. Key-only authentication is worth more than everything below it combined, and it is the control most often deferred because copying a key feels like a chore. The 1,404 attempts from one address are what deferring it costs you.
2. SSH: The Control That Does the Work
Generate the key pair on your workstation. Never on the server, because a private key that has touched the machine you are protecting is no longer private to you.
# On your local machine
ssh-keygen -t ed25519 -a 100 -C "deploy@$(hostname)" -f ~/.ssh/id_ed25519_prod
ssh-copy-id -i ~/.ssh/id_ed25519_prod.pub root@YOUR_SERVER_IP
# Confirm the key works BEFORE changing anything on the server
ssh -i ~/.ssh/id_ed25519_prod root@YOUR_SERVER_IP 'echo key auth OK'Ed25519 rather than RSA: shorter keys, faster verification, and no key-size decision to get wrong. The -a 100 raises the KDF rounds protecting the private key on disk, which matters if your laptop is ever stolen.
Create the unprivileged account first
adduser --gecos "" deploy
usermod -aG sudo deploy
install -d -m 700 -o deploy -g deploy /home/deploy/.ssh
cp /root/.ssh/authorized_keys /home/deploy/.ssh/authorized_keys
chown deploy:deploy /home/deploy/.ssh/authorized_keys
chmod 600 /home/deploy/.ssh/authorized_keys
# Verify from a SECOND terminal before proceeding. Do not close the first.
ssh -i ~/.ssh/id_ed25519_prod deploy@YOUR_SERVER_IP 'sudo -n true && echo sudo OK'Do this before disabling root login, not after. We audited our own server while writing this article and found exactly the wrong order: three keys installed for root, a deploy-equivalent account in the sudo group with zero keys, and PermitRootLogin without-password still in effect. Setting PermitRootLogin no in that state locks you out of your own machine. Check before you tighten:
# Which accounts can actually log in with a key?
for h in /root /home/*; do
[ -f "$h/.ssh/authorized_keys" ] && \
printf "%-20s %s key(s)\n" "$(basename $h)" \
"$(grep -c '^ssh-\|^ecdsa-' $h/.ssh/authorized_keys)"
done
# Who has sudo?
getent group sudo | cut -d: -f4Harden the daemon
cat > /etc/ssh/sshd_config.d/99-hardening.conf <<'EOF'
PermitRootLogin no
PasswordAuthentication no
KbdInteractiveAuthentication no
PubkeyAuthentication yes
AuthenticationMethods publickey
PermitEmptyPasswords no
X11Forwarding no
AllowAgentForwarding no
MaxAuthTries 3
MaxSessions 4
ClientAliveInterval 300
ClientAliveCountMax 2
LoginGraceTime 30
EOF
# Parse before reload. Exits non-zero on error, so && prevents a lockout.
sshd -t && systemctl reload sshUbuntu 24.04 places drop-in files in /etc/ssh/sshd_config.d/ and includes them from the main config. Editing sshd_config directly works but is overwritten by package upgrades; a drop-in survives them.
Then confirm what the daemon actually resolved, rather than what you wrote. These differ more often than people expect, because a directive earlier in the include order wins:
sshd -T | grep -E "^(permitrootlogin|passwordauthentication|pubkeyauthentication|kbdinteractiveauthentication|maxauthtries|permitemptypasswords)"
# On a correctly hardened server:
# permitrootlogin no
# passwordauthentication no
# pubkeyauthentication yes
# kbdinteractiveauthentication no
# maxauthtries 3
# permitemptypasswords nopermitrootlogin without-password in that output means root can still authenticate with a key. That is a defensible position on a single-admin machine and a weaker one than no, because root is the account every scanner tries first.
Managing keys once there is more than one
Key-only authentication moves the problem rather than removing it: the question becomes which keys are trusted, and whether anyone remembers. Our own audit found three keys on root and no record of what the third one was.
# List trusted keys with their fingerprints and comments
ssh-keygen -lf /root/.ssh/authorized_keys
# A key with no comment, or a comment naming someone who left,
# is a key you cannot account for. Remove it.
# Keep one key per person or per machine, never a shared one:
# revoking a shared key locks out everyone at once.Use the comment field as an inventory. ssh-keygen -C "alice@laptop-2026" costs nothing at creation and is the difference between a confident removal and leaving an unknown key in place because nobody dares delete it.
# Removing a key: keep a copy, verify access still works, then commit
cp /root/.ssh/authorized_keys /root/authorized_keys.bak-$(date +%F)
grep -v "old-laptop-2023" /root/.ssh/authorized_keys > /tmp/ak && \
mv /tmp/ak /root/.ssh/authorized_keys
chmod 600 /root/.ssh/authorized_keys
# Then open a NEW session before closing this one.The last line is the rule that governs every change in this article. Any modification to authentication is made with a working session held open, and verified from a second one. There is no recovery from a locked-out VPS other than the provider’s console, and on some plans that console is slower to reach than the outage it is meant to fix.
3. Firewall, Updates and Service Exposure
ufw default deny incoming
ufw default allow outgoing
ufw allow OpenSSH
ufw allow 80/tcp
ufw allow 443/tcp
ufw --force enable
ufw status verboseDeny by default, then open the two ports the internet needs plus the one you need. Note what is deliberately absent: MySQL’s 3306, PostgreSQL’s 5432 and Redis’s 6379. Those bind to loopback and are reached over it. An internet-facing Redis without authentication is compromised within hours, and the attack requires no exploit — CONFIG SET dir is a documented feature that writes files where you point it.
# Verify what is actually listening on a public interface
ss -tlnp | awk 'NR==1 || $4 !~ /^(127\.|\[::1\])/'
# Anything answering on 0.0.0.0 or * is reachable from the internet
# unless the firewall stops it. Databases and caches must show 127.0.0.1.The Docker exception that defeats your firewall
If Docker is installed, UFW is not the whole story. Docker writes its own iptables rules in a chain consulted before UFW’s, so a container published as -p 5432:5432 is reachable from the internet while UFW reports the port as denied. This is documented behaviour, not a bug, and it is the most common way a self-managed server leaks a database.
# WRONG on a firewalled host
ports:
- "5432:5432"
# CORRECT: reachable only from the host itself
ports:
- "127.0.0.1:5432:5432"# Verify from OUTSIDE the machine, which is the only test that counts
nmap -Pn -p 3306,5432,6379,8080 YOUR_PUBLIC_IP
# Expect: filtered or closed on every oneAutomatic security updates
apt-get update && apt-get install -y unattended-upgrades apt-listchanges
dpkg-reconfigure --priority=low unattended-upgrades
systemctl enable --now unattended-upgrades
# Prove it is armed rather than merely installed
systemctl is-active unattended-upgrades
unattended-upgrade --dry-run --debug 2>&1 | tail -20The dry run is the point. A package installed with its timer disabled looks identical to a working one in every check except this one, and the failure mode is silent for months.
# Reboot when the kernel requires it, at a time you choose
# /etc/apt/apt.conf.d/50unattended-upgrades
Unattended-Upgrade::Automatic-Reboot "true";
Unattended-Upgrade::Automatic-Reboot-Time "04:00";
# Then confirm your services actually come back after one
systemctl is-enabled nginx php8.3-fpm mysql redis-server dockerAutomatic reboots are worth enabling only if you have verified that everything restarts unattended. A queue worker that does not survive a reboot fails silently, and you discover it from the work that never happened. The same applies to Docker stacks: restart: unless-stopped on every service, and systemctl is-enabled docker.
4. fail2ban: The Part Almost Everyone Gets Wrong
Two distinct failures, both silent, both present on our own server before we audited it for this article.
Failure one: it was not running
The 1,404-attempt burst happened between 23 and 30 August. Our fail2ban service last started on 2 September. The offending address appears zero times in the fail2ban log, and was never banned. The package was installed; the service was not up during the attack.
# Is it running, and since when?
systemctl is-active fail2ban
systemctl show fail2ban -p ActiveEnterTimestamp --value
# Has it ever actually banned anything?
grep -c " Ban " /var/log/fail2ban.log
fail2ban-client status sshdFailure two: the default filter ignores your attackers
This one is subtler and survives a working installation. fail2ban’s sshd filter defaults to mode = normal, which matches lines like Failed password for invalid user admin. On a server with PasswordAuthentication no, sshd never writes that line, because no password was ever offered. What it writes instead is:
sshd[229299]: Invalid user bot from 203.0.113.9 port 45308
sshd[229299]: Connection closed by invalid user bot 203.0.113.9 port 45308 [preauth]Read the shipped filter and the reason becomes explicit. In normal mode that second line is wrapped in <F-NOFAIL>, which tells fail2ban to treat it as not a failure:
# /etc/fail2ban/filter.d/sshd.conf
mdre-normal-other = ^<F-NOFAIL><F-MLFFORGET>(Connection (?:closed|reset)|Disconnected)...
# The aggressive variant matches the same line WITHOUT the NOFAIL wrapper:
mdre-ddos-other = ^<F-MLFFORGET>(Connection (?:closed|reset)|Disconnected)... \[preauth\]\s*$
mdre-aggressive = %(mdre-ddos)s %(mdre-extra)sYou can test this in one command, against a real log line, without waiting for an attack:
L="Sep 06 03:22:57 host sshd[229299]: Connection closed by invalid user bot 203.0.113.9 port 45308 [preauth]"
fail2ban-regex "$L" /etc/fail2ban/filter.d/sshd.conf
# Lines: 1 lines, 1 ignored, 0 matched, 0 missed <-- IGNORED
fail2ban-regex "$L" "/etc/fail2ban/filter.d/sshd.conf[mode=aggressive]"
# Lines: 1 lines, 0 ignored, 1 matched, 0 missed <-- COUNTEDThose two outputs are verbatim from our server. The default configuration ignores exactly the log line that a hardened server produces. The fix is one directive:
# /etc/fail2ban/jail.local
[DEFAULT]
bantime = 3600
findtime = 600
maxretry = 5
ignoreip = 127.0.0.0/8 ::1 YOUR.OFFICE.IP.HERE
[sshd]
enabled = true
mode = aggressive
# With PasswordAuthentication no, bots only produce
# "Connection closed ... [preauth]" lines, ignored in normal mode.
[nginx-http-auth]
enabled = true
[nginx-badbots]
enabled = truefail2ban-client -t # validate before reloading
systemctl reload fail2ban
fail2ban-client get sshd mode # expect: aggressiveWhy you cannot test it from the server
We tried, and the counter did not move. The reason is in the configuration, not the software:
$ fail2ban-client get sshd ignoreip
These IP addresses/networks are ignored:
|- 127.0.0.0/8
|- 169.254.0.0/16
`- ::1Loopback is ignored by design, so a test connection from the machine to itself is invisible to fail2ban. Test with fail2ban-regex against a log line, as above, or from a different host you control — and put your own office address in ignoreip before you do, unless you enjoy waiting out a one-hour ban.
What it looks like when it works
2026-09-06 02:56:38,169 fail2ban.filter [sshd] Found 146.148.9.100
2026-09-06 02:56:38,452 fail2ban.filter [sshd] Found 146.148.9.100
2026-09-06 02:56:38,903 fail2ban.actions [sshd] Ban 146.148.9.100
2026-09-06 03:12:45,955 fail2ban.filter [sshd] Found 197.1.156.241Three detections inside one second, then a ban. That is from our log this morning, with the corrected configuration in place. Compare it to the 1,404 attempts in August that produced nothing at all.
Keep fail2ban in proportion, though. Once key-only authentication is enforced, every one of those attempts was already failing. fail2ban reduces log noise and stops resource waste; it is not what is protecting you. If you had to choose one control, it would not be this one.
Kernel parameters: most of the advice is already the default
Hardening guides routinely publish a block of a dozen sysctl settings to paste into /etc/sysctl.d/. We checked every one of them against a stock Ubuntu 24.04 install. Almost all were already correct out of the box:
| Parameter | Hardened value | Ubuntu 24.04 default | Action needed |
|---|---|---|---|
net.ipv4.tcp_syncookies | 1 | 1 | none |
net.ipv4.conf.all.rp_filter | 1 or 2 | 2 | none |
net.ipv4.conf.all.accept_redirects | 0 | 0 | none |
net.ipv4.conf.all.accept_source_route | 0 | 0 | none |
kernel.randomize_va_space | 2 | 2 | none |
fs.protected_hardlinks | 1 | 1 | none |
fs.protected_symlinks | 1 | 1 | none |
kernel.dmesg_restrict | 1 | 1 | none |
net.ipv4.icmp_echo_ignore_broadcasts | 1 | 1 | none |
net.ipv4.conf.all.send_redirects | 0 | 1 | set it |
Nine of ten already hardened, shipped by Ubuntu in /etc/sysctl.d/10-network-security.conf and its siblings. One genuinely is not: a host that is not a router has no reason to send ICMP redirects.
# /etc/sysctl.d/60-hardening.conf
net.ipv4.conf.all.send_redirects = 0
net.ipv4.conf.default.send_redirects = 0
sysctl --system
sysctl -n net.ipv4.conf.all.send_redirects # expect: 0# Audit your own machine rather than pasting someone else's block.
# Anything already at the hardened value needs no file.
for k in net.ipv4.tcp_syncookies net.ipv4.conf.all.rp_filter \
net.ipv4.conf.all.accept_redirects net.ipv4.conf.all.send_redirects \
net.ipv4.conf.all.accept_source_route kernel.randomize_va_space \
fs.protected_hardlinks fs.protected_symlinks kernel.dmesg_restrict; do
printf "%-44s %s\n" "$k" "$(sysctl -n $k 2>/dev/null)"
doneThe reason to audit rather than paste matters beyond tidiness. A pasted block that contradicts a distribution default is a change nobody remembers making, and it surfaces two years later as behaviour that does not match the documentation. Set what is genuinely wrong, and leave the rest to the people who maintain the distribution.
5. Web Layer, TLS and Certificate Renewal
The certificate is not the risk. The renewal is. A certificate obtained today fails in ninety days, at a moment you have long stopped thinking about it, and on a booking page or a checkout that failure stops revenue rather than merely embarrassing you.
apt-get install -y certbot python3-certbot-nginx
certbot --nginx -d example.com -d www.example.com \
--redirect --agree-tos -m you@example.com --non-interactive
# Prove renewal works. This is the check nobody runs until it is too late.
certbot renew --dry-run
# Then automate the proof, not just the renewal
( crontab -l 2>/dev/null; \
echo "0 4 1 * * certbot renew --dry-run || echo 'CERT RENEWAL CHECK FAILED' | mail -s 'cert check' you@example.com" ) \
| crontab -Certbot edits only the server block whose server_name matches, which is safe on a multi-site machine — but verify that claim rather than trusting it, because a mistake here takes down sites unrelated to the one you were working on:
# Which files did certbot touch?
find /etc/nginx -newermt "-5 minutes" -type f | sed 's/^/ /'
nginx -t && systemctl reload nginxOne detail worth checking afterwards: certbot's generated redirect sends www to https://$host, which produces a two-hop chain (http://www to https://www to https://apex). Rewriting it to redirect straight to the apex saves a round trip on every first visit.
# In the port-80 www block, prefer:
return 301 https://example.com$request_uri;
# over certbot's default:
return 301 https://$host$request_uri;
# Verify the chain is one hop
curl -sIL http://www.example.com/ | grep -E "^HTTP|^[Ll]ocation"Security headers and application isolation
# In each server block
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
add_header X-Frame-Options "SAMEORIGIN" always;
add_header X-Content-Type-Options "nosniff" always;
add_header Referrer-Policy "strict-origin-when-cross-origin" always;
# Deny the paths attackers probe first
location = /xmlrpc.php { return 403; }
location ~ /\.(?!well-known) { deny all; }
location ~* ^/(uploads|storage|cache)/.*\.(php|phtml|phar)$ { deny all; }On a machine hosting several applications, give each its own PHP-FPM pool. A dedicated pool means a runaway script in one site cannot exhaust the worker slots of another, and a compromise in one cannot read another's files through the shared process user. The full pool configuration is in our CodeCanyon deployment guide, and the framework-level equivalent in our Laravel production setup.
Reading your own logs
Monitoring products are not required to answer the questions that matter. Four commands cover most of it, and the first is how the central finding of this article surfaced.
# Who is trying to log in, and how insistently?
grep -oE "from ([0-9]{1,3}\.){3}[0-9]{1,3}" /var/log/auth.log \
| awk '{print $2}' | sort | uniq -c | sort -rn | head -10
# Which usernames? A list of service accounts means a targeted scan;
# admin, pi and guest mean a generic one.
grep -oE "Invalid user [a-zA-Z0-9._-]+" /var/log/auth.log \
| awk '{print $3}' | sort | uniq -c | sort -rn | head -10
# What is probing the web layer? Filter your own testing out first.
awk '$9==404 || $9==403 {print $7}' /var/log/nginx/*access.log \
| sort | uniq -c | sort -rn | head -15
# Did anyone SUCCEED in logging in, and was it you?
grep -E "Accepted (publickey|password)" /var/log/auth.log | tail -20The fourth command is the one to run first after any suspicion. A successful authentication you cannot account for changes the situation from hardening to incident response, and the difference between the two is whether you preserve evidence before you start deleting things.
One caution from our own data: on a server you are actively working on, your own activity dominates the logs. Our web access log held 10,950 requests, of which the overwhelming majority were our own benchmarks and health checks. Filter ApacheBench, curl and loopback before drawing conclusions, or you will diagnose yourself as the attacker.
The log volume trap
Attack traffic writes to disk, and on a small VPS that is a capacity question rather than a security one. A sustained brute-force run against SSH produced measurable log growth on our machine, and an unrotated log filling the root filesystem takes the database read-only with it.
# What is actually consuming your disk?
df -h /
du -sh /var/log/* 2>/dev/null | sort -h | tail -8
# Is rotation configured and running?
systemctl status logrotate.timer --no-pager | head -4
logrotate -d /etc/logrotate.conf 2>&1 | grep -c "considering log"Ubuntu rotates logs by default, so this is usually already handled. It stops being handled the moment you add an application that writes its own log outside /var/log without a rotation rule, which is the common case with self-hosted Docker stacks and PHP applications writing into their storage directory.
6. Verification: Prove Each Control, Do Not Assume It
Every control above has a way of appearing correct while doing nothing. This section is the audit we ran on our own machine, which is how the two fail2ban failures surfaced.
# --- SSH: what the daemon RESOLVED, not what you wrote ---
sshd -T | grep -E "^(permitrootlogin|passwordauthentication|pubkeyauthentication|maxauthtries)"
# --- Which accounts can actually get in? ---
for h in /root /home/*; do
[ -f "$h/.ssh/authorized_keys" ] && printf "%-16s %s key(s)\n" \
"$(basename $h)" "$(grep -c '^ssh-\|^ecdsa-' $h/.ssh/authorized_keys)"
done
# --- Firewall: enforcing, and nothing unexpected exposed ---
ufw status verbose
ss -tlnp | awk 'NR==1 || $4 !~ /^(127\.|\[::1\])/'
nmap -Pn -p 3306,5432,6379 YOUR_PUBLIC_IP # from another machine
# --- fail2ban: running, correct mode, and has actually banned something ---
systemctl is-active fail2ban
fail2ban-client get sshd mode # expect: aggressive
fail2ban-client status sshd
grep -c " Ban " /var/log/fail2ban.log
# --- Updates: armed, not merely installed ---
systemctl is-active unattended-upgrades
unattended-upgrade --dry-run --debug 2>&1 | tail -5
# --- TLS: renewal proven ---
certbot renew --dry-run
openssl s_client -connect example.com:443 -servername example.com </dev/null 2>/dev/null \
| openssl x509 -noout -dates
# --- What is actually attacking you right now ---
grep -cE "Failed password|Invalid user" /var/log/auth.log
grep -oE "from ([0-9]{1,3}\.){3}[0-9]{1,3}" /var/log/auth.log \
| awk '{print $2}' | sort | uniq -c | sort -rn | head -5That last block is worth running monthly even when nothing is wrong. It took ninety seconds on our server and produced the central finding of this article. A control you have never verified is a belief, not a defence.
The integrity baseline
# Record what the filesystem looks like when you believe it is clean
find /var/www -name '*.php' -type f -exec sha256sum {} + | sort -k2 \
> /root/baseline-$(date +%F).sha256
# Weekly, from cron
find /var/www -name '*.php' -type f -exec sha256sum {} + | sort -k2 \
> /tmp/now.sha256
diff /root/baseline-*.sha256 /tmp/now.sha256 \
|| echo "PHP files changed since baseline" | mail -s "integrity" you@example.comThis is the highest-value item on the page and the one nobody runs. Every other control describes the state of the machine on the day you configured it. The baseline is the only one that answers the question you will genuinely be asked during an incident: what changed, and when. The related protocol for third-party code is in our security audit checklist.
What hardening does not protect you from
Everything above defends the perimeter. None of it helps once an attacker arrives through the front door, and on a web server the front door is your application. Keeping the two categories separate in your head is what stops a hardened server from feeling safer than it is.
- A vulnerable plugin or dependency. Requests to it arrive on port 443, which your firewall is required to allow. SSH keys are irrelevant. The controls that matter here are dependency auditing and a file integrity baseline.
- A stolen credential. An API token in a leaked
.env, or a password reused from another breach, authenticates legitimately. fail2ban sees a successful login. - A supply-chain compromise. A backdoored package installs with your blessing over an encrypted connection. Our audit protocol for third-party code covers the scanning, including the four-line backdoor that evaded every pattern in the standard checklist.
- Your own mistake. The most common cause of data loss on a self-managed server is a command typed by its administrator, not an intrusion.
That last point reframes the whole exercise. Across the servers we operate, the incidents that actually cost time have been a dropped database, an rm in the wrong directory, and a certificate nobody renewed. Not one was an intrusion. Harden the perimeter because it is cheap and it removes the noise, then spend the remaining effort on being able to recover.
Backups are a security control
Ransomware, a compromised application and a mistyped command all resolve to the same question: can you restore. Three properties turn a backup script into a control.
#!/bin/bash
# /usr/local/bin/site-backup
set -euo pipefail
STAMP=$(date +%Y%m%d-%H%M)
DEST=/var/backups/site
BUCKET=s3://your-bucket/site
RETENTION_DAYS=14
install -d -m 700 "$DEST"
# 1. Consistent: a snapshot, not a copy of live files
mysqldump --no-tablespaces --single-transaction --quick \
-u "$DB_USER" -p"$DB_PASS" "$DB_NAME" \
| gzip -9 \
| gpg --batch --yes --symmetric --cipher-algo AES256 \
--passphrase-file /root/.backup-passphrase \
> "$DEST/db-$STAMP.sql.gz.gpg"
# 2. Off the machine, encrypted BEFORE it leaves
aws s3 cp "$DEST/db-$STAMP.sql.gz.gpg" "$BUCKET/" --storage-class STANDARD_IA
find "$DEST" -name '*.gpg' -mtime +$RETENTION_DAYS -delete
echo "$(date -Is) OK $(stat -c%s "$DEST/db-$STAMP.sql.gz.gpg")o" >> "$DEST/journal.log"Encrypted before upload, so the storage provider never holds readable customer data. Off the machine, because a backup on the server it protects is destroyed by the same event. And the third property, which is the one people skip:
# 3. Proven. Restore into a throwaway database and count the rows.
gpg --batch --quiet --passphrase-file /root/.backup-passphrase \
-d /var/backups/site/db-$(date +%Y%m%d)-*.sql.gz.gpg | gunzip \
| mysql restore_test
mysql restore_test -e "SHOW TABLES;" | head
mysql restore_test -e "SELECT COUNT(*) FROM users;"
mysql -e "DROP DATABASE restore_test;"Run that quarterly and treat a failure as a production incident. We have watched a backup job run successfully every night for months while producing dumps that contained no rows, because the credentials it used had lost their SELECT grant. The job's exit code was zero throughout. Only a restore reveals that class of failure, and only if somebody performs one.
7. Frequently Asked Questions
How many attacks does a new VPS actually receive?
Our server logged 1,422 failed SSH attempts from 27 addresses in one week, with a single host responsible for 1,404 of them. Every attempt failed because password authentication was disabled. The volume is unremarkable; what matters is that key-only authentication neutralises all of it with one configuration line.
Why is fail2ban not banning anyone on my server?
Most likely its filter runs in mode = normal, which matches "Failed password" lines your server no longer produces once password authentication is off. Test with fail2ban-regex against a real log line: on ours, normal mode ignored it and aggressive mode matched it. Set mode = aggressive in the sshd jail.
Should I change the SSH port from 22?
It reduces log noise, not risk. Automated scanners find services on any port, and a non-standard port breaks tooling that assumes 22. Spend the effort on key-only authentication instead, which removed 100% of the 1,422 attempts we measured. Move the port only if the log volume is genuinely costing you attention.
Does UFW protect my Docker containers?
No. Docker writes iptables rules in a chain consulted before UFW's, so a container published as -p 5432:5432 is reachable from the internet while UFW reports the port denied. Publish as 127.0.0.1:5432:5432 instead, then verify from another machine with nmap rather than trusting the firewall status.
Is disabling root login worth the risk of a lockout?
Yes, but only after an unprivileged sudo account has a working key. We audited our own server and found three keys on root and none on the sudo account, where setting PermitRootLogin no would have locked us out. Verify with ssh deploy@host 'sudo -n true' from a second terminal before touching the daemon.
8. Final Hardening Checklist
# --- Do first. These carry the weight. ---
[ ] Ed25519 key generated on your workstation, never on the server
[ ] Unprivileged sudo account has a WORKING key (tested from a 2nd terminal)
[ ] PasswordAuthentication no, PermitRootLogin no, verified with: sshd -T
[ ] ufw default deny incoming, only 22/80/443 open
[ ] Databases and caches bound to 127.0.0.1, verified with nmap FROM OUTSIDE
[ ] unattended-upgrades active, proven with: unattended-upgrade --dry-run
# --- Do next ---
[ ] fail2ban running, mode = aggressive, has actually banned something
[ ] Your own office IP in ignoreip before you test from outside
[ ] certbot renew --dry-run passes, and runs monthly from cron
[ ] Security headers present: curl -sI https://site | grep -i strict-transport
[ ] Each application in its own PHP-FPM pool
[ ] Integrity baseline recorded, diffed weekly from cron
# --- Verify monthly, in 90 seconds ---
sshd -T | grep -E "^(permitrootlogin|passwordauthentication)"
fail2ban-client status sshd
systemctl is-active unattended-upgrades fail2ban
certbot renew --dry-run
grep -cE "Failed password|Invalid user" /var/log/auth.log
# --- The test that matters most ---
[ ] Restore a backup into a throwaway database and count the rows.
An untested backup is a hypothesis, and hardening does not
protect you from the mistake you make yourself.If you run the audit in section 6 and find something we did not cover, send us the output with identifying details removed. The two fail2ban failures documented here were found on our own machine while writing this article, and we would rather publish a correction than a clean report.