Symptom: You Suspect Your Server Has Been Breached
Something is wrong. Maybe your site is serving pop-up ads you never installed. Maybe outbound traffic spiked at 3 AM and your hosting provider sent an abuse notice. Maybe you try to SSH in and get a “connection refused” because someone changed the SSH port — or the CPU is pegged at 100% from a process called java or httpdz that you definitely did not deploy.
You’re not sure what happened yet, but your gut says something is not right. And in server security, your gut is usually the first detection system you have — there is no “breach alarm” button on most self-managed servers.
This guide walks you through exactly what to do, in order, when you suspect your Linux server has been compromised. It applies to Ubuntu, Debian, Rocky Linux, AlmaLinux, and most other distributions.
Quick Fix: Isolate and Preserve
Before you investigate anything, stop the bleeding. If your server is compromised, every minute it stays online is another minute an attacker can exfiltrate data (or pivot to other machines).
Step 1 — Disconnect the server from the network. If you are in a data centre, disable the network port. If this is a VPS, use your provider’s control panel to stop the instance. On the server itself:
# Emergency containment: prefer provider firewall or console access firstn# If you still have SSH access, capture evidence before changing host firewall policy.n# Then restrict inbound access to your management IP in the cloud/provider firewall.n# Avoid setting OUTPUT DROP over SSH unless you have console access and a rollback plan.
Step 2 — Take a forensic snapshot. Before you touch anything on the server, capture its state. If your hosting provider offers VPS snapshots, create one right now. On a physical or dedicated server, use dd to image the drive if you have a secondary volume:
# Capture running processes and network connections BEFORE reboot
ps aux --sort=-%mem > /tmp/process_snapshot.txt
ss -tunap > /tmp/network_snapshot.txt
lsof -i > /tmp/open_connections.txt
# Copy these off-server if possible (use an isolated management jump box)
Step 3 — Change ALL service account passwords and rotate API keys. The attacker may have collected credentials from memory or config files. Rotate MySQL, PostgreSQL, Redis, and any API keys stored on that server. If you use SSH key authentication, generate new key pairs and update authorized_keys.
Step 4 — Determine the scope. Do not assume the compromise is limited to one application. The attacker may have moved laterally within your infrastructure. Check every server that shared authentication, network access, or credentials with the affected machine.
Your goal in the Quick Fix phase is not to understand the breach. It is to prevent it from getting worse. Investigation comes next, once the blast radius is contained.
Ops Note: Preserve the Timeline
In real support work, the fastest way to lose the root cause is to reboot first and investigate later. Before cleanup, capture process lists, network sockets, recent logins, cron entries, web access logs, and file modification times. If the server is business-critical, clone or snapshot it before remediation so the production recovery and forensic review do not fight each other.
Root Causes: How They Got In
Once the server is isolated and snapshots are saved, you need to identify the entry point. Here are the most common causes of server compromise, ordered by frequency.
1. Weak SSH Credentials — The Most Common Entry
How to confirm: Check your SSH authentication logs for brute-force patterns — hundreds of failed logins followed by a successful one at an unusual hour.
# Count failed SSH attempts
sudo grep "Failed password" /var/log/auth.log | wc -l
# Show all successful SSH logins in the last 7 days
sudo grep "Accepted" /var/log/auth.log | grep -E "sshd" | awk '{print $1, $2, $3, $9, $11, $14}'
If you see a successful login from an IP you don’t recognise, or a username like admin or test that had no business logging in, that is your entry point.
The fix: Disable password authentication for SSH entirely. Only allow key-based login. Remove the compromised user’s authorized_keys entry and change their password.
# In /etc/ssh/sshd_config, set:
PasswordAuthentication no
PermitRootLogin prohibit-password
# Then restart SSH
sudo systemctl reload ssh || sudo systemctl reload sshd
2. Outdated or Vulnerable Web Application
How to confirm: Review your web access logs for suspicious POST requests — especially to wp-admin/admin-ajax.php (WordPress), PHP files you don’t recognise, or file-upload endpoints.
# Check for POST requests to unknown PHP files in the last 24 hours
sudo grep "POST" /var/log/nginx/access.log | grep ".php" | awk '{print $7, $1, $9}' | sort | uniq -c | sort -rn | head -20
# Check for recently modified files in the web root
sudo find /var/www -name "*.php" -mtime -3 -type f
WordPress plugins are a common vector — outdated versions of plugins like Elementor, WooCommerce, or contact form plugins have been exploited in the wild. Check wp-content/plugins/ for anything unfamiliar.
The fix: Update the vulnerable application immediately. Delete unknown files and any backdoor scripts (look for eval(base64_decode(...)), system($_GET['cmd']), or obfuscated PHP). Re-audit user permissions on application directories.
3. Exposed Database Port
How to confirm: Check if MySQL/MariaDB/PostgreSQL is listening on a public IP rather than localhost only.
# Check which ports are listening on which interfaces
sudo ss -tlnp | grep -E "3306|5432|6379"
# Expected output for a secure setup:
# LISTEN 127.0.0.1:3306 (MySQL listening locally only)
# If you see 0.0.0.0:3306 — that means the database is exposed to the internet
If your database was exposed with a weak password (or default credentials), attackers may have dumped the entire database. This is especially common with Redis instances left open on port 6379.
The fix: Bind your database to 127.0.0.1 in the configuration file. For MySQL: bind-address = 127.0.0.1 in /etc/mysql/my.cnf. Change ALL database passwords. Use a firewall to restrict access to only necessary ports.
4. Cryptominer Payload
How to confirm: High CPU usage from an unknown process is the classic sign of a cryptominer. Check for odd process names, high memory usage, or processes running as nobody or www-data.
# Check top CPU consumers
ps aux --sort=-%cpu | head -10
# Check for suspicious cron jobs (miners often install persistence via cron)
for user in root www-data deploy; do
sudo crontab -u "$user" -l 2>/dev/null
done
# Check system-wide cron
sudo ls -la /etc/cron.d/ /etc/cron.hourly/ /etc/cron.daily/
# Look for base64-encoded or wget/curl commands in cron entries
The fix: Kill the mining process, delete the binary (usually in /tmp, /var/tmp, or a hidden directory), remove the cron job, and audit how the miner got installed in the first place — it arrived through one of the other vectors listed here.
5. Default or Weak Administration Panel Credentials
How to confirm: Check your admin panels (phpMyAdmin, Adminer, Cockpit, Webmin, cPanel, Plesk) for default login credentials. Check your application’s admin user passwords — especially if you use the same admin password across multiple services.
# Check for admin panels that might be exposed
sudo find /var/www /usr/share -name "admin*" -o -name "phpmyadmin" -o -name "adminer*" 2>/dev/null
The fix: Change every default password. Remove admin panels that aren’t actively needed. If you must keep phpMyAdmin or similar tools, restrict them to specific IP addresses using a firewall rule or HTTP basic auth.
Diagnostic Flowchart: Identify the Entry Point
Use this decision tree to systematically identify how the attacker got in. Start at the top and work down until you find a match.
| If you see… | Then check… | Likely cause |
|---|---|---|
| Successful SSH login from unknown IP | /var/log/auth.log for the accepted connection |
Weak SSH credentials |
| Unknown PHP files in web root | Web server access logs for POST requests | Vulnerable web app / plugin |
High CPU from xmrig or unknown process |
Cron jobs, /tmp binaries |
Cryptominer (vector = any of the above) |
| Unknown MySQL/PostgreSQL queries in logs | Database was accessible from external IP | Exposed database port |
| Spam emails sent from your server | /var/log/mail.log for unauthorised relaying |
Compromised web app or weak mail credentials |
| New user accounts you didn’t create | /etc/passwd and /etc/shadow for suspicious UIDs |
Weak SSH or exploited web app |
| Files with recent modification dates from unknown users | find / -mtime -2 -type f -not -path '/proc/*' -not -path '/sys/*' |
Attacker gained shell access — method TBD |
Once you identify the vector, follow the specific fix for that root cause above. If you find multiple entry points, fix all of them — experienced attackers often leave multiple backdoors.
Prevention: Locking the Door After the Intrusion
A breach is a forcing function for better security. Here is what to implement once you have recovered:
SSH Hardening
Disable root login, disable password authentication, use Ed25519 keys, and consider moving SSH to a non-standard port (even though security-by-obscurity is weak on its own, it dramatically reduces automated scanning volume).
# Quick SSH hardening checklist
# /etc/ssh/sshd_config:
# Port 2222 (optional — reduces brute-force noise)
# PermitRootLogin no
# PasswordAuthentication no
# PubkeyAuthentication yes
# MaxAuthTries 3
# ClientAliveInterval 300
# ClientAliveCountMax 0
Install a Host-Based Firewall
UFW on Ubuntu, firewalld on Rocky/AlmaLinux, or CSF for cPanel servers. Block everything except the ports you explicitly need:
# Example UFW rules for a web server
sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow 2222/tcp # SSH
sudo ufw allow 80/tcp # HTTP
sudo ufw allow 443/tcp # HTTPS
sudo ufw enable
Implement Intrusion Detection
Consider a lightweight IDS like Wazuh (open-source SIEM) to detect anomalous behaviour automatically, or fail2ban for automated brute-force blocking:
# Install fail2ban for SSH protectionnsudo apt-get updatensudo apt-get install fail2bannsudo mkdir -p /etc/fail2ban/jail.dnprintf "[sshd]nenabled = truenmaxretry = 5nbantime = 1hn" | sudo tee /etc/fail2ban/jail.d/sshd.localnsudo systemctl enable --now fail2bann# Check banned IPsnsudo fail2ban-client status sshd
Keep Everything Updated
Enable automatic security updates and set up a weekly manual update review:
# Ubuntu/Debian automatic security updates
sudo apt-get install unattended-upgrades
sudo dpkg-reconfigure --priority=low unattended-upgrades
# Weekly audit command
sudo apt-get list --upgradable 2>/dev/null | grep -i security
Backup Strategically
Maintain immutable backups — backups that cannot be deleted or modified from the compromised server. CWH CloudSafe Backup provides offsite, versioned backups with retention policies. Test your restore process quarterly — a backup you have never restored is a backup you do not have.
When to Escalate
Some compromises require professional incident response. Escalate in these situations:
- Customer data was involved. If the compromised server held personal information, payment data, or PHI, you have disclosure obligations. Contact CWH Managed Security for a SOC 2-level investigation.
- The attacker had root access. If they escalated to root, assume every binary on the system is compromised. A rebuild from known-good backups (or from scratch) is the only safe recovery path.
- You cannot identify the entry point. If you have followed all the diagnostic steps above and are still unsure how the attacker got in, you have not contained the threat. They may still have access.
- Lateral movement is suspected. If other servers in your infrastructure show similar signs of compromise, the attacker may have moved beyond this one machine.
For businesses without dedicated security staff, CWH’s Managed Support tiers include security monitoring, patch management, and incident response triage. If you prefer to hand off security entirely, Managed Security provides SOC 2-compliant infrastructure with continuous monitoring.
Sources and Command Notes
This incident-response guide was refreshed in May 2026. The SSH, UFW, and Fail2Ban examples were smoke-tested in an Ubuntu 24.04 container where practical. Host firewall isolation is still environment-dependent: if you only have remote SSH access, use provider-level firewall controls or console access before applying rules that could lock you out.
- OpenSSH sshd_config manual
- Ubuntu UFW manual page
- Fail2Ban manual
- Ubuntu automatic security updates
- NGINX access log module documentation
Related Issues and Next Steps
A server compromise is rarely a one-off event. Once you have recovered, invest time in preventative security:
- Password Policies for Hosting Accounts: A Practical Guide — Weak credentials are the most common entry point. Learn how to enforce strong password policies across cPanel, Plesk, WordPress, and server-level accounts.
- WordPress Security Hardening: A Practical Guide — Essential if your compromised server was running WordPress. Covers file permissions, database hardening, and security plugin recommendations.
- WAF vs Firewall vs Managed Security: What Small Teams Need — Understand the difference between network firewalls, web application firewalls, and managed security services, and which combination fits your setup.
- Diagnosing High Server Load — Sometimes high CPU is a cryptominer, sometimes it is a misconfigured application. This guide helps you tell the difference.
- Docker Logs Are Filling Your Server Disk — Log-based indicators of compromise and how to detect unusual activity in containerised environments.
Remember the golden rule of incident response: Contain first, investigate second, recover third. Do not try to understand the breach while the attacker is still inside the server. Isolate. Snapshot. Then diagnose.
Of course, the best incident is the one that never happens. For a complete walkthrough of preventing SSH-based attacks before they reach your server, see our SSH Hardening Guide. Your hosting provider — whether that is CWH managed support or your own team — needs a clean starting point to rebuild securely.
Be First to Comment