Your server’s disk is filling up and you can’t figure out why. You’ve checked your files, cleaned up old kernels, and pruned Docker images — but the space keeps disappearing. Sound familiar?

If you run Docker containers, the culprit is almost certainly Docker container log files. Docker’s default logging driver (json-file) stores every line of stdout and stderr as a JSON file on your host’s disk — and by default, no rotation is performed. A single chatty container can silently consume gigabytes per day.

Here is how to diagnose the issue, recover space carefully, and set log rotation so it does not happen again.

The Symptom: Warnings That Point to Docker Logs

Here’s what you might be seeing:

  • No space left on device errors in your terminal or application logs
  • docker logs <container> returns nothing or truncates output
  • New containers fail to start with write /var/lib/docker/containers/...: no space left on device
  • df -h shows /var or /var/lib at 90%+ usage, but du -sh /* doesn’t add up
  • docker system df shows modest image/container/volume sizes, but the disk is still nearly full

That last one is the biggest clue. When docker system df reports healthy numbers but your disk is full, Docker’s JSON log files (which live outside the Docker storage system) are the missing piece.

The log files live at /var/lib/docker/containers/<container-id>/<container-id>-json.log. Each container gets its own file, and without rotation, they grow without bound.

Quick Fix: Reclaim Your Disk Right Now

If your disk is critically full and you need space immediately:

Step 1: Find the biggest log files

sudo du -sh /var/lib/docker/containers/*/*-json.log 2>/dev/null | sort -rh | head -10

Expected output:

2.3G    /var/lib/docker/containers/abc123/abc123-json.log
1.1G    /var/lib/docker/containers/def456/def456-json.log
856M    /var/lib/docker/containers/ghi789/ghi789-json.log

Step 2: Truncate the largest files

# Truncate ALL container log files (safe — Docker holds the file handle)
sudo truncate -s 0 /var/lib/docker/containers/*/*-json.log

Important: truncate -s 0 is safe. Docker keeps a file descriptor open, so it continues writing to the (now-empty) file. The container doesn’t restart, and you don’t lose current log data — you just free the disk space the old data was taking.

Step 3: Set up log rotation so this doesn’t happen again

# Edit or create /etc/docker/daemon.json
sudo tee /etc/docker/daemon.json <<'EOF'
{
  "log-driver": "local",
  "log-opts": {
    "max-size": "10m",
    "max-file": "3"
  }
}
EOF

# Restart Docker to apply
sudo systemctl restart docker

The local driver is Docker’s recommended alternative to json-file. It stores logs in an internal binary format with built-in rotation — and it still supports docker logs. The max-size: 10m + max-file: 3 settings cap each container’s log footprint at 30 MB.

Caveat: This only applies to NEW containers. Existing containers keep their old log files and old settings. See the root causes below for how to fix existing containers.

Root Causes: What’s Eating Your Disk

1. Default json-file Driver Has No Rotation

Docker’s default logging driver writes each container’s stdout and stderr to a single JSON file on disk. As the Docker documentation warns: “Tip: Use the local logging driver to prevent disk-exhaustion. By default, no log-rotation is performed.” This is not a bug — it’s a design choice that assumes you’ll configure rotation. Many administrators don’t realize this until the disk fills up.

How to confirm:

# Check a running container's logging config
docker inspect <container-name> | python3 -c "
import json, sys
d = json.load(sys.stdin)
lc = d[0]['HostConfig']['LogConfig']
print(f'Driver: {lc["Type"]}')
print(f'Config: {json.dumps(lc["Config"], indent=2)}')"

If the output shows "Type": "json-file" and the Config section is empty (no max-size or max-file), you’ve confirmed the root cause. Every byte of stdout and stderr the container has ever produced is sitting on your disk.

Fix — per-container rotation in docker-compose.yml:

services:
  web:
    image: nginx:latest
    logging:
      driver: json-file
      options:
        max-size: "5m"
        max-file: "5"

  db:
    image: postgres:16
    logging:
      driver: json-file
      options:
        max-size: "10m"
        max-file: "3"

After updating your compose files, recreate the containers: docker compose up -d. The old oversized log files remain on disk (truncate them manually), but new rotation limits will apply going forward.

Alternative — switch to the local driver system-wide: Edit /etc/docker/daemon.json (see Quick Fix Step 3) and restart Docker. All new containers will use the local driver.

2. Application Flood-Logging (Error Loop or Debug Mode)

Sometimes the problem isn’t the logging driver — it’s the application writing far too much data. Common scenarios:

  • A container in debug mode that logs every HTTP request body
  • An application in an error loop, printing the same stack trace 50 times per second
  • A database running with log_statement = all in production
  • A web server in full verbose mode logging every header and cookie

How to confirm:

# Check log file size and growth
sudo du -sh /var/lib/docker/containers/*/*-json.log | sort -rh | head -5

# Check recent log volume
docker logs --tail 50 <container-name>

# If you see the same error repeating every second, you have an error loop.
# A single container writing 50 lines/second at ~200 bytes/line consumes
# about 864 MB per day.

Fix — application level:

  1. Diagnose the root cause of the error loop (stuck database connection, missing config file, dependency timeout)
  2. Set log level flags: --log-level error, --quiet, or environment variables like LOG_LEVEL=warn
  3. Use app-specific logging config to reduce verbosity (e.g., Nginx access log off, MySQL general log off)

Fix — immediate relief while debugging:

# Truncate just the offending container's log
sudo truncate -s 0 /var/lib/docker/containers/$(docker ps -q --filter name=offending-container)/*-json.log

This buys you time to fix the application without filling the disk again.

3. Stopped Containers Still Keep Their Log Files

When you docker stop a container, the JSON log file stays on disk. Stopped containers with months of accumulated logs can quietly consume hundreds of megabytes — or gigabytes — that you’ve already forgotten about.

How to confirm:

# List all containers, including stopped ones
docker container ls -a | head -20

# Check log file sizes for stopped containers
for cid in $(docker ps -aq --filter status=exited); do
  logfile="/var/lib/docker/containers/$cid/${cid}-json.log"
  if [ -f "$logfile" ]; then
    echo "$(docker inspect $cid --format='{{.Name}}' | sed 's////') $(du -sh $logfile | cut -f1)"
  fi
done

Fix — remove stopped containers:

# Remove a single stopped container (this deletes its log file too)
docker rm <container-id>

# Remove ALL stopped containers
docker container prune

# Confirm the log file is deleted
ls -la /var/lib/docker/containers/<container-id>/*-json.log
# Expected: ls: cannot access ...: No such file or directory

4. No Log Rotation in docker-compose.yml

Even if you set global defaults in daemon.json, individual docker-compose.yml service definitions can override them — often unintentionally. Many community-maintained compose files don’t include logging configuration, so they inherit the Docker daemon’s defaults (no rotation).

How to confirm:

# Check a running container's log config
docker inspect <container> | python3 -c "
import json, sys
d = json.load(sys.stdin)
lc = d[0]['HostConfig']['LogConfig']
print(json.dumps(lc, indent=2))"

# Missing "max-size" and "max-file" in the Config means no rotation
# Even if daemon.json has them, the compose file's logging: block
# (or lack thereof) determines the actual config.

Fix — add logging to each compose service:

services:
  app:
    image: my-app:latest
    logging:
      driver: json-file
      options:
        max-size: "10m"
        max-file: "3"

After updating, recreate the container: docker compose up -d.

Rule of thumb: Every service in every docker-compose.yml should have an explicit logging: block. Do not rely on daemon defaults — they change between Docker versions and across different servers.

5. Build Cache and Dangling Resources (Related but Different)

Log files aren’t the only Docker disk hog. Build cache from docker compose build or docker buildx can accumulate hundreds of megabytes per build. Dangling images (untagged layers) and unused volumes also take space.

How to confirm:

docker system df

Expected output shows all categories:

TYPE            TOTAL     ACTIVE    SIZE      RECLAIMABLE
Images          5         3         2.1GB     500MB (23%)
Containers      8         5         50MB      30MB (60%)
Local Volumes   4         3         1.2GB     800MB (66%)
Build Cache     12                 900MB     900MB (100%)

If the Reclaimable column shows significant space, you can clean up:

# Full cleanup (images, containers, volumes, build cache)
docker system prune -a --volumes

# Or surgical — just build cache
docker builder prune

# Only dangling images
docker image prune

Note: docker system prune -a --volumes is aggressive. It removes all unused images (not just dangling), all stopped containers, all unused networks, all build cache, and all volumes not attached to at least one container. Recommended as a monthly maintenance task, not a daily habit.

Diagnostic Flowchart: Where to Start

Symptom Check This First Likely Cause
df -h shows disk full, docker system df looks fine du -sh /var/lib/docker/containers/*/*-json.log json-file logs without rotation
One container’s log is 10x larger than others docker logs --tail 50 <container> Error loop or flood-logging
Space keeps filling up after truncation Check daemon.json and compose logging config No rotation configured on new containers
Disk used by /var/lib/docker but docker system df says otherwise du -sh /var/lib/docker/* | sort -rh Stopped containers, old overlay2 data, or build cache
Large docker system df Build Cache Run docker builder prune CI/CD builds or frequent compose builds

Ops Note: Logs Are a Symptom, Not Always the Root Cause

Truncating a Docker log buys time; it does not fix the application that is writing thousands of lines per minute. After you recover disk space, check the repeating error, restart loop, debug logging level, or failed dependency that caused the growth. Otherwise the same server will be full again tomorrow.

Prevention: Keep Docker Logs Under Control

Set Global Defaults (Do This First)

Configure the Docker daemon to apply log rotation to every new container by default:

sudo tee /etc/docker/daemon.json <<'EOF'
{
  "log-driver": "local",
  "log-opts": {
    "max-size": "10m",
    "max-file": "3"
  }
}
EOF

sudo systemctl restart docker

The local driver stores logs in a compact binary format with built-in rotation. It’s Docker’s own recommendation for preventing disk exhaustion, and it supports docker logs just like json-file does.

If you need json-file (for compatibility with log shippers like Filebeat or Fluentd):

{
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "10m",
    "max-file": "3"
  }
}

Monitor Log File Growth

Set up a cron job to alert you when Docker log files grow unexpectedly:

# Add to /etc/cron.daily/docker-log-watch
#!/bin/bash
THRESHOLD_MB=500
for logfile in /var/lib/docker/containers/*/*-json.log; do
  size_mb=$(du -m "$logfile" | cut -f1)
  if [ "$size_mb" -gt "$THRESHOLD_MB" ]; then
    echo "WARNING: $logfile is ${size_mb}MB" | 
      mail -s "Docker log size alert" admin@example.com
  fi
done

For production servers, consider using CWH Monitoring which provides infrastructure-level disk usage alerts with your choice of notification channels (email, SMS, Slack, PagerDuty).

Ship Logs Externally (Recommended for Production)

The best way to prevent local disk exhaustion from logs is to not store them locally. Ship logs to a centralized system and keep local rotation tight:

  • CWH Graylog — use Docker’s gelf driver: 
    docker run --log-driver gelf --log-opt gelf-address=udp://graylog.example.com:12201 ...
  • Grafana Loki — install the Docker plugin: 
    docker plugin install grafana/loki-docker-driver
  • Syslog — use the syslog driver: 
    docker run --log-driver syslog --log-opt syslog-address=tcp://syslog.example.com:514 ...

With external shipping, you can set local logs to max-size: 5m and max-file: 2 while keeping full history in the central store.

When to Escalate

Most Docker log disk issues are solvable with the steps above. Here’s when you might need help:

  • Disk is 100% full and Docker won’t start: Use sudo truncate -s 0 directly from the host shell on the largest -json.log files. If Docker itself won’t start, you may need to manually delete log files (not truncate) to free inodes.
  • Legitimate high-volume logs (e.g., a busy web server or API gateway): This is a scaling sign — consider CWH Monitoring for log aggregation or upgrading to Dedicated Servers for higher I/O and larger storage.
  • Multiple VPSes with the same problem: Set up centralized logging with CWH Cloud VPS running Graylog or Loki, so you can search logs across all servers from one place.
  • You don’t want to manage this yourself: CWH Managed Support can configure Docker logging, set up log shipping, monitor disk usage, and respond to alerts on your behalf. Our Tier 1 plan covers setup and configuration; Tier 2 adds proactive monitoring and incident response.

Sources and Command Notes

This post was refreshed in May 2026 against Docker documentation. The Docker logging configuration example and inspect command were smoke-tested locally with Docker 29.2.0. Restarting Docker affects running containers, so plan a maintenance window on production servers.

Related Issues

Disk space is a finite resource, but with the right Docker logging configuration, it doesn’t have to be a constant worry. Configure rotation once, set up monitoring, and you’ll only think about container logs again when you’re shipping them somewhere useful — not scrambling to free up space at 2 AM.