Fix Nginx "upstream prematurely closed connection" for Django

Problem statement

Some requests return 502 Bad Gateway and the Nginx error log shows:

upstream prematurely closed connection while reading response header from upstream,
client: 203.0.113.7, server: example.com, request: "POST /reports/export/ HTTP/1.1",
upstream: "http://unix:/run/gunicorn.sock:/reports/export/"

Nginx connected to Gunicorn and sent the request, then the connection closed before any response header arrived. Something on the Gunicorn side ended the request without answering: the worker was killed for timing out, killed by the OOM killer, crashed in native code, or restarted mid-request. Nginx is reporting the symptom; the cause is in the application server's logs.

1. Match the Nginx timestamp with the Gunicorn log: look for WORKER TIMEOUT, Worker was sent SIGKILL, or Booting worker.
2. Check the kernel log for OOM kills: journalctl -k | grep -i "killed process".
3. If it is a timeout, make the request faster or move it to a background task; only then align Gunicorn --timeout with Nginx proxy_read_timeout.
4. If it is memory, reduce per-request memory or worker count.
5. If it happens during deploys, use graceful reloads instead of restarts.

Step-by-step solution

1. Correlate the logs

sudo tail -n 50 /var/log/nginx/error.log
sudo journalctl -u gunicorn --since "-15min" | grep -E "TIMEOUT|SIGKILL|Booting|Exception|Error"
sudo journalctl -k --since "-15min" | grep -iE "out of memory|killed process"

The matching line at the same second tells you which of the causes below applies.

2. Worker timeout

[CRITICAL] WORKER TIMEOUT (pid:4123)
[ERROR] Worker (pid:4123) was sent SIGKILL! Perhaps out of memory?

Despite the wording, a WORKER TIMEOUT just before the SIGKILL means the request ran longer than Gunicorn's --timeout (30 seconds by default) and the arbiter killed it. Find which view is slow with request logs or an APM, then:

• Optimise the query or code; see Fix Gunicorn Worker Timeout.
• Move exports, imports, and report generation to Celery and return immediately.
• If a long request is genuinely expected, raise --timeout for that service and make Nginx wait as long.

3. Out-of-memory kill

If the kernel log shows Killed process 4123 (gunicorn), the worker exceeded available memory. Typical culprits are loading an entire table, building a large file in memory, or image processing. Stream large responses, use .iterator(), and see Fix Gunicorn Workers Killed by the OOM Killer.

4. Crashes and recycling

A worker that disappears with no Python traceback may have segfaulted in a C extension (image libraries, XML parsers, database drivers built against the wrong library). Check dmesg for segfault lines. Separately, --max-requests recycles workers after N requests; Gunicorn waits for the current request to finish, so recycling alone should not cause this, but combined with a short --graceful-timeout it can.

5. Deploy restarts

If the errors cluster around deploys, the service is being restarted rather than reloaded. systemctl restart gunicorn kills workers mid-request. Use:

sudo systemctl reload gunicorn    # sends HUP: new workers start, old ones finish

with ExecReload=/bin/kill -s HUP $MAINPID in the unit, and set --graceful-timeout long enough for in-flight requests.

6. Align timeouts across layers

When a slow request is expected, every layer must allow it, and the outer layers should wait slightly longer than the inner ones so the app server, which can log the cause, gives up first:

location / {
    proxy_pass http://unix:/run/gunicorn.sock;
    proxy_read_timeout 65s;
}

gunicorn config.wsgi --timeout 60 --graceful-timeout 30

Verification check: reproduce the slow request with curl -w "%{http_code} %{time_total}\n" and confirm it either succeeds or fails with a clear application error, and that the Nginx error log no longer shows the message during normal traffic or deploys.

Explanation

Nginx reads response headers from the upstream socket. If the socket reaches end-of-file before a complete header block, Nginx cannot build a response and returns 502 with this message. Unlike "connection refused" (nothing listening) or "upstream timed out" (Nginx gave up waiting), "prematurely closed" means the upstream actively closed the connection. With Gunicorn that nearly always means the worker process holding it died or was killed.

Edge cases / notes

• "while reading upstream" instead of "response header" means the response started but was cut off, typically a streaming response killed mid-way.
• uWSGI produces the same Nginx message when its harakiri timeout kills a worker.
• Keepalive to upstream. With keepalive in an upstream block, Gunicorn closing an idle keepalive connection can occasionally race with a new request. Gunicorn's sync workers do not support keepalive; do not enable upstream keepalive to sync workers.
• Containers. In Kubernetes, a container restart from a failed liveness probe produces the same pattern.

Next steps

• Work through every 502 cause with Debug Django 502 Bad Gateway.
• Review proxy timeouts in Configure Nginx for Django in Production.

FAQ

Is this an Nginx problem?

Rarely. Nginx reports that the upstream closed the connection; the reason is in Gunicorn or kernel logs.

Should I raise proxy_read_timeout?

Only if Gunicorn's timeout is also raised and the slow request is expected. Otherwise the worker is still killed first.

Why does it say SIGKILL, perhaps out of memory?

Gunicorn prints that hint for any SIGKILL. If a WORKER TIMEOUT line comes just before it, the cause was the timeout.

Why only on some requests?

Usually only certain views are slow or memory-heavy. Group the Nginx errors by request path.