Fix Cloudflare 524 Timeouts on Slow Django Requests

Problem statement

A report export, CSV import, or admin bulk action shows Cloudflare's page:

Error 524: A timeout occurred
The origin web server timed out responding to this request.

Your Gunicorn logs show the request finishing successfully a minute later, or being killed by its own timeout. Cloudflare connected to your origin and sent the request, but did not receive a response within its proxy read timeout, 100 seconds on Free, Pro, and Business plans. The origin may still be working; the visitor has already been given an error.

1. Find the slow endpoint in Gunicorn or Nginx logs (request times near or over 100 seconds).
2. Make it faster if you can: indexes, fewer queries, pagination.
3. Move long work to Celery: return immediately with a job ID, let the browser poll or get notified, then download the result.
4. For generated files, stream the response so bytes start flowing within the limit.
5. For rare admin-only tasks, use a DNS-only (grey cloud) hostname, or a management command.

Step-by-step solution

1. Identify the slow requests

Add request time to the Nginx log format and look for the long ones:

log_format timed '$remote_addr "$request" $status $request_time';
access_log /var/log/nginx/access.log timed;

awk '$NF > 60 {print $NF, $2, $3}' /var/log/nginx/access.log | sort -rn | head

Gunicorn's access log with %(L)s (request time in seconds) shows the same from the app side.

2. Make it faster first

Many long requests are accidental: an N+1 query in an export loop, a missing index on a filter, or loading an entire table into memory. Profile the view (see Profile Slow Django Endpoints in Production) before re-architecting. A 3-minute export that becomes a 20-second export no longer needs special handling.

3. Move long work to a background job

The robust pattern for exports and imports:

# views.py
def start_export(request):
    job = ExportJob.objects.create(user=request.user, params=request.GET.dict())
    transaction.on_commit(lambda: build_export.delay(job.pk))
    return JsonResponse({"job": job.pk, "status_url": reverse("export-status", args=[job.pk])}, status=202)

def export_status(request, pk):
    job = get_object_or_404(ExportJob, pk=pk, user=request.user)
    return JsonResponse({"status": job.status, "download": job.file.url if job.file else None})

# tasks.py
@shared_task
def build_export(job_id):
    job = ExportJob.objects.get(pk=job_id)
    job.file.save(f"export-{job.pk}.csv", generate_csv(job.params))
    job.status = "done"
    job.save(update_fields=["status", "file"])

The browser polls status_url every few seconds and offers the download when ready, or you email a link. Each HTTP request is short, so no proxy timeout applies.

4. Stream when the work is incremental

If the output can be produced row by row, streaming sends the first bytes quickly, and Cloudflare's timer only waits for the start of the response:

import csv
from django.http import StreamingHttpResponse

class Echo:
    def write(self, value):
        return value

def export_csv(request):
    writer = csv.writer(Echo())
    rows = Order.objects.values_list("id", "email", "total").iterator(chunk_size=2000)
    resp = StreamingHttpResponse((writer.writerow(r) for r in rows), content_type="text/csv")
    resp["Content-Disposition"] = 'attachment; filename="orders.csv"'
    resp["X-Accel-Buffering"] = "no"
    return resp

Disable Nginx buffering for the response so bytes reach Cloudflare as they are produced, and make sure Gunicorn's --timeout covers the total duration for sync workers.

5. Bypass the proxy for rare long tasks

For an occasional admin-only operation, a separate hostname with the Cloudflare proxy turned off (DNS-only) avoids the limit. Restrict it by IP or VPN, since it exposes the origin directly. Better still, run such tasks as management commands over SSH. Enterprise plans can raise the proxy timeout, but designing around it is cheaper.

Verification check: trigger the export through the public hostname and confirm the initial request returns within a second, status polling works, and the download completes. Cloudflare Analytics should show no 524s for that path.

Explanation

Cloudflare's edge holds a connection to your origin while waiting for the response headers. To protect its infrastructure and visitors from hanging connections, it closes that wait after about 100 seconds on standard plans and returns a 524. Raising Nginx or Gunicorn timeouts cannot change a limit enforced at Cloudflare. The only durable fixes are making responses start sooner (faster code or streaming) or not doing long work inside a request at all.

Edge cases / notes

• WebSockets are not subject to the 100-second response timeout once upgraded, but idle connections are closed after about 100 seconds without traffic.
• Large uploads are limited by plan size (100 MB on Free and Pro) and produce 413s, not 524s.
• Origin timeouts. Keep Gunicorn and Nginx timeouts sensible anyway; the work continuing after a 524 wastes resources and can run twice if the user retries.
• Idempotency. Users who see a 524 often click again; make export and import endpoints safe to repeat.

Next steps

• If Cloudflare cannot reach the origin at all, see Fix Cloudflare 521 and 522 Errors.
• Without Cloudflare, the same problem appears as Fix Django 504 Gateway Timeout.

FAQ

What is Cloudflare's timeout?

About 100 seconds for the origin to start responding on Free, Pro, and Business plans. Enterprise can raise it.

Will increasing Gunicorn's timeout fix a 524?

No. The limit is at Cloudflare's edge, not on your server.

Does the request keep running after a 524?

Usually yes, until it finishes or hits your own timeout. The visitor just never sees the result.

Can I stream to avoid the timeout?

Yes, if the response starts within the limit and keeps sending data. Disable proxy buffering for that response.