Celery Production Checklist for Django

Problem statement

Celery is easy to add to a Django project and easy to run badly in production. Common results: tasks that disappear during deploys, emails sent twice, a queue backlog nobody notices for days, workers holding stale database connections, a scheduler running on every server, and tasks that fail silently because no result backend or error tracking is configured.

Unlike web requests, background work has no user watching it. Failures are invisible unless you build visibility deliberately. This checklist reviews a Celery deployment for Django across configuration, reliability, operations, and security.

Quick answer

1. Broker: durable and monitored; separate from the cache; credentials and network access restricted.
2. Queues: separate queues per workload with dedicated workers; explicit routing.
3. Tasks: idempotent, with bounded retries, time limits, and small serializable arguments.
4. Scheduling: exactly one Beat process; schedules in code or the database, never both for the same task.
5. Workers: run under a process manager, with concurrency and memory limits, connection handling, and graceful shutdown.
6. Visibility: task failures in the error tracker, queue depth and task metrics in monitoring, heartbeats for the scheduler.
7. Deploys: workers restarted with new code, backward-compatible task signatures, and a plan for in-flight tasks.

Checklist

1. Broker configuration

What to check:

python manage.py shell -c "from config.celery import app; print(app.conf.broker_url.split('@')[-1]); print(app.conf.result_backend)"

Guidelines:

• The broker (Redis or RabbitMQ) is reachable only from application and worker hosts, with authentication enabled.
• Broker data is not shared with the cache: a cache instance with an eviction policy can delete queued task messages. Use a separate instance or at least a separate database number with a non-evicting policy.
• Redis persistence is configured appropriately if losing queued tasks is unacceptable; RabbitMQ queues and messages are durable.
• broker_connection_retry_on_startup is set explicitly (Celery 6 changes the default behavior), so workers wait for a broker that is still starting.
• A visibility timeout longer than your longest task when using Redis:

CELERY_BROKER_TRANSPORT_OPTIONS = {"visibility_timeout": 60 * 60 * 2}

2. Result backend

Guidelines:

• A result backend exists only if results or states are actually used; otherwise set none and avoid storing data nobody reads.
• With Redis results, result_expires is set so results do not accumulate.
• Long-lived status (job progress shown to users) lives in your own models, not in Celery results.

3. Queues and routing

Guidelines:

• Separate queues by workload characteristics: fast and user-facing (emails, notifications) versus slow and heavy (reports, imports).
• task_routes maps tasks to queues explicitly.
• Every queue has at least one worker consuming it (celery -A config inspect active_queues).
• Worker settings differ per queue: concurrency, prefetch, and time limits.

4. Task design

For each task, confirm:

• Idempotent: running it twice is harmless, because redelivery and retries happen. Use unique constraints, status checks, or locks.
• Small arguments: pass primary keys and simple values, not model instances or large payloads. Objects are serialized into the broker, and a stale serialized object may no longer match the database.
• Reads fresh data: fetch the object inside the task by ID.
• Queued after commit: use transaction.on_commit(lambda: task.delay(obj.pk)) (or delay_on_commit) so workers never see uncommitted data.
• Bounded retries with backoff and jitter:

@shared_task(
    bind=True,
    autoretry_for=(RequestException,),
    retry_backoff=True,
    retry_backoff_max=600,
    retry_jitter=True,
    max_retries=5,
)
def sync_to_crm(self, customer_id):
    ...

• Time limits so a hung call cannot hold a worker forever, plus timeouts on the network calls themselves.
• Serializer: JSON (Celery's default) rather than pickle, which executes arbitrary code on deserialization.

CELERY_TASK_SERIALIZER = "json"
CELERY_RESULT_SERIALIZER = "json"
CELERY_ACCEPT_CONTENT = ["json"]

5. Scheduling

Guidelines:

• Exactly one Beat process across all hosts and containers; never -B on workers.
• Schedules defined in one place: CELERY_BEAT_SCHEDULE in settings, or django-celery-beat in the database, not duplicated.
• CELERY_TIMEZONE matches Django's TIME_ZONE, ideally UTC.
• Periodic tasks set expires so a backlog does not run hours late.
• Beat's schedule file (file scheduler) is on persistent, writable storage.

6. Worker processes

Guidelines:

• Workers run under systemd, Docker, or an orchestrator with restart policies, not in a terminal session.
• Each worker has a unique node name (-n worker1@%h) when several run on one host.
• Concurrency matches the workload type and available memory.
• worker_max_tasks_per_child or worker_max_memory_per_child recycles processes to limit memory growth.
• Database connections are closed around tasks (close_old_connections on task signals) so long-lived workers do not hold broken connections.
• Graceful shutdown: the stop timeout in systemd or the orchestrator is longer than typical task duration, so SIGTERM lets tasks finish.

[Service]
KillSignal=SIGTERM
TimeoutStopSec=300

7. Visibility

Guidelines:

• Task failures are reported to the error tracker. With Sentry, the Celery integration captures task exceptions; otherwise log exceptions explicitly in a task_failure signal handler.
• Worker logs go to the same place as application logs and are retained.
• Metrics collected: queue depth per queue, oldest queued task age, task success and failure counts, task duration.
• Alerts: queue backlog beyond a threshold, failure rate spikes, scheduler heartbeat missing, worker count below expected.
• A dashboard shows queues and task outcomes next to web metrics, so backlogs are visible during incidents.

8. Deploys

Guidelines:

• Workers are restarted on every deploy that changes task code; otherwise they run old code against new messages.
• Task names are stable; renaming a task requires a transition period where both names exist, or messages for the old name are drained first.
• Task signatures stay backward compatible for at least one release (add optional arguments rather than changing positions).
• Restart order documented: for new tasks, deploy and restart workers before the code that enqueues them.
• In-flight tasks: with acks_late, a worker killed mid-task causes redelivery (safe only for idempotent tasks); without it, the task is lost.

9. Security

Guidelines:

• Broker not exposed publicly; credentials in the secret store, not in code.
• JSON serializer only (no pickle).
• Task arguments contain no secrets; they are stored in the broker and may appear in logs and monitoring tools.
• Flower or other dashboards are authenticated and not publicly reachable.
• Workers run as an unprivileged user with the same sandboxing as the web service.

10. Capacity

Guidelines:

• Peak task volume and processing time estimated; worker capacity exceeds it with headroom.
• Database connection usage from workers included in the database connection budget.
• Rate limits for tasks that call external APIs, aware that Celery's rate_limit is per worker instance.
• Backlog behavior understood: what happens during a traffic spike, and whether users see delays (for example password reset emails arriving late).

Verify

celery -A config inspect ping
celery -A config inspect active_queues
celery -A config inspect registered | head -20
celery -A config inspect stats | grep -E '"pool"|"max-concurrency"'
pgrep -af "celery.*beat" | wc -l    # expect exactly 1 across the deployment
redis-cli -n 0 LLEN celery          # queue depth for the default queue

Verification check:

• all expected workers reply to inspect ping
• every routed queue has a consumer
• exactly one Beat process exists
• a deliberately failing task appears in the error tracker
• queue depth returns to zero after a burst of tasks
• restarting workers mid-task does not lose or duplicate critical work

Explanation

Celery introduces a second execution environment with its own failure modes: messages can be delayed, redelivered, or lost; workers can run stale code; schedulers can duplicate; and nothing is visible unless instrumented. A production-ready setup makes each of those explicit: durable brokers and sensible acknowledgement settings for delivery, idempotent tasks for redelivery, one scheduler for duplication, restarts for code freshness, and metrics plus error tracking for visibility.

Edge cases / notes

Alternatives for simple needs

If you only need a handful of scheduled jobs and no queueing, systemd timers running management commands avoid the broker and worker fleet entirely. Django 6.0 also introduces a built-in background tasks framework, which may cover simple cases without Celery.

Priority work

Broker priority support is limited and behaves differently per broker. Separate queues with dedicated workers give more predictable results.

Long-running tasks

Tasks measured in hours complicate deploys and redelivery. Split them into resumable chunks that record progress.

Common mistakes

• Sharing one Redis instance and database between cache and broker with an eviction policy.
• Running Beat on every application server.
• Passing model instances as task arguments.
• Deploying web code without restarting workers.

Next steps

• Set up the broker and workers with Configure Redis and Celery for Django Production.
• Manage worker services with Run Celery Workers with systemd for Django.
• Tune throughput with Tune Celery Worker Concurrency and Prefetch for Django.
• Fix duplicate scheduling with Fix Celery Beat Running Tasks Twice.

FAQ

Can Celery share Redis with the Django cache?

It can, but the eviction policy applies to the whole instance, so cache eviction can delete queued tasks. Use separate instances, or a non-evicting policy with timeouts on all cache keys.

Do Celery workers need to be restarted on every deploy?

Whenever task code or shared application code changes, yes. Workers keep the code they imported at startup, so old code would otherwise process new messages.

Should Celery tasks use acks_late?

For idempotent tasks where losing work on a crash is worse than running twice, yes, combined with prefetch_multiplier = 1. For non-idempotent tasks, keep the default early acknowledgement.

How do I monitor Celery queues in production?

Track queue depth and the age of the oldest queued task per queue, task failure rates, and worker liveness, and alert when backlogs grow or the Beat heartbeat stops.