Fix PostgreSQL "deadlock detected" in Django

Problem statement

Under load, some requests or Celery tasks fail with:

django.db.utils.OperationalError: deadlock detected
DETAIL:  Process 4123 waits for ShareLock on transaction 99812; blocked by process 4188.
Process 4188 waits for ShareLock on transaction 99807; blocked by process 4123.
HINT:  See server log for query details.

Two transactions each hold a lock the other needs. Neither can continue, so PostgreSQL picks one, aborts it, and lets the other finish. The data stays consistent, but the aborted request returns a 500. Deadlocks are a design issue in how transactions take locks, and they are fixable: take locks in the same order everywhere, hold them for less time, and retry the occasional loser.

1. Read the PostgreSQL log entry for the full queries of both processes.
2. Lock rows in a consistent order, for example select_for_update().order_by("pk").
3. Keep transactions short: no network calls or slow work while holding row locks.
4. Avoid updating the same "counter" row from many concurrent transactions; use F() expressions or a separate aggregation.
5. Retry transactions that fail with a deadlock, a small number of times with jitter.

Step-by-step solution

1. Read the deadlock report

The Django exception shows little; the PostgreSQL server log has both queries:

sudo grep -A 12 "deadlock detected" /var/log/postgresql/postgresql-16-main.log | tail -40

ERROR:  deadlock detected
DETAIL:  Process 4123 waits for ShareLock on transaction 99812; blocked by process 4188.
        Process 4188 waits for ShareLock on transaction 99807; blocked by process 4123.
        Process 4123: UPDATE "inventory_stock" SET "quantity" = ... WHERE "id" = 17
        Process 4188: UPDATE "inventory_stock" SET "quantity" = ... WHERE "id" = 9

Find the code that issues these statements. On managed databases, enable log_lock_waits and view the logs in the provider console.

2. Lock in a consistent order

The classic case: an order with several items decrements stock rows in the order the items appear in the cart. Cart A has items 9 then 17, cart B has 17 then 9:

with transaction.atomic():
    for line in order.lines.all():
        stock = Stock.objects.select_for_update().get(pk=line.stock_id)
        stock.quantity -= line.qty
        stock.save()

Lock all needed rows up front, sorted by primary key:

with transaction.atomic():
    ids = sorted(order.lines.values_list("stock_id", flat=True))
    stocks = {s.pk: s for s in Stock.objects.select_for_update().filter(pk__in=ids).order_by("pk")}
    for line in order.lines.all():
        s = stocks[line.stock_id]
        s.quantity -= line.qty
    Stock.objects.bulk_update(stocks.values(), ["quantity"])

When every transaction acquires locks in ascending ID order, a cycle is impossible.

3. Shorten transactions

Locks are held until the transaction ends. Code like this holds row locks while calling a payment API:

with transaction.atomic():
    order = Order.objects.select_for_update().get(pk=pk)
    charge = stripe.PaymentIntent.create(...)    # seconds, with locks held
    order.status = "paid"
    order.save()

Do slow work outside the transaction, then lock briefly to record the result. With ATOMIC_REQUESTS = True, every view is one transaction for its whole duration, which widens the window for deadlocks; consider turning it off and using atomic() around only the writes that need it.

4. Avoid hot rows

Many transactions updating one row, such as a global counter or a parent's updated_at, serialise on that row and deadlock when combined with other locks. Use an atomic update without reading first:

Product.objects.filter(pk=pk).update(view_count=F("view_count") + 1)

or move counters to Redis and write them back in batches. Foreign keys also take locks: inserting child rows takes a KEY SHARE lock on the parent, which conflicts with updates to the parent's key in other transactions.

5. Retry the losers

Even with good ordering, rare deadlocks can occur between unrelated code paths. Retry the whole transaction:

import random, time
from django.db import OperationalError, transaction

def atomic_with_retry(fn, attempts=3):
    for i in range(attempts):
        try:
            with transaction.atomic():
                return fn()
        except OperationalError as e:
            if "deadlock detected" not in str(e) or i == attempts - 1:
                raise
            time.sleep(0.05 * (2 ** i) + random.random() * 0.05)

Only retry code that is safe to repeat: no emails or external calls inside the function.

Verification check: count deadlocks before and after with SELECT deadlocks FROM pg_stat_database WHERE datname = current_database(); sampled daily. Load-test the checkout or import path with concurrent requests using overlapping items; it should complete without deadlock detected.

Explanation

PostgreSQL's lock manager checks for cycles after a transaction has waited deadlock_timeout (1 second by default). If it finds one, it cancels one transaction so the others can progress. This is correct behaviour, not a database bug: the database cannot know which order your application intended. Row locks come from UPDATE, DELETE, SELECT ... FOR UPDATE, and foreign key checks, so ORM code that looks harmless can take locks in data-dependent orders. Imposing a global order on lock acquisition is the standard cure in every database.

Edge cases / notes

• Bulk updates. UPDATE ... WHERE id IN (...) locks rows in physical scan order, not ID order. Lock with an ordered SELECT ... FOR UPDATE first if concurrent bulk updates overlap.
• skip_locked=True is useful for job queues: workers skip rows already locked instead of waiting.
• Migrations. DDL takes heavy table locks; running migrations during traffic can deadlock with application transactions. See the lock-wait guide.
• MySQL reports deadlocks similarly (Deadlock found when trying to get lock); the same fixes apply, and gap locks make them more frequent.

Next steps

• Long transactions also cause Postgres Idle in Transaction problems.
• Bound query time with Set a PostgreSQL statement_timeout for Django.

FAQ

Does a deadlock corrupt data?

No. PostgreSQL rolls back one transaction completely, and the other commits normally.

Can I raise deadlock_timeout to avoid errors?

No. It only delays detection. The cycle still exists and one transaction is still aborted.

Should I use select_for_update everywhere?

Only where you read then write the same rows. Unnecessary locks increase contention.

Why do deadlocks only appear under load?

They need two transactions overlapping in time on the same rows, which is rare at low traffic.