Automatic ORM Query Caching in Django With Cachalot or Cacheops

Problem statement

Your Django app runs the same read queries over and over: lookup tables, settings rows, category trees, and product lists that change a few times a day. Adding cache.get_or_set around each query is tedious and easy to get wrong when data changes. ORM caching libraries hook into the query layer and cache results automatically, invalidating them when the underlying tables or objects are written. They can remove a large share of database load, or cause subtle stale-data bugs if their rules do not match how your app writes data.

1. django-cachalot caches every query and invalidates per table on any write; simple and safe, best when writes are rare.
2. django-cacheops caches selected models or querysets with finer-grained invalidation based on conditions; more control, more configuration.
3. Use Redis as the cache backend, shared by all processes.
4. Exclude write-heavy tables (sessions, logs, queues, counters) from caching.
5. Measure: compare database query counts and cache hit rates before and after, on production-like traffic.

Step-by-step solution

1. Decide if your workload fits

ORM caching helps when the same queries repeat and tables change rarely relative to reads. It helps little or hurts when:

• Tables receive constant writes (every write invalidates every cached query on that table with cachalot).
• Most queries are unique (filters by user ID, timestamps, search terms).
• You already cache rendered fragments or pages higher up.

Check with pg_stat_statements: a few queries with very high calls and low variation are good candidates.

2. Try django-cachalot

INSTALLED_APPS += ["cachalot"]
CACHES = {"default": {"BACKEND": "django_redis.cache.RedisCache", "LOCATION": os.environ["REDIS_URL"]}}
CACHALOT_ENABLED = True
CACHALOT_TIMEOUT = 60 * 60
CACHALOT_UNCACHABLE_TABLES = frozenset((
    "django_migrations", "django_session", "django_celery_results_taskresult",
    "audit_log_entry", "analytics_event",
))

Cachalot caches query results keyed by the SQL and invalidates all cached queries touching a table whenever any row in that table is inserted, updated, or deleted through Django. It works with transactions, raw SQL through connection.cursor() (detected by parsing), and QuerySet.update().

3. Or use django-cacheops for finer control

INSTALLED_APPS += ["cacheops"]
CACHEOPS_REDIS = os.environ["REDIS_URL"]
CACHEOPS_DEFAULTS = {"timeout": 60 * 15}
CACHEOPS = {
    "catalog.category": {"ops": "all"},
    "catalog.product": {"ops": ("get", "fetch"), "timeout": 60 * 5},
    "auth.user": {"ops": "get", "timeout": 60 * 15},
    "*.*": {},          # everything else uncached
}

Cacheops tracks the conditions of each cached query (for example category_id = 7) and invalidates only queries whose conditions match a changed object. That keeps more cached results valid when writes are frequent but localized.

4. Watch for invisible writes

Both libraries rely on Django to tell them about writes. Data changed outside Django, such as another service, a psql session, a database trigger, or a bulk COPY import, does not invalidate the cache. Either route those writes through Django, invalidate manually afterwards (cachalot.api.invalidate("catalog_product"), cacheops.invalidate_model(Product)), or exclude those tables.

5. Measure before trusting it

Enable it in staging with production-like traffic. Compare:

• Database queries per request (django-debug-toolbar, APM, or pg_stat_statements deltas).
• Redis memory and operations per second.
• p95 latency of the affected views.

If invalidation churn is high, the cache adds Redis round-trips without saving database work.

Explanation

Both libraries patch Django's query compiler. Before executing a read, they compute a cache key from the SQL and parameters and check Redis; on a miss they run the query and store the result along with metadata about which tables (cachalot) or which conditions (cacheops) it depends on. Write paths in the ORM trigger invalidation of dependent keys. The approach is powerful because it needs no changes to views, and risky because correctness depends on every write going through the ORM.

Edge cases / notes

• Transactions. Invalidation must respect transaction boundaries; both libraries handle common cases, but test long transactions that read and write the same tables.
• Version compatibility. Check each library's supported Django versions before upgrading Django.
• Memory. Large querysets cached in Redis consume memory; set timeouts and Redis maxmemory with an eviction policy.

Next steps

• Choose layers deliberately with Django Caching Strategy for Production.
• Remove wasted queries first with Fix Django N+1 Queries.
• Protect hot keys with Prevent Django Cache Stampedes.

FAQ

Is automatic ORM caching safe?

It is safe when all writes go through Django and you exclude tables changed elsewhere. Otherwise cached results can be stale.

Cachalot or cacheops?

Cachalot for simplicity on read-mostly apps; cacheops when you need per-model rules and finer invalidation.

Do I need Redis?

A shared cache is required with multiple processes. Redis is the usual choice.

Should I fix N+1 queries first?

Yes. Caching hides inefficient queries but does not remove the load on cache misses.