Write a Disaster Recovery Plan for a Django App
Write a practical disaster recovery plan for Django: RPO and RTO targets, failure scenarios, point-in-time recovery, runbooks, access, and drills.
Problem statement
Most Django teams have backups. Far fewer can answer these questions quickly:
- If the production server disappeared right now, how long until the site is back?
- How much data would be lost?
- Who has the credentials to rebuild everything, and where are they if the password manager account is locked?
- When did anyone last restore the database from backup?
A disaster recovery (DR) plan answers them in advance. It turns a stressful incident into a sequence of known steps, and it exposes gaps (a backup stored in the same account as production, a server nobody can rebuild because its configuration was done by hand) while there is still time to fix them.
This guide walks through writing a DR plan sized for a typical Django deployment: one or a few application servers or containers, PostgreSQL, Redis, object storage, and a CDN or DNS provider.
Quick answer
- Define business targets: RPO (maximum acceptable data loss) and RTO (maximum acceptable downtime) per service.
- List failure scenarios: server loss, database corruption, accidental deletion, region outage, account compromise, and bad deploys.
- Make recovery possible: backups and point-in-time recovery for the database, media backups, infrastructure as code, and documented configuration.
- Store backups and emergency credentials outside the production account and infrastructure.
- Write runbooks for each scenario with exact commands and decision points.
- Assign roles and communication responsibilities.
- Rehearse restores and full recovery drills on a schedule, measure actual recovery time, and update the plan.
Step-by-step solution
1. Define RPO and RTO
- RPO (Recovery Point Objective): how much recent data you can afford to lose. Nightly database dumps alone give an RPO of up to 24 hours.
- RTO (Recovery Time Objective): how long the service may be unavailable.
Set them with the people who own the business consequences:
| Component | RPO | RTO | Justification |
|---|---|---|---|
| Orders and payments data | 5 minutes | 2 hours | revenue and legal records |
| User uploads | 24 hours | 8 hours | inconvenient but recoverable |
| Analytics events | 24 hours | 3 days | not customer-facing |
| Public website | n/a | 1 hour | marketing and support |
These targets drive technical choices: a 5-minute RPO requires continuous WAL archiving or a managed database with point-in-time recovery, not daily dumps.
2. Map the system and its dependencies
Document every component recovery depends on:
- application code (repository, CI, container registry)
- runtime configuration and secrets
- compute (servers, containers, platform accounts)
- PostgreSQL (primary, replicas, backups)
- Redis (cache, broker; usually rebuildable, but queued tasks may matter)
- object storage for media and static files
- DNS provider, CDN, TLS certificates
- email provider, payment provider webhooks, third-party APIs
- monitoring and alerting
For each, note where it lives, who has access, and how it would be recreated.
3. Enumerate failure scenarios
| Scenario | Example cause | Primary recovery |
|---|---|---|
| Single server lost | hardware failure, deleted VM | rebuild from infrastructure code, redeploy |
| Database lost or corrupted | disk failure, bad migration | restore backup or point-in-time recovery |
| Accidental data deletion | bug or admin action deletes rows | point-in-time restore into a separate database, copy data back |
| Media deleted | code bug, compromised credentials | object versions or media backup |
| Bad deploy | broken release | roll back release |
| Region or provider outage | cloud region unavailable | restore in another region (if RTO requires) |
| Account compromise | leaked cloud credentials | rotate credentials, restore from backups stored outside the account |
| DNS or domain problem | expired domain, DNS provider outage | registrar access, secondary DNS or documented migration |
Each scenario gets a runbook section.
4. Database recovery capabilities
Match the database setup to the RPO:
- Daily logical dumps (
pg_dump): simple, portable, RPO up to 24 hours. Good for small apps and as an extra copy. - Continuous archiving and point-in-time recovery (PITR): base backups plus archived WAL, using tools such as pgBackRest or WAL-G, or a managed database's built-in PITR. RPO of minutes, and the ability to restore to just before an accidental deletion.
- Replicas: reduce RTO for hardware failures, but replicate mistakes instantly; they are not backups.
Document retention (how far back you can restore) and test restoring to a specific timestamp.
5. Put backups out of reach of production failures
A backup that the same compromised credential can delete is not a disaster recovery backup:
- copy database and media backups to a separate account or provider with separate credentials
- use write-only or append-only credentials for backup jobs where possible, and object lock or versioning at the destination
- keep at least one copy in a different region
- encrypt backups and store the encryption keys where they remain accessible if production is gone
6. Make infrastructure reproducible
Rebuilding a hand-configured server under pressure takes far longer than the RTO usually allows. Aim for:
- infrastructure as code (Terraform, Pulumi, or the platform's configuration files) for servers, networks, databases, buckets, and DNS
- configuration management (Ansible) or container images for application servers
- a documented, tested deploy path that works on a fresh environment
- all non-secret configuration in the repository; secrets in a secret manager with documented recovery access
7. Secure emergency access
Plan for the people and access needed during a disaster:
- at least two people who can access each critical account (cloud, DNS registrar, database provider, email provider, domain registrar)
- break-glass credentials or recovery codes stored offline or in a separate password manager vault, with documented procedures
- MFA recovery options that do not depend on a single phone or person
- a contact list for providers' support, including account IDs
8. Write runbooks
A runbook entry should be executable by someone on the team who did not write it. Example for database loss with PITR on a managed service:
Scenario: Production database unavailable or corrupted
1. Declare incident; assign incident lead and communications lead.
2. Put the site in maintenance mode (enable maintenance flag or Nginx maintenance page).
3. Determine recovery point:
- hardware failure: latest available time
- bad migration or deletion: time just before the event (check deploy logs and audit logs)
4. Create a new database instance from point-in-time restore to that timestamp.
5. Verify restored data: row counts for key tables, latest order timestamp, sample records.
6. Update DATABASE_URL secret to the new instance; deploy/restart app and workers.
7. Run smoke tests (login, checkout, admin).
8. Disable maintenance mode; monitor error rates and latency.
9. Keep the old instance until post-incident review is complete.
10. Record actual RPO and RTO achieved.
Include exact commands where possible, links to consoles, and expected durations for long steps (for example "restore of 200 GB takes about 45 minutes").
A maintenance page served directly by Nginx, independent of Django and the database, is worth preparing in advance:
if (-f /srv/myapp/maintenance.on) {
return 503;
}
error_page 503 /maintenance.html;
location = /maintenance.html {
root /srv/myapp/static-pages;
internal;
}
Creating /srv/myapp/maintenance.on switches the site to the maintenance page without touching application code.
9. Communication plan
Decide in advance:
- who updates the status page and customers
- templates for the first message ("We are investigating an outage affecting...")
- update frequency during the incident
- when and how to notify customers about data loss, including legal or contractual obligations
10. Rehearse
Plans that are never tested fail in predictable ways: expired credentials, missing permissions, undocumented steps, restores slower than expected. Schedule:
- monthly: automated restore of the latest database backup into a scratch environment with basic checks
- quarterly: restore database and media into staging from backups and run the application against them
- yearly (or after major changes): full DR drill rebuilding the environment from infrastructure code in a separate account or region, timed against the RTO
Record results, time taken, and issues found, then update runbooks.
Verification check:
- RPO and RTO targets are agreed and written down
- each scenario has a runbook with commands and owners
- backups exist outside the production account and region
- a restore drill completed within the RTO in the last quarter
- at least two people can access every critical account
Explanation
Disaster recovery planning shifts work from the worst possible moment to a calm one. Defining RPO and RTO turns vague reliability goals into specific backup frequencies and rebuild capabilities. Separating backups and credentials from production protects against the scenarios where both are lost together. Runbooks and drills convert individual knowledge into a repeatable process, and each drill reveals the gaps that would otherwise surface during a real outage.
Edge cases / notes
Small teams
For a solo developer or small team, a one-page plan with RPO and RTO, backup locations, account access, and a tested restore procedure already covers the most likely disasters.
Platform-as-a-service deployments
PaaS providers handle hardware failures, but not accidental deletions, account issues, or provider-wide outages. You still need independent database and media backups and the ability to deploy elsewhere if RTO demands it.
Redis and queues
Caches can be rebuilt, but queued Celery tasks in Redis may be lost. For critical workflows, record intent in the database so tasks can be re-enqueued after recovery.
Common mistakes
- Treating replicas as backups.
- Storing backups in the same cloud account with the same credentials as production.
- Runbooks that say "restore the database" without commands, timings, or verification.
- Never timing a real restore against the RTO.
Next steps
- Implement database backups with Backup and Restore PostgreSQL for Django Apps.
- Protect uploads with Back Up Django Media Files.
- Check coverage with the Django Backup and Recovery Checklist.
- Make servers reproducible with Provision a Django Server Baseline with Terraform.
FAQ
What is the difference between RPO and RTO?
RPO is how much data you can afford to lose, measured in time before the incident. RTO is how long the service can be unavailable before it must be restored.
Are database replicas enough for disaster recovery?
No. Replicas copy deletions and corruption immediately. You also need backups with retention and ideally point-in-time recovery.
How often should I test Django backup restores?
Automate a basic database restore check monthly and run fuller restores into staging at least quarterly, with a complete recovery drill yearly or after major infrastructure changes.
What belongs in a disaster recovery runbook?
The scenario, decision points, exact commands and consoles, expected durations, verification steps, owners, and communication tasks, written so another team member can follow them.