Fix Load Balancer Health Checks Failing for Django

Problem statement

The Django site works when you open it in a browser, but the load balancer marks every target unhealthy. Depending on the platform, you see:

• AWS target groups with Health checks failed with these codes: [400] or [301]
• Elastic Beanstalk environments in Severe health
• Kubernetes pods that never become ready
• a platform that repeatedly restarts or refuses to route traffic to new instances

Health checks differ from browser traffic in ways that trip Django's safety features:

• they connect to instance or container IPs, so the Host header is an IP address that is not in ALLOWED_HOSTS
• they use plain HTTP to the target, so SECURE_SSL_REDIRECT answers with 301
• they have no session, so login-required middleware redirects them
• they run frequently with short timeouts, so a health endpoint that queries the database or external services can time out under load

The symptoms often start after a security hardening change, such as enabling HTTPS redirects or tightening ALLOWED_HOSTS.

Quick answer

1. Read the status code the load balancer reports for failed checks.
2. 400: DisallowedHost; answer the health path in a middleware before host validation, or configure the check's host header where the platform allows it.
3. 301 or 302: HTTPS redirect or login redirect; exempt the health path from SECURE_SSL_REDIRECT with SECURE_REDIRECT_EXEMPT, or answer it before SecurityMiddleware.
4. 404: the path or port in the health check does not match your app.
5. Timeouts or 5xx under load: make the liveness check cheap; keep dependency checks separate and time-limited.
6. Match the check's port, path, matcher (expected status codes), interval, timeout, and thresholds to your app's startup time.
7. Verify by requesting the health path the way the load balancer does: target IP, plain HTTP, no cookies.

Step-by-step solution

1. Get the failure reason

AWS Application Load Balancer:

aws elbv2 describe-target-health --target-group-arn "$TG_ARN" \
  --query 'TargetHealthDescriptions[].{target:Target.Id,port:Target.Port,state:TargetHealth.State,reason:TargetHealth.Reason,desc:TargetHealth.Description}'

The description includes the response codes, for example Health checks failed with these codes: [400].

Kubernetes:

kubectl describe pod <pod> | grep -A5 -E "Readiness|Liveness|Warning"

Nginx or app logs: find health check requests by path and user agent (ALB uses ELB-HealthChecker/2.0):

sudo grep "ELB-HealthChecker" /var/log/nginx/access.log | tail -5

2. Reproduce the request exactly

From another host in the same network (or the instance itself), request the target IP directly over HTTP, without a hostname:

curl -si http://10.0.1.23:8000/healthz/ | head -5
curl -si http://10.0.1.23:8000/healthz/ -H "Host: 10.0.1.23:8000" | head -5

This usually reproduces the failure immediately, while curl https://www.example.com/healthz/ succeeds.

3. Fix 400: DisallowedHost

With DEBUG = False, Django rejects hosts not in ALLOWED_HOSTS. Adding private IP addresses to ALLOWED_HOSTS is brittle (instance IPs change) and weakens host validation. Instead, handle the health path before anything reads the host:

# core/middleware.py
from django.http import HttpResponse


class HealthCheckMiddleware:
    def __init__(self, get_response):
        self.get_response = get_response

    def __call__(self, request):
        if request.path == "/healthz/":
            return HttpResponse("ok", content_type="text/plain")
        return self.get_response(request)

MIDDLEWARE = [
    "core.middleware.HealthCheckMiddleware",
    "django.middleware.security.SecurityMiddleware",
    # ...
]

ALLOWED_HOSTS is enforced when request.get_host() is called, which normal middleware and views trigger; this middleware returns before that happens. It also runs before SecurityMiddleware, so the HTTPS redirect does not apply to it.

Some platforms let you set the Host header used by health checks, and Kubernetes probes accept httpHeaders. That is an alternative, but the middleware works everywhere without depending on platform features.

4. Fix 301: HTTPS redirects

With SECURE_SSL_REDIRECT = True, plain HTTP requests are redirected. Load balancers talk to targets over HTTP and treat 301 as unhealthy unless configured to accept it. Options:

• the middleware above, placed before SecurityMiddleware
• exempt the path:

SECURE_REDIRECT_EXEMPT = [r"^healthz/$"]

The patterns match the path without the leading slash.

Do not "fix" this by accepting 301 in the health check matcher; a redirect proves only that Django is running up to SecurityMiddleware, not that the app works.

5. Fix 302: authentication redirects

Global login requirements (LoginRequiredMiddleware in Django 5.1+, or custom middleware) redirect anonymous requests to the login page. Exempt the health view:

from django.contrib.auth.decorators import login_not_required
from django.http import HttpResponse


@login_not_required
def healthz(request):
    return HttpResponse("ok", content_type="text/plain")

login_not_required exists in Django 5.1 and later for use with LoginRequiredMiddleware. With custom middleware, add the path to its exemption list, or use the early middleware approach.

6. Fix 404: path and port mismatch

• the load balancer checks /health while Django routes /healthz/
• APPEND_SLASH redirects /healthz to /healthz/, returning 301
• the check uses the traffic port of Nginx on the instance while only Gunicorn has the route, or the other way round

Align the check path exactly, including the trailing slash, and confirm which process listens on the checked port.

7. Fix timeouts and flapping

A health check that queries the database, cache, and third-party APIs can exceed the check's timeout during load or brief dependency slowness, removing all targets at once and turning a partial problem into a full outage.

Design two endpoints:

• Liveness or load balancer check (/healthz/): returns quickly without external calls. It answers "is this process able to serve requests?"
• Readiness or diagnostic check (/readyz/): checks database connectivity with a short timeout. Use it for deploy gates and monitoring, and for readiness probes only if you accept that a database outage removes every pod from rotation.

from django.db import connection
from django.http import JsonResponse


def readyz(request):
    try:
        with connection.cursor() as cursor:
            cursor.execute("SELECT 1")
        return JsonResponse({"database": "ok"})
    except Exception:
        return JsonResponse({"database": "error"}, status=503)

Keep such endpoints free of sensitive details, and consider restricting them to internal networks.

8. Tune health check timing

Typical ALB settings that suit a Django app with a few seconds of boot time:

For container platforms, set a grace period (ECS healthCheckGracePeriodSeconds, Kubernetes initialDelaySeconds or a startupProbe) longer than the time Gunicorn needs to import the app and boot workers.

9. Verify

curl -s -o /dev/null -w "%{http_code}\n" http://10.0.1.23:8000/healthz/
curl -s -o /dev/null -w "%{http_code}\n" http://10.0.1.23:8000/healthz/ -H "Host: 10.0.1.23"
curl -s -o /dev/null -w "%{http_code}\n" https://www.example.com/admin/ -H "Host: evil.example"

Verification check:

• health path returns 200 for requests to the target IP over HTTP
• a normal page with a bogus Host header still returns 400 (host validation intact)
• all targets show healthy in the load balancer
• a rolling deploy keeps healthy capacity throughout

Explanation

Load balancers verify targets by talking to them directly, bypassing DNS names, TLS termination, and user sessions. Django's production defaults are designed for real client traffic and correctly reject requests with unknown hosts, redirect HTTP, and require authentication. A dedicated, cheap health endpoint that runs before those checks lets both work: the load balancer gets a precise signal about process health, and user-facing traffic keeps every protection.

Edge cases / notes

Health checks through Nginx

If the load balancer checks port 80 on an instance where Nginx proxies to Gunicorn, Nginx's server blocks matter too: a default_server block that returns 444 for unknown hosts drops health checks. Add a location for the health path in the default server, or check Gunicorn's port directly.

Logging noise

Health checks every few seconds per target flood access logs. Exclude the health path from access logs in Nginx with a map and access_log ... if=$loggable;.

Deep checks and cascading failures

If every service's health check depends on a shared database, a database blip marks the entire fleet unhealthy simultaneously. Keep load balancer checks shallow.

Testing the health endpoint like a load balancer

Add tests that send requests the way health checkers do, so later changes to middleware order, security settings, or login requirements cannot break the check silently:

from django.test import TestCase, override_settings


@override_settings(
    ALLOWED_HOSTS=["www.example.com"],
    SECURE_SSL_REDIRECT=True,
    DEBUG=False,
)
class HealthCheckTests(TestCase):
    def test_health_check_with_ip_host_over_http(self):
        response = self.client.get("/healthz/", HTTP_HOST="10.0.1.23:8000")
        self.assertEqual(response.status_code, 200)
        self.assertEqual(response.content, b"ok")

    def test_normal_page_with_ip_host_is_rejected(self):
        response = self.client.get("/", HTTP_HOST="10.0.1.23:8000")
        self.assertEqual(response.status_code, 400)

The second test proves the health exemption did not weaken host validation for other URLs.

Excluding health checks from access logs

Frequent checks make access logs noisy and inflate request metrics. In Nginx:

map $request_uri $loggable {
    ~^/healthz/ 0;
    default     1;
}

access_log /var/log/nginx/access.log combined if=$loggable;

In Gunicorn, access log filtering requires a custom logger class; many teams filter health checks in their log pipeline instead.

Deploy behavior

During rolling deploys, new targets must pass the configured number of consecutive successful checks before receiving traffic, and old targets drain connections. If deploys cause brief errors:

• make sure the health endpoint only returns 200 after Gunicorn workers have booted
• set the load balancer's deregistration delay (connection draining) longer than your slowest normal request
• keep Gunicorn's graceful timeout at least as long as the draining period, so in-flight requests finish

Common mistakes

• Adding * or internal IP ranges to ALLOWED_HOSTS to satisfy health checks.
• Accepting 301 or 400 as healthy in the load balancer matcher.
• Health endpoints that call external APIs or run slow queries.
• Too short a grace period for app startup, causing restart loops during deploys.

Next steps

• Design useful endpoints with How to Build a Django Health Check Endpoint for Deployments.
• Apply the fix on Beanstalk with Deploy Django on AWS Elastic Beanstalk.
• Configure ECS target groups with Deploy Django on AWS ECS Fargate.
• Understand host validation in How to Fix DisallowedHost in Django Production.

FAQ

Why does my load balancer health check get 400 from Django?

The check uses the target's IP address as the Host header, which is not in ALLOWED_HOSTS, so Django raises DisallowedHost. Answer the health path in middleware before host validation.

Should I add the instance IP to ALLOWED_HOSTS?

It works only while IPs stay the same and weakens host validation. Handling the health path in early middleware is more robust.

Why does the health check fail with 301 after enabling HTTPS?

SECURE_SSL_REDIRECT redirects the load balancer's plain HTTP request. Exempt the health path with SECURE_REDIRECT_EXEMPT or answer it before SecurityMiddleware.

Should the load balancer health check test the database?

Usually not. A shallow check avoids removing every target during brief database issues. Use a separate readiness or monitoring endpoint for dependency checks.