Fix Kubernetes Probes Failing With 400 DisallowedHost in Django

Problem statement

The pod starts, but it never becomes ready, or it keeps restarting. kubectl describe pod shows:

Warning  Unhealthy  Readiness probe failed: HTTP probe failed with statuscode: 400
Warning  Unhealthy  Liveness probe failed: HTTP probe failed with statuscode: 400

and the Django logs contain:

django.security.DisallowedHost: Invalid HTTP_HOST header: '10.42.1.17:8000'.
You may need to add '10.42.1.17' to ALLOWED_HOSTS.

The kubelet sends HTTP probes directly to the pod IP, and unless you set a header, the Host header is that IP and port. Django's ALLOWED_HOSTS lists your public domain, not ephemeral pod IPs, so it rejects the probe with 400. The app is healthy; the probe is being refused at the door.

1. Add a host header to the probe: httpHeaders: [{name: Host, value: app.example.com}].
2. Or serve the health endpoint from middleware placed before host validation, so it never checks Host.
3. Avoid ALLOWED_HOSTS = ["*"]; it removes protection against host header attacks.
4. Keep liveness light (process alive) and readiness meaningful (database reachable).
5. Use a startupProbe so slow boots are not mistaken for failures.

Step-by-step solution

1. Confirm the cause

kubectl describe pod web-7d9f8c-abcde | grep -A3 Unhealthy
kubectl logs web-7d9f8c-abcde | grep DisallowedHost | tail -3

Status 400 together with DisallowedHost lines that name the pod IP confirms it. A 500 or a timeout is a different problem.

2. Send a Host header in the probe

The simplest fix keeps Django's host checks intact:

readinessProbe:
  httpGet:
    path: /healthz/ready
    port: 8000
    httpHeaders:
      - name: Host
        value: app.example.com
  periodSeconds: 10
  failureThreshold: 3
livenessProbe:
  httpGet:
    path: /healthz/live
    port: 8000
    httpHeaders:
      - name: Host
        value: app.example.com
  periodSeconds: 20
  failureThreshold: 3

The value must be in ALLOWED_HOSTS. If SECURE_SSL_REDIRECT is on, the probe also receives a 301 to HTTPS, which kubelet counts as success but tells you nothing; exempt the health path with SECURE_REDIRECT_EXEMPT = [r"^healthz/"].

3. Or answer health checks before host validation

Host validation happens when Django reads request.get_host(), which CommonMiddleware and others call. A tiny middleware at the top of the list can answer probes first:

# core/middleware.py
from django.http import HttpResponse

class HealthCheckMiddleware:
    def __init__(self, get_response):
        self.get_response = get_response

    def __call__(self, request):
        if request.path == "/healthz/live":
            return HttpResponse("ok", content_type="text/plain")
        return self.get_response(request)

MIDDLEWARE = ["core.middleware.HealthCheckMiddleware", "django.middleware.security.SecurityMiddleware", ...]

This endpoint must not do anything sensitive, because it bypasses host checks and every later middleware. Keep the readiness check, which touches the database, behind normal routing with the Host header from step 2.

4. Do not allow every host

ALLOWED_HOSTS = ["*"] makes probes pass but lets attackers inject arbitrary host names into password reset links and cache keys. Adding the pod CIDR dynamically at startup (reading the pod IP from the Downward API) works, but is more moving parts than a header.

5. Separate liveness from readiness

A liveness probe that queries the database restarts every pod when the database blips, turning a short outage into a long one. Use:

• startupProbe: generous, lets migrations or warm-up finish.
• livenessProbe: process responds; no dependencies.
• readinessProbe: database and cache reachable; removes the pod from the Service when not.

Explanation

Kubernetes probes are made by the kubelet on the node, straight to the pod's IP, bypassing Services and Ingress. Unlike browser traffic, they carry no domain name. Django validates the Host header against ALLOWED_HOSTS to stop host header poisoning, and pod IPs change with every reschedule, so they cannot reasonably be listed. Giving the probe a real host name, or handling the probe before validation, resolves the mismatch without weakening security.

Edge cases / notes

• Ingress health checks. Cloud load balancers created by an Ingress controller run their own health checks with their own headers; see Fix Django Health Check Failing Behind a Load Balancer.
• TCP probes. A tcpSocket probe avoids HTTP entirely but only proves the port is open, not that Django works.
• CSRF and sessions. Health endpoints should be plain GET views with no session or CSRF requirements.

Next steps

• Design probes properly with Configure Kubernetes Liveness and Readiness Probes for Django.
• For host errors outside Kubernetes, see Fix DisallowedHost in Django Production.
• If pods restart for other reasons, read Fix Kubernetes CrashLoopBackOff for Django.

FAQ

Why does the probe use the pod IP as Host?

The kubelet connects to the pod IP directly and, unless told otherwise, sends that address as the Host header.

Can I add the pod IP to ALLOWED_HOSTS?

You can read it from the Downward API at startup, but a probe header is simpler and does not depend on pod networking details.

Is ALLOWED_HOSTS = "*" acceptable inside a cluster?

Traffic from the internet still reaches Django through the Ingress with attacker-controlled Host headers, so no.

Should the liveness probe check the database?

No. Restarting pods does not fix a database outage. Check dependencies in readiness, not liveness.