Fix boto3 NoCredentialsError in Django

Problem statement

Uploads to S3 through django-storages, emails through SES, or any boto3 call fails in production with:

botocore.exceptions.NoCredentialsError: Unable to locate credentials

boto3 searched every place it knows for AWS credentials and found none. It never reached AWS, so this is not a permissions problem (that would be AccessDenied). The process running Django simply has no credentials: environment variables were not loaded into Gunicorn or Celery, the EC2 or ECS role is missing or unreachable, or settings pass empty strings that block the default chain.

1. On AWS compute (EC2, ECS, EKS, Lambda), attach an IAM role and do not set access keys at all.
2. Elsewhere, provide AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY to every process (Gunicorn, Celery, cron) via the service environment.
3. Do not set AWS_ACCESS_KEY_ID = env("AWS_ACCESS_KEY_ID", default="") in settings; an empty string disables the role lookup in django-storages.
4. On EC2 inside Docker, raise the IMDSv2 hop limit to 2 so containers can reach the role credentials.
5. Test in the same context as the app: python manage.py shell -c "import boto3; print(boto3.client('sts').get_caller_identity())".

Step-by-step solution

1. Test from the app's own context

Run as the same user, with the same environment, as the failing process:

sudo -u deploy bash -c 'set -a; . /etc/myapp/env; set +a; /srv/app/.venv/bin/python -c "import boto3; print(boto3.client(\"sts\").get_caller_identity()[\"Arn\"])"'
docker compose exec web python -c "import boto3; print(boto3.client('sts').get_caller_identity()['Arn'])"

If this prints an ARN, credentials are available in that context and the problem is in how the app process is started. If it raises NoCredentialsError, the context itself lacks them.

2. Use an IAM role on AWS compute

On EC2, attach an instance profile with a policy for your bucket. On ECS, set the task role (not only the execution role, which is for pulling images and reading secrets). On EKS, use IRSA or Pod Identity. Then remove access keys from settings entirely; boto3 finds the role automatically.

# settings: no keys at all when running on AWS
STORAGES = {
    "default": {
        "BACKEND": "storages.backends.s3.S3Storage",
        "OPTIONS": {"bucket_name": env("AWS_STORAGE_BUCKET_NAME"), "region_name": env("AWS_S3_REGION_NAME")},
    },
    "staticfiles": {"BACKEND": "whitenoise.storage.CompressedManifestStaticFilesStorage"},
}

3. Avoid empty-string credentials

This common pattern breaks role-based credentials:

AWS_ACCESS_KEY_ID = env("AWS_ACCESS_KEY_ID", default="")
AWS_SECRET_ACCESS_KEY = env("AWS_SECRET_ACCESS_KEY", default="")

django-storages passes these to boto3; empty strings can be treated as explicit (invalid) values rather than absent ones depending on versions. Use None as the default, or omit the settings when keys are not configured.

4. Load environment variables into every process

Off AWS, keys come from the environment. The web service, Celery workers, Beat, and scheduled commands each need them:

• systemd: EnvironmentFile=/etc/myapp/env in each unit (check with systemctl show gunicorn -p Environment, or cat /proc/<pid>/environ | tr '\0' '\n' | grep AWS_).
• Docker Compose: env_file: on every service that touches S3.
• Celery started by a different unit often lacks the file; uploads from tasks then fail while web uploads work.

5. Fix IMDSv2 from containers on EC2

EC2 instances that require IMDSv2 default to a response hop limit of 1. Docker's bridge network adds a hop, so containers cannot fetch role credentials and boto3 falls through to NoCredentialsError:

aws ec2 modify-instance-metadata-options --instance-id i-0abc123 --http-put-response-hop-limit 2 --http-tokens required

6. Set the region

Some calls fail with NoRegionError alongside credential issues. Set AWS_S3_REGION_NAME (or AWS_DEFAULT_REGION in the environment) to the bucket's region.

Verification check: from inside the running service context, get_caller_identity() returns the expected role or user ARN, and a test upload with default_storage.save("healthcheck.txt", ContentFile(b"ok")) succeeds.

Explanation

boto3 resolves credentials lazily through a provider chain: explicit parameters, environment variables, shared credential files, container credential endpoints, and finally the EC2 instance metadata service. The first source that returns credentials wins. NoCredentialsError means the chain was exhausted without success, which is always a property of the process environment, not of IAM policies. Because Django apps run as several processes started by different units or containers, it is common for one process to have credentials and another not.

Edge cases / notes

• AccessDenied after fixing this means credentials work but the policy lacks permissions; see the S3 access denied guide.
• Expired temporary credentials produce ExpiredToken, usually from manually pasted session tokens.
• Non-AWS S3 (R2, Spaces, MinIO) always needs explicit keys and an endpoint URL; there is no role.
• Never commit keys to the repository or bake them into images; use roles or a secrets manager.

Next steps

• Configure S3 storage fully with Store Django Media Files on S3.
• If AWS now returns AccessDenied, see Fix Django S3 Upload Access Denied.

FAQ

Is NoCredentialsError a permissions problem?

No. boto3 found no credentials at all. Permission problems show up as AccessDenied.

Should I put AWS keys in Django settings on EC2?

No. Attach an IAM role and leave keys unset so boto3 uses the role.

Why does it work in my shell but not in Gunicorn?

Your shell has environment variables or ~/.aws files that the service user or unit does not.

Why do containers on EC2 fail when the host works?

IMDSv2's default hop limit of 1 blocks containers. Raise it to 2.