Secrets Management in Docker Compose

1. The Problem: Why Environment Variables Can Be Risky

Before we look at solutions, it's essential to understand the problem. It's common for developers to pass configuration, including secrets like API keys or database passwords, using environment variables. While this is better than hardcoding them directly into a Dockerfile or docker-compose.yml, it's not without risks.

As the article highlights, any process within the container can read all environment variables. They can also be accidentally leaked in application logs, especially during error stack traces, and are easily inspectable from the Docker host. Our goal is to find a method that limits this exposure.

2. Strategy 1: Using .env Files

A first step away from hardcoding secrets is to use an environment file (typically named .env). Docker Compose automatically detects and loads a .env file from your project's root directory, making its variables available for substitution in your docker-compose.yml file.

Here's the pattern:

1. Create a .env file:

   # .env
   # This file should NOT be committed to git!
   POSTGRES_PASSWORD=mysecretpassword
   

2. Reference it in docker-compose.yml:

   # docker-compose.yml
   services:
     db:
       image: postgres
       environment:
         - POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
   

3. Ignore the .env file in version control:
   Your .gitignore file must include .env to prevent secrets from being committed.

   # .gitignore
   .env
   

This approach externalizes secrets from your Compose file, which is a good practice. However, it still results in the secret being set as a plain-text environment variable inside the container, with the risks we discussed earlier.

3. Strategy 2: Docker Compose Secrets (The Secure Standard)

The most secure, recommended way to handle runtime secrets is with Docker's native secrets feature. Instead of injecting a secret as an environment variable, Docker securely mounts it as a file into a temporary in-memory filesystem inside the container, typically at /run/secrets/<secret_name>.

This has several key advantages:

• The secret is not part of the container's environment, preventing accidental leakage through logs.
• The secret is stored in memory (tmpfs) on the container's filesystem and is never written to disk on the host or inside the container.
• You can explicitly control which services have access to which secrets.

Here’s how it works:

1. Create a file containing your secret:

   # On your host machine
   echo "super-secret-db-password" > db_password.txt
   

2. Define the secret in docker-compose.yml:
   You define a top-level secrets block and then grant services access to it.

   services:
     db:
       image: postgres:16
       environment:
         # This tells the postgres image to read its password from a file
         POSTGRES_PASSWORD_FILE: /run/secrets/db_password
       secrets:
         - db_password # Grant this service access to the secret
   
   secrets:
     db_password:
       file: ./db_password.txt # Define the secret and its source file
   

Notice the POSTGRES_PASSWORD_FILE environment variable. This is a common convention used by many official images (like PostgreSQL and MySQL) to read a secret from a file instead of an environment variable. This pattern is crucial for using Docker secrets effectively.

4. Advanced Control: The Long Syntax for Secrets

By default, secrets mounted in a container are world-readable (permissions 0444). For enhanced security, especially in a professional context, you should restrict access to the specific user running the application process. Docker Compose's "long syntax" for secrets allows you to do this.

Here's an example applying what you just read, restricting a secret to be readable only by the postgres user and group (which both have the ID 999 inside the official image):

services:
  db:
    image: postgres:16
    environment:
      POSTGRES_PASSWORD_FILE: /run/secrets/db_password
    secrets:
      - source: db_password
        target: db_password # Optional: name of file inside /run/secrets/
        uid: '999'           # User ID of the owner
        gid: '999'           # Group ID of the owner
        mode: 0400           # Permissions: read-only for the owner only

secrets:
  db_password:
    file: ./db_password.txt

This level of granular control is a hallmark of a production-grade, secure setup, applying the principle of least privilege directly within your container definitions.

version: "3.8"
services:
  api:
    image: my-api:1.0
    ports:
      - "8000:8000"
    environment:
      - DATABASE_URL=postgresql://user:hardcoded_password@db:5432/myapp
      - STRIPE_API_KEY=sk_test_12345
  db:
    image: postgres:16
    environment:
      - POSTGRES_USER=user
      - POSTGRES_PASSWORD=hardcoded_password
      - POSTGRES_DB=myapp

hardcoded_password

sk_test_12345

version: "3.8"
services:
  api:
    image: my-api:1.0
    ports:
      - "8000:8000"
    environment:
      # We still construct the DB URL here, but now we're not exposing the password.
      # The password will be read by the postgres driver inside the app from the file.
      # Note: This assumes the application logic can handle a file path in the password field.
      # A more robust solution is to set individual DB vars and have the app build the URL.
      # For this exercise, we'll keep the URL but will handle the Stripe key via file.
      - DATABASE_URL=postgresql://user@db:5432/myapp 
    secrets:
      - stripe_api_key

  db:
    image: postgres:16
    environment:
      - POSTGRES_USER=user
      - POSTGRES_DB=myapp
      - POSTGRES_PASSWORD_FILE=/run/secrets/db_password
    secrets:
      - db_password

secrets:
  db_password:
    file: ./db_pass.txt
  stripe_api_key:
    file: ./stripe_key.txt

Conclusion

You've now learned the professional standard for managing secrets in Docker Compose. While .env files offer a basic level of separation, Docker's native secrets feature provides a much more robust and secure solution by mounting secrets as in-memory files with controllable permissions.

Key Takeaways:

• Avoid hardcoding secrets. Never commit secrets to version control.
• .env files are a simple way to externalize configuration, but secrets still exist as plain-text environment variables inside the container.
• Docker Compose secrets are the recommended approach. They mount secrets as files in an in-memory filesystem (/run/secrets), limiting their exposure.
• Use the *_FILE convention for images that support reading secrets from files (e.g., POSTGRES_PASSWORD_FILE).
• Apply the principle of least privilege by using the long syntax for secrets to set restrictive uid, gid, and mode permissions.

In our next lesson, we'll build on our knowledge of managing configurations by learning how to override and extend Compose configurations for different environments. This will allow us to create a base docker-compose.yml and then layer on development- or production-specific settings, a powerful pattern for maintaining clean and scalable projects.