Secure Local Secrets with Docker Compose

1. The Dangers of Exposed Secrets

First, what is a "secret"? It's any piece of configuration that grants access or is otherwise sensitive. Passwords and API keys are obvious examples. If these are exposed, the consequences can be severe.

Leaking secrets can happen easily, especially when an application is in debug mode or misconfigured. The following image shows a real-world example from a Django application where sensitive environment variables, including database credentials and AWS keys, were leaked on a public-facing error page.

The most common way secrets are unintentionally exposed is by hardcoding them in source code or configuration files (like docker-compose.yml) and then committing them to a Git repository.

To understand why this is such a critical issue, let's watch a short explanation.

2. The Standard Solution: Environment Files

The industry-standard solution for managing secrets in local development is to use environment files (commonly named .env).

An environment file is a simple text file that stores configuration as KEY=VALUE pairs. Your application or orchestration tool (like Docker Compose) reads this file at runtime to load the configuration. The crucial rule is: the .env file is never committed to version control. It is always listed in your project's .gitignore file.

This diagram illustrates the basic workflow:

While the concept is simple, the mechanics of how Docker Compose uses these variables can be nuanced. Let's dig into the details.

3. How Docker Compose Manages Environment Variables

Docker Compose provides several ways to get variables into your services, and understanding their differences and precedence is key to using them correctly.

To summarize the key points from the reading:

• A .env file in your project root is loaded automatically by Docker Compose. Its primary purpose is to provide values for variable substitution inside your docker-compose.yml file (e.g., POSTGRES_PASSWORD: ${DB_PASSWORD}).
• The env_file: directive loads a file's contents directly into a container's environment. It does not support variable substitution within the file itself.
• The environment: directive has the highest precedence (besides the command line) and will override values from env_file: or an image's ENV instruction.

The most important security practice, repeated in all professional guides, is ensuring your secrets file is never committed.

4. Hands-On Exercise: Securing a Service

Let's put this into practice. Below is a docker-compose.yml file with hardcoded database credentials—a clear security anti-pattern. Your task is to refactor it to use a .env file securely.

Setup:

1. Create a new project folder.
2. Inside it, create a file named docker-compose.yml with the content below.

version: '3.8'

services:
  db:
    image: postgres:16-alpine
    container_name: my_postgres_db
    environment:



      # TODO: Move these hardcoded secrets to a .env file
      POSTGRES_USER: admin
      POSTGRES_PASSWORD: a_very_bad_password
      POSTGRES_DB: myapp
    ports:
      - "5432:5432"
    volumes:
      - postgres_data:/var/lib/postgresql/data

volumes:
  postgres_data:

Instructions:

1. Create a .env file in the same directory. Inside it, define the three secret variables: POSTGRES_USER, POSTGRES_PASSWORD, and POSTGRES_DB. Assign them secure, non-default values.

   
   
   
   # .env
   POSTGRES_USER=myuser
   POSTGRES_PASSWORD=somesecurepassword123
   POSTGRES_DB=servicedb
   

2. Modify docker-compose.yml. Change the environment section for the db service to use variable substitution to read the values from your .env file.
   Hint: Use the ${VARIABLE_NAME} syntax.
3. Create a .gitignore file. Add the .env file to it to ensure it's never committed to version control. This is a critical step.
4. Run and Verify. Start the service using docker compose up -d. Then, inspect the running container's environment variables to confirm it received the values from your .env file:

   docker inspect my_postgres_db | grep POSTGRES_
   

   You should see the values you set in your .env file, not the old hardcoded ones.

version: '3.8'

services:
  db:
    image: postgres:16-alpine
    container_name: my_postgres_db
    environment:
      POSTGRES_USER: ${POSTGRES_USER}
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
      POSTGRES_DB: ${POSTGRES_DB}
    ports:
      - "5432:5432"
    volumes:
      - postgres_data:/var/lib/postgresql/data

volumes:
  postgres_data:

POSTGRES_USER=myuser
POSTGRES_PASSWORD=somesecurepassword123
POSTGRES_DB=servicedb

.env

5. Limitations and More Secure Alternatives

Using .env files is a great practice for local development, but it's important to understand its limitations, especially as you approach production environments.

The primary weakness is that any variable passed via the environment or env_file key is stored in plain text in the container's configuration and can be easily viewed by anyone with Docker access using docker inspect.

For more sensitive scenarios, even in development, Docker Compose offers a more secure mechanism: the top-level secrets directive. Instead of injecting secrets as environment variables, this feature mounts them as files in a temporary in-memory filesystem inside the container at /run/secrets/.

Your application can then read the secret from this file. This is more secure because:

1. The secrets are not exposed via docker inspect.
2. The files are read-only and exist only in memory.
3. You can control file permissions with fine-grained access.

Many official Docker images, including Postgres, support this pattern out-of-the-box. They look for a _FILE suffix on environment variables. For example, if you set POSTGRES_PASSWORD_FILE=/run/secrets/db_password, the Postgres container will read its password from that file instead of from the POSTGRES_PASSWORD environment variable.

The following article shows how to implement this pattern.

While we won't implement this more advanced pattern today, it's crucial to know it exists as you move toward production-grade security.

Conclusion

You have now mastered a fundamental security practice for local development. By externalizing secrets into a git-ignored .env file, you drastically reduce the risk of accidentally exposing sensitive credentials.

Key Takeaways:

• Never hardcode secrets in source code or configuration files that are checked into version control.
• For local development, use a .env file to store secrets and add it to your .gitignore file.
• Docker Compose automatically loads a .env file from the project root for variable substitution within docker-compose.yml.
• Environment variables injected into a container are visible via docker inspect, making them less secure than file-based secrets.
• For a more secure approach, use the top-level secrets directive in Docker Compose to mount secrets as in-memory files.

Preview of the Next Lesson

We have now covered several patterns for making our system more resilient and secure. In the next lesson, we will return to runtime resilience by exploring how to design for graceful degradation when a non-critical downstream service is unavailable. This will ensure our system remains partially functional even when parts of it fail, a core tenet of robust microservice architecture.