Managing Environment Configurations with Docker Compose

1. Externalizing Configuration with .env Files

The first principle of environment-agnostic applications is to separate configuration from code. Sensitive data like API keys, database passwords, or external hostnames should never be committed to version control. Docker Compose provides a simple and powerful way to handle this using .env files.

By default, when you run docker compose up, Docker looks for a file named .env in the same directory. It reads the variables from this file and makes them available for substitution within your docker-compose.yml.

Let's see how this works in practice.

As the video shows, the process involves two steps:

1. Create a .env file: In your project root, create a file named .env with key-value pairs.

   # .env
   MYSQL_ROOT_PASSWORD=my-secret-pw
   WEB_APP_PORT=8080
   

2. Reference variables in docker-compose.yml: Use the ${VARIABLE_NAME} syntax to substitute the values.

   # docker-compose.yml
   services:
     db:
       image: mysql:8.0
       environment:
         - MYSQL_ROOT_PASSWORD=${MYSQL_ROOT_PASSWORD}
     
     web:
       build: .
       ports:
         - "${WEB_APP_PORT}:80"
   

Targeting Different Environments with --env-file

The default .env file is great, but what if you have separate configurations for development, staging, and production? You can create multiple environment files (e.g., .env.dev, .env.prod) and specify which one to use at runtime with the --env-file flag.

For example, you could have:

• .env.dev: DEBUG=true
• .env.prod: DEBUG=false

And run them like this:

• Development: docker compose --env-file .env.dev up
• Production: docker compose --env-file .env.prod up

This technique provides a clean way to manage variable configuration values across different deployment targets.

2. Overriding Compose Files for Structural Changes

Environment variables are perfect for values that change, but what about structural differences? For example:

• In development, you want to use a bind mount to sync your source code for live-reloading.
• In production, you want to use the pre-built image without any mounts.
• In development, you might expose a database port for debugging, but in production, it should only be accessible within the Docker network.

Trying to manage this with environment variables would be clumsy. The solution is to use multiple Compose files. The strategy is known as base and override.

The core idea is to have a docker-compose.yml with the base configuration shared across all environments. Then, you create additional files to override or extend that base.

The Automatic Override for Development

By convention, Docker Compose automatically looks for a file named docker-compose.override.yml. If it exists, its configuration is merged on top of the base docker-compose.yml. This is perfect for local development tweaks because you don't need to type any extra flags.

Let's review a detailed explanation of this powerful feature.

Here's a typical example:

docker-compose.yml (Base)

services:
  web:
    build: .
    command: npm start
    environment:
      - NODE_ENV=production

docker-compose.override.yml (Development)

services:
  web:
    command: npm run dev # Override the command for dev
    environment:
      - NODE_ENV=development # Override the environment
    volumes:
      - .:/app # Add a volume for live-reloading
    ports:
      - "3000:3000" # Add a port mapping for local access

When you run docker compose up, Docker automatically combines these, giving you the perfect development setup. For production, you would simply deploy without the override file.

Explicit Overrides with the -f Flag

The automatic override is great for one-off tweaks, but for managing distinct environments like staging and production, you'll want to be more explicit. The -f (or --file) flag lets you specify a list of Compose files. Docker merges them in the order you provide them, with later files overriding earlier ones.

This video demonstrates the -f flag and a couple of other powerful tricks.

To summarize the workflow:

1. Base file: docker-compose.yml
2. Production override: docker-compose.prod.yml
3. Command to run for production:
   docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d

The docker compose config command is your best friend when working with multiple files. It prints the final, merged YAML to the console, allowing you to see exactly what Docker Compose will execute. Always use it to verify your setup before deploying.

# docker-compose.yml
services:
  webapp:
    build: .
    command: gunicorn --bind 0.0.0.0:8000 myapp.wsgi
    environment:
      - DJANGO_SETTINGS_MODULE=myapp.settings.production

# docker-compose.override.yml
services:
  webapp:
    command: python manage.py runserver 0.0.0.0:8000
    environment:
      - DJANGO_SETTINGS_MODULE=myapp.settings.development
    volumes:
      - .:/app
    ports:
      - "8000:8000"

Conclusion

Today you've learned the two primary techniques for managing environment-specific configurations in Docker Compose, moving you closer to creating professional, production-ready applications.

Key Takeaways:

• .env files are used to externalize configuration values (like secrets and ports), keeping your Compose files clean and secure. Use --env-file to specify environment-specific files.
• Multiple Compose files are used for structural overrides (like adding volumes, changing commands, or modifying port mappings).
• docker-compose.override.yml is loaded automatically, making it ideal for local development changes.
• The -f flag gives you explicit control to combine multiple files, which is the standard pattern for managing distinct dev, staging, and prod environments.
• Always use docker compose config to verify the final merged configuration before deployment.

In our next lesson, we will build upon our production setup by learning how to scale services to run multiple container instances for high availability. This is a key feature of Docker Compose for handling increased load and improving the fault tolerance of your applications.