Managing Services with Compose Profiles

1. What Are Docker Compose Profiles?

Imagine you have services that you only need for specific tasks:

• A database admin tool like phpmyadmin or pgadmin for development.
• An email-catching service like mailhog for local testing.
• A separate service that runs your integration test suite.
• A monitoring stack (prometheus, grafana) that you might not want running all the time.

In the past, you might have managed this by commenting out services in your docker-compose.yml, using multiple override files, or maintaining separate files altogether (docker-compose.dev.yml, docker-compose.test.yml). These approaches can lead to configuration drift and become difficult to manage.

Compose profiles solve this by letting you assign one or more "profile" names to a service. A service with a profile will only start if that profile is explicitly activated.

• Services without a profiles key are considered part of a default group; they are enabled when you run docker compose up without any profile flags.
• Services with a profiles key are disabled by default and only run when their profile is active.

Let's look at a visual example.

This image neatly illustrates the core idea: one file, multiple configurations, activated on demand. This is perfect for a situation where you might use a standard CPU-based service for development and testing, but a more powerful GPU-based service in production.

2. Defining and Activating Profiles

The syntax is straightforward. You add a profiles key to any service, followed by a list of profile names.

services:
  web:
    image: nginx
    # No profiles key, so this is an "always-on" service.

  db:
    image: postgres
    profiles: ["prod"] # Only runs when the 'prod' profile is active.

  tests:
    image: my-app-tests
    profiles: ["test"] # Only runs when the 'test' profile is active.

You can activate profiles using the --profile flag on the command line.

# Start only the 'web' service (default)
docker compose up -d

# Start 'web' and 'db' services
docker compose --profile prod up -d

# Start 'web' and 'tests' services
docker compose --profile test up -d

Notice that the service without a profile (web) is included when you activate other profiles.

3. A Practical Use Case: Managing Dev, Prod, and Test Environments

The real power of profiles comes from managing entire application environments. As a DevOps Engineer, this is a pattern you'll find incredibly useful.

Let's walk through an example that sets up distinct prod, dev, and test environments using profiles.

This article clearly demonstrates how you can define entirely different services (like db for production and db-dev for development) and use profiles to select the correct one at runtime.

services:
  webapp:
    profiles: ["core"]
    # ...
  database:
    profiles: ["core"]
    # ...
  cache:
    profiles: ["core"]
    # ...
  monitoring:
    profiles: ["monitor"]
    # ...

4. Advanced Concepts and Best Practices

For professional use, especially in automated CI/CD pipelines, there are a couple of advanced features you should know. The following video from Nick Janetakis, a well-regarded expert in the Docker community, covers these perfectly.

Let's summarize the two key takeaways from the video:

1. Use the COMPOSE_PROFILES Environment Variable: Instead of --profile flags, you can define an environment variable, typically in your .env file:

   # In .env file
   COMPOSE_PROFILES=core,monitor
   

   Now, a simple docker compose up -d will activate the core and monitor profiles. This is the preferred method for CI/CD pipelines and production deployments because it keeps the configuration self-contained and the command simple.

2. depends_on Works Intelligently: In modern Docker Compose, if service-A has depends_on: service-B, but service-B is not in an active profile, Compose simply ignores the dependency. This is a huge benefit. For example, your webapp can depend_on a database service for local development, but in production (where you use a managed cloud database), you can simply run with a profile that excludes the database service, and your application will start without errors.

Conclusion

You've now learned how to use Docker Compose profiles to create flexible, manageable, and environment-aware application stacks. This technique is fundamental to professional DevOps workflows.

Key Takeaways:

• Profiles allow you to label services to be enabled or disabled in groups.
• They are a superior alternative to multiple Compose files or commented-out services for managing different environments (dev, test, prod).
• Activate profiles with the docker compose --profile <name> flag or, preferably, by setting the COMPOSE_PROFILES environment variable.
• Modern Docker Compose handles depends_on intelligently, ignoring dependencies on services in inactive profiles, which simplifies using external services (like managed databases) in production.

In our next lesson, we will tackle another critical aspect of multi-service applications: implementing a strategy for managing secrets in Docker Compose. We'll look at how to handle sensitive data like API keys and passwords securely, comparing different methods like environment files and Docker's built-in secrets management.