Defining Multi-Service Applications with Docker Compose

1. The Problem with Manual Container Management

Before we dive into the solution, let's solidify our understanding of the problem Docker Compose solves. Imagine our e-commerce application has a web frontend, a gRPC user service, a gRPC order service, and a PostgreSQL database. Manually, you'd have to:

1. Create a network: docker network create my_network
2. Run the database container: docker run --network my_network --name db ... postgres
3. Run the user service container: docker run --network my_network --name user-service ...
4. Run the order service container: docker run --network my_network --name order-service ...
5. And so on...

This process is error-prone, hard to version control, and difficult to share with teammates. There has to be a better way.

2. Introducing Docker Compose

Docker Compose is a tool for defining and running multi-container Docker applications. It uses a YAML file, typically named docker-compose.yml, to configure an application's services. With a single command, you can create and start all the services from your configuration.

To get a high-level overview of what Docker Compose is and the problems it was designed to solve, please watch the introduction of this excellent tutorial from TechWorld with Nana.

3. Anatomy of a docker-compose.yml File

The docker-compose.yml file is the heart of Docker Compose. It's where you declaratively define your application stack. Let's break down its structure.

A docker-compose.yml file typically has a few top-level keys, with services being the most important.

The following video segment demonstrates how the long docker run commands we've seen before are mapped directly to YAML configuration under the services key.

As you saw, the mapping is quite direct. Here are the most common keys you'll use when defining a service:

• services: The top-level key that contains the definition for all your individual containers (services).
• <service-name>: A name you choose for your service (e.g., api, db, redis). This name is also the hostname the service will use on the Docker network.
• image: Specifies the Docker image to use for the container. This is used for pre-built images, like postgres:15 or redis:alpine.
• build: Specifies the build information for an image that needs to be created from a Dockerfile.
  • context: The path to the directory containing the Dockerfile and source code. . refers to the current directory.
  • dockerfile: The name of the Dockerfile (if it's not named Dockerfile).
• ports: Maps ports between the host machine and the container, in the format "HOST_PORT:CONTAINER_PORT".
• environment: Sets environment variables inside the container. This is how you pass configuration like database passwords or API keys.
• volumes: Mounts host paths or named volumes into the container, primarily used for persisting data or providing live code reloading during development.
• networks: Specifies which networks the service should connect to. If you don't specify this, Docker Compose creates a single default network for you.
• depends_on: Controls the startup order of services. For example, you can make your API wait for the database to start.

4. A Practical Example: gRPC Service and API Gateway

Let's look at a complete example that mirrors the kind of architecture we are building in this course. The "Python Microservices With gRPC" article provides a perfect docker-compose.yaml for a marketplace service (acting as our API gateway) and a recommendations gRPC service.

This example beautifully illustrates the power of Docker Compose:

1. Service Definition: Two distinct services are clearly defined.
2. Building Images: Both services are built from their respective Dockerfiles using the build directive.
3. Networking is Automated: Docker Compose creates the microservices network. The services can communicate using their service names (recommendations) as hostnames because they are on the same network. This automates the docker network create and --network steps we did manually.
4. Configuration via Environment Variables: The marketplace service is configured to find the recommendations service via an environment variable. This is a clean, standard practice.

To start this entire application, you would navigate to the project's root directory (where the docker-compose.yaml file is) and run a single command:

docker compose up

Docker Compose reads the file, builds any images that need building, creates the network, and starts the containers in the correct order. To stop and remove everything, another simple command suffices:

docker compose down

5. Hands-on Exercise: Define Your Own Multi-Service App

Now, it's your turn to apply what you've learned. Let's create a docker-compose.yml file for a simple application consisting of a FastAPI backend and a PostgreSQL database.

Scenario:
You have a project directory structured as follows:

.
├── api/
│   ├── main.py
│   └── Dockerfile
└── docker-compose.yml

• The api/Dockerfile builds your FastAPI application, which runs on port 8000.
• The API needs to connect to a PostgreSQL database.
• The database service should be named db.
• The API service should be named api.

Your Task:

Write the content of the docker-compose.yml file. It should accomplish the following:

1. Define two services: api and db.
2. The api service should be built using the Dockerfile in the ./api directory.
3. The api service should expose port 8000 on the host, mapping it to port 8000 in the container.
4. The db service should use the public postgres:15-alpine image.
5. Set the required POSTGRES_PASSWORD environment variable for the db service to a value of your choice (e.g., supersecret).
6. The api service needs to know the database credentials. Pass the database password to the api service as an environment variable named DATABASE_PASSWORD.

version: '3.8'

services:
  api:
    build:
      context: ./api
    ports:
      - "8000:8000"
    environment:
      - DATABASE_PASSWORD=supersecret



      # You would also pass other DB connection info here, like host, user, etc.
      # The hostname for the database is 'db', the service name.
      - DATABASE_HOST=db
      - DATABASE_USER=postgres
    depends_on:
      - db

  db:
    image: postgres:15-alpine
    environment:
      - POSTGRES_PASSWORD=supersecret



      # This is the default user, you could change it if needed
      - POSTGRES_USER=postgres



    # The default postgres port is 5432. We don't need to expose it to the host
    # because the 'api' service can reach it over the internal Docker network.

Conclusion

You've now learned how to use Docker Compose to define and manage a multi-service application. This is a fundamental skill for microservices development, as it provides a reproducible and easy-to-use environment for both local development and testing.

Key Takeaways:

• Declarative over Imperative: Docker Compose lets you declare the desired state of your application in a YAML file, rather than running a series of manual commands.
• Single File, Single Command: An entire application stack can be defined in docker-compose.yml and managed with commands like docker compose up and docker compose down.
• Automated Networking: Services defined in the same file can automatically find each other using their service names as hostnames.
• Standard for Local Development: Docker Compose is the industry standard for spinning up local development environments that mirror a production setup.

Preview of the Next Lesson

We've successfully defined our services in a docker-compose.yml file. In the next lesson, we will dive deeper into two crucial aspects: "Configure service-to-service communication and persistent data volumes using Docker Compose." We'll explore how to ensure data in our database isn't lost when a container restarts and how to mount our source code into a container for live-reloading during development.