"Docker Compose for Multi-Service Applications"

1. Persisting Data with Volumes

A core principle of containers is that they are ephemeral, or short-lived. When you run docker compose down, the containers are not just stopped; they are removed. If you then run docker compose up, new containers are created from the base images.

For a stateless service like an API, this is fine. But for a stateful service like a PostgreSQL database, this is a critical problem. Any data written to the database—new users, orders, etc.—would be completely wiped out.

The solution is to store the data outside the container's filesystem in a Docker volume. Think of a volume as a dedicated hard drive for your container, managed by Docker, which persists even when the container is removed.

There are two main ways to mount data into a container:

• Bind Mounts: This maps a directory from your host machine directly into the container (e.g., - ./my-app:/app). This is great for development, like mounting your source code for live reloading, but it's not ideal for database data as it ties your application to a specific host file structure.
• Named Volumes: This is the preferred method for persisting application data. You give a volume a name (e.g., postgres-data), and Docker takes care of managing the actual storage location on the host machine. This makes your setup more portable and decouples it from the host's filesystem.

The following video provides an excellent demonstration of this concept. It first shows the problem of data loss, then illustrates how both bind mounts and named volumes solve it, highlighting why named volumes are superior for database persistence.

Configuring Volumes in docker-compose.yml

To use a named volume, you need to do two things in your docker-compose.yml:

1. Declare a top-level volumes key to define the named volume.
2. In your service definition, mount this named volume to the correct path inside the container.

Here's a typical example for a PostgreSQL database:

version: '3.8'

services:
  db:
    image: postgres:15-alpine
    environment:
      - POSTGRES_PASSWORD=supersecret
    volumes:



      # Mounts the named volume 'db-data' to the directory where Postgres stores its data
      - db-data:/var/lib/postgresql/data




# Top-level declaration of the named volume
volumes:
  db-data:

Now, when you run docker compose down -v (the -v flag is needed to explicitly remove volumes), the db-data volume and its contents will be preserved. The next time you run docker compose up, a new db container will be created and will attach to the existing db-data volume, restoring all your data.

2. Service-to-Service Communication with Networks

As we touched on in the last lesson, when you run docker compose up, Docker Compose automatically creates a default bridge network and connects all defined services to it. This is why services can seamlessly communicate with each other using their service names as hostnames (e.g., your API can connect to postgres://db:5432).

While the default network is convenient, for more complex or production-like setups, using custom networks provides better organization and security.

The most important reason to use custom networks is isolation. Consider a typical web application architecture:

In this model, you can create two networks: frontend and backend.

• The nginx reverse proxy is on the frontend network to accept public traffic.
• The database is only on the backend network.
• Your api service acts as the bridge, connecting to both the frontend network (to receive requests from Nginx) and the backend network (to query the database).

This setup ensures that even if your Nginx proxy were compromised, the attacker would have no direct network path to your database, significantly improving your security posture.

The following video segments demonstrate how to create and use custom networks to achieve this secure architecture.

Configuring Networks in docker-compose.yml

Similar to volumes, configuring custom networks involves a top-level networks key and a networks key within each service.

Here is the YAML structure for the public/private network architecture described above:

version: '3.8'

services:
  nginx:
    image: nginx:alpine
    ports:
      - "80:80"
    networks:
      - frontend

  api:
    build: ./api
    networks:
      - frontend
      - backend

  db:
    image: postgres:15-alpine
    volumes:
      - db-data:/var/lib/postgresql/data
    networks:
      - backend

networks:
  frontend:
  backend:

volumes:
  db-data:

3. Hands-on Exercise

Let's put this all together. Below is a docker-compose.yml for a simple e-commerce system with an API gateway, a user service, and a database. Analyze the file and answer the questions that follow.

version: '3.8'

services:
  api-gateway:
    build: ./api_gateway
    ports:
      - "8000:8000"
    environment:
      - USER_SERVICE_HOST=user-service
    networks:
      - public-net

  user-service:
    build: ./user_service
    environment:
      - DATABASE_HOST=user-db
      - DATABASE_PASSWORD=supersecret
    networks:
      - public-net
      - private-net

  user-db:
    image: postgres:15-alpine
    environment:
      - POSTGRES_PASSWORD=supersecret
    volumes:
      - user-data:/var/lib/postgresql/data
    networks:
      - private-net

networks:
  public-net:
  private-net:

volumes:
  user-data:

Questions:

1. How is the data for the user-db service being persisted? What would happen to the data if you ran docker compose down followed by docker compose up?
2. Explain the networking setup. Why is user-service on two networks while api-gateway and user-db are each on only one?
3. What is a potential security vulnerability in this configuration? How would you change the networks configuration to fix it?

Conclusion

You have now mastered two of the most critical aspects of using Docker Compose for microservices: ensuring data persistence with named volumes and structuring communication securely with custom networks. These are not just configuration details; they are fundamental architectural decisions that impact the resilience and security of your entire system.

Key Takeaways:

• Use named volumes to persist data for stateful services like databases.
• Use custom networks to isolate services and improve security. A multi-tier network design (e.g., public, internal, private) is a standard pattern.
• Services on the same Docker network can resolve each other using their service names as hostnames.

Preview of the Next Lesson

We've connected our services and made their data persistent. However, there's still a race condition lurking. What happens if our api container starts faster than our db container and tries to connect before the database is ready? The API will crash. In the next lesson, we will solve this by learning how to "Manage service startup order and dependencies using depends_on in Docker Compose."