Docker Multi-Stage Builds for Testing

Part 1: The 'Test' Stage: Your Self-Contained QA Environment

The foundation of testing within Docker is the multi-stage build. You're already familiar with using multi-stage builds to create lean production images. We can extend this concept by creating dedicated stages for other tasks, like testing.

The core idea is to define a stage that contains your source code, your test code, and all the build-time and test-time dependencies (like a compiler, test framework, or linter). Inside this stage, you execute your test command.

Let's watch a video that perfectly illustrates this process.

As you saw in the video, the pattern is straightforward: in an early stage (e.g., named builder or test), you copy your code and then use RUN to execute your test command.

Why RUN and not CMD?

A critical detail is using the RUN instruction for your tests.

• RUN instructions are executed at build time. If any RUN command fails (i.e., exits with a non-zero status code), the Docker build process stops immediately. This is exactly what we want: the image build fails if the tests fail.
• CMD or ENTRYPOINT instructions are executed at run time, when a container is started from the image. They have no effect on the build process itself.

This pattern is language-agnostic. Let's look at how it's applied in a couple of different ecosystems.

.
├── app.py
├── requirements.txt
└── tests/
    └── test_app.py

# Stage 1: Run tests
FROM python:3.9-slim as test
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . .
# YOUR COMMAND HERE

# Stage 2: Final application image
FROM python:3.9-slim as final
WORKDIR /app
COPY --from=test /app /app
CMD ["python", "app.py"]

# YOUR COMMAND HERE
RUN pytest

Part 2: Targeting Stages and Extracting Test Artifacts

In a CI environment, you'll often want to do two things:

1. Run only the tests as a quick validation step.
2. Extract test reports or code coverage results to be displayed in your CI system's UI.

Multi-stage builds make this easy. You can tell Docker to build an image only up to a specific target stage using the --target flag.

Furthermore, once that intermediate image is built, you can create a temporary container from it and copy the artifacts out. This is a powerful pattern for CI/CD.

This build --target -> create -> cp -> rm workflow is the standard playbook for running tests and extracting evidence within a CI job.

Part 3: A Note on Integration Tests

So far, we've focused on unit tests, which typically run without external dependencies. Integration tests are more complex because they often require services like a database or a message queue. How do you handle this inside a docker build?

1. Service-in-the-Stage: For some cases, you can install and run the dependency directly within your test stage. For example, you could RUN apt-get install -y postgresql && service postgresql start && ./run-integration-tests.sh. This can get complicated but keeps the build self-contained.
2. Docker Compose (for Local/CI): A very common pattern is to use Docker Compose to define your application and its dependent services. Your CI script would run docker-compose up -d, wait for the services to be healthy, and then execute tests against them (e.g., using docker-compose exec my-app npm test). This happens outside the Dockerfile build process but is a practical way to manage test environments.
3. Testcontainers: This is an increasingly popular library-based approach. Your test code itself uses the Testcontainers library to programmatically start and stop the Docker containers it needs for a specific test. This gives you fine-grained control over dependencies. While powerful, this shifts the orchestration logic from your CI script into your application's test suite. The video "The Best Way to Run Integration Tests in Your CI/CD Pipeline" discusses this approach if you are curious to learn more.

For this course, we'll focus on tests that can be run within the Dockerfile itself or managed by the CI script, as this is a direct extension of the build process.

Part 4: Integrating Testing into Your GitHub Actions Workflow

Let's bring this all together and update the CI pipeline we built in the last lesson. We will add a test job that runs before our build-and-push job.

The new test job will:

1. Check out the code.
2. Run docker build but with --target test to only build and run the test stage.

The existing build-and-push job will be modified to only run if the test job succeeds by adding needs: test.

Here’s what the updated .github/workflows/docker-publish.yml would look like:

name: Test, Build, and Push Docker Image
on:
  push:
    branches: ['main']

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Run tests in Docker
        run: |
          # Build the image only up to the 'test' stage.
          # The build will fail if the tests fail.
          docker build --target test -t my-app:test .
  
  build-and-push:
    # This job will only run if the 'test' job completes successfully
    needs: test
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3

      - name: Login to Docker Hub
        uses: docker/login-action@v3
        with:
          username: ${{ secrets.DOCKERHUB_USERNAME }}
          password: ${{ secrets.DOCKERHUB_TOKEN }}

      - name: Build and push with cache
        uses: docker/build-push-action@v5
        with:
          context: .
          push: true
          tags: your-dockerhub-username/my-app:latest
          platforms: linux/amd64,linux/arm64
          cache-from: type=registry,ref=your-dockerhub-username/my-app:buildcache
          cache-to: type=registry,ref=your-dockerhub-username/my-app:buildcache,mode=max

With this setup, every push to main first triggers the test job. If it fails, the pipeline stops. If it succeeds, the build-and-push job runs, benefiting from the layers that were already built and cached during the test job!

Conclusion

You've now integrated a crucial validation step into your automated pipeline, making it significantly more robust.

Key Takeaways:

• Isolate with Stages: Use a dedicated test stage in your multi-stage Dockerfile to create a self-contained testing environment.
• Fail Fast with RUN: Always use the RUN instruction to execute tests so that a test failure stops the image build.
• Target and Extract in CI: Use docker build --target <stage> in your CI pipeline to run only the tests, and use docker cp to extract test reports or coverage data for analysis.
• Chain Your CI Jobs: Make your build-and-push job dependent on the successful completion of your test job to ensure only validated code is deployed.

In our next lesson, we will focus on tagging and versioning images based on CI/CD workflow triggers. Now that we can automatically build and test our images, the final step is to give them meaningful tags (e.g., based on Git tags or branches) before pushing them to the registry.