Data Persistence with Docker Volumes and Bind Mounts

1. The Problem: The Ephemeral Container Filesystem

First, let's experience the problem firsthand. As we've discussed, when you run a container, Docker creates a thin, writable layer on top of the read-only image layers. This writable layer is where all changes inside the container are stored. However, this layer is tied directly to the life of that specific container.

Let's run a quick experiment:

1. Start a container and create a file:

   docker run -it --name test-container ubuntu bash
   

   You are now inside the container's shell. Let's create a file.

   # Inside the container
   echo "This data is important!" > /data.txt
   cat /data.txt
   exit
   

   You should see the content of your file printed. Now you're back on your host machine. The test-container is stopped but still exists.

2. Remove the container and check for the data:

   # On your host machine
   docker rm test-container
   

   The container is now gone. Let's start a new one from the same image and look for our file.

   docker run -it --name another-container ubuntu bash
   

   Once inside, try to view the file:

   # Inside the new container
   cat /data.txt
   exit
   

   You'll get an error: cat: /data.txt: No such file or directory. The data is gone because it only existed in the writable layer of the first container, which we deleted.

This behavior is fine for stateless applications, but for anything that needs to maintain state (like a database or a web app with user uploads), we need a better solution.

2. The Solution: Docker Mounts

Docker solves the data persistence problem with mounts. A mount allows you to connect a storage location on the host machine to a directory inside the container. This means data written to that directory by the container is actually being saved on the host, completely independent of the container's lifecycle.

There are three main types of mounts in Docker:

• Volumes: Storage is fully managed by Docker. This is the preferred method for persisting data in production.
• Bind Mounts: You map a specific file or directory from your host's filesystem into the container. This is very useful for development.
• tmpfs Mounts: Data is stored in the host's memory and is not written to the filesystem. This is temporary and non-persistent.

Let's dive into the two most important types for data persistence: volumes and bind mounts.

3. Docker Volumes: The Production-Ready Choice

Volumes are the recommended way to persist data generated by and used by Docker containers. A volume is a directory on the host that is entirely managed by Docker.

Key advantages of volumes:

• Managed by Docker: You don't have to worry about the exact location on the host. Docker handles it for you, making your setup cleaner and more portable.
• Decoupled from Container Lifecycle: Volumes persist even after the containers using them are deleted. You can safely remove and recreate a container and re-attach the same volume to it.
• Safe to Share: Multiple containers can safely use the same volume simultaneously.
• Better Performance: On macOS and Windows, volumes have significantly better performance than bind mounts because they exist within the Linux VM that Docker uses.
• Work with Volume Drivers: You can use volume drivers to store data on remote hosts or cloud providers.

Hands-On: Using a Named Volume

Let's repeat our earlier experiment, but this time with a volume.

1. Create a named volume:

   docker volume create my-app-data
   

   You can see your new volume by running docker volume ls.

2. Run a container with the volume mounted:
   We'll mount our my-app-data volume to a directory called /data inside the container. We can use either the -v flag (shorter) or the --mount flag (more explicit).

   Using --mount (recommended for clarity):

   docker run -it --name app1 --mount source=my-app-data,target=/data ubuntu bash
   

   Using -v:

   docker run -it --name app1 -v my-app-data:/data ubuntu bash
   

   Both commands achieve the same result.

3. Create data in the mounted directory:
   Now that you're in the container, create a file inside the /data directory.

   # Inside the container
   echo "This data will survive!" > /data/important.txt
   cat /data/important.txt
   exit
   

4. Remove the container and verify persistence:

   # On your host machine
   docker rm app1
   

   Now, let's launch a completely new container and mount the same volume.

   docker run -it --name app2 --mount source=my-app-data,target=/data ubuntu bash
   

5. Check for the data:

   # Inside the new container (app2)
   cat /data/important.txt
   

   Success! The file important.txt is still there. Your data has been persisted in the my-app-data volume, completely independent of the container.

Managing Volumes

You can manage volumes with a few simple commands:

• docker volume ls: List all volumes.
• docker volume inspect <volume_name>: Get detailed information about a volume, including its location on the host.
• docker volume rm <volume_name>: Remove a volume. You can only remove volumes that are not currently in use by a container.
• docker volume prune: Remove all unused local volumes to free up space.

4. Bind Mounts: The Developer's Friend

A bind mount maps a file or directory from your host machine directly into a container. You control the exact path on the host. This provides a direct, real-time link between your host and the container.

The primary use case for bind mounts is local development. Imagine you are editing your application's source code in your favorite IDE on your host machine. By bind-mounting the source code directory into your development container, any changes you save are immediately reflected inside the container, enabling features like hot-reloading.

Risks and limitations:

• Host-Dependent: Bind mounts rely on a specific directory structure on the host. This makes your setup less portable.
• Security Concerns: A container can modify the host filesystem, which can be a security risk, especially if the container is compromised or running as root.
• Performance Issues: On macOS and Windows, file synchronization between the host and the container can be slow.

Hands-On: Using a Bind Mount for Development

Let's simulate a development workflow with a simple nginx web server.

1. Create a project directory and an HTML file on your host:

   # On your host machine
   mkdir my-website
   cd my-website
   echo "<h1>Welcome to My Website! Version 1</h1>" > index.html
   

2. Run an nginx container with a bind mount:
   We will mount our current directory (my-website) into the directory where nginx serves files from (/usr/share/nginx/html). We'll use ${PWD} (on Linux/macOS) or %CD% (on Windows Command Prompt) to specify the current directory.

   Using --mount:

   docker run -d --name web-server -p 8080:80 --mount type=bind,source="${PWD}",target=/usr/share/nginx/html nginx
   

   Using -v:

   docker run -d --name web-server -p 8080:80 -v "${PWD}":/usr/share/nginx/html nginx
   

3. Verify in your browser:
   Open your web browser and go to http://localhost:8080. You should see "Welcome to My Website! Version 1".

4. Update the code on your host:
   Now, without stopping the container, edit the index.html file on your host machine.

   # In your my-website directory on the host
   echo "<h1>This is Version 2, updated live!</h1>" > index.html
   

5. Refresh your browser:
   Refresh http://localhost:8080. The page updates instantly to show "Version 2"! This is the power of bind mounts for development.

6. Clean up:

   docker stop web-server
   docker rm web-server
   cd ..
   rm -rf my-website
   

Conclusion

You now have the tools to build and run stateful applications with Docker. By separating data from the container's lifecycle, you can manage applications like databases, content management systems, and any service that needs to remember information across restarts and updates.

Key Takeaways:

• Container filesystems are ephemeral by default; data is lost when a container is removed.
• Docker volumes are the standard, production-safe way to persist container data. They are managed by Docker, portable, and performant.
• Bind mounts are ideal for development, allowing you to map a directory from your host directly into a container for real-time code changes.
• The choice between a volume and a bind mount depends on the use case: use volumes for application data and bind mounts for development source code or host-managed configuration files.
• The core commands to remember are docker volume create, docker volume rm, and using the -v or --mount flags with docker run.

Next Up: Tying It All Together with Docker Compose

We've learned how to build images with Dockerfile, connect containers with docker network, and persist data with docker volume. Managing all these separate commands for a multi-service application can become cumbersome. In our next lesson, we will be introduced to Docker Compose, a tool for defining and running multi-container Docker applications with a single, simple YAML file.