Blue/Green Deployment on EC2/ECS

1. The "Why" Behind Blue/Green Deployments

Before diving into the "how," let's establish the "why." In a traditional deployment, you might stop the old version of your application and start the new one, causing downtime. A rolling update improves this but can result in running two different versions of your code simultaneously, and rollbacks can be slow.

Blue/green deployments offer a powerful alternative. The core idea is to have two identical, independent production environments: "Blue" (the current live version) and "Green" (the new version).

To understand the trade-offs, it's useful to compare it with other common strategies.

As the article highlights, the key advantages of blue/green are:

• Zero Downtime: The traffic switch is instantaneous at the load balancer level.
• Instant Rollback: If something goes wrong with the green environment, you simply switch traffic back to the still-running blue environment.
• Isolated Testing: You can run a full suite of tests on the green environment with real infrastructure before it ever receives live traffic.

The main trade-off is the cost, as you temporarily run double the infrastructure during the deployment.

2. Blue/Green Deployments on EC2 with AWS CodeDeploy

Let's start with the classic implementation for applications running on EC2 instances. The key service here is AWS CodeDeploy, which automates the entire process.

The core components you'll work with are:

• Application Load Balancer (ALB): Routes traffic to one of two Target Groups.
• Two Target Groups: One for the blue environment, one for the green.
• Auto Scaling Group (ASG): Manages the EC2 instances for your application.
• CodeDeploy: Orchestrates the deployment, including provisioning a new ASG for the green environment and managing the ALB traffic shift.
• AppSpec file (appspec.yml): A configuration file that tells CodeDeploy how to deploy your application onto an instance and validate its health.

The following video from AWS gives a comprehensive walkthrough.

The appspec.yml and Lifecycle Hooks

The magic of the deployment on the instance itself is defined in the appspec.yml file. This file uses lifecycle hooks to run scripts at various stages of the deployment. For a DevOps Engineer, understanding these hooks is vital.

The ValidateService hook is your opportunity to run health checks, synthetic transactions, or any other tests to confirm the green environment is healthy before CodeDeploy shifts production traffic to it.

Finally, the deployment process itself involves CodeDeploy cloning your ASG, running through the lifecycle hooks on the new instances, and then, upon successful validation, instructing the ALB to shift traffic.

3. Blue/Green Deployments on Amazon ECS

For containerized applications, especially on AWS Fargate where you don't manage the underlying instances, the process is more streamlined. Amazon ECS has native support for blue/green deployments, also orchestrated by CodeDeploy.

The concepts are similar (ALB, two target groups), but the implementation differs. Instead of cloning an entire ASG, ECS manages two versions of your application within a single service using Task Sets.

With ECS, the blue/green process looks like this:

1. You have a single ECS service configured for CODE_DEPLOY deployments.
2. When you deploy a new version, CodeDeploy creates a new "green" task set within the service.
3. ECS launches the new tasks, which are registered with the "green" target group.
4. CodeDeploy can direct test traffic to the green environment via a test listener.
5. After optional tests and a "bake time" (a monitoring period), CodeDeploy swaps the production listener on the ALB to point to the green target group.
6. The old "blue" task set is scaled down and terminated.

This next video demonstrates the process and highlights key features like "bake time" and lifecycle hooks using Lambda.

The ECS approach is powerful because it's tightly integrated and abstracts away much of the underlying infrastructure management, which fits perfectly with the operational model of Fargate.

Behavioral Interview Preparation

Understanding advanced deployment strategies is a great way to showcase your expertise in an interview.

Question: "Describe a time you were responsible for improving the reliability of the deployment process for a critical application."

How to Answer using today's concepts:

• Situation: "At a previous role, our team managed a critical API service with high traffic. Our deployment process used rolling updates, but we frequently saw a spike in errors during the rollout window as two different API versions were running simultaneously. Furthermore, a full rollback was a slow, manual process that could take over 15 minutes, which was unacceptable for this tier-1 service."
• Task: "My goal was to implement a deployment strategy that would achieve zero downtime, eliminate mid-deployment errors, and provide a near-instantaneous rollback capability."
• Action: "I chose to implement a blue/green deployment strategy. Since the application was containerized and running on ECS, I leveraged the native CodeDeploy integration.
  1. I configured a second target group for our Application Load Balancer.
  2. I updated our ECS service's Terraform configuration to use the CODE_DEPLOY deployment controller and specified the blue and green target groups.
  3. I added a ValidateService Lambda hook that was triggered by CodeDeploy after the green environment was up. This Lambda function ran a small suite of automated API integration tests against the green target group's private endpoint.
  4. I also configured a 10-minute 'bake time' post-traffic-shift, giving our monitoring systems time to detect any subtle performance degradations before the old environment was terminated."
• Result: "This new process completely eliminated deployment-related errors. We reduced our rollback time to under 60 seconds—it was as simple as re-running the CodeDeploy action to revert the listener swap. This dramatically reduced the risk of releases and increased our team's confidence, allowing us to deploy more frequently."

Conclusion

Great job! You've now learned how to design and implement one of the most effective deployment strategies for modern, highly-available applications on both EC2 and ECS.

Key Takeaways:

• Blue/Green is a Risk-Reduction Strategy: Its primary purpose is to enable safe, zero-downtime releases with fast, reliable rollbacks.
• The Core is Traffic Switching: The mechanism always involves redirecting traffic at a load balancer from an old environment to a new one.
• EC2 vs. ECS Implementation: On EC2, you typically manage full Auto Scaling Groups orchestrated by CodeDeploy via an appspec.yml. On ECS, the process is more integrated, with CodeDeploy managing Task Sets within a single service.
• Validation is Key: A robust ValidateService step (whether a script or a Lambda function) is essential to ensure you don't switch traffic to a faulty environment.
• Cost is the Main Trade-off: The reliability and safety of blue/green deployments come at the cost of temporarily running double the infrastructure.

What's Next?

This lesson concludes our module on Advanced AWS for DevOps. You've covered a wide range of topics essential for a senior DevOps role, from event-driven automation and advanced IAM to compliance-as-code and now, sophisticated deployment strategies.

In our next lesson, we will begin a new module: Kubernetes Ecosystem and Operations. Building on your CKA knowledge, we will start with the most critical aspect of running Kubernetes in production: security. The first lesson will be: "Secure a Kubernetes cluster using a combination of RBAC, Network Policies, and Pod Security Standards."