Auto-scaling Deployments with HPA and CPU Utilization

1. The "Why" and "How" of Horizontal Pod Autoscaling

Manually adjusting the number of replicas for a service is not a viable strategy in a dynamic production environment. Traffic patterns change—think of an e-commerce site during a flash sale or a fintech platform at market open. We need a system that responds automatically.

Horizontal Pod Autoscaling (HPA) is Kubernetes' solution to this problem. It automatically increases or decreases the number of Pod replicas (scaling "out" or "in") based on observed metrics like CPU or memory utilization.

The mechanism behind HPA involves a few key components in a continuous control loop:

1. Metrics Server: This is a cluster-wide aggregator of resource usage data. It collects metrics from the kubelet on each node (which gets them from a container monitoring agent called cAdvisor) and exposes them through a Kubernetes API.
2. HPA Controller: This controller, running within the Kubernetes control plane, periodically checks the metrics from the Metrics Server.
3. HPA Object: This is a resource you define, specifying your scaling rules, such as the minimum/maximum number of replicas and the target metric value (e.g., "keep average CPU utilization at 70%").
4. Deployment/ReplicaSet: The HPA controller doesn't create pods directly. Instead, it updates the replicas field in the target object (like a Deployment), which then handles the creation or termination of Pods.

The following diagram provides a clear visual of this control loop.

To dive deeper into this architecture and the core scaling algorithm, the following video provides an excellent explanation.

The Scaling Algorithm in Detail

As the video explained, the HPA's decision-making is based on a simple ratio. For CPU utilization, the formula is:

This is where our last lesson becomes critical. The currentCPUUtilization is not an absolute value but is measured against the Pod's CPU request.

Let's say:

• Your Pod has requests.cpu: "500m" (0.5 cores).
• Your HPA has targetCPUUtilizationPercentage: 80.
• This means your actual target CPU usage per pod is 500m * 80% = 400m.
• If the current average usage across all Pods is 600m, the HPA controller calculates: desiredReplicas = ceil[currentReplicas * (600m / 400m)]. It will scale up.
• If the current average is 200m, the calculation becomes ceil[currentReplicas * (200m / 400m)], and it will scale down.

This direct link between requests and HPA targets is a key concept often probed in interviews.

2. Configuring HPA for a Spring Boot Application

Now, let's put theory into practice. We'll walk through the steps to enable HPA for a Spring Boot application. This requires three YAML files: a Deployment, a Service to expose it, and the HPA object itself.

The following article provides a clear, step-by-step guide with all the necessary manifests.

Let's break down the two most important manifests.

1. The Deployment Manifest (Prerequisite)

# From deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: spring-boot-hpa-app
spec:
  replicas: 1
  template:
    spec:
      containers:
      - name: spring-boot-hpa-app
        image: bansikah/spring-boot-hpa:latest
        ports:
        - containerPort: 8082
        resources:
          requests:
            cpu: "250m" # This is CRITICAL for HPA
            memory: "512Mi"
          limits:
            cpu: "500m"
            memory: "1Gi"

The most important line here is requests.cpu: "250m". Without a CPU request, the HPA has no baseline against which to calculate utilization percentage.

2. The HPA Manifest

# From hpa.yaml
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: spring-boot-hpa-app
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: spring-boot-hpa-app # Points to our Deployment
  minReplicas: 1
  maxReplicas: 5
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 50 # Target 50% of the requested 250m CPU

Here we define the scaling behavior:

• scaleTargetRef: Identifies the Deployment the HPA should manage.
• minReplicas & maxReplicas: Sets the lower and upper bounds for scaling.
• metrics: Specifies the metric to monitor. Here, we target an average CPU utilization of 50% across all pods.

To see this all come together in a live demonstration, let's return to the video from Pavan Elthepu.

3. Interview-Level Discussion: HPA Gotchas & Best Practices

Understanding how to configure an HPA is expected. A senior engineer is expected to understand why it might not behave as expected.

For a deeper dive into debugging and optimizing HPA behavior, the following article provides invaluable insights from a real-world scenario.

Let's summarize the key interview talking points from this material:

• Why isn't HPA scaling down immediately? Kubernetes uses a stabilization window (defaulting to 5 minutes for scaling down) to prevent "flapping"—rapidly scaling replicas up and down. It waits for a period to ensure the load reduction is stable before removing pods.
• Why did my pods scale up to the max on startup? A brief, high CPU spike during application startup can be enough to trigger the scaling algorithm. If currentCPU is much higher than targetCPU even for a short time, the formula can calculate a high desiredReplicas count. This highlights the importance of setting a realistic targetCPUUtilizationPercentage that accounts for normal bursts.
• Should I scale on Memory? For many applications, especially Java applications, memory usage is not directly proportional to traffic. The JVM claims a large heap on startup and keeps it. Scaling on memory can lead to a stable number of pods regardless of load. CPU utilization is almost always a better metric for scaling stateless, traffic-driven services.

Conclusion

Today you've learned how to make your Kubernetes deployments dynamically adapt to real-world demand. This is a powerful combination of resource management and automation that is essential for building robust, cloud-native systems.

Key Takeaways:

• HPA automatically scales the number of Pods based on metrics, with CPU utilization being the most common for stateless services.
• The HPA control loop relies on the Metrics Server to collect data and the HPA controller to make scaling decisions.
• The scaling behavior is dictated by the formula desiredReplicas = ceil[currentReplicas * (current / target)], where the CPU metric is calculated against the Pod's CPU request.
• For production, it's crucial to understand nuances like the scale-down stabilization window and to choose the right metric for scaling (typically CPU over memory).

Preview of the Next Lesson:
We are now at the end of the "Containerization & Kubernetes" module. You've learned how to package your application with Docker, and then deploy, configure, probe, and automatically scale it in Kubernetes.

The next module, "CI/CD & System Design Interview Practice," will focus on automating this entire process. In the first lesson, "Create a CI/CD pipeline using GitHub Actions to build, test, and package a Spring Boot application," you will learn how to automate the steps of turning your source code into a runnable artifact, laying the groundwork for automated deployments.