Resource Management for Pods

1. The Core Concepts: Requests vs. Limits

In Kubernetes, we manage a container's resource consumption using two primary parameters for both CPU and memory:

• Requests: This is the amount of resources that Kubernetes guarantees for your container. The Kubernetes scheduler uses this value to find a node with enough available capacity to run your Pod. Think of it as reserving a minimum amount of resources.
• Limits: This is the maximum amount of resources your container is allowed to use. The container runtime enforces this limit.

It's vital to understand the difference in how CPU and memory are handled:

• CPU is a compressible resource. If a container hits its CPU limit, it gets throttled, meaning its CPU access is restricted. The application slows down but continues to run.
• Memory is an incompressible resource. If a container tries to use more memory than its limit, it is terminated by the system. In Kubernetes, this results in an OOMKilled (Out of Memory Killed) error, and the container is restarted.

To understand the mechanics of how Kubernetes uses requests for planning and limits for enforcement, let's watch a segment from a CNCF talk that goes "under the hood."

As the video explained, these settings also determine a Pod's Quality of Service (QoS) class, which Kubernetes uses to decide which Pods to evict first when a node is under resource pressure:

• Guaranteed: requests are set equal to limits for all resources. These are the last Pods to be killed.
• Burstable: At least one resource has a request that is lower than its limit.
• BestEffort: No requests or limits are set. These are the first Pods to be killed.

2. Practical Consequences of Misconfiguration

Theory is important, but seeing the real-world impact of these settings is crucial. Let's explore the common failure modes you'll encounter in production and will almost certainly be asked about in an interview.

The following video provides excellent hands-on demonstrations of what happens when resource settings go wrong.

Let's summarize the key scenarios:

3. Production Best Practices for Spring Boot

Now, let's bridge the gap from general Kubernetes concepts to specific, actionable advice for your Java and Spring Boot applications. Given the unique behavior of the JVM, applying generic advice can be counterproductive.

Memory: Guarantee Your Resources

For Java applications, the memory footprint is relatively predictable once the application is warmed up. However, if the JVM needs more memory and it isn't available, the result is fatal.

Best Practice: For Java workloads, always set memory requests and limits to the same value.

This approach gives your Pod a Guaranteed QoS class. It prevents your Pod from being killed due to node memory pressure (as long as it stays within its limit) and provides a stable, predictable memory environment for the JVM.

CPU: Request, But Don't Limit

Unlike memory, CPU is compressible. A typical Spring Boot application has "bursty" CPU needs—it requires more CPU during startup, JIT compilation, or handling a spike in traffic than during steady-state operation.

Best Practice: Set a realistic CPU request based on your application's typical needs, but do not set a CPU limit.

This is a powerful, non-obvious strategy perfect for a senior-level interview discussion.

• Why set a request? It guarantees your application gets its fair share of CPU when the node is busy.
• Why not set a limit? A CPU limit artificially prevents your application from using idle CPU cycles on the node. By omitting the limit, your app can "burst" and use available CPU during its high-demand phases, improving startup time and response latency without being throttled.

The following article provides a fantastic, in-depth justification for these practices.

JVM Tuning: Making Java Container-Aware

Setting the container resources is only half the battle. You also need to ensure the JVM running inside the container respects these boundaries. Modern JDKs (10+) are container-aware by default (-XX:+UseContainerSupport), but there are still nuances.

1. Explicitly Set Active Processor Count: The way the JVM calculates the number of available CPUs can be unreliable inside a container. It might be based on CPU shares, which can fluctuate. To avoid ambiguity and ensure consistent performance of thread pools (like Fork/Join), it's best to set it explicitly.

   • Flag: -XX:ActiveProcessorCount=N

2. Manage Heap Size: You must allocate enough container memory for both the JVM heap and non-heap memory (metaspace, thread stacks, direct memory, etc.). A common rule of thumb is to allocate about 75-80% of the container's memory to the heap.

   • Flag: -XX:MaxRAMPercentage=75.0
   • Important: This flag lets the JVM calculate the max heap size based on the container's memory limit. Your container's memory limit must be high enough to accommodate the heap plus non-heap memory. For example, if your app needs 1.5GB of heap and 500MB of non-heap, you must set the container memory limit to at least 2Gi.

The following article provides more detail on these JVM flags and how they interact with Kubernetes.

A sample configuration in your deployment.yaml would look like this:

spec:
  template:
    spec:
      containers:
      - name: my-spring-app
        image: my-app:1.0.0
        env:
        - name: JAVA_TOOL_OPTIONS
          value: "-XX:ActiveProcessorCount=2 -XX:MaxRAMPercentage=75.0"
        resources:
          requests:
            cpu: "1000m"      # Request 1 CPU core
            memory: "2Gi"     # Request 2 GiB of memory
          limits:
            memory: "2Gi"     # Limit memory to the same value
            # No CPU limit is set, allowing the app to burst

Conclusion

You have now learned how to manage one of the most critical aspects of running applications in Kubernetes. Proper resource management is the foundation of a stable and efficient cluster.

Key Takeaways:

• Requests are for scheduling, Limits are for enforcement. They are fundamental to how Kubernetes allocates and protects resources.
• Exceeding a memory limit is fatal (OOMKilled), while exceeding a CPU limit leads to performance degradation (throttling).
• For Java/Spring Boot applications, a proven production strategy is to set memory requests equal to limits, and to set a CPU request but no CPU limit.
• Always complement your Kubernetes resource settings with container-aware JVM flags like -XX:MaxRAMPercentage and -XX:ActiveProcessorCount for a truly robust setup.

Preview of the Next Lesson:
Manually setting the replica count for a service is inflexible. What happens when traffic spikes? In our next lesson, "Explain how Horizontal Pod Autoscaling (HPA) works and configure it based on CPU utilization," we will build directly on today's concepts. You will learn how to configure Kubernetes to automatically scale the number of your Pods up or down based on their CPU utilization, which is measured against the very requests we configured today.