Troubleshooting JVM Errors

1. StackOverflowError: When the Call Stack Runs Too Deep

Let's start with the simpler of the two errors. As we reviewed in the previous lesson, each thread in a Java application has its own stack. The stack is used to store frames for each method call, holding local variables and references to objects on the heap. It operates on a Last-In, First-Out (LIFO) basis.

A java.lang.StackOverflowError occurs when a thread's call stack grows so large that it exhausts the memory allocated to it.

Causes and Diagnosis

The overwhelming cause of a StackOverflowError is infinite or excessively deep recursion. This happens when a method calls itself without a proper termination condition (a "base case").

How to Diagnose:
The stack trace of the error is your primary tool. It will clearly show the same sequence of method calls repeating over and over again until the JVM gives up.

For example:

Exception in thread "main" java.lang.StackOverflowError
	at com.example.MyClass.recursiveMethod(MyClass.java:10)
	at com.example.MyClass.recursiveMethod(MyClass.java:10)
	at com.example.MyClass.recursiveMethod(MyClass.java:10)
	...

Solutions

1. Fix the Code (Primary Solution): The correct solution in almost all cases is to debug your code and fix the faulty logic. Find the recursive method and ensure it has a base case that will eventually be met.

2. Increase Stack Size (Use with Caution): In rare cases, an application might have a legitimate need for very deep recursion (e.g., traversing a very deep tree structure). If you've confirmed the logic is correct, you can increase the thread stack size using the -Xss JVM argument. For example, -Xss2m sets the stack size to 2 megabytes.

It's crucial to understand that increasing stack size without fixing a bug is just a temporary fix that masks the underlying problem and wastes memory.

public long factorial(int n) {
    if (n == 0) {
        return 1;
    }
    return n * factorial(n); // What's wrong here?
}

public long factorial(int n) {
    if (n < 0) { // Good practice to handle negative input
        throw new IllegalArgumentException("Factorial not defined for negative numbers");
    }
    if (n == 0) {
        return 1;
    }
    return n * factorial(n - 1); // Corrected recursive call
}

2. OutOfMemoryError: When the Heap is Tapped Out

Unlike the relatively straightforward StackOverflowError, java.lang.OutOfMemoryError (OOME) is a class of errors with multiple variations, each pointing to a different underlying problem.

For your interviews, you should be deeply familiar with the three most common types.

A. OutOfMemoryError: Java heap space

This is the most frequent OOME. It occurs when the JVM tries to allocate an object on the heap, but there isn't enough space, and the Garbage Collector cannot free up enough memory to satisfy the request.

As the guide explains, there are two main culprits:

1. Memory Leak: This is the more insidious problem. Objects that are no longer needed are still referenced somewhere in your application, preventing the GC from reclaiming them. Over time, these objects accumulate in the Old Generation until it's completely full.
2. Undersized Heap: The application genuinely needs more memory than the heap is configured to provide. This might happen due to a sudden spike in traffic or processing a particularly large data set.

The Diagnostic Process: A Step-by-Step Guide

This is the structured response interviewers are looking for.

Step 1: Gather Data Automatically
Configure your production JVMs with these essential flags:

• -XX:+HeapDumpOnOutOfMemoryError: This tells the JVM to automatically generate a heap dump file when an OOME occurs. A heap dump is a snapshot of all objects on the heap at that moment.
• -XX:HeapDumpPath=/path/to/dumps/: Specifies where to save the heap dump file.
• -Xlog:gc*:file=gc.log: Enables detailed GC logging to a file. This helps you analyze memory usage patterns leading up to the crash.

Step 2: Analyze the Heap Dump
This is the core of the investigation. The heap dump file, which can be several gigabytes, is analyzed using a tool like Eclipse Memory Analyzer Tool (MAT) or VisualVM.

When you open a heap dump in MAT, you're looking for:

• Leak Suspects Report: MAT can often automatically identify potential memory leaks.
• Dominator Tree: This view shows you which objects are "dominating" memory—that is, which objects are holding on to the largest amount of memory through their references. This is invaluable for tracing a leak back to its root.
• Histogram: Shows the number of instances and total size for each class, helping you spot unusual accumulations of certain object types.

Step 3: Correlate with GC Logs
Analyze the GC logs from the time leading up to the OOME.

• If you see the Old Generation usage steadily climbing over time, even after Full GCs, it strongly suggests a memory leak.
• If heap usage is generally stable but a sudden spike causes the crash, it might be an undersized heap or a specific, memory-intensive operation.

By combining heap dump analysis with GC logs, you can definitively determine the root cause and fix it—either by plugging the memory leak in your code or by appropriately tuning the heap size (-Xmx).

B. OutOfMemoryError: GC overhead limit exceeded

This is a specific variant of a heap space error. The JVM throws this if it's spending more than 98% of its time doing garbage collection and is recovering less than 2% of the heap. In short, the application is grinding to a halt, making virtually no progress because it's constantly trying—and failing—to free up memory.

The key takeaway is that this error is a symptom of a full heap. Treat its diagnosis exactly like you would for a Java heap space OOME.

C. OutOfMemoryError: unable to create new native thread

This error is fundamentally different because it's not about the Java heap. It's about native memory—the memory the OS gives to the JVM process itself.

The JVM throws this error when it asks the operating system to create a new thread, and the OS refuses.

Common Causes:

1. Process Thread Limit Reached: The operating system imposes a limit on the number of threads a single process can create. You may have hit this limit (check with ulimit -u on Linux).
2. System Memory Exhaustion: The OS is out of RAM and swap space to allocate the memory required for a new thread's stack. Each thread requires its own stack space (defined by -Xss), which is allocated from native memory.
3. Thread Leak: Your application is creating threads but never shutting them down properly. Over time, the thread count grows until it hits an OS limit.

Diagnosis:

• Monitor Thread Count: Use tools like jstack <pid> to take a thread dump and count the number of active threads. jstack <pid> | wc -l gives a quick count. Is the number abnormally high or constantly growing?
• Check OS Limits: Consult your system administrator or use ulimit -a to check the maximum user processes allowed.
• Monitor System Memory: Use OS-level tools (top, free, htop) to check the overall memory usage on the machine. Is another process consuming all the memory?

The solution usually involves fixing a thread leak in the code, increasing OS-level limits, or reducing the thread stack size (-Xss) if it's unnecessarily large.

Conclusion

Mastering JVM error diagnosis is a hallmark of a senior engineer. It demonstrates a deep understanding of how the JVM interacts with the underlying system and shows you can handle high-pressure production issues methodically.

Key Takeaways:

• StackOverflowError is a stack issue, usually from infinite recursion. The fix is almost always in the code.
• OutOfMemoryError: Java heap space is a heap issue, caused by memory leaks or an undersized heap. Your primary diagnostic tools are heap dumps and GC logs.
• OutOfMemoryError: unable to create new native thread is a native memory issue, caused by OS limits or overall system memory exhaustion. Diagnose with thread dumps and OS-level monitoring.
• A Structured Approach is Key: For any OutOfMemoryError, the process is the same: enable logging (-XX:+HeapDumpOnOutOfMemoryError), collect the artifacts (dump files, logs), analyze them with the right tools (MAT, jstack), and form a hypothesis to guide your fix.

In our next lesson, we'll shift back to a core Java concept and explore the difference between primitive types and wrapper classes, including the subtle but important performance implications of autoboxing.