Java app OOMKilled in Kubernetes: why it happens and how to fix it

Your Spring Boot service restarts every few hours. kubectl get pods shows a climbing restart count, the reason is OOMKilled, and yet the heap graphs look perfectly healthy. This is one of the most common problems with Java on Kubernetes — and it is almost never fixed by simply raising -Xmx.

OOMKilled vs OutOfMemoryError

These are two different failures, and telling them apart is the first step:

• OutOfMemoryError is thrown by the JVM when the Java heap (or metaspace) is full. You get a Java stack trace in the logs.
• OOMKilled means the Linux kernel killed the whole process because the container used more memory than its limit. The JVM gets no chance to log anything — the container just disappears with exit code 137 (killed by signal 9).

kubectl describe pod orders-7d9f8b6c4-x2klm
#   Last State:  Terminated
#     Reason:    OOMKilled
#     Exit Code: 137

If you see OOMKilled and no OutOfMemoryError in the logs, the heap is not the problem — the total memory of the process is.

Why the heap looks fine

The container limit covers everything the JVM process uses, and the heap is only one part of it:

• Heap — your objects; capped by -Xmx or MaxRAMPercentage.
• Metaspace — class metadata; grows with the number of loaded classes (Spring apps load a lot).
• Thread stacks — typically around 1 MB reserved per thread; hundreds of threads add up.
• Direct buffers — off-heap memory used by NIO, Netty, Kafka clients and many HTTP clients.
• Code cache, GC and JIT structures, plus native memory allocated by libraries.

So a container with a 1 GiB limit and -Xmx1g is guaranteed to be killed eventually: the heap alone is allowed to fill the entire limit, leaving nothing for the rest.

How the JVM sizes itself in a container

Modern JVMs (JDK 10+, and JDK 8 from update 191) are container-aware: they read the container’s memory limit and base their defaults on it. By default the maximum heap is only 25% of the container limit — safe, but often wasteful, which tempts teams to set a large -Xmx by hand.

Newer clusters use cgroup v2. Older JDK builds don’t read cgroup v2 limits correctly and may size themselves from the node’s total memory instead — so run a current JDK patch release.

The fix: size memory as a whole

1. Size the heap as a share of the limit, not a fixed number. Around 70–75% for the heap usually leaves enough headroom for everything else. Lower it if you use many threads or a lot of direct memory.
2. Set the memory request equal to the limit, so the pod is scheduled where that memory really exists, instead of relying on a node that is already overcommitted.
3. Exit cleanly on heap exhaustion with -XX:+ExitOnOutOfMemoryError, so a broken JVM restarts instead of limping along.

containers:
  - name: orders
    image: registry.example.com/orders:1.4.2
    resources:
      requests:
        memory: "1Gi"
        cpu: "500m"
      limits:
        memory: "1Gi"
    env:
      - name: JAVA_TOOL_OPTIONS
        value: "-XX:MaxRAMPercentage=75 -XX:+ExitOnOutOfMemoryError"

JAVA_TOOL_OPTIONS is read by the JVM itself, so this works without changing your image’s entrypoint. If you build images with Cloud Native Buildpacks, their Java memory calculator already sets these flags from the container limit.

When it still gets killed: find the native memory

If the heap is capped and the container is still killed, something outside the heap is growing. Turn on Native Memory Tracking and compare snapshots over time:

# add to JAVA_TOOL_OPTIONS (small overhead, fine for diagnosis)
-XX:NativeMemoryTracking=summary

# then, inside the container
jcmd 1 VM.native_memory baseline
# ...wait while memory grows...
jcmd 1 VM.native_memory summary.diff

The category that keeps growing points at the cause:

• Thread — a thread leak or an oversized pool; cap pool sizes.
• Class (metaspace) — classloader leaks, often from redeploys or dynamic proxies; consider -XX:MaxMetaspaceSize to fail fast.
• Other / Internal — often direct buffers; cap them with -XX:MaxDirectMemorySize.

If the JVM’s own numbers stay flat while the container’s memory keeps climbing, suspect native allocation by libraries or glibc memory fragmentation. Setting the environment variable MALLOC_ARENA_MAX=2 is a common, low-risk fix for the latter.

What to monitor

• Container working set — container_memory_working_set_bytes in Prometheus is the number that matters for the memory limit; alert well before it reaches the limit.
• JVM heap and non-heap from Micrometer or Actuator, to see which part is growing.
• Restart count and last termination reason for every pod.

Checklist

1. Confirm it’s OOMKilled (exit code 137), not an OutOfMemoryError.
2. Remove any fixed -Xmx close to the container limit.
3. Set -XX:MaxRAMPercentage to around 70–75 and memory request equal to limit.
4. Run a current JDK patch release so cgroup limits are read correctly.
5. Still killed? Use Native Memory Tracking to find what’s growing outside the heap.
6. Alert on container working set, not just heap usage.