OOMKilled: memory limit too low
echo-server pods keep restarting and the service is flapping. Restart counts climb; the kernel is killing the container the moment it allocates memory.
What the symptom looks like
echo-server pods keep restarting and the service is flapping. Restart counts climb; the kernel is killing the container the moment it allocates memory.
Affects echo-server.
The hint ladder
Each hint narrows the search a little further. They are collapsed on purpose — open one only when you are properly stuck, because working out the next command yourself is the whole exercise.
Hint 1
The pods are restarting on a loop. `kubectl get pods -n echo-server` shows climbing RESTARTS — but *why* are they dying? `kubectl describe pod` and read the container's **Last State** carefully.
Hint 2
`Last State: Terminated, Reason: OOMKilled, Exit Code: 137`. The kernel is killing the container for exceeding its memory limit. So what *is* the limit? Check the deployment's `resources` block.
Hint 3
A 16Mi limit can't even hold a Go runtime. Compare with what the app actually needs (Grafana's container memory panels, or `kubectl top pod -n echo-server` on a healthy replica) and raise it: `kubectl -n echo-server set resources deploy/echo-server --limits=memory=256Mi --requests=memory=64Mi`.
Further reading
- Kubernetes — Assign memory resources to containersrequests vs limits, and what the kernel does when a container exceeds its memory limit.
- Kubernetes — Debug an OOMKilled container / exit code 137
- Prometheus — kube-state-metrics container restart & OOM signals
Related incidents
- Noisy neighbor: CPU-hungry tenantLatency on go-api crept up and node CPU is pegged, but go-api itself didn't change. Someone else moved in — a workload with big CPU requests, no limits, and a hot loop.
- Bad deploy: image that doesn't existA release went out referencing an image tag that was never pushed. The rollout is stuck — new pods sit in ImagePullBackOff while the old version keeps serving.
- CrashLoop: broken container commandUsers report the latest go-api deploy never finished. New pods start, die instantly, and start again — classic CrashLoopBackOff. The old pods are still serving, so traffic mostly works… for now.