CrashLoop: broken container command
Users report the latest go-api deploy never finished. New pods start, die instantly, and start again — classic CrashLoopBackOff. The old pods are still serving, so traffic mostly works… for now.
What the symptom looks like
Users report the latest go-api deploy never finished. New pods start, die instantly, and start again — classic CrashLoopBackOff. The old pods are still serving, so traffic mostly works… for now.
Affects go-api.
The hint ladder
Each hint narrows the search a little further. They are collapsed on purpose — open one only when you are properly stuck, because working out the next command yourself is the whole exercise.
Hint 1
Something changed about the go-api deployment recently. Start where you always should: `kubectl get pods -n go-api` and look at the STATUS and RESTARTS columns. What's different between the old and new pods?
Hint 2
A pod that dies instantly usually logs why — or dies too fast to log anything, which is itself a clue. Compare `kubectl logs -n go-api <crashing-pod>` (and `--previous`) with `kubectl describe pod` — check the container's Last State and exit code.
Hint 3
Exit code 1 with no app logs means the app never started. Inspect the pod *spec*, not its status: `kubectl get deploy go-api -n go-api -o yaml` and look at the container's `command`. Does that look like it belongs there? Remove the override and watch the rollout recover.
Related incidents
- Bad deploy: image that doesn't existA release went out referencing an image tag that was never pushed. The rollout is stuck — new pods sit in ImagePullBackOff while the old version keeps serving.
- Network blackhole: service unreachablego-api went dark. Every request through the ingress times out or 5xxes — yet the pods are Running and Ready, logs are quiet, and nothing was deployed. Something is eating the traffic.
- Noisy neighbor: CPU-hungry tenantLatency on go-api crept up and node CPU is pegged, but go-api itself didn't change. Someone else moved in — a workload with big CPU requests, no limits, and a hot loop.