SnowOps Labs builds a production-shaped Kubernetes cluster on your laptop, breaks it the way real clusters break, and checks whether your fix actually worked. Real Prometheus, real Istio, real failures. Free and open source.
For engineers who passed the CKA and still freeze when the pager goes off.
Watch the setup videoBuild a realistic platform, run a scenario on it, break it on purpose, and find out how you did. One CLI, labctl, drives all four.
One command creates a local k3d cluster and installs a real platform on it: Traefik for ingress, Prometheus and Grafana for metrics. Add logs, traces, GitOps, a service mesh, secrets or autoscaling when a scenario needs them.
labctl initThese are the real faults from the lab, with the same descriptions and the same hints labctl gives you. Start the clock and see how far you get before you need one.
What you are told
A release went out referencing an image tag that was never pushed. The rollout is stuck - new pods sit in ImagePullBackOff while the old version keeps serving.
This is a dry run in the browser, not a live cluster. In the lab the fault breaks a running app and a check against the cluster decides whether you fixed it.
Scenarios with checks that grade you, reversible faults with hints, and timed challenges with par times. All of it is YAML and shell in the repo, so you can read exactly what each one does.
13 scenarios
Watch KEDA scale go-api on Prometheus RPS: a spike drives it from 1 to several replicas, then cooldown brings it back. The flagship 'autoscaling actually works' demo, verified under traffic from the load generator.
labctl scenario up autoscaling-under-load1 stageSourceRun the chaos engineering loop against go-api with Chaos Mesh: establish a steady state under load, form a hypothesis, inject a failure (pod kill, network delay, CPU and memory stress) and measure the blast radius on a Grafana dashboard. go-api ships as a single replica behind a PodDisruptionBudget, so it starts out with disruptionsAllowed=0: a node drain would block forever while a pod kill is still a full outage. Your job is to find that with an experiment and fix it, then prove the same experiment no longer hurts.
labctl scenario up chaos-engineeringSourcego-api arrives reserving 2000m CPU and 1Gi of memory - forty times what it uses. Drive real traffic, read the gap between reserved and used on the dashboard and in OpenCost, then right-size the requests. Grading has two sides: a request over the waste ceiling fails, and so does one under the peak the workload actually reached, because under-provisioning is how workloads get throttled and OOM-killed.
labctl scenario up cost-right-sizing1 stageSourceRehearse the loop every operator needs before they need it. Capture a namespace to a manifest archive with 'kubectl get -o json | jq', delete something that matters, and put it back with 'kubectl apply --server-side'. Then find the edge of what that buys you: a data-writer holds a boot-id on a PersistentVolume, and restoring its claim gives you back the object bound to a brand-new empty disk. The objects round-trip; the bytes do not. A dependency-light way to learn where a manifest backup stops and a volume snapshot has to start.
labctl scenario up backup-restore-drill2 stagesSourceTake a worker node out of service while go-api serves live traffic. labctl gives the workload enough replicas to survive; you write the PodDisruptionBudget, cordon and drain the node with kubectl, watch the pods reschedule, and put the node back. The grade is the measured request success rate across the window.
labctl scenario up node-drain-drill1 stageSourceRoll the cluster's worker nodes onto a new Kubernetes version one at a time while go-api serves live traffic. You write the PodDisruptionBudget, you cordon and drain each node, and you watch the PDB block the eviction that would have taken the app down. The grade is the measured request success rate across the upgrade window and a cluster left with no node behind.
labctl scenario up cluster-upgrade-drill1 stageSourceAn 'orders' topic on Kafka (Strimzi) with a producer/consumer flow already running, and producers ramped to five times what the consumer group can process. Orders are piling up. Diagnose the backlog from Kafka's own tooling and from the lag dashboard, then restore throughput by scaling the group up to the topic's partition count, and no further, because a fourth consumer gets no partition and adds nothing.
labctl scenario up event-driven-arch2 stagesSourceRun a real GitOps loop against a Git repository you can actually push to. The lab serves its own repo in-cluster; ArgoCD reconciles it. You clone it, change the declared image and replica count, push, and watch the cluster follow, then prove the three things that make GitOps different from kubectl: self-heal reverts a change you make by hand, prune deletes a resource you remove from Git, and a release that syncs perfectly can still be broken - the lab ships one that is, and you repair it in Git.
labctl scenario up gitops-cicd4 stagesSourceRun a canary release for go-api on Istio. labctl stages two versions behind one Service and a meshed load client; you write the routing. Shift a 90/10 weighted split, rehearse a slow release with a header-matched latency fault, and lock the canary down with STRICT mTLS, then prove each one from mesh telemetry rather than from the fact that a CR exists.
labctl scenario up mesh-traffic-management3 stagesSourceRun a real release pipeline by hand: build versioned go-api images, deploy one to dev, and promote it forward to staging then prod with plain kubectl. Promotion here is a genuine image rollout (new ReplicaSet), not a ConfigMap edit - you build vNEXT, roll it out per environment, and watch each env's /version change as the release moves down the pipeline.
labctl scenario up env-promotion4 stagesSourcelabctl stands up the three signals (Loki for logs, Tempo behind an Alloy collector for traces, Prometheus rules and SLO dashboards for metrics) and then leaves the wiring to you. Export go-api's spans through the collector, drive real load with k6, follow one request from a log line to its trace, and take an SRE alert from inactive to firing by breaking readiness on purpose.
labctl scenario up observability-sre3 stagesSourceA workload mounts a Vault-backed secret that External Secrets keeps in sync. You rotate the value in Vault; it must reach the running pod through ESO and the kubelet with no restart and no redeploy. The grade is what the running container reads from disk.
labctl scenario up secrets-management1 stageSourceTake a cluster from 'policies exist' to 'policies bite'. Kyverno ships four Pod Security and best-practice ClusterPolicies in Audit mode, so nothing is blocked and every violation lands in a policy report. Your job is the part a real team owns: read the report, remediate the workload at the source, promote the policies to Enforce, prove the admission webhook now rejects a non-compliant Pod, and issue a real TLS certificate from the lab CA with cert-manager - on a namespace that is default-deny in both directions.
labctl scenario up security-compliance4 stagesSourceThe same Helm charts teams run in production: Prometheus, Istio, Kafka, Argo CD. Faults hit real systems, so the dashboards you learn to read here are the ones you have at work.
autoscaling
KEDA
chaos
Chaos Mesh
cost
OpenCost
dashboard
Kubernetes Dashboard
data
KafkaPostgreSQL
gitops
Argo CD
ingresspick one
NginxTraefik
logging
Loki
meshpick one
IstioLinkerd
monitoring
GrafanaPrometheus
secrets
External SecretsVault
security
Network policiesKyvernoSealed Secretscert-manager
tracing
Tempo
Where a category says “pick one”, the lab runs one provider at a time, the same choice a platform team has to make. Swap the provider, run the scenario again, and compare with evidence instead of opinion.
labctl ui serves a web dashboard at localhost:3939. Nothing extra to install, and everything the CLI does is a click away.
Production is usually the only place these skills get tested, and it is the worst place to learn them.
Learn → scenario → incident
Work through a path and come out able to debug that kind of failure on your own.
Build → swap provider → compare
Run the same scenario on Istio and on Linkerd, and decide with evidence in a day instead of a month.
Break → measure
Surprise faults, a shared cluster and recorded time to resolve, without weeks of setup.
Challenge → leaderboard
Timed challenges graded by the cluster give a fair, repeatable read on incident response.
The open-source lab is the product today, and it stays free. A hosted version is something we want to build in the long run, once enough people use the lab.
Everything on this site runs on your own machine today. Free, open source, and yours to read and change.
The same scenarios on a cluster we run, opened in a browser. We'll build it once the open-source lab has a solid base of users, and what it looks like will depend on what they need.
The Go code runs, records and grades. The lab content itself is YAML and shell, so adding a scenario or a fault never means touching the engine.
Scenarios, faults, paths and checks are YAML plus shell scripts. If you can write YAML, you can add to the lab. No Go required.
Your first scenarioGood first issues are labelled, and the PR bar is written down, so you know what review will ask for before you start.
Good first issuesThe docs live next to the code and this site publishes them. Fix a line in the repo and it shows up here on the next sync.
Contributing guideIt's an early release and it moves fast. A bug report with your labctl doctor output and the command that failed helps a lot.
Open an issueFree and open source. Clone it, run the installer, and the first lab is up in about ten minutes.
git clone --branch stable https://github.com/sagar2395/snowopslabs.git
cd snowopslabs && ./install.sh
labctl initYou need Docker with at least 2 CPUs and 4 GB of memory and about 10 GB of free disk. On macOS, labctl init sets up colima at that size for you. Heavier scenarios such as Kafka or a service mesh want 6 GB.
New scenarios and faults, release notes, and news about hosted labs if and when there is some. A few emails a month at most.
We'll email you about new releases, new lab content and paid offerings such as hosted labs, courses and workshops. Unsubscribe any time.