Everything that ships with the lab. Scenarios set up a situation and grade you on it. Incidents break something real, and you can always undo them. Challenges put a clock on it.
What ships with labctl v1.7.0.
13 scenarios
Watch KEDA scale go-api on Prometheus RPS: a spike drives it from 1 to several replicas, then cooldown brings it back. The flagship 'autoscaling actually works' demo, verified under traffic from the load generator.
labctl scenario up autoscaling-under-load1 stageSourceRun the chaos engineering loop against go-api with Chaos Mesh: establish a steady state under load, form a hypothesis, inject a failure (pod kill, network delay, CPU and memory stress) and measure the blast radius on a Grafana dashboard. go-api ships as a single replica behind a PodDisruptionBudget, so it starts out with disruptionsAllowed=0: a node drain would block forever while a pod kill is still a full outage. Your job is to find that with an experiment and fix it, then prove the same experiment no longer hurts.
labctl scenario up chaos-engineeringSourcego-api arrives reserving 2000m CPU and 1Gi of memory - forty times what it uses. Drive real traffic, read the gap between reserved and used on the dashboard and in OpenCost, then right-size the requests. Grading has two sides: a request over the waste ceiling fails, and so does one under the peak the workload actually reached, because under-provisioning is how workloads get throttled and OOM-killed.
labctl scenario up cost-right-sizing1 stageSourceRehearse the loop every operator needs before they need it. Capture a namespace to a manifest archive with 'kubectl get -o json | jq', delete something that matters, and put it back with 'kubectl apply --server-side'. Then find the edge of what that buys you: a data-writer holds a boot-id on a PersistentVolume, and restoring its claim gives you back the object bound to a brand-new empty disk. The objects round-trip; the bytes do not. A dependency-light way to learn where a manifest backup stops and a volume snapshot has to start.
labctl scenario up backup-restore-drill2 stagesSourceTake a worker node out of service while go-api serves live traffic. labctl gives the workload enough replicas to survive; you write the PodDisruptionBudget, cordon and drain the node with kubectl, watch the pods reschedule, and put the node back. The grade is the measured request success rate across the window.
labctl scenario up node-drain-drill1 stageSourceRoll the cluster's worker nodes onto a new Kubernetes version one at a time while go-api serves live traffic. You write the PodDisruptionBudget, you cordon and drain each node, and you watch the PDB block the eviction that would have taken the app down. The grade is the measured request success rate across the upgrade window and a cluster left with no node behind.
labctl scenario up cluster-upgrade-drill1 stageSourceAn 'orders' topic on Kafka (Strimzi) with a producer/consumer flow already running, and producers ramped to five times what the consumer group can process. Orders are piling up. Diagnose the backlog from Kafka's own tooling and from the lag dashboard, then restore throughput by scaling the group up to the topic's partition count, and no further, because a fourth consumer gets no partition and adds nothing.
labctl scenario up event-driven-arch2 stagesSourceRun a real GitOps loop against a Git repository you can actually push to. The lab serves its own repo in-cluster; ArgoCD reconciles it. You clone it, change the declared image and replica count, push, and watch the cluster follow, then prove the three things that make GitOps different from kubectl: self-heal reverts a change you make by hand, prune deletes a resource you remove from Git, and a release that syncs perfectly can still be broken - the lab ships one that is, and you repair it in Git.
labctl scenario up gitops-cicd4 stagesSourceRun a canary release for go-api on Istio. labctl stages two versions behind one Service and a meshed load client; you write the routing. Shift a 90/10 weighted split, rehearse a slow release with a header-matched latency fault, and lock the canary down with STRICT mTLS, then prove each one from mesh telemetry rather than from the fact that a CR exists.
labctl scenario up mesh-traffic-management3 stagesSourceRun a real release pipeline by hand: build versioned go-api images, deploy one to dev, and promote it forward to staging then prod with plain kubectl. Promotion here is a genuine image rollout (new ReplicaSet), not a ConfigMap edit - you build vNEXT, roll it out per environment, and watch each env's /version change as the release moves down the pipeline.
labctl scenario up env-promotion4 stagesSourcelabctl stands up the three signals (Loki for logs, Tempo behind an Alloy collector for traces, Prometheus rules and SLO dashboards for metrics) and then leaves the wiring to you. Export go-api's spans through the collector, drive real load with k6, follow one request from a log line to its trace, and take an SRE alert from inactive to firing by breaking readiness on purpose.
labctl scenario up observability-sre3 stagesSourceA workload mounts a Vault-backed secret that External Secrets keeps in sync. You rotate the value in Vault; it must reach the running pod through ESO and the kubelet with no restart and no redeploy. The grade is what the running container reads from disk.
labctl scenario up secrets-management1 stageSourceTake a cluster from 'policies exist' to 'policies bite'. Kyverno ships four Pod Security and best-practice ClusterPolicies in Audit mode, so nothing is blocked and every violation lands in a policy report. Your job is the part a real team owns: read the report, remediate the workload at the source, promote the policies to Enforce, prove the admission webhook now rejects a non-compliant Pod, and issue a real TLS certificate from the lab CA with cert-manager - on a namespace that is default-deny in both directions.
labctl scenario up security-compliance4 stagesSourceOrdered modules that chain scenarios and incidents together. Each one ends with a check you have to pass before you move on.
A hands-on tour of the lab, written for someone starting from no cluster at all: spin one up, deploy the demo apps, enable observability, and face your first production incident. By the end you will have used kubectl, Helm, Prometheus, and the incident engine on a real Kubernetes cluster. If you already have a lab running, the first two modules will pass immediately.
labctl runtime uplabctl app deploy go-apiobservability-sreservice-selector-brokenlabctl learn start kubernetes-foundationsPoint SNOWOPS_CONTENT_PATH at your content and it shows up next to the built-in catalog. No package format, no registry, and no fork to maintain.
New scenarios and faults, release notes, and news about hosted labs if and when there is some. A few emails a month at most.
We'll email you about new releases, new lab content and paid offerings such as hosted labs, courses and workshops. Unsubscribe any time.