Everything that ships with the lab. Scenarios stage a situation and grade you against it; incidents break something real and reversible; challenges put a clock on it.
Synced from the repository on 3 Sept 2026.
13 scenarios
Watch KEDA scale go-api on Prometheus RPS: a spike drives it from 1 to several replicas, then cooldown brings it back. The flagship 'autoscaling actually works' demo, verified under traffic from the load generator.
labctl scenario up autoscaling-under-load1 stageSourceFailure injection with Chaos Mesh — pod kills, network delays, CPU/memory stress, partitions. Correlate chaos experiments with application metrics via Grafana dashboard.
labctl scenario up chaos-engineeringSourceDeploy go-api with deliberately over-provisioned CPU and memory requests, observe the inflated cost in OpenCost, then right-size the requests to a realistic baseline. Checks verify that requests were reduced to the target thresholds while go-api stays healthy under traffic. Teaches the observe-measure-reduce loop that keeps platform costs rational.
labctl scenario up cost-right-sizing1 stageSourceBack up a namespace's resources to a manifest archive, simulate accidental data loss (delete a resource — or the whole namespace), then restore from the backup and verify the round-trip with checks. A stateful data-writer on a PersistentVolume makes the crucial lesson concrete: the manifest backup round-trips Kubernetes OBJECTS, not the DATA on a volume. A dependency-light alternative to Velero that teaches the backup/restore loop every operator must rehearse before they need it.
labctl scenario up backup-restore-drill2 stagesSourceCordon and drain a worker node while go-api serves live traffic. A PodDisruptionBudget keeps a minimum number of replicas available, so the scheduler reschedules pods onto the remaining nodes without dropping requests. The grade is the measured request success rate through the drain.
labctl scenario up node-drain-drill1 stageSourceRoll the cluster's worker nodes to a newer Kubernetes version one at a time while go-api serves live traffic. Each node is drained (respecting the PodDisruptionBudget) before it is replaced, so the application stays available throughout. The grade is the measured request success rate across the upgrade window.
labctl scenario up cluster-upgrade-drill1 stageSourceA producer/consumer flow through Kafka (Strimzi): an 'orders' topic with a continuous producer and a consumer group. Stage 2 ramps producers to build consumer lag — observe it, then scale consumers (manually or with KEDA's Kafka scaler) to drain it.
labctl scenario up event-driven-arch2 stagesSourceArgoCD-based GitOps deployment with Application CRDs for declarative app management.
labctl scenario up gitops-cicdSourceIstio canary release for go-api: a 90/10 weighted split between v1 and v2, mesh-level latency fault injection, and STRICT mTLS on the canary workload. Watch the split and faults in mesh telemetry.
labctl scenario up mesh-traffic-management2 stagesSourceRun a real release pipeline by hand: build versioned go-api images, deploy one to dev, and promote it forward to staging then prod with plain kubectl. Promotion here is a genuine image rollout (new ReplicaSet), not a ConfigMap edit — you build vNEXT, roll it out per environment, and watch each env's /version change as the release moves down the pipeline.
labctl scenario up env-promotion4 stagesSourceFull observability stack with log aggregation, distributed tracing, alerting rules, and SLO dashboards.
labctl scenario up observability-sre3 stagesSourcego-api consumes a Vault-backed secret synced by External Secrets. Stage 1 wires the sync and seeds a baseline value; stage 2 rotates it in Vault. A check verifies the new value propagated to the Kubernetes Secret with no redeploy.
labctl scenario up secrets-management2 stagesSourcePolicy enforcement with Kyverno, TLS certificates via cert-manager, encrypted secrets with Sealed Secrets, namespace network isolation, and security best-practice policies.
labctl scenario up security-complianceSourceOrdered modules that chain scenarios and incidents into a curriculum, each gated by a check you have to actually pass.
A hands-on tour of the lab: spin up a cluster, deploy the demo apps, enable observability, and face your first production incident. By the end you will have used kubectl, Helm, Prometheus, and the incident engine on a real Kubernetes cluster.
labctl runtime uplabctl app deploy go-apiobservability-sreservice-selector-brokenlabctl learn start kubernetes-foundationsPoint SNOWOPS_CONTENT_PATH at your content and it appears alongside the built-in catalog. No pack format, no registry, no fork — a deliberate design decision, written up in ADR-0008.