Lesson 1 of 4 · Free to read
Session lesson
Kubernetes Observability on Kind
Self-paced phase · ~8–12 focused hours across Lab A/B/C
Audience: Final-year students & early-career engineers
Prerequisites: obs-04-otel-collector; Kind installed
Why this matters (5 minutes)
Compose taught the signals. Kubernetes is where MNCs actually run them — ServiceMonitors, Operators, and burn-rate alerts as CRDs.
Learning outcomes
- Install kube-prometheus-stack on Kind
- Scrape workloads via ServiceMonitor/PodMonitor
- Inject auto-instrumentation with the OTel Operator
- Express burn-rate alerts as PrometheusRule YAML
Shared lab constants
| System | api (Express) → SNS → SQS → worker → S3 (+ Postgres/Redis from phase 07) |
| Stack | Grafana · Prometheus/Mimir · Loki · Tempo · Pyroscope · OTel Collector · Alloy |
| Rule | Extend the same repo every phase — never start a new toy app. |
What MNCs do in 2026
kube-prometheus-stack, OpenTelemetry Operator (auto-instrumentation injection), and PrometheusRule CRDs are the default golden path on internal platforms.
Teachable chunks (compressed from days 41–50)
From compose to Kind
Same services, now Deployments/Services; scrape via Kubernetes SD.
ServiceMonitor is the contract
App teams expose ports + labels; platform scrapes without forking prometheus.yml.
Operator injection
One annotation mutates Pods — but SNS→SQS propagation gaps still need humans.
Burn-rate as code
Multi-window multi-burn-rate alerts in PrometheusRule, reviewed in PRs.
Mental model
Watch & learn
Course video
Course video
Worked example
Annotate the api Deployment for Node auto-instrumentation. Confirm traces without rebuilding the image. Re-apply the async propagation fix from phase 03 — Operator does not magically fix messaging.
Hands-on path
Open Exercises next. Complete Lab A → B → C in order. Stretch is optional.
Phase checklist
- ServiceMonitor scrapes api
- Operator injection demonstrated
- Burn-rate PrometheusRule applied
Key takeaways
- Keep the same order pipeline — depth beats new toy apps
- Correlate signals before you SSH
- Write policies and catalogs your future on-call self can use
Credit
Curriculum themes inspired by the public Observability100 course (https://observability100.vercel.app/course/index.html); lessons and labs rewritten for AI-Tutor.
Observability phase lesson — Kubernetes Observability on Kind.