Lesson 1 of 4 · Free to read
Session lesson
OpenTelemetry Collector Pipelines
Self-paced phase · ~8–12 focused hours across Lab A/B/C
Audience: Final-year students & early-career engineers
Prerequisites: obs-03-distributed-tracing
Why this matters (5 minutes)
Apps should speak OTLP to one local endpoint and forget backends. The Collector is the data plane MNCs use to batch, scrub PII, sample, and fan-out.
Learning outcomes
- Configure receivers → processors → exporters pipelines
- Apply batch, attributes, and filter processors
- Use OTTL for light transforms and redaction
- Explain auto-instrumentation floor vs manual spans
Shared lab constants
| System | api (Express) → SNS → SQS → worker → S3 (+ Postgres/Redis from phase 07) |
| Stack | Grafana · Prometheus/Mimir · Loki · Tempo · Pyroscope · OTel Collector · Alloy |
| Rule | Extend the same repo every phase — never start a new toy app. |
What MNCs do in 2026
Agent + gateway Collector topologies, OTTL transforms, tail sampling, and PII redaction processors are standard — often alongside Grafana Alloy for scrape/profile duties.
Teachable chunks (compressed from days 31–40)
Why a collector
Decouple app deploy from backend choice; centralize retries, batching, and policy.
Processors that matter
batch, memory_limiter, attributes, filter, tail_sampling — order matters.
OTTL + redaction
Strip emails/tokens before export — compliance requirement in MNCs.
Tail sampling
Keep error and slow traces; drop boring success at the gateway.
Mental model
Watch & learn
Course video
Course video
Worked example
Split config into agent (receive OTLP, forward) and gateway (tail_sampling + export). Redact user.email attributes. Confirm Tempo still receives error traces under load.
Hands-on path
Open Exercises next. Complete Lab A → B → C in order. Stretch is optional.
Phase checklist
- Agent/gateway story explained in your own words
- PII redaction proven
- Tail sampling effect measured
Key takeaways
- Keep the same order pipeline — depth beats new toy apps
- Correlate signals before you SSH
- Write policies and catalogs your future on-call self can use
Credit
Curriculum themes inspired by the public Observability100 course (https://observability100.vercel.app/course/index.html); lessons and labs rewritten for AI-Tutor.
Observability phase lesson — OpenTelemetry Collector Pipelines.