CompCode Academy

Lesson 1 of 4 · Free to read

Session lesson

OpenTelemetry Collector Pipelines

Self-paced phase · ~8–12 focused hours across Lab A/B/C
Audience: Final-year students & early-career engineers
Prerequisites: obs-03-distributed-tracing

Why this matters (5 minutes)

Apps should speak OTLP to one local endpoint and forget backends. The Collector is the data plane MNCs use to batch, scrub PII, sample, and fan-out.

Learning outcomes

  • Configure receivers → processors → exporters pipelines
  • Apply batch, attributes, and filter processors
  • Use OTTL for light transforms and redaction
  • Explain auto-instrumentation floor vs manual spans

Shared lab constants

Systemapi (Express) → SNS → SQS → worker → S3 (+ Postgres/Redis from phase 07)
StackGrafana · Prometheus/Mimir · Loki · Tempo · Pyroscope · OTel Collector · Alloy
RuleExtend the same repo every phase — never start a new toy app.

What MNCs do in 2026

Agent + gateway Collector topologies, OTTL transforms, tail sampling, and PII redaction processors are standard — often alongside Grafana Alloy for scrape/profile duties.

Teachable chunks (compressed from days 31–40)

Why a collector

Decouple app deploy from backend choice; centralize retries, batching, and policy.

Processors that matter

batch, memory_limiter, attributes, filter, tail_sampling — order matters.

OTTL + redaction

Strip emails/tokens before export — compliance requirement in MNCs.

Tail sampling

Keep error and slow traces; drop boring success at the gateway.

Mental model

Watch & learn

Course video

Open on YouTube
OpenTelemetry Collector on Kubernetes — Collector agent/gateway pattern on a cluster — receivers to exporters.

Course video

Open on YouTube
OpenTelemetry Deep Dive (Big Tent) — Standards, tracing, and why OTel is the 2026 vendor-neutral data plane.

Worked example

Split config into agent (receive OTLP, forward) and gateway (tail_sampling + export). Redact user.email attributes. Confirm Tempo still receives error traces under load.

Hands-on path

Open Exercises next. Complete Lab A → B → C in order. Stretch is optional.

Phase checklist

  • Agent/gateway story explained in your own words
  • PII redaction proven
  • Tail sampling effect measured

Key takeaways

  • Keep the same order pipeline — depth beats new toy apps
  • Correlate signals before you SSH
  • Write policies and catalogs your future on-call self can use

Credit

Curriculum themes inspired by the public Observability100 course (https://observability100.vercel.app/course/index.html); lessons and labs rewritten for AI-Tutor.


Observability phase lesson — OpenTelemetry Collector Pipelines.

New free lessons and course updates. Unsubscribe from any email.