Lesson 2 of 4 · Free to read
Exercises
Exercises — Distributed Tracing & TraceQL
Watch & learn
Course video
Course video
Keep a lab notebook: hypothesis → command → result. Speed without evidence is not a skill.
How to use this file
- Work Lab A → B → C on the same
orders-labrepo - Prefer Docker Desktop; use Kind when the lab says so
- Definition of Done at the bottom is the phase gate
Lab A — SpanKinds + attributes
Time: 45–60 min
Goal: Waterfall shows SERVER/CLIENT/CONSUMER/INTERNAL correctly.
Setup
OTel SDK on api and worker.
Steps
- Review auto-instrumented spans for Express and AWS SDK.
- Add one business INTERNAL span for validation.
- Verify semantic convention attributes (http.route, messaging.system).
Success criteria
- Kinds look correct in Tempo UI
If stuck
Wrong kind usually means manual span started with defaults — set kind explicitly.
Lab B — Async propagation
Time: 60–90 min
Goal: One trace_id across api → worker → S3.
Setup
SNS→SQS pipeline from phase 01.
Steps
- Confirm propagator injects into SNS/SQS attributes.
- Worker extracts context before processing.
- Break propagation on purpose, observe two traces, then fix.
Success criteria
- Single trace spans the async hop
If stuck
Request MessageAttributeNames: ['All'] on receive.
Lab C — TraceQL mastery
Time: 45–60 min
Goal: Attribute + structural queries return real traces.
Setup
Sampled traces in Tempo.
Steps
- Run an attribute filter for http.status_code = 500.
- Run a structural query for descendant errors under
/orders. - Document sampling rate and its effect on empty results.
Snippet — TraceQL example
{ resource.service.name = "api" && name = "POST /orders" } >> { status = error }
Success criteria
- Two TraceQL queries saved in README
- Sampling caveat written
If stuck
Empty results? Check sampler and time range first.
Stretch goals
- Enable service graph metrics and open the Grafana service map
Definition of Done
- Async hop stays in one trace
- You can explain sampling vs “no data”
- At least one structural TraceQL query works
Exercises — Distributed Tracing & TraceQL.