Traces

The traces add-on collects distributed traces from your app and shows them in Grafana. Use it when “this request was slow” needs to become “this request was slow because of this downstream call to this service.”

Attach it

watasu addons:create traces --app my-app

If this is your first observability add-on, Grafana is provisioned automatically.

What your app needs to do

Tracing requires your app to emit spans. Use a standard tracing library — OpenTelemetry is the recommended path:

instrument the inbound request boundary (your HTTP framework or RPC server)
propagate context across outgoing calls (HTTP clients, RPC calls, queue producers)
let the auto-instrumentation libraries cover database and cache calls

You don’t need to configure an export destination. When the traces add-on attaches, your app receives:

Variable	Value
`OTEL_EXPORTER_OTLP_ENDPOINT`	The app’s private trace ingest endpoint
`OTEL_EXPORTER_OTLP_TRACES_ENDPOINT`	The same endpoint with the `/v1/traces` path
`OTEL_EXPORTER_OTLP_PROTOCOL`	`http/protobuf`

Standard OpenTelemetry SDKs read these automatically — instrument your app and the spans flow. Watasu collects, stores, and presents the traces in Grafana.

What you can do in Grafana

search by service name, operation, duration, or tags
inspect individual traces with the full call tree
compare latency distributions across operations
jump from a slow trace into the matching log lines (correlation works out of the box)

When traces help most

Traces pay off when:

your app talks to several services or has multiple process types
one inbound request fans out into many downstream calls
p99 latency matters and logs alone can’t explain it
you’re debugging a system, not a single binary

For a single-process app with two endpoints, traces are overkill. For a five-service mesh with queues in between, they’re indispensable.

Tips

instrument the main user-facing request path first, then expand
use service and operation names a human can read at 3am
make sure spans reflect actual work boundaries (one giant 200ms span tells you nothing)
use traces alongside logs and metrics — none of the three alone is enough