Tempo

Tempo is Grafana’s distributed tracing backend. Nexus queries it to follow a request across your services: which service handled it, where it slowed down, and where it errored.

​

What we support

A Tempo data source gives Nexus three capabilities, all queried with TraceQL: When Nexus retrieves a trace, you get it as a timeline: each span with its service, operation, duration, and status, nested under the span that called it. You can see where the request spent its time and where the failure started.

​

Aggregating over span attributes

Span attributes are rarely exposed as labeled metrics elsewhere: a customer ID, an endpoint, a SQL fingerprint, a status. TraceQL metrics let Nexus chart rates and latency quantiles grouped by those attributes directly. A question like “p95 latency on POST /v1/incidents in production” can be answered from traces even when no metric was set up to track it. Trace-derived metrics depend on Tempo’s metrics-generator; where it isn’t running, Nexus falls back to span search over the same period. Nexus learns the structure of your traces automatically: your services, the resource and span attributes you emit, and which of them are high-cardinality. That way, queries target attributes that actually exist. How that works is covered in How telemetry works.

​

Connecting Tempo

Connect Grafana, and Nexus discovers every Tempo data source behind it automatically, using Grafana’s own credentials, with nothing separate to configure. Discovered Tempo data sources start disabled, so you opt in deliberately: enable the ones your team uses from your telemetry settings. If an enabled Tempo data source holds no spans yet, Nexus sets it aside rather than run queries that can’t return anything. It starts using the source once spans arrive.

​

Best practice

• Enable the Tempo data sources your responders reach for during incidents, rather than every source available.
• If you want trace-derived metrics, run Tempo’s metrics-generator. Without it, Nexus can still search spans and retrieve traces, but can’t chart rates or latency quantiles.
• Connect the Grafana dashboards that query Tempo. Nexus learns your real trace query patterns from them, which makes Tempo queries more accurate.

​

Related