> ## Documentation Index
> Fetch the complete documentation index at: https://docs.incident.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Prometheus

> Query your Prometheus metrics to see how your services behaved during an incident.

Prometheus is a time-series metric store. Nexus queries it to see how your services behaved around the time of an incident: the error rates that climbed, the latency that crept up, the saturation that tipped a service over.

<Note>
  Prometheus is connected through [Grafana](/nexus/telemetry/grafana): connect Grafana, and Nexus discovers every
  Prometheus data source behind it automatically, with nothing separate to configure.
</Note>

## What we support

Nexus queries Prometheus with PromQL, its query language. It goes beyond reading a metric's raw value: PromQL turns counters and histograms into the rates and percentiles you actually reason about during an incident.

* **Rates from counters.** Counters only ever climb, so the raw number means little on its own. Nexus wraps them in `rate()` to ask the real question: how fast are requests failing right now, and was that different before the incident started.
* **Percentiles from histograms.** Latency lives in histogram buckets, not a single number. Nexus uses `histogram_quantile()` to pull out the p95 or p99 your responders care about, rather than an average that hides the tail.
* **Aggregations across dimensions.** With `sum by`, `avg by`, and `max by`, Nexus rolls a metric up to the dimension that matters: per service, per route, per namespace. That shows which slice of your fleet is misbehaving.

### Discovering your metrics and labels

Prometheus exposes thousands of metrics and labels, and a query that names the wrong one returns nothing. Rather than guess, Nexus reads what your Prometheus actually holds, focusing on what's been recently active: your metric names, their types, and the help text you've attached. It also reads your labels, with a sense of how many distinct values each one takes.

Types and help text depend on your backend serving metric metadata. Many Prometheus-compatible backends, such as VictoriaMetrics, Mimir, and Thanos, don't populate it. There Nexus works from the metric names alone.

Cardinality shapes how Nexus builds queries. Grouping by a low-cardinality label like `service` or `namespace` gives a readable breakdown; grouping by a high-cardinality one like `pod` or `instance` produces noise. Nexus learns which labels are which, so it groups on the ones that clarify and filters on the ones that would overwhelm.

Nexus learns this structure automatically: your metrics, their types, and your labels and their cardinality. How that works is covered in [How telemetry works](/nexus/telemetry/how-it-works#learning-your-stack).

## Connecting Prometheus

Connect [Grafana](/nexus/telemetry/grafana), and Nexus discovers every Prometheus data source behind it automatically, using Grafana's own credentials, with nothing separate to configure. That covers any backend Grafana fronts as a Prometheus data source: Amazon Managed Service for Prometheus, Thanos, Cortex, Mimir, and VictoriaMetrics all work the same way.

Discovered Prometheus data sources start disabled, so you opt in deliberately: enable the ones your team uses from your [telemetry settings](https://app.incident.io/~/nexus/telemetry).

If you run several Prometheus servers (per cluster, per region, or HA replicas), you can group their data sources into one logical view: add a **Prometheus aggregate** from the connect wizard and pick its members. A query against the aggregate reaches every member and merges the results, so a sum or a rate is one correct global answer rather than a per-server fragment.

## Best practice

* Connect the Grafana dashboards that query Prometheus. Nexus learns your real query patterns from them (which metrics matter, how they're filtered and grouped), which makes Prometheus queries more accurate.
* Enable the Prometheus data sources your responders reach for during incidents, rather than every source available.

## Related

<CardGroup cols={2}>
  <Card title="Grafana" icon="database" href="/nexus/telemetry/grafana">
    Connect Prometheus through Grafana.
  </Card>

  <Card title="How telemetry works" icon="magnifying-glass" href="/nexus/telemetry/how-it-works">
    How Nexus queries your metrics.
  </Card>
</CardGroup>
