Pyroscope

Pyroscope is Grafana’s continuous profiling backend. Nexus queries it to see where your services were spending CPU and allocating memory around the time of an incident, broken down by the functions and call paths responsible.

​

What we support

Profiling answers a question logs, metrics, and traces can’t. A metric tells you CPU hit 90%, a trace tells you which request was slow, but neither tells you which function inside the process was burning the cycles. Pyroscope does: it samples your running services continuously, so Nexus can look back at the incident window and see exactly which code paths were responsible. Nexus queries Pyroscope by profile type and a label selector, for example the CPU profile for {service_name="api"} over the incident window. Profile types depend on what your services emit, and commonly include:

• CPU: where processes spent their time on the CPU.
• Memory: allocations and in-use memory, by where they were allocated.
• Goroutines and other types: concurrency and other resource profiles, where your services produce them.

​

Reading a flame graph as findings

A raw profile is a flame graph: thousands of stack frames that are hard to read without clicking around. Nexus turns one into two things you can act on:

• Top functions: the functions that consumed the most CPU or memory in their own right, ranked, with each one’s share of the total.
• The hottest path: the single heaviest call chain from entry point down to the leaf, so you can see not just which function was expensive but how the code reached it.

Alongside those summaries, Nexus reads the flame graph itself as a rendered image, the way an engineer scanning it would. Nexus learns the profile types, labels, and service names in your Pyroscope instance automatically, so it queries for profile types you actually emit and filters on labels that exist. How that works is covered in How telemetry works.

​

Connecting Pyroscope

Connect Grafana, and Nexus discovers Pyroscope automatically as one of the data sources behind it, using Grafana’s credentials, with nothing separate to configure. Pyroscope appears in your data source list once the Grafana connection finishes syncing. Discovered Pyroscope data sources start disabled, so you opt in deliberately: enable the ones your team uses from your telemetry settings.

​

Best practice

• Enable the Pyroscope data sources whose CPU and memory behavior you investigate, rather than every source available.
• Connect the Grafana dashboards your team uses for profiling. Nexus learns your real query patterns from them.

​

Related