Splunk Observability Cloud

Splunk Observability Cloud (formerly SignalFx) keeps your infrastructure, application, and APM metrics in a single store. Investigations query it to find the metric that moved around the time of an incident, and read your dashboards to learn what your team watches.

​

What we support

Connecting Splunk Observability Cloud gives investigations two capabilities:

​

Metrics

Infrastructure, custom, and APM metrics all live in one place in Splunk Observability Cloud, and investigations query all of it. They graph a metric for the incident’s time window, so a CPU saturation, a latency change, or a queue backing up shows up against the period that matters. Queries are grounded in the metrics your account actually reports. Investigations learn the metric names and dimensions that carry live data, rather than every name your account has ever registered, so they filter and group by dimensions you really have instead of guessing. How that works is covered in How telemetry works. If you send traces to Splunk APM, Splunk derives request, error, and duration metrics per service from them. Investigations read those to see which service’s error rate climbed or which one slowed down, broken down by service and environment, and take percentiles from the underlying distribution rather than an average that hides the slow tail.

​

Dashboards

Investigations read the dashboards in your account and the queries behind their charts. That tells them which metrics your team relies on, and how you filter and group them, so the metrics an investigation reaches for are the ones your team would reach for.

​

Logs

Splunk Observability Cloud has no log store of its own. Your logs live in Splunk Cloud Platform or Splunk Enterprise, and Log Observer Connect gives you a view onto them from the Observability Cloud interface: it queries your platform indexes in place, and doesn’t store or index anything itself. Wherever your logs are actually stored, check whether we have an integration for it, or get in touch and tell us what you need!

​

Connecting Splunk Observability Cloud

You connect Splunk Observability Cloud directly. There’s no provider in front of it. What you’ll need:

• An access token with the API authorization scope, created under Settings → Access Tokens → New Token. Choose API token as the scope: that’s the one that can read data back. The ingest tokens your OpenTelemetry Collector or Smart Agent uses to send data in can’t read it, and a connection made with one fails the connection test. You need to be an organization administrator to create a token. Splunk covers this in its access token documentation.
• An expiry you’ve set deliberately on that token. Splunk expires access tokens by default, and the connection stops working the day it does.
• Your realm: the deployment your organization is hosted on, such as us0, eu0, or au0. Find it under Settings → your name → Organizations. The realm has to match your organization, because Splunk doesn’t carry credentials across realms.

1. From the Investigations settings, add a telemetry data source and choose Splunk Observability Cloud.
2. Enter your access token and realm, then test the connection.

Connecting adds a metrics data source, which you can turn off if you don’t want investigations to query it. Dashboards are read through the connection itself.

​

Best practice

• Create a dedicated access token for the connection rather than reusing one, and give it a long expiry. An access token belongs to your organization rather than to a person, so the connection keeps working when people move teams.
• Leave the token’s SignalFlow job start limit unset, which is Splunk’s default. Investigations query in bursts, so a low cap rejects queries and leaves an investigation without the metrics half of its picture. If your policy requires a limit, you can find it under Settings → Access Tokens → the token’s actions menu → Manage limits.
• Select the dashboards that your responders use in your configuration. Investigations learn from the charts your team built, so a dashboard that reflects how you actually debug a service makes for better queries than one nobody looks at.

​

Related