“OpenObserve helped us migrate from Datadog in under an hour... reducing observability costs by 4x.”
4x
Lower observability cost
< 1 hour
Migrated off Datadog
Flat pricing
No month-end surprises
Preview
Debug Your Agents Like You Debug Your Services.
OpenTelemetry-native. Every prompt, tool call and token.
Every prompt, tool call and response captured as a span - with token cost attached, scored for quality, and sitting in the same store as the logs, metrics and infrastructure underneath it.
trace 4f2a91c8·support-agent·14 spans·8.42s·$0.0412SLOW
spantimelinetokens
POST /chat-
↳ agent.router412
↳ retriever.pgvector-
↳ postgres · lock wait3.4s
↳ tool.lookup_order-
↳ llm.anthropic · sonnet6,204
41% of this trace was a Postgres lock wait, not the model. Purpose-built LLM tools don't carry that span.
span detail · llm.anthropic
gen_ai.input_messages
system: You are a support agent… user: where is order 88214
gen_ai.output_messages
Your order shipped Tuesday and is due…
cost_total
$0.0186
faithfulness
0.91
Session sess_9f31 · turn 4 of 6 · scored at session scope
Every Span Down to the Prompt
Read the exact prompt the model actually got, the response it gave back, and the tool call in between - at any depth of the execution graph.
Cost Per Call, Not Per Month
Input and output token cost is computed onto every span from your own model pricing - so the expensive step is a row you can sort, not a line on an invoice.
Prompts Stay on Your Infrastructure
Self-host or run in your own cloud, with redaction at ingest and field-level masking. Your prompts are your customers' data.
What Is LLM Observability?
LLM observability is the practice of capturing every LLM call, tool call and agent step in a production application as telemetry you can query. In OpenObserve it is OpenTelemetry: each call becomes a span carrying the prompt, the response, model parameters, token counts, computed cost and errors - so you can debug an AI application the same way you debug a service.
What Is Agent Observability?
Agent observability extends the same idea to multi-step and multi-agent systems. Rather than scoring one completion, it reassembles the whole run - planner, tool calls, retries and handoffs across services - into a single trace using W3C context propagation, then groups related traces into conversations by session ID.
How Is It Different From APM?
APM tells you a request took nine seconds. LLM observability tells you which prompt, retrieval step or tool call spent that time and what it cost in tokens. OpenObserve does both in one store, so an LLM span and the Postgres span beneath it share a trace ID.
Span, Trace or Session?
A span is one step - a single model call or tool call. A trace is one complete run assembled from its spans. A session is a whole conversation across many traces. Evaluations here can score at any of the three.
Inside the Product
From Execution Graph to Quality Score
One store, four views of the same run: reassemble what happened, watch score health across every config, run the jobs that produce those scores, and open the scorer that made the call.
Traces -> Agent Graph
planner.llm
agent.billing
2 tools · 1.1s
agent.shipping
4 retries · 5.2s
agent.refunds
skipped
synthesizer.llmresponse
The handoff that broke the run is a node you can click, not a log line you have to find. agent.shipping retried four times and burned 5.2 seconds of a 9-second answer.
Multi-Agent Runs, One Trace
Spans stitch across process boundaries with W3C context propagation. See which handoff introduced the hallucination, the timeout or the retry loop - by agent name, across every service the run touched.
Prompt-to-Response Analysis
Every span carries gen_ai.input_messages and gen_ai.output_messages. Diff what the model was given against what it produced to catch context contamination and prompt drift at the step where it started.
Session Capture Across Turns
Traces group into conversations on session_id or gen_ai.conversation.id. Replay a whole multi-turn conversation in order, with cost and quality rolled up per session - because the failure is usually turn 5, not turn 1.
LLM-Native Trace Search
Query across traces, spans, models, sessions and any request attribute to find the high-cost, high-latency or abnormal-output tail.
Only Here
Your Agent Wasn't Slow. Your Vector Database Was.
Purpose-built LLM tools can tell you the model took nine seconds. They cannot tell you why, because the Postgres lock, the pod eviction, the GPU throttle and the retry storm live in a different product. Here, the LLM span and the infrastructure span are rows in the same store - so "model problem or infra problem?" is one click, not one week.
Traces
LLM spans and service spans in one waterfall, one trace ID.
Logs
Jump from a failed completion to the application log that threw.
Metrics
Correlate token burn with GPU saturation, pod restarts and queue depth.
Kubernetes
The cluster the agent runs on, already instrumented, already retained.
Online EvaluationsEnterprise
Score Quality Where the Trace Already Lives
Continuously score production traces with LLM-as-a-judge using your own provider keys, or call your existing scoring service over HTTP. Scores land next to the cost and latency of the same span - no export, no second platform, no separate bill.
Span, Trace or Session Scope
Score a single completion, a whole run once it settles, or an entire conversation. Trace and session jobs wait for an idle window or a terminal span you nominate, then assemble the full payload.
Everything Is Versioned
Scorers and score configs carry a stable entity ID and a version number. Jobs pin to a version or track latest - so tightening a rubric never silently rewrites last quarter's numbers.
Sample, or Score Everything
Filter to the traffic that matters, evaluate 10% or 100%, and re-run a job manually against a known trace after changing a scorer.
Keep the Evaluator You Already Trust
Remote scorers call your own endpoint with retries and auth. Already using another eval tool? Send its scores in as span attributes and see quality beside cost, latency and infrastructure.
Qualitylast 24h · scope: session
Faithfulness
0.94
healthy ≥ 0.70
Relevance
0.88
v3 · latest
Tone
0.61
↓ 0.14 wk/wk
unhealthy runs
sess_9f31 · turn 50.42tone · v3
sess_a012 · turn 20.55tone · v3
trace 4f2a91c80.68faith · v2
Every row links back to the evaluator's own trace - the judge is observable too.
Cost
Token Cost Is a Span Attribute, Not a Monthly Surprise
Input, output and total cost are computed onto every LLM span from model pricing you define - matching your real billing agreements, including negotiated and self-hosted rates. Sort by it, group by it, charge back on it.
- Cost per request, per span, per session, per agent
- Configurable pricing per model and per provider
- Internal chargeback by team, feature or customer
Act on it
Alert on Token Burn. Page Someone. Open the Incident.
Most tracing tools stop at the dashboard. Alerts, SLOs and incident management are already part of this platform - so an AI regression runs the same response path as a production outage, and the on-call rotation you already have picks it up.
- Alerts on cost spikes, latency, error rate and eval-score drops
- SLOs on agent success rate and p95 answer time
- Incidents, on-call and AI SRE in the same platform
Integrations
Keep the Instrumentation You Already Have
Anything that speaks OpenTelemetry lands here - including the tools you might otherwise be choosing between us and. Browse the catalogue inside the product under Ingestion → AI Integrations, which hands you the endpoint, org and token for each one, then normalises every framework and provider onto one schema so cross-model debugging isn't six different field names.
Agent Frameworks
Trace orchestration, agent runs and tool calls at the framework level.
LangChain CrewAI LlamaIndex AutoGen
Model Providers
Token counts, model parameters, latency and errors per provider call.
OpenAI Anthropic Gemini Mistral Ollama
AI Gateways
Monitor routed traffic where the fan-out across models actually happens.
Portkey LiteLLM Proxy OpenRouter Kong AI
No-Code Builders
Observe the agents your team built without writing code - same spans, same cost.
n8n Flowise LangFlow OpenWebUI
Coding Agents
Native OTel metrics and events, plus optional per-turn tracing for the agents writing your code.
Claude Code GitHub Copilot
Tools, Vector Stores & MCP
Retrieval and eval tooling in the same trace - plus MCP, so your own agents can query this data in natural language.
Milvus Promptfoo Firecrawl MCP Server
Already sending traces to LangChain, LangFuse or another OTel collector?
Point the exporter here. No pipeline re-architecting, no re-instrumentation, and no requirement to move your prompts out of the tool your team already likes.
Setup
How Do I Instrument My LLM App?
Four steps, no application rewrite. If you already emit OpenTelemetry, it is step three onwards.
- 1
Install the SDK
Add the OpenObserve telemetry SDK and the instrumentation for your provider. It is a thin wrapper over the standard OpenTelemetry Python SDK - if you already use OpenTelemetry, skip it and export straight to the OTLP endpoint.
- 2
Instrument, Then Initialise
Call the instrumentor for your provider and initialise the SDK at your application entry point, before any model calls. Then use your LLM client exactly as before - spans are captured automatically.
- 3
Point the Exporter at Your Org
Take the endpoint, organisation and Base64 auth token from Ingestion → AI Integrations in the product. Set them as environment variables; no code changes required.
- 4
Open Traces
Filter by service, model or time range and click any span to inspect prompt, response, token counts and computed cost. Turn on model pricing to populate the cost fields, and add an eval job when you want quality scored.
Or Let a Coding Agent Do It
OpenObserve ships an MCP server. Connect it in Claude Code, Cursor or another MCP client and ask for the instrumentation in one prompt - the agent reads your stack and wires the exporter itself.
# 1 · add the MCP serverclaude mcp add openobserve --transport http \ https://api.openobserve.ai/mcp # 2 · one prompt> Add OpenObserve LLM observability to my projectAlready on Another Collector?
Anything OpenTelemetry-compatible can ship straight to the OTLP endpoint. There is nothing special about an LLM trace - the work happens server-side, so your existing exporter is enough.
Prompts Are the Most Sensitive Payload in Observability
A full-fidelity LLM trace contains customer PII, internal documents and source code. That is why this one runs in your cloud if you want it to - and why the answer to "where does the prompt go?" is a configuration you control, not a policy you have to accept.
Self-Host or Your Own Cloud
Open source, single binary. Prompts and responses never leave your infrastructure.
Redaction at Ingest
Mask or drop fields in the pipeline, before anything is written to disk.
Your Own Judge Keys
Evaluations call providers you configure, with credentials you hold and can rotate.
SOC 2 Type II · ISO 27001
Role-based access control down to individual evaluation resources.
Compare
Purpose-Built LLM Tool, APM Suite, or Both?
Where each category actually lands. We are strong on two axes the LLM-native tools cannot reach, and honest about the one where they lead.
| Capability | OpenObserve | Purpose-Built LLM Tools Langfuse, Arize, Braintrust | APM Suites Datadog, New Relic |
|---|---|---|---|
| Prompt and response per span Full payload capture on the LLM call | ✓ | ✓ | ✓ |
| Token cost attribution Input, output and total cost per span | ✓ | ✓ | ✓ |
| Session-scope evaluation Score a whole multi-turn conversation, not one turn | ✓ | ✓ | ~ |
| Versioned scorers and score configs Pin a rubric version so history stays comparable | ✓ | ✓ | ~ |
| Prompt management and playground Where the LLM-native tools genuinely lead | ~ | ✓ | ~ |
| Correlation with logs, metrics and Kubernetes The Postgres lock and the pod eviction in the same trace | ✓ | - | ✓ |
| Self-host the full product, open source Prompts never leave your infrastructure | ✓ | ~ | - |
| Alerts, SLOs and incident response built in An AI regression pages the on-call rotation you already have | ✓ | ~ | ✓ |
| Retention economics for full payloads Keep prompts for a year rather than a fortnight | ✓ | ~ | - |
✓ full support · ~ partial or add-on · - not available. Category-level summary as of August 2026; individual products vary, and evaluation features on OpenObserve are an enterprise capability.
Benchmarks
LLM Traces Are the Heaviest Telemetry You Own. Price Them That Way
A full prompt-and-response payload is kilobytes per span, and this category bills by span or by gigabyte with a fortnight or a month of retention. That is why teams sample - and then can't investigate the incident that only shows up in the tail. Every number below is measured against a named vendor, not an industry average.
vs. Elastic
140×
storage efficiency
Keep full prompts and responses for a year instead of a fortnight.
Read case studyvs. Datadog
8×
more cost-effective
AI observability and the rest of your stack on one bill, not two.
Read case studyvs. Grafana stack
5–15×
faster queries
Search a month of agent traffic in milliseconds, not minutes.
Read case study
Comparing us to a purpose-built LLM tool instead? See how much you would save switching today.
Testimonials
What Teams Say Running OpenObserve
Correlate between environments and signals across various sources with OpenObserve's built-in correlation engine.
LLM & Agent Observability FAQs
Ready to Get Started?
Point your OTel exporter here and watch the first trace land. No re-instrumentation, no pipeline rebuild, no migrating your prompts.
