LLM Observability with OpenObserve

Monitor, trace, and debug your LLM-powered applications in production using OpenObserve and OpenTelemetry.

Note: LLM Observability is available on both OpenObserve Cloud and self-hosted deployments.

What is LLM Observability?

LLM Observability gives you visibility into the behaviour of large language model calls inside your application , similar to traditional APM, but purpose-built for AI workloads.

With it you can track:

• Token usage: prompt, completion, and total tokens per request
• Latency : end-to-end duration of every LLM call
• Model metadata : model name, temperature, max tokens, and other parameters
• Errors : rate limit events, API failures, and timeouts with full context

Under the hood: LLM Observability is OpenTelemetry. There is nothing special about LLM traces compared to regular distributed traces, the work happens on the server side. Any OpenTelemetry-compatible SDK or exporter can ship traces to OpenObserve.

Prerequisites

• Python 3.9+
• uv package manager (or pip)
• An OpenObserve account (cloud or self-hosted)
• Your OpenObserve organisation ID and auth token, copied from Data > Data sources > Custom > Traces > OpenTelemetry in the OpenObserve UI

Configuration

Go to Data > Data sources > Custom > Traces > OpenTelemetry in the OpenObserve UI and copy the endpoint and the Authorization header value under OTLP HTTP. OPENOBSERVE_URL is the part of the endpoint before /api/; the SDK adds /api/<org>/v1/traces itself.

Create a .env file in your project root:

# OpenObserve instance URL
# Default for self-hosted: http://localhost:5080
OPENOBSERVE_URL=https://api.openobserve.ai/

# Your OpenObserve organisation slug or ID
OPENOBSERVE_ORG=your_org_id

# Auth token: copy the Authorization header value from Data > Data sources > Custom > Traces > OpenTelemetry
OPENOBSERVE_AUTH_TOKEN="Basic <token_from_ui>"

# Enable or disable tracing (default: true)
OPENOBSERVE_ENABLED=true

# API keys for services you're using (optional, based on instrumentation)
OPENAI_API_KEY="your-openai-key"
ANTHROPIC_API_KEY="your-anthropic-key"

The OTEL Collector tab next to it shows the same endpoint and Authorization header in a collector configuration.

Option A : Quickstart with the bundled example

Clone the SDK repository and run the included OpenAI example to see traces flowing into OpenObserve with minimal setup.

1. Clone the repository

git clone https://github.com/openobserve/openobserve-python-sdk/
cd openobserve-python-sdk

2. Install dependencies

uv pip install openobserve-python-sdk openai opentelemetry-instrumentation-openai python-dotenv
uv pip install -r requirements.txt

3. Add your .env file to the project root (see Configuration above), including OPENAI_API_KEY.

4. Run the example

uv run examples/openai_example.py

Open Traces in OpenObserve to see the spans appear.

Option B: Integrate into your own project using the OpenObserve SDK

Use this if you want the simplest possible integration without cloning the repository.

1. Install dependencies

uv pip install openobserve-python-sdk opentelemetry-instrumentation-openai python-dotenv

2. Initialise the SDK at your application entry point and Use your LLM client as normal

Sample OpenAI Instrumentation:

from opentelemetry.instrumentation.openai import OpenAIInstrumentor
from openobserve import openobserve_init

# Initialize OpenObserve and instrument OpenAI
OpenAIInstrumentor().instrument()
openobserve_init()

from openai import OpenAI

# Use OpenAI as normal - traces are automatically captured
client = OpenAI()
response = client.chat.completions.create(
    model="gpt-3.5-turbo",
    messages=[{"role": "user", "content": "Hello!"}]
)
print(response.choices[0].message.content)

Sample Anthropic Instrumentation:

from opentelemetry.instrumentation.anthropic import AnthropicInstrumentor
from openobserve import openobserve_init

# Initialize OpenObserve and instrument Anthropic
AnthropicInstrumentor().instrument()
openobserve_init()

from anthropic import Anthropic

# Use Claude as normal - traces are automatically captured
client = Anthropic()
response = client.messages.create(
    model="claude-3-5-sonnet-20241022",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Hello!"}]
)
print(response.content[0].text)

Every call is now captured as a trace span and exported to OpenObserve.

Note: The openobserve-python-sdk is an optional thin wrapper around the standard OpenTelemetry Python SDK that simplifies setup and exporter configuration. If you already use OpenTelemetry, you can send telemetry directly to the OpenObserve OTLP endpoint without it.

What gets captured

The opentelemetry-instrumentation-openai library attaches request, response, and token attributes to each span. OpenObserve stores attribute names with dots replaced by underscores, for example:

See OpenAI (Python) for the full list.

In addition, OpenObserve computes and stores the following token and cost fields on each LLM span at ingestion:

The cost fields are filled from the span's own reported cost when it has one; otherwise Model Pricing calculates them from the model name and token counts (ZO_MODEL_PRICING_ENABLED, on by default). Spans with prompt caching also get gen_ai_usage_cache_read_input_tokens, gen_ai_usage_cache_creation_input_tokens, and matching gen_ai_usage_cost_cache_* fields.

Viewing traces in OpenObserve

1. Log in to your OpenObserve instance

2. Navigate to Traces in the left sidebar

3. Filter by service name, model, or time range

4. Click any span to inspect token counts, latency, and full request metadata

Troubleshooting

Traces are not appearing in OpenObserve

• Confirm OPENOBSERVE_ENABLED=true in your .env
• Check that OPENOBSERVE_URL is the base URL of your instance, without /api/<org>
• Verify OPENOBSERVE_AUTH_TOKEN matches the Authorization header value from Data > Data sources > Custom > Traces > OpenTelemetry (Basic <token>)
• Ensure the SDK or tracer provider is initialised before any LLM calls

ModuleNotFoundError: No module named 'dotenv'

• Install the correct package: uv pip install python-dotenv (not dotenv)

ModuleNotFoundError: No module named 'openobserve'

• Run: uv pip install openobserve-python-sdk

Read More

• OpenObserve Python SDK
• LLM Evaluations: Continuously score live LLM traces using LLM-as-a-judge or remote scorers.
• LLM Experiments: Run offline, batch evaluations against versioned datasets and compare results.