OpenObserve Docs
IntegrationAIGateways

Kong AI Gateway → OpenObserve

Updated View MarkdownOpen in ChatGPTOpen in Claude

Automatically capture token usage, latency, and model metadata for every request routed through Kong AI Gateway. Kong acts as a transparent OpenAI-compatible proxy, so the standard OpenAI instrumentor captures spans with no additional configuration.

Prerequisites

  • Python 3.9+
  • Kong Gateway running in DB-less mode with an OpenAI proxy route (Docker recommended)
  • An OpenObserve account (cloud or self-hosted)
  • Your OpenObserve organisation ID and Base64-encoded auth token
  • An OpenAI API key

Installation

pip install openobserve-python-sdk openinference-instrumentation-openai openai python-dotenv

Configuration

Start Kong in DB-less mode with a declarative config that proxies to OpenAI:

# kong.yml
_format_version: "3.0"

services:
  - name: openai-service
    url: https://api.openai.com
    routes:
      - name: openai-route
        paths:
          - /openai
        strip_path: true
docker run -d --name kong \
  -e KONG_DATABASE=off \
  -e KONG_DECLARATIVE_CONFIG=/kong/declarative/kong.yml \
  -e KONG_PROXY_LISTEN=0.0.0.0:8000 \
  -v $(pwd)/kong.yml:/kong/declarative/kong.yml \
  -p 8000:8000 \
  kong:latest

Create a .env file in your project root:

OPENOBSERVE_URL=https://api.openobserve.ai/
OPENOBSERVE_ORG=your_org_id
OPENOBSERVE_AUTH_TOKEN=Basic <your_base64_token>
OPENAI_API_KEY=your-openai-api-key
KONG_GATEWAY_URL=http://localhost:8000/openai/v1

Instrumentation

Call OpenAIInstrumentor().instrument() before creating the OpenAI client. Point the client at the Kong proxy URL and pass your OpenAI API key directly.

from dotenv import load_dotenv
load_dotenv()

from openinference.instrumentation.openai import OpenAIInstrumentor
from openobserve import openobserve_init

OpenAIInstrumentor().instrument()
openobserve_init()

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.environ["OPENAI_API_KEY"],
    base_url=os.environ.get("KONG_GATEWAY_URL", "http://localhost:8000/openai/v1"),
)

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "Explain distributed tracing in one sentence."}],
    max_tokens=20,
)
print(response.choices[0].message.content)

What Gets Captured

AttributeDescription
llm_systemopenai
llm_model_nameResolved model returned by the API (e.g. gpt-4o-mini-2024-07-18)
gen_ai_request_max_tokensmax_tokens value from the request
gen_ai_response_modelSame as llm_model_name
gen_ai_operation_namechat
llm_token_count_promptPrompt tokens consumed
llm_token_count_completionCompletion tokens returned
llm_token_count_totalTotal tokens consumed
llm_token_count_prompt_details_cache_readCached prompt tokens
llm_token_count_prompt_details_audioAudio prompt tokens
llm_token_count_completion_details_reasoningReasoning tokens
llm_token_count_completion_details_audioAudio completion tokens
gen_ai_usage_input_tokensInput tokens (mirrors prompt count)
gen_ai_usage_output_tokensOutput tokens (mirrors completion count)
gen_ai_usage_total_tokensTotal tokens
gen_ai_usage_cost_inputEstimated input cost in USD
gen_ai_usage_cost_outputEstimated output cost in USD
llm_invocation_parametersJSON string of model config sent with the request
openinference_span_kindLLM
operation_nameChatCompletion
input_mime_typeapplication/json
output_mime_typeapplication/json
durationEnd-to-end latency including Kong proxy overhead
span_statusOK on success, ERROR on failure

Viewing Traces

  1. Log in to OpenObserve and navigate to Traces
  2. Spans appear with operation_name: ChatCompletion and llm_system: openai
  3. Since Kong is a transparent proxy, spans look identical to direct OpenAI calls. Use service_name or a custom span attribute to tag Kong traffic separately
  4. Use duration to measure Kong proxy overhead versus direct API latency

Kong Gateway trace in OpenObserve, Waterfall view with the ChatCompletion span attributes

Next Steps

With Kong AI Gateway instrumented, every proxied request is recorded in OpenObserve. From here you can monitor latency per route, track token usage across consumers, and set alerts on error rates.

Read More

Was this page helpful?

On this page