OpenObserve Docs

OpenAI JS/TS → OpenObserve

Updated View MarkdownOpen in ChatGPTOpen in Claude

Capture LLM call latency, token usage, model name, and finish reason for every OpenAI API call made from a Node.js application. Instrumentation uses @arizeai/openinference-instrumentation-openai to automatically patch the OpenAI SDK and export spans to OpenObserve via OTLP.

Prerequisites

  • Node.js 18+
  • An OpenObserve account (cloud or self-hosted)
  • Your OpenObserve organisation ID and Base64-encoded auth token
  • An OpenAI API key

Installation

npm install openai @arizeai/openinference-instrumentation-openai \
  @opentelemetry/sdk-node @opentelemetry/exporter-trace-otlp-http \
  @opentelemetry/sdk-trace-base @opentelemetry/resources dotenv

Configuration

Create a .env file in your project root:

OPENAI_API_KEY=your-openai-api-key

Instrumentation

Initialise the OTel SDK with OpenAIInstrumentation before requiring the OpenAI SDK. The instrumentation patches the module at require() time, so OpenAI must be imported after sdk.start().

require('dotenv').config();

const { NodeSDK } = require('@opentelemetry/sdk-node');
const { OTLPTraceExporter } = require('@opentelemetry/exporter-trace-otlp-http');
const { SimpleSpanProcessor } = require('@opentelemetry/sdk-trace-base');
const { resourceFromAttributes } = require('@opentelemetry/resources');
const { OpenAIInstrumentation } = require('@arizeai/openinference-instrumentation-openai');

const sdk = new NodeSDK({
  resource: resourceFromAttributes({ 'service.name': 'my-openai-app' }),
  spanProcessors: [
    new SimpleSpanProcessor(
      new OTLPTraceExporter({
        url: 'https://api.openobserve.ai/api/your_org_id/v1/traces',
        headers: { Authorization: 'Basic <your_base64_token>' },
      })
    ),
  ],
  instrumentations: [new OpenAIInstrumentation()],
});
sdk.start();

const OpenAI = require('openai');

async function main() {
  const client = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

  const response = await client.chat.completions.create({
    model: 'gpt-4o-mini',
    messages: [{ role: 'user', content: 'What is distributed tracing?' }],
    max_tokens: 256,
  });

  console.log(response.choices[0].message.content);
  await sdk.shutdown();
}

main().catch(console.error);

For self-hosted OpenObserve, replace the URL with http://localhost:5080/api/default/v1/traces.

What Gets Captured

AttributeDescription
operation_nameOpenAI Chat Completions
llm_model_nameResolved model version (e.g. gpt-4o-mini-2024-07-18)
gen_ai_response_modelSame as llm_model_name
llm_systemopenai
llm_provideropenai
gen_ai_operation_namechat
openinference_span_kindLLM
llm_finish_reasonReason generation stopped (e.g. stop, length)
llm_invocation_parametersJSON-serialised request parameters
gen_ai_request_max_tokensmax_tokens from the request
llm_token_count_promptPrompt tokens consumed
llm_token_count_completionCompletion tokens generated
llm_token_count_totalTotal tokens consumed
gen_ai_usage_input_tokensInput tokens (mirrors llm_token_count_prompt)
gen_ai_usage_output_tokensOutput tokens (mirrors llm_token_count_completion)
gen_ai_usage_total_tokensTotal tokens
gen_ai_usage_cost_inputEstimated cost for input tokens
gen_ai_usage_cost_outputEstimated cost for output tokens
input_mime_typeapplication/json
output_mime_typeapplication/json
span_kindInternal
span_statusOK on success, ERROR on failure
durationEnd-to-end request latency

Viewing Traces

  1. Log in to OpenObserve and navigate to Traces
  2. Filter by service_name to find your application's spans
  3. Spans appear with operation_name: OpenAI Chat Completions
  4. Note that llm_invocation_parameters holds the alias you requested while llm_model_name shows the resolved version (e.g. gpt-4o-mini vs gpt-4o-mini-2024-07-18)
  5. Filter by span_status = ERROR to find failed calls
  6. Use llm_finish_reason = length to identify responses truncated by max_tokens

OpenAI JS trace in OpenObserve, Waterfall view with the OpenAI Chat Completions span attributes

Next Steps

With the OpenAI JS/TS SDK instrumented, every API call is recorded in OpenObserve. From here you can build dashboards tracking token consumption, compare response times across models, and alert on high error rates.

Read More

Was this page helpful?

On this page