This video explains the need for observability in complex AI/LLM applications and demonstrates how to monitor them using tools like OpenObserve and OpenTelemetry. It covers key metrics such as traces, latency, token usage, cost, and logs, along with practical dashboards and visualization techniques for debugging and performance optimization.
Why observability is critical for complex LLM applications
The four key signals: traces, latency, token usage, and logs
How to instrument LLM apps using OpenTelemetry
How to monitor end-to-end LLM traces and workflows
How to measure latency (TTFT and total response time)
How to track token usage and cost across models and users
How to debug using prompt and response logs
How to use waterfall views and span details for analysis
How to identify bottlenecks with flame graphs
How to visualize system architecture using service maps and DAGs
How to correlate logs and metrics using trace IDs
How to use dashboards for LLM cost monitoring
This video explores why modern AI applications—especially those using multiple LLMs, large context windows, and tool integrations—require strong observability to ensure reliability and efficiency. It introduces the four key monitoring signals: traces, latency, token usage/cost, and logs.
The tutorial walks through instrumenting an LLM application using OpenTelemetry GenAI specifications and integrating it with OpenObserve via a Python SDK or collector. Using real-world trace data from a production SRE agent, the video demonstrates how to analyze system performance through metrics like request rate, errors, and duration.
It further showcases advanced visualization tools including waterfall trace views, span-level input/output inspection, flame graphs for identifying bottlenecks, service maps, and DAG flow diagrams. The video also highlights how logs and metrics can be correlated using trace IDs, and presents pre-built dashboards for tracking LLM costs across models, users, and features.
Simran specializes in DevOps, cloud-native technologies, and observability, with hands-on experience in Kubernetes, Docker, and AWS. Creates practical, accessible technical content and solutions that help teams simplify complex workflows and improve system reliability.
