OpenObserve
Get DemoStar
Mobile View Menu

Observability Pipelines

Transform, enrich, redact, reduce, and parse your observability data with real-time and scheduled pipelines.

Start Free Cloud TrialStart Free Cloud Trial
OpenObserve Pipelines Dashboard
Protect Senstive Data

Protect Sensitive Data

Strengthen your security posture by automatically redacting and transforming sensitive data

Clear User Insight

Accelerate Data Insights

Unlock faster decision-making with real-time data parsing and enrichment using VRL functions and enrichment tables

Improved Data Quality

Improve Data Quality

Make data more valuable and actionable by pre-processing and enriching data

OpenObserve Observability Pipelines

Pipeline Types

  • Real-time

    Transform data the instant it arrives. Parse, filter, and enrich streams on-the-fly—turning raw data into immediate insights.

  • Scheduled

    Orchestrate batch transformations precisely when you need them. Aggregate massive datasets, convert logs to actionable metrics, and process data on your timeline.

Pipeline Types

Data Transformation

  • VRL Functions

    Bend data to your will with Vector Remap Language (VRL). Build custom transformations that parse, enrich, and filter your data streams with powerful functions that handle even your most complex logic requirements.

  • Data Parsing

    Discover insights regardless of data format. Transform chaotic logs into structured gold using pre-built parsing functions—making even the most complex data instantly queryable.

Data Transformation

Data Enrichment

  • Enrichment Tables

    Transform raw events into context-rich intelligence using simple CSV lookup tables. Instantly add location data, user details, and critical metadata that turns isolated signals into meaningful patterns.

  • Dynamic Lookups

    Connect your data to the world with real-time API enrichment. Pull in vital external context that elevates your observability data from interesting metrics to business-changing insights.

Data Enrichment

Pipeline Components

  • Function Nodes

    Build data processing workflows with VRL function nodes. Execute custom logic for complex transformations and create reusable components that make your data pipelines more efficient and maintainable.

  • Stream Operations

    Master your data flows with flexible stream operations. Filter, clone, and route streams to multiple destinations—ensuring every bit of data reaches exactly where it needs to go.

Pipeline Components

Observability Pipelines FAQs

Latest From Our Blogs

Sending Data from OpenObserve Pipelines to Splunk Using Splunk HTTP Event Collector (HEC)
How to
OpenobservePipelines

Sending Data from OpenObserve Pipelines to Splunk Using Splunk HTTP Event Collector (HEC)

How to send data from OpenObserve Pipelines to Splunk using the Splunk HTTP Event Collector (HEC) in 5 easy steps.

Md Mosaraf
Md Mosaraf
2025-06-20
June '25 Pricing Policy Updates for OpenObserve Cloud
Announcement
Openobserve

June '25 Pricing Policy Updates for OpenObserve Cloud

OpenObserve is transitioning to a fully usage-based pricing model effective June 2nd, 2025, eliminating minimum fees and the free tier. The new pay-as-you-go structure charges $0.30 per GB for log, metric, and trace ingestion, with additional fees for queries, pipelines, and advanced features like RUM and error tracking. Existing free-tier customers receive a 30-day exemption period, while current standard-tier customers will have their flat monthly fees removed after 30 days. New customers can access a 14-day free trial. The company emphasizes that their open-source version remains completely free for self-hosting, providing users with flexibility to choose between managed cloud services or self-managed deployments. This pricing change aims to create a more transparent, scalable cost structure that aligns charges with actual resource consumption while supporting OpenObserve's continued development and platform improvements.Description - OpenObserve announces usage-based pricing starting June 2025. Free tier ends, pay-as-you-go begins at $0.30/GB ingestion. 30-day transition period for existing users. Self-hosting remains free.

Jake Swiss
Jake Swiss
2025-06-02
Implementing Real-Time Anomaly Detection with OpenObserve and Random Cut Forest
How to
OpenobserveEnterprise

Implementing Real-Time Anomaly Detection with OpenObserve and Random Cut Forest

This comprehensive guide explains how to implement real-time anomaly detection using OpenObserve and the Random Cut Forest algorithm for time series data. Learn what anomaly detection is, how OpenObserve streamlines monitoring, and step-by-step instructions for setting up a robust anomaly detection system. Discover common challenges, troubleshooting tips, and best practices for scalable, cost-effective anomaly detection solutions.

Prabhat Sharma
Prabhat Sharma
2025-06-01
How to Deploy OpenObserve on Heroku: A Complete Guide
How to
OpenobserveIntegrations

How to Deploy OpenObserve on Heroku: A Complete Guide

Deploy OpenObserve on Heroku without DevOps complexity. Get logs, metrics, and traces monitoring running in minutes with our comprehensive deployment guide.

Chaitanya Sistla
Chaitanya Sistla
2025-05-14
Complete Guide to Retool Enterprise Error Monitoring and Observability
Engineering
EnterpriseObservability

Complete Guide to Retool Enterprise Error Monitoring and Observability

Learn how to implement enterprise-grade error monitoring and observability for Retool applications. Compare native monitoring vs OpenObserve, setup workflows, configure dashboards, and establish best practices for tracking Retool app errors.

Chaitanya Sistla
Chaitanya Sistla
2025-05-14
Complete Fortinet Firewall Monitoring Guide: Log Analysis
Engineering
SecurityLogging

Complete Fortinet Firewall Monitoring Guide: Log Analysis

Learn how to monitor Fortinet firewalls using OpenObserve. Step-by-step guide for syslog setup, log transformation, and creating dashboards for real-time security monitoring.

Chaitanya Sistla
Chaitanya Sistla
2025-05-11
Token Exchange & OpenObserve Service accounts
Engineering
EnterpriseOpenobserve

Token Exchange & OpenObserve Service accounts

Discover OpenObserve’s Service Accounts feature, designed for secure programmatic access to APIs. Learn how token exchange enhances security and simplifies automation.

Manas Sharma
Manas Sharma
2025-05-05
OpenObserve Reaches 15,000 GitHub Stars: A Journey to Provide Simple, Efficient, and Performant Observability for All
Announcement
Openobserve

OpenObserve Reaches 15,000 GitHub Stars: A Journey to Provide Simple, Efficient, and Performant Observability for All

OpenObserve has just surpassed 15,000 stars on GitHub, a milestone that fills me with both pride and gratitude. When we started this project three years ago, the goal was simple yet ambitious: to build an open-source observability platform that is easier, faster, and dramatically more cost-effective than anything out there.

Prabhat Sharma
Prabhat Sharma
2025-04-28
How to Configure Microsoft ADDS LDAP with OpenObserve and Dex: Complete Integration Guide
How to
MicrosoftOpenobserveEnterprise

How to Configure Microsoft ADDS LDAP with OpenObserve and Dex: Complete Integration Guide

Learn how to securely integrate Microsoft Active Directory Domain Services (ADDS) LDAP with OpenObserve and Dex. This step-by-step guide covers authentication configuration, troubleshooting tips, and best practices for enterprise observability solutions.

Chaitanya Sistla
Chaitanya Sistla
2025-04-23
Monitoring Java GC Logs: Enhance Application Performance with OpenObserve
Engineering
LoggingApplication

Monitoring Java GC Logs: Enhance Application Performance with OpenObserve

Learn how to leverage Java GC logs to diagnose memory issues, optimize application performance, and prevent production outages. This guide explains how to interpret GC log patterns and use OpenObserve for practical monitoring and analysis.

Manas Sharma
Manas Sharma
2025-04-18

Ready to get started?

Try OpenObserve today for more efficient and performant observability.

Get Started For FreeGet Started For FreeSchedule DemoSchedule Demo