OpenObserve
  • Platform
    • Logs
    • Metrics
    • Traces
    • Frontend Monitoring
    • Pipelines
    • Alerts
    • Visualization & Dashboards
  • Solutions
    • AWS Monitoring
    • Azure Monitoring
    • GCP Monitoring
    • Kubernetes Monitoring
    • Database Monitoring
    • OpenTelemetry
  • Resources
    • Documentation
    • Customer Stories
    • Blog
    • Freqently Asked Questions
    • Webinar and Videos
    • Partners
    • Events
    • Downloads
  • Pricing
Get Demo
AWS US East-2 cloud.openobserve.ai Ohio, USA · AWS Azure US West 2 us2.openobserve.ai California, USA · Azure AWS EU Central-1 eu1.openobserve.ai Frankfurt, Germany · AWS
Star
OpenObserve
Get Demo Star
Open Menu Close Menu
  • Platform Arrow

    Full Stack Observability Platform

    View Platform
    • Logs
    • Metrics
    • Traces
    • Frontend Monitoring
    • Pipelines
    • Alerts
    • Visualization & Dashboards
  • Solutions Arrow

    Full Stack Observability Solutions

    View Solutions
    • AWS Monitoring
    • Azure Monitoring
    • GCP Monitoring
    • Kubernetes Monitoring
    • Database Monitoring
    • OpenTelemetry
  • Resources Arrow

    Full Stack Observability Resources

    View Resources
    • Documentation
    • Customer Stories
    • Blog
    • Freqently Asked Questions
    • Webinar and Videos
    • Partners
    • Events
    • Downloads
Pricing
Docs Icon Light Docs Icon Dark
Documentation
Initializing search
      • Introduction
      • Our Principles
      • Comparison with Alternatives
    • Architecture
    • Getting started
      • Log Management
      • Metric Monitoring
      • Distributed Tracing
      • Frontend Observability
      • Enterprise Features
      • What's next
    • Releases
        • OpenObserve Query Examples
        • Telemetry
          • Federated Search Overview
          • How to Use Federated Search
          • Federated Search Architecture
          • Logs Overview
          • Logs in OpenObserve
          • Refresh Cache Run Query
          • Search Around
          • Insights
          • Quick Mode and Interesting Fields
          • Explain and Analyze Query
          • Metrics Overview
          • File Access Time
          • Downsampling Metrics
          • Overview
            • RUM Overview
            • Setup Guide
            • Performance Monitoring
            • Session Tracking
            • Error Tracking
            • Session Replay
            • Metrics Reference
            • Advanced Features
            • Use Cases
            • Best Practices
            • Troubleshooting
          • Traces Overview
          • Traces in OpenObserve
          • Service Graph
          • Actions Overview
          • Actions in OpenObserve
          • Create and Use Real-time Actions
          • Create and Use Scheduled Actions
          • Declare Python Dependencies in Actions
          • Environment Variables in Actions
          • Enrichment Tables Overview
          • Create and Use Enrichment Tables
          • Upload, Caching, and Restart Behavior
          • Functions Overview
          • Functions in OpenObserve
          • OpenObserve Python SDK
          • Pipelines Overview
          • Pipelines in OpenObserve
          • Create and Use Real-time Pipeline
          • Create and Use Scheduled Pipeline
          • Remote Destination
          • Import and Export Pipelines
          • Manage Pipelines
          • Configurable Delay in Scheduled Pipelines
          • Streams Overview
          • Streams in OpenObserve
          • Stream Details
          • Schema Settings
          • Extended Retention
          • Summary Streams
          • Field and Index Types in Streams
          • Query Recommendations Stream
          • Distinct Values
          • Alerts Overview
          • Alerts in OpenObserve
          • Alert Folders
          • Alert History
          • Alert Conditions
          • Import and Export Alerts
          • Multi-window Selector in Scheduled Alerts (SQL Mode)
          • Use Multi-window Selector in Scheduled Alerts
          • Use Telegram for Alerts
          • Dashboards Overview
          • Dashboards in OpenObserve
          • Manage Dashboards
          • Move and Export Dashboards in Bulk
            • Panels Overview
            • Manage Panels
            • Troubleshooting
            • Config Overview
            • Trellis Layout
            • Show Symbol
            • Variables Overview
            • Variables in OpenObserve
            • Variable Types
            • Variable Dependencies
            • Filters
          • Comparison Against in Dashboards
            • Custom Charts Overview
            • What Are Custom Charts
            • Custom Charts with Flat Data
            • Custom Charts with Nested Data
            • Event Handlers and Custom Functions
            • Custom Charts for Metrics Using PromQL
            • Custom Charts for Metrics Using Multiple PromQL Queries
          • Histogram Caching
          • Reports Overview
          • Reports FAQ
        • Work group
          • Identity and Access Management Overview
          • Role-Based Access Control (RBAC)
          • Enable Role-Based Access Control (RBAC) in Enterprise Edition
          • Single Sign-On (SSO)
          • Organizations
            • Quotas Overview
            • Quotas
            • How Quotas Work
          • Manage Users & Passwords
          • General Settings
          • Management Overview
          • Streaming Search
          • Streaming Aggregation
          • Query Management
          • Alert Destinations
          • Templates
          • Audit trail
          • Cipher Keys
          • Nodes in OpenObserve
          • SSO Domain Restrictions
          • Sensitive Data Redaction
          • License Management
          • Profile Overview
          • UI Languages in OpenObserve
        • SIEM Platform - Security Information and Event Management Solution | OpenObserve
          • Best Practices
          • Prevent UI Crashes From Large Queries
          • Ensure the S3 Configuration is Correct
          • Performance Overview
          • Download Manager
          • Result Cache
          • Monitor Download Queue Size and Disk Cache Metrics
          • Configure Disk Cache Eviction Strategy
          • Tantivy Index
          • Broadcast Join
      • Ingestion Overview
        • otel-collector
        • Vector
        • Filebeat
        • Fluent-bit
        • Fluentd
        • Kinesis Firehose
        • Syslog
        • Python
        • Go
        • Curl
        • Index
        • Prometheus
        • OTEL Collector
        • Telegraf
        • Index
        • OpenTelemetry
        • Typescript
        • Node.js
        • Python
        • Go
        • Rust
      • Integrations Overview
        • Claude Code Tracing
        • **LLM Observability with OpenObserve**
          • MCP Overview
          • Claude code CLI
        • Integration with Cloudflare
        • Heroku Application Logs Integration Guide
        • Vercel Application Logs Integration Guide
          • AWS Integrations Overview
          • Amazon EC2
          • Application Load Balancer(ALB)
          • Amazon Virtual Private Cloud
          • Amazon Cognito
          • AWS Network Firewall
          • AWS Cloudwatch logs
          • AWS Cloudwatch metrics
          • Amazon Relational Database Service (RDS)
          • Amazon Lambda Function
          • Amazon Elastic Container Service (ECS)
          • AWS Route 53
          • AWS Web Application Firewall (WAF)
          • API Gateway logs
          • Amazon CloudFront
          • Amazon EventBridge
          • GCP Integrations Overview
          • GCP Logs
          • Google Cloud Run
        • Apache Airflow Integration Guide
        • Cribl Integration Guide
        • MuleSoft AMC Logs Integration Guide
        • Database Integrations Overview
        • PostgreSQL
        • Oracle Database
        • Snowflake
        • MySQL
        • MongoDB
        • Redis
        • Amazon DynamoDB
        • Apache Zookeeper
        • Cassandra
        • Aerospike
        • DevOps Integrations Overview
        • Jenkins
        • Ansible
        • Terraform
        • Github Actions
        • Message Queues
        • Kafka
        • RabbitMQ
        • NATS
        • Security Integraions
        • Microsoft 365
        • Servers Integrations Overview
        • Nginx
        • Oracle Weblogic
        • Kubernetes Monitoring - K8s Cluster Monitoring and Container Observability | OpenObserve
        • Linux Server Monitoring - System Logs and Performance Metrics | OpenObserve
        • Windows Server Monitoring - Event Logs and Performance Metrics | OpenObserve
        • Capacity Planning Guide - Resource Sizing for Production Deployment | OpenObserve
        • High Availability Deployment Guide - HA Setup on Kubernetes | OpenObserve
        • Openobserve enterprise edition installation guide
        • Performance Optimization Guide - Ingestion and Query Performance Tuning | OpenObserve
        • O2 SRE Agent Setup Guide
        • Environment variables
          • OpenObserve CLI
          • K8s Operator Overview
          • Deployment Script
          • Manual Deployment
          • Features
          • Troubleshooting Guide
        • Monitoring OpenObserve
          • Operator Guide Overview
          • Systemd
          • SIMD
          • Mimalloc
          • Grafana plugin
          • Etcd maintenance
          • Etcd restore
          • Nginx proxy
          • Configuration Management
          • Storage Management Overview
          • Storage Configuration
          • Configure Multiple Object Storage Accounts
          • Schema
          • Settings
          • List
          • Delete
            • JSON
            • Bulk
            • Multi
            • Loki
            • Otlp
            • Prometheus
            • JSON
            • OTLP
          • Search
          • Around
          • Value
          • Profile
          • Create
          • Update
          • Delete
          • List
          • Examples
          • Create
          • Update
          • Add to org
          • Remove from org
          • List
          • Cluster Info API
          • Traces API Overview
          • Search Traces
        • SQL Functions Overview
        • Full-Text Search Functions
        • Secondary Index Functions
        • Array Functions
        • Aggregate Functions
          • Approximate Aggregate Overview
          • approx_topk
          • approx-topk-distinct
      • SQL Reference
        • Overview
        • Architecture & Terminology
        • Migrating Metrics
        • Migrating Traces
        • Migrating Logs
        • Migrating Dashboards & Alerts

    Pipelines Overview

    In OpenObserve, pipelines are data processing workflows used to transform logs, metrics, or traces during ingestion or before storage.

    Learn more:

    • Pipelines in OpenObserve
    • Create and Use Real-time Pipeline
    • Create and Use Scheduled Pipeline
    • Import and Export Pipelines
    • Manage Pipelines
    • Configurable Delay in Scheduled Pipelines
    Was this page helpful?
    Thank you for your feedback.
    Thank you, we will review this page.
    Stay Updated

    Get the latest OpenObserve insights

    By subscribing, you agree to receive product and marketing related updates from OpenObserve.

    Official AICPA SOC badge

    SOC2 Type II

    Certified

    Star

    OpenObserve Inc. ©

    3000 Sand Hill Rd Building 1, Suite 260, Menlo Park, CA 94025

    • Terms of Service
    • Privacy Policy

      Thank you!

      Your response
      has been submitted successfully!

      This is for Website feedback only - for general support go to our Community Slack
      Failed to submit feedback. Please try again.
    • slack
    • twitter
    • linkedin
    • github
    • youtube