Data Ingestion - Logs, Metrics, Traces & Profiles

Collect logs, metrics, traces, and profiles into OpenObserve from a variety of data sources for unified observability. This data ingestion guide describes how to ingest observability data from log forwarders, monitoring agents, OpenTelemetry Collector, APIs, and programmatic methods.

In the OpenObserve UI, Data > Data sources shows ready-to-copy endpoints, headers, and agent configurations for your organization, with an ingestion token already filled in.

Logs

Ship application and infrastructure logs to OpenObserve through agents, HTTP APIs, or your own code.

Log forwarders

1. OTEL Collector
2. Vector
3. Filebeat
4. Fluent Bit
5. Fluentd
6. Amazon Kinesis Firehose
7. Splunk HEC
8. Syslog (alternatives)
9. Python
10. Go
11. Curl

APIs

Logs can also be ingested into OpenObserve Cloud / OpenObserve through one of these 3 HTTP APIs.

1. _json
2. _multi
3. _bulk

You can call the above APIs directly in your code to ingest data.

From Code

Here are 2 examples on how you can do it programmatically:

1. Go
2. Python

Curl

You can also use curl command to ingest logs:

1. curl

Metrics

Send time-series metrics from Prometheus, Telegraf, or the OpenTelemetry Collector.

1. OTEL Collector
2. Prometheus
3. Vector
4. Telegraf

Traces

Send distributed traces using OpenTelemetry SDKs or the Collector.

1. OTEL Collector
2. OpenTelemetry
3. TypeScript
4. Node.js
5. Python
6. Go

Profiles

Send continuous CPU and memory profiles over OTLP Profiles.

1. OTLP / Collector
2. Java (async-profiler)
3. Go (pprof)
4. Rust (pprof)
5. eBPF profiler

Need some help?

• Join our Community Slack
• Or Contact support