OpenObserve vs Prometheus & Mimir: Metrics Benchmark

TLDR

• We pushed the same ~2.2 billion samples and ~1.09 million active bucket series through one OTel Collector into four deployments simultaneously (Prometheus, Grafana Mimir, OpenObserve on Parquet, and OpenObserve on Vortex), stopped ingestion, and ran identical PromQL over a frozen window. The full configuration and every query are in the benchmark repository.
• Everyday queries: OpenObserve leads by an order of magnitude. On irate at 3 hours, OpenObserve answered in 507 ms against Prometheus's 7,589 ms (~15x) and Mimir's 8,324 ms (~16x); both OpenObserve variants stayed under a second on every window.
• The million-series unfiltered histogram is where the systems diverge. At stock limits Prometheus and Mimir refuse the query outright. Raised to match, all four complete: OpenObserve in ~27 seconds at 3 hours against ~3.2 minutes for Prometheus and ~4.1 minutes for Mimir.
• Ingestion memory is OpenObserve's clearest win: flat at 1.5–2.1 GB RSS against Prometheus's 3.2–4.1 GB and Mimir's 4.5–5.5 GB, and under 600 MB once ingestion stops. The trade-off is ~2.5x Prometheus's disk.
• Format matters as much as engine on filtered queries. Vortex is consistently ~3x faster than Parquet on filtered histograms; at the 6-hour window Vortex beats both Prometheus and Mimir while Parquet falls behind them.
• Run again at a 14 GB memory limit, only Prometheus changes behavior: it is OOMKilled mid-query on the million-series histogram, peaking at 18.7 GB RSS on a single 3-hour query.

This is not a database-versus-database verdict so much as a test of what a metrics system does when cardinality crosses a million. Prometheus is the default for metrics, and Grafana Mimir is the natural scale-out answer for it. The benchmark asks the question teams actually face when they hit that wall: whether a columnar engine designed for high-cardinality data (OpenObserve) changes the trade-offs, and how much of the answer comes from the file format underneath it.

We wanted a straight, reproducible answer, so we ran all four systems on matched hardware with the same data and the same queries, twice, changing only the memory limit. OpenObserve is a single Rust binary that stores logs, metrics, and traces in columnar formats (Parquet or Vortex) with object storage as the durable layer. Prometheus and Mimir both come from the Prometheus lineage, with TSDB block storage tuned specifically for time series.

Why compare OpenObserve with Prometheus and Mimir

Prometheus earned its place as the reference metrics database honestly. It is the storage engine behind a majority of self-hosted monitoring, its PromQL is the lingua franca of metrics queries, and it is battle-tested at the scale of most single-node deployments. Grafana Mimir extends that same data model and query language across a cluster, which is why it is the common answer when Prometheus runs out of headroom.

The difference is what happens as cardinality grows. Prometheus and Mimir are built around a time-series database that is extremely efficient for a specific access pattern (sequential scans over contiguous time series), but their cost on large, high-cardinality queries scales with the number of output points and the series the query touches. OpenObserve takes the other path: it stores metrics in a general-purpose columnar format, which is why the same data can serve PromQL, SQL, and dimensional analysis together, and why its query cost scales with the data actually scanned rather than the points emitted.

Cost keeps this comparison honest. In Grafana's 2025 observability survey, 74 percent of respondents said cost is a top factor when picking observability tools, and the average organization runs eight observability technologies (Grafana 2025 survey). Metrics cardinality is a big part of that bill: the difference between a million-series query that answers in seconds and one that refuses or restarts a container is not a latency annoyance, it is an incident. See our metrics monitoring and Prometheus metrics types guides for the broader picture on where cardinality shows up.

How we ran it

The methodology matters more than any single number, so here it is in full. The driver, the configurations, and every query live in the benchmark repository, so you can reproduce or dispute anything below.

Hardware and versions

Four deployments, fully aligned: Prometheus, Grafana Mimir, OpenObserve on Parquet, and OpenObserve on Vortex. Each gets a dedicated EC2 m7gd.2xlarge (7 CPU, 474 GB local NVMe) with its data on the node's local instance store, so neither the disk nor a shared host decides the outcome. All are single-node deployments.

• Prometheus quay.io/prometheus/prometheus:v3.6.0
• Mimir grafana/mimir:latest
• OpenObserve v0.92.0, deployed twice as an A/B differing only in ZO_FILE_FORMAT: one parquet, one vortex

The dataset

The workload comes from openobserve/fake-webserver:v2: 24 pods, each producing ~45,000 time series, so the histogram's _bucket metric carries 1,085,760 active series. A separately deployed OTel Collector writes the same data to all four systems simultaneously, so every system receives the exact same samples: ~2.2 billion across the codelab_api_* metrics. Cardinality of the two key metrics:

• codelab_api_request_duration_seconds_bucket: 1,085,760 series (the histogram queries)
• codelab_api_request_duration_seconds_count: 41,760 series (the irate query)

Ingestion ran 2026-08-09 04:47–13:10 UTC (8h23m) and then stopped. Every query runs against a frozen dataset over absolutely-pinned time windows, so a run repeated an hour later sees the same bytes.

How the measurement is taken

Four choices here move the numbers more than any configuration, so they are worth stating outright.

• Queries are issued from inside the cluster. Driving a benchmark from a laptop over kubectl port-forward invalidates the result: the fastest query in this set answers in ~100 ms, and forwarding added 1,656 ms to 2,862 ms of pure overhead across repeats. Everything is measured from a pod in the same availability zone as all four systems, on a node that is not under test.
• Every system's data is on node-local NVMe. Storage is not a variable, and no query is measuring EBS throughput.
• Ingestion is stopped before anything is measured. Load still arriving during measurement would make each system's numbers depend on what it happened to be compacting.
• The step is computed, not pinned. A fixed 15s step makes the point count grow with the window (1,440 points at 6h), which no dashboard ever requests. Grafana asks for roughly the panel width in pixels, so this benchmark uses its rule: max(15s, range / 1000) rounded up to a tidy interval. That is 15s at 30m/1h/3h and 30s at 6h. It matters because Prometheus and Mimir cost scales with the number of output points, while OpenObserve's scales with the data scanned: widening the step on a fixed 3-hour window from 36 to 720 points moved Prometheus 1.68s→3.92s and Mimir 1.14s→4.11s, but OpenObserve only 3.45s→4.63s.

Every query runs three times and the tables report the median, plus one cold first-touch request recorded separately and excluded from it.

Fairness controls

Four systems are only comparable if they answer the same questions the same way. All query caches were disabled, query limits and timeouts were raised uniformly across all four systems so protective defaults did not decide the outcome, and one OpenObserve setting was deliberately turned off: Parquet's filter pushdown (ZO_FEATURE_PUSHDOWN_FILTER_ENABLED=false), because in these metrics tests it actually slowed queries by ~20%. The switch only affects Parquet; Vortex's filter pushdown is always on.

And one disclosure that belongs in every vendor benchmark: we make OpenObserve. Treat the numbers accordingly; the repository exists so you can rerun everything on your own account and tell us where we are wrong.

Config disclosure

The most common way a vendor benchmark goes wrong is tuning your own tool and running the competitor near defaults, so here is exactly what each side ran. Read this before the numbers.

Key OpenObserve environment variables:

extraEnv:
  - name: ZO_METRICS_MAX_SERIES_RESPONSE
    value: "40000"
  - name: ZO_METRICS_MAX_POINTS_PER_SERIES
    value: "10000000"
  - name: ZO_FEATURE_PUSHDOWN_FILTER_ENABLED
    value: "false"
  - name: ZO_RESULT_CACHE_ENABLED
    value: "false"
  # A/B: one deployment on parquet, one on vortex
  - name: ZO_FILE_FORMAT
    value: "vortex"

Prometheus launch arguments: remote write receiver enabled so the OTel Collector can push, and the last two raised from their defaults so the million-series histogram runs instead of being refused:

args:
  - --config.file=/etc/prometheus/prometheus.yml
  - --storage.tsdb.path=/prometheus
  - --web.enable-lifecycle
  - --web.enable-remote-write-receiver
  - --query.max-samples=1000000000   # default 50e6
  - --query.timeout=600s             # default 2m

Key Mimir configuration. Ingestion limits are raised so the write path is never throttled by defaults; the query-side ones are raised for the same reason as Prometheus's. Mimir needs both timeouts: http_server_write_timeout also defaults to 2m and fires first:

querier:
  timeout: 600s                        # default 2m

server:
  http_server_write_timeout: 600s      # default 2m

store_gateway:
  sharding_ring:
    replication_factor: 1

limits:
  ingestion_rate: 20000000
  ingestion_burst_size: 20000000
  max_global_series_per_user: 150000000
  max_fetched_chunks_per_query: 20000000   # default 2e6
  ha_cluster_label: prometheus
  ha_replica_label: prometheus_replica

Results

Ingestion: resource usage

Sampled every 10 minutes across the 8h23m ingestion, at the 28 GB memory limit. Disk is the settled figure, 14.8 hours after the last write:

Three observations.

Memory is OpenObserve's clear win. Sampled as RSS (the memory the process actually holds, not page cache the kernel is free to reclaim), Prometheus ran 3.2–4.1 GB and Mimir 4.5–5.5 GB, both moving in a sawtooth as head blocks fill and compact. Both OpenObserve variants stay flat at 1.5–2.1 GB, roughly half of Prometheus and a third of Mimir. Once ingestion stops everything collapses: Prometheus to 2.4 GB, Mimir to 2.5 GB, and both OpenObserve deployments to under 600 MB; the ingestion-time figures are buffers and WAL, not resident working set.

OpenObserve pays for it in CPU. ~1.6–2.2 cores versus Prometheus's ~1.3–1.9 and Mimir's ~0.7–1.1. The extra cycles go into columnar encoding and compression; at identical ingest throughput, that is the price paid for query performance and low memory.

Disk usage is largest for OpenObserve, and it deserves context. 28 GB versus Prometheus's 11 GB and Mimir's 18 GB over 8h23m of ingestion. A general-purpose columnar format keeps full detail rather than applying TSDB-tuned XOR/delta compression; in exchange, the same data simultaneously serves SQL, PromQL, and arbitrary dimensional analysis, and maps naturally onto object storage, where it typically lives in production at a very different unit cost. Parquet and Vortex tie, at 28 and 27 GB. At object store economics the difference in overall TCO is minimal.

One caution on reading disk numbers at all: disk is the one figure you cannot measure right after stopping ingestion. Watched live, Mimir's volume went up after the writes stopped (21.6 GB at 13:10, 29.5 GB by 14:23), because the compactor writes the merged block first and keeps the sources for half a day. It fell back to 18 GB only once deletion_delay expired. Read at stop+2h, Mimir looks the same size as OpenObserve; settled, OpenObserve uses 1.6x more. OpenObserve and Prometheus settle in ~2–3 hours; Mimir needs ~14h, and neither of its timers can usefully be shortened without risking silently incomplete query results. Wait ~14 hours after the last write, or label your disk figures provisional.

Query latency

The query tests cover the four most common dashboard workloads (irate, unfiltered histogram, regex-filtered histogram, equality-filtered histogram), each run over 30-minute, 1-hour, 3-hour and 6-hour windows. Tables show the median of three runs, in ms.

irate

sum by (path) (irate(codelab_api_request_duration_seconds_count[1m]))

The two OpenObserve variants run neck and neck and lead throughout. On the 3-hour window, Parquet's median of 507 ms is about 15x faster than Prometheus (7,589 ms) and about 16x faster than Mimir (8,324 ms). Both stay under a second on every window, including 6h, where Parquet is 9.7x faster than Prometheus and Vortex 10.0x. Over 41,760 series this is the cheapest query in the set, and it is still an order of magnitude apart.

Unfiltered histogram: the million-series stress test

histogram_quantile(
  0.9,
  sum by(le, path) (
    rate(codelab_api_request_duration_seconds_bucket{}[5m])
  )
)

This query has no label filter at all; it must run rate + aggregation over all 1,085,760 series. It is the most brutal query in the test.

All four complete every window, but only because the limits were raised, on all three systems. At stock settings this query does not run at all. Prometheus refuses it outright:

execution: query processing would load too many samples
into memory in query execution

Mimir survives 30m and 1h, then refuses the wider windows:

execution: the query exceeded the maximum number of chunks
(limit: 2000000 chunks) (err-mimir-max-chunks-per-query).
Consider reducing the time range and/or number of series
selected by the query.

OpenObserve's stock metrics limits reject it too. Every one of these is a protective default doing its job, and this benchmark raises all of them (--query.max-samples=1e9, max_fetched_chunks_per_query=20e6, ZO_METRICS_MAX_SERIES_RESPONSE=40000, ZO_METRICS_MAX_POINTS_PER_SERIES=1e7) together with a uniform 600s timeout. Given the same allowances and the same clock, all four finish every window, and the spread is an order of magnitude: at 6h OpenObserve finishes in ~46 seconds against Prometheus's ~4.5 minutes and Mimir's ~4.3 minutes, about 6x faster. At 3h it is ~27 seconds against ~3.2 and ~4.1 minutes.

Against a million-series unfiltered histogram, the defaults decide the outcome before the engines do.

Filtered histograms (regex and equality)

# regex match
histogram_quantile(0.9, sum by(le, path) (
  rate(codelab_api_request_duration_seconds_bucket{path=~"/api/bar"}[5m])
))

# equality match
histogram_quantile(0.9, sum by(le, path) (
  rate(codelab_api_request_duration_seconds_bucket{path="/api/bar"}[5m])
))

Regex match:

Equality match:

The two filter styles behave almost identically: regex and equality matching cost essentially the same across all four systems; the filter type isn't the variable, scan volume is. Ranking the 6-hour medians: Vortex 3,022 ms < Mimir 7,955 ms < Parquet 8,336 ms < Prometheus 9,237 ms. Note the order: Vortex is 3.1x faster than Prometheus, while Parquet sits between the two TSDBs, a position that moves with how selective the filter is.

Parquet vs Vortex: same engine, two file formats

The only difference between the two OpenObserve deployments is ZO_FILE_FORMAT. The results split cleanly by query type.

• irate and unfiltered histogram: a tie. When queries must scan all the data, the two formats trade places within 5–10%, with no decisive winner. Disk usage ties as well (28 and 27 GB).
• Filtered histogram: Vortex is consistently ~3x faster. The gap is stable and widens with the window: 2.9x at 30m, 2.9x at 1h, 3.2x at 3h and 3.2x at 6h. At the 6-hour window Parquet is slower than both Prometheus and Mimir on this query, while Vortex is roughly twice as fast as either.

Put differently: the "filter by service/endpoint" queries that dominate real dashboards are exactly the shape where Vortex gains the most. For dashboard-style filtered workloads, the format choice is worth more than the engine choice.

Round 2: the same benchmark at 14 GB of memory

28 GB is a comfortable sizing for a million active series. 14 GB is a plausible one. Running the identical benchmark at both (same frozen dataset, same queries, same absolute time windows, only the container limit changed) turns "how fast is it" into a second question: what still runs at all?

Halving the memory changes almost nothing, except that Prometheus stops being able to answer. Every cell landed within 8% of its 28 GB value, with two exceptions: the million-series unfiltered histogram at 3h and at 6h, where Prometheus was OOMKilled both times, mid-query, after 158s and 192s respectively (restartCount=2, exitCode=137).

The reason shows up in a direct measurement. A single 3-hour unfiltered histogram peaks at 18.7 GB RSS in Prometheus against 4.25 GB in Mimir: Prometheus loads samples into memory, and at --query.max-samples=1e9 and ~16 bytes per sample that is ~16 GB, matching the measured rise almost exactly. Raising the limit so the query can run, and having the query consume 20 GB, are the same decision. Mimir answers it in a quarter of the memory and takes 35% longer; on a 14 GB box that is the difference between an answer and a restart.

Worth noting how this was captured at all: curl cannot distinguish an OOMKill from a network failure; both surface as HTTP 000. The harness records a per-request timestamp and samples container restart state before and after each run, so the exact request that triggered the kill could be matched. Without that, this result would read "connection failure" and mean nothing.

Where Prometheus and Mimir win

An honest benchmark reports its losses, and the two TSDBs beat OpenObserve in real ways here.

• Ingest CPU. Mimir ran ~0.7–1.1 cores against OpenObserve's ~1.6–2.2, and Prometheus ~1.3–1.9. If your ingest nodes are CPU-bound, that is a real advantage.
• Disk footprint. Prometheus stored 11 GB against OpenObserve's 28 GB, the benefit of XOR/delta compression tuned specifically for time series. On local NVMe that is a smaller bill; though for larger deployments once you you move to s3 backed storage, that advantage vanishes.

Where OpenObserve wins

• Query latency on the workloads that hurt. On irate, OpenObserve is ~15–16x faster at 3h and ~10x faster at 6h; on the unfiltered million-series histogram it is ~6–7x faster. These are the queries that make dashboards unusable and paging a gamble.
• Memory footprint. Half of Prometheus and a third of Mimir while ingesting, under 600 MB idle, and no OOMKill at the 14 GB limit. The 18.7 GB peak Prometheus hit on a single query is the cost of loading samples into memory; OpenObserve's columnar scan never approaches it.
• High-cardinality stability. At a million series, the dividing line is not a few hundred milliseconds of latency; it is whether the query returns at all. OpenObserve answers where Prometheus refuses at stock limits and is OOMKilled at 14 GB.
• The Vortex format. On filtered queries it turns a loss into a win: ~3x faster than Parquet, and faster than both TSDBs at 6h where Parquet trails.
• One store for more than metrics. Because the data is a general-purpose columnar format, the same bytes serve PromQL, SQL, and dimensional analysis, and map onto object storage where it typically lives in production.

When to pick which

Query performance is the last piece of this decision, not the whole decision. The benchmark asked a narrow question: at the high cardinality that real production metrics reach, do Prometheus and Mimir hold up, and does a columnar engine change the trade-offs? The answer has an honest asymmetry.

Prometheus remains the right answer for modest cardinality and the ecosystem it anchors: a single node, a handful of targets, and the full PromQL/Alertmanager world at your fingertips. Mimir is the natural scale-out path when you are committed to that data model and want it to span a cluster, and on ingest CPU and disk footprint it is the most frugal system here.

OpenObserve's wins concentrate where the pain concentrates: high cardinality, memory pressure, and the specific query shapes that fill real dashboards. The price is more ingest CPU and more disk on local NVMe, a trade that inverts once the durable layer is object storage, which this run deliberately did not measure. And the format choice compounds it: Vortex is the better default for filtered, dashboard-style workloads, a result that holds across every window tested.

So the practical answer is not "use OpenObserve for histograms and Prometheus for counters." The split is what you are choosing. If you are picking a metrics database in the Prometheus lineage, Prometheus and Mimir are excellent at what they are built for. If you are running metrics at the cardinality where defaults start refusing queries and memory starts killing containers, OpenObserve, and specifically its Vortex format, changes what is even possible on the same hardware.

For teams whose metrics have crossed roughly a million active series, start with OpenObserve on Vortex. It does not win every benchmark: the TSDBs beat it on ingest CPU and disk footprint. But the query latency, memory profile, and high-cardinality stability are no longer a reason to stay within the TSDB's limits.

Try OpenObserve

If a columnar metrics engine that answers the million-series histogram in seconds and holds ingestion to a couple of gigabytes of memory appeals to you, the fastest way to feel the difference is to point some real metrics at it. OpenObserve Cloud gives you ingestion, PromQL and SQL, dashboards, and alerting without running the binary yourself, and the same engine is open source if you prefer to self-host. Every configuration and query from this benchmark is in the benchmark repository if you want to rerun it yourself.