ObsessionDB vs. ClickHouse Cloud in the 10B Row Benchmark

ClickHouse recently published a detailed cost-performance comparison of cloud data warehouses. They ran ClickBench - 43 real-world analytical queries - at 1B, 10B, and 100B rows across ClickHouse Cloud, Snowflake, Databricks, and BigQuery.

The results were impressive. ClickHouse Cloud dominated: 67 seconds for 43 queries over 10 billion rows, at a cost of $4.27 per run. The competition wasn't close - Snowflake, Databricks, and BigQuery were 7-13x more expensive for similar or worse performance.

We read the post and thought: we should try this ourselves.

What We're Working With

For this benchmark, we used 20 nodes with 64GB memory each. The total always-on monthly cost for our infrastructure is $25,600.

For context: ClickHouse Cloud used 20 compute nodes as well. Based on their publicly available pricing and the 236GiB memory instance size, a comparable always-on configuration runs well north of $150,000/month.

How We Built the 10B Row Dataset

We replicated ClickHouse's methodology by following their public benchmark repository. The process has two steps:

1. Load the 100M-row ClickBench dataset from datasets.clickhouse.com/hits_compatible/athena_partitioned/hits_{0..99}.parquet
2. Duplicate it iteratively up to 6.4B rows, then top off to 10B

Apples to Apples - It's Complicated

Here it starts to get complicated to compare apples with apples. Two things matter:

1. Compression: ClickHouse Cloud uses a slightly different compression codec and rate than ObsessionDB. This alone leads to meaningful differences in on-disk size.
2. Part merging: ClickHouse Cloud merges parts up to 15GB - at least that's their default setting. In their results, they report a table size of 95,656,386,960 bytes, which is roughly 89GB. So their table likely consisted of around 6 parts.

Because of this, we ran two benchmark configurations:

• Run A - Comparable: We merged our dataset down to 6 parts as well, resulting in ~92GB on disk - close to ClickHouse Cloud's 89GB. This is the apples-to-apples comparison.
• Run B - Optimized: We merged the entire dataset into a single part at 54GB. The large difference in overall data size comes from the data redundancy in the ClickBench dataset and the resulting compression potential when everything is in one part.

The Results

Matching the methodology of ClickHouse by: "We report hot runtimes, defined as the best of three runs, and we disabled query result caches everywhere they exist".

To be fair: with ObsessionDB's 63s vs ClickHouse Cloud's 67s in Run A, we see both on eyesight - even though we ran the benchmark on smaller sized instances (64GB vs 236GB). The workload likely just can't benefit from more resources. Our gut feeling tells us that they would get similar results on smaller sized instances, too. The same most likely counts for a table merged down to one single part - we'd expect results around 40-50 seconds on a ClickHouse Cloud cluster as well.

Even on comparable sized machines, ObsessionDB customers get around 50% more performance per dollar.

How We Got There

1. Get the Best Out of Parallel Replicas

ClickHouse has a feature called parallel replicas - it distributes a single query across multiple nodes for faster execution. We used this heavily, and we want to be transparent about how.

We didn't blanket-enable parallel replicas for all 43 queries. While parallel replicas dramatically speed up heavy queries (aggregations, COUNT DISTINCT, regex scans), they actually hurt performance on lightweight queries that complete in under a few seconds. Sending a query to 20 nodes when one node can answer it in 15 milliseconds just adds overhead.

So we split the queries:

• 38 queries ran with parallel replicas across all 20 nodes - these are the compute-heavy ones (heavy aggregations, distinct counts, regex pattern matching)
• 5 queries ran on a single node - these are fast queries where the coordination overhead of parallel execution isn't worth it.

We don't know whether ClickHouse Cloud applied a similar optimization. Most query results are pretty similar, but on both systems there are some outliers - they balance out at the end.

Where Parallel Replicas Shined

Some of the speedups from distributing across 20 nodes were remarkable:

Where Single-Node Was Better

Those differences are tremendous, and we have one plausible hypothesis. When we ran the benchmark with 3 rounds, we'd expect - due to its determinism - that every distributed part of the query gets processed on the same instance. But with parallel replicas, nodes that finished their work can steal work from other nodes. The robbed node stops working. The consequence is that determinism is broken and the nodes don't have the chance to fully warm up in the first 2 of 3 runs. This explains why we see such a high variance in each run. We will repeat those queries later and see if the hypothesis turns out to be true or not.

2. Resource Efficiency

We obsess over maximizing every resource available. Two key techniques drive this: our distributed cache mesh and locality-aware merges.

To be transparent: for this benchmark, the distributed cache was disabled - we used standard local NVMe cache. But in production, these optimizations allow us to merge parts up to 150GB without risking network congestion - which is exactly what enabled Run B's single-part, 54GB dataset and its 35% performance improvement.

Our merge engine is also locality-aware, preferring parts already cached on the local node's NVMe to minimize cross-node data movement. We'll publish a detailed deep dive on this soon.

We also reproduced the benchmark with 25, 30, and 40 nodes. Performance did improve, but only by a few seconds - a gain that does not justify the proportionally higher infrastructure costs.

The Cost Picture

This is where it gets interesting.

ClickHouse Cloud charged $4.27 for a 67-second benchmark run. Our equivalent cost - calculated as our monthly infrastructure cost amortized per second - comes to $0.66 for Run A (~63s) and just $0.46 for Run B (~44s).

That's roughly 6x cheaper at equivalent performance, and up to 9x cheaper with the optimized dataset.

*Based on ClickHouse Cloud's public pricing for a comparable 20-node always-on configuration. Actual costs may vary with autoscaling, reserved capacity, or negotiated enterprise pricing.

We want to be clear: this comparison has limits. ClickHouse Cloud offers in their enterprise tier a level of maturity that we do not offer today. You're paying for more than compute when you use a managed service.

But if your primary concern is analytical query performance per dollar the economics here are hard to ignore.

Transparency

We'll be publishing the full raw results shortly at GitHub.

If you'd like to reproduce these numbers, run the benchmark on your own setup, or challenge ours - we welcome it. Drop us a line at marc@obsessiondb.com and we'll share everything you need.

---

Benchmark conducted February 16, 2026. 43 ClickBench queries, best of 3 runs, 20 nodes, 64GB each. Dataset: 10,000,000,000 rows following ClickHouse's exact duplication methodology.