What a Terabyte Actually Costs: TPC-DS on Cazpian
· 15 min read
There is a long-standing assumption in data engineering: if you want fast SQL, you do not use Spark. Spark is for pipelines. For interactive analytics you bring in a dedicated query engine, run it alongside, and accept the cost of operating two systems over one set of tables.
We wanted to know how true that still is. So we ran the full TPC-DS benchmark — all 103 queries — against a terabyte of Apache Iceberg data on a four-worker Cazpian cluster, and published everything: every query time, the cost, the hardware utilisation, and the parts that are not yet good enough.