Every time I start a data project I re-Google the same thing. "Is Glue the AWS Dataproc?" "Where does dbt actually run on Databricks?" So I finally drew the map once, properly, and the act of drawing it surfaced the one idea that makes the whole comparison simple. I'll get to that — but first, the frame that makes any of these stacks legible.
Everything is one of three layers
Strip the branding off and every cloud data stack is the same three layers: ingest (get data in), transformation (shape it), and storage & query (keep it, ask it questions). GCP, AWS and Databricks each have a full set of tools for all three — Pub/Sub or Kinesis or Auto Loader for streaming ingest; BigQuery or Redshift or Databricks SQL for the warehouse; and so on. Once you sort tools into those three buckets, the "which cloud" noise drops away and you can actually compare like for like.
The idea that makes it click: dbt isn't in any of them
Here's what I'd never quite articulated. dbt isn't an ingest tool, isn't storage, and isn't a compute engine. It's a SQL compiler, test runner and dependency graph that runs inside whichever warehouse already owns the compute. It doesn't move a single byte of data or spin up a cluster of its own — it turns your SELECTs into a managed, tested, documented DAG and hands them to BigQuery, or Redshift, or Databricks SQL to execute.
Which is why it plugs into all three clouds through a thin per-platform adapter:
| Warehouse | dbt adapter | Runs on |
|---|---|---|
| BigQuery | dbt-bigquery | BigQuery slots |
| Redshift | dbt-redshift | Redshift cluster / Serverless |
| Athena (lake on S3) | dbt-athena | Athena / Trino over S3 |
| Databricks | dbt-databricks | Databricks SQL warehouse |
| Snowflake | dbt-snowflake | Snowflake (the most common pairing of all) |
Follow that thread and a nice conclusion falls out: the warehouse is the only real variable. Swap warehouses and you swap one adapter; your ingest (Fivetran, Airbyte), your transforms (dbt models), and your orchestration (Airflow) all travel with you. Choosing a stack is mostly choosing where the compute lives.
Where dbt wins — and where the native tool beats it
The honest part, because "just use dbt everywhere" is lazy advice. Each platform has a native transform layer that can genuinely beat dbt for its home turf:
- Dataform (GCP) — now free inside BigQuery, Git-in-the-console, zero setup. For a BigQuery-only shop it's less to run.
- Delta Live Tables (Databricks) — streaming-first, with a built-in data-quality expectations framework dbt doesn't have natively.
- AWS Glue Workflows — not even a dbt competitor; it's a different layer (Spark/compute orchestration), and conflating them is a common mistake.
So the rule of thumb I settled on: dbt wins on portability, community, testing ergonomics and auto-generated docs. Native tools (Dataform, DLT) win on zero setup and streaming. Reach for dbt when the team already knows it, or when there's any real chance you'll switch warehouses. And know its edges: dbt doesn't move data, doesn't do heavy Spark/Python work, and doesn't schedule itself in production — it still needs Airflow, dbt Cloud, or Databricks Workflows to run on a cadence.
Three default stacks (and the hybrid everyone actually runs)
At a readable altitude, the out-of-the-box paths are:
- GCP: GCS → Dataflow / BigQuery Data Transfer → BigQuery → dbt or Dataform → Looker.
- AWS: S3 → DMS / Firehose / Glue → S3 + Glue Catalog (lakehouse) or Redshift → dbt or Glue Spark → QuickSight.
- Databricks: object store → Auto Loader / DLT → Delta Lake on Unity Catalog → dbt or DLT or notebooks → Databricks SQL.
But the stack most teams actually run isn't any single vendor's poster — it's the hybrid: Fivetran or Airbyte for SaaS ingest, dbt for the SQL transforms, Airflow for orchestration, pointed at whichever warehouse they picked. Which is the earlier point again, from the other direction: the warehouse is the variable, everything around it is portable.
So which do I pick?
The short heuristics that survive contact with a real project:
- Cheapest to start, small data: BigQuery — pay-per-query, no cluster idling. S3 + Athena a close second.
- ML-adjacent, notebook-first: Databricks — the only one where notebooks are a first-class production artifact.
- Already on Kafka: AWS (MSK) or Databricks (Structured Streaming).
- Best governance out of the box: Unity Catalog on Databricks.
- Most vendor-neutral: dbt + Iceberg + object store — avoids lock-in, at the cost of writing more glue yourself.
The transferable bit
When a tool seems to fit everywhere, it's usually because it's solving a different problem than the ones it sits next to. dbt feeling cloud-agnostic isn't magic — it's because it deliberately owns only the transform-as-code layer and delegates compute, ingest and scheduling to whatever's around it. Find the one layer a tool actually owns, and both its portability and its limits stop being surprising.
Two honest caveats before you take any of this as gospel: I haven't put concrete pound figures on a real workload here (a 1 TB/day medallion build would tell a sharper cost story), and the Iceberg-everywhere and dbt-Fusion stories are both moving fast enough in 2026 that the adapter picture may shift under us. Treat this as the decision map, not the final invoice.
If you're choosing a data stack and want the version grounded in your workload and budget rather than a vendor diagram, that's the kind of work I do at twentytwotensors. Get in touch.