Fractional Data Architect
Book a discovery call →

Warehouse Or Lakehouse At 40 People

Warehouse Or Lakehouse At 40 People
Warehouse Or Lakehouse At 40 People

At 40 people, a lakehouse needs someone whose job is running it.

A lakehouse at that size means files in S3 or ADLS, a table format like Delta or Iceberg, a catalog, compaction jobs, and access rules on both buckets and tables. Taken together, that’s a small platform, and someone has to look after it.

A managed warehouse hides most of that. You pay for the hiding, and at 40 people that’s usually a good trade.

The filter I use:

  • No platform or infra engineer, and nobody who wants to become one: warehouse.
  • Heavy ML, large raw files, or data that has to stay in your own cloud account: a lakehouse is worth a look, if someone owns it.
  • Not sure yet: warehouse now, and keep the raw files in cheap storage on the side.

The painful version I’ve seen is a half-run lakehouse. Nobody compacts. The catalog has no owner, and after a year nobody can say which tables are still current.

Which way are you leaning? I compared the main platforms for small teams here: https://thomasnys.com/snowflake-vs-bigquery-vs-databricks/

Written by Thomas Nys

Fractional Data Architect helping startups and scaleups build data platforms that scale.

More about Thomas Nys →

Recognise the problem? Let's talk about it.