Rust Is Quietly Becoming the Language of Data Infra

The fastest tools in your data stack are written in a language nobody on your team writes.

Polars was built in Rust. So was delta-rs, and uv, the package manager a lot of Python teams picked up this year. Your team stays a Python team: these tools hand out Python interfaces, though adopting one is still a migration, not a free upgrade.

The consequence is a budget one. One large machine now handles work that got handed to a cluster by default a few years ago, and a cluster is a monthly bill, an on-call rota and another thing to keep patched.

So before you sign off the cluster line in next year’s budget, have someone run your most expensive job on a single machine and measure it. A week of someone’s time, against a year of the bill.

It doesn’t hold for everything. Large volumes, a lot of concurrent users, or jobs that need distributed compute for good reasons stay where they are. Someone also has to know how to size a machine, which is a skill plenty of teams lost while the cluster did it for them.

That test needs no Rust from anyone. The tools expose Python and a command line, which is the whole point of where the language sits.

What’s in your stack because it was the sensible default three years ago?