Gene Kim wrote The Unicorn Project about a developer trapped in bureaucracy. Data engineers live that story every day.
The book defines five ideals. Locality, focus, joy, improvement, safety. Let me translate what breaking each one looks like in a data team.
Locality is broken when you need three teams to change one pipeline. A schema change in source requires a ticket to platform, a review from governance, and a two-week wait for the next deploy window. Your data engineers don’t own their domain end-to-end. They own fragments.
Focus dies when engineers spend Monday morning triaging Slack messages instead of building. “Can you check why this number looks off?” becomes the actual job. The pipeline work happens in the gaps between interruptions.
Joy disappears when your best engineer spends a sprint manually reconciling data between two systems that should have been integrated a year ago. Nobody quits over hard problems. They quit over pointless ones.
Improvement of daily work is the one I find most neglected. Teams ship new pipelines but never go back to fix existing ones. That transformation job held together with three workarounds and a prayer? It’s still running. Debt compounds silently until the platform is unmaintainable.
Safety - the psychological kind - is broken when data incidents get blame attached. “Who changed this pipeline?” instead of “what made this possible?” Without it, teams hide problems instead of surfacing them early. I’ve seen this destroy velocity more than any technical constraint.
And customer focus: data teams measure pipeline uptime and row counts. Meanwhile, the dashboard nobody opens has been green for six months. Adoption rate beats uptime as a success metric.
Which of these five ideals can be most improved on your data team right now?