Data Architecture & Strategy
91 posts filed under Data Architecture & Strategy.
Pipeline Replay Decision Tree
Most pipelines are designed for the happy path. The first real outage makes recovery the only thing that matters.
Read →Lakehouse Convergence Risks
The lakehouse pitch is one platform for raw files and clean tables. The trap is inheriting the data lake's oldest habit: land everything, …
Read →Premature Real-Time Tax
Streaming has a cost nobody puts in the business case: the weekly tax of running it after the demo works.
Read →Semantic Layer Pitfalls - Invisible Technical Debt
You bought a semantic layer to end the "which number is right" argument. Eighteen months later you have 200 metrics and the same argument, …
Read →Deterministic vs Non-Deterministic AI
Run your AI feature twice on the same input. If a different answer would be a problem, you've found a deterministic requirement.
Read →Your AI Initiative Is Blocked by Context, Not Tokens
The team bought more tokens. The agent still gave confident, wrong answers. The bottleneck was never compute.
Read →Idempotency In Orchestration Failures
Your pipeline failed at 3am. The on-call engineer is now paid to clean up duplicates.
Read →Conway's Law Pipeline Silos
Two teams sitting in different Slack channels build two pipelines that compute the same metric.
Read →Contract-First Data For Cross-Team Handoffs
Most cross-team data fights are caused by an undocumented assumption upstream.
Read →Orchestration Tool Audit For Small Data Teams
Two-engineer data teams keep picking Airflow because it's "industry standard." Six months in, half the DAGs are commented out.
Read →