The team bought more tokens. The agent still gave confident, wrong answers. The bottleneck was never compute.
I keep seeing AI budgets pour into bigger models and more tokens while the thing actually blocking the agent sits upstream: it has no reliable way to know what your data means.
An agent answering a business question needs three things most stacks don’t have ready:
- A semantic layer. One governed definition of “active user,” “revenue,” “churn,” so the agent isn’t guessing which of four tables is the real one.
- Governance and access. What this agent is allowed to see, and a way to prove it later. Added after launch, this becomes the thing that stalls the rollout.
- A protocol for tools and context. Something like MCP, so the agent reaches data through a defined interface instead of a pile of bespoke glue.
That’s all data foundation work, the kind that’s been undervalued for a decade and now has a deadline attached, because the agent exposes every gap instantly. Compute doesn’t touch any of it.
The order that works: get definitions, governance, and access right first. Then the model has something trustworthy to reason over. Skip it and a bigger model gets to the wrong answer sooner. That’s all you bought.
Before the next compute invoice clears, find out what the agent reads from and whether a single person owns it.
Does your AI feature read from governed definitions, or from whichever table someone wired up first?