AI Modeling Crisis Patterns

Every failed RAG project I’ve reviewed had the same upstream issue: nobody modeled the data underneath it.

The pattern repeats: a heap of unstructured JSON, an embedding model on top, and disappointment six weeks later.

Embeddings can’t recover meaning nobody defined. If “customer”, “account” and “user” are three overlapping things in your source systems, the vector store faithfully preserves the confusion. Retrieval returns plausible chunks about the wrong entity, and the model presents them with full confidence.

Joe Reis has called this the data modeling crisis, and I think the AI wave is exposing it. Modeling quietly stopped being taught, then quietly stopped being done, and now the bill arrives through AI projects that can’t answer basic questions.

What worked at a client this spring: define the core entities and their relationships first. About 15 of them. Plain semantic modeling, two weeks of unglamorous work. Same embedding model afterwards, usable answers.

Vectors amplify whatever structure exists, including none.

There are cases where raw retrieval is enough: narrow corpora, single-entity domains. Growing SaaS data is rarely one of them.

Could your systems agree on what a “customer” is before you embed them?