Run your AI feature twice on the same input. If a different answer would be a problem, you’ve found a deterministic requirement.
It sounds trivial. It’s the fastest way I know to tell whether you’re about to put AI in the wrong place.
Non-deterministic means the same input can produce different output: an LLM, most ML scoring. Deterministic means same input, same output, every time: SQL, a function, a rules engine.
Run-it-twice sorts the work into two piles:
- A different answer is fine, even useful. Brainstorming, drafting, summaries, fuzzy matching with a human in the loop. A non-deterministic model fits.
- A different answer is a problem. Your month-end total, a compliance figure, a data transform, anything you have to reproduce or audit. You need a deterministic core.
The trap I see most often is a model doing the second kind of job. An agent that computes the figure directly. A “smart” pipeline that re-decides its schema every run. You can’t unit-test a vibe, and “the model said so” won’t satisfy a regulator.
So keep the deterministic core deterministic, and let the model build it. The LLM writes the SQL, you review it, the SQL returns the same answer forever. The model is excellent at generating the tool; running it in production is the tool’s job.
Run it twice before you ship. If the two answers have to match, put a deterministic engine at the center and keep the model on the edges.
Where in your stack is a non-deterministic model doing a job that needs the same answer every time?