Why does the AI give you a different figure when you ask again?
· Toni Martir, arquitecto de software
In the morning, someone in sales asks the company's AI tool «how many active customers do we have?» and gets a figure. In the afternoon, someone in accounts types the same question, word for word, and gets one almost four times bigger. Not a single customer was added in between.
The database is the same; what has changed is the SQL statement. The AI translates every question into an SQL statement (the technical instruction that fetches the data), and every time you ask, it translates again. If it writes a different statement, with a different criterion, you get a different figure. The question is why this happens and how to get a figure that repeats.
«How many active customers do we have?», asked twice on the same day
An office example with made-up figures. The database is the same in the morning and in the afternoon: what changes is the statement.
Nobody guarantees the same answer twice
The vendors say so themselves. OpenAI explains in its documentation that, by default, «model outputs may differ from request to request». There is a setting to reduce that variation, the temperature, which controls how much randomness goes into the way the model picks each word. But Google and Anthropic both warn that even with the temperature at zero the answer can still vary. And on Anthropic's models released after Claude Opus 4.6, that setting can no longer be changed at all.
So the same question can produce two different statements. And a small difference is enough to change the criterion in a question like this one, because «active customers» can be read more than one way: the customer records flagged as active, or the customers who have bought something in the last year. Both are reasonable, and the AI can take one in the morning and the other in the afternoon. The database runs each statement to the letter: both counts are right; they answer two different questions.
Checking one figure does not settle the next
When a figure matters, you check what the statement behind it does; I explained how to do that without being a programmer in how do you know the AI figure is right. But that check covers that statement, not the question. Ask again the next day and the AI may write a different one, which then needs checking all over again.
For a one-off question, you can live with that. For the figure the management team looks at every month, it means checking the same thing twelve times a year, and risking that one month's figure cannot be compared with the month before.
What repeats is the statement
Every answer from the AI is written from scratch; the statement, by contrast, is fixed text, and the database runs it the same way every time. Run on October's data, it gives October's figure by September's criterion, and the figure only changes when the data does.
So when a question turns into a figure you are going to track, you check the statement once and keep it (in a report, or with your IT people) with its explanation in words alongside, so that whoever reads the figure knows what it counts. That leaves the AI for what it does well: new questions.
So why does it give you a different figure?
Because each time it translates your question afresh and may write a different SQL statement, with a different criterion. The figure you are going to track always comes from the same statement: you check it once and you keep it.
At Reportman AI the SQL statement the AI writes is in plain view and can be reviewed, edited and run. The blind audit, optional and switched on by default, explains in words what each statement does without knowing your question: compare two explanations and you can see whether two answers use the same criterion. Both work with cloud AI and with a model installed on your own machine, with no subscription: you supply the machine.
Sources
- OpenAI, Advanced usage, section «Reproducible outputs»: «Chat Completions are non-deterministic by default (which means model outputs may differ from request to request).»
- Google Cloud, Experiment with parameter values, section «Temperature»: «Temperature controls the degree of randomness in token selection.»; «A temperature of 0 means that the highest probability tokens are always selected. In this case, responses for a given prompt are mostly deterministic, but a small amount of variation is still possible.»
- Anthropic, Create a Message,
temperatureparameter: «Note that even with temperature of 0.0, the results will not be fully deterministic.»; and «Models released after Claude Opus 4.6 do not support setting temperature.» - Reportman AI: home page (the optional blind audit), developers and local AI.
Toni Martir, arquitecto de software · · sources checked on