The AI gives you a figure from your database: how do you know it is right?

16 October 2026 · Toni Martir, arquitecto de software

Someone in sales types «how much did we invoice in September?» into the company's AI tool and gets back a figure down to the cent. It can be added up perfectly and still not be the figure they asked for.

The vendors say so themselves. The documentation for Gemini in BigQuery, Google's tool that writes SQL statements from a plain question, warns that it «can generate output that seems plausible but is factually incorrect» and recommends validating all of it before use. The question is how to validate a figure if you do not work in IT.

The sum is right; the question may not be

A tool like that turns your question into an SQL statement (a technical instruction in the language databases understand), and it is that statement, not the AI, that fetches the data: the database runs it and the arithmetic comes out right. What can go wrong is the translation. «What we invoiced in September» can be read several ways, and if nobody settles it, the AI picks one. On BIRD, the public benchmark used to compare these systems, the best one returns the correct result for 82.95 out of 100 questions.

«How much did we invoice in September?»: five readings

Including VAT Adds up the invoice totals with tax included

Another question

Including cancelled invoices Also adds the invoices that were cancelled or corrected afterwards

Another question

By payment date Counts what was collected in September, even if it was invoiced earlier

Another question

Once per line Repeats each invoice's total on every one of its lines: a three-line invoice counts three times

Double-counted

Before VAT, without cancelled invoices, by invoice date The pre-tax amount invoiced in September

The one asked for

An everyday office example. If the person asking wanted the pre-tax amount invoiced, only the last row answers them; and all five come out to the cent, looking just as exact as each other.

The check you can always run: a blind audit

You rarely have the correct figure to compare with: if you had it, you would not be asking the AI. What you can do is ask for the SQL statement and have someone tell you in words exactly what it does: what it adds up, which dates it covers, what it leaves out and whether it counts anything twice. It can be a person from IT or another AI (even the same one, in a fresh conversation), on one condition: they must not know what you asked. That is a blind audit, and the final call is yours: only you know what you meant to ask. If that AI does not charge in euros, it charges in another currency: the statement describes your business.

Why the AI that wrote it cannot audit it

The easy option is to ask the AI that wrote the statement to explain it, in the same conversation. That is exactly who you should not ask. It has your question and everything said so far in front of it (what engineers call its context), and it reads the statement through all that: it tells you what it meant to do, which is what you asked for, not what the statement does.

So if it misunderstood you, it will make the same mistake explaining itself. If it took «what we invoiced» to mean the total including VAT, it will say «adds up what was invoiced in September», and you will notice nothing. Someone who does not know the question has to read the statement as it stands: «adds up the VAT-inclusive total of September's invoices». That is when you see it. It works like accounting: you do not ask whoever made an entry to check it.

So how do you know it is right?

You know it when you know which question it answers, and a blind audit tells you that: a reading of the statement by someone who does not know your question. If the tool hides the statement, the figure stays unchecked, however good it sounds.

At Reportman AI the AI writes the SQL statement, your database supplies the figure, and the statement is in plain view and can be reviewed and edited. The blind audit is a separate call to the AI that receives the statement and the description of your tables, but not your question or the conversation, and explains in words what it does. It is optional and switched on by default: leave it that way. All of it also works with a model installed on your own machine, with no subscription: you supply the machine.

Sources

Google Cloud, Write queries with Gemini assistance, section «Generate a SQL query»: «As an early-stage technology, Gemini for Google Cloud products can generate output that seems plausible but is factually incorrect. We recommend that you validate all output from Gemini for Google Cloud products before you use it.»
BIRD, execution accuracy leaderboard, «Overall Leaderboard» table, test set column: top system, GrainSQL (Purdue University, 26 September 2026), 82.95. Checked on 16 October 2026.
Reportman AI: home page (the three ways of answering and the optional blind audit), developers and local AI.

Toni Martir, arquitecto de software · 16 October 2026 · sources checked on 16 October 2026