Can you trust the AI's calculations?
· Toni Martir, arquitecto de software
Someone in accounts copies the two hundred unpaid invoices out of the business software, pastes them into the AI assistant and asks for the total. The answer comes back instantly, down to the cent: €48,317.25.
Whether that figure is a sum depends on something you cannot see: whether the AI did the arithmetic with a program or in its head, writing the number the same way it writes the rest of the answer. The vendors treat those two cases differently.
The total of the unpaid invoices: who does the arithmetic
An office example. All four figures come out with cents and look the same. What changes is who did the arithmetic, what you can check and which rows were added up: what gets pasted into a chat may not be the whole list.
Writing a figure is not calculating it
A language model, the engine behind these assistants, is trained to predict the next word from the preceding text, and that is how it writes its answer: word by word. To the model, a figure is just one more word.
For simple arithmetic, Anthropic lets its model, Claude, answer that way. For longer calculations it gives Claude a calculator: Claude runs code when a request involves "non-trivial math (large numbers, many steps, precision-sensitive results)" or data analysis. Microsoft says its assistant, Copilot, "can also run code to complete complex calculations". Adding up two hundred invoices takes many steps, and every cent of the result matters.
When Anthropic brought that calculator to its apps in 2024, it promised answers that are "mathematically precise and reproducible": the same figure every time.
How to tell whether it did the arithmetic
First, the code tool has to be switched on. Anthropic and Google offer it to the software that uses their models, but each application has to ask for it: an AI tool built on them may not have it. In Claude's own apps it is on by default, and on business plans the organization owner can turn it off.
Second, even when it is on, the model decides: Claude "evaluates whether code execution would help answer your question", and for borderline cases Anthropic advises asking explicitly: "run code to verify this". Google's own example spells it out: "Generate and run code for the calculation, and make sure you get all 50." So always ask for it, and ask to see the code: if there is no code, the arithmetic was done in its head.
Do the arithmetic where the data lives
The safest calculation is the one done by the program that already holds the data: a spreadsheet formula or, if the figures come from your business software, its database. The AI is just as useful there: instead of the figure, you ask it for the formula or the SQL statement (the technical instruction that pulls the data), and the spreadsheet or the database does the sum, over every row. It also spares you the copy and paste, and with it another way of losing invoices: if the screen only showed the first fifty, the AI adds up fifty, calculator or not.
That leaves checking that the instruction adds up what you asked for (without VAT, without cancelled invoices…); I explained how to check that without being technical in how do you know the AI figure is right.
So, can you trust the AI's calculations?
Only if a program did them: code the AI runs and shows you, a spreadsheet formula or your database. If the AI does the arithmetic in its head, the figure will have cents and look exact, but no program calculated it.
At Reportman AI, the Normal Response does not ask the AI to do the arithmetic: the AI writes the SQL statement, which is in plain view and can be reviewed before it runs, and your database calculates the figure. Every statement goes through a blind audit, optional and on by default, which explains in plain words what it does without knowing your question. It works with cloud AI, on tiered price plans, and with a model installed on your own machine: unlimited queries on every plan, with you providing the machine.
Sources
- Anthropic, Glossary, entry "Pretraining": "autoregressive language models (such as Claude’s underlying model) are pretrained to predict the next word, given the previous context of text in the document."
- Anthropic, Code execution tool, section "How code execution works": "When you add the code execution tool to your API request: 1. Claude evaluates whether code execution would help answer your question." And under "When Claude runs code": "Claude runs code when the request benefits from computation or file handling: Non-trivial math (large numbers, many steps, precision-sensitive results); Data analysis, file parsing, or visualization", among other cases; "Claude answers directly without running code for: Simple arithmetic and well-known math facts", among others; and "If you want Claude to run code for a borderline request, ask explicitly (for example, “run code to verify this”)."
- Microsoft, Transparency Note for Microsoft Copilot: "Copilot can also run code to complete complex calculations and generate graphs."
- Anthropic, Introducing the analysis tool in Claude.ai (October 24, 2024; a note dated November 5, 2025 says code execution is replacing it): "With the analysis tool, you get answers that are not just well-reasoned, but are mathematically precise and reproducible."
- Google, Code execution (Gemini API): "The Gemini API provides a code execution tool that enables the model to generate and run Python code."; "To enable code execution, configure the code execution tool on the model."; and the prompt in its example: "What is the sum of the first 50 prime numbers? Generate and run code for the calculation, and make sure you get all 50."
- Anthropic, Claude Help Center, Create and edit files with Claude, section "Availability": on the Free, Pro, Max and Ultra plans, "Code execution and file creation is enabled by default"; on Team, enabled by default and "Organization owners can disable it in organization settings"; on Enterprise, enabled by default "for new Enterprise plan organizations", and owners can disable it there too.
- Reportman AI: home page (the Normal Response and the blind audit, optional and on by default), developers ("Every query generated by the AI can be reviewed, modified, and audited by an expert before execution"), plans ("Unlimited Local Queries" on every plan) and local AI ("Reportman.AI allows local AI queries in all plans").
Toni Martir, arquitecto de software · · sources checked on