Why does AI get my numbers wrong?

The failure mode has a shape

An AI that hallucinates a fact is a problem you can see. An AI that reads a correct number and draws the opposite conclusion is a problem you cannot see — because nothing in the output looks wrong.

Most people worry about the first. The second is the one that reaches your investor deck.

Researchers have a name for the second kind: an ecological error. Not a fabricated fact, but a precise mechanical failure — correct logic applied to a badly extracted variable. The model does everything right, on the wrong number. And the output looks polished, which is the problem: a wrong number close to the right one is the hardest bug to catch.

Here is the shape it takes. Every financial metric has cases where it genuinely has no value. A company that never runs out of cash has no "months of runway". A company with no paid acquisition has no "CAC payback period". A human writing a spreadsheet leaves those cells blank, or writes "n/a", and every other human understands.

Software cannot write "n/a" into a number field. So it writes 0. And 0 is a perfectly valid number.

What it did to our own engine

Startkeel produces a financial model plus a calibrated prompt: a .txt file containing your real numbers, which you paste into your own ChatGPT or Claude for a second opinion. The numbers come from a deterministic engine; the AI only interprets them. The engine internal types were honest — a number, or nothing — but the function that serialised them collapsed "nothing" into 0. Five fields were affected.

In every one of them, what the AI concluded was wrong — and in two, it was the exact opposite of the truth. A runway of 0 meant "never runs out of cash"; the AI read "zero months, it is dead". A CAC payback of 0 meant "never recovers acquisition cost"; the AI read "instant payback". A burn multiple of 0 meant "not applicable"; the AI read "perfect efficiency". An LTV:CAC of 0 meant "not computable"; the AI read "broken unit economics". A CAC of 0 meant "no measured acquisition spend"; the AI read "free customers".

Read those pairs again. Three of the five errors flatter the business. A company with no payback at all is described as recovering its cost instantly. A company burning cash with nothing to show is described as perfectly efficient.

Our prompt opens by asking the AI to be brutally honest. We were feeding it data that pushed it, silently, toward flattery.

We found it while calibrating the engine — not because anything looked broken. Nothing did. That is the whole point of this failure mode: it does not announce itself. It surfaced only because we went looking at the data contract itself, field by field.

The bug was fossilised in its own guardrail

This is the part worth stealing, because it generalises far beyond financial models. If you do not read code, the idea survives the translation: the check that was supposed to protect us had banned the only correct fix.

Two of our tests asserted that the output must never contain a null value. The intent was right: NaN and Infinity are corruption — they mean a computation broke. But the same line also banned null, which is not corruption. It is information. It is the field saying "I have no value, and I know it".

So the test forbade the correct fix. As long as that line existed, writing null would have turned the suite red, and a diligent engineer would have "fixed" it by writing 0 again.

A guardrail that bans the solution is worse than no guardrail. It converts a bug into a rule. The test now bans only NaN and Infinity. Corruption stays banned. Absence is allowed.

The rule that closes it

We replaced a paragraph of prose in the prompt — which asked the AI to interpret sentinel values correctly — with one sentence, and made the data honest instead: null never means unknown. It means: does not apply, or does not happen. Every value present has been calculated.

The asymmetry is the whole point. A null cannot be mistaken for a number. A 0 can.

We tried the alternative first: extra boolean fields, such as a flag saying the business does not run out of cash, to disambiguate. We rejected them. Two fields carrying the same truth drift apart the moment one is updated and the other is not. One honest field beats two that must agree.

What to do with your own numbers

Compute outside the model. Spreadsheet, engine, script — anything deterministic. A language model that is not executing code does not calculate: it predicts the text that usually follows. It reads 847 as a string of characters, not as the integer 847. Give it finished arithmetic and ask it to explain.

Never let a sentinel value stand in for absence. Not 0, not -1, not 999. A human reads 0 in a runway field and thinks "n/a". A model reads it and thinks "zero".

Say what absence means, once. If your data can be absent, define absence explicitly, and state that everything else is real.

Suspect flattery. When a number is wrong, ask which direction the error points. Errors that flatter you survive review, because nobody argues with good news.

Why this makes us more careful, not less

Elsewhere on this site we say our engine cannot invent a figure. That is still true, and this bug is not a counter-example — it is the boundary between two different failures. The engine computed correctly. What broke was how we represented the absence of a number on its way out. No arithmetic was wrong. A null was flattened into a 0, and a 0 is a number.

That distinction matters, because it is the one thing a language model cannot make for you. Give it a 0 and it has no way to ask whether you meant "zero" or "nothing". It will answer either way, fluently.

We are not claiming our engine is infallible. We are claiming something narrower and more useful: it is checkable. The verdict Startkeel gives you — default alive or default dead, the benchmarked ranges, the score — is computed by code that a test can pin down and a mutation can prove. When we broke it, a test could be made to fail. When we fixed it, the same test went green for the right reason.

An LLM asked to compute the same verdict cannot be pinned down that way. Independent work says the same: a CFA Institute review of large language models in finance documents hallucination and numerical-reasoning limits, and 2026 benchmarking found that several leading models fabricate financial data when the source document is incomplete — some confidently, and without disclosure.

That is why the AI prompt we ship is deliberately constrained. It is handed the numbers. It is told not to recompute them, not to invent them, and not to re-litigate the simplifications we have already declared. Its job is to explain, argue and challenge — on top of arithmetic it did not do.

We caught this one because our numbers live somewhere a test can reach them. That is the whole argument for keeping the arithmetic out of the model.

FAQ

Related guides

• Startkeel vs ChatGPT: should you model your SaaS in an LLM?
• Do you need an AI CFO agent?

Related tools

• SaaS Viability Score — Does my whole business hold up? One 0–100 score across five pillars.
• Runway & Burn Calculator — How long until I run out of cash? Your zero-cash date and break-even month.
• See all: Cash & runway

Related benchmarks

• What is a good CAC payback period for SaaS?