Most writing about AI hallucination is about the world: a fabricated court case, an invented citation, a library function that does not exist. Those are model problems, they are measurable on public benchmarks, and they have improved a great deal.
The hallucination that costs companies money is a different animal. It is a model stating, fluently and in your house style, that a customer's renewal is in March. There is no benchmark for that, nobody catches it in review, and the only person who could have caught it is the person who asked.
Why your data is the worst case
A model that does not know a public fact often has some signal that it does not know. The question is unusual, the entity is obscure, the training data was thin. Hedging is a learned behaviour and it fires more often where the model is genuinely uncertain.
Your internal facts give it none of that. The question when does BigCo renew? is grammatically identical to a thousand questions it answered well. "BigCo" is a plausible company. "March" is a plausible month. Nothing about the shape of the request signals danger, so nothing triggers the hedge. The model produces the most probable continuation of a sentence that begins BigCo's renewal is in — and that is a guess wearing the clothes of an answer.
This is why "the model is getting better" does not solve it. A better model guesses more plausibly.
Four mechanisms, not one
In practice, wrong answers about company data come from four distinguishable places, and they want different fixes.
No evidence was retrieved. The renewal date exists in the CRM, nothing fetched it, and the model answered from priors. The fix is a tool call, not a better prompt.
Partial evidence was retrieved. This one is the most dangerous, because the answer is partly grounded. Similarity search returned the support thread — it sounds the most like a customer question — and the invoice and the contract never came back. The model answers from a quarter of the record and sounds entirely certain, because from where it sits the evidence was consistent. We have written about why retrieval misses on relational questions.
Identity collapsed. sam@bigco.com in the mailbox, "Sam Lee" on the ticket and contact 0012f in the CRM are one person, or they are three, depending on whether anything resolved them. Get this wrong and the model confidently merges two customers, or splits one, and every number downstream is wrong in a way no spot-check reveals.
The evidence disagreed with itself. Two systems hold different renewal dates because somebody changed one and not the other. There is no correct answer to retrieve. A model asked to produce one will pick, silently. What you want is for it to say these two records disagree — which means something has to be looking for contradictions rather than hoping they do not exist.
What actually reduces it
Cite everything, and make the citation checkable. Not "according to your CRM" — the record, the field, the timestamp. The value of a citation is not that it reassures the reader; it is that it makes the wrong answer findable. An uncited figure cannot be audited, so it is never audited. This is why every figure Memtro returns is cited to the record it came from.
Make "I don't know" a first-class answer. A system that cannot return nothing will always return something. If the tool found no renewal date, the right output names the gap. This is mostly a product decision rather than a model one: somewhere in your stack, an empty result has to survive all the way to the reader instead of being smoothed over.
Resolve identity before retrieval, not after. If the question is about a customer, the system needs to know what that customer is across every system before it decides what to fetch. A graph does this structurally; a pile of embeddings cannot, because similarity is not identity.
Keep history, do not overwrite it. When somebody asks why a decision was made in July, the answer has to be what was true in July. Memory that overwrites can only ever tell you what is true now, which means it cannot distinguish "the model was wrong" from "the facts changed". Both look the same afterwards, and only one is your fault. That is the whole argument for reading memory as of a date.
Read-only by default. A hallucination that only produces a sentence is recoverable. One that produces a sentence and updates a record is not. Keeping write access to specific, individually granted tools is what keeps a confident mistake from becoming a durable one.
The uncomfortable part
None of this gets you to zero, and you should be suspicious of anyone who says otherwise. Models will always be able to produce a fluent sentence about a fact nobody fetched.
What you can change is whether a wrong answer is detectable. An uncited answer and a correct answer look identical. A cited answer that points at the wrong record looks wrong immediately, to anyone who clicks. That is a much lower bar than truthfulness, and it is the one that actually protects a business: not a system that never errs, but one where every claim has an address you can check.
If you want to test this on your own systems, the quickest experiment is to connect one and ask it something you already know the answer to — then look at whether the citation points where it should.