In short
An AI hallucination is output from a language model that is fluent and confident but false, unsupported or invented: a fact that is wrong, a source that does not exist, a quotation never said, a function that is not in the library, a policy the company does not have. It happens because models generate text by predicting likely continuations rather than by consulting a store of facts, and plausibility is not truth. Hallucinations cannot be eliminated but can be reduced sharply by grounding the model in retrieved sources, constraining it to cite them, verifying claims programmatically, allowing it to say it does not know, and designing interfaces that show provenance and invite checking.
The mechanism
A language model is trained to predict the next token given the ones before it, across an enormous body of text. It does not store facts as facts; it stores patterns of how text about facts tends to go. Asked who founded a company, it produces the name that most plausibly follows that question, which is usually the right one because the training text was usually right. Asked about a company it has little data on, it still produces a plausible name, because producing plausible text is the only thing it does. There is no internal signal that distinguishes the remembered answer from the invented one; both come out fluent.
The name hallucination is a metaphor and a slightly misleading one; confabulation, the confident filling of gaps, is closer to what happens. The practical definition is any output that asserts something false or unsupported as though it were established.
The kinds that appear in products
Invented facts. Wrong dates, figures, names, specifications, prices. The most common and the hardest to notice, since they are embedded in otherwise correct text.
Fabricated citations and sources. Papers, articles, cases and URLs that do not exist, formatted perfectly. Lawyers have filed briefs citing invented cases; the pattern is well documented.
Invented quotes and attributions. Plausible sentences assigned to real people who never said them.
Non-existent code. Functions, parameters and packages that do not exist in the library the model is confidently describing, which is a risk in AI-generated code with a security dimension when the invented package name is registered by an attacker.
Invented policies and capabilities. A customer-service bot describing a refund policy the company does not have, or a feature the product lacks, which has produced real legal liability.
Unfaithful summaries. A summary that includes claims not in the source, or reverses the source's meaning.
Confident reasoning to a wrong conclusion, where each step reads sensibly and the chain is wrong.
Why they cannot be eliminated
Because generation is prediction, and prediction is never certain. Newer models hallucinate less, particularly on well-covered topics, and techniques below reduce the rate substantially, but any system that generates free text can generate false text, and the rate is never zero. Products that assume otherwise discover the residual cases in front of customers. The design goal is to reduce the rate, catch what remains, and limit the damage when something gets through.
The mitigation stack
Grounding through retrieval. Give the model the relevant source documents in its context and instruct it to answer from them. This is retrieval-augmented generation, and it is the single largest reduction available: the model paraphrases what it was given rather than recalling what it half-remembers.
Require citations. Instruct the model to cite the passage supporting each claim, and check programmatically that the cited passage exists and contains the claim. Uncited or unsupported claims are dropped or flagged.
Constrain the task. Narrow instructions, defined output formats, and explicit scope reduce the space for invention. A model asked to extract fields from a document hallucinates less than one asked to write freely about it.
Allow abstention. Instruct the model to say it does not know or the source does not cover the question, and reward that in evaluation. Models forced to answer invent; models permitted to decline often do.
Verify programmatically. Check facts against a database, run generated code, validate references against a catalogue, cross-check numbers. Anything checkable should be checked by software, not by the user.
Use a second model as a judge, reviewing the first's output for unsupported claims, which catches a meaningful share at a modest cost.
Lower the temperature for factual tasks, reducing the randomness that produces less likely, and often less accurate, continuations.
Evaluate continuously with a test set of questions with known answers, tracking the hallucination rate as prompts and models change, which is part of evaluating any LLM feature.
Designing for the residual cases
Show provenance: the sources the answer drew on, linked, so the user can check. Signal confidence honestly, and never dress an unsupported answer as a verified one. Keep humans in the loop for consequential outputs: legal, medical, financial, anything the company will be held to. Make correction easy, and feed corrections back. Label the assistant as an AI. And scope the product so that a hallucination is an inconvenience rather than a liability; a bot that drafts an internal summary can be wrong cheaply, and a bot that quotes prices to customers cannot.
Hallucination is a property of the technology, not a defect that a vendor will patch away, and products built on language models are built on managing it. If you would like an AI feature reviewed for where its hallucinations would land and what would catch them, book a call.
Common questions
Why do AI models hallucinate?
Because they generate text by predicting the most plausible continuation from patterns in training data, not by looking up facts. When the training data covered a topic well, the plausible answer is usually right; when it did not, the model still produces a plausible answer, with the same fluency and confidence, because plausibility is all it optimises for. There is no internal signal separating recalled facts from invented ones.
Can AI hallucinations be prevented completely?
No. Any system that generates free text can generate false text, and the rate is never zero. It can be reduced sharply by grounding the model in retrieved sources, requiring citations that are checked, constraining the task, allowing the model to say it does not know, verifying claims programmatically and using a second model to review. Products should be designed for the residual cases.
What is an example of an AI hallucination?
A chatbot citing a court case that does not exist, complete with a plausible name and reference; a coding assistant calling a library function that was never written; a customer-service bot describing a refund policy the company does not offer; a summary that attributes a claim to a document that never made it. Each is fluent, confident and wrong.
Does RAG stop hallucinations?
It reduces them substantially, because the model answers from documents placed in its context rather than from memory, and it enables citations that can be verified. It does not eliminate them: the model can still misread a source, combine passages wrongly, or invent when the retrieved material does not cover the question. Pair retrieval with citation checking and permission to abstain.
How do I detect hallucinations in an AI product?
Require citations and check programmatically that the cited source exists and supports the claim; verify checkable facts against databases and run generated code; use a second model to review outputs for unsupported statements; and maintain a test set of questions with known answers to measure the hallucination rate over time. Show sources to users so they can check too.
