Why Models Hallucinate

You ask an assistant for three studies to cite in a report. It gives you three neat references: authors, year, journal, a one-line finding each. Two check out. The third looks just as real, with a plausible author and a journal that exists, but the paper does not. Nobody wrote it.

This is called a hallucination: output that is fluent and confident but false or unsupported. It is not a rare glitch, and the model is not lying. It follows directly from how a model makes text.

#Plausible is not the same as true

A language model produces text one token at a time, each picked from chances that say what usually comes next (What Is an LLM). Pretraining made those chances match real text, and real text is mostly true, so the model is usually right about things that are written about often.

But nothing in the loop checks facts. In pretraining, the model was rewarded for being a good continuation, not for being true. When it knows a fact well, the true continuation and the plausible one are the same. When it does not, it still produces a plausible one. Every step of the loop must output some token, and there is no separate step that stops when the model is unsure. "I don't know" only comes out if those words are themselves a likely continuation.

#Watch a fake reference being built

The chances below are invented to show the pattern. A model is asked for a study showing that houseplants improve exam scores, and suppose no such study is well known. Watch how sure it is at each step.

repeat:  chances = model(text so far)  token = sample(chances)  append token to text

What the model really knew was what a reference looks like. That is why the result is convincing. The facts inside were filled in like any other blank.

#The main causes

• Rare facts are stored weakly. A fact seen thousands of times in training gets sharp, confident chances. A fact seen once or twice, like a small company's founding year or a minor paper's author, leaves only a blurry trace. Specific details (names, numbers, dates, quotes, URLs) are the most at risk.
• Every step commits. Once a wrong token is written, it becomes part of the text the model reads. Later tokens stay consistent with it, so one bad guess can grow into a detailed, coherent false story.
• Guessing can be rewarded. If training or evaluation scores answers only as right or wrong, a confident guess sometimes scores and "I'm not sure" never does. Researchers point to this as one reason models guess rather than hold back.
• The answer is not in the text it has. Questions about your private documents, events after its cut-off, or anything it was never told cannot be answered from memory, so a plausible guess is all it can produce.
• False premises. "Why did the city ban bicycles in 2015?" assumes it happened. A good continuation of that question is an explanation, so the model may supply one.

To be accurate about what is known: research suggests models often carry some internal signal of how sure they are, and for simple facts that signal is partly reliable. Training can teach a model to say "I don't know" more often. But the signal does not reliably show in the wording, and no current method removes hallucinations completely. Treat any specific factual claim as unverified until you check it.

#What actually reduces it

The strongest fix is to stop asking the model to remember, and ask it to read instead. Put the source material in the prompt and ask for answers grounded in it.

The second prompt does three things: it supplies the facts, it asks for a quote you can check in seconds, and it gives the model an honest way out. Systems that look up documents automatically and add them to the prompt use the same idea; see RAG Basics. Letting the model run a search or a calculator through tools helps for the same reason.

The last line of defence is you. Check names, numbers, dates, quotes and links before you use them, as carefully as a mistake would cost. Verifying AI Output turns this into a routine.

#Common mistakes

#Try it yourself

#Check yourself

#What to practise next

The Prompting Basics lesson starts putting this module to work, and Verifying AI Output shows how to check answers quickly when the stakes are real.