LLMs don't hallucinate

A test with nowhere to hide

We took a nine-billion-parameter model and gave it one job. Read a short customer review. Say whether it is positive or negative. Answer with a single digit, 1 or 0. We wrote 1,050 reviews, a third of them deliberately wishy-washy ("The hotel was fine, I guess"), because the interesting cases are the close calls. Here is a real question and a real dead heat.

Same question, same answer

The same sum in a different order

The machine never wavers, so the variety has to come from somewhere else. The first place to look is the arithmetic.

In January 1982 the Vancouver Stock Exchange launched an index at 1,000. Twenty-two months later it stood at 524. Nobody could explain it, because the market had not fallen. The index was recalculated after every trade. Each time, the computer dropped everything past the third decimal, thousands of times a day. Each drop was worth nothing. Together they had eaten half the index. Nothing random had happened; the rounding had simply happened at different moments.

Who else is in the queue and how long the conversation runs

In 2025 a research lab asked a large model "Tell me about Richard Feynman" a thousand times with every random setting switched off. It got eighty different answers. All thousand were identical for the first hundred-odd words, then parted, over whether Feynman was born in "Queens, New York" or "New York City". The lab traced the cause to the queue. A busy service does not answer one question at a time; it lays many requests side by side and works through them together. Nothing in the other requests can reach a user's own, but the sheet is a different shape, so the sums are added in a different order and the last digits move. We ran the same 256 reviews alone, then in groups of two, four, eight and sixteen, then in groups of eight with different companions.

The dice

Now to the thing every user of a chatbot has actually experienced. The one-digit question skipped a step. The model does not choose a word, or even a digit. It holds up 151,936 scorecards and stops. Something else has to pick. Taking the top card needs no dice. It is what every experiment above did. Almost every chatbot in use rolls instead.

There are three reasons to add randomness to a machine whose great virtue is that it has none. First, the top card is not the whole answer: the model was trained to spread its bets. Always taking its favourite throws the rest of its judgement away. In long stretches of writing this makes the text repeat itself, a problem researchers at the University of Washington named "neural text degeneration" in 2019 before proposing the fix most products still use. Second, variety is useful; a user who asks for five headlines wants five different ones. And third, the reason people suspect: it feels right. Consider what Apple learned in 2005. The iPod's shuffle picked songs genuinely at random. Listeners hated it, because genuine randomness produces runs: the same artist three times in a row, the same album twice in ten minutes. It felt broken. So Apple added a rule that steered the picks away from recent repeats. Steve Jobs said from the stage that it made shuffle less random, even though people would think it more random. The randomness people enjoy is a designed impression of variety, tuned until it feels natural and adjusted when it doesn't. It has little to do with the mathematical kind. A chatbot's dice are the same kind of thing: the temperature is set where replies feel lively and a penalty on repeated words does for sentences what Apple's rule did for songs. The machine underneath has no such property. The dice are a setting, chosen for how it feels. Turing knew it.

"It would deliberately introduce mistakes in a manner calculated to confuse the interrogator."Alan Turing, Computing Machinery and Intelligence, 1950

The dice went in for the first two reasons. Turing was right about the third anyway. In 1966 Joseph Weizenbaum's secretary, who had watched him write ELIZA and knew it was a few hundred lines of pattern-matching, asked him to leave the room so she could talk to it in private. We are easily convinced.

The scorecard has no truth column

In the 1970s the neuroscientist Michael Gazzaniga worked with patients whose two brain hemispheres had been surgically separated. He showed one such patient two pictures at once, each visible to only one side of the brain: a chicken claw to the side that speaks, a snow scene to the side that does not. Asked to pick matching cards, the patient's hands did the right thing, a chicken for the claw and a shovel for the snow. Then Gazzaniga asked why the shovel. The speaking half of the brain had never seen the snow. It answered without a flicker: to clean out the chicken shed. Fluent, confident, invented and to the patient indistinguishable from a real reason. Gazzaniga called it the interpreter. It is worth knowing about because a system built to produce the most plausible next thing will produce one whether or not it knows.

The model's job was only ever to hold up the most likely scorecard given everything it has read. When what it has read contains the answer, the top card is the answer, every time. When it does not, the top card is still the most plausible-sounding thing, every time, produced by exactly the same procedure. Nothing inside distinguishes the two cases; there is no column on the card for true. Instruments that read a model's internal state, sparse autoencoders (SAEs) and the Jacobian lens (J-lens) among them, can show which features fired and at which layer a verdict settled. Those readings differ between a name the model knows and one it is inventing. But that is a measurement taken from outside. The card itself still has no truth column. That is what a hallucination is: the correct answer to the question the machine was actually asked.

We asked this model a question this page has already answered. First bare. Then with the Vancouver paragraph from earlier on this page placed in front of it.

Why a chatbot gives different answers

Put it together. In almost every hosted product, the dice are on: temperature somewhere around 0.7, seed from the clock, every reply a fresh roll. That alone accounts for most of what users see. Turn the dice off and the quieter things remain, the saved subtotals and the queue, which move last digits. Last digits decide photo finishes and a paragraph is a long row of them. This is why a service will promise "mostly" the same answer even at its most careful setting and why "mostly" is the honest word.

And some things change the machine itself, which is not randomness at all: a new version of the model behind the old name, an updated hidden instruction or today's date slipped into the prompt, a committee model (a mixture of experts, MoE) whose small router picks a specialist for each word, so a dead heat there is a dead heat inside the model, an assistant model drafting words for the big one to check, a different chip adding in a different order. Every one of these is a different machine answering, or a different question asked.

Who holds which lever depends on where the model runs. An open model, one whose weights are published so that it can be run on the user's own machine or servers, puts every lever except the training data in the user's hands: the dice can be switched off, the queue is the user's own and the version changes only when the user changes it. A hosted service, including the frontier models reached through an API or a chat window, keeps the middle column for itself and exposes some of the first, on its own terms. Here is the whole set, sorted by who holds them.

So when a chatbot gives a different answer to the same question, there are only a few places to look. We could not make the model waver. The last digits depend on how the work was arranged. They can only tip a decision the model had already left as a dead heat. Temperature and the seed are the dice. The filing of faces is top-p and its cousins. Even the dice are a calculation, fed a starting number. And when the answer is wrong rather than different, that is the scorecard, not the dice: the machine did what it always does with what it was given. Different answers are a setting. Wrong answers are a supply problem. Both are under the user's control, wholly with an open model and partly with a hosted one. The first can be switched off.

So who is hallucinating?

Nobody. A hallucination, in the sense the word has carried since Esquirol defined it in 1817, is a perception with nothing behind it. The model perceives nothing and believes nothing; it has no column for true. It cannot hallucinate. It answered the question it was actually asked, which is what usually comes next. Given its training and what was in front of it, that answer was the exact right output. It will be the same output every time. The first half of this page measured nothing else.

The confusion is on the other side of the screen. The question typed was "what is so?" The question the machine answers is "what usually follows?" When the two agree, because the source was in front of it, the answer is right. When nothing was in front of it, the answer is the most plausible-sounding thing, delivered with exactly the same confidence. A person reads that confidence as knowledge. That is a misunderstanding of what was asked and of what came back. Weizenbaum's secretary, with better grammar.

The word describes the expectation, not the output. Change the expectation, or change what is in front of the machine. The second is easier.