A Beginner’s Guide to AI Hallucinations

In the last post we learned that Large Language Models (LLMs), and tools based on them like ChatGPT, can "make up" information. We called this hallucination or confabulation. This article unpacks the core reasons behind the phenomenon, focusing on the simplest case: text‑only LLMs, meaning taking only text as input, and producing only text as output.

I try to keep this intentionally as simple and jargon-free as I can.

💡 LLMs use tokens, not words

You can think of an LLM as a "word‑guessing" machine. Instead of looking at whole words, it looks at tiny pieces of words, we call them tokens. A token can be a whole word (e.g., “is”, “and”), or a word fragment (e.g., “sup”, “‑ing”) so that even rare words can be handled by LLMs by putting multiple tokens together. When you ask an LLM to write something, it reads your request as a list of tokens, and then produces another list of tokens as a response. In paid services you usually pay for both the request and response tokens - that's the LLM currency, not words.

💡 How LLMs are actually trained

Training an LLM starts with a huge pile of human‑written text. That text is split into tokens. During training LLM is shown a piece of text with all but the last token, and it is asked to guess this last missing token. While guessing, the LLM keeps a guess‑score (called probability) for every token it could possibly produce. The guess‑score is simply a number that represents how confident the LLM thinks that each token is the right one for this spot in this particular context.

For example, suppose LLM sees "Merry Christmas and Happy New ___". The LLM looks at every token it could put in the blank (e.g., "Year", "Chair", "yellow", "go", "-market"). For each token it assigns a guess-score.

• If it is very confident, one token will get a score that's far higher than the others.
• If it's unsure, several tokens will have similar scores.

During training LLM adjusts its guess‑scores so that the correct token, in this case "Year", gets as high a score as possible, while all the others get as low a score as possible. It repeats this trillions of times on billions of texts, always learning which token is most likely to come next. When training is finished, we have a model that, hopefully, captures how language works. That internal language model is the statistical engine that powers the LLM.

💡 Why English works best with LLMs

LLMs see far more English text than any other language because most free text online is in English. So the guess scores for English are more accurate than for other languages, simply because it had much more chances to guess and adjust the scores for texts in English.

🎯 Why halucinations are a feature, and not always a bug

When an LLM needs to produce its own text, it simply uses its learned internal language model and tries to guess the next token, until it reaches a special stop‑token.

• If the internal language model is very confident, LLM will almost always pick the token with the highest score.
• If the internal language model is unsure, LLM may pick a token that looks plausible but is actually wrong.

You can influence this behavior with a setting called temperature: the closer to 0, the more the LLM favors the highest‑scoring token; the closer to 1, the more it explores other possibilities (though it still follows its internal language model).

In those moments when the internal language model is unsure, the wrong token becomes the seed for the rest of the sentence. The mistake can snowball, producing a whole paragraph that looks believable but is made up. You could call these hallucinations autocomplete-style.

If we look closer, the reason why this happens is that learned guess-scores are trained to fit a huge number of different sentences at once. In trying to satisfy all those contexts, the LLM settles on a sort of "average" set of scores that is good enough for many situations. Language itself is inherently uncertain and unpredictable - any given sentence can have many valid endings.

A helpful way to picture an LLM is as an average‑skilled expert: it often produces a reasonable answer, but it's far from infallible. Take, for example, the sentence "The average temperature in Paris in July is ___ degrees Celsius". Take a guess yourself! If you don't know (as I don't) any number between 20 °C and 35 °C would be almost equally likely. This is the kind of situation leading to LLM hallucinations.

Another factor to consider is that the LLM's token set includes tokens from many languages, but it has seen far fewer non‑English tokens than English ones. As a result, when you prompt the LLM in a language other than English, it can still produce English words, which is a form of hallucination that can be considered a bug.

However because language is inherently unpredictable, LLMs must rely on probability scores, and hallucination within a single language is an unavoidable side effect. Yet this very uncertainty is desirable, as it is what lets LLMs generate human‑like text simply by predicting the next token.

LLMs are sometimes called "autocomplete on steroids" - a description that would hold if we used only the LLMs themselves. In practice, LLMs serve as the foundation (hence the term "foundational models") for other tools such as ChatGPT. Those tools introduce additional sources of hallucinations, which we’ll explore in the next post.

Bonus for extra curious

The internal language model of the LLM is encoded in something called artificial neural network. To the best of my knowledge, the most accessible explanation of what it is and how it works comes from Geoffrey Hinton (who dedicated his life to research on artificial neural nets). Here are some quotes to get you started:

I was trying to come up with ideas about how the brain actually learned. And there're some things we know about that -- it learns by changing the strengths of connections between brain cells.

So if you think of it from the point of view of a neuron in the middle of the brain, a brain cell, all it can do in life is sometimes go ping. That's all it's got. Unless it happens to be connected to a muscle. It can sometimes go ping. And it has to decide when to go ping. There's other neurons going ping, and when it sees particular patterns of other neurons going ping, it goes ping.

You can think of this neuron as receiving pings from other neurons. And each time it receives a ping, it treats that as a number of votes for whether it should turn on, or it should go ping, or should not go ping. And you can change how many votes another neuron has for it. And you can change it by changing the strength of the connection.

The strength of the connection, think of as the number of votes this other neuron gives for you to go ping. It's very like political coalitions. There'll be groups of neurons that go ping together. And the neurons in that group will all be telling each other go ping. And then there might be a different coalition, and they'll be telling other neurons, don't go ping.

-- Geoffrey Hinton (discussion starts at roughly 5:27)