Here is the entire mechanism in one sentence:
A large language model takes the text so far and predicts what token is most likely to come next — then does it again, with its own output added to the input.
That is it. Every capability and every failure follows from that loop.
Watching it work
Give a model the text The capital of France is.
It does not look up France in a table. It computes a probability for every token it knows:
| Candidate next token | Probability |
| --- | --- |
| Paris | very high |
| located | low |
| a | low |
| banana | vanishingly small |
One is chosen. It is appended. Now the input is The capital of France is Paris and the process repeats to produce the next token — perhaps ., perhaps , which.
Generation is this loop, running until the model produces a token meaning "I am finished" or hits a length limit.
Why this explains so much
Why it is fluent. Fluency is precisely what it optimises. It was trained on enormous quantities of text to predict continuations that look like the real thing. Sounding right is the objective.
Why it can be confidently wrong. Nothing in that loop checks truth. The model produces the most plausible continuation. Plausible text and true text overlap heavily — which is why it works at all — but they are not the same thing, and where they diverge you get fluent nonsense.
Why the same prompt varies. The next token is usually sampled from the distribution rather than always taking the top choice. That randomness is a setting, often called temperature. Low temperature means "almost always take the most likely token" — repeatable, safer, sometimes flat. Higher means more variety and more risk.
Why prompting works at all. The only lever you have is the text that comes before. Everything in prompt engineering is a way of making the desired continuation the most probable one. Asking for "a summary for a non-technical manager" shifts the distribution towards text that looks like exactly that.
Why it cannot count reliably. Nothing in next-token prediction does arithmetic. It produces the token that usually follows in text like this. Sometimes that is correct. Sometimes it is confidently, precisely wrong.
The word "large"
The models that behave this well are large in three ways at once: many parameters (the tuned weights), enormous training data, and substantial compute. Scaling all three produced abilities that smaller versions simply do not show.
But scale did not change the mechanism. A frontier model and a tiny one are doing the same thing. One is far better at it.
What this earns you
Once you internalise "it predicts the next token", you stop being surprised. You expect confident errors on obscure facts. You expect it to invent a citation that looks perfect. You expect that giving it better context produces better output.
And you stop asking it whether it is sure — because you know that the answer to that question is itself just predicted text.