Explain easily the way how LLMs learn before they begin inference.
ㅡㅡ
Before an LLM begins answering us, it has already gone through a very large learning process called training.
The easiest way to understand it is to imagine a student playing a gigantic “guess the next word” game.
1. Give the LLM enormous amounts of text
During training, the model processes huge amounts of written material. For example, it might encounter:
The Earth goes around the Sun.
But the computer does not experience the Earth or Sun directly. It receives tokens representing the text.
This is important in terms of our recent discussion:
Human: WORD ↔ THOUGHT ↔ THING
Basic LLM training: principally WORD ↔ WORD
The model learns from relations represented in its training data rather than experiencing the things described.
2. Hide the next token and make it predict
Imagine that the training system gives:
The Earth goes around the ____
The model has to predict what comes next.
At an early stage, it might produce probabilities such as:
Moon — 30%
world — 20%
Sun — 10%
house — 5%
…
The training system knows that the actual next token is Sun.
So the model learns:
“My prediction was poor. I should make Sun more probable in contexts like this.”
3. Change the internal numerical weights
An LLM contains enormous numbers of adjustable numerical values called parameters or weights.
After prediction, training measures the error. A mathematical procedure then changes many weights by tiny amounts so that the model becomes somewhat better at prediction.
Very simply:
context → prediction → compare with actual text → calculate error → adjust weights
This happens again and again—on an enormous scale.
4. It gradually learns contextual relationships
Now consider:
He deposited his money at the bank.
and:
He sat beside the river bank.
The word bank occurs in both. But its neighboring words differ.
Across enormous amounts of training text, the model learns that:
money, deposit, account, loan
tend to occur in one kind of context,
while:
river, water, fishing, shore
tend to occur in another.
It therefore learns something resembling different meanings of bank through what we have called statistical context.
This is why context is so fundamental to LLM learning.
5. Something surprising emerges
To become very good at predicting language, the model cannot learn only which words commonly stand beside other words.
Consider:
Paris is the capital of France.
Tokyo is the capital of Japan.
Seoul is the capital of South Korea.
Across enormous amounts of text, it learns deeper regularities such as:
CITY — is capital of — COUNTRY
It similarly learns patterns involving people, actions, causes, time, grammar, arguments, stories, computer programs, mathematics, and much else.
Thus simple next-token training produces surprisingly complicated internal representations.
This is one reason LLMs seem much more intelligent than the phrase “next-word prediction” suggests.
6. Then training stops
After training, the learned weights are largely fixed.
Now I receive a new prompt such as:
“Explain metaphor easily.”
I do not normally retrain myself on that sentence. Instead, I use the patterns already represented in the trained weights together with the new context you provide.
That is inference:
trained model + present context → probabilities for next token → selected token → repeat
So there are two importantly different stages:
TRAINING: learn statistical relationships from enormous amounts of previous data.
INFERENCE: use those learned relationships plus the present context to generate a new response.
The most interesting point for our discussion
Training turns enormous amounts of previous verbal context into numerical relationships within the model.
We could describe the process very roughly as:
human WORDS in many contexts
↓
statistical learning
↓
weights representing learned relationships
↓
new WORDS + new context
↓
inference
↓
new WORDS
And here lies both the power and weakness of LLMs.
Much human knowledge about THOUGHT and THING has left traces in WORD. Therefore learning WORD in enormous contextual depth indirectly teaches the model a great deal about thoughts and the world.
But:
learning what humans have written about THING is not identical to observing THING.
That distinction helps explain why an LLM can give an impressively coherent answer and nevertheless sometimes give a false one. The statistical relationships learned from WORD do not by themselves guarantee correspondence with THING.
That is exactly where our earlier discussion of context, evidence, and hallucination becomes important.
ㅡㅡㅡㅡ
2026-08-24 Mark Park
Our Library 