04
Language Models
A large language model does one simple thing over and over: predict the next word. Trained on huge amounts of text, it learns which word is likely to come next in any context.
Build a sentence by picking words. The bars show the model's probability for each candidate next word. Choose one and watch it predict again.
The▋
model's guess for the next word
Pick a word to add it. The model only ever predicts the next word — stringing those predictions together is how it writes whole sentences.
Check yourself
If the model only ever predicts one word ahead, how does it write a whole paragraph?