Language Models
A large language model does one simple thing over and over: predict the next word. Trained on huge amounts of text, it learns which word is likely to come next in any context.
Build a sentence by picking words. The bars show the model's probability for each candidate next word. Choose one and watch it predict again.
model's guess for the next word
Pick a word to add it. The model only ever predicts the next word — stringing those predictions together is how it writes whole sentences.
How it works
A large language model is a transformer trained on one task: given some text, predict the next token. Every word is turned into an embedding, attention lets the words share context, and a final layer produces a probability for every token in the vocabulary — the bars you see. To write more than one word, the model appends its choice to the text and runs again, predicting the next token, then the next; this loop is called autoregressive generation. That single next-token objective, repeated over a huge amount of text, is enough to make the model absorb grammar, facts, and reasoning patterns as a side effect. At generation time you can shape the output: a setting called temperature controls how adventurous the choice is — low temperature almost always takes the top bar and reads as safe and predictable, higher temperature samples further down the bars for more variety. After this base training, models are usually fine-tuned to follow instructions and to be helpful, which is what turns a raw next-word predictor into a chat assistant.
Check yourself
If the model only ever predicts one word ahead, how does it write a whole paragraph?
Go deeper (free): Karpathy — Intro to Large Language Models (video) ↗