Context Window
A model has no memory between requests — it only sees the tokens currently in its context window, its working memory. Everything you have typed, every document pasted, every reply so far must fit inside it. Anything that falls out simply stops existing for the model.
Shrink the window and watch older messages fall away. At some point the peanut allergy (line 3) drops out — and the model, now blind to it, could cheerfully suggest peanut chikki. Note the cost readout too: attention compares every token to every other, so work grows with the SQUARE of the window. The KV cache softens this by storing each token's key/value vectors so only the newest token's attention is computed fresh.
✓ Model can still see the peanut allergy → suggests a safe snack.
attention cost ∝ window² : 8² = 64 comparisons
Doubling the window quadruples the attention work — why long context is expensive, and why models cache each token's key/value vectors (the KV cache) instead of recomputing them for every new token.
Check yourself
Why does a chatbot sometimes ‘forget’ something you told it early in a very long conversation?
Go deeper (free): IBM — What is a context window? ↗