03
Attention
Attention lets a model look at every other word when processing one word, and decide how much each one matters. It is the core idea behind transformers and modern language models.
Click any word to make it the query. The lines fan out to every other word, and their thickness shows how much attention the query word pays to each.
Click any word in the bottom row to make it the query. Line thickness shows how much attention “sat” pays to each other word.
Check yourself
When a word is the query, why is its line to itself the strongest but hidden here?