The Transformer Block
You have now seen every ingredient — embeddings, positions, attention, small neural nets. The transformer block is how they are assembled into one repeatable unit: normalise, attend, add the result back, normalise, run a feed-forward net, add back again. A modern LLM is just this block stacked 30 to 100+ times.
Step through the block to follow one token's journey. Then toggle residuals off: those green shortcut paths carry the original signal past each sub-layer, so a layer only has to learn the CHANGE it wants to make and gradients can flow straight down a deep stack. Remove them and very deep networks simply fail to train.
The token arrives as a vector with its position signal already added.
Check yourself
Attention mixes information ACROSS tokens; what does the feed-forward network act on, and why does the block need both?
Go deeper (free): 3Blue1Brown — Transformers, visually ↗