← All lessons
05

Training

Training is how a model learns. It makes a guess, measures how wrong it was, then adjusts its weights to be a little less wrong next time. Repeat millions of times and the model gets good.

The curve is the error. Take steps to roll the ball downhill toward the lowest point. Change the learning rate to see it crawl when small, or overshoot and bounce when too large.

lowest error
learning rate0.30
step 0 · error 20.4

Each step nudges the ball downhill toward lower error. Small learning rate crawls; too large overshoots and bounces. Backpropagation is what computes which way is downhill for every weight.

How it works

Training runs a loop. A forward pass makes a prediction, then a loss measures how far off it was. Backpropagation then works backward through the network, using the chain rule to compute a gradient for every weight — a number saying which direction and how much to change it to reduce the error. Each weight is nudged in that direction by an amount set by the learning rate. The gradient picks the direction downhill; the learning rate picks how big a step to take. Too small and training crawls; too large and it overshoots the minimum and bounces.

Check yourself

What happens to the ball when the learning rate is set very high?

Next: Layers