Loss Functions
Training needs a single number that says how wrong the model is — the loss. Gradient descent can only roll downhill on whatever curve this number defines, so the choice of loss function shapes what the model learns. The two workhorses: MSE (squared distance, for predicting numbers) and cross-entropy (for predicting probabilities).
The truth is a cat; the slider is the model's predicted probability of cat. Both loss curves are drawn over every possible prediction. Drag toward zero — confidently wrong. MSE flattens out near 1.0, but cross-entropy shoots toward infinity, punishing confident mistakes hard. That steep wall is exactly why classifiers train with cross-entropy: the gradient stays strong where the model most needs correcting.
Drag toward 0 to see why classifiers use cross-entropy.
Check yourself
Why is a loss that saturates (goes flat) on confident mistakes bad for learning?
Go deeper (free): Google ML Crash Course — Loss ↗