Train / Validation / Test
A model graded on questions it studied tells you nothing about how it handles new ones. So the data is split three ways before training begins: the training set teaches the model, the validation set steers your choices (which model size, when to stop, which settings), and the test set stays locked away until the very end, opened once for the final honest grade. The subtle trap is that tuning is itself a kind of learning — tune long enough against the validation set and you overfit to it too, and if the test set ever leaks into your decisions, your reported score becomes fiction.
Tune the model a few rounds: train accuracy climbs, validation peaks then slips. Then hit the cheat button and watch the reported test score inflate while true skill on genuinely unseen data stays flat.
Train teaches, validation steers your choices, test is opened once at the very end. Cheat with the test set and your reported score climbs while true skill does not — the exam leaked into the study notes.
Check yourself
You tried fifty model variants and picked the one with the best validation score. Why is even that score now a slightly dishonest estimate of real-world performance?
Go deeper (free): Google ML Crash Course — Dividing datasets ↗