Transfer Learning
A network trained on millions of images has already learned the universal groundwork — edges, textures, shapes in its early layers. Transfer learning reuses it: take the pretrained network as a backbone, keep those general layers, and retrain only the final layers on your own small dataset. Your few hundred labeled examples teach just the last distinction (cat breeds, defect vs fine) instead of re-deriving vision from scratch. Fine-tuning an LLM is this same idea — the pretraining did the expensive general learning once, and everyone downstream inherits it.
Slide the amount of labeled data. With a few hundred examples the pretrained curve crushes training from scratch; only near ten thousand does scratch catch up — a budget most projects never have.
With little data the gap is enormous: the pretrained backbone already knows edges, textures, and shapes — your examples only need to teach the final distinction.
Check yourself
Why are a pretrained vision model's early layers reusable for almost any image task, while its final layers usually must be replaced?
Go deeper (free): CS231n — Transfer learning notes ↗