Autoencoders
An autoencoder is a network trained on the simplest possible objective: output what you were given. The trick is the shape — an encoder squeezes the input through a narrow bottleneck, and a decoder rebuilds it from that code. Because the bottleneck is far smaller than the input, perfect copying is impossible; the network is forced to keep only what matters, discovering a compressed representation with no labels at all. That latent space is the payoff: it powers compression, denoising, anomaly detection (things that reconstruct badly are unusual), and — as the VAE — the latent space that Stable Diffusion generates inside.
Shrink the bottleneck and watch the reconstruction triage: broad structure survives longest, fine details (the eyes, the mouth) vanish first. Wrong-pixel count shows the price of each squeeze.
Check yourself
Nobody labels which pixels are important, yet the network consistently sacrifices fine detail before broad structure. What in the training objective forces that priority?
Go deeper (free): Lilian Weng — From Autoencoder to Beta-VAE ↗