PCA & Dimensionality
Real data has too many dimensions to see or store efficiently, but most of them barely vary. PCA (Principal Component Analysis) finds the directions along which the data actually spreads out — the principal components — and lets you keep just the top few, discarding the rest with minimal loss. Project a 2D cloud onto its best single axis and one number per point preserves almost all the structure. This is the same idea behind flattening 768-dimensional embeddings into a 2D scatter you can look at.
Rotate the axis and watch two things: how far apart the projected points stay on the 1D strip, and the variance-captured percentage. The best angle — the first principal component — is the direction the cloud is longest.
This is (nearly) the first principal component — the direction of maximum variance. One number per point now tells almost the whole story.
Check yourself
After projecting onto the best axis, two originally different points can land on the same spot. What information was in the direction PCA threw away?
Go deeper (free): Setosa — PCA explained visually ↗