Cost & Energy
AI feels free in a chat window, but every token is a forward pass through billions of weights on power-hungry GPUs. At scale that becomes real money and real electricity — the economics that quietly shape which models get built and which get deployed.
Set a usage pattern and switch tiers. Two things to notice: cost scales with tokens × model size, and the gap between a small and a frontier model is roughly 50× per token. That gap explains modern architecture choices — route easy queries to cheap models, cache aggressively, and apply every trick from this course (quantization, distillation, speculative decoding) to squeeze the per-token price. The numbers are illustrative, but the proportions are the point.
tokens / month: 1.2M
cost: ₹480 / month
energy: 7.2 kWh ≈ 480 phone charges
Every technique in this course — quantization, distillation, speculative decoding, LoRA — exists mostly to push these two numbers down.
Figures are illustrative orders of magnitude, not live prices.
Check yourself
Your app answers 90% simple questions and 10% hard ones. How would you design it to cut cost by 10× without hurting quality?
Go deeper (free): Epoch AI — Trends in ML compute ↗