World Models
An agent that must try every action in the real world learns slowly and expensively. A world model is the agent's internal simulator: a network trained to predict the next state given the current state and an action. With one, the agent can imagine rollouts — try thousands of action sequences in its head, keep the best, and only then act. The catch is compounding error: each predicted step feeds the next, so small mistakes snowball, and long imagined futures drift away from reality.
Drag the horizon slider. The solid line is the real environment; the dashed line is the model's imagined rollout from the same start. Short horizons track well; long ones drift into fiction.
Check yourself
Why is a world model's 10-step prediction so much worse than ten separate 1-step predictions?
Go deeper (free): World Models — interactive paper ↗