Fine-Tuning
A base model only predicts the next token, so it tends to continue or echo a prompt rather than answer it. Fine-tuning turns that raw predictor into something that follows instructions.
First, instruction tuning trains the model on many prompt-and-good-answer pairs, so it learns to respond rather than ramble. Then preference tuning (RLHF) shows the model pairs of answers with a human or reward signal marking the better one, nudging its behaviour toward helpful, aligned replies. Knowledge comes from pretraining; behaviour comes from fine-tuning.
prompt: How do I reset my password?
How do I reset my password? How do I change my email? How do I delete my account? How do I...
Variants & real-world flavors
The alignment pipeline has stages. SFT (supervised fine-tuning) teaches format and behaviour from example conversations — the base model learns to answer rather than continue. RLHF then optimises against human preferences: people rank pairs of answers, a reward model learns the ranking, and reinforcement learning pushes the model toward it — the recipe behind ChatGPT. DPO gets most of the same effect without the reinforcement-learning machinery, tuning directly on the preference pairs, and became the open-weight default for its simplicity. RLAIF swaps human raters for AI feedback guided by written principles (constitutional AI). Modern pipelines also add RL on verifiable rewards — math and code where correctness is checkable — which is what trains reasoning models.
Check yourself
The base and tuned models hold the same facts. So why does only the tuned one actually answer the question?
Go deeper (free): Hugging Face LLM Course — Fine-tuning ↗