Foundations

RLHF: Reward Modeling, PPO, and the DPO Trade-off

How human preference data becomes a reward model, how PPO uses it to fine-tune an LLM, and why DPO often replaces the whole pipeline.

intermediate · 3 min read · Premium