Direct Preference Optimization (DPO) — AI Glossary

Direct preference optimization (DPO) is a training technique that aligns AI models with human preferences by learning directly from pairs of preferred and rejected outputs — without needing a separate reward model. Unlike RLHF which trains a reward model as an intermediate step, DPO simplifies the alignment process into a single supervised learning objective, making it faster and more stable to train.