medium10 pts · first answer countstransformersfine-tuningrlhf

How DPO differs from PPO-based RLHF

Direct Preference Optimization (DPO) became popular as an alternative to PPO-based RLHF. What is the key difference?

A.DPO needs human ratings on a 1–10 scale instead of pairsB.DPO trains the reward model but never updates the chat modelC.DPO can only be used during pretraining, not after SFTD.DPO learns from preference pairs with no reward model or RL loop

Discussion 0

Join free to post in the discussion.

How DPO differs from PPO-based RLHF

Discussion 0

Related questions