Hugging Face· PlatformResearch paper

Preference Tuning LLMs with Direct Preference Optimization Methods

This blog post empirically evaluates three preference tuning methods (DPO, IPO, KTO) on two 7B LLMs, sweeping hyperparameters and evaluating with MT-Bench.

From the source

We evaluate three promising methods to align language models without reinforcement learning (or preference tuning) on a number of models and hyperparameter settings. In particular we train using different hyperparameters and evaluate on: Direct Preference Optimization (DPO) Identity Preference Optimisation (IPO) Kahneman-Tversky Optimisation (KTO)

huggingface.co

Announced

18 Jan 2024 00:00 UTC

Detected

03 Aug 2026 21:12 UTC (+928d after)