From the source
Lead story
Top stories
Models & availability
Latest
Lead story
Top stories
Models & availability
Latest
From the source
New guidance method for diffusion language models
Apple researchers introduce probe guidance, a method to guide flow matching models that uses frozen internal states from existing diffusion models to construct guidance signals.
Applied to a 1.7B diffusion language model, probe guidance achieves state-of-the-art performance on unconditional generation and improves multiple choice question answering benchmarks while eliminating the need for additional forward passes at inference time.
From the source
Our approach, which we call probe guidance, uses the frozen internal states of an existing diffusion model to construct a guidance signal. This works using a similar principle as autoguidance, but eliminates the need for an additional forward pass at inference time and provides a reliable path to ensure that the weak and strong model share similar dynamics.
machinelearning.apple.com