From the source
Apple researchers prove that discrete diffusion models using per-position distributions (including remasking and uniform-state samplers) cannot match the training distribution when the tokens they write are dependent, and that per-position marginals alone cannot detect such dependence.
- On a synthetic task, they measure the generated distribution's total variation to be 29 times the sampling-noise floor.