Sakana AI· Sakana modelsResearch paper

DiffusionBlocks: Training Neural Networks One Block at a Time

Sakana AI introduces DiffusionBlocks, a method to train neural networks one block at a time by reinterpreting the forward pass as a diffusion process, reducing memory requirements while matching end-to-end performance across ViT, DiT, masked diffusion, autoregressive transformers, and recurrent-depth transformers. The paper was accepted at ICLR 2026.

From the source

We found a new way to break the network into blocks and train them independently. The trick? Treating the network’s forward pass like a diffusion model denoising a signal.

sakana.ai

Announced

27 May 2026 15:00 UTC

Detected

28 Aug 2026 21:48 UTC (+93d after)