From the source
Lead story
Ask your AI
Top stories
Models & availability
Latest
Training 1000-layer networks without backpropagation
A local-learning method trains 1000-layer networks without backpropagation.
Sakana AI introduced PC-ALM, a local-learning alternative to backpropagation that trains 1000-layer neural networks using only local dynamics.
The method generalizes predictive coding with an augmented Lagrangian, introducing dual neurons (Lagrange multipliers) as part of layer-local dynamics, enabling signal propagation to arbitrary depth in deep narrow networks where standard predictive coding struggles.
From the source
Our method trains 1000-layer neural nets using only local dynamics, and without backprop.
sakana.ai
