Sakana AI researchers Jeffrey Seely and Julian Gould published a method that trains residual multilayer perceptrons up to 1,000 layers deep without backpropagation, the credit-assignment algorithm that underlies nearly all of modern deep learning. They call it Augmented Lagrangian Predictive Coding, or PC-ALM, and report it stays within roughly two percentage points of backprop’s test accuracy on MNIST across the full depth range tested, according to the Sakana AI write-up.
Backpropagation trains a network in three locked phases: a forward pass, a backward pass, and a weight update, with every layer waiting on its neighbors before it can act. Neuroscientists have long noted that biological brains have no obvious mechanism for that kind of network-wide synchronization, which is why predictive coding, a decades-old framework rooted in Hermann von Helmholtz’s theories of perception, has stayed attractive as a candidate for how cortex might actually compute. Standard predictive coding lets each layer act as a local dynamical system that passes only its prediction error to its neighbor, avoiding backprop’s synchronization requirement entirely.
That local approach has a known weakness. Sakana AI’s authors describe a “signal decay problem” in standard predictive coding: supervision entering at the output layer has to propagate backward through a chain of local compromises, and when a network is both deep and narrow, that credit signal fades before the early layers ever see it. Prior predictive coding work, cited in the post, managed only 128-layer networks and needed wide layers of 512 neurons to do it.
PC-ALM’s fix borrows a tool from distributed optimization rather than neuroscience. Each layer gets a second set of local variables, Lagrange multipliers, turning its update rule into what the authors describe as a proportional-integral feedback controller. The Sakana AI team traces the idea to Yann LeCun’s 1988 paper showing that a constrained network’s Lagrange multipliers correspond to its backpropagation credit signals at equilibrium. PC-ALM combines that observation with predictive coding’s existing local error terms, and the authors show algebraically that in linear networks, the accumulated multipliers converge to the exact backprop gradient using only communication between adjacent layers.
The practical result is what the researchers call “ballistic” credit propagation: instead of decaying diffusively through the network the way standard predictive coding’s signal does, PC-ALM’s credit wavefront advances through all 1,000 layers within a training-step budget that scales linearly with depth. On CIFAR-10 and Tiny ImageNet with ResNet-18, the authors report PC-ALM improves on standard predictive coding at every depth tested, with the gap widening as networks get deeper.
Sakana AI frames the motivation as scientific rather than immediately commercial: understanding how a distributed system without global synchronization, such as the brain, could compute something functionally equivalent to a gradient. The authors also flag a possible engineering payoff on neuromorphic hardware, where simulating local dynamical systems is reportedly cheaper than running GPU-style forward and backward passes, though they do not provide benchmark numbers for that claim. They note a trade-off too: a related predictive coding property called “prospective configuration,” shown elsewhere to improve sample efficiency over backprop, appears to be given up in PC-ALM at convergence.
This is a research write-up making its own claims, not an independently verified result, and the authors describe it as tested only on simple tasks (Fashion-MNIST, CIFAR-10, Tiny ImageNet scale) rather than anything approaching frontier-model training. For teams following alternatives to backprop for energy-efficient or neuromorphic inference, PC-ALM is now a concrete baseline to benchmark against rather than a shipped training method, and the open question is whether the local-dynamics advantage survives contact with the scale and self-supervised objectives used in production model training.
Sakana AI, “Augmented Lagrangian Predictive Coding,” published by Jeffrey Seely and Julian Gould, September 2026.
