Sakana AI Trains 1,000-Layer Networks Without Backpropagation Using Layer-Local Feedback Controllers

Sakana AI / Hacker News (103pts, 37 comments)·medium signalXBlueskyLinkedInCopy link

Jeffrey Seely and Julian Gould published PC-ALM (Augmented Lagrangian Predictive Coding) on September 14, adding dual neurons as Lagrange multipliers to each layer so its local recurrence acts as a PI feedback controller. That turns backprop's forward-then-backward phase lock into layer-local dynamics that still distribute credit through 1,000-layer residual MLPs, nearly matching backprop where standard predictive coding decays away. In the linear limit the dual neurons converge to exact backprop credit signals using only local computation, which is the theoretical hook; the practical pitch is neuromorphic hardware where simulating dynamical systems is cheaper than a GPU.