Sakana AI’s Error Diffusion Trains Dale-Compliant Dual-Stream Networks, Reaching 96.7% MNIST and 61.7% CIFAR-10 Without Backpropagation
Backpropagation dominates deep studying, but it makes use of a mechanism the mind probably can’t. Specifically, the backward cross wants actual transposes of ahead weight matrices. This is the weight transport drawback. Sakana AI’s new paper, Diffusing Blame, confronts this constraint immediately. The analysis staff trains networks that obey Dale’s precept whereas avoiding weight transport…
