论文标题
步态螺纹：从错误反向传播中得出的生物学上合理的学习规则
GAIT-prop: A biologically plausible learning rule derived from backpropagation of error
论文作者
论文摘要
传统的错误反向传播，尽管在人工神经网络模型中学习了一种非常成功的学习算法，但具有在实际神经回路中学习在生物学上令人难以置信的功能。一种称为目标传播的替代方案提议通过使用自上而下的神经活动模型来解决这种不可使用的性能，以将神经网络的输出的误差转换为层次，并为每个单元转换出合理的“目标”。然后，这些目标可用于产生网络培训的重量更新。但是，到目前为止，启发式提出了目标传播，而没有明显的等效性与返回传播。在这里，我们得出了反向传播和目标传播形式（步态 - 普罗普）之间的确切对应关系，其中目标是向前通行证的小扰动。具体而言，当突触重量矩阵正交时，反向传播和步态 - 可以提供相同的更新。在一系列简单的计算机视觉实验中，我们通过柔软的正交性诱导正规器显示了反向传播和步态prop之间几乎相同的性能。
Traditional backpropagation of error, though a highly successful algorithm for learning in artificial neural network models, includes features which are biologically implausible for learning in real neural circuits. An alternative called target propagation proposes to solve this implausibility by using a top-down model of neural activity to convert an error at the output of a neural network into layer-wise and plausible 'targets' for every unit. These targets can then be used to produce weight updates for network training. However, thus far, target propagation has been heuristically proposed without demonstrable equivalence to backpropagation. Here, we derive an exact correspondence between backpropagation and a modified form of target propagation (GAIT-prop) where the target is a small perturbation of the forward pass. Specifically, backpropagation and GAIT-prop give identical updates when synaptic weight matrices are orthogonal. In a series of simple computer vision experiments, we show near-identical performance between backpropagation and GAIT-prop with a soft orthogonality-inducing regularizer.
