论文标题
颤抖的神经网络
Quiver neural networks
论文作者
论文摘要
我们通过介绍Quiver神经网络的概念来开发一种统一的理论方法来分析各种神经网络连通性体系结构。受箭量表示理论的启发，这种方法提供了一种紧凑的方法，可以捕获复杂的网络体系结构中精心的数据流。作为应用程序，我们使用参数空间对称性来证明一种无损模型压缩算法的箭量神经网络，其某些非重点激活称为重新激活。在径向重新激活的情况下，我们证明，梯度下降训练压缩模型等同于用投影梯度下降训练原始模型。
We develop a uniform theoretical approach towards the analysis of various neural network connectivity architectures by introducing the notion of a quiver neural network. Inspired by quiver representation theory in mathematics, this approach gives a compact way to capture elaborate data flows in complex network architectures. As an application, we use parameter space symmetries to prove a lossless model compression algorithm for quiver neural networks with certain non-pointwise activations known as rescaling activations. In the case of radial rescaling activations, we prove that training the compressed model with gradient descent is equivalent to training the original model with projected gradient descent.
