论文标题
使用可区分口罩的操作感知的软通道修剪
Operation-Aware Soft Channel Pruning using Differentiable Masks
论文作者
论文摘要
我们提出了一种简单但有效的数据驱动的通道修剪算法，该算法通过利用操作的特征来以可不同的方式压缩深层神经网络。所提出的方法是联合考虑批处理（BN）和用于通道修剪的整流线性单元（RELU）；它估计了两个连续的操作将每个特征图停用的可能性有多大，并以较高的概率修剪通道。为此，我们在整个优化过程中学习了各个渠道的可区分掩码，并在整个优化过程中做出软决策，这有助于探索更大的搜索空间并培训更稳定的网络。提出的框架使我们能够通过模型参数的联合学习和通道修剪识别压缩模型，而无需进行额外的微调程序。与最先进的方法相比，我们在输出网络的准确性方面进行了广泛的实验并取得了出色的性能。
We propose a simple but effective data-driven channel pruning algorithm, which compresses deep neural networks in a differentiable way by exploiting the characteristics of operations. The proposed approach makes a joint consideration of batch normalization (BN) and rectified linear unit (ReLU) for channel pruning; it estimates how likely the two successive operations deactivate each feature map and prunes the channels with high probabilities. To this end, we learn differentiable masks for individual channels and make soft decisions throughout the optimization procedure, which facilitates to explore larger search space and train more stable networks. The proposed framework enables us to identify compressed models via a joint learning of model parameters and channel pruning without an extra procedure of fine-tuning. We perform extensive experiments and achieve outstanding performance in terms of the accuracy of output networks given the same amount of resources when compared with the state-of-the-art methods.
