论文标题
DCT PESCEPTRON层：卷积层的转换域方法
DCT Perceptron Layer: A Transform Domain Approach for Convolution Layer
论文作者
论文摘要
在本文中，我们提出了一种新型的离散余弦变换（DCT）的神经网络层，我们称之为DCT-PERCEPTRON，以取代残留神经网络（RESNET）中的$ 3 \ TIMES3 $ CONS2D层。通过利用傅立叶和DCT卷积定理，使用元素乘积在DCT域中在DCT域中进行卷积过滤操作。可训练的软阈值层用作DCT感知的非线性。与Resnet的conv2d层相比，该层是空间不合时式和通道特异性的，所提出的层是特定于位置的且特定于通道的层。 DCT- perceptron层大大减少了参数和乘法的数量，同时保持了CIFAR-10和Imagenet-1K中常规重新NET的可比精度结果。此外，可以在常规重新NET中的全局平均池层作为附加层以提高分类精度之前的全局平均合并层，在全局平均池层之前，可以使用批处理层插入DCT- perceptron层。
In this paper, we propose a novel Discrete Cosine Transform (DCT)-based neural network layer which we call DCT-perceptron to replace the $3\times3$ Conv2D layers in the Residual neural Network (ResNet). Convolutional filtering operations are performed in the DCT domain using element-wise multiplications by taking advantage of the Fourier and DCT Convolution theorems. A trainable soft-thresholding layer is used as the nonlinearity in the DCT perceptron. Compared to ResNet's Conv2D layer which is spatial-agnostic and channel-specific, the proposed layer is location-specific and channel-specific. The DCT-perceptron layer reduces the number of parameters and multiplications significantly while maintaining comparable accuracy results of regular ResNets in CIFAR-10 and ImageNet-1K. Moreover, the DCT-perceptron layer can be inserted with a batch normalization layer before the global average pooling layer in the conventional ResNets as an additional layer to improve classification accuracy.
