论文标题
高斯噪声注射中的显式正则化
Explicit Regularisation in Gaussian Noise Injections
论文作者
论文摘要
我们研究了通过高斯噪声注射（GNIS）在神经网络中诱导的正则化。尽管将这种注射应用于数据时已被广泛研究，但很少有研究了解它们在应用于网络激活时诱导的正则化效果。在这里，我们得出了GNIS的显式正规机，通过将注入的噪声边缘化而获得，并表明它在傅立叶域中用高频组件对功能进行了惩罚；特别是在更接近神经网络输出的层中。我们在分析和经验上表明，这种正则化会产生具有较大分类边缘的校准分类器。
We study the regularisation induced in neural networks by Gaussian noise injections (GNIs). Though such injections have been extensively studied when applied to data, there have been few studies on understanding the regularising effect they induce when applied to network activations. Here we derive the explicit regulariser of GNIs, obtained by marginalising out the injected noise, and show that it penalises functions with high-frequency components in the Fourier domain; particularly in layers closer to a neural network's output. We show analytically and empirically that such regularisation produces calibrated classifiers with large classification margins.
