论文标题
分类过度良性：可证明具有较大型号的反式标签噪声
Benign Overfitting in Classification: Provably Counter Label Noise with Larger Models
论文作者
论文摘要
良性过度拟合的研究为过度参数深度学习模型的成功提供了见解。在这项工作中，我们检查了过度拟合在现实世界分类任务中是否真正良性。我们首先观察到，重新网络模型在CIFAR10上过度良好，而不是在Imagenet上进行良性良性。为了理解为什么良性过度拟合在Imagenet实验中失败，我们理论上分析了在更限制的设置下良性过度拟合的，其中参数数量的数量不大于数据点的数量。在这种温和的过度参数设置下，我们的分析确定了相变的变化：与以前的重量过度参数设置不同，现在在存在标签噪声的情况下，良性过度拟合可能会失败。我们的分析解释了我们的经验观察，并通过一组具有重新NET的控制实验来验证。我们的工作突出了理解不足的制度作为未来方向的隐性偏见的重要性。
Studies on benign overfitting provide insights for the success of overparameterized deep learning models. In this work, we examine whether overfitting is truly benign in real-world classification tasks. We start with the observation that a ResNet model overfits benignly on Cifar10 but not benignly on ImageNet. To understand why benign overfitting fails in the ImageNet experiment, we theoretically analyze benign overfitting under a more restrictive setup where the number of parameters is not significantly larger than the number of data points. Under this mild overparameterization setup, our analysis identifies a phase change: unlike in the previous heavy overparameterization settings, benign overfitting can now fail in the presence of label noise. Our analysis explains our empirical observations, and is validated by a set of control experiments with ResNets. Our work highlights the importance of understanding implicit bias in underfitting regimes as a future direction.
