论文标题
基于表示的复杂性度量，以预测深度学习的概括
Representation Based Complexity Measures for Predicting Generalization in Deep Learning
论文作者
论文摘要
深层神经网络仍可以概括性地过多兼容。最近的研究试图从各个角度检查这种现象，并根据这些观点（例如基于规范，基于PAC-Bayes和基于边缘的分析）提供有关概括错误或衡量概括差距的界限。在这项工作中，我们从深层神经网络的内部表示质量的角度提供了对概括的解释，这些神经科学理论是关于人类视觉系统如何创建不变且无与伦比的对象表示的。我们没有提供理论上的界限，而是展示了实践复杂性措施，可以在深层模型中计算临时性的概括行为。我们还提供了有关解决方案的详细描述，该说明赢得了神经竞争，以预测2020年Neurips在2020年举行的深度学习中的概括。我们的解决方案的实施可在https://github.com/parthnatekar/pgdl上获得。
Deep Neural Networks can generalize despite being significantly overparametrized. Recent research has tried to examine this phenomenon from various view points and to provide bounds on the generalization error or measures predictive of the generalization gap based on these viewpoints, such as norm-based, PAC-Bayes based, and margin-based analysis. In this work, we provide an interpretation of generalization from the perspective of quality of internal representations of deep neural networks, based on neuroscientific theories of how the human visual system creates invariant and untangled object representations. Instead of providing theoretical bounds, we demonstrate practical complexity measures which can be computed ad-hoc to uncover generalization behaviour in deep models. We also provide a detailed description of our solution that won the NeurIPS competition on Predicting Generalization in Deep Learning held at NeurIPS 2020. An implementation of our solution is available at https://github.com/parthnatekar/pgdl.
