论文标题
表征过度参数化模型中标记数据的结构规律性
Characterizing Structural Regularities of Labeled Data in Overparameterized Models
论文作者
论文摘要
人类习惯于包含规律性和例外的环境。例如，在大多数加油站，一个付费之前的付款，但偶尔的乡村车站不提前接受付款。同样，深度神经网络可以概括共享共同模式或结构的实例，但有能力记住罕见或不规则形式。我们分析了如何通过一致性评分来处理模型的个人实例。得分表征了持有实例的预期准确性给定从数据分布采样的尺寸变化的训练集。我们在多个数据集中获得了单个实例的该分数的经验估计，我们表明该分数在连续体的一端识别出分布和标记的示例错误，另一端是强烈的定期示例。我们使用培训期间收集的统计数据确定计算廉价的一致性评分的代理。我们展示了对深度学习系统分析的潜在应用的示例。
Humans are accustomed to environments that contain both regularities and exceptions. For example, at most gas stations, one pays prior to pumping, but the occasional rural station does not accept payment in advance. Likewise, deep neural networks can generalize across instances that share common patterns or structures, yet have the capacity to memorize rare or irregular forms. We analyze how individual instances are treated by a model via a consistency score. The score characterizes the expected accuracy for a held-out instance given training sets of varying size sampled from the data distribution. We obtain empirical estimates of this score for individual instances in multiple data sets, and we show that the score identifies out-of-distribution and mislabeled examples at one end of the continuum and strongly regular examples at the other end. We identify computationally inexpensive proxies to the consistency score using statistics collected during training. We show examples of potential applications to the analysis of deep-learning systems.
