论文标题
简单的技术对于神经网络测试优先和主动学习（可复制性研究）出乎意料地奏效
Simple Techniques Work Surprisingly Well for Neural Network Test Prioritization and Active Learning (Replicability Study)
论文作者
论文摘要
测试深度神经网络（DNN）的测试输入优先次数（TIP）是有效处理非常大的测试数据集的重要技术，可以节省计算和标记成本。对于大规模部署的系统尤其如此，其中记录了在生产中观察到的输入，以作为系统的下一个版本的潜在测试或培训数据。冯等人al。提议Deepgini是一个非常快速和简单的提示，并表明它的表现优于更精致的技术，例如神经元和惊喜覆盖范围。在一项大规模研究（4个案例研究，8个测试数据集，32'200训练的模型）中，我们验证了他们的发现。但是，我们还发现，不确定性量化领域的其他可比较甚至更简单的基线，例如预测的软磁性可能性或预测的软马克斯可能性的熵与deepgini的性能同样出色。
Test Input Prioritizers (TIP) for Deep Neural Networks (DNN) are an important technique to handle the typically very large test datasets efficiently, saving computation and labeling costs. This is particularly true for large-scale, deployed systems, where inputs observed in production are recorded to serve as potential test or training data for the next versions of the system. Feng et. al. propose DeepGini, a very fast and simple TIP, and show that it outperforms more elaborate techniques such as neuron- and surprise coverage. In a large-scale study (4 case studies, 8 test datasets, 32'200 trained models) we verify their findings. However, we also find that other comparable or even simpler baselines from the field of uncertainty quantification, such as the predicted softmax likelihood or the entropy of the predicted softmax likelihoods perform equally well as DeepGini.
