论文标题
主动测试：一种公正的评估方法，用于远距离监督关系提取
Active Testing: An Unbiased Evaluation Method for Distantly Supervised Relation Extraction
论文作者
论文摘要
遥远的监督是一种广泛使用的方法，用于自动标记数据集，用于神经关系提取。但是，现有的有关远距离监督关系提取的作品遭受了低质量测试集的影响，这导致了相当大的偏见性能评估。这些偏见不仅会导致不公平的评估，还误导了神经关系提取的优化。为了减轻此问题，我们通过利用嘈杂的测试集和一些手动注释，提出了一种名为主动测试的新型评估方法。广泛使用基准的实验表明，我们提出的方法可以对远距离监督的关系提取器产生近似无偏见的评估。
Distant supervision has been a widely used method for neural relation extraction for its convenience of automatically labeling datasets. However, existing works on distantly supervised relation extraction suffer from the low quality of test set, which leads to considerable biased performance evaluation. These biases not only result in unfair evaluations but also mislead the optimization of neural relation extraction. To mitigate this problem, we propose a novel evaluation method named active testing through utilizing both the noisy test set and a few manual annotations. Experiments on a widely used benchmark show that our proposed approach can yield approximately unbiased evaluations for distantly supervised relation extractors.
