论文标题
集体相关性标签用于通过
Collective Relevance Labeling for Passage Retrieval
论文作者
论文摘要
深度学习信息检索（IR）需要大量的高质量查询文档相关性标签，但此类标签本质上是稀疏的。标签平滑的将一些观察到的概率质量重新分布，通常是统一的，对真实分布的不知情。相比之下，我们提出了知识蒸馏以进行知情标记，而不会在评估时产生高度计算开销。我们的贡献是设计一个简单但有效的教师模型，该模型利用集体知识，超过了更复杂的教师模型的最先进。具体来说，我们比最先进的老师更快地训练X8，同时使排名更好。我们的代码可在https://github.com/jihyukkim-nlp/collactivekd上公开获取
Deep learning for Information Retrieval (IR) requires a large amount of high-quality query-document relevance labels, but such labels are inherently sparse. Label smoothing redistributes some observed probability mass over unobserved instances, often uniformly, uninformed of the true distribution. In contrast, we propose knowledge distillation for informed labeling, without incurring high computation overheads at evaluation time. Our contribution is designing a simple but efficient teacher model which utilizes collective knowledge, to outperform state-of-the-arts distilled from a more complex teacher model. Specifically, we train up to x8 faster than the state-of-the-art teacher, while distilling the rankings better. Our code is publicly available at https://github.com/jihyukkim-nlp/CollectiveKD
