论文标题
多任务检索授权的文本生成和相关抽样
Multi-Task Retrieval-Augmented Text Generation with Relevance Sampling
论文作者
论文摘要
本文研究了针对知识密集型任务的检索型生成模型的多任务培训。我们建议通过利用知识密集型一代的独特属性来清理设定的训练：查询 - 答案对与知识库中的项目的联系。我们通过对相关性标签的信心阈值过滤训练示例，无论一对对知识基础是否可以回答。我们在苏格兰语基准的七个组合任务上训练单个融合中的二十个发电机（FID）发电机。实验结果表明，我们简单而有效的方法基本上改善了两个强烈不平衡任务的基线。并显示其余任务的改进较小或没有重大回归。此外，我们通过相关性标签采样量表很好地展示了我们的多任务培训，并且具有增加的模型容量，并实现了最先进的训练，从而在七个苏格兰短裙任务中五个。
This paper studies multi-task training of retrieval-augmented generation models for knowledge-intensive tasks. We propose to clean the training set by utilizing a distinct property of knowledge-intensive generation: The connection of query-answer pairs to items in the knowledge base. We filter training examples via a threshold of confidence on the relevance labels, whether a pair is answerable by the knowledge base or not. We train a single Fusion-in-Decoder (FiD) generator on seven combined tasks of the KILT benchmark. The experimental results suggest that our simple yet effective approach substantially improves competitive baselines on two strongly imbalanced tasks; and shows either smaller improvements or no significant regression on the remaining tasks. Furthermore, we demonstrate our multi-task training with relevance label sampling scales well with increased model capacity and achieves state-of-the-art results in five out of seven KILT tasks.
