论文标题
语言模型的即时信息检索增强
On-The-Fly Information Retrieval Augmentation for Language Models
论文作者
论文摘要
在这里，我们尝试使用信息检索作为预训练语言模型的增强。信息检索中使用的文本语料库可以视为随着时间的流逝而增长的情节记忆形式。通过通过信息检索增强GPT 2.0，我们在吉格沃德语料库中相对减少了15％的相对减少，而没有任何重新训练。我们还验证了事件共同参考任务的IR扩展。
Here we experiment with the use of information retrieval as an augmentation for pre-trained language models. The text corpus used in information retrieval can be viewed as form of episodic memory which grows over time. By augmenting GPT 2.0 with information retrieval we achieve a zero shot 15% relative reduction in perplexity on Gigaword corpus without any re-training. We also validate our IR augmentation on an event co-reference task.
