论文标题
通过神经不可能训练的多种键形生成
Diverse Keyphrase Generation with Neural Unlikelihood Training
论文作者
论文摘要
在本文中，我们从多样性的角度研究了序列到序列（S2S）键形生成模型。神经自然语言产生的最新进展已通过改进诸如F1得分等质量指标的改进来证明，在键形生成的任务上取得了显着进步。但是，多样性在键形生成中的重要性在很大程度上被忽略了。我们首先分析了使用最大似然估计（MLE）训练的基线模型产生的输出中存在的信息冗余的程度。我们的发现表明，重复键形是MLE培训的主要问题。为了减轻这个问题，我们采用神经不可能（UL）目标来培训S2S模型。我们的UL培训版本在（1）目标令牌级别以阻止重复令牌的产生； （2）复制令牌级别，以避免从源文本复制重复令牌。此外，为了鼓励在解码过程中进行更好的模型计划，我们将k-step的前面标记预测目标纳入了未来令牌的MLE和UL损失。通过来自三个不同领域的数据集的广泛实验，我们证明了所提出的方法在保持竞争性产出质量的同时，获得了相当大的多样性增长。
In this paper, we study sequence-to-sequence (S2S) keyphrase generation models from the perspective of diversity. Recent advances in neural natural language generation have made possible remarkable progress on the task of keyphrase generation, demonstrated through improvements on quality metrics such as F1-score. However, the importance of diversity in keyphrase generation has been largely ignored. We first analyze the extent of information redundancy present in the outputs generated by a baseline model trained using maximum likelihood estimation (MLE). Our findings show that repetition of keyphrases is a major issue with MLE training. To alleviate this issue, we adopt neural unlikelihood (UL) objective for training the S2S model. Our version of UL training operates at (1) the target token level to discourage the generation of repeating tokens; (2) the copy token level to avoid copying repetitive tokens from the source text. Further, to encourage better model planning during the decoding process, we incorporate K-step ahead token prediction objective that computes both MLE and UL losses on future tokens as well. Through extensive experiments on datasets from three different domains we demonstrate that the proposed approach attains considerably large diversity gains, while maintaining competitive output quality.
