论文标题
定向光束搜索：插件词汇限制语言生成
Directed Beam Search: Plug-and-Play Lexically Constrained Language Generation
论文作者
论文摘要
大型预训练的语言模型能够生成逼真的文本。但是，控制这些模型以使生成的文本满足词汇约束，即包含特定的词，这是一个具有挑战性的问题。鉴于最先进的语言模型太大了，无法在可管理的时间内从头开始训练，因此希望在不重新训练的情况下控制这些模型。能够执行此操作的方法称为插件。最近的插件方法已成功地限制了具有限制搜索空间（例如机器翻译）任务中的小型双向语言模型以及向前模型。但是，控制基于变压器的大型模型在不重新训练的情况下满足词汇约束仍然是一个挑战。在这项工作中，我们提出了定向光束搜索（DB），这是一种用于词汇约束语言生成的插件方法。我们的方法可以应用于任何语言模型，易于实现，并且可以用于通用语言。在我们的实验中，我们使用DBS控制GPT-2。我们展示了其在关键字到短语生成上的性能，并获得了可比的结果，作为词汇限制的故事生成的最先进的非插电模型。
Large pre-trained language models are capable of generating realistic text. However, controlling these models so that the generated text satisfies lexical constraints, i.e., contains specific words, is a challenging problem. Given that state-of-the-art language models are too large to be trained from scratch in a manageable time, it is desirable to control these models without re-training them. Methods capable of doing this are called plug-and-play. Recent plug-and-play methods have been successful in constraining small bidirectional language models as well as forward models in tasks with a restricted search space, e.g., machine translation. However, controlling large transformer-based models to meet lexical constraints without re-training them remains a challenge. In this work, we propose Directed Beam Search (DBS), a plug-and-play method for lexically constrained language generation. Our method can be applied to any language model, is easy to implement and can be used for general language generation. In our experiments we use DBS to control GPT-2. We demonstrate its performance on keyword-to-phrase generation and we obtain comparable results as a state-of-the-art non-plug-and-play model for lexically constrained story generation.
