论文标题
戒断：歧视者合作不可能的及时及时调整可控文本生成
DisCup: Discriminator Cooperative Unlikelihood Prompt-tuning for Controllable Text Generation
论文作者
论文摘要
通过非常大的休闲语言模型（CLM）的及时学习已显示出可属性可控制的文本生成（CTG）有希望的。但是，香草及时调整倾向于模仿超出控制属性的训练语料库特征，从而导致概括能力差。此外，捕获不同属性之间的关系并进一步限制了控制性能，它的能力较低。在本文中，我们提出了一种新的CTG方法，即戒断，该方法结合了歧视者的属性知识以优化控制效果，并转向冷冻的CLM来产生特定于属性的文本。具体而言，能够生成众多文本的冷冻CLM模型首先用于基于上下文生成下一步的候选者，以确保可以预测代币的多样性。然后，我们利用属性 - 歧视器从这些候选人中选择所需的/不希望的令牌，从而提供属性知识。最后，我们通过迅速调整的不可能的目标来弥合上述两个特征。广泛的实验结果表明，杂物可以在保持高效且高质量的文本生成的同时，仅依靠10个虚拟令牌。
Prompt learning with immensely large Casual Language Models (CLMs) has been shown promising for attribute-controllable text generation (CTG). However, vanilla prompt tuning tends to imitate training corpus characteristics beyond the control attributes, resulting in a poor generalization ability. Moreover, it is less able to capture the relationship between different attributes, further limiting the control performance. In this paper, we propose a new CTG approach, namely DisCup, which incorporates the attribute knowledge of discriminator to optimize the control-prompts, steering a frozen CLM to produce attribute-specific texts. Specifically, the frozen CLM model, capable of producing multitudinous texts, is first used to generate the next-token candidates based on the context, so as to ensure the diversity of tokens to be predicted. Then, we leverage an attribute-discriminator to select desired/undesired tokens from those candidates, providing the inter-attribute knowledge. Finally, we bridge the above two traits by an unlikelihood objective for prompt-tuning. Extensive experimental results show that DisCup can achieve a new state-of-the-art control performance while maintaining an efficient and high-quality text generation, only relying on around 10 virtual tokens.
