论文标题
语言产生的潜在扩散
Latent Diffusion for Language Generation
论文作者
论文摘要
扩散模型在建模连续数据模式（例如图像，音频和视频）方面取得了巨大成功，但在离散域（例如语言）中的使用有限。最新的试图适应扩散到语言的尝试已将扩散作为现有语言模型的替代方法。我们将扩散和现有语言模型视为互补。我们证明，可以利用编码器语言模型有效地学习高质量的语言自动编码器。然后，我们证明可以在语言自动编码器的潜在空间中学习连续的扩散模型，从而使我们能够采样连续的潜在表示，可以通过预审计的解码器将其解码为自然语言。我们验证了我们对无条件，阶级和顺序到序列语言产生的方法的有效性。我们在多个不同的数据集中证明，我们的潜在语言扩散模型比以前的扩散语言模型更有效。
Diffusion models have achieved great success in modeling continuous data modalities such as images, audio, and video, but have seen limited use in discrete domains such as language. Recent attempts to adapt diffusion to language have presented diffusion as an alternative to existing pretrained language models. We view diffusion and existing language models as complementary. We demonstrate that encoder-decoder language models can be utilized to efficiently learn high-quality language autoencoders. We then demonstrate that continuous diffusion models can be learned in the latent space of the language autoencoder, enabling us to sample continuous latent representations that can be decoded into natural language with the pretrained decoder. We validate the effectiveness of our approach for unconditional, class-conditional, and sequence-to-sequence language generation. We demonstrate across multiple diverse data sets that our latent language diffusion models are significantly more effective than previous diffusion language models.
