论文标题
从模型分布和辍学产生多样的翻译
Generating Diverse Translation from Model Distribution with Dropout
论文作者
论文摘要
尽管翻译质量提高了，但神经机器翻译（NMT）通常会遭受其一代人缺乏多样性的影响。在本文中，我们建议通过从它们的推理中衍生出大量的贝叶斯建模和采样模型来生成各种翻译。通过将混凝土辍学量应用于NMT模型来获得可能的模型，并且每个模型对其预测都具有特定的信心，该模型对应于贝叶斯建模原理下的特定训练数据下的后验模型分布。通过变异推断，可以通过变异分布近似后验模型分布，从中取样了最终模型。我们进行了有关中文和英语 - 德语翻译任务的实验，结果表明，我们的方法可以在多样性和准确性之间进行更好的权衡。
Despite the improvement of translation quality, neural machine translation (NMT) often suffers from the lack of diversity in its generation. In this paper, we propose to generate diverse translations by deriving a large number of possible models with Bayesian modelling and sampling models from them for inference. The possible models are obtained by applying concrete dropout to the NMT model and each of them has specific confidence for its prediction, which corresponds to a posterior model distribution under specific training data in the principle of Bayesian modeling. With variational inference, the posterior model distribution can be approximated with a variational distribution, from which the final models for inference are sampled. We conducted experiments on Chinese-English and English-German translation tasks and the results shows that our method makes a better trade-off between diversity and accuracy.
