论文标题
探测神经机器翻译中幻觉的原因
Probing Causes of Hallucinations in Neural Machine Translations
论文作者
论文摘要
幻觉是一种困扰神经机器翻译的一种病理翻译，最近引起了很多关注。简而言之，幻觉翻译是流利的句子，但与源输入几乎没有关系。可以说，如何发生幻觉仍然是一个悬而未决的问题。在本文中，我们建议使用探测方法从模型架构的角度研究幻觉的原因，旨在避免将来的架构设计中的此类问题。通过对各种NMT数据集进行实验，我们发现幻觉通常伴随着不足的编码器，尤其是嵌入和脆弱的交叉分离，而有趣的是，跨煽动性减轻了由编码器引起的一些错误。
Hallucination, one kind of pathological translations that bothers Neural Machine Translation, has recently drawn much attention. In simple terms, hallucinated translations are fluent sentences but barely related to source inputs. Arguably, it remains an open problem how hallucination occurs. In this paper, we propose to use probing methods to investigate the causes of hallucinations from the perspective of model architecture, aiming to avoid such problems in future architecture designs. By conducting experiments over various NMT datasets, we find that hallucination is often accompanied by the deficient encoder, especially embeddings, and vulnerable cross-attentions, while, interestingly, cross-attention mitigates some errors caused by the encoder.
