论文标题
这不是希腊人的麦伯特：诱导多语言伯特的单词级翻译
It's not Greek to mBERT: Inducing Word-Level Translations from Multilingual BERT
论文作者
论文摘要
最近的作品表明，多语言伯特（Mbert）学习丰富的跨语性表示，可以跨语言转移。我们研究嵌入在姆伯特中的单词级翻译信息，并提出了两种简单的方法，这些方法揭示了显着的翻译功能而没有微调。结果表明，大多数此信息都是以非线性方式编码的，而其中一些信息也可以用纯线性工具恢复。作为我们分析的一部分，我们检验了Mbert学习的表示的假设，该假设既包含语言编码组件和抽象的跨语性组成部分，又明确地识别Mbert表示中的经验语言 - 认同子空间。
Recent works have demonstrated that multilingual BERT (mBERT) learns rich cross-lingual representations, that allow for transfer across languages. We study the word-level translation information embedded in mBERT and present two simple methods that expose remarkable translation capabilities with no fine-tuning. The results suggest that most of this information is encoded in a non-linear way, while some of it can also be recovered with purely linear tools. As part of our analysis, we test the hypothesis that mBERT learns representations which contain both a language-encoding component and an abstract, cross-lingual component, and explicitly identify an empirical language-identity subspace within mBERT representations.
