论文标题
多语言模型对新语言的系统发育启发的改编
Phylogeny-Inspired Adaptation of Multilingual Models to New Languages
论文作者
论文摘要
经过数十种语言培训的大型多语言模型，由于各种语言任务的跨语性学习能力，取得了令人鼓舞的结果。进一步将这些模型改编成特定语言，尤其是在预训练期间看不见的语言，是扩大语言技术覆盖范围的重要目标。在这项研究中，我们展示了如何使用语言系统发育信息来以结构化的，语言知名的方式改善跨语性传递，从而利用密切相关的语言。我们对来自不同语言家庭（日耳曼语，乌拉尔，图宗，UTO-Aztecan）的语言进行基于适配器的培训，并对句法和语义任务进行评估，从而获得了20％以上的相对性能改善，尤其是在预训练期间未见的语言，尤其是对语言。
Large pretrained multilingual models, trained on dozens of languages, have delivered promising results due to cross-lingual learning capabilities on variety of language tasks. Further adapting these models to specific languages, especially ones unseen during pre-training, is an important goal towards expanding the coverage of language technologies. In this study, we show how we can use language phylogenetic information to improve cross-lingual transfer leveraging closely related languages in a structured, linguistically-informed manner. We perform adapter-based training on languages from diverse language families (Germanic, Uralic, Tupian, Uto-Aztecan) and evaluate on both syntactic and semantic tasks, obtaining more than 20% relative performance improvements over strong commonly used baselines, especially on languages unseen during pre-training.
