论文标题
句子歧义，语法和复杂性探针
Sentence Ambiguity, Grammaticality and Complexity Probes
论文作者
论文摘要
目前尚不清楚大型预训练的语言模型是否，如何和何处捕获了歧义，语法和句子复杂性等微妙的语言特征。我们介绍了这些特征自动分类的结果，并比较了它们跨表示类型的生存能力和模式。我们证明，基于模板的数据集不应用于探测，应与基准进行仔细的比较，并且不应使用T-SNE图来确定密集矢量表示中特征的存在。我们还展示了这些模型的特征如何在这些型号的层中进行高度局限，并在上层中丢失。
It is unclear whether, how and where large pre-trained language models capture subtle linguistic traits like ambiguity, grammaticality and sentence complexity. We present results of automatic classification of these traits and compare their viability and patterns across representation types. We demonstrate that template-based datasets with surface-level artifacts should not be used for probing, careful comparisons with baselines should be done and that t-SNE plots should not be used to determine the presence of a feature among dense vectors representations. We also show how features might be highly localized in the layers for these models and get lost in the upper layers.
