论文标题
神经自然推理模型部分嵌入了词汇和否定的理论
Neural Natural Language Inference Models Partially Embed Theories of Lexical Entailment and Negation
论文作者
论文摘要
我们使用四种方法：（1）挑战测试集的行为评估方法和（2）系统的概括任务以及（3）探针和（4）介入的结构评估方法。为了促进这项整体评估，我们提出了单调性NLI（MONLI），这是一个针对词汇范围和否定的新自然主义数据集。在我们的行为评估中，我们发现在通用NLI数据集中训练的模型在包含否定的Monli示例上系统地失败，但Monli微调解决了这一失败。在我们的结构评估中，我们寻找证据表明，我们最表现的基于BERT的模型已经学会了实施Monli背后的单调性算法。探针产生与该结论一致的证据，我们的干预实验会加强这一点，表明该模型的因果动力学反映了该算法在Monli子集上的因果动力学。这表明，BERT模型至少部分地嵌入了词汇层面和否定理论，以算法层面。
We address whether neural models for Natural Language Inference (NLI) can learn the compositional interactions between lexical entailment and negation, using four methods: the behavioral evaluation methods of (1) challenge test sets and (2) systematic generalization tasks, and the structural evaluation methods of (3) probes and (4) interventions. To facilitate this holistic evaluation, we present Monotonicity NLI (MoNLI), a new naturalistic dataset focused on lexical entailment and negation. In our behavioral evaluations, we find that models trained on general-purpose NLI datasets fail systematically on MoNLI examples containing negation, but that MoNLI fine-tuning addresses this failure. In our structural evaluations, we look for evidence that our top-performing BERT-based model has learned to implement the monotonicity algorithm behind MoNLI. Probes yield evidence consistent with this conclusion, and our intervention experiments bolster this, showing that the causal dynamics of the model mirror the causal dynamics of this algorithm on subsets of MoNLI. This suggests that the BERT model at least partially embeds a theory of lexical entailment and negation at an algorithmic level.
