论文标题
GPT-3对新型英语名词化合物的解释中的系统性
Systematicity in GPT-3's Interpretation of Novel English Noun Compounds
论文作者
论文摘要
Levin等。 （2019年）在实验上表明，新型英语名词化合物的解释（例如，炖煮的煎锅）虽然不是完全组成，但基于修饰符和头部是否指文物或天然种类，是可以高度预测的。大型语言模型GPT-3是否由相同的解释原理支配？为了解决这个问题，我们首先将Levin等人的实验数据与GPT-3代相提并论，从而找到了高度的相似性。但是，该证据与GPT3仅针对特定词汇项目进行推理，而不是Levin等人的理论的更抽象的概念类别。为了更深入地探究，我们构建了需要相关类型的概念推理的提示。在这里，我们找不到令人信服的证据，表明GPT-3不仅是单个词汇项目的推理。这些结果强调了控制低水平分布规律性的重要性，当评估大型语言模型是否延迟编码更深层次的理论时。
Levin et al. (2019) show experimentally that the interpretations of novel English noun compounds (e.g., stew skillet), while not fully compositional, are highly predictable based on whether the modifier and head refer to artifacts or natural kinds. Is the large language model GPT-3 governed by the same interpretive principles? To address this question, we first compare Levin et al.'s experimental data with GPT-3 generations, finding a high degree of similarity. However, this evidence is consistent with GPT3 reasoning only about specific lexical items rather than the more abstract conceptual categories of Levin et al.'s theory. To probe more deeply, we construct prompts that require the relevant kind of conceptual reasoning. Here, we fail to find convincing evidence that GPT-3 is reasoning about more than just individual lexical items. These results highlight the importance of controlling for low-level distributional regularities when assessing whether a large language model latently encodes a deeper theory.
