论文标题
Physnlu：一种用于评估自然语言理解和解释的语言资源
PhysNLU: A Language Resource for Evaluating Natural Language Understanding and Explanation Coherence in Physics
论文作者
论文摘要
为了使语言模型有助于物理研究，它们必须首先编码数学和自然语言话语的表示，这会导致连贯的解释，并具有正确的订单和陈述相关性。我们提出了开发的数据集的集合，以评估这方面的语言模型的性能，该方面衡量了有关句子顺序，位置，部分预测和话语一致性的功能。对数据的分析揭示了在物理话语中最常见的方程式和子学科，以及方程式和表达式的句子级频率。我们介绍了基准，即使在数学自然语言目标训练时，也证明了当代语言模型如何受到物理相关任务的挑战。
In order for language models to aid physics research, they must first encode representations of mathematical and natural language discourse which lead to coherent explanations, with correct ordering and relevance of statements. We present a collection of datasets developed to evaluate the performance of language models in this regard, which measure capabilities with respect to sentence ordering, position, section prediction, and discourse coherence. Analysis of the data reveals equations and sub-disciplines which are most common in physics discourse, as well as the sentence-level frequency of equations and expressions. We present baselines that demonstrate how contemporary language models are challenged by coherence related tasks in physics, even when trained on mathematical natural language objectives.
