论文标题
评估变更
Evaluation for Change
论文作者
论文摘要
评估是评估，理解和交流NLP模型的中心手段。在该立场论文中，我们认为评估应该不仅仅是：这是推动变革的力量，具有超出其技术方面的社会学和政治特征。作为一支力量，评估的力量源于其采用：在我们看来，当评估实现该领域所需的变化时，评估就成功了。此外，通过将评估作为一支力量，我们考虑它如何与其他力量竞争。在我们的分析下，我们猜测NLP的当前轨迹表明，尽管它可能在该领域实现更多多元化的野心，但评估的力量正在减弱。我们通过讨论了这种权力的合法性，谁获得了这种权力及其分配方式。最终，我们希望研究界能够更积极地利用变更评估。
Evaluation is the central means for assessing, understanding, and communicating about NLP models. In this position paper, we argue evaluation should be more than that: it is a force for driving change, carrying a sociological and political character beyond its technical dimensions. As a force, evaluation's power arises from its adoption: under our view, evaluation succeeds when it achieves the desired change in the field. Further, by framing evaluation as a force, we consider how it competes with other forces. Under our analysis, we conjecture that the current trajectory of NLP suggests evaluation's power is waning, in spite of its potential for realizing more pluralistic ambitions in the field. We conclude by discussing the legitimacy of this power, who acquires this power and how it distributes. Ultimately, we hope the research community will more aggressively harness evaluation for change.
