论文标题
探索文本摘要中编辑后有效性的探索
An Exploration of Post-Editing Effectiveness in Text Summarization
论文作者
论文摘要
自动摘要方法是有效的，但质量低。相比之下，手动摘要昂贵，但质量更高。人类和人工智能可以协作以提高总结性能吗？在类似的文本生成任务（例如机器翻译）中，人类AI的合作形式是“后编辑” AI生成的文本，可减少人类的工作量并提高AI输出的质量。因此，我们探讨了邮政编辑是否在文本摘要中提供了优势。具体来说，我们对72名参与者进行了实验，比较了在正式（XSUM新闻）和非正式（Reddit帖子）文本的摘要质量，人为效率和用户体验的手动摘要中提供的摘要。这项研究对何时编辑后的文本摘要提供了宝贵的见解：在某些情况下（例如，何时参与者缺乏域名知识），但在其他情况下却没有帮助（例如，何时提供的摘要包括不准确的信息）。参与者的不同编辑策略和援助需求为未来的人类摘要系统提供了影响。
Automatic summarization methods are efficient but can suffer from low quality. In comparison, manual summarization is expensive but produces higher quality. Can humans and AI collaborate to improve summarization performance? In similar text generation tasks (e.g., machine translation), human-AI collaboration in the form of "post-editing" AI-generated text reduces human workload and improves the quality of AI output. Therefore, we explored whether post-editing offers advantages in text summarization. Specifically, we conducted an experiment with 72 participants, comparing post-editing provided summaries with manual summarization for summary quality, human efficiency, and user experience on formal (XSum news) and informal (Reddit posts) text. This study sheds valuable insights on when post-editing is useful for text summarization: it helped in some cases (e.g., when participants lacked domain knowledge) but not in others (e.g., when provided summaries include inaccurate information). Participants' different editing strategies and needs for assistance offer implications for future human-AI summarization systems.
