论文标题
个性化的多模式反馈生成教育
Personalized Multimodal Feedback Generation in Education
论文作者
论文摘要
对学校作业的自动评估是AI在教育领域的重要应用。在这项工作中，我们专注于个性化的多模式反馈生成的任务，该任务旨在为各种老师生成个性化反馈，以评估学生的作业，涉及涉及图像，音频和文本等多模式输入。此任务涉及多模式信息和自然语言生成的表示和融合，这提出了三个方面的挑战：1）如何编码和集成多模式输入； 2）如何生成针对每种模式的反馈； 3）如何实现个性化的反馈生成。在本文中，我们提出了一个新型个性化的多式联运反馈生成网络（PMFGN），该网络（PMFGN）配备了模态门机制和一个个性化偏见机制来应对这些挑战。关于现实世界K-12教育数据的广泛实验表明，我们的模型通过产生更准确和多样化的反馈来大大优于几个基线。此外，进行了详细的消融实验，以加深我们对所提出框架的理解。
The automatic evaluation for school assignments is an important application of AI in the education field. In this work, we focus on the task of personalized multimodal feedback generation, which aims to generate personalized feedback for various teachers to evaluate students' assignments involving multimodal inputs such as images, audios, and texts. This task involves the representation and fusion of multimodal information and natural language generation, which presents the challenges from three aspects: 1) how to encode and integrate multimodal inputs; 2) how to generate feedback specific to each modality; and 3) how to realize personalized feedback generation. In this paper, we propose a novel Personalized Multimodal Feedback Generation Network (PMFGN) armed with a modality gate mechanism and a personalized bias mechanism to address these challenges. The extensive experiments on real-world K-12 education data show that our model significantly outperforms several baselines by generating more accurate and diverse feedback. In addition, detailed ablation experiments are conducted to deepen our understanding of the proposed framework.
