论文标题
探索和解释：自我监督的导航和叙述
Explore and Explain: Self-supervised Navigation and Recounting
论文作者
论文摘要
体现的AI最近引起了人们的关注，因为它旨在促进自主和智能代理的发展。在本文中，我们设计了一个新颖的体现环境，在该环境中，代理需要探索以前未知的环境，同时叙述其在路径中看到的环境。在这种情况下，代理需要浏览由探索目标驱动的环境，选择适当的时刻进行描述，并输出相关对象和场景的自然语言描述。我们的模型将新颖的自我监督探索模块与惩罚结合在一起，并为解释提供了完全专心的字幕模型。此外，我们研究了选择适当的解释时刻的不同政策，这是由来自环境和导航的信息驱动的。实验是从MatterPort3D数据集中在光真逼真的环境上进行的，并研究了代理的导航和解释功能以及其相互作用的作用。
Embodied AI has been recently gaining attention as it aims to foster the development of autonomous and intelligent agents. In this paper, we devise a novel embodied setting in which an agent needs to explore a previously unknown environment while recounting what it sees during the path. In this context, the agent needs to navigate the environment driven by an exploration goal, select proper moments for description, and output natural language descriptions of relevant objects and scenes. Our model integrates a novel self-supervised exploration module with penalty, and a fully-attentive captioning model for explanation. Also, we investigate different policies for selecting proper moments for explanation, driven by information coming from both the environment and the navigation. Experiments are conducted on photorealistic environments from the Matterport3D dataset and investigate the navigation and explanation capabilities of the agent as well as the role of their interactions.
