论文标题
将自然语言说明转换为机器人操纵的计算机程序
Translating Natural Language Instructions to Computer Programs for Robot Manipulation
论文作者
论文摘要
对于与人类一起工作的机器人可以理解自然语言的说明是非常理想的。现有的语言模仿学习模型直接从图像观察和指令文本中预测执行器命令。我们没有直接预测执行器命令，而是建议将自然语言指令转换为Python函数，该功能通过访问对象检测器的输出并控制机器人执行指定任务来查询场景。这使得在计算机器人命令时使用非差异性模块，例如约束求解器。此外，此设置中的标签具有更大信息的计算机程序，可以捕获专家的意图，而不是详细的演示。我们表明，所提出的方法比训练神经网络直接预测机器人动作要好。
It is highly desirable for robots that work alongside humans to be able to understand instructions in natural language. Existing language conditioned imitation learning models directly predict the actuator commands from the image observation and the instruction text. Rather than directly predicting actuator commands, we propose translating the natural language instruction to a Python function which queries the scene by accessing the output of the object detector and controls the robot to perform the specified task. This enables the use of non-differentiable modules such as a constraint solver when computing commands to the robot. Moreover, the labels in this setup are significantly more informative computer programs that capture the intent of the expert rather than teleoperated demonstrations. We show that the proposed method performs better than training a neural network to directly predict the robot actions.
