论文标题
无监督的强化学习，可转移的操纵技能发现
Unsupervised Reinforcement Learning for Transferable Manipulation Skill Discovery
论文作者
论文摘要
由于特定于特定于任务的培训范式，机器人技术中的当前强化学习（RL）通常会在推广到新的下游任务方面遇到困难。为了减轻无监督的RL，该框架是以任务不合时宜的方式预先培训代理的框架，而无需获得特定于任务的奖励，利用主动探索将多样化的经验蒸馏成基本技能或可重复使用的知识。为了利用此类好处在机器人操作中，我们提出了一种无监督的方法，用于转移操纵技能发现，将结构化探索与相互作用的行为和可转移的技能学习联系起来。它不仅使代理商能够学习互动行为，这是机器人操纵学习的关键方面，而无需访问环境奖励，而且还可以通过学习的任务不可能的技能推广到任意下游操纵任务。通过比较实验，我们表明我们的方法实现了最多样化的相互作用行为，并显着提高了下游任务中的样本效率，包括扩展到多对象，多任务问题。
Current reinforcement learning (RL) in robotics often experiences difficulty in generalizing to new downstream tasks due to the innate task-specific training paradigm. To alleviate it, unsupervised RL, a framework that pre-trains the agent in a task-agnostic manner without access to the task-specific reward, leverages active exploration for distilling diverse experience into essential skills or reusable knowledge. For exploiting such benefits also in robotic manipulation, we propose an unsupervised method for transferable manipulation skill discovery that ties structured exploration toward interacting behavior and transferable skill learning. It not only enables the agent to learn interaction behavior, the key aspect of the robotic manipulation learning, without access to the environment reward, but also to generalize to arbitrary downstream manipulation tasks with the learned task-agnostic skills. Through comparative experiments, we show that our approach achieves the most diverse interacting behavior and significantly improves sample efficiency in downstream tasks including the extension to multi-object, multitask problems.
