论文标题
学习用于机器人操作的稳定正常化控制
Learning Stable Normalizing-Flow Control for Robotic Manipulation
论文作者
论文摘要
尽管取得了令人印象深刻的成功，但机器人操纵技巧的强化学习（RL）仍可以从控制理论中纳入领域知识中受益。感兴趣的最重要的特性之一是控制稳定性。理想情况下，人们希望在保持最先进的深度RL算法框架内实现稳定性保证。这种解决方案通常不存在，尤其是一个扩展到复杂的操作任务的解决方案。我们通过引入$ \ textit {normolization-flow} $控制结构来缩小这一差距，该结构可以部署在任何最新的深度RL算法中。尽管不能保证稳定的探索，但我们的方法旨在最终生产具有可证明稳定性的确定控制器。除了展示我们有关挑战接触式操纵任务的方法外，我们还表明，可以实现相当大的勘探效率 - 减少了国家空间覆盖范围和驱动工作 - 而不会失去学习效率。
Reinforcement Learning (RL) of robotic manipulation skills, despite its impressive successes, stands to benefit from incorporating domain knowledge from control theory. One of the most important properties that is of interest is control stability. Ideally, one would like to achieve stability guarantees while staying within the framework of state-of-the-art deep RL algorithms. Such a solution does not exist in general, especially one that scales to complex manipulation tasks. We contribute towards closing this gap by introducing $\textit{normalizing-flow}$ control structure, that can be deployed in any latest deep RL algorithms. While stable exploration is not guaranteed, our method is designed to ultimately produce deterministic controllers with provable stability. In addition to demonstrating our method on challenging contact-rich manipulation tasks, we also show that it is possible to achieve considerable exploration efficiency--reduced state space coverage and actuation efforts--without losing learning efficiency.
