论文标题
RARA：零射击SIM2REAL视觉导航，带有前景提示
RARA: Zero-shot Sim2Real Visual Navigation with Following Foreground Cues
论文作者
论文摘要
模拟与现实世界之间的差距限制了计算机视觉中的许多机器学习突破，并从适用于现实世界中的增强学习中的学习限制。在这项工作中，我们解决了基于相机导航的特定情况的差距，将其制定为遵循具有任意背景的前景中的视觉提示。前景中的视觉提示通常可以真实地模拟，例如线，门或锥体。然后，挑战在于应对未知背景并将两者整合在一起。因此，目的是在空的模拟环境中捕获的数据上训练视觉代理，除了该前景提示并直接在视觉上多样化的现实世界中测试该模型。为了弥合这一巨大差距，我们表明将以下技术结合起来至关重要：即前后和背景的随机增强，同时使用深度监督和三重态损失以及最终使用路点的动态抽象，而不是直接的速度命令。在定性和定量上，我们的实验结果中都消融了各种技术，最终证明了从模拟到现实世界的成功转移。
The gap between simulation and the real-world restrains many machine learning breakthroughs in computer vision and reinforcement learning from being applicable in the real world. In this work, we tackle this gap for the specific case of camera-based navigation, formulating it as following a visual cue in the foreground with arbitrary backgrounds. The visual cue in the foreground can often be simulated realistically, such as a line, gate or cone. The challenge then lies in coping with the unknown backgrounds and integrating both. As such, the goal is to train a visual agent on data captured in an empty simulated environment except for this foreground cue and test this model directly in a visually diverse real world. In order to bridge this big gap, we show it's crucial to combine following techniques namely: Randomized augmentation of the fore- and background, regularization with both deep supervision and triplet loss and finally abstraction of the dynamics by using waypoints rather than direct velocity commands. The various techniques are ablated in our experimental results both qualitatively and quantitatively finally demonstrating a successful transfer from simulation to the real world.
