论文标题
调试RL的交互式可视化
Interactive Visualization for Debugging RL
论文作者
论文摘要
监督学习的可视化工具使用户可以解释，内省并获得其模型成功和失败的直觉。强化学习从业人员提出了许多相同的问题，但现有工具不适用于RL设置，因为这些工具解决了监督学习制度中通常发现的挑战。在这项工作中，我们设计并实施了用于调试和解释RL算法的交互式可视化工具。我们的系统解决了以前工具中缺少的许多功能，例如（1）用于监督学习的工具通常不是交互式的； （2）在调试RL政策的同时，研究人员使用的国家表示与代理商所见的国家表示不同； （3）旨在使调试RL策略更有益的框架。我们提供了如何使用该系统以及未来扩展的想法的示例工作流程。
Visualization tools for supervised learning allow users to interpret, introspect, and gain an intuition for the successes and failures of their models. While reinforcement learning practitioners ask many of the same questions, existing tools are not applicable to the RL setting as these tools address challenges typically found in the supervised learning regime. In this work, we design and implement an interactive visualization tool for debugging and interpreting RL algorithms. Our system addresses many features missing from previous tools such as (1) tools for supervised learning often are not interactive; (2) while debugging RL policies researchers use state representations that are different from those seen by the agent; (3) a framework designed to make the debugging RL policies more conducive. We provide an example workflow of how this system could be used, along with ideas for future extensions.
