论文标题
大型网络系统中的分布式影响力增强的本地模拟器用于并行MAL
Distributed Influence-Augmented Local Simulators for Parallel MARL in Large Networked Systems
论文作者
论文摘要
由于其样本的复杂性很高，截至目前，模拟对于成功应用增强学习至关重要。但是，许多现实世界中的问题都表现出过度复杂的动力学，这使其全尺度模拟在计算上很慢。在本文中，我们展示了如何将许多代理的大型网络系统分解为多个局部组件，以便我们可以构建独立和并行运行的单独模拟器。为了监视不同局部组件彼此施加的影响，这些模拟器中的每个模拟器都配备了一个学识渊博的模型，该模型经过定期训练对实际轨迹。我们的经验结果表明，在不同的过程之间分配仿真不仅可以在短短几个小时内训练大型多机构系统，还可以帮助减轻同时学习的负面影响。
Due to its high sample complexity, simulation is, as of today, critical for the successful application of reinforcement learning. Many real-world problems, however, exhibit overly complex dynamics, which makes their full-scale simulation computationally slow. In this paper, we show how to decompose large networked systems of many agents into multiple local components such that we can build separate simulators that run independently and in parallel. To monitor the influence that the different local components exert on one another, each of these simulators is equipped with a learned model that is periodically trained on real trajectories. Our empirical results reveal that distributing the simulation among different processes not only makes it possible to train large multi-agent systems in just a few hours but also helps mitigate the negative effects of simultaneous learning.
