论文标题
个性的出现
The Emergence of Individuality
论文作者
论文摘要
个性在人类社会中至关重要，这会导致劳动分裂，从而提高效率和生产力。同样，它也应该是多代理合作的关键。受这种个性的启发是与他人分开的个体，我们提出了一种简单而有效的方法，用于在多机构增强学习（MARL）中出现个性（EOI）。 EOI学习了一个概率分类器，该分类器可以预测鉴于其观察结果的概率分布，并为每个代理提供由分类器正确预测的固有奖励。内在的奖励鼓励代理商访问自己熟悉的观察结果，并通过此类观察来学习分类器，使内在的奖励信号更强，并且代理更容易识别。为了进一步增强内在的奖励并促进个性的出现，提出了两个正规化器来提高分类器的可区分性。我们在流行的MARL算法上实现EOI。从经验上讲，我们表明EOI在各种多机构合作社场景中的表现大大优于现有方法。
Individuality is essential in human society, which induces the division of labor and thus improves the efficiency and productivity. Similarly, it should also be the key to multi-agent cooperation. Inspired by that individuality is of being an individual separate from others, we propose a simple yet efficient method for the emergence of individuality (EOI) in multi-agent reinforcement learning (MARL). EOI learns a probabilistic classifier that predicts a probability distribution over agents given their observation and gives each agent an intrinsic reward of being correctly predicted by the classifier. The intrinsic reward encourages the agents to visit their own familiar observations, and learning the classifier by such observations makes the intrinsic reward signals stronger and the agents more identifiable. To further enhance the intrinsic reward and promote the emergence of individuality, two regularizers are proposed to increase the discriminability of the classifier. We implement EOI on top of popular MARL algorithms. Empirically, we show that EOI significantly outperforms existing methods in a variety of multi-agent cooperative scenarios.
