论文标题
分析网络代理中的独立学习：数据包转发用例
Analysis of Independent Learning in Network Agents: A Packet Forwarding Use Case
论文作者
论文摘要
如今，多代理增强学习（MARL）被广泛用于解决各个领域中的现实世界和复杂决策。虽然MAL可以归类为独立和合作的方法，但我们将独立方法视为一种简单，更可扩展且成本较低的方法，用于大规模分布式系统（例如网络数据包转发）。在本文中，我们使用指定的数据网络（NDN）体系结构作为驱动示例，定量和定性地评估了利用此类独立代理学习方法，尤其是基于IQL的算法的好处，以用于计算机网络中的数据包转发。我们将基于IQL的多个转发策略（IDQF）进行了测试，并将其性能与非常基本的转发方案和简单的拓扑/交通模型进行比较，以突出主要的挑战和问题。我们讨论了与IDQF性能不佳有关的主要问题，并在训练和测试IDQF模型下在不同的模型调整参数和网络拓扑/特征下训练和测试IDQF模型时量化了这些问题对隔离的影响。
Multi-Agent Reinforcement Learning (MARL) is nowadays widely used to solve real-world and complex decisions in various domains. While MARL can be categorized into independent and cooperative approaches, we consider the independent approach as a simple, more scalable, and less costly method for large-scale distributed systems, such as network packet forwarding. In this paper, we quantitatively and qualitatively assess the benefits of leveraging such independent agents learning approach, in particular IQL-based algorithm, for packet forwarding in computer networking, using the Named Data Networking (NDN) architecture as a driving example. We put multiple IQL-based forwarding strategies (IDQF) to the test and compare their performances against very basic forwarding schemes and simple topologies/traffic models to highlight major challenges and issues. We discuss the main issues related to the poor performance of IDQF and quantify the impact of these issues on isolation when training and testing the IDQF models under different model tuning parameters and network topologies/characteristics.
