论文标题
分布式非凸优化的通信压缩
Communication Compression for Distributed Nonconvex Optimization
论文作者
论文摘要
本文认为分布的非凸优化，其成本函数是在代理上分布的。指出信息压缩是随着代理与邻居进行迭代通信的分布式算法的重大通信负载的关键工具，我们提出了三种带有压缩通信的分布式原始算法。前两种算法适用于具有有界相对压缩误差的通用压缩机类别，第三算法适用于两个具有界限绝对压缩误差的通用压缩机类。我们表明，带有压缩通信的提议的分布式算法具有可比的收敛属性，即具有精确通信的最先进算法。具体来说，我们表明他们可以找到具有sublinear收敛速率$ \ MATHCAL {O}（1/T）$的一阶固定点时，当每个本地成本函数都很顺畅，其中$ t $是迭代总数，并在全球成本函数的附加条件下找到具有线性收敛速度的全球最佳次数，使全球成本函数满足Polyak-goloak-golojasiew-golojasiew-golojasiew条件。提供数值模拟以说明理论结果的有效性。
This paper considers distributed nonconvex optimization with the cost functions being distributed over agents. Noting that information compression is a key tool to reduce the heavy communication load for distributed algorithms as agents iteratively communicate with neighbors, we propose three distributed primal--dual algorithms with compressed communication. The first two algorithms are applicable to a general class of compressors with bounded relative compression error and the third algorithm is suitable for two general classes of compressors with bounded absolute compression error. We show that the proposed distributed algorithms with compressed communication have comparable convergence properties as state-of-the-art algorithms with exact communication. Specifically, we show that they can find first-order stationary points with sublinear convergence rate $\mathcal{O}(1/T)$ when each local cost function is smooth, where $T$ is the total number of iterations, and find global optima with linear convergence rate under an additional condition that the global cost function satisfies the Polyak--Łojasiewicz condition. Numerical simulations are provided to illustrate the effectiveness of the theoretical results.
