论文标题
强烈凹入式函数的分布式鞍点问题
Distributed saddle point problems for strongly concave-convex functions
论文作者
论文摘要
在本文中，我们提出了GT-GDA，一种分布式优化方法来解决形式的鞍点问题：$ \ min _ {\ MathBf {x}}} \ max _ {\ MathBf {y Mathbf {y}}} \ {f（ \ Mathbf {y}，\ clyline {p} \ MathBf {x} \ rangle -h（\ Mathbf {y}）\} $，其中函数$ g（\ cdot）$，$ h（\ cdot）$，以及coupling matrix $ \ + ednelline newsern intrane contruse and and proved a proped a proped a n contruse and contruse aft a fortly contruse and contruse and od and n of n n of n ofd and od and {p}。 GT-GDA是一种使用梯度跟踪来消除节点之间异质数据分布引起的差异的一阶方法。在最通用的形式中，GT-GDA对局部耦合矩阵有了共识，以达到最佳（独特的）马鞍点，但是，以增加通信为代价。为了避免这种情况，我们提出了一个更有效的变体GT-GDA-LITE，该变体不会引起额外的交流并在各种情况下分析其收敛性。我们表明，当$ g（\ cdot）$平稳且凸，$ h（\ cdot）$平滑且强烈凸面时，GT-GDA线性收敛到唯一的鞍点解决方案，并且全局耦合矩阵$ \ overline {p} $具有完整的列等级。我们进一步表征了GT-GDA表现出与网络拓扑无关的收敛行为的制度。接下来，我们将显示GT-GDA对唯一鞍点的错误的线性收敛性，当耦合成本$ {\ langle \ mathbf y，\ overline {p} \ mathbf x \ rangle} $时，该耦合成本为零。数值实验说明了GT-GDA和GT-GDA-LITE对多种应用的收敛属性和重要性。
In this paper, we propose GT-GDA, a distributed optimization method to solve saddle point problems of the form: $\min_{\mathbf{x}} \max_{\mathbf{y}} \{F(\mathbf{x},\mathbf{y}) :=G(\mathbf{x}) + \langle \mathbf{y}, \overline{P} \mathbf{x} \rangle - H(\mathbf{y})\}$, where the functions $G(\cdot)$, $H(\cdot)$, and the the coupling matrix $\overline{P}$ are distributed over a strongly connected network of nodes. GT-GDA is a first-order method that uses gradient tracking to eliminate the dissimilarity caused by heterogeneous data distribution among the nodes. In the most general form, GT-GDA includes a consensus over the local coupling matrices to achieve the optimal (unique) saddle point, however, at the expense of increased communication. To avoid this, we propose a more efficient variant GT-GDA-Lite that does not incur the additional communication and analyze its convergence in various scenarios. We show that GT-GDA converges linearly to the unique saddle point solution when $G(\cdot)$ is smooth and convex, $H(\cdot)$ is smooth and strongly convex, and the global coupling matrix $\overline{P}$ has full column rank. We further characterize the regime under which GT-GDA exhibits a network topology-independent convergence behavior. We next show the linear convergence of GT-GDA to an error around the unique saddle point, which goes to zero when the coupling cost ${\langle \mathbf y, \overline{P} \mathbf x \rangle}$ is common to all nodes, or when $G(\cdot)$ and $H(\cdot)$ are quadratic. Numerical experiments illustrate the convergence properties and importance of GT-GDA and GT-GDA-Lite for several applications.
