论文标题
关于原始二重偶发梯度法及以后的无限次分化大小
On the Infimal Sub-differential Size of Primal-Dual Hybrid Gradient Method and Beyond
论文作者
论文摘要
原始双重混合梯度法（PDHG，又称Chambolle和Pock方法）是一种良好的算法，用于使用双线性相互作用项，用于最小值优化问题。最近，PDHG用作新的LP求解器PDLP的基本算法，该算法旨在通过利用现代计算资源（例如GPU和分布式系统）来解决大型LP实例。 PDHG的大多数先前收敛结果都处于二元间隔或与最佳解决方案集的距离上，通常在求解过程中很难计算。在本文中，我们提出了一个新的进度度量，用于分析PDHG，我们通过利用PDHG迭代的几何形状来对其进行虚拟的亚差异大小（IDS）。 IDS是平滑问题的梯度规范到非平滑问题的自然扩展，在LP的情况下，它与KKT误差有关。与PDHG的传统进度指标相比，IDS始终具有有限的值，并且只能使用当前解决方案的信息来计算。我们表明ID是单调衰减的，并且它具有$ \数学o（\ frac {1} {k}）$ sublinear速率，用于求解convex-concave primal偶联问题，并且它具有线性收敛速度，并且如果问题进一步满足了诸如计划的计划，则可以满足诸如固定的计划，quadratial oferation grane oferation groment oferation等线性状态，quadratial oferation等线性，quadratial oferiative，quadratial oferation等。 IDS的单调衰减表明IDS是分析PDHG的自然进步指标。作为我们分析的副产品，我们表明原始二间隙具有$ \ MATHCAL O（\ frac {1} {\ sqrt {k}}} $ CENTGENCES $ CENCTGENCE pdhg的最后一次迭代率用于CONVEX-CONCONCAVE问题。 PDHG上的分析和结果可以直接推广到其他原始偶偶有算法，例如近端方法（PPM），乘数的交替方向方法（ADMM）和线性化的乘数交替方向方法（L-ADMM）。
Primal-dual hybrid gradient method (PDHG, a.k.a. Chambolle and Pock method) is a well-studied algorithm for minimax optimization problems with a bilinear interaction term. Recently, PDHG is used as the base algorithm for a new LP solver PDLP that aims to solve large LP instances by taking advantage of modern computing resources, such as GPU and distributed system. Most of the previous convergence results of PDHG are either on duality gap or on distance to the optimal solution set, which are usually hard to compute during the solving process. In this paper, we propose a new progress metric for analyzing PDHG, which we dub infimal sub-differential size (IDS), by utilizing the geometry of PDHG iterates. IDS is a natural extension of the gradient norm of smooth problems to non-smooth problems, and it is tied with KKT error in the case of LP. Compared to traditional progress metrics for PDHG, IDS always has a finite value and can be computed only using information of the current solution. We show that IDS monotonically decays, and it has an $\mathcal O(\frac{1}{k})$ sublinear rate for solving convex-concave primal-dual problems, and it has a linear convergence rate if the problem further satisfies a regularity condition that is satisfied by applications such as linear programming, quadratic programming, TV-denoising model, etc. The simplicity of our analysis and the monotonic decay of IDS suggest that IDS is a natural progress metric to analyze PDHG. As a by-product of our analysis, we show that the primal-dual gap has $\mathcal O(\frac{1}{\sqrt{k}})$ convergence rate for the last iteration of PDHG for convex-concave problems. The analysis and results on PDHG can be directly generalized to other primal-dual algorithms, for example, proximal point method (PPM), alternating direction method of multipliers (ADMM) and linearized alternating direction method of multipliers (l-ADMM).
