论文标题
割线惩罚的BFG：通过惩罚割线条件的噪声强大的准Newton方法
Secant Penalized BFGS: A Noise Robust Quasi-Newton Method Via Penalizing The Secant Condition
论文作者
论文摘要
在本文中，我们介绍了BFGS方法的新变体，旨在在噪声损坏梯度测量时表现良好。我们表明，通过使用正规最小二乘估计动机的惩罚方法处理割线条件，可以通过使用原始BFGS更新公式更新逆Hessian近似值，并且不更新逆Hessian近似值。此外，当插值移动到不更新逆黑板近似值时，我们发现曲率条件平稳放松，当不更新逆Hessian近似值时，完全消失了。这些发展使我们能够开发一种我们称为SCANT惩罚的BFG（SP-BFG）的方法，该方法使人们可以根据梯度测量中的噪声量放松割线状况。 SP-BFGS提供了一种逐步更新新的逆Hessian近似值，对先前的逆Hessian近似具有受控量的偏置，这使得人们可以用平均性质替换原始BFGS更新的覆盖性质，以抵抗噪声的破坏性效应，并能够应对噪声的破坏性效应，并能够应对负弯曲的测量。我们讨论了SP-BFG的理论特性，包括当在均匀界限噪声的情况下最大程度地凸出凸功能时收敛。最后，我们使用最可爱的测试问题集中的30多个问题提出了广泛的数值实验，这些实验表明，在噪声功能和梯度评估的情况下，与BFG相比，SP-BFG的出色性能。
In this paper, we introduce a new variant of the BFGS method designed to perform well when gradient measurements are corrupted by noise. We show that by treating the secant condition with a penalty method approach motivated by regularized least squares estimation, one can smoothly interpolate between updating the inverse Hessian approximation with the original BFGS update formula and not updating the inverse Hessian approximation. Furthermore, we find the curvature condition is smoothly relaxed as the interpolation moves towards not updating the inverse Hessian approximation, disappearing entirely when the inverse Hessian approximation is not updated. These developments allow us to develop a method we refer to as secant penalized BFGS (SP-BFGS) that allows one to relax the secant condition based on the amount of noise in the gradient measurements. SP-BFGS provides a means of incrementally updating the new inverse Hessian approximation with a controlled amount of bias towards the previous inverse Hessian approximation, which allows one to replace the overwriting nature of the original BFGS update with an averaging nature that resists the destructive effects of noise and can cope with negative curvature measurements. We discuss the theoretical properties of SP-BFGS, including convergence when minimizing strongly convex functions in the presence of uniformly bounded noise. Finally, we present extensive numerical experiments using over 30 problems from the CUTEst test problem set that demonstrate the superior performance of SP-BFGS compared to BFGS in the presence of both noisy function and gradient evaluations.
