论文标题
LSO：非convex有限总和的线路搜索二阶随机优化方法
LSOS: Line-search Second-Order Stochastic optimization methods for nonconvex finite sums
论文作者
论文摘要
我们开发了一个线路搜索二阶算法框架，以最大程度地减少有限总和。我们没有做出任何凸度的假设，而是要求总和的术语持续可区分，并且具有Lipschitz-conledun带来的梯度。适合此框架的方法结合了线路搜索和适当的衰减步长。关键问题是在每次迭代时进行两步采样，这使我们能够控制线路搜索过程中存在的错误。限制点的平稳性在几乎是纯正的意义上证明，而近似值序列的几乎呈固定的收敛性则具有额外的假设，即功能强烈凸出。数值实验，包括与最先进的随机优化方法的比较，显示了我们方法的效率。
We develop a line-search second-order algorithmic framework for minimizing finite sums. We do not make any convexity assumptions, but require the terms of the sum to be continuously differentiable and have Lipschitz-continuous gradients. The methods fitting into this framework combine line searches and suitably decaying step lengths. A key issue is a two-step sampling at each iteration, which allows us to control the error present in the line-search procedure. Stationarity of limit points is proved in the almost-sure sense, while almost-sure convergence of the sequence of approximations to the solution holds with the additional hypothesis that the functions are strongly convex. Numerical experiments, including comparisons with state-of-the art stochastic optimization methods, show the efficiency of our approach.
