论文标题
在不精确梯度评估和动态的Hessian精度下对自适应立方正则化方法的随机分析
Stochastic Analysis of an Adaptive Cubic Regularisation Method under Inexact Gradient Evaluations and Dynamic Hessian Accuracy
论文作者
论文摘要
在这里，我们将自适应立方正规化方法的扩展版本使用动态不精确的Hessian信息，用于[3]中的非convex优化，以适应随机优化设置。尽管仍然考虑了确切的功能评估，但这种新型变体继承了[3]中引入的Hessian近似值的自适应精度要求的创新使用，并还采用了梯度的不精确计算。我们假设这些近似值不受差异的限制，我们假设这些近似值在足够大但固定的概率内可用，我们本着[18]的精神扩展，将框架的确定性分析延伸到其随机对应物，表明预期的迭代次数可以达到一阶平稳点与已知的众所周知的最糟糕的最糟糕的最佳相匹配。实际上，这仍然由O（epsilon^（-3/2））给出，相对于一阶Epsilon公差。最后，对非convex有限和最小化的数值测试证实，使用不精确的一阶和二阶导数可以在计算节省方面有益。
We here adapt an extended version of the adaptive cubic regularisation method with dynamic inexact Hessian information for nonconvex optimisation in [3] to the stochastic optimisation setting. While exact function evaluations are still considered, this novel variant inherits the innovative use of adaptive accuracy requirements for Hessian approximations introduced in [3] and additionally employs inexact computations of the gradient. Without restrictions on the variance of the errors, we assume that these approximations are available within a sufficiently large, but fixed, probability and we extend, in the spirit of [18], the deterministic analysis of the framework to its stochastic counterpart, showing that the expected number of iterations to reach a first-order stationary point matches the well known worst-case optimal complexity. This is, in fact, still given by O(epsilon^(-3/2)), with respect to the first-order epsilon tolerance. Finally, numerical tests on nonconvex finite-sum minimisation confirm that using inexact first and second-order derivatives can be beneficial in terms of the computational savings.
