论文标题
结合控制变体和自适应重要性抽样的正交规则
A Quadrature Rule combining Control Variates and Adaptive Importance Sampling
论文作者
论文摘要
在几种成功的应用中，例如在随机梯度下降或贝叶斯计算中，控制变体已成为蒙特卡洛整合的主要工具。但是，标准方法不允许在算法期间分布颗粒的分布，就像顺序仿真方法中一样。在标准的自适应重要性采样框架内，提出了一种简单的加权最小二乘方法，以通过控制变量来改善程序。该过程采用具有适应性正交权重的正交规则的形式，以反映控制变体带来的信息。正交点和权重不取决于整数，这是多个集成的计算优势。此外，目标密度只需要知道到乘法常数。我们的主要结果是在该过程的概率误差上绑定了一个非反应性结合。边界证明，为了提高估计的准确性，可以合并自适应重要性抽样和控制变体所带来的好处。该方法的良好行为在贝叶斯线性回归的综合示例和现实世界中进行了经验说明。
Driven by several successful applications such as in stochastic gradient descent or in Bayesian computation, control variates have become a major tool for Monte Carlo integration. However, standard methods do not allow the distribution of the particles to evolve during the algorithm, as is the case in sequential simulation methods. Within the standard adaptive importance sampling framework, a simple weighted least squares approach is proposed to improve the procedure with control variates. The procedure takes the form of a quadrature rule with adapted quadrature weights to reflect the information brought in by the control variates. The quadrature points and weights do not depend on the integrand, a computational advantage in case of multiple integrands. Moreover, the target density needs to be known only up to a multiplicative constant. Our main result is a non-asymptotic bound on the probabilistic error of the procedure. The bound proves that for improving the estimate's accuracy, the benefits from adaptive importance sampling and control variates can be combined. The good behavior of the method is illustrated empirically on synthetic examples and real-world data for Bayesian linear regression.
