论文标题
通往各种紧密度的上坡道路：单调性和蒙特卡洛目标
Uphill Roads to Variational Tightness: Monotonicity and Monte Carlo Objectives
论文作者
论文摘要
我们重新审视重要性加权变量推理（IWVI）的理论，这是一种学习潜在变量模型的有前途的策略。 IWVI使用新的变分边界，称为蒙特卡洛目标（MCO），通过通过蒙特卡洛估计替换顽固的积分来获得 - 通常仅通过重要性采样而获得。 Burda，Grosse和Salakhutdinov（2016）表明，增加重要性样本的数量可以收紧界限与可能性之间的差距。受这个简单的单调性定理的启发，我们提出了一系列的非沉淀结果，这些结果将蒙特卡洛估计的特性与MCOS的紧密度联系起来。我们挑战了较小的蒙特卡洛方差导致更好界限的理由。从理论上讲，我们证明了最近的几篇论文的经验发现，表明从精确的意义上说，负相关降低了变异差距。我们还通过考虑不均匀的权重来概括原始的单调定理。我们讨论了理论结果的几个实际后果。我们的作品借鉴了随机秩序理论的许多想法和结果。
We revisit the theory of importance weighted variational inference (IWVI), a promising strategy for learning latent variable models. IWVI uses new variational bounds, known as Monte Carlo objectives (MCOs), obtained by replacing intractable integrals by Monte Carlo estimates -- usually simply obtained via importance sampling. Burda, Grosse and Salakhutdinov (2016) showed that increasing the number of importance samples provably tightens the gap between the bound and the likelihood. Inspired by this simple monotonicity theorem, we present a series of nonasymptotic results that link properties of Monte Carlo estimates to tightness of MCOs. We challenge the rationale that smaller Monte Carlo variance leads to better bounds. We confirm theoretically the empirical findings of several recent papers by showing that, in a precise sense, negative correlation reduces the variational gap. We also generalise the original monotonicity theorem by considering non-uniform weights. We discuss several practical consequences of our theoretical results. Our work borrows many ideas and results from the theory of stochastic orders.
