论文标题
学习基于能量的潜在变量模型的双层得分匹配
Bi-level Score Matching for Learning Energy-based Latent Variable Models
论文作者
论文摘要
得分匹配（SM）通过避免计算分区功能，为学习基于能量的模型（EBM）提供了令人信服的方法。但是，除某些特殊情况外，它仍然在学习基于能量的潜在变量模型（EBLVM）方面仍然开放。本文提出了一种双层分数匹配（BISM）方法，通过将SM作为双层优化问题来学习一般结构的EBLVM。较高的级别引入了潜在变量的变异后部，并优化了修改后的SM目标，并且下层优化了变异后部，以适合真实的后部。为了有效地解决BISM，我们开发了一种随机优化算法，并具有梯度展开。从理论上讲，我们分析了偶然的一致性和随机算法的收敛性。从经验上讲，我们在高斯有限的玻尔兹曼机器和高度非结构性的EBLVM中表现出了卑鄙的诺言的希望。 Bism与适用时广泛采用的对比差异和SM方法相媲美；并可以学习具有棘手的后代的复杂EBLVM，以产生自然图像。
Score matching (SM) provides a compelling approach to learn energy-based models (EBMs) by avoiding the calculation of partition function. However, it remains largely open to learn energy-based latent variable models (EBLVMs), except some special cases. This paper presents a bi-level score matching (BiSM) method to learn EBLVMs with general structures by reformulating SM as a bi-level optimization problem. The higher level introduces a variational posterior of the latent variables and optimizes a modified SM objective, and the lower level optimizes the variational posterior to fit the true posterior. To solve BiSM efficiently, we develop a stochastic optimization algorithm with gradient unrolling. Theoretically, we analyze the consistency of BiSM and the convergence of the stochastic algorithm. Empirically, we show the promise of BiSM in Gaussian restricted Boltzmann machines and highly nonstructural EBLVMs parameterized by deep convolutional neural networks. BiSM is comparable to the widely adopted contrastive divergence and SM methods when they are applicable; and can learn complex EBLVMs with intractable posteriors to generate natural images.
