论文标题
通过贝叶斯深网进行分类的快速预测不确定性
Fast Predictive Uncertainty for Classification with Bayesian Deep Networks
论文作者
论文摘要
在贝叶斯深度学习中，通常通过首先在权重上构造高斯分布，然后从中取样以接收SoftMax输出的分布，从而近似分类神经网络输出的分布。这是昂贵的。我们重新考虑了旧工作（拉普拉斯桥）以构建此软磁输出分布的差异近似，该分布在logit空间中的高斯分布与输出空间中的dirichlet分布（分类分布之前的共轭）之间产生了分析图。重要的是，香草拉普拉斯桥有一定的局限性。我们分析了这些问题，并提出了一个简单的解决方案，该解决方案与Softmax-Gaussian积分的其他常用估计值有利。我们证明，所得的Dirichlet分布具有多个优点，特别是对大型数据集和网络（如ImageNet和Densenet）的不确定性估计和扩展的更有效计算。我们通过使用它来构建ImageNet的轻量级不确定性输出排名来进一步证明此Dirichlet近似值的有用性。
In Bayesian Deep Learning, distributions over the output of classification neural networks are often approximated by first constructing a Gaussian distribution over the weights, then sampling from it to receive a distribution over the softmax outputs. This is costly. We reconsider old work (Laplace Bridge) to construct a Dirichlet approximation of this softmax output distribution, which yields an analytic map between Gaussian distributions in logit space and Dirichlet distributions (the conjugate prior to the Categorical distribution) in the output space. Importantly, the vanilla Laplace Bridge comes with certain limitations. We analyze those and suggest a simple solution that compares favorably to other commonly used estimates of the softmax-Gaussian integral. We demonstrate that the resulting Dirichlet distribution has multiple advantages, in particular, more efficient computation of the uncertainty estimate and scaling to large datasets and networks like ImageNet and DenseNet. We further demonstrate the usefulness of this Dirichlet approximation by using it to construct a lightweight uncertainty-aware output ranking for ImageNet.
