论文标题
使用最近的邻居取样的有条件互信息的神经估计器
Neural Estimators for Conditional Mutual Information Using Nearest Neighbors Sampling
论文作者
论文摘要
从一组样品中估算相互信息（MI）或条件互信息（CMI）是一个长期存在的问题。该领域最近的一项工作利用了人工神经网络的近似能力，并显示出比常规方法的改善。在这种新方法中，一个重要的挑战是，考虑到原始数据集，需要获得另一个集合，其中样品是根据特定产品密度函数分配的。估计CMI时，这尤其具有挑战性。 在本文中，我们基于K最近的邻居（K-NN）介绍了一种新技术，以进行重新采样并获得样品平均值的高信心浓度边界。然后采用该技术来训练神经网络分类器，并估算CMI。我们使用此技术提出了三个估计量，并证明了它们的一致性，对它们进行比较与文献中的类似方法，并在实验中显示出在估计器的准确性和差异方面估算CMI的改进。
The estimation of mutual information (MI) or conditional mutual information (CMI) from a set of samples is a long-standing problem. A recent line of work in this area has leveraged the approximation power of artificial neural networks and has shown improvements over conventional methods. One important challenge in this new approach is the need to obtain, given the original dataset, a different set where the samples are distributed according to a specific product density function. This is particularly challenging when estimating CMI. In this paper, we introduce a new technique, based on k nearest neighbors (k-NN), to perform the resampling and derive high-confidence concentration bounds for the sample average. Then the technique is employed to train a neural network classifier and the CMI is estimated accordingly. We propose three estimators using this technique and prove their consistency, make a comparison between them and similar approaches in the literature, and experimentally show improvements in estimating the CMI in terms of accuracy and variance of the estimators.
