论文标题
使用高斯过程的不确定性估计中的单调性和双重下降
Monotonicity and Double Descent in Uncertainty Estimation with Gaussian Processes
论文作者
论文摘要
尽管它们对于评估预测的可靠性的重要性，但对机器学习模型的不确定性量化（UQ）措施最近才开始严格地表征。一个突出的问题是维度的诅咒：通常认为边际可能性应该让人联想到交叉验证指标，并且两者都应随着较大的输入维度而恶化。我们证明，通过调整超参数以最大程度地提高边际可能性（经验贝叶斯程序），通过边际可能性测量的性能可以通过输入维度单调地改善。另一方面，我们证明，交叉验证指标在质量上表现出不同的行为，这是双重下降的特征。最近由于某些情况下的性能提高而引起了人们的兴趣，这些寒冷的后代似乎加剧了这些现象。我们从经验上验证了我们的结果对实际数据持续，超出了我们考虑的假设，并且我们探索了涉及合成协变量的后果。
Despite their importance for assessing reliability of predictions, uncertainty quantification (UQ) measures for machine learning models have only recently begun to be rigorously characterized. One prominent issue is the curse of dimensionality: it is commonly believed that the marginal likelihood should be reminiscent of cross-validation metrics and that both should deteriorate with larger input dimensions. We prove that by tuning hyperparameters to maximize marginal likelihood (the empirical Bayes procedure), the performance, as measured by the marginal likelihood, improves monotonically} with the input dimension. On the other hand, we prove that cross-validation metrics exhibit qualitatively different behavior that is characteristic of double descent. Cold posteriors, which have recently attracted interest due to their improved performance in certain settings, appear to exacerbate these phenomena. We verify empirically that our results hold for real data, beyond our considered assumptions, and we explore consequences involving synthetic covariates.
