论文标题
高维中的内核回归：超越双重下降的精制分析
Kernel regression in high dimensions: Refined analysis beyond double descent
论文作者
论文摘要
在本文中，根据训练数据N的数量是否超过特征维度d，我们提供了高维内核脊回归的概括性能的精确表征。通过建立预期过量风险的偏置变异分解，我们表明，尽管偏差（几乎）与d无关，而单调的偏差随着n而单调降低，但在不同的正则化方案下，偏差依赖于n，d，并且可以单峰或单调降低。我们的精致分析超出了双重下降理论，这表明，根据数据本征核和正则化水平，内核回归风险曲线可以是n的双衰变，钟形或单调的函数。进行合成和真实数据的实验以支持我们的理论发现。
In this paper, we provide a precise characterization of generalization properties of high dimensional kernel ridge regression across the under- and over-parameterized regimes, depending on whether the number of training data n exceeds the feature dimension d. By establishing a bias-variance decomposition of the expected excess risk, we show that, while the bias is (almost) independent of d and monotonically decreases with n, the variance depends on n, d and can be unimodal or monotonically decreasing under different regularization schemes. Our refined analysis goes beyond the double descent theory by showing that, depending on the data eigen-profile and the level of regularization, the kernel regression risk curve can be a double-descent-like, bell-shaped, or monotonic function of n. Experiments on synthetic and real data are conducted to support our theoretical findings.
