论文标题
对抗重量扰动改善了图神经网络中的概括
Adversarial Weight Perturbation Improves Generalization in Graph Neural Networks
论文作者
论文摘要
许多理论和经验证据表明，局部最小的最小值倾向于改善概括。对抗重量扰动（AWP）是一种新兴技术，可有效地找到这种最小值。在AWP中，我们最大程度地减少了W.R.T.的损失模型参数的有界最坏情况扰动，从而有利于本地的最小值，而周围的社区却很少损失。 AWP的好处以及更普遍的平坦与概括之间的联系已被广泛研究为I.I.D.数据，例如图像。在本文中，我们广泛研究了图形数据的这种现象。一路上，我们首先得出了一个限制于非i.i.d的概括。节点分类任务。然后，我们通过所有现有的AWP表述确定一个消失的梯度问题，并提出了一个新的加权截短AWP（WT-AWP）来减轻此问题。我们表明，将图形神经网络与WT-AWP进行正规化，可以始终改善许多不同的图表学习任务和模型的自然概括和稳健的概括。
A lot of theoretical and empirical evidence shows that the flatter local minima tend to improve generalization. Adversarial Weight Perturbation (AWP) is an emerging technique to efficiently and effectively find such minima. In AWP we minimize the loss w.r.t. a bounded worst-case perturbation of the model parameters thereby favoring local minima with a small loss in a neighborhood around them. The benefits of AWP, and more generally the connections between flatness and generalization, have been extensively studied for i.i.d. data such as images. In this paper, we extensively study this phenomenon for graph data. Along the way, we first derive a generalization bound for non-i.i.d. node classification tasks. Then we identify a vanishing-gradient issue with all existing formulations of AWP and we propose a new Weighted Truncated AWP (WT-AWP) to alleviate this issue. We show that regularizing graph neural networks with WT-AWP consistently improves both natural and robust generalization across many different graph learning tasks and models.
