论文标题
快速顾问
Fast AdvProp
论文作者
论文摘要
对抗性传播（Advprop）是改善识别模型，利用对抗性例子的有效方法。尽管如此，Advprop的训练速度极为慢，主要是因为：a）产生对抗性示例需要额外的前进和向后传球； b）原始样本及其对抗性对应物均用于培训（即2 $ \ times $数据）。在本文中，我们介绍了Fast Advprop，该快速顾问会积极地改善Advprop昂贵的培训组件，从而使该方法几乎与香草培训一样便宜。具体而言，我们在快速顾问中进行的修改是通过以下假设来指导的，即用对抗性示例进行分解学习是改进性能的关键，而其他训练食谱（例如，配对的清洁和对抗性训练样本，多步兵对抗性攻击者）可以很大程度上简化。 我们的经验结果表明，与Vanilla培训基线相比，Fast Advprop能够在视觉基准测试中进一步建模性能，而不会产生额外的培训成本。此外，如果使用较大的模型，我们的消融将找到快速的advprop量表，与现有数据增强方法兼容（即混合和cutmix），并且可以轻松地适应其他识别任务，例如对象检测。该代码可在此处提供：https：//github.com/meijieru/fast_advprop。
Adversarial Propagation (AdvProp) is an effective way to improve recognition models, leveraging adversarial examples. Nonetheless, AdvProp suffers from the extremely slow training speed, mainly because: a) extra forward and backward passes are required for generating adversarial examples; b) both original samples and their adversarial counterparts are used for training (i.e., 2$\times$ data). In this paper, we introduce Fast AdvProp, which aggressively revamps AdvProp's costly training components, rendering the method nearly as cheap as the vanilla training. Specifically, our modifications in Fast AdvProp are guided by the hypothesis that disentangled learning with adversarial examples is the key for performance improvements, while other training recipes (e.g., paired clean and adversarial training samples, multi-step adversarial attackers) could be largely simplified. Our empirical results show that, compared to the vanilla training baseline, Fast AdvProp is able to further model performance on a spectrum of visual benchmarks, without incurring extra training cost. Additionally, our ablations find Fast AdvProp scales better if larger models are used, is compatible with existing data augmentation methods (i.e., Mixup and CutMix), and can be easily adapted to other recognition tasks like object detection. The code is available here: https://github.com/meijieru/fast_advprop.
