论文标题
平方$ \ ell_2 $ norm作为利用增强数据的一致性损失，以了解强大而不变的表示形式
Squared $\ell_2$ Norm as Consistency Loss for Leveraging Augmented Data to Learn Robust and Invariant Representations
论文作者
论文摘要
数据增强是改善神经网络鲁棒性的最流行技术之一。除了直接使用原始样品和增强样品训练模型外，还引入了一系列方法，将原始样品的嵌入/表示及其增强物之间的距离定期。在本文中，我们探讨了这些各种正规化选择，试图对我们应该如何使嵌入方式进行一般了解。我们的分析表明，正则化的理想选择对应于各种假设。通过不变性测试，我们认为正规化很重要，如果该模型在更广泛的上下文中要比准确驱动的设置更广泛，因为非规范化方法在学习不变的概念方面受到限制，尽管精度同样很高。最后，我们还表明，我们确定的通用方法（平方$ \ ell_2 $ norm norm正规化增强）优于最近的几种方法，这些方法是专门为一项任务设计的，并且比我们的三个不同任务更为复杂。
Data augmentation is one of the most popular techniques for improving the robustness of neural networks. In addition to directly training the model with original samples and augmented samples, a torrent of methods regularizing the distance between embeddings/representations of the original samples and their augmented counterparts have been introduced. In this paper, we explore these various regularization choices, seeking to provide a general understanding of how we should regularize the embeddings. Our analysis suggests the ideal choices of regularization correspond to various assumptions. With an invariance test, we argue that regularization is important if the model is to be used in a broader context than the accuracy-driven setting because non-regularized approaches are limited in learning the concept of invariance, despite equally high accuracy. Finally, we also show that the generic approach we identified (squared $\ell_2$ norm regularized augmentation) outperforms several recent methods, which are each specially designed for one task and significantly more complicated than ours, over three different tasks.
