论文标题
没有任何监督的理由：改善了监督模型的概括
No Reason for No Supervision: Improved Generalization in Supervised Models
论文作者
论文摘要
我们考虑在给定的分类任务（例如Imagenet-1k（IN1K））上训练深层神经网络的问题，以便它在培训任务以及其他（未来）转移任务上都符合训练任务。这两个看似矛盾的属性在改善模型的概括和维持其在原始任务上的绩效之间实现了权衡。接受自我监督学习训练的模型倾向于比其受监督的转移学习更好地概括。但是，他们仍然落后于In1k上的受监督模型。在本文中，我们提出了一个有监督的学习设置，以利用两全其美的方式。我们使用多尺度农作物进行数据增强和消耗性投影仪负责人进行了广泛的分析培训，并揭示了投影仪的设计使我们能够控制培训任务上的绩效与可转移性的绩效之间的权衡。我们将进一步的班级权重代替了使用记忆库在飞行中计算的班级原型的最后一层原型，并得出了两种模型：T-Rex实现了转移学习的新技术，并且优于in in1k上的Dino和Paws，以及T-Rex*等高优化的RSB-A1模型在ENS1K上进行更优化的RESB-A1模型，同时可以在in1k上进行更优化的RSB-A1模型。代码和预算模型：https：//europe.naverlabs.com/t-rex
We consider the problem of training a deep neural network on a given classification task, e.g., ImageNet-1K (IN1K), so that it excels at both the training task as well as at other (future) transfer tasks. These two seemingly contradictory properties impose a trade-off between improving the model's generalization and maintaining its performance on the original task. Models trained with self-supervised learning tend to generalize better than their supervised counterparts for transfer learning; yet, they still lag behind supervised models on IN1K. In this paper, we propose a supervised learning setup that leverages the best of both worlds. We extensively analyze supervised training using multi-scale crops for data augmentation and an expendable projector head, and reveal that the design of the projector allows us to control the trade-off between performance on the training task and transferability. We further replace the last layer of class weights with class prototypes computed on the fly using a memory bank and derive two models: t-ReX that achieves a new state of the art for transfer learning and outperforms top methods such as DINO and PAWS on IN1K, and t-ReX* that matches the highly optimized RSB-A1 model on IN1K while performing better on transfer tasks. Code and pretrained models: https://europe.naverlabs.com/t-rex
