论文标题
day2ark：超出静音日光的伪监督活动识别
Day2Dark: Pseudo-Supervised Activity Recognition beyond Silent Daylight
论文作者
论文摘要
本文努力认识到黑暗和当天的活动。我们首先确定最先进的活动识别者在白天有效，但在黑暗中不值得信赖。主要原因是标记为“黑暗视频”的可用性有限，以及在测试时向较低颜色对比度的分布转移。为了弥补缺乏标记的黑暗视频，我们介绍了一种伪监督的学习计划，该计划利用易于获得未标记和任务含量的黑暗视频，以在低光下提高活动识别器。由于较低的对比导致视觉信息丢失，我们进一步建议将互补活动信息纳入音频，这是照明的不变。由于音频和视觉特征的有用性取决于照明量，因此我们介绍了“黑暗自适应”音频识别器。关于Epic-Kitchens，Kinetics-Sound和Charades的实验表明，我们的建议优于图像增强，域的适应性和替代视听融合方法，甚至可以改善遮挡引起的局部黑暗的稳健性。项目页面：https：//xiaobai1217.github.io/day2dark/
This paper strives to recognize activities in the dark, as well as in the day. We first establish that state-of-the-art activity recognizers are effective during the day, but not trustworthy in the dark. The main causes are the limited availability of labeled dark videos to learn from, as well as the distribution shift towards the lower color contrast at test-time. To compensate for the lack of labeled dark videos, we introduce a pseudo-supervised learning scheme, which utilizes easy to obtain unlabeled and task-irrelevant dark videos to improve an activity recognizer in low light. As the lower color contrast results in visual information loss, we further propose to incorporate the complementary activity information within audio, which is invariant to illumination. Since the usefulness of audio and visual features differs depending on the amount of illumination, we introduce our `darkness-adaptive' audio-visual recognizer. Experiments on EPIC-Kitchens, Kinetics-Sound, and Charades demonstrate our proposals are superior to image enhancement, domain adaptation and alternative audio-visual fusion methods, and can even improve robustness to local darkness caused by occlusions. Project page: https://xiaobai1217.github.io/Day2Dark/
