论文标题
单帧监督时间动作本地化的扩张渗透
Dilation-Erosion for Single-Frame Supervised Temporal Action Localization
论文作者
论文摘要
为了平衡注释劳动和监督的粒度，在时间行动定位中引入了单帧注释。它为动作提供了一个粗略的时间位置，但隐含地夸大了训练过程中带注释的框架的监督，从而导致动作与背景之间的混乱，即动作不完整和背景误报。为了应对这两个挑战，在这项工作中，我们介绍了摘要分类模型和扩张渗透模块。在扩张渗透模块中，我们以宽松的标准扩展了潜在的动作段，以减轻行动不完整问题，然后从潜在的动作段中删除背景，以减轻行动不完整的问题。依靠单帧注释和摘要分类的输出，扩张渗透模块地雷矿山pseudo sippet级地面真实，硬背景和明显的背景，这又进一步训练了摘要分类模型。它形成了循环依赖性。此外，我们提出了一种新的嵌入损失，以汇总具有相同标签的动作实例功能，并将动作的特征与背景分开。 Thumos14和ActivityNet上的实验1.2验证了所提出的方法的有效性。代码已公开可用（https://github.com/lingjun123/single-frame-tal）。
To balance the annotation labor and the granularity of supervision, single-frame annotation has been introduced in temporal action localization. It provides a rough temporal location for an action but implicitly overstates the supervision from the annotated-frame during training, leading to the confusion between actions and backgrounds, i.e., action incompleteness and background false positives. To tackle the two challenges, in this work, we present the Snippet Classification model and the Dilation-Erosion module. In the Dilation-Erosion module, we expand the potential action segments with a loose criterion to alleviate the problem of action incompleteness and then remove the background from the potential action segments to alleviate the problem of action incompleteness. Relying on the single-frame annotation and the output of the snippet classification, the Dilation-Erosion module mines pseudo snippet-level ground-truth, hard backgrounds and evident backgrounds, which in turn further trains the Snippet Classification model. It forms a cyclic dependency. Furthermore, we propose a new embedding loss to aggregate the features of action instances with the same label and separate the features of actions from backgrounds. Experiments on THUMOS14 and ActivityNet 1.2 validate the effectiveness of the proposed method. Code has been made publicly available (https://github.com/LingJun123/single-frame-TAL).
