论文标题
PIC 4th挑战：语义辅助多功能编码和多头解码，用于密集的视频字幕
PIC 4th Challenge: Semantic-Assisted Multi-Feature Encoding and Multi-Head Decoding for Dense Video Captioning
论文作者
论文摘要
密集的视频字幕（DVC）的任务旨在为一个视频中的多个事件生成带有时间戳的字幕。语义信息对于DVC的本地化和描述都起着重要作用。我们提出了基于编码编码框架的语义辅助密集的视频字幕字幕模型。在编码阶段，我们设计了一个概念检测器来提取语义信息，然后将其与多模式的视觉特征融合在一起，以充分代表输入视频。在解码阶段，我们设计了一个与本地化和字幕的分类头，以提供语义监督。我们的方法在DVC评估指标下对Youmakeup数据集进行了重大改进，并在PIC 4th挑战的化妆密集视频字幕（MDVC）任务中实现了高性能。
The task of Dense Video Captioning (DVC) aims to generate captions with timestamps for multiple events in one video. Semantic information plays an important role for both localization and description of DVC. We present a semantic-assisted dense video captioning model based on the encoding-decoding framework. In the encoding stage, we design a concept detector to extract semantic information, which is then fused with multi-modal visual features to sufficiently represent the input video. In the decoding stage, we design a classification head, paralleled with the localization and captioning heads, to provide semantic supervision. Our method achieves significant improvements on the YouMakeup dataset under DVC evaluation metrics and achieves high performance in the Makeup Dense Video Captioning (MDVC) task of PIC 4th Challenge.
