论文标题
跨认同视频运动重新定位，并进行联合转换和综合
Cross-identity Video Motion Retargeting with Joint Transformation and Synthesis
论文作者
论文摘要
在本文中，我们提出了一个新型的双分支转换合成网络（TS-NET），用于视频运动重新定位。鉴于一个主题视频和一个驾驶视频，TS-NET可以制作一个新的合理视频，并带有主题视频和驾驶视频的运动模式的主题。 TS-NET由一个基于经过的转换分支和无经翘曲的合成分支组成。双重分支的新型设计结合了基于变形网格的转换和无经翘曲的生成的优势，从而在合成视频中具有更好的身份保存和稳健性。进一步将面具感知的相似性模块引入转换分支，以减少计算开销。面部和舞蹈数据集的实验结果表明，TS-NET比几种最先进的模型及其单分支变体可以在视频运动重新定位中取得更好的性能。我们的代码可在https://github.com/nihaomiao/wacv23_tsnet上找到。
In this paper, we propose a novel dual-branch Transformation-Synthesis network (TS-Net), for video motion retargeting. Given one subject video and one driving video, TS-Net can produce a new plausible video with the subject appearance of the subject video and motion pattern of the driving video. TS-Net consists of a warp-based transformation branch and a warp-free synthesis branch. The novel design of dual branches combines the strengths of deformation-grid-based transformation and warp-free generation for better identity preservation and robustness to occlusion in the synthesized videos. A mask-aware similarity module is further introduced to the transformation branch to reduce computational overhead. Experimental results on face and dance datasets show that TS-Net achieves better performance in video motion retargeting than several state-of-the-art models as well as its single-branch variants. Our code is available at https://github.com/nihaomiao/WACV23_TSNet.
