论文标题

与图形卷积网络的统一图嵌入功能,用于基于骨架的动作识别

Unifying Graph Embedding Features with Graph Convolutional Networks for Skeleton-based Action Recognition

论文作者

Yang, Dong, Li, Monica Mengqi, Fu, Hong, Fan, Jicong, Zhang, Zhao, Leung, Howard

论文摘要

将骨骼结构与图形卷积网络相结合,在人类行动识别方面取得了显着的性能。由于当前的研究重点是设计用于表示骨骼数据的基本图形,因此这些嵌入功能包含基本拓扑信息,这些信息无法从骨骼数据中学习更多的系统观点。在本文中,我们通过提出一个新颖的框架来克服这一限制,该框架将15个图形嵌入到人类动作识别的图形卷积网络中,旨在最好地利用图形信息来区分人类动作中的关键关节,骨骼和身体部位,而不是独占单个特征或域。此外,我们全面研究了如何找到骨骼结构的最佳图形特征,以改善人类的作用识别。此外,还探索了骨骼序列的拓扑信息,以进一步提高多流框架中的性能。此外,统一的图形特征是通过训练过程中的自适应方法提取的,这进一步得到了改进。我们的模型通过三个大规模数据集验证,即NTU-RGB+D,动力学和SYSU-3D,并且优于最先进的方法。总体而言,我们的工作统一图嵌入功能以促进有关人类行动识别的系统研究。

Combining skeleton structure with graph convolutional networks has achieved remarkable performance in human action recognition. Since current research focuses on designing basic graph for representing skeleton data, these embedding features contain basic topological information, which cannot learn more systematic perspectives from skeleton data. In this paper, we overcome this limitation by proposing a novel framework, which unifies 15 graph embedding features into the graph convolutional network for human action recognition, aiming to best take advantage of graph information to distinguish key joints, bones, and body parts in human action, instead of being exclusive to a single feature or domain. Additionally, we fully investigate how to find the best graph features of skeleton structure for improving human action recognition. Besides, the topological information of the skeleton sequence is explored to further enhance the performance in a multi-stream framework. Moreover, the unified graph features are extracted by the adaptive methods on the training process, which further yields improvements. Our model is validated by three large-scale datasets, namely NTU-RGB+D, Kinetics and SYSU-3D, and outperforms the state-of-the-art methods. Overall, our work unified graph embedding features to promotes systematic research on human action recognition.

扫码加入交流群

加入微信交流群

微信交流群二维码

扫码加入学术交流群,获取更多资源