论文标题
时间戳独立于触觉 - 视觉同步
Timestamp-independent Haptic-Visual Synchronization
论文作者
论文摘要
蓬勃发展的触觉数据可显着改善多媒体相互作用期间的用户immersion。结果，对触觉，视听环境（HAT）的研究吸引了多媒体社区的注意力。要实现这样的系统，具有挑战性的解决方案是对用户体验至关重要的多个感官信号的同步。尽管进行了视听同步工作，但仍然缺乏触觉感知的多媒体同步模型。在这项工作中，我们提出了一个独立于时间戳的同步，以用于触觉 - 视觉信号传递。首先，我们利用触觉 - 视觉通信系统的交付和播放期间的顺序相关性。其次，我们根据力反馈特性开发了触觉信号的关键样本提取，以及基于深对象检测的视觉信号的关键框架提取。第三，我们将关键样品和帧结合起来，以同步相应的触觉 - 视觉信号。没有信号流中的时间戳，提出的方法仍然对复杂的网络条件仍然有效且更健壮。主观评估还显示了通过提出的方法对用户体验的显着改善。
The booming haptic data significantly improves the users'immersion during multimedia interaction. As a result, the study of Haptic,Audio-Visual Environment(HAVE)has attracted attentions of multimedia community. To realize such a system, a challenging tack is the synchronization of multiple sensorial signals that is critical to user experience. Despite of audio-visual synchronization efforts, there is still a lack of haptic-aware multimedia synchronization model. In this work, we propose a timestamp-independent synchronization for haptic-visual signal transmission. First, we exploit the sequential correlations during delivery and playback of a haptic-visual communication system. Second, we develop a key sample extraction of haptic signals based on the force feedback characteristics, and a key frame extraction of visual signals based on deep object detection. Third, we combine the key samples and frames to synchronize the corresponding haptic-visual signals. Without timestamps in signal flow, the proposed method is still effective and more robust to complicated network conditions. Subjective evaluation also shows a significant improvement of user experience with the proposed method.
