论文标题
混合现实服务的基于云渲染的容量视频流系统
Cloud Rendering-based Volumetric Video Streaming System for Mixed Reality Services
论文作者
论文摘要
体积视频是一种新兴技术，用于对3D空间的沉浸式表示，可使用多个摄像机从各个方向捕获对象，并创建场景的动态3D模型。但是，处理容量内容需要大量的处理能力，对于当今的移动设备来说，仍然是一项非常苛刻的任务。为了减轻这种情况，我们提出了一个体积视频流系统，该系统将渲染效果卸载到功能强大的云/边缘服务器上，并且仅将渲染的2D视图发送给客户端，而不是完整的体积内容。我们使用6DOF头部运动预测技术，WEBRTC协议和硬件视频编码，以确保处理链不同部分的低延迟。我们使用基于浏览器的客户端和Microsoft Hololens客户端展示了我们的系统。我们的应用程序包含通用接口，可轻松使用同一服务器实现来轻松部署各种增强/混合现实客户端。
Volumetric video is an emerging technology for immersive representation of 3D spaces that captures objects from all directions using multiple cameras and creates a dynamic 3D model of the scene. However, processing volumetric content requires high amounts of processing power and is still a very demanding task for today's mobile devices. To mitigate this, we propose a volumetric video streaming system that offloads the rendering to a powerful cloud/edge server and only sends the rendered 2D view to the client instead of the full volumetric content. We use 6DoF head movement prediction techniques, WebRTC protocol and hardware video encoding to ensure low-latency in different parts of the processing chain. We demonstrate our system using both a browser-based client and a Microsoft HoloLens client. Our application contains generic interfaces that allow for easy deployment of various augmented/mixed reality clients using the same server implementation.
