论文标题
透明玻璃：透明形状的神经3D重建
Through the Looking Glass: Neural 3D Reconstruction of Transparent Shapes
论文作者
论文摘要
使用少量不受限制的自然图像恢复透明物体的3D形状是一个问题。折射和反射引起的复杂光路已经阻止了传统和深度多览立体声的解决挑战。我们提出了一个基于物理的网络，以在已知但任意的环境图下使用手机相机获取的几张图像来恢复透明对象的3D形状。我们的新颖贡献包括一种正常表示，使网络能够通过局部计算进行建模复杂的光传输，该计算是建模折射和反射层的渲染层，这是专门设计用于正常细化透明形状的成本量，以及基于3D点云重建的预测的正常绘制的特征映射。我们渲染一个合成数据集，以鼓励模型学习跨不同视图的折射光传输。我们的实验表明，使用少于5-12个自然图像的复杂透明形状成功恢复了高质量的3D几何形状。代码和数据将公开发布。
Recovering the 3D shape of transparent objects using a small number of unconstrained natural images is an ill-posed problem. Complex light paths induced by refraction and reflection have prevented both traditional and deep multiview stereo from solving this challenge. We propose a physically-based network to recover 3D shape of transparent objects using a few images acquired with a mobile phone camera, under a known but arbitrary environment map. Our novel contributions include a normal representation that enables the network to model complex light transport through local computation, a rendering layer that models refractions and reflections, a cost volume specifically designed for normal refinement of transparent shapes and a feature mapping based on predicted normals for 3D point cloud reconstruction. We render a synthetic dataset to encourage the model to learn refractive light transport across different views. Our experiments show successful recovery of high-quality 3D geometry for complex transparent shapes using as few as 5-12 natural images. Code and data are publicly released.
