论文标题
武士：不受约束的现实世界任意图像收集的形状和材料
SAMURAI: Shape And Material from Unconstrained Real-world Arbitrary Image collections
论文作者
论文摘要
在完全未知的捕获条件下，对象的逆渲染是计算机视觉和图形的基本挑战。 NERF等神经方法在新型视图合成上取得了逼真的结果，但它们需要已知的相机姿势。用未知相机姿势解决此问题是高度挑战的，因为它需要对形状，辐射和姿势进行关节优化。当在野外捕获带有不同背景和照明的野外捕获输入图像时，此问题会加剧。由于图像之间的估计对应关系很少，因此在野外这种图像收集中，标准姿势估计技术失败。此外，NERF无法在任何照明下重新占据场景，因为它在Radiance（反射率和照明的产物）上运行。我们提出了一个联合优化框架，以估计形状，BRDF和每印象相机的姿势和照明。我们的方法可用于对象的野外线图像集合，并为AR/VR等多个用例生成可靠的3D资产。据我们所知，我们的方法是第一个通过最少的用户互动来解决这项严重不受限制的任务的方法。项目页面：https：//markboss.me/publication/2022-samurai/视频：https：//youtu.be/llyugdjxp-8
Inverse rendering of an object under entirely unknown capture conditions is a fundamental challenge in computer vision and graphics. Neural approaches such as NeRF have achieved photorealistic results on novel view synthesis, but they require known camera poses. Solving this problem with unknown camera poses is highly challenging as it requires joint optimization over shape, radiance, and pose. This problem is exacerbated when the input images are captured in the wild with varying backgrounds and illuminations. Standard pose estimation techniques fail in such image collections in the wild due to very few estimated correspondences across images. Furthermore, NeRF cannot relight a scene under any illumination, as it operates on radiance (the product of reflectance and illumination). We propose a joint optimization framework to estimate the shape, BRDF, and per-image camera pose and illumination. Our method works on in-the-wild online image collections of an object and produces relightable 3D assets for several use-cases such as AR/VR. To our knowledge, our method is the first to tackle this severely unconstrained task with minimal user interaction. Project page: https://markboss.me/publication/2022-samurai/ Video: https://youtu.be/LlYuGDjXp-8
