论文标题
一致性引导场景流估计
Consistency Guided Scene Flow Estimation
论文作者
论文摘要
一致性引导场景流量估计（CGSF）是一个自我监督的框架，用于联合重建3D场景结构和立体声视频的运动。该模型将两个时间立体声对作为输入，并预测差异和场景流。该模型在测试时间进行自适应，通过迭代完善其预测。完善过程以一致性损失为指导，该损失结合了立体声和时间的光合段与几何术语，该术语将差异和3D运动结合在一起。为了处理一致性损失（例如兰伯特假设）中的固有建模误差，并为了更好地概括，我们进一步引入了学习的，输出改进网络，该网络以初始预测，损失和梯度为输入，并有效地预测了相关的输出更新。在包括消融研究在内的多个实验中，我们表明，所提出的模型可以可靠地预测具有挑战性的图像中的差异和场景流，比最新的模型更好地获得概括，并快速，稳健地适应了看不见的域。
Consistency Guided Scene Flow Estimation (CGSF) is a self-supervised framework for the joint reconstruction of 3D scene structure and motion from stereo video. The model takes two temporal stereo pairs as input, and predicts disparity and scene flow. The model self-adapts at test time by iteratively refining its predictions. The refinement process is guided by a consistency loss, which combines stereo and temporal photo-consistency with a geometric term that couples disparity and 3D motion. To handle inherent modeling error in the consistency loss (e.g. Lambertian assumptions) and for better generalization, we further introduce a learned, output refinement network, which takes the initial predictions, the loss, and the gradient as input, and efficiently predicts a correlated output update. In multiple experiments, including ablation studies, we show that the proposed model can reliably predict disparity and scene flow in challenging imagery, achieves better generalization than the state-of-the-art, and adapts quickly and robustly to unseen domains.
