Learning with unreliability : Fast few-shot voxel radiance fields with relative geometric consistency

• Main Authors: XU, Yingjie, LIU, Bangzhen, TANG, Hao, DENG, Bailin, HE, Shengfeng
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2024
• Subjects: Computer vision; Accuracy; Smoothing methods; Codes; Image color analysis; Navigation; Pattern recognition; Artificial Intelligence and Robotics; Graphics and Human Computer Interfaces
• Online Access: https://ink.library.smu.edu.sg/sis_research/9775; https://ink.library.smu.edu.sg/context/sis_research/article/10775/viewcontent/2403.17638v1.pdf
• Institution: Singapore Management University

We propose a voxel-based optimization framework, Re VoRF, for few-shot radiance fields that strategically ad-dress the unreliability in pseudo novel view synthesis. Our method pivots on the insight that relative depth relationships within neighboring regions are more reliable than the ab-solute color values in disoccluded areas. Consequently, we devise a bilateral geometric consistency loss that carefully navigates the trade-off between color fidelity and geometric accuracy in the context of depth consistency for uncertain regions. Moreover, we present a reliability-guided learning strategy to discern and utilize the variable quality across syn-thesized views, complemented by a reliability-aware voxel smoothing algorithm that smoothens the transition between reliable and unreliable data patches. Our approach allows for a more nuanced use of all available data, promoting en-hanced learning from regions previously considered unsuit-able for high-quality reconstruction. Extensive experiments across diverse datasets reveal that our approach attains significant gains in efficiency and accuracy, delivering ren-dering speeds of 3 FPS, 7 mins to train a 360° scene, and a 5% improvement in PSNR over existing few-shot methods.