PRFusion: toward effective and robust multi-modal place recognition with image and point cloud fusion

• Main Authors: Wang, Sijie, Kang, Qiyu, She, Rui, Zhao, Kai, Song, Yang, Tay, Wee Peng
• Other Authors: School of Electrical and Electronic Engineering
• Format: Article
• Language: English
• Published: 2025
• Subjects: Engineering; Place recognition; Multi-modal fusion
• Online Access: https://hdl.handle.net/10356/182557
• Institution: Nanyang Technological University

Place recognition plays a crucial role in the fields of robotics and computer vision, finding applications in areas such as autonomous driving, mapping, and localization. Place recognition identifies a place using query sensor data and a known database. One of the main challenges is to develop a model that can deliver accurate results while being robust to environmental variations. We propose two multi-modal place recognition models, namely PRFusion and PRFusion++. PRFusion utilizes global fusion with manifold metric attention, enabling effective interaction between features without requiring camera-LiDAR extrinsic calibrations. In contrast, PRFusion++ assumes the availability of extrinsic calibrations and leverages pixel-point correspondences to enhance feature learning on local windows. Additionally, both models incorporate neural diffusion layers, which enable reliable operation even in challenging environments. We verify the state-of-the-art performance of both models on three large-scale benchmarks. Notably, they outperform existing models by a substantial margin of +3.0 AR@1 on the demanding Boreas dataset. Furthermore, we conduct ablation studies to validate the effectiveness of our proposed methods.