PatchAugNet: patch feature augmentation-based heterogeneous point cloud place recognition in large-scale street scenes

Point Cloud Place Recognition (PCPR) in street scenes is an essential task in the fields of autonomous driving, robot navigation, and urban map updating. However, the domain gap between heterogeneous point clouds and the difficulty of feature characterization in large-scale complex street scenes pos...

Full description

Saved in:
Bibliographic Details
Main Authors: Zou, Xianghong, Li, Jianping, Wang, Yuan, Liang, Fuxun, Wu, Weitong, Wang, Haiping, Yang, Bisheng, Dong, Zhen
Other Authors: School of Electrical and Electronic Engineering
Format: Article
Language:English
Published: 2024
Subjects:
Online Access:https://hdl.handle.net/10356/173337
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Nanyang Technological University
Language: English
id sg-ntu-dr.10356-173337
record_format dspace
spelling sg-ntu-dr.10356-1733372024-01-29T01:33:19Z PatchAugNet: patch feature augmentation-based heterogeneous point cloud place recognition in large-scale street scenes Zou, Xianghong Li, Jianping Wang, Yuan Liang, Fuxun Wu, Weitong Wang, Haiping Yang, Bisheng Dong, Zhen School of Electrical and Electronic Engineering Engineering::Electrical and electronic engineering Point Cloud Place Recognition Point Cloud Place Recognition (PCPR) in street scenes is an essential task in the fields of autonomous driving, robot navigation, and urban map updating. However, the domain gap between heterogeneous point clouds and the difficulty of feature characterization in large-scale complex street scenes pose significant challenges for existing PCPR methods. Most PCPR methods only take into account point clouds collected by the same platforms and sensors, thus they are with poor domain transferability. In this paper, we propose PatchAugNet, which utilizes patch feature augmentation and adaptive pyramid feature aggregation to achieve better performance and generalizability for Heterogeneous Point Cloud-based Place Recognition (HPCPR) tasks. Firstly, multi-scale local features are extracted by the pyramid feature extraction module. Secondly, local features are enhanced by the patch feature augmentation module to overcome the domain gap problem and achieve better feature representation as well as network generalizability. Finally, a global feature is generated using an adaptive pyramid feature aggregation module, which automatically adjusts and balances the proportion of intra-scale and inter-scale features according to the scene content. To evaluate the performance of PatchAugNet, a large-scale heterogeneous point cloud dataset consisting of high-precision Mobile Laser Scanning (MLS) point clouds and helmet-mounted Portable Laser Scanning (PLS) point clouds is collected. The dataset covers various street scenes with a length of over 20 km. The comprehensive experimental results indicate that PatchAugNet achieves State-Of-The-Art (SOTA) performance with 83.43 % recall@top1% and 60.34 % recall@top1 on unseen large-scale street scenes, outperforming existing SOTA PCPR methods by + 9.57 recall@top1% and + 15.50 recall@top1, while exhibiting better generalizability. For source code and detailed experimental results, please refer to: https://github.com/WHU-USI3DV/PatchAugNet. This study was jointly supported by the National Natural Science Foundation Project (No. 42130105, No. 42201477), the National Key Research and Development Program of China (No.2022YFB3904100), China Postdoctoral Science Foundation (2022 M712441, 2022TQ0234), the Open Fund of Hubei Luojia Laboratory (No. 2201000054). 2024-01-29T01:33:19Z 2024-01-29T01:33:19Z 2023 Journal Article Zou, X., Li, J., Wang, Y., Liang, F., Wu, W., Wang, H., Yang, B. & Dong, Z. (2023). PatchAugNet: patch feature augmentation-based heterogeneous point cloud place recognition in large-scale street scenes. ISPRS Journal of Photogrammetry and Remote Sensing, 206, 273-292. https://dx.doi.org/10.1016/j.isprsjprs.2023.11.005 0924-2716 https://hdl.handle.net/10356/173337 10.1016/j.isprsjprs.2023.11.005 2-s2.0-85177176296 206 273 292 en ISPRS Journal of Photogrammetry and Remote Sensing © 2023 International Society for Photogrammetry and Remote Sensing, Inc. (ISPRS). Published by Elsevier B.V. All rights reserved.
institution Nanyang Technological University
building NTU Library
continent Asia
country Singapore
Singapore
content_provider NTU Library
collection DR-NTU
language English
topic Engineering::Electrical and electronic engineering
Point Cloud
Place Recognition
spellingShingle Engineering::Electrical and electronic engineering
Point Cloud
Place Recognition
Zou, Xianghong
Li, Jianping
Wang, Yuan
Liang, Fuxun
Wu, Weitong
Wang, Haiping
Yang, Bisheng
Dong, Zhen
PatchAugNet: patch feature augmentation-based heterogeneous point cloud place recognition in large-scale street scenes
description Point Cloud Place Recognition (PCPR) in street scenes is an essential task in the fields of autonomous driving, robot navigation, and urban map updating. However, the domain gap between heterogeneous point clouds and the difficulty of feature characterization in large-scale complex street scenes pose significant challenges for existing PCPR methods. Most PCPR methods only take into account point clouds collected by the same platforms and sensors, thus they are with poor domain transferability. In this paper, we propose PatchAugNet, which utilizes patch feature augmentation and adaptive pyramid feature aggregation to achieve better performance and generalizability for Heterogeneous Point Cloud-based Place Recognition (HPCPR) tasks. Firstly, multi-scale local features are extracted by the pyramid feature extraction module. Secondly, local features are enhanced by the patch feature augmentation module to overcome the domain gap problem and achieve better feature representation as well as network generalizability. Finally, a global feature is generated using an adaptive pyramid feature aggregation module, which automatically adjusts and balances the proportion of intra-scale and inter-scale features according to the scene content. To evaluate the performance of PatchAugNet, a large-scale heterogeneous point cloud dataset consisting of high-precision Mobile Laser Scanning (MLS) point clouds and helmet-mounted Portable Laser Scanning (PLS) point clouds is collected. The dataset covers various street scenes with a length of over 20 km. The comprehensive experimental results indicate that PatchAugNet achieves State-Of-The-Art (SOTA) performance with 83.43 % recall@top1% and 60.34 % recall@top1 on unseen large-scale street scenes, outperforming existing SOTA PCPR methods by + 9.57 recall@top1% and + 15.50 recall@top1, while exhibiting better generalizability. For source code and detailed experimental results, please refer to: https://github.com/WHU-USI3DV/PatchAugNet.
author2 School of Electrical and Electronic Engineering
author_facet School of Electrical and Electronic Engineering
Zou, Xianghong
Li, Jianping
Wang, Yuan
Liang, Fuxun
Wu, Weitong
Wang, Haiping
Yang, Bisheng
Dong, Zhen
format Article
author Zou, Xianghong
Li, Jianping
Wang, Yuan
Liang, Fuxun
Wu, Weitong
Wang, Haiping
Yang, Bisheng
Dong, Zhen
author_sort Zou, Xianghong
title PatchAugNet: patch feature augmentation-based heterogeneous point cloud place recognition in large-scale street scenes
title_short PatchAugNet: patch feature augmentation-based heterogeneous point cloud place recognition in large-scale street scenes
title_full PatchAugNet: patch feature augmentation-based heterogeneous point cloud place recognition in large-scale street scenes
title_fullStr PatchAugNet: patch feature augmentation-based heterogeneous point cloud place recognition in large-scale street scenes
title_full_unstemmed PatchAugNet: patch feature augmentation-based heterogeneous point cloud place recognition in large-scale street scenes
title_sort patchaugnet: patch feature augmentation-based heterogeneous point cloud place recognition in large-scale street scenes
publishDate 2024
url https://hdl.handle.net/10356/173337
_version_ 1789483129687244800