CVFNet: Real-time 3D object detection by learning cross view features

In recent years 3D object detection from LiDAR point clouds has made great progress thanks to the development of deep learning technologies. Although voxel or point based methods are popular in 3D object detection, they usually involve time-consuming operations such as 3D convolutions on voxels or b...

Full description

Saved in:
Bibliographic Details
Main Authors: GU, Jiaqi, XIANG, Zhiyu, ZHAO, Pan, BAI, Tingming, WANG, Lingxuan, ZHAO, Xijun, ZHANG, Zhiyuan
Format: text
Language:English
Published: Institutional Knowledge at Singapore Management University 2022
Subjects:
Online Access:https://ink.library.smu.edu.sg/sis_research/7945
https://ink.library.smu.edu.sg/context/sis_research/article/8948/viewcontent/2203.06585.pdf
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Singapore Management University
Language: English
id sg-smu-ink.sis_research-8948
record_format dspace
spelling sg-smu-ink.sis_research-89482023-07-20T07:48:08Z CVFNet: Real-time 3D object detection by learning cross view features GU, Jiaqi XIANG, Zhiyu ZHAO, Pan BAI, Tingming WANG, Lingxuan ZHAO, Xijun ZHANG, Zhiyuan In recent years 3D object detection from LiDAR point clouds has made great progress thanks to the development of deep learning technologies. Although voxel or point based methods are popular in 3D object detection, they usually involve time-consuming operations such as 3D convolutions on voxels or ball query among points, making the resulting network inappropriate for time critical applications. On the other hand, 2D view-based methods feature high computing efficiency while usually obtaining inferior performance than the voxel or point based methods. In this work, we present a real-time view-based single stage 3D object detector, namely CVFNet to fulfill this task. To strengthen the cross-view feature learning under the condition of demanding efficiency, our framework extracts the features of different views and fuses them in an efficient progressive way. We first propose a novel Point-Range feature fusion module that deeply integrates point and range view features in multiple stages. Then, a special Slice Pillar is designed to well maintain the 3D geometry when transforming the obtained deep point-view features into bird's eye view. To better balance the ratio of samples, a sparse pillar detection head is presented to focus the detection on the nonempty grids. We conduct experiments on the popular KITTI and NuScenes benchmark, and state-of-the-art performances are achieved in terms of both accuracy and speed. 2022-10-01T07:00:00Z text application/pdf https://ink.library.smu.edu.sg/sis_research/7945 info:doi/10.1109/iros47612.2022.9981087 https://ink.library.smu.edu.sg/context/sis_research/article/8948/viewcontent/2203.06585.pdf http://creativecommons.org/licenses/by-nc-nd/4.0/ Research Collection School Of Computing and Information Systems eng Institutional Knowledge at Singapore Management University Representation learning Point cloud compression Three-dimensional displays Laser radar Object detection Feature extraction Real-time systems Artificial Intelligence and Robotics Graphics and Human Computer Interfaces
institution Singapore Management University
building SMU Libraries
continent Asia
country Singapore
Singapore
content_provider SMU Libraries
collection InK@SMU
language English
topic Representation learning
Point cloud compression
Three-dimensional displays
Laser radar
Object detection
Feature extraction
Real-time systems
Artificial Intelligence and Robotics
Graphics and Human Computer Interfaces
spellingShingle Representation learning
Point cloud compression
Three-dimensional displays
Laser radar
Object detection
Feature extraction
Real-time systems
Artificial Intelligence and Robotics
Graphics and Human Computer Interfaces
GU, Jiaqi
XIANG, Zhiyu
ZHAO, Pan
BAI, Tingming
WANG, Lingxuan
ZHAO, Xijun
ZHANG, Zhiyuan
CVFNet: Real-time 3D object detection by learning cross view features
description In recent years 3D object detection from LiDAR point clouds has made great progress thanks to the development of deep learning technologies. Although voxel or point based methods are popular in 3D object detection, they usually involve time-consuming operations such as 3D convolutions on voxels or ball query among points, making the resulting network inappropriate for time critical applications. On the other hand, 2D view-based methods feature high computing efficiency while usually obtaining inferior performance than the voxel or point based methods. In this work, we present a real-time view-based single stage 3D object detector, namely CVFNet to fulfill this task. To strengthen the cross-view feature learning under the condition of demanding efficiency, our framework extracts the features of different views and fuses them in an efficient progressive way. We first propose a novel Point-Range feature fusion module that deeply integrates point and range view features in multiple stages. Then, a special Slice Pillar is designed to well maintain the 3D geometry when transforming the obtained deep point-view features into bird's eye view. To better balance the ratio of samples, a sparse pillar detection head is presented to focus the detection on the nonempty grids. We conduct experiments on the popular KITTI and NuScenes benchmark, and state-of-the-art performances are achieved in terms of both accuracy and speed.
format text
author GU, Jiaqi
XIANG, Zhiyu
ZHAO, Pan
BAI, Tingming
WANG, Lingxuan
ZHAO, Xijun
ZHANG, Zhiyuan
author_facet GU, Jiaqi
XIANG, Zhiyu
ZHAO, Pan
BAI, Tingming
WANG, Lingxuan
ZHAO, Xijun
ZHANG, Zhiyuan
author_sort GU, Jiaqi
title CVFNet: Real-time 3D object detection by learning cross view features
title_short CVFNet: Real-time 3D object detection by learning cross view features
title_full CVFNet: Real-time 3D object detection by learning cross view features
title_fullStr CVFNet: Real-time 3D object detection by learning cross view features
title_full_unstemmed CVFNet: Real-time 3D object detection by learning cross view features
title_sort cvfnet: real-time 3d object detection by learning cross view features
publisher Institutional Knowledge at Singapore Management University
publishDate 2022
url https://ink.library.smu.edu.sg/sis_research/7945
https://ink.library.smu.edu.sg/context/sis_research/article/8948/viewcontent/2203.06585.pdf
_version_ 1772829247245647872