Learning nighttime semantic segmentation the hard way

Nighttime semantic segmentation is an important but challenging research problem for autonomous driving. The major challenges lie in the small objects or regions from the under-/over-exposed areas or suffer from motion blur caused by the camera deployed on moving vehicles. To resolve this, we propos...

Full description

Saved in:
Bibliographic Details
Main Authors: LIU, Wenxi, CAI, Jiaxin, LI, Qi, LIAO, Chenyang, CAO, Jingjing, HE, Shengfeng, YU, Yuanlong
Format: text
Language:English
Published: Institutional Knowledge at Singapore Management University 2024
Subjects:
Online Access:https://ink.library.smu.edu.sg/sis_research/9801
https://ink.library.smu.edu.sg/context/sis_research/article/10801/viewcontent/NighttimeSemanticSeg_av.pdf
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Singapore Management University
Language: English
id sg-smu-ink.sis_research-10801
record_format dspace
spelling sg-smu-ink.sis_research-108012024-12-18T00:56:16Z Learning nighttime semantic segmentation the hard way LIU, Wenxi CAI, Jiaxin LI, Qi LIAO, Chenyang CAO, Jingjing HE, Shengfeng YU, Yuanlong Nighttime semantic segmentation is an important but challenging research problem for autonomous driving. The major challenges lie in the small objects or regions from the under-/over-exposed areas or suffer from motion blur caused by the camera deployed on moving vehicles. To resolve this, we propose a novel hard- class-aware module that bridges the main network for full-class segmentation and the hard-class network for segmenting aforementioned hard-class objects. In specific, it exploits the shared focus of hard-class objects from the dual-stream network, enabling the contextual information flow to guide the model to concentrate on the pixels that are hard to classify. In the end, the estimated hard-class segmentation results will be utilized to infer the final results via an adaptive probabilistic fusion refinement scheme. Moreover, to overcome over- smoothing and noise caused by extreme exposures, our model is modulated by a carefully crafted pretext task of constructing an exposure-aware semantic gradient map, which guides the model to faithfully perceive the structural and semantic information of hard-class objects while mitigating the negative impact of noises and uneven exposures. In experiments, we demonstrate that our unique network design leads to superior segmentation performance over existing methods, featuring the strong ability of perceiving hard-class objects under adverse conditions. 2024-05-01T07:00:00Z text application/pdf https://ink.library.smu.edu.sg/sis_research/9801 info:doi/10.1145/3650032 https://ink.library.smu.edu.sg/context/sis_research/article/10801/viewcontent/NighttimeSemanticSeg_av.pdf http://creativecommons.org/licenses/by-nc-nd/4.0/ Research Collection School Of Computing and Information Systems eng Institutional Knowledge at Singapore Management University Nighttime semantic segmentation dual-branch hard class pretext task fusion refinement scheme Artificial Intelligence and Robotics Graphics and Human Computer Interfaces Transportation
institution Singapore Management University
building SMU Libraries
continent Asia
country Singapore
Singapore
content_provider SMU Libraries
collection InK@SMU
language English
topic Nighttime semantic segmentation
dual-branch
hard class
pretext task
fusion refinement scheme
Artificial Intelligence and Robotics
Graphics and Human Computer Interfaces
Transportation
spellingShingle Nighttime semantic segmentation
dual-branch
hard class
pretext task
fusion refinement scheme
Artificial Intelligence and Robotics
Graphics and Human Computer Interfaces
Transportation
LIU, Wenxi
CAI, Jiaxin
LI, Qi
LIAO, Chenyang
CAO, Jingjing
HE, Shengfeng
YU, Yuanlong
Learning nighttime semantic segmentation the hard way
description Nighttime semantic segmentation is an important but challenging research problem for autonomous driving. The major challenges lie in the small objects or regions from the under-/over-exposed areas or suffer from motion blur caused by the camera deployed on moving vehicles. To resolve this, we propose a novel hard- class-aware module that bridges the main network for full-class segmentation and the hard-class network for segmenting aforementioned hard-class objects. In specific, it exploits the shared focus of hard-class objects from the dual-stream network, enabling the contextual information flow to guide the model to concentrate on the pixels that are hard to classify. In the end, the estimated hard-class segmentation results will be utilized to infer the final results via an adaptive probabilistic fusion refinement scheme. Moreover, to overcome over- smoothing and noise caused by extreme exposures, our model is modulated by a carefully crafted pretext task of constructing an exposure-aware semantic gradient map, which guides the model to faithfully perceive the structural and semantic information of hard-class objects while mitigating the negative impact of noises and uneven exposures. In experiments, we demonstrate that our unique network design leads to superior segmentation performance over existing methods, featuring the strong ability of perceiving hard-class objects under adverse conditions.
format text
author LIU, Wenxi
CAI, Jiaxin
LI, Qi
LIAO, Chenyang
CAO, Jingjing
HE, Shengfeng
YU, Yuanlong
author_facet LIU, Wenxi
CAI, Jiaxin
LI, Qi
LIAO, Chenyang
CAO, Jingjing
HE, Shengfeng
YU, Yuanlong
author_sort LIU, Wenxi
title Learning nighttime semantic segmentation the hard way
title_short Learning nighttime semantic segmentation the hard way
title_full Learning nighttime semantic segmentation the hard way
title_fullStr Learning nighttime semantic segmentation the hard way
title_full_unstemmed Learning nighttime semantic segmentation the hard way
title_sort learning nighttime semantic segmentation the hard way
publisher Institutional Knowledge at Singapore Management University
publishDate 2024
url https://ink.library.smu.edu.sg/sis_research/9801
https://ink.library.smu.edu.sg/context/sis_research/article/10801/viewcontent/NighttimeSemanticSeg_av.pdf
_version_ 1819113149762109440