FASFLNet: feature adaptive selection and fusion lightweight network for RGB-D indoor scene parsing

• Main Authors: Qian, Xiaohong, Lin, Xingyang, Yu, Lu, Zhou, Wujie
• Other Authors: School of Computer Science and Engineering
• Format: Article
• Language: English
• Published: 2023
• Subjects: Engineering::Computer science and engineering; Lightweight Backbone Model; RGB-D
• Online Access: https://hdl.handle.net/10356/171471
• Institution: Nanyang Technological University

RGB-D indoor scene parsing is a challenging task in computer vision. Conventional scene-parsing approaches based on manual feature extraction have proved inadequate in this area because indoor scenes are both unordered and complex. This study proposes a feature adaptive selection, and fusion lightweight network (FASFLNet) for RGB-D indoor scene parsing that is both efficient and accurate. The proposed FASFLNet utilizes a lightweight classification network (MobileNetV2), constituting the backbone of the feature extraction. This lightweight backbone model guarantees that FASFLNet is not only highly efficient but also provides good performance in terms of feature extraction. The additional information provided by depth images (specifically, spatial information such as the shape and scale of objects) is used in FASFLNet as supplemental information for feature-level adaptive fusion between the RGB and depth streams. Furthermore, during decoding, the features of different layers are fused from top-bottom and integrated at different layers for final pixel-level classification, resulting in an effect similar to that of pyramid supervision. Experimental results obtained on the NYU V2 and SUN RGB-D datasets indicate that the proposed FASFLNet outperforms existing state-of-the-art models and is both highly efficient and accurate.