Spatial context-aware object-attentional network for multi-label image classification
Multi-label image classification is a fundamental but challenging task in computer vision. To tackle the problem, the label-related semantic information is often exploited, but the background context and spatial semantic information of related objects are not fully utilized. To address these issues,...
Saved in:
Main Authors: | , , , , |
---|---|
Other Authors: | |
Format: | Article |
Language: | English |
Published: |
2024
|
Subjects: | |
Online Access: | https://hdl.handle.net/10356/174558 |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Institution: | Nanyang Technological University |
Language: | English |
id |
sg-ntu-dr.10356-174558 |
---|---|
record_format |
dspace |
spelling |
sg-ntu-dr.10356-1745582024-04-05T15:41:52Z Spatial context-aware object-attentional network for multi-label image classification Zhang, Jialu Ren, Jianfeng Zhang, Qian Liu, Jiang Jiang, Xudong School of Electrical and Electronic Engineering Engineering Multi-label image classification Adaptive patch expansion Multi-label image classification is a fundamental but challenging task in computer vision. To tackle the problem, the label-related semantic information is often exploited, but the background context and spatial semantic information of related objects are not fully utilized. To address these issues, a multi-branch deep neural network is proposed in this paper. The first branch is designed to extract the discriminant information from regions of interest to detect target objects. In the second branch, a spatial context-aware approach is proposed to better capture the contextual information of an object in its surroundings by using an adaptive patch expansion mechanism. It helps the detection of small objects that are easily lost without the support of context information. The third one, the object-attentional branch, exploits the spatial semantic relations between the target object and its related objects, to better detect partially occluded, small or dim objects with the support of those easily detectable objects. To better encode such relations, an attention mechanism jointly considering the spatial and semantic relations between objects is developed. Two widely used benchmark datasets for multi-labeling classification, MS COCO and PASCAL VOC, are used to evaluate the proposed framework. The experimental results demonstrate that the proposed method outperforms the state-of-the-art methods for multi-label image classification. Published version This work was supported in part by the National Natural Science Foundation of China under Grant 72071116 and in part by the Ningbo Municipal Bureau of Science and Technology under Grant 2019B10026 and Grant 2021Z089. 2024-04-02T06:41:17Z 2024-04-02T06:41:17Z 2023 Journal Article Zhang, J., Ren, J., Zhang, Q., Liu, J. & Jiang, X. (2023). Spatial context-aware object-attentional network for multi-label image classification. IEEE Transactions On Image Processing, 32, 3000-3012. https://dx.doi.org/10.1109/TIP.2023.3266161 1057-7149 https://hdl.handle.net/10356/174558 10.1109/TIP.2023.3266161 37163392 2-s2.0-85159840794 32 3000 3012 en IEEE Transactions on Image Processing © The Authors. This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ application/pdf |
institution |
Nanyang Technological University |
building |
NTU Library |
continent |
Asia |
country |
Singapore Singapore |
content_provider |
NTU Library |
collection |
DR-NTU |
language |
English |
topic |
Engineering Multi-label image classification Adaptive patch expansion |
spellingShingle |
Engineering Multi-label image classification Adaptive patch expansion Zhang, Jialu Ren, Jianfeng Zhang, Qian Liu, Jiang Jiang, Xudong Spatial context-aware object-attentional network for multi-label image classification |
description |
Multi-label image classification is a fundamental but challenging task in computer vision. To tackle the problem, the label-related semantic information is often exploited, but the background context and spatial semantic information of related objects are not fully utilized. To address these issues, a multi-branch deep neural network is proposed in this paper. The first branch is designed to extract the discriminant information from regions of interest to detect target objects. In the second branch, a spatial context-aware approach is proposed to better capture the contextual information of an object in its surroundings by using an adaptive patch expansion mechanism. It helps the detection of small objects that are easily lost without the support of context information. The third one, the object-attentional branch, exploits the spatial semantic relations between the target object and its related objects, to better detect partially occluded, small or dim objects with the support of those easily detectable objects. To better encode such relations, an attention mechanism jointly considering the spatial and semantic relations between objects is developed. Two widely used benchmark datasets for multi-labeling classification, MS COCO and PASCAL VOC, are used to evaluate the proposed framework. The experimental results demonstrate that the proposed method outperforms the state-of-the-art methods for multi-label image classification. |
author2 |
School of Electrical and Electronic Engineering |
author_facet |
School of Electrical and Electronic Engineering Zhang, Jialu Ren, Jianfeng Zhang, Qian Liu, Jiang Jiang, Xudong |
format |
Article |
author |
Zhang, Jialu Ren, Jianfeng Zhang, Qian Liu, Jiang Jiang, Xudong |
author_sort |
Zhang, Jialu |
title |
Spatial context-aware object-attentional network for multi-label image classification |
title_short |
Spatial context-aware object-attentional network for multi-label image classification |
title_full |
Spatial context-aware object-attentional network for multi-label image classification |
title_fullStr |
Spatial context-aware object-attentional network for multi-label image classification |
title_full_unstemmed |
Spatial context-aware object-attentional network for multi-label image classification |
title_sort |
spatial context-aware object-attentional network for multi-label image classification |
publishDate |
2024 |
url |
https://hdl.handle.net/10356/174558 |
_version_ |
1814047212468961280 |