Pyramid graph networks with connection attentions for region-based one-shot semantic segmentation
One-shot image segmentation aims to undertake the segmentation task of a novel class with only one training image available. The difficulty lies in that image segmentation has structured data representations, which yields a many-to-many message passing problem. Previous methods often simplify it to...
Saved in:
Main Authors: | , , , , , |
---|---|
Other Authors: | |
Format: | Conference or Workshop Item |
Language: | English |
Published: |
2020
|
Subjects: | |
Online Access: | https://hdl.handle.net/10356/144393 |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Institution: | Nanyang Technological University |
Language: | English |
id |
sg-ntu-dr.10356-144393 |
---|---|
record_format |
dspace |
spelling |
sg-ntu-dr.10356-1443932020-11-03T02:44:36Z Pyramid graph networks with connection attentions for region-based one-shot semantic segmentation Zhang, Chi Lin, Guosheng Liu, Fayao Guo, Jiushuang Wu, Qingyao Yao, Rui School of Computer Science and Engineering 2019 IEEE/CVF International Conference on Computer Vision (ICCV) Engineering::Computer science and engineering Task Analysis Image Segmentation One-shot image segmentation aims to undertake the segmentation task of a novel class with only one training image available. The difficulty lies in that image segmentation has structured data representations, which yields a many-to-many message passing problem. Previous methods often simplify it to a one-to-many problem by squeezing support data to a global descriptor. However, a mixed global representation drops the data structure and information of individual elements. In this paper, we propose to model structured segmentation data with graphs and apply attentive graph reasoning to propagate label information from support data to query data. The graph attention mechanism could establish the element-to-element correspondence across structured data by learning attention weights between connected graph nodes. To capture correspondence at different semantic levels, we further propose a pyramid-like structure that models different sizes of image regions as graph nodes and undertakes graph reasoning at different levels. Experiments on PASCAL VOC 2012 dataset demonstrate that our proposed network significantly outperforms the baseline method and leads to new state-of-the-art performance on 1-shot and 5-shot segmentation benchmarks. AI Singapore Ministry of Education (MOE) Accepted version This work is supported by the National Research Foundation Singapore under its AI Singapore Programme [AISG-RP-2018-003] and the MOE Tier-1 research grant [RG126/17 (S)]. We would like to thank NVIDIA for GPU donation. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not reflect the views of National Research Foundation, Singapore. 2020-11-03T02:44:36Z 2020-11-03T02:44:36Z 2019 Conference Paper Zhang, C., Lin, G., Liu, F., Guo, J., Wu, Q., & Yao, R. (2019). Pyramid graph networks with connection attentions for region-based one-shot semantic segmentation. Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV). doi:10.1109/ICCV.2019.00968 https://hdl.handle.net/10356/144393 10.1109/ICCV.2019.00968 en AISG-RP-2018-003 RG126/17 (S) © 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work is available at: https://doi.org/10.1109/ICCV.2019.00968 application/pdf |
institution |
Nanyang Technological University |
building |
NTU Library |
continent |
Asia |
country |
Singapore Singapore |
content_provider |
NTU Library |
collection |
DR-NTU |
language |
English |
topic |
Engineering::Computer science and engineering Task Analysis Image Segmentation |
spellingShingle |
Engineering::Computer science and engineering Task Analysis Image Segmentation Zhang, Chi Lin, Guosheng Liu, Fayao Guo, Jiushuang Wu, Qingyao Yao, Rui Pyramid graph networks with connection attentions for region-based one-shot semantic segmentation |
description |
One-shot image segmentation aims to undertake the segmentation task of a novel class with only one training image available. The difficulty lies in that image segmentation has structured data representations, which yields a many-to-many message passing problem. Previous methods often simplify it to a one-to-many problem by squeezing support data to a global descriptor. However, a mixed global representation drops the data structure and information of individual elements. In this paper, we propose to model structured segmentation data with graphs and apply attentive graph reasoning to propagate label information from support data to query data. The graph attention mechanism could establish the element-to-element correspondence across structured data by learning attention weights between connected graph nodes. To capture correspondence at different semantic levels, we further propose a pyramid-like structure that models different sizes of image regions as graph nodes and undertakes graph reasoning at different levels. Experiments on PASCAL VOC 2012 dataset demonstrate that our proposed network significantly outperforms the baseline method and leads to new state-of-the-art performance on 1-shot and 5-shot segmentation benchmarks. |
author2 |
School of Computer Science and Engineering |
author_facet |
School of Computer Science and Engineering Zhang, Chi Lin, Guosheng Liu, Fayao Guo, Jiushuang Wu, Qingyao Yao, Rui |
format |
Conference or Workshop Item |
author |
Zhang, Chi Lin, Guosheng Liu, Fayao Guo, Jiushuang Wu, Qingyao Yao, Rui |
author_sort |
Zhang, Chi |
title |
Pyramid graph networks with connection attentions for region-based one-shot semantic segmentation |
title_short |
Pyramid graph networks with connection attentions for region-based one-shot semantic segmentation |
title_full |
Pyramid graph networks with connection attentions for region-based one-shot semantic segmentation |
title_fullStr |
Pyramid graph networks with connection attentions for region-based one-shot semantic segmentation |
title_full_unstemmed |
Pyramid graph networks with connection attentions for region-based one-shot semantic segmentation |
title_sort |
pyramid graph networks with connection attentions for region-based one-shot semantic segmentation |
publishDate |
2020 |
url |
https://hdl.handle.net/10356/144393 |
_version_ |
1683493304588566528 |