Causal intervention for weakly-supervised semantic segmentation

We present a causal inference framework to improve Weakly-Supervised Semantic Segmentation (WSSS). Specifically, we aim to generate better pixel-level pseudo-masks by using only image-level labels --- the most crucial step in WSSS. We attribute the cause of the ambiguous boundaries of pseudo-masks t...

Full description

Saved in:
Bibliographic Details
Main Authors: ZHANG Dong, ZHANG, Hanwang, TANG, Jinhui, HUA, Xian-Sheng, SUN, Qianru
Format: text
Language:English
Published: Institutional Knowledge at Singapore Management University 2020
Subjects:
Online Access:https://ink.library.smu.edu.sg/sis_research/5597
https://ink.library.smu.edu.sg/context/sis_research/article/6600/viewcontent/NeurIPS_2020_causal_intervention_for_weakly_supervised_semantic_segmentation_Paper.pdf
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Singapore Management University
Language: English
id sg-smu-ink.sis_research-6600
record_format dspace
spelling sg-smu-ink.sis_research-66002021-01-07T13:58:58Z Causal intervention for weakly-supervised semantic segmentation ZHANG Dong, ZHANG, Hanwang TANG, Jinhui HUA, Xian-Sheng SUN, Qianru We present a causal inference framework to improve Weakly-Supervised Semantic Segmentation (WSSS). Specifically, we aim to generate better pixel-level pseudo-masks by using only image-level labels --- the most crucial step in WSSS. We attribute the cause of the ambiguous boundaries of pseudo-masks to the confounding context, e.g., the correct image-level classification of "horse'' and "person'' may be not only due to the recognition of each instance, but also their co-occurrence context, making the model inspection (e.g., CAM) hard to distinguish between the boundaries. Inspired by this, we propose a structural causal model to analyze the causalities among images, contexts, and class labels. Based on it, we develop a new method: Context Adjustment (CONTA), to remove the confounding bias in image-level classification and thus provide better pseudo-masks as ground-truth for the subsequent segmentation model. On PASCAL VOC 2012 and MS-COCO, we show that CONTA boosts various popular WSSS methods to new state-of-the-arts. 2020-12-01T08:00:00Z text application/pdf https://ink.library.smu.edu.sg/sis_research/5597 https://ink.library.smu.edu.sg/context/sis_research/article/6600/viewcontent/NeurIPS_2020_causal_intervention_for_weakly_supervised_semantic_segmentation_Paper.pdf http://creativecommons.org/licenses/by-nc-nd/4.0/ Research Collection School Of Computing and Information Systems eng Institutional Knowledge at Singapore Management University Artificial Intelligence and Robotics Databases and Information Systems
institution Singapore Management University
building SMU Libraries
continent Asia
country Singapore
Singapore
content_provider SMU Libraries
collection InK@SMU
language English
topic Artificial Intelligence and Robotics
Databases and Information Systems
spellingShingle Artificial Intelligence and Robotics
Databases and Information Systems
ZHANG Dong,
ZHANG, Hanwang
TANG, Jinhui
HUA, Xian-Sheng
SUN, Qianru
Causal intervention for weakly-supervised semantic segmentation
description We present a causal inference framework to improve Weakly-Supervised Semantic Segmentation (WSSS). Specifically, we aim to generate better pixel-level pseudo-masks by using only image-level labels --- the most crucial step in WSSS. We attribute the cause of the ambiguous boundaries of pseudo-masks to the confounding context, e.g., the correct image-level classification of "horse'' and "person'' may be not only due to the recognition of each instance, but also their co-occurrence context, making the model inspection (e.g., CAM) hard to distinguish between the boundaries. Inspired by this, we propose a structural causal model to analyze the causalities among images, contexts, and class labels. Based on it, we develop a new method: Context Adjustment (CONTA), to remove the confounding bias in image-level classification and thus provide better pseudo-masks as ground-truth for the subsequent segmentation model. On PASCAL VOC 2012 and MS-COCO, we show that CONTA boosts various popular WSSS methods to new state-of-the-arts.
format text
author ZHANG Dong,
ZHANG, Hanwang
TANG, Jinhui
HUA, Xian-Sheng
SUN, Qianru
author_facet ZHANG Dong,
ZHANG, Hanwang
TANG, Jinhui
HUA, Xian-Sheng
SUN, Qianru
author_sort ZHANG Dong,
title Causal intervention for weakly-supervised semantic segmentation
title_short Causal intervention for weakly-supervised semantic segmentation
title_full Causal intervention for weakly-supervised semantic segmentation
title_fullStr Causal intervention for weakly-supervised semantic segmentation
title_full_unstemmed Causal intervention for weakly-supervised semantic segmentation
title_sort causal intervention for weakly-supervised semantic segmentation
publisher Institutional Knowledge at Singapore Management University
publishDate 2020
url https://ink.library.smu.edu.sg/sis_research/5597
https://ink.library.smu.edu.sg/context/sis_research/article/6600/viewcontent/NeurIPS_2020_causal_intervention_for_weakly_supervised_semantic_segmentation_Paper.pdf
_version_ 1770575523436560384