Class re-activation maps for weakly-supervised semantic segmentation

• Main Authors: CHEN, Zhaozheng, WANG, Tan, WU, Xiongwei, HUA, Xian-Sheng, ZHANG, Hanwang, SUN, Qianru
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2022
• Subjects: class activation maps; weakly supervised learning; semantic segmentation; Databases and Information Systems; Graphics and Human Computer Interfaces
• Online Access: https://ink.library.smu.edu.sg/sis_research/7511; https://ink.library.smu.edu.sg/context/sis_research/article/8514/viewcontent/Chen_Class_Re_Activation_Maps_for_Weakly_Supervised_Semantic_Segmentation_CVPR_2022_paper.pdf
• Institution: Singapore Management University

Extracting class activation maps (CAM) is arguably the most standard step of generating pseudo masks for weakly supervised semantic segmentation (WSSS). Yet, we find that the crux of the unsatisfactory pseudo masks is the binary cross-entropy loss (BCE) widely used in CAM. Specifically, due to the sum-over-class pooling nature of BCE, each pixel in CAM may be responsive to multiple classes co-occurring in the same receptive field. To this end, we introduce an embarrassingly simple yet surprisingly effective method: Reactivating the converged CAM with BCE by using softmax crossentropy loss (SCE), dubbed ReCAM. Given an image, we use CAM to extract the feature pixels of each single class, and use them with the class label to learn another fully-connected layer (after the backbone) with SCE. Once converged, we extract ReCAM in the same way as in CAM.