Cross-domain few-shot segmentation via iterative support-query correspondence mining

• Main Authors: Nie, Jiahao, Xing, Yun, Zhang, Gongjie, Yan, Pei, Xiao, Aoran, Tan, Yap Peng, Kot, Alex Chichung, Lu, Shijian
• Other Authors: Interdisciplinary Graduate School (IGS)
• Format: Conference or Workshop Item
• Language: English
• Published: 2024
• Subjects: Engineering; Computer vision; Codes
• Online Access: https://hdl.handle.net/10356/177386; https://openaccess.thecvf.com/CVPR2024?day=all
• Institution: Nanyang Technological University

Cross-Domain Few-Shot Segmentation (CD-FSS) poses the challenge of segmenting novel categories from a distinct domain using only limited exemplars. In this paper, we undertake a comprehensive study of CD-FSS and uncover two crucial insights: (i) the necessity of a fine-tuning stage to effectively transfer the learned meta-knowledge across domains, and (ii) the overfitting risk during the naive fine-tuning due to the scarcity of novel category examples. With these insights, we propose a novel cross-domain fine-tuning strategy that addresses the challenging CD-FSS tasks. We first design Bi-directional Few-shot Prediction (BFP), which establishes support-query correspondence in bi-directional manner, crafting augmented supervision to reduce the overfitting risk. Then we further extend BFP into Iterative Few-shot Adaptor (IFA), which is a recursive framework to capture the support-query correspondence iteratively, targeting maximal exploitation of supervisory signals from the sparse novel category samples. Extensive empirical evaluations show that our method significantly outperforms the state-of-the-arts (+7.8%), which verifies that IFA tackles the cross-domain challenges and mitigates the overfitting simultaneously.