Cross-domain few-shot segmentation via iterative support-query correspondence mining

Cross-Domain Few-Shot Segmentation (CD-FSS) poses the challenge of segmenting novel categories from a distinct domain using only limited exemplars. In this paper, we undertake a comprehensive study of CD-FSS and uncover two crucial insights: (i) the necessity of a fine-tuning stage to effectively tr...

Full description

Saved in:
Bibliographic Details
Main Authors: Nie, Jiahao, Xing, Yun, Zhang, Gongjie, Yan, Pei, Xiao, Aoran, Tan, Yap Peng, Kot, Alex Chichung, Lu, Shijian
Other Authors: Interdisciplinary Graduate School (IGS)
Format: Conference or Workshop Item
Language:English
Published: 2024
Subjects:
Online Access:https://hdl.handle.net/10356/177386
https://openaccess.thecvf.com/CVPR2024?day=all
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Nanyang Technological University
Language: English
id sg-ntu-dr.10356-177386
record_format dspace
spelling sg-ntu-dr.10356-1773862024-10-15T02:37:39Z Cross-domain few-shot segmentation via iterative support-query correspondence mining Nie, Jiahao Xing, Yun Zhang, Gongjie Yan, Pei Xiao, Aoran Tan, Yap Peng Kot, Alex Chichung Lu, Shijian Interdisciplinary Graduate School (IGS) School of Electrical and Electronic Engineering 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Engineering Computer vision Codes Cross-Domain Few-Shot Segmentation (CD-FSS) poses the challenge of segmenting novel categories from a distinct domain using only limited exemplars. In this paper, we undertake a comprehensive study of CD-FSS and uncover two crucial insights: (i) the necessity of a fine-tuning stage to effectively transfer the learned meta-knowledge across domains, and (ii) the overfitting risk during the naive fine-tuning due to the scarcity of novel category examples. With these insights, we propose a novel cross-domain fine-tuning strategy that addresses the challenging CD-FSS tasks. We first design Bi-directional Few-shot Prediction (BFP), which establishes support-query correspondence in bi-directional manner, crafting augmented supervision to reduce the overfitting risk. Then we further extend BFP into Iterative Few-shot Adaptor (IFA), which is a recursive framework to capture the support-query correspondence iteratively, targeting maximal exploitation of supervisory signals from the sparse novel category samples. Extensive empirical evaluations show that our method significantly outperforms the state-of-the-arts (+7.8%), which verifies that IFA tackles the cross-domain challenges and mitigates the overfitting simultaneously. Ministry of Education (MOE) Nanyang Technological University Submitted/Accepted version This study is supported by the Interdisciplinary Graduate Programme, Nanyang Technological University, and the Ministry of Education Singapore, under the Tier-1 scheme with project number RG18/22. 2024-09-18T06:54:37Z 2024-09-18T06:54:37Z 2024 Conference Paper Nie, J., Xing, Y., Zhang, G., Yan, P., Xiao, A., Tan, Y. P., Kot, A. C. & Lu, S. (2024). Cross-domain few-shot segmentation via iterative support-query correspondence mining. 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 3380-3390. https://dx.doi.org/10.1109/CVPR52733.2024.00325 979-8-3503-5300-6 2575-7075 https://hdl.handle.net/10356/177386 10.1109/CVPR52733.2024.00325 https://openaccess.thecvf.com/CVPR2024?day=all 3380 3390 en RG18/22 © 2024 IEEE. All rights reserved. This article may be downloaded for personal use only. Any other use requires prior permission of the copyright holder. The Version of Record is available online at http://doi.org/10.1109/CVPR52733.2024.00325. application/pdf
institution Nanyang Technological University
building NTU Library
continent Asia
country Singapore
Singapore
content_provider NTU Library
collection DR-NTU
language English
topic Engineering
Computer vision
Codes
spellingShingle Engineering
Computer vision
Codes
Nie, Jiahao
Xing, Yun
Zhang, Gongjie
Yan, Pei
Xiao, Aoran
Tan, Yap Peng
Kot, Alex Chichung
Lu, Shijian
Cross-domain few-shot segmentation via iterative support-query correspondence mining
description Cross-Domain Few-Shot Segmentation (CD-FSS) poses the challenge of segmenting novel categories from a distinct domain using only limited exemplars. In this paper, we undertake a comprehensive study of CD-FSS and uncover two crucial insights: (i) the necessity of a fine-tuning stage to effectively transfer the learned meta-knowledge across domains, and (ii) the overfitting risk during the naive fine-tuning due to the scarcity of novel category examples. With these insights, we propose a novel cross-domain fine-tuning strategy that addresses the challenging CD-FSS tasks. We first design Bi-directional Few-shot Prediction (BFP), which establishes support-query correspondence in bi-directional manner, crafting augmented supervision to reduce the overfitting risk. Then we further extend BFP into Iterative Few-shot Adaptor (IFA), which is a recursive framework to capture the support-query correspondence iteratively, targeting maximal exploitation of supervisory signals from the sparse novel category samples. Extensive empirical evaluations show that our method significantly outperforms the state-of-the-arts (+7.8%), which verifies that IFA tackles the cross-domain challenges and mitigates the overfitting simultaneously.
author2 Interdisciplinary Graduate School (IGS)
author_facet Interdisciplinary Graduate School (IGS)
Nie, Jiahao
Xing, Yun
Zhang, Gongjie
Yan, Pei
Xiao, Aoran
Tan, Yap Peng
Kot, Alex Chichung
Lu, Shijian
format Conference or Workshop Item
author Nie, Jiahao
Xing, Yun
Zhang, Gongjie
Yan, Pei
Xiao, Aoran
Tan, Yap Peng
Kot, Alex Chichung
Lu, Shijian
author_sort Nie, Jiahao
title Cross-domain few-shot segmentation via iterative support-query correspondence mining
title_short Cross-domain few-shot segmentation via iterative support-query correspondence mining
title_full Cross-domain few-shot segmentation via iterative support-query correspondence mining
title_fullStr Cross-domain few-shot segmentation via iterative support-query correspondence mining
title_full_unstemmed Cross-domain few-shot segmentation via iterative support-query correspondence mining
title_sort cross-domain few-shot segmentation via iterative support-query correspondence mining
publishDate 2024
url https://hdl.handle.net/10356/177386
https://openaccess.thecvf.com/CVPR2024?day=all
_version_ 1814777734117720064