Cross-domain few-shot segmentation via iterative support-query correspondence mining
Cross-Domain Few-Shot Segmentation (CD-FSS) poses the challenge of segmenting novel categories from a distinct domain using only limited exemplars. In this paper, we undertake a comprehensive study of CD-FSS and uncover two crucial insights: (i) the necessity of a fine-tuning stage to effectively tr...
Saved in:
Main Authors: | , , , , , , , |
---|---|
Other Authors: | |
Format: | Conference or Workshop Item |
Language: | English |
Published: |
2024
|
Subjects: | |
Online Access: | https://hdl.handle.net/10356/177386 https://openaccess.thecvf.com/CVPR2024?day=all |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
Institution: | Nanyang Technological University |
Language: | English |
id |
sg-ntu-dr.10356-177386 |
---|---|
record_format |
dspace |
spelling |
sg-ntu-dr.10356-1773862024-10-15T02:37:39Z Cross-domain few-shot segmentation via iterative support-query correspondence mining Nie, Jiahao Xing, Yun Zhang, Gongjie Yan, Pei Xiao, Aoran Tan, Yap Peng Kot, Alex Chichung Lu, Shijian Interdisciplinary Graduate School (IGS) School of Electrical and Electronic Engineering 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Engineering Computer vision Codes Cross-Domain Few-Shot Segmentation (CD-FSS) poses the challenge of segmenting novel categories from a distinct domain using only limited exemplars. In this paper, we undertake a comprehensive study of CD-FSS and uncover two crucial insights: (i) the necessity of a fine-tuning stage to effectively transfer the learned meta-knowledge across domains, and (ii) the overfitting risk during the naive fine-tuning due to the scarcity of novel category examples. With these insights, we propose a novel cross-domain fine-tuning strategy that addresses the challenging CD-FSS tasks. We first design Bi-directional Few-shot Prediction (BFP), which establishes support-query correspondence in bi-directional manner, crafting augmented supervision to reduce the overfitting risk. Then we further extend BFP into Iterative Few-shot Adaptor (IFA), which is a recursive framework to capture the support-query correspondence iteratively, targeting maximal exploitation of supervisory signals from the sparse novel category samples. Extensive empirical evaluations show that our method significantly outperforms the state-of-the-arts (+7.8%), which verifies that IFA tackles the cross-domain challenges and mitigates the overfitting simultaneously. Ministry of Education (MOE) Nanyang Technological University Submitted/Accepted version This study is supported by the Interdisciplinary Graduate Programme, Nanyang Technological University, and the Ministry of Education Singapore, under the Tier-1 scheme with project number RG18/22. 2024-09-18T06:54:37Z 2024-09-18T06:54:37Z 2024 Conference Paper Nie, J., Xing, Y., Zhang, G., Yan, P., Xiao, A., Tan, Y. P., Kot, A. C. & Lu, S. (2024). Cross-domain few-shot segmentation via iterative support-query correspondence mining. 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 3380-3390. https://dx.doi.org/10.1109/CVPR52733.2024.00325 979-8-3503-5300-6 2575-7075 https://hdl.handle.net/10356/177386 10.1109/CVPR52733.2024.00325 https://openaccess.thecvf.com/CVPR2024?day=all 3380 3390 en RG18/22 © 2024 IEEE. All rights reserved. This article may be downloaded for personal use only. Any other use requires prior permission of the copyright holder. The Version of Record is available online at http://doi.org/10.1109/CVPR52733.2024.00325. application/pdf |
institution |
Nanyang Technological University |
building |
NTU Library |
continent |
Asia |
country |
Singapore Singapore |
content_provider |
NTU Library |
collection |
DR-NTU |
language |
English |
topic |
Engineering Computer vision Codes |
spellingShingle |
Engineering Computer vision Codes Nie, Jiahao Xing, Yun Zhang, Gongjie Yan, Pei Xiao, Aoran Tan, Yap Peng Kot, Alex Chichung Lu, Shijian Cross-domain few-shot segmentation via iterative support-query correspondence mining |
description |
Cross-Domain Few-Shot Segmentation (CD-FSS) poses the challenge of segmenting novel categories from a distinct domain using only limited exemplars. In this paper, we undertake a comprehensive study of CD-FSS and uncover two crucial insights: (i) the necessity of a fine-tuning stage to effectively transfer the learned meta-knowledge across domains, and (ii) the overfitting risk during the naive fine-tuning due to the scarcity of novel category examples. With these insights, we propose a novel cross-domain fine-tuning strategy that addresses the challenging CD-FSS tasks. We first design Bi-directional Few-shot Prediction (BFP), which establishes support-query correspondence in bi-directional manner, crafting augmented supervision to reduce the overfitting risk. Then we further extend BFP into Iterative Few-shot Adaptor (IFA), which is a recursive framework to capture the support-query correspondence iteratively, targeting maximal exploitation of supervisory signals from the sparse novel category samples. Extensive empirical evaluations show that our method significantly outperforms the state-of-the-arts (+7.8%), which verifies that IFA tackles the cross-domain challenges and mitigates the overfitting simultaneously. |
author2 |
Interdisciplinary Graduate School (IGS) |
author_facet |
Interdisciplinary Graduate School (IGS) Nie, Jiahao Xing, Yun Zhang, Gongjie Yan, Pei Xiao, Aoran Tan, Yap Peng Kot, Alex Chichung Lu, Shijian |
format |
Conference or Workshop Item |
author |
Nie, Jiahao Xing, Yun Zhang, Gongjie Yan, Pei Xiao, Aoran Tan, Yap Peng Kot, Alex Chichung Lu, Shijian |
author_sort |
Nie, Jiahao |
title |
Cross-domain few-shot segmentation via iterative support-query correspondence mining |
title_short |
Cross-domain few-shot segmentation via iterative support-query correspondence mining |
title_full |
Cross-domain few-shot segmentation via iterative support-query correspondence mining |
title_fullStr |
Cross-domain few-shot segmentation via iterative support-query correspondence mining |
title_full_unstemmed |
Cross-domain few-shot segmentation via iterative support-query correspondence mining |
title_sort |
cross-domain few-shot segmentation via iterative support-query correspondence mining |
publishDate |
2024 |
url |
https://hdl.handle.net/10356/177386 https://openaccess.thecvf.com/CVPR2024?day=all |
_version_ |
1814777734117720064 |