Generalized logit adjustment: Calibrating fine-tuned models by removing label bias in foundation models

• Main Authors: ZHU, Beier, TANG, Kaihua, SUN, Qianru, ZHANG, Hanwang
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2023
• Subjects: Databases and Information Systems
• Online Access: https://ink.library.smu.edu.sg/sis_research/8473; https://ink.library.smu.edu.sg/context/sis_research/article/9476/viewcontent/Generalized_Logit_Adjustment__Calibrating_Fine_tuned_Models_by_Removing_Label_Bias_in_Foundation_Models__1_.pdf
• Institution: Singapore Management University

Foundation models like CLIP allow zero-shot transfer on various tasks without additional training data. Yet, the zero-shot performance is less competitive than a fully supervised one. Thus, to enhance the performance, fine-tuning and ensembling are also commonly adopted to better fit the downstream...