Open-vocabulary object detection via debiased curriculum self-training

• Main Authors: Zhang, Hanlue, Guan, Dayan, Ke, Xiangrui, El Saddik, Abdulmotaleb, Lu, Shijian
• Other Authors: College of Computing and Data Science
• Format: Article
• Language: English
• Published: 2024
• Subjects: Computer and Information Science; Object detection; Open-vocabulary recognition
• Online Access: https://hdl.handle.net/10356/180718
• Institution: Nanyang Technological University

Open-vocabulary object detection aims to train a detector capable of recognizing various novel classes. Most existing studies exploit image-level weak supervision to generate pseudo object boxes for novel class training. However, the generated pseudo boxes are often noisy and biased towards base classes, leading to sub-optimal open-vocabulary detectors. We propose DCS, a novel Debiased Curriculum Self-Training technique that generates pseudo object boxes progressively and adaptively for training accurate open-vocabulary detectors. DCS consists of two complementary designs, namely, progressive pseudo-label filtering (PPF) and adaptive pseudo-label selection (APS). Specifically, PPF discards confident but mismatched detection progressively at the early training stage when the trained detector is biased towards the base classes, APS instead fuses class-aware and class-agnostic pseudo labels by prioritizing class-aware pseudo labels at the late training stage when the detector can better recognize novel classes. Without bells and whistles, DCS achieves superior detection performance over two open-vocabulary detection benchmarks.