Towards a smaller student: Capacity dynamic distillation for efficient image retrieval

• Main Authors: XIE, Yi, ZHANG, Huaidong, XU, Xuemiao, ZHU, Jianqing, HE, Shengfeng
• Format: text
• Language: English
• Published: Institutional Knowledge at Singapore Management University 2023
• Subjects: Deep learning architectures and techniques; Databases and Information Systems; Graphics and Human Computer Interfaces
• Online Access: https://ink.library.smu.edu.sg/sis_research/8448; https://ink.library.smu.edu.sg/context/sis_research/article/9451/viewcontent/TowardsSmallerStudent_IR_av_cc_by.pdf
• Institution: Singapore Management University

Previous Knowledge Distillation based efficient image retrieval methods employ a lightweight network as the student model for fast inference. However, the lightweight student model lacks adequate representation capacity for effective knowledge imitation during the most critical early training period...