Towards a smaller student: Capacity dynamic distillation for efficient image retrieval

• Main Authors: XIE, Yi, ZHANG, Huaidong, XU, Xuemiao, ZHU, Jianqing, HE, Shengfeng
• 格式: text
• 語言: English
• 出版: Institutional Knowledge at Singapore Management University 2023
• 主題: Deep learning architectures and techniques; Databases and Information Systems; Graphics and Human Computer Interfaces
• 在線閱讀: https://ink.library.smu.edu.sg/sis_research/8448; https://ink.library.smu.edu.sg/context/sis_research/article/9451/viewcontent/TowardsSmallerStudent_IR_av_cc_by.pdf
• 機構: Singapore Management University

Previous Knowledge Distillation based efficient image retrieval methods employ a lightweight network as the student model for fast inference. However, the lightweight student model lacks adequate representation capacity for effective knowledge imitation during the most critical early training period...