Computation-efficient knowledge distillation via uncertainty-aware mixup

• Main Authors: Xu, Guodong, Liu, Ziwei, Loy, Chen Change
• Other Authors: School of Computer Science and Engineering
• Format: Article
• Language: English
• Published: 2023
• Subjects: Engineering::Computer science and engineering; Knowledge Distillation; Training Cost
• Online Access: https://hdl.handle.net/10356/172038
• Institution: Nanyang Technological University

Knowledge distillation (KD) has emerged as an essential technique not only for model compression, but also other learning tasks such as continual learning. Given the richer application spectrum and potential online usage of KD, knowledge distillation efficiency becomes a pivotal component. In this w...