Knowledge distillation in computer vision models

• Main Author: Yeoh, Yu Shyan
• Other Authors: Lin Guosheng
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2024
• Subjects: Computer and Information Science; Classification
• Online Access: https://hdl.handle.net/10356/181128
• Institution: Nanyang Technological University

Knowledge distillation has gained significant popularity in the Vision Transformer (ViT) space as a powerful approach to enhance the efficiency of a small lightweight model. Knowledge distillation enables a larger and complex “teacher” model to relay its knowledge to a smaller “student” model. Th...