The effect of softmax temperature on recent knowledge distillation algorithms

• المؤلف الرئيسي: Poh, Dominique
• مؤلفون آخرون: Weichen Liu
• التنسيق: Final Year Project
• اللغة: English
• منشور في: Nanyang Technological University 2023
• الموضوعات: Engineering::Computer science and engineering::Computing methodologies::Artificial intelligence
• الوصول للمادة أونلاين: https://hdl.handle.net/10356/172431
• المؤسسة: Nanyang Technological University

Knowledge distillation is a technique to transfer the knowledge from a large and complex teacher model to a smaller and faster student model, and an important category among methods of model compression. In this study, I survey various knowledge distillation algorithms that have been proposed in re...