The effect of softmax temperature on recent knowledge distillation algorithms

• 主要作者: Poh, Dominique
• 其他作者: Weichen Liu
• 格式: Final Year Project
• 語言: English
• 出版: Nanyang Technological University 2023
• 主題: Engineering::Computer science and engineering::Computing methodologies::Artificial intelligence
• 在線閱讀: https://hdl.handle.net/10356/172431

Knowledge distillation is a technique to transfer the knowledge from a large and complex teacher model to a smaller and faster student model, and an important category among methods of model compression. In this study, I survey various knowledge distillation algorithms that have been proposed in re...