Boosting knowledge distillation and interpretability

• Main Author: Song, Huan
• Other Authors: Ponnuthurai Nagaratnam Suganthan
• Format: Thesis-Master by Coursework
• Language: English
• Published: Nanyang Technological University 2021
• Subjects: Engineering::Computer science and engineering::Computing methodologies::Pattern recognition
• Online Access: https://hdl.handle.net/10356/150315
• Institution: Nanyang Technological University

Deep Neural Network (DNN) can be applied in many fields to predict classification and can obtain high accuracy. However, Deep Neural Network is a black box, which means that it’s hard to explain how the Deep Neural Network can derive specific classification directly. The generally accepted interpretable model is the decision tree. Although decision tree classification accuracy is not as good as deep neural networks, it is a more intuitive and interpretable common model. By combining a deep neural network with a decision tree, it is possible to show the inner architecture of model without loss of accuracy. It can be helpful to learn why certain inputs can get specific output by distilling the knowledge from DNN model into a decision tree.