Automatic hyperparameter optimization for machine learning

• Main Author: Tan, Xavier Jun Sheng
• Other Authors: Mao Kezhi
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2020
• Subjects: Engineering::Electrical and electronic engineering
• Online Access: https://hdl.handle.net/10356/139152
• Institution: Nanyang Technological University

This project focuses on the concept of hyperparameters in a Machine Learning classifi- cation problem. Hyperparameters are parameter values that have direct control over the behaviour of the Machine Learning classification model. Automatic hyperparameter op- timization is an active area of research which aims to facilitate the exhaustive process of manually tuning them by hand. In recent years, the success of the bayesian optimization model in the sequential model-based optimization approach shed light and direction for the future of hyperparameter optimization. It led to the development of many automatic hyperparameter optimization frameworks such as Hyperopt, Scikit-optimize and Optuna, to name a few. Each differs in the underlying optimization algorithms and performance. A considerable amount of effort is required to understand the complex and rigid struc- ture of the framework, and they are not easily reproducible onto other practical Machine Learning problems. This project aims to address those problems through the development of a unified func- tion combining Hyperopt, Scikit-optimize and Optuna, offering more flexibility, improved usability and visual aids. Experiments conducted demonstrated that the program devel- oped had consistently produced a model that outperforms an untuned classification model and that Hyperopt is superior in terms of accuracy score and speed. However, an ob- servation from the results was that the program hadn’t accounted for several factors which could have impeded with the performance of the random forest classifier. Hence, as future works, functional ANOVA can be incorporated into the program to address the limitations of the program.