Feature selection for demand forecasting incorporating external covariates

• Main Author: Mantri, Raghav
• Other Authors: Jagath C Rajapakse
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2021
• Subjects: Engineering::Computer science and engineering::Computing methodologies::Artificial intelligence
• Online Access: https://hdl.handle.net/10356/153745
• Institution: Nanyang Technological University

Feature selection is used to select a subset of features from a dataset, when developing a predictive model, and use only these selected features for prediction. This helps in not only reducing the computational cost but also in improving the forecasting performance of a machine learning model. Popular approaches in this domain engineer features from the target variable, and then proceed to independently rank these features using some scoring function. These ranks then decide the features used for forecasting the target variable. However, such scoring functions discard feature inter-dependencies, and these frameworks fail to capture the characteristics of the features that could benefit the model in prediction. To solve this, wrapper approaches are used to test out the actual feature subsets with the target model, but these approaches fail to capture the wider characteristics of a rolling window of features within each time step, when being fit in the model for evaluation. In this paper, we forecast the demand for medical products using historical in-market-sales data for a multinational medical company. We utilise machine learning methods for this purpose and perform feature selection to use only relevant features. To enhance the forecast accuracy, we incorporate external variables like Google search word trends. We also enhance this method by utilising a fixed rolling window on each of the external covariates, and reducing them dimensionally using Principal Component Analysis, to fit in the predictive model. These features are then used and have shown reduction in errors when compared to benchmark techniques. For this project, the dataset provided is confidential, so information regarding the products under study has not been shared in this report.