Investigating spatial and non-spatial cross-validation techniques

• Main Author: Kan, Kai Huai Feng
• Other Authors: Michele Nguyen
• Format: Final Year Project
• Language: English
• Published: Nanyang Technological University 2024
• Subjects: Computer and Information Science; Cross-validation techniques
• Online Access: https://hdl.handle.net/10356/181133
• Institution: Nanyang Technological University

Accurately evaluating predictive models, especially when built for spatial data, remains a challenge due to the limitations of traditional cross-validation techniques. While spatial cross-validation techniques have been developed to address those challenges, there is still a lack of clear guidance on when a spatial or non-spatial technique would yield the most precise and dependable assessment of a model's performance. This study investigates the effectiveness of the cross-validation method, spatial or non-spatial, in yielding accurate estimates by comparing three spatial techniques (Spatial K-Fold, Blocked, Buffered) and three non-spatial techniques (Random K-Fold, Bootstrap, Importance-Weighted). Using simulated landscapes, model performance is assessed across a range of metrics including Root Mean Square Error (RMSE), Mean Absolute Error (MAE), Coefficient of Determination (R-Squared), and Bias. The findings emphasize the importance of considering spatial autocorrelation and covariate shifts, offering practical guidance on the selection of cross-validation techniques in spatial modelling contexts.