Improving performance of spatio-temporal machine learning models using forward feature selection and target-oriented validation

Importance of target-oriented validation strategies for spatio-temporal prediction models is illustrated using two case studies: (1) modelling of air temperature () in Antarctica, and (2) modelling of volumetric water content (VW) for the R.J. Cook Agronomy Farm, USA. Performance of a random k-fold...

Full description

Bibliographic Details
Published in:Environmental Modelling & Software
Main Authors: Meyer, Hanna, Reudenbach, Christoph, Hengl, Tomislav, Katurji, Marwan, Nauss, Thomas
Format: Article in Journal/Newspaper
Language:English
Published: Elsevier 2018
Subjects:
Antarc*
Antarctica
Online Access:https://oceanrep.geomar.de/id/eprint/50207/
https://oceanrep.geomar.de/id/eprint/50207/1/Meyer.pdf
https://doi.org/10.1016/j.envsoft.2017.12.001
Description
Summary:Importance of target-oriented validation strategies for spatio-temporal prediction models is illustrated using two case studies: (1) modelling of air temperature () in Antarctica, and (2) modelling of volumetric water content (VW) for the R.J. Cook Agronomy Farm, USA. Performance of a random k-fold cross-validation (CV) was compared to three target-oriented strategies: Leave-Location-Out (LLO), Leave-Time-Out (LTO), and Leave-Location-and-Time-Out (LLTO) CV. Results indicate that considerable differences between random k-fold ( = 0.9 for and 0.92 for VW) and target-oriented CV (LLO = 0.24 for and 0.49 for VW) exist, highlighting the need for target-oriented validation to avoid an overoptimistic view on models. Differences between random k-fold and target-oriented CV indicate spatial over-fitting caused by misleading variables. To decrease over-fitting, a forward feature selection in conjunction with target-oriented CV is proposed. It decreased over-fitting and simultaneously improved target-oriented performances (LLO CV = 0.47 for and 0.55 for VW).

Similar Items