Reproducibility of an Earth System Model under a Change in Computing Environment

• Main Authors: Massonnet, François, Ménégoz, Martin, Acosta, Mario, Yepes-Arbós, Xavier, Exarchout, Eleftheria, Doblas-Reyes, Francisco
• Other Authors: UCL - SST/ELI/ELIC - Earth & Climate
• Format: Report
• Language: English
• Published: 2018
• Subjects: High Performance Computing; Climate model; Reproducibility; Antarctic; Antarc*
• Online Access: http://hdl.handle.net/2078.1/203820

Most Earth System Models (ESMs) are running under different high performance computing (HPC) environments. This has several advantages, from allowing different groups to work with the same tool in parallel, to leveraging the burden of ensemble climate simulations but also offering alternative solutions in case of shutdown (expected or not) of any of the environments. However, for obvious scientific reasons, it is critical to ensure that ESMs provide reproducible results under changes in computing environment. While reproducibility in a strong sense (bit-for-bit) is in general unfeasible, it can be hoped that results obtained under one computing environment are statistically indistinguishable from those obtained under another environment (reproducibility in a weak sense). Here, we develop a protocol to assess the reproducibility (in a weak sense) of the EC-Earth ESM. Using two versions of that ESM, we present one case of non-reproducibility and one case of reproducibility. The non-reproducible case occurs with the older version of the model and likely finds its origin in the treatment of river runoffs along Antarctic coasts. However, it was not possible to nail the problem further down. By contrast, the more recent version of the model provides reproducible results. The protocol introduced in this note can help users of EC-Earth and, by extensión other ESMs, assessing whether their model can be ported from one HPC environment to another. Our results and experience in this work suggest that the default assumption should be that an ESM is not reproducible under changes in the HPC environment, until proven otherwise.