Replicability of the EC-Earth3 Earth system model under a change in computing environment
Most Earth system models (ESMs) are running under different high-performance computing (HPC) environments. This has several advantages, from allowing different groups to work with the same tool in parallel to leveraging the burden of ensemble climate simulations, but it also offers alternative solut...
Published in: | Geoscientific Model Development |
---|---|
Main Authors: | , , , , , |
Other Authors: | , |
Format: | Article in Journal/Newspaper |
Language: | English |
Published: |
2020
|
Subjects: | |
Online Access: | http://hdl.handle.net/2117/193046 https://doi.org/10.5194/gmd-13-1165-2020 |
id |
ftupcatalunyair:oai:upcommons.upc.edu:2117/193046 |
---|---|
record_format |
openpolar |
institution |
Open Polar |
collection |
Universitat Politècnica de Catalunya, BarcelonaTech: UPCommons - Global access to UPC knowledge |
op_collection_id |
ftupcatalunyair |
language |
English |
topic |
Àrees temàtiques de la UPC::Desenvolupament humà i sostenible::Degradació ambiental::Canvi climàtic Àrees temàtiques de la UPC::Matemàtiques i estadística::Matemàtica aplicada a les ciències Weather forecasting -- Mathematical models Climatic changes -- Mathematical models High performance computing Previsió del temps -- Models matemàtics Canvis climàtics -- Models matemàtics Càlcul intensiu (Informàtica) |
spellingShingle |
Àrees temàtiques de la UPC::Desenvolupament humà i sostenible::Degradació ambiental::Canvi climàtic Àrees temàtiques de la UPC::Matemàtiques i estadística::Matemàtica aplicada a les ciències Weather forecasting -- Mathematical models Climatic changes -- Mathematical models High performance computing Previsió del temps -- Models matemàtics Canvis climàtics -- Models matemàtics Càlcul intensiu (Informàtica) Massonnet, François Menegoz, Martín Acosta Cobos, Mario César Yepes-Arbós, Xavier Exarchou, Eleftheria Doblas Reyes, Francisco Replicability of the EC-Earth3 Earth system model under a change in computing environment |
topic_facet |
Àrees temàtiques de la UPC::Desenvolupament humà i sostenible::Degradació ambiental::Canvi climàtic Àrees temàtiques de la UPC::Matemàtiques i estadística::Matemàtica aplicada a les ciències Weather forecasting -- Mathematical models Climatic changes -- Mathematical models High performance computing Previsió del temps -- Models matemàtics Canvis climàtics -- Models matemàtics Càlcul intensiu (Informàtica) |
description |
Most Earth system models (ESMs) are running under different high-performance computing (HPC) environments. This has several advantages, from allowing different groups to work with the same tool in parallel to leveraging the burden of ensemble climate simulations, but it also offers alternative solutions in the case of shutdown (expected or not) of any of the environments. However, for obvious scientific reasons, it is critical to ensure that ESMs provide identical results under changes in computing environment. While strict bit-for-bit reproducibility is not always guaranteed with ESMs, it is desirable that results obtained under one computing environment are at least statistically indistinguishable from those obtained under another environment, which we term a “replicability” condition following the metrology nomenclature. Here, we develop a protocol to assess the replicability of the EC-Earth ESM. Using two versions of EC-Earth, we present one case of non-replicability and one case of replicability. The non-replicable case occurs with the older version of the model and likely finds its origin in the treatment of river runoff along Antarctic coasts. By contrast, the more recent version of the model provides replicable results. The methodology presented here has been adopted as a standard test by the EC-Earth consortium (27 institutions in Europe) to evaluate the replicability of any new model version across platforms, including for CMIP6 experiments. To a larger extent, it can be used to assess whether other ESMs can safely be ported from one HPC environment to another for studying climate-related questions. Our results and experience with this work suggest that the default assumption should be that ESMs are not replicable under changes in the HPC environment, until proven otherwise. This research has been supported by the EU Commission – H2020 (grant nos. 727862: APPLICATE, 312979: IS-ENES2, 675191: ESiWACE, and 641727: PRIMAVERA) and the Spanish Ministerio de Economía y Competitividad (MINECO) (grant no. CGL2015-70353-R: HIATUS). Peer Reviewed Postprint (published version) |
author2 |
Universitat Politècnica de Catalunya. Departament d'Arquitectura de Computadors Barcelona Supercomputing Center |
format |
Article in Journal/Newspaper |
author |
Massonnet, François Menegoz, Martín Acosta Cobos, Mario César Yepes-Arbós, Xavier Exarchou, Eleftheria Doblas Reyes, Francisco |
author_facet |
Massonnet, François Menegoz, Martín Acosta Cobos, Mario César Yepes-Arbós, Xavier Exarchou, Eleftheria Doblas Reyes, Francisco |
author_sort |
Massonnet, François |
title |
Replicability of the EC-Earth3 Earth system model under a change in computing environment |
title_short |
Replicability of the EC-Earth3 Earth system model under a change in computing environment |
title_full |
Replicability of the EC-Earth3 Earth system model under a change in computing environment |
title_fullStr |
Replicability of the EC-Earth3 Earth system model under a change in computing environment |
title_full_unstemmed |
Replicability of the EC-Earth3 Earth system model under a change in computing environment |
title_sort |
replicability of the ec-earth3 earth system model under a change in computing environment |
publishDate |
2020 |
url |
http://hdl.handle.net/2117/193046 https://doi.org/10.5194/gmd-13-1165-2020 |
geographic |
Antarctic |
geographic_facet |
Antarctic |
genre |
Antarc* Antarctic Arctic |
genre_facet |
Antarc* Antarctic Arctic |
op_relation |
https://gmd.copernicus.org/articles/13/1165/2020/ info:eu-repo/grantAgreement/EC/FP7/312979/EU/Infrastructure for the European Network for Earth System modelling/IS-ENES2 info:eu-repo/grantAgreement/EC/H2020/727862/EU/Advanced Prediction in Polar regions and beyond: Modelling, observing system design and LInkages associated with ArctiC ClimATE change/APPLICATE info:eu-repo/grantAgreement/EC/H2020/675191/EU/Excellence in SImulation of Weather and Climate in Europe/ESiWACE info:eu-repo/grantAgreement/EC/H2020/641727/EU/PRocess-based climate sIMulation: AdVances in high resolution modelling and European climate Risk Assessment/PRIMAVERA info:eu-repo/grantAgreement/MINECO/1PE/CGL2015-70353-R Massonnet, F. [et al.]. Replicability of the EC-Earth3 Earth system model under a change in computing environment. "Geoscientific model development", 12 Març 2020, vol. 13, p. 1165-1178. 1991-959X http://hdl.handle.net/2117/193046 doi:10.5194/gmd-13-1165-2020 |
op_rights |
Attribution 4.0 International https://creativecommons.org/licenses/by/4.0/ Open Access |
op_rightsnorm |
CC-BY |
op_doi |
https://doi.org/10.5194/gmd-13-1165-2020 |
container_title |
Geoscientific Model Development |
container_volume |
13 |
container_issue |
3 |
container_start_page |
1165 |
op_container_end_page |
1178 |
_version_ |
1766013113338953728 |
spelling |
ftupcatalunyair:oai:upcommons.upc.edu:2117/193046 2023-05-15T13:30:50+02:00 Replicability of the EC-Earth3 Earth system model under a change in computing environment Massonnet, François Menegoz, Martín Acosta Cobos, Mario César Yepes-Arbós, Xavier Exarchou, Eleftheria Doblas Reyes, Francisco Universitat Politècnica de Catalunya. Departament d'Arquitectura de Computadors Barcelona Supercomputing Center 2020-03-12 14 p. application/pdf http://hdl.handle.net/2117/193046 https://doi.org/10.5194/gmd-13-1165-2020 eng eng https://gmd.copernicus.org/articles/13/1165/2020/ info:eu-repo/grantAgreement/EC/FP7/312979/EU/Infrastructure for the European Network for Earth System modelling/IS-ENES2 info:eu-repo/grantAgreement/EC/H2020/727862/EU/Advanced Prediction in Polar regions and beyond: Modelling, observing system design and LInkages associated with ArctiC ClimATE change/APPLICATE info:eu-repo/grantAgreement/EC/H2020/675191/EU/Excellence in SImulation of Weather and Climate in Europe/ESiWACE info:eu-repo/grantAgreement/EC/H2020/641727/EU/PRocess-based climate sIMulation: AdVances in high resolution modelling and European climate Risk Assessment/PRIMAVERA info:eu-repo/grantAgreement/MINECO/1PE/CGL2015-70353-R Massonnet, F. [et al.]. Replicability of the EC-Earth3 Earth system model under a change in computing environment. "Geoscientific model development", 12 Març 2020, vol. 13, p. 1165-1178. 1991-959X http://hdl.handle.net/2117/193046 doi:10.5194/gmd-13-1165-2020 Attribution 4.0 International https://creativecommons.org/licenses/by/4.0/ Open Access CC-BY Àrees temàtiques de la UPC::Desenvolupament humà i sostenible::Degradació ambiental::Canvi climàtic Àrees temàtiques de la UPC::Matemàtiques i estadística::Matemàtica aplicada a les ciències Weather forecasting -- Mathematical models Climatic changes -- Mathematical models High performance computing Previsió del temps -- Models matemàtics Canvis climàtics -- Models matemàtics Càlcul intensiu (Informàtica) Article 2020 ftupcatalunyair https://doi.org/10.5194/gmd-13-1165-2020 2021-02-26T15:22:22Z Most Earth system models (ESMs) are running under different high-performance computing (HPC) environments. This has several advantages, from allowing different groups to work with the same tool in parallel to leveraging the burden of ensemble climate simulations, but it also offers alternative solutions in the case of shutdown (expected or not) of any of the environments. However, for obvious scientific reasons, it is critical to ensure that ESMs provide identical results under changes in computing environment. While strict bit-for-bit reproducibility is not always guaranteed with ESMs, it is desirable that results obtained under one computing environment are at least statistically indistinguishable from those obtained under another environment, which we term a “replicability” condition following the metrology nomenclature. Here, we develop a protocol to assess the replicability of the EC-Earth ESM. Using two versions of EC-Earth, we present one case of non-replicability and one case of replicability. The non-replicable case occurs with the older version of the model and likely finds its origin in the treatment of river runoff along Antarctic coasts. By contrast, the more recent version of the model provides replicable results. The methodology presented here has been adopted as a standard test by the EC-Earth consortium (27 institutions in Europe) to evaluate the replicability of any new model version across platforms, including for CMIP6 experiments. To a larger extent, it can be used to assess whether other ESMs can safely be ported from one HPC environment to another for studying climate-related questions. Our results and experience with this work suggest that the default assumption should be that ESMs are not replicable under changes in the HPC environment, until proven otherwise. This research has been supported by the EU Commission – H2020 (grant nos. 727862: APPLICATE, 312979: IS-ENES2, 675191: ESiWACE, and 641727: PRIMAVERA) and the Spanish Ministerio de Economía y Competitividad (MINECO) (grant no. CGL2015-70353-R: HIATUS). Peer Reviewed Postprint (published version) Article in Journal/Newspaper Antarc* Antarctic Arctic Universitat Politècnica de Catalunya, BarcelonaTech: UPCommons - Global access to UPC knowledge Antarctic Geoscientific Model Development 13 3 1165 1178 |