A fast input/output library for high-resolution climate models
We describe the design and implementation of climate fast input/output (CFIO), a fast input/output (I/O) library for high-resolution climate models. CFIO provides a simple method for modelers to overlap the I/O phase with the computing phase automatically, so as to shorten the running time of numeri...
Published in: | Geoscientific Model Development |
---|---|
Main Authors: | , , , , , |
Format: | Text |
Language: | English |
Published: |
2018
|
Subjects: | |
Online Access: | https://doi.org/10.5194/gmd-7-93-2014 https://gmd.copernicus.org/articles/7/93/2014/ |
id |
ftcopernicus:oai:publications.copernicus.org:gmd21903 |
---|---|
record_format |
openpolar |
spelling |
ftcopernicus:oai:publications.copernicus.org:gmd21903 2023-05-15T18:18:27+02:00 A fast input/output library for high-resolution climate models Huang, X. M. Wang, W. C. Fu, H. H. Yang, G. W. Wang, B. Zhang, C. 2018-09-27 application/pdf https://doi.org/10.5194/gmd-7-93-2014 https://gmd.copernicus.org/articles/7/93/2014/ eng eng doi:10.5194/gmd-7-93-2014 https://gmd.copernicus.org/articles/7/93/2014/ eISSN: 1991-9603 Text 2018 ftcopernicus https://doi.org/10.5194/gmd-7-93-2014 2020-07-20T16:25:12Z We describe the design and implementation of climate fast input/output (CFIO), a fast input/output (I/O) library for high-resolution climate models. CFIO provides a simple method for modelers to overlap the I/O phase with the computing phase automatically, so as to shorten the running time of numerical simulations. To minimize the code modifications required for porting, CFIO provides similar interfaces and features to parallel Network Common Data Form (PnetCDF), which is one of the most widely used I/O libraries in climate models. We deployed CFIO in three high-resolution climate models, including two ocean models (POP and LICOM) and one sea ice model (CICE). The experimental results show that CFIO improves the performance of climate models significantly versus the original serial I/O approach. When running with CFIO at 0.1° resolution with about 1000 CPU cores, we managed to reduce the running time by factors of 7.9, 4.6 and 2.0 for POP, CICE, and LICOM, respectively. We also compared the performance of CFIO against two existing libraries, PnetCDF and parallel I/O (PIO), in different scenarios. For scenarios with both data output and computations, CFIO decreases the I/O overhead compared to PnetCDF and PIO. Text Sea ice Copernicus Publications: E-Journals Geoscientific Model Development 7 1 93 103 |
institution |
Open Polar |
collection |
Copernicus Publications: E-Journals |
op_collection_id |
ftcopernicus |
language |
English |
description |
We describe the design and implementation of climate fast input/output (CFIO), a fast input/output (I/O) library for high-resolution climate models. CFIO provides a simple method for modelers to overlap the I/O phase with the computing phase automatically, so as to shorten the running time of numerical simulations. To minimize the code modifications required for porting, CFIO provides similar interfaces and features to parallel Network Common Data Form (PnetCDF), which is one of the most widely used I/O libraries in climate models. We deployed CFIO in three high-resolution climate models, including two ocean models (POP and LICOM) and one sea ice model (CICE). The experimental results show that CFIO improves the performance of climate models significantly versus the original serial I/O approach. When running with CFIO at 0.1° resolution with about 1000 CPU cores, we managed to reduce the running time by factors of 7.9, 4.6 and 2.0 for POP, CICE, and LICOM, respectively. We also compared the performance of CFIO against two existing libraries, PnetCDF and parallel I/O (PIO), in different scenarios. For scenarios with both data output and computations, CFIO decreases the I/O overhead compared to PnetCDF and PIO. |
format |
Text |
author |
Huang, X. M. Wang, W. C. Fu, H. H. Yang, G. W. Wang, B. Zhang, C. |
spellingShingle |
Huang, X. M. Wang, W. C. Fu, H. H. Yang, G. W. Wang, B. Zhang, C. A fast input/output library for high-resolution climate models |
author_facet |
Huang, X. M. Wang, W. C. Fu, H. H. Yang, G. W. Wang, B. Zhang, C. |
author_sort |
Huang, X. M. |
title |
A fast input/output library for high-resolution climate models |
title_short |
A fast input/output library for high-resolution climate models |
title_full |
A fast input/output library for high-resolution climate models |
title_fullStr |
A fast input/output library for high-resolution climate models |
title_full_unstemmed |
A fast input/output library for high-resolution climate models |
title_sort |
fast input/output library for high-resolution climate models |
publishDate |
2018 |
url |
https://doi.org/10.5194/gmd-7-93-2014 https://gmd.copernicus.org/articles/7/93/2014/ |
genre |
Sea ice |
genre_facet |
Sea ice |
op_source |
eISSN: 1991-9603 |
op_relation |
doi:10.5194/gmd-7-93-2014 https://gmd.copernicus.org/articles/7/93/2014/ |
op_doi |
https://doi.org/10.5194/gmd-7-93-2014 |
container_title |
Geoscientific Model Development |
container_volume |
7 |
container_issue |
1 |
container_start_page |
93 |
op_container_end_page |
103 |
_version_ |
1766195025707794432 |