Portal de Periódicos da CAPES

Improving the I/O of large geophysical models using PnetCDF and BeeGFS

2021; Elsevier BV; Volume: 104-105; Linguagem: Inglês

10.1016/j.parco.2021.102786

1872-7336

Jared Brzenski, Christopher Paolini, José E. Castillo,

Parallel Computing and Optimization Techniques

Large scale geophysical modeling uses high performance computing systems to expedite the solutions of very large, complex systems. High disk latencies, low IOPS, and low read/write data transfer rates are relegating many numerical simulations to I/O bound jobs, where the run time is bound not by CPU rate, but by I/O rate. In this paper we seek to improve the I/O of two geophysical modeling applications and take full advantage of the parallel nature of the programs, as well as the file management system for the large output files. Parallelizing output for these programs is achieved using PnetCDF, a parallel implementation of the netCDF format, and BeeGFS, an open-source parallel file system. Using these solutions, we have significantly decreased the amount of time spent saving data to disk, and give analysis of the features used in relation to PnetCDF with BeeGFS I/O optimization.