[Wrf-users] Parallel NETCDF4?

[Dominikus Heinzeller]

Hi Chris,

The parallel netCDF library (pnetCDF, Northwestern University and Argonne National Laboratory) and the netCDF4 libraries(Unidata) are different pieces of software. The former one reads and writes netCDF3 data in parallel in a format that the Unidata netCDF3/4 libraries can understand (CDF2) and also in a large data format proprietary to pnetCDF (CDF5). The netCDF4 library relies on the HDF5 library for parallel reading and writing, which means that the underlying format of the data is entirely different. It is therefore impossible to use the pnetCDF library for reading and writing netCDF4 data.

If you want to stick with smaller files in netCDF4 format but are unhappy with the I/O performance, you should try the I/O quilting features in WRF. Or use the on-the-fly calculation of variables on pressure levels (pressure level diagnostics) in WRF to write out the data on fewer vertical levels of interest rather than on all model levels.

Cheers,

Dom

> On 4/03/2016, at 10:54 AM, Christopher Thomas <christopherthomas at cmt.id.au> wrote:
> 
> Hi there,
> 
> Does anyone know how to compile WRF to use parallel NETCDF but still output NETCDF4 files? 
> 
> I am using 512 cores and outputting data at high temporal and spatial resolutions; parallel NETCDF reduces my wall times by about 1/3 but more than doubles output file sizes over NETCDF4 serial output.
> 
> Chris 
> _______________________________________________
> Wrf-users mailing list
> Wrf-users at ucar.edu
> http://mailman.ucar.edu/mailman/listinfo/wrf-users