[Go-essp-tech] +2Gb CMIP5 files

[stephen.pascoe at stfc.ac.uk]

The latest UKMO extraction for CMIP5 has produced some files in the 30Gb
range.  We had discussed previously the assumption that all files would
be <2Gb.  Do we feel it is important to enforce a <2Gb limit or should
this just be a recommendation on modelling centres?
 
To my knowledge there is two issues with +2Gb files:
 
 1. +2GB NetCDF files will be in 64-bit offset format. Therefore NetCDF
libraries prior to v3.6 will not be able to read them.
 2. Older file systems may have a 2Gb file limit. This will mainly
affect 32-bit systems that are a few years old. FAT32 has a 4Gb limit.

These are end-user issues, is there any reason why the ESG software
might have problems with files over 2Gb?  If we do want to ensure files
are <2Gb do we want to mandate the modelling centres deliver that or
will the data centres need to split files?

Stephen.
 
---
Stephen Pascoe  +44 (0)1235 445980
British Atmospheric Data Centre
Rutherford Appleton Laboratory
-- 
Scanned by iCritical.