[Go-essp-tech] +2Gb CMIP5 files
stephen.pascoe at stfc.ac.uk
stephen.pascoe at stfc.ac.uk
Fri May 14 03:52:16 MDT 2010
The latest UKMO extraction for CMIP5 has produced some files in the 30Gb
range. We had discussed previously the assumption that all files would
be <2Gb. Do we feel it is important to enforce a <2Gb limit or should
this just be a recommendation on modelling centres?
To my knowledge there is two issues with +2Gb files:
1. +2GB NetCDF files will be in 64-bit offset format. Therefore NetCDF
libraries prior to v3.6 will not be able to read them.
2. Older file systems may have a 2Gb file limit. This will mainly
affect 32-bit systems that are a few years old. FAT32 has a 4Gb limit.
These are end-user issues, is there any reason why the ESG software
might have problems with files over 2Gb? If we do want to ensure files
are <2Gb do we want to mandate the modelling centres deliver that or
will the data centres need to split files?
Stephen.
---
Stephen Pascoe +44 (0)1235 445980
British Atmospheric Data Centre
Rutherford Appleton Laboratory
--
Scanned by iCritical.
More information about the GO-ESSP-TECH
mailing list