[ncl-talk] Selecting files from systemfunc(ls...) list based on latest version

[Tess Parker]

Hi,

I need to access a large number of netcdf files from a central repository.

There is a directory for each year, with a subdirectory for each month.
This contains the 6-hourly files for every day of that year/month. The
files are named according to the following protocol:

var_name-input_type_1-input_type_2-data_set-version_yyyymmddhh.nc

The problematic part is that, if a 6-hourly file has been regenerated, then
the version number in the file name will change, but all previous versions
of the file will also be retained for operational reasons. So every month's
directory may contain several versions of the file for the same time step.
The initial version number is "v1", with subsequent versions numbered
"v1.1", "v1.2", etc.

My question is, how can I write an NCL script that will access several
years worth of data for each variable, but dynamically retrieve only the
latest version of the files for each time step? (This would be slightly
simpler, I feel, if "v1" were instead "v1.0" but this is not the case.) The
options seem to be to find duplicate date/times in the list of file names,
and then check for either the version number or perhaps the file
modification date?

The code needs to be flexible enough to be applied to varying lengths of
the components of the file name protocol, such as different variable names
etc., and also to the retrieval of just e.g. the 06 UTC time steps, etc. It
also needs to be efficient in terms of processing time.

Many thanks for your help!
Tess

-- 
Dr Tess Parker
Research Fellow
Civil Engineering & School of Earth, Atmosphere and Environment
Room 232, Building 28
9 Rainforest Walk
Monash University, Clayton VIC 3800

http://users.monash.edu.au/~tess/index.html
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20181004/065de6de/attachment.html>