[ncl-talk] Selecting files from systemfunc(ls...) list based on latest version

[Tess Parker]

Thanks Dennis. To be clear, an example filename would be:

mslp-an-sfc-ERAI_G-v1-2018010100.nc

and the updated version would be:

mslp-an-sfc-ERAI_G-v1.1-2018010100.nc

-- 
Dr Tess Parker
Research Fellow
Civil Engineering & School of Earth, Atmosphere and Environment
Room 232, Building 28
9 Rainforest Walk
Monash University, Clayton VIC 3800

http://users.monash.edu.au/~tess/index.html


On Thu, 4 Oct 2018 at 16:01, Tess Parker <tess.parker at monash.edu> wrote:

> Hi,
>
> I need to access a large number of netcdf files from a central repository.
>
> There is a directory for each year, with a subdirectory for each month.
> This contains the 6-hourly files for every day of that year/month. The
> files are named according to the following protocol:
>
> var_name-input_type_1-input_type_2-data_set-version_yyyymmddhh.nc
>
> The problematic part is that, if a 6-hourly file has been regenerated,
> then the version number in the file name will change, but all previous
> versions of the file will also be retained for operational reasons. So
> every month's directory may contain several versions of the file for the
> same time step. The initial version number is "v1", with subsequent
> versions numbered "v1.1", "v1.2", etc.
>
> My question is, how can I write an NCL script that will access several
> years worth of data for each variable, but dynamically retrieve only the
> latest version of the files for each time step? (This would be slightly
> simpler, I feel, if "v1" were instead "v1.0" but this is not the case.) The
> options seem to be to find duplicate date/times in the list of file names,
> and then check for either the version number or perhaps the file
> modification date?
>
> The code needs to be flexible enough to be applied to varying lengths of
> the components of the file name protocol, such as different variable names
> etc., and also to the retrieval of just e.g. the 06 UTC time steps, etc. It
> also needs to be efficient in terms of processing time.
>
> Many thanks for your help!
> Tess
>
> --
> Dr Tess Parker
> Research Fellow
> Civil Engineering & School of Earth, Atmosphere and Environment
> Room 232, Building 28
> 9 Rainforest Walk
> Monash University, Clayton VIC 3800
>
> http://users.monash.edu.au/~tess/index.html
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20181005/14dbcc1e/attachment.html>