[ncl-talk] Selecting files from systemfunc(ls...) list based on latest version

[Dennis Shea]

Hi Tess,

The attached is a 'toy' example.

%> ncl toy.ncl

INPUT:  (any order)

(0)     mslp-an-sfc-ERAI_G-v1-2018010100.nc
(1)     mslp-an-sfc-ERAI_G-v1-2018010106.nc
(2)     mslp-an-sfc-ERAI_G-v1-2018010112.nc
(3)     mslp-an-sfc-ERAI_G-v1-2018010118.nc
(4)     mslp-an-sfc-ERAI_G-v1-2018010206.nc
(5)     mslp-an-sfc-ERAI_G-v1-2018010218.nc
(6)     mslp-an-sfc-ERAI_G-v*1.1*-2018010100.nc
(7)     mslp-an-sfc-ERAI_G-v*1.1*-2018010112.nc
(8)     mslp-an-sfc-ERAI_G-v*1.1*-2018010118.nc
(9)     mslp-an-sfc-ERAI_G-v*1.2*-2018010112.nc
(10)    mslp-an-sfc-ERAI_G-v*1.3*-2018010112.nc

OUTPUT:

(0)    mslp-an-sfc-ERAI_G-*v1.1-2018010100*.nc
(1)    mslp-an-sfc-ERAI_G-*v1-2018010106*.nc
(2)    mslp-an-sfc-ERAI_G-*v1.3-2018010112*.nc
(3)    mslp-an-sfc-ERAI_G-*v1.1-2018010118*.nc
(4)    mslp-an-sfc-ERAI_G-*v1-2018010206*.nc
(5)    mslp-an-sfc-ERAI_G-*v1-2018010218*.nc


On Thu, Oct 4, 2018 at 12:02 AM Tess Parker <tess.parker at monash.edu> wrote:

> Hi,
>
> I need to access a large number of netcdf files from a central repository.
>
> There is a directory for each year, with a subdirectory for each month.
> This contains the 6-hourly files for every day of that year/month. The
> files are named according to the following protocol:
>
> var_name-input_type_1-input_type_2-data_set-version_yyyymmddhh.nc
>
> The problematic part is that, if a 6-hourly file has been regenerated,
> then the version number in the file name will change, but all previous
> versions of the file will also be retained for operational reasons. So
> every month's directory may contain several versions of the file for the
> same time step. The initial version number is "v1", with subsequent
> versions numbered "v1.1", "v1.2", etc.
>
> My question is, how can I write an NCL script that will access several
> years worth of data for each variable, but dynamically retrieve only the
> latest version of the files for each time step? (This would be slightly
> simpler, I feel, if "v1" were instead "v1.0" but this is not the case.) The
> options seem to be to find duplicate date/times in the list of file names,
> and then check for either the version number or perhaps the file
> modification date?
>
> The code needs to be flexible enough to be applied to varying lengths of
> the components of the file name protocol, such as different variable names
> etc., and also to the retrieval of just e.g. the 06 UTC time steps, etc. It
> also needs to be efficient in terms of processing time.
>
> Many thanks for your help!
> Tess
>
> --
> Dr Tess Parker
> Research Fellow
> Civil Engineering & School of Earth, Atmosphere and Environment
> Room 232, Building 28
> 9 Rainforest Walk
> Monash University, Clayton VIC 3800
>
> http://users.monash.edu.au/~tess/index.html
> _______________________________________________
> ncl-talk mailing list
> ncl-talk at ucar.edu
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20181006/6ac103d7/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: toy.ncl
Type: application/octet-stream
Size: 2270 bytes
Desc: not available
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20181006/6ac103d7/attachment.obj>