[ncl-talk] Selecting files from systemfunc(ls...) list based on latest version

[Tess Parker]

Thanks Dennis, that works perfectly! Quick and easy to use. Also easily
adapted to changing directory and/or file structures. Appreciate your quick
response.

Tess



On Sun, 7 Oct 2018 at 00:33, Dennis Shea <shea at ucar.edu> wrote:

> Hi Tess,
>
> The attached is a 'toy' example.
>
> %> ncl toy.ncl
>
> INPUT:  (any order)
>
> (0)     mslp-an-sfc-ERAI_G-v1-2018010100.nc
> (1)     mslp-an-sfc-ERAI_G-v1-2018010106.nc
> (2)     mslp-an-sfc-ERAI_G-v1-2018010112.nc
> (3)     mslp-an-sfc-ERAI_G-v1-2018010118.nc
> (4)     mslp-an-sfc-ERAI_G-v1-2018010206.nc
> (5)     mslp-an-sfc-ERAI_G-v1-2018010218.nc
> (6)     mslp-an-sfc-ERAI_G-v*1.1*-2018010100.nc
> (7)     mslp-an-sfc-ERAI_G-v*1.1*-2018010112.nc
> (8)     mslp-an-sfc-ERAI_G-v*1.1*-2018010118.nc
> (9)     mslp-an-sfc-ERAI_G-v*1.2*-2018010112.nc
> (10)    mslp-an-sfc-ERAI_G-v*1.3*-2018010112.nc
>
> OUTPUT:
>
> (0)    mslp-an-sfc-ERAI_G-*v1.1-2018010100*.nc
> (1)    mslp-an-sfc-ERAI_G-*v1-2018010106*.nc
> (2)    mslp-an-sfc-ERAI_G-*v1.3-2018010112*.nc
> (3)    mslp-an-sfc-ERAI_G-*v1.1-2018010118*.nc
> (4)    mslp-an-sfc-ERAI_G-*v1-2018010206*.nc
> (5)    mslp-an-sfc-ERAI_G-*v1-2018010218*.nc
>
>
> On Thu, Oct 4, 2018 at 12:02 AM Tess Parker <tess.parker at monash.edu>
> wrote:
>
>> Hi,
>>
>> I need to access a large number of netcdf files from a central
>> repository.
>>
>> There is a directory for each year, with a subdirectory for each month.
>> This contains the 6-hourly files for every day of that year/month. The
>> files are named according to the following protocol:
>>
>> var_name-input_type_1-input_type_2-data_set-version_yyyymmddhh.nc
>>
>> The problematic part is that, if a 6-hourly file has been regenerated,
>> then the version number in the file name will change, but all previous
>> versions of the file will also be retained for operational reasons. So
>> every month's directory may contain several versions of the file for the
>> same time step. The initial version number is "v1", with subsequent
>> versions numbered "v1.1", "v1.2", etc.
>>
>> My question is, how can I write an NCL script that will access several
>> years worth of data for each variable, but dynamically retrieve only the
>> latest version of the files for each time step? (This would be slightly
>> simpler, I feel, if "v1" were instead "v1.0" but this is not the case.) The
>> options seem to be to find duplicate date/times in the list of file names,
>> and then check for either the version number or perhaps the file
>> modification date?
>>
>> The code needs to be flexible enough to be applied to varying lengths of
>> the components of the file name protocol, such as different variable names
>> etc., and also to the retrieval of just e.g. the 06 UTC time steps, etc. It
>> also needs to be efficient in terms of processing time.
>>
>> Many thanks for your help!
>> Tess
>>
>> --
>> Dr Tess Parker
>> Research Fellow
>> Civil Engineering & School of Earth, Atmosphere and Environment
>> Room 232, Building 28
>> 9 Rainforest Walk
>> Monash University, Clayton VIC 3800
>>
>> http://users.monash.edu.au/~tess/index.html
>> _______________________________________________
>> ncl-talk mailing list
>> ncl-talk at ucar.edu
>> List instructions, subscriber options, unsubscribe:
>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20181008/1e59a0d2/attachment.html>