[Go-essp-tech] CMIP5 parameter versions
Ashish Bhardwaj
abhipsl at ipsl.jussieu.fr
Tue Sep 6 03:52:45 MDT 2011
Hi Stephen,
stephen.pascoe at stfc.ac.uk wrote:
> Hi Mark,
>
> 1) if the ESG system is designed to always supply the latest version
> (and hide away old versions)?
>
> Yes. The ESG publisher keeps THREDDS catalogs of old versions but the Gateway will always display the most recent version. Previous versions aren't directly accessible through a Gateway (at least not in the versions I've evaluated, 1.2.x and 1.3.x) but the existence of previous versions are visible through a Gateway.
>
> Of course this presumes you have separated the files of old and new versions in some way and correctly told ESG publisher which files constitute a new version. See below.
>
>
If I added a new version for a dataset containing new variables, is it
possible both versions to be accessible through gateway ?
Thanks.
Ashish
>> 2) as the file system that is scanned by the ESG system is shared by
>> our analysts, as it stands old versions can still be seen and copied
>> and potentially end up in local archives, especially when distributed
>> by 3rd parties. We would like to try and avoid this from happening
>> _without_ messing up the ESG metadata.
>>
>
> Are you using the DRS directory structure which includes a version directory "vYYYYMMDD"? This is how we are separating the current version from previous versions. The tool drslib[1] is designed to manage duplicate files across versions using symbolic links.
>
> [1] http://esgf.org/esgf-drslib-site
>
> Cheers,
> Stephen.
>
> ---
> Stephen Pascoe +44 (0)1235 445980
> Centre of Environmental Data Archival
> STFC Rutherford Appleton Laboratory, Harwell Oxford, Didcot OX11 0QX, UK
>
> -----Original Message-----
> From: go-essp-tech-bounces at ucar.edu [mailto:go-essp-tech-bounces at ucar.edu] On Behalf Of mark collier
> Sent: 06 September 2011 02:30
> To: go-essp-tech at ucar.edu
> Cc: Jeffrey Stephen; Aspendale) Rotstayn Leon (CMAR; Ben Evans
> Subject: [Go-essp-tech] CMIP5 parameter versions
>
> Hi,
> just a general question about versions of parameters.
>
> As we identify problems our list of new versions is growing, however,
> still manageable.
>
> What concerns us more is the possibility of outdated versions which
> may still be accessed. In terms of reconciling file (differences) it
> can be good to have all versions available, however, nightmarish if
> the wrong files get into analysts repositories and accidently (or
> unknowingly used if they aren't aware of them being outdated) used.
>
> I would like to know:
>
> 1) if the ESG system is designed to always supply the latest version
> (and hide away old versions)?
>
> 2) as the filesystem that is scanned by the ESG system is shared by
> our analysts, as it stands old versions can still be seen and copied
> and potentially end up in local archives, especially when distributed
> by 3rd parties. We would like to try and avoid this from happening
> _without_ messing up the ESG metadata.
>
> One solution for 2) is to ("in-situ") change the variable name from
> say pr to pr_error in the wrong/outdated file versions - the DRS
> structure (including filename) will stay the same but anyone trying to
> read the file will instantly be alerted to the problem.
>
> This is probably our biggest concern at the moment in terms of making
> data available to the CMIP5 community.
>
> Regards,
> Mark Collier.
> _______________________________________________
> GO-ESSP-TECH mailing list
> GO-ESSP-TECH at ucar.edu
> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
>
--
Ashish Bhardwaj
Software Engineer
Institut Pierre Simon Laplace
UPMC, Case 101, 4 place Jussieu,
75252 Paris Cedex 5
Tour 45-55 2ème étage Bureau 207
Tel: 33 1 44 27 49 10
More information about the GO-ESSP-TECH
mailing list