[Go-essp-tech] CMIP5 parameter versions

[Ashish Bhardwaj]

Hi Stephen,


stephen.pascoe at stfc.ac.uk wrote:
> Hi Mark,
>
> 1) if the ESG system is designed to always supply the latest version  
> (and hide away old versions)?
>
> Yes.  The ESG publisher keeps THREDDS catalogs of old versions but the Gateway will always display the most recent version.  Previous versions aren't directly accessible through a Gateway (at least not in the versions I've evaluated, 1.2.x and 1.3.x) but the existence of previous versions are visible through a Gateway.
>
> Of course this presumes you have separated the files of old and new versions in some way and correctly told ESG publisher which files constitute a new version.  See below.
>
>   

If I added a new version for a dataset containing new variables, is it 
possible both versions to be accessible through gateway ?

Thanks.
Ashish



>> 2) as the file system that is scanned by the ESG system is shared by  
>> our analysts, as it stands old versions can still be seen and copied  
>> and potentially end up in local archives, especially when distributed  
>> by 3rd parties. We would like to try and avoid this from happening  
>> _without_ messing up the ESG metadata.
>>     
>
> Are you using the DRS directory structure which includes a version directory "vYYYYMMDD"?  This is how we are separating the current version from previous versions.  The tool drslib[1] is designed to manage duplicate files across versions using symbolic links.
>
> [1] http://esgf.org/esgf-drslib-site 
>
> Cheers,
> Stephen.
>
> ---
> Stephen Pascoe  +44 (0)1235 445980
> Centre of Environmental Data Archival
> STFC Rutherford Appleton Laboratory, Harwell Oxford, Didcot OX11 0QX, UK
>
> -----Original Message-----
> From: go-essp-tech-bounces at ucar.edu [mailto:go-essp-tech-bounces at ucar.edu] On Behalf Of mark collier
> Sent: 06 September 2011 02:30
> To: go-essp-tech at ucar.edu
> Cc: Jeffrey Stephen; Aspendale) Rotstayn Leon (CMAR; Ben Evans
> Subject: [Go-essp-tech] CMIP5 parameter versions
>
> Hi,
> just a general question about versions of parameters.
>
> As we identify problems our list of new versions is growing, however,  
> still manageable.
>
> What concerns us more is the possibility of outdated versions which  
> may still be accessed. In terms of reconciling file (differences) it  
> can be good to have all versions available, however, nightmarish if  
> the wrong files get into analysts repositories and accidently (or  
> unknowingly used if they aren't aware of them being outdated) used.
>
> I would like to know:
>
> 1) if the ESG system is designed to always supply the latest version  
> (and hide away old versions)?
>
> 2) as the filesystem that is scanned by the ESG system is shared by  
> our analysts, as it stands old versions can still be seen and copied  
> and potentially end up in local archives, especially when distributed  
> by 3rd parties. We would like to try and avoid this from happening  
> _without_ messing up the ESG metadata.
>
> One solution for 2) is to ("in-situ") change the variable name from  
> say pr to pr_error in the wrong/outdated file versions - the DRS  
> structure (including filename) will stay the same but anyone trying to  
> read the file will instantly be alerted to the problem.
>
> This is probably our biggest concern at the moment in terms of making  
> data available to the CMIP5 community.
>
> Regards,
> Mark Collier.
> _______________________________________________
> GO-ESSP-TECH mailing list
> GO-ESSP-TECH at ucar.edu
> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
>   


-- 
Ashish Bhardwaj

Software Engineer
Institut Pierre Simon Laplace
UPMC, Case 101, 4 place Jussieu,
75252 Paris Cedex 5

Tour 45-55 2ème étage Bureau 207
Tel: 33 1 44 27 49 10