[Go-essp-tech] CMIP5 parameter versions

[stephen.pascoe at stfc.ac.uk]

Hi Mark,

1) if the ESG system is designed to always supply the latest version  
(and hide away old versions)?

Yes.  The ESG publisher keeps THREDDS catalogs of old versions but the Gateway will always display the most recent version.  Previous versions aren't directly accessible through a Gateway (at least not in the versions I've evaluated, 1.2.x and 1.3.x) but the existence of previous versions are visible through a Gateway.

Of course this presumes you have separated the files of old and new versions in some way and correctly told ESG publisher which files constitute a new version.  See below.

> 2) as the file system that is scanned by the ESG system is shared by  
> our analysts, as it stands old versions can still be seen and copied  
> and potentially end up in local archives, especially when distributed  
> by 3rd parties. We would like to try and avoid this from happening  
> _without_ messing up the ESG metadata.

Are you using the DRS directory structure which includes a version directory "vYYYYMMDD"?  This is how we are separating the current version from previous versions.  The tool drslib[1] is designed to manage duplicate files across versions using symbolic links.

[1] http://esgf.org/esgf-drslib-site 

Cheers,
Stephen.

---
Stephen Pascoe  +44 (0)1235 445980
Centre of Environmental Data Archival
STFC Rutherford Appleton Laboratory, Harwell Oxford, Didcot OX11 0QX, UK

-----Original Message-----
From: go-essp-tech-bounces at ucar.edu [mailto:go-essp-tech-bounces at ucar.edu] On Behalf Of mark collier
Sent: 06 September 2011 02:30
To: go-essp-tech at ucar.edu
Cc: Jeffrey Stephen; Aspendale) Rotstayn Leon (CMAR; Ben Evans
Subject: [Go-essp-tech] CMIP5 parameter versions

Hi,
just a general question about versions of parameters.

As we identify problems our list of new versions is growing, however,  
still manageable.

What concerns us more is the possibility of outdated versions which  
may still be accessed. In terms of reconciling file (differences) it  
can be good to have all versions available, however, nightmarish if  
the wrong files get into analysts repositories and accidently (or  
unknowingly used if they aren't aware of them being outdated) used.

I would like to know:

1) if the ESG system is designed to always supply the latest version  
(and hide away old versions)?

2) as the filesystem that is scanned by the ESG system is shared by  
our analysts, as it stands old versions can still be seen and copied  
and potentially end up in local archives, especially when distributed  
by 3rd parties. We would like to try and avoid this from happening  
_without_ messing up the ESG metadata.

One solution for 2) is to ("in-situ") change the variable name from  
say pr to pr_error in the wrong/outdated file versions - the DRS  
structure (including filename) will stay the same but anyone trying to  
read the file will instantly be alerted to the problem.

This is probably our biggest concern at the moment in terms of making  
data available to the CMIP5 community.

Regards,
Mark Collier.
_______________________________________________
GO-ESSP-TECH mailing list
GO-ESSP-TECH at ucar.edu
http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
-- 
Scanned by iCritical.