[Go-essp-tech] CMIP5 Version directory structure update

[martin.juckes at stfc.ac.uk]

I think there are a couple of issues that need to be resolved here, but
which can be addressed after Stephen has implemented his versioning
system:

(1) How does a modelling system provide annotation of a new data
version? E.g. if they find that the institute name has been misspelt in
one of the attributes of the netcdf files, there should be a brief
comment lodged somewhere. I suppose the logical way to do this is for a
comment placed in the ESG data node in such a way that it can
subsequently be harvested into the METAFOR CIM by the gateway.

(2) When a DOI is issued, how is this flagged at the data node and, more
importantly, what steps are taken to guarantee curation of the flagged
data version? We might, for instance, want to create a clean copy of
this version in a new subdirectory starting at the same level as
Stephen's "files" subdirectory, and place links in files subdirectory to
avoid having multiple copies of files. But there might be better ideas,
and we don't need to finalise this aspect now.

In the meantime, I think Stephen's versioning system deals cleanly with
all the issues that have been raised about supporting versioning in the
directory structure,

Cheers,
Martin

> -----Original Message-----
> From: go-essp-tech-bounces at ucar.edu [mailto:go-essp-tech-
> bounces at ucar.edu] On Behalf Of Martina Stockhause
> Sent: 07 June 2010 14:07
> To: Pascoe, Stephen (STFC,RAL,SSTD)
> Cc: go-essp-tech at ucar.edu
> Subject: Re: [Go-essp-tech] CMIP5 Version directory structure update
> 
> Hi Stephen,
> 
> It's good that you push the version control ahead.
> 
> I'd like to bring the version control on the DOI level into the
> discussion. We plan to assign DOIs on experiments and on models or
> institutes. I think we should use a similar mechanism for that as for
> the version management for the portal (on the model_realm level). I
see
> only one difference: We do not publish data but only add to the
> metadata, which makes it simpler.
> 
> Would a DOI on an experiment be another version tree on the experiment
> level with links to the associated model_realm versions?
> 
> How do we publish an assigned DOI among ESGF? Via the ESG publisher in
> a
> metadata publishing mode or via the CIM repository?
> For the first we could use a lot of the version control mechanism of
> the
> model_realms, maybe even the tool you develop. And for the latter we
> would need another version management, e.g. in CERA, to store the
links
> to the associated model_realm or file versions and to resolve them.
> User
> questions might be: Which DOI do I cite for the downloaded data I used
> for my analyses? Which data is associated with the DOI cited in a
> publication?
> 
> So, is it preferable to follow a similar solution for DOI versions as
> for the model_realm version management or are the differences so large
> that we need a different solution?
> 
> Best wishes,
> Martina
> 
> 
> stephen.pascoe at stfc.ac.uk wrote:
> > Hi All,
> >
> > Attached is a new version of the version directory structure
document
> > that reflects our decision to publish at the realm level and
> therefore
> > manage versions of realm-datasets rather than atomic datasets.  Of
> > particular interest should be Section 2 which suggests how a version
> > management tool would transform the CMOR directory structure into a
> > version-controlled structure.  Also note I'm suggesting the DRS
> version
> > directory is moved to just below realm.
> >
> > Comments are still welcome but I am now going to push ahead at
> turning
> > this plan into code.  I intend a tool that can be run each time more
> > data arrives at the datanode, before esgpublish is executed.  This
> could
> > be merged into esgpublish if we wish.  I hope to have something
> testable
> > by the end of the week.
> >
> > Cheers,
> > Stephen.
> >
> > ---
> > Stephen Pascoe  +44 (0)1235 445980
> > British Atmospheric Data Centre
> > Rutherford Appleton Laboratory
> >
> >
> >
> >
---------------------------------------------------------------------
> ---
> >
> > _______________________________________________
> > GO-ESSP-TECH mailing list
> > GO-ESSP-TECH at ucar.edu
> > http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
> 
> _______________________________________________
> GO-ESSP-TECH mailing list
> GO-ESSP-TECH at ucar.edu
> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
-- 
Scanned by iCritical.