[Go-essp-tech] Updating the DRS document

[martin.juckes at stfc.ac.uk]

Hello Karl,

 

I think the following changes are what is needed - I'm happy to put them
into the document and circulate the result for approval if you send me
the latest version in word. I don't think this affects what modelling
groups are doing, since, if they archive in the DRS, everything they
archive will be in the 'full' section of the directory tree and it will
be up to the data node managers to create the 'requested' section.

 

In section 2.1: Modify the atomic dataset definition: 

The collection of data that is output from a single model run and 

characterized by sharing a single activity, activity component,
institute, model, 

experiment/scenario, data frequency, modeling-realm, variable name, 

local ensemble member, and version. 

 

I'm not sure about the terminology "activity component", but it is
reasonably descriptive.

 

In the following paragraph, change `first six' to `first seven' and add
'activity component' to the list. 

 

In section 2.2: Insert an `activity component' definition after the
'activity' definition: 

 

Activity component 

 

The DRS will distinguish between 'full' and 'requested' atomic datasets.
The 'full' component  

will contain all the data archived, while the 'requested' component will
contain only  

those sections of the data in the PCMDI CMIP5 data request. The atomic
datasets within  

the 'requested' component will thus either be subsets of those 'full'
component or identical  

to them when only the requested data is archived. 

 

In section 3.1: insert `/<category>' after '/<activity>/', and similar
changes to 

other URLs in this section. 

 

In section 3.2: as above.

 

Add a new section 3.4:

 

3.4 Replication

 

A subset of the data will be replicated at PCMDI, BADC and DKRZ. This
subset will consist of a selection of complete atomic datasets from the
"requested" activity component.

 

 

Cheers,

Martin

 

> -----Original Message-----

> From: Pascoe, Stephen (STFC,RAL,SSTD)

> Sent: 07 January 2010 12:05

> To: Juckes, Martin (STFC,RAL,SSTD); Karl Taylor

> Cc: go-essp-tech at ucar.edu

> Subject: Updating the DRS document

> 

> Hi Martin and Karl,

> 

> I am keen that the recent discussions on DRS aren't forgotten in the

> rush to implement.  Therefore I would like to see the DRS document

> updated with the following:

> 

>  1. Product/Category Component.

> 

> As agreed in the versioning telco

> (http://proj.badc.rl.ac.uk/go-essp/wiki/CMIP5/Meetings/telco091208) we

> need an extra component in the DRS to distinguish between requested

> and unrequested atomic datasets.  I think the proposal was for a

> category between Activity and Institute with a the value "full" or

> "requested".  I like the term "Product" for this component but I think

> various names were discussed.

> 

>  2. Hostname Component in URLs.

> 

> I believe this needs to be replaced by a host prefix, thus allowing

> some site-specific path elements to preceed the activity component.

> Knowing how the datanode software is designed I think it is too

> restrictive to insist the first element after the hostname is Activity

> because in reality multiple services will be running on a datanode.

> One could keep the current scheme with virtual hosts and redirects but

> it seems unnecessary.

> 

>  3. Section 3.3 Filenames.

> 

> There is no version number in the filename convention: Shouldn't there

> be?  A recent comment from Sylvia suggested the ESG metadata display

> will depend on this naming scheme so I would like to see more detail

> here.  Is the temporal_subset element manditory for all files in the

> system?

> 

> 

> I am happy to help with preparing a new draft if you wish.

> 

> Thanks,

> Stephen.

 


-- 
Scanned by iCritical.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/go-essp-tech/attachments/20100112/cc5ae514/attachment.html