[Go-essp-tech] Updating the DRS document with attachments

Tue Jan 19 09:45:42 MST 2010

Hi Phillip,

As far as I know it does comply. Unless something changed in the last  
couple of weeks?

C.

On Jan 18, 2010, at 4:21 AM, Bentley, Philip wrote:

> Hi Karl,
>
> Thanks for sending out these revised DRS documents. Are you of the
> opinion that this specification can now be branded as v1.0 and as such
> will not be subject to further significant updates for the immediate
> future?
>
> Like all the other CMIP5 participants, we are in the process of
> developing software that conforms to the CMIP5 DRS. Consequently it
> would be extremely helpful to know that we have a reasonably static
> target to aim at over the coming 3 months or so. If further changes to
> the DRS are anticipated then we (here at UKMO) may wish to hold off on
> our development of DRS-dependent software.
>
> Also - and a question for Charles I suspect - does the latest CMOR
> software release comply with this latest DRS specification?
>
> Regards,
> Phil
>
>> -----Original Message-----
>> From: go-essp-tech-bounces at ucar.edu
>> [mailto:go-essp-tech-bounces at ucar.edu] On Behalf Of Karl Taylor
>> Sent: 13 January 2010 00:11
>> To: martin.juckes at stfc.ac.uk
>> Cc: go-essp-tech at ucar.edu
>> Subject: [Go-essp-tech] Updating the DRS document *with attachments*
>>
>> Karl Taylor wrote:
>>> Dear Martin and Stephen,
>>>
>>> thanks very much for thinking about this a providing some specific
>>> suggestions. This has prompted me to get back to this. I
>> thought I had
>>> already placed on the web version 22 of the document
>> (attached), which
>>> defined a "Product" component for DRS. After reading your input, I
>>> have altered this slightly, and also completed the
>> definition of the
>>> experiment names (see the last appendix) in version 23
>> (also attached).
>>>
>>> Please let me know if this meets all the requirements.
>>>
>>> Best regards,
>>>
>>> martin.juckes at stfc.ac.uk wrote:
>>>>
>>>> Hello Karl,
>>>>
>>>> I think the following changes are what is needed - I'm
>> happy to put
>>>> them into the document and circulate the result for
>> approval if you
>>>> send me the latest version in word. I don't think this
>> affects what
>>>> modelling groups are doing, since, if they archive in the DRS,
>>>> everything they archive will be in the 'full' section of the
>>>> directory tree and it will be up to the data node managers
>> to create
>>>> the 'requested' section.
>>>>
>>>> In section 2.1: Modify the atomic dataset definition:
>>>>
>>>> /The collection of data that is output from a single model
>> run and /
>>>>
>>>> /characterized by sharing a single activity, _activity component_,
>>>> institute, model, /
>>>>
>>>> /experiment/scenario, data frequency, modeling-realm,
>> variable name,
>>>> /
>>>>
>>>> /local ensemble member, and version. /
>>>>
>>>> I'm not sure about the terminology "activity component", but it is
>>>> reasonably descriptive.
>>>>
>>>> In the following paragraph, change `first six' to `first
>> seven' and
>>>> add 'activity component' to the list.
>>>>
>>>> In section 2.2: Insert an `activity component' definition
>> after the
>>>> 'activity' definition:
>>>>
>>>> *Activity component *
>>>>
>>>> The DRS will distinguish between 'full' and 'requested' atomic
>>>> datasets. The 'full' component
>>>>
>>>> will contain all the data archived, while the 'requested'
>> component
>>>> will contain only
>>>>
>>>> those sections of the data in the PCMDI CMIP5 data request. The
>>>> atomic datasets within
>>>>
>>>> the 'requested' component will thus either be subsets of
>> those 'full'
>>>> component or identical
>>>>
>>>> to them when only the requested data is archived.
>>>>
>>>> In section 3.1: insert `/<category>' after '/<activity>/', and
>>>> similar changes to
>>>>
>>>> other URLs in this section.
>>>>
>>>> In section 3.2: as above.
>>>>
>>>> Add a new section 3.4:
>>>>
>>>> 3.4 Replication
>>>>
>>>> A subset of the data will be replicated at PCMDI, BADC and
>> DKRZ. This
>>>> subset will consist of a selection of complete atomic
>> datasets from
>>>> the "requested" activity component.
>>>>
>>>> Cheers,
>>>>
>>>> Martin
>>>>
>>>>> -----Original Message-----
>>>>
>>>>> From: Pascoe, Stephen (STFC,RAL,SSTD)
>>>>
>>>>> Sent: 07 January 2010 12:05
>>>>
>>>>> To: Juckes, Martin (STFC,RAL,SSTD); Karl Taylor
>>>>
>>>>> Cc: go-essp-tech at ucar.edu
>>>>
>>>>> Subject: Updating the DRS document
>>>>
>>>>>
>>>>
>>>>> Hi Martin and Karl,
>>>>
>>>>>
>>>>
>>>>> I am keen that the recent discussions on DRS aren't forgotten in
>>>>> the
>>>>
>>>>> rush to implement. Therefore I would like to see the DRS document
>>>>
>>>>> updated with the following:
>>>>
>>>>>
>>>>
>>>>> 1. Product/Category Component.
>>>>
>>>>>
>>>>
>>>>> As agreed in the versioning telco
>>>>
>>>>>
>>>>
>> (http://**proj.badc.rl.ac.uk/go-essp/wiki/CMIP5/Meetings/telco091208)
>>>> we
>>>>
>>>>> need an extra component in the DRS to distinguish
>> between requested
>>>>
>>>>> and unrequested atomic datasets. I think the proposal was for a
>>>>
>>>>> category between Activity and Institute with a the value
>> "full" or
>>>>
>>>>> "requested". I like the term "Product" for this component but I
>>>>> think
>>>>
>>>>> various names were discussed.
>>>>
>>>>>
>>>>
>>>>> 2. Hostname Component in URLs.
>>>>
>>>>>
>>>>
>>>>> I believe this needs to be replaced by a host prefix,
>> thus allowing
>>>>
>>>>> some site-specific path elements to preceed the activity
>> component.
>>>>
>>>>> Knowing how the datanode software is designed I think it is too
>>>>
>>>>> restrictive to insist the first element after the hostname is
>>>>> Activity
>>>>
>>>>> because in reality multiple services will be running on
>> a datanode.
>>>>
>>>>> One could keep the current scheme with virtual hosts and
>> redirects
>>>>> but
>>>>
>>>>> it seems unnecessary.
>>>>
>>>>>
>>>>
>>>>> 3. Section 3.3 Filenames.
>>>>
>>>>>
>>>>
>>>>> There is no version number in the filename convention: Shouldn't
>>>>> there
>>>>
>>>>> be? A recent comment from Sylvia suggested the ESG
>> metadata display
>>>>
>>>>> will depend on this naming scheme so I would like to see more
>>>>> detail
>>>>
>>>>> here. Is the temporal_subset element manditory for all
>> files in the
>>>>
>>>>> system?
>>>>
>>>>>
>>>>
>>>>>
>>>>
>>>>> I am happy to help with preparing a new draft if you wish.
>>>>
>>>>>
>>>>
>>>>> Thanks,
>>>>
>>>>> Stephen.
>>>>
>>>>
>>>> --
>>>> Scanned by iCritical.
>>>>
>>>>
>>>
>>>
>>
>>
>