[Go-essp-tech] DRS syntax into ESG

[Luca Cinquini]

Hi,
	the purpose of this email is to start a conversation, and a plan of  
action, on how to incorporate the DRS syntax into the ESG system.
As a reminder, the current DRS specification states that a CMIP5  
dataset will be uniquely identified by the following URL:

http://<hostname>/<activity>/<institute>/<model>/<experiment>/ 
<frequency>/<modeling realm>/<variable>/<ensemble member>/<version>/  
[<endpoint>]

where most of the <...> fields are controlled vocabularies
for example:

http://badc.nerc.ac.uk/activity/institute/model/experiment/frequency/realm/varname/r 
  1/v1/

The first question would be what does it mean to capture the semantics  
of the DRS syntax within ESG ? I can see at least two answers:

a) The user is able to browse the CMIP5 datasets hierarchically  
according to the DRS hierarchy of fields
b) The user is able to search for data based on facets that reflect  
the DRS syntax: activity, institute, experiment, etc..

So how do we get there ? A straw-man workflow could be the following:

o) The ESG Data Node publishing client, when building the THREDDS  
catalogs, creates a hierarchy of datasets that reflects the syntax.  
There is probably also a need to mark up these catalogs as "DRS" or  
"CMIP5".
o) The ESG Gateway, when parsing these catalogs, invokes a specific  
handler that creates the same datasets hierarchy (this is actually  
automatic, I believe), and additionally associates corresponding  
objects at each level of the hierarchy. For example, at first level  
the dataset will be associated with an activity, at second level with  
an institute, and so on. An alternative way would be to associate all  
the objects only to the leaf level dataset.
o) When the metadata for the leaf nodes datasets is harvested into RDF  
triples for searching, the dataset - object associations must be  
transfered to the triple store
o) Specific CMIP5 facets can be configured to search by DRS fields  
(perhaps only on the PCMDI Gateway, or perhaps on all gateways).

As mentioned, this is just a start. I do believe though that this is  
an extremely important issue that must be tackled as soon as possible.

thanks, Luca