[Go-essp-tech] information in the ESG federation

[Luca Cinquini]

Hi Bryan,
	this is indeed an excellent document, thanks for putting it together  
and I apologize for not attending the telco that tuesday. I thought  
about answering some of the questions that you pose in the document  
about the Gateway:

10.ESG Gateway databases and Catalogs
(None of this was discussed on the 27th , since we didn't have a  
gateway representative
on the telco. We'll try and hijack a Curator call to discuss most of  
this material.
Action Bryan to setup.

• Configured to receive publication information from ESG data nodes  
(via TDS
catalogs) and other gateways (via OAI).

• Receives Curator OWL documents via a manual process. Can we automate  
this.
Luca would like to use OAI. Bryan would like to use atom-pub-sub.

The reason I like OAI is that it already provides an API not only to  
ingest documents, but also for figuring out if a document has been  
updated with a new version.
Does atom do that ? Additionally, the gateways already exchange  
metadata via OAI, so it would be good to not introduce yet another  
protocol. And we already have the
user interface that an administrator (Sylvia) can use to harvest the  
records starting from a given date. And yes, the process could be made  
automatic if necessary.

• How does it align model and data?
Each dataset has a property that connects it to the simulation that  
generated it (if available).

• How does it align gridspec, data and models?
A dataset can be generically associated with a grid, but I believe we  
don't actually store this information while publishing. If the  
gridspec was to be
conained in the thredds catalog, we can certainly parse that  
information out and store it.

• What is the internal data model?
Please see attached slide. In a very semplified version, it's made of  
datasets, that contain other datasets and files, and that can have  
arbotrary access points. Really, the data model is the same as thredds.

• What is the format of the material moved around by OAI between  
gateways?
It's RDF triples serialized in RDF/XML and embedded in OAI envelops.

• What does it do about service endpoints?
Service endpoints are stored in database tables, and also stored as  
RDF triples in the triple store.

Also:

Action: ESG gateway team to advise on what concept of “dataset” exists  
in the gateway.

this is probably the same as the data model question...


I also have two additional action items:
o) We need to parse the forthcoming PCMDI controlled vocabulary and  
convert it to owl instances, for use by the faceted search in ESG
o) Have we talked about how to enforce the DRS syntax ? Is this a  
responsability of the publisher ?

thanks, Luca

-------------- next part --------------
A non-text attachment was scrubbed...
Name: ESG data model.tiff
Type: image/tiff
Size: 435446 bytes
Desc: not available
Url : http://mailman.ucar.edu/pipermail/go-essp-tech/attachments/20091104/0a4c9af6/attachment-0001.tiff 
-------------- next part --------------



On Nov 2, 2009, at 6:40 AM, Bryan Lawrence wrote:

> Hi Folks
>
> Attached is a revised version of the document I sent out before the  
> last telco.
>
> It's got explanatory text added from the last telco, along with some  
> action items. (Actually a lot).
>
> I've not gathered them together yet, we plan to move the action  
> items onto a trac instance, but in the mean time, at least I've got  
> these notes out ...
>
> Cheers
> Bryan
>
> -- 
> Bryan Lawrence
> Director of Environmental Archival and Associated Research
> (NCAS/British Atmospheric Data Centre and NCEO/NERC NEODC)
> STFC, Rutherford Appleton Laboratory
> Phone +44 1235 445012; Fax ... 5848;
> Web: home.badc.rl.ac.uk/lawrence
> <ESG Information  
> V03.pdf>_______________________________________________
> GO-ESSP-TECH mailing list
> GO-ESSP-TECH at ucar.edu
> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech