[Go-essp-tech] [ESGF-Search] Opensearch and XML representation of a search query

Wed Jun 8 07:47:22 MDT 2011

On Jun 8, 2011, at 6:53 AM, <stephen.pascoe at stfc.ac.uk> <stephen.pascoe at stfc.ac.uk> wrote:

> Is there more detail on what it means to "federate search with ESIP"?  

Yes, I can answer some of those questions, as one of the coordinators of the ESIP Discovery Cluster.

> Is there infrastructure behind the pattern described on the ESIP wiki?

In terms of a single coherent infrastructure, the answer is no.  The ESIP federated search convention is limited to the convention.  Particpants follow the convention through their own implementations, though they often add some application-specific extensions.  The cluster is fine with that, though we encourage the extensions to be made optional so that, say, clients that do not understand them can still function with the server, albeit at a basic-service level.
> 
> I'm asking because I'm slightly cautions about tying ourselves to an infrastructure that may not meet our needs.  For instance the wiki page suggests ESIP OpenSearch engines will support a single query string plus optional time constraints.  If this is a hard requirement it means we can't use arbitrary OpenSearch extensions.  The time extension that is supported may not meet our needs: I bet it doesn't do 360 day calendars!  Multiple time dimensions anyone?
> 

Quite right that it doesn't do 360 day calendars, or multiple time dimensions. (Yet.) From the Discovery Cluster point of view, our preferences for "federating search with ESG" would be for ESG to participate in our convention-making process.  We would like to get recommendations for enhancements of the convention from the modeling community, especially because some of the data providers in the Discovery Cluster (such as my own organization) deal with both observational data and model output.

> I've been here before in the context of OGC standards.  We end up swimming against the tide in trying to fit our requirements into what the rest of the community want.  Of course there are many great advantages to sharing standards like this.  I just advocate caution.
> 

I can appreciate the need for caution in this circumstance.  And I also appreciate that you have a schedule to meet that does not allow for "clearing" your conventions with the rest of the ESIP Discovery Cluster, or possibly even going with OpenSearch at all.  I would say go ahead and implement what you need to, but with eyes wide open so that you don't needlessly preclude a more full-featured "federation with ESIP".

(Having said all that, though, I will mention that the reason we went with OpenSearch rather than, say, OGC CSW, is the simplicity and rapidity with which both clients and servers can be implemented with the OpenSearch mechanism.  So I think any project that needs a distributed search up and running in a hurry should take a hard look at it.)

> Thanks,
> Stephen.
> 
> ---
> Stephen Pascoe  +44 (0)1235 445980
> Centre of Environmental Data Archival
> STFC Rutherford Appleton Laboratory, Harwell Oxford, Didcot OX11 0QX, UK
> 
> 
> -----Original Message-----
> From: Cinquini, Luca (3880) [mailto:Luca.Cinquini at jpl.nasa.gov] 
> Sent: 08 June 2011 11:37
> To: Mattmann, Chris A (388J)
> Cc: Pascoe, Stephen (STFC,RAL,RALSP); go-essp-tech at ucar.edu
> Subject: Re: [Go-essp-tech] [ESGF-Search] Opensearch and XML representation of a search query
> 
> Thanks Chris, indeed one of our goals is to federate the ESG search with ESIP...
> Luca
> 
> On Jun 7, 2011, at 11:18 PM, Mattmann, Chris A (388J) wrote:
> 
>> Guys you may want to check out the work in ESIP going on with the Federated Search folks:
>> 
>> http://wiki.esipfed.org/index.php/How-To_Guide_for_Implementing_ESIP_Federated_Search_Servers
>> 
>> Just passing it along.
>> 
>> Thanks!
>> 
>> Cheers,
>> Chris
>> 
>> On Jun 7, 2011, at 7:21 AM, <stephen.pascoe at stfc.ac.uk> <stephen.pascoe at stfc.ac.uk> wrote:
>> 
>>> In off-list discussions about QC we've found we need to attach QC documents to data at the simulation level.  This would seem to be semantically similar to selecting all datasets for a given activity/product/institute/model/experiment which could be viewed as a query.
>>> 
>>> Unfortunately I wasn't on the Search API call yesterday but I hear OpenSearch has been floated as an implementation-neutral interface.  What about using OpenSearch's XML Query syntax for representing this?  E.g.
>>> 
>>> <Query xmlns="http://a9.com/-/spec/openseaerch/1.1/"
>>>     xmlns:esgf="http://esgf.org/ns/search/"
>>>     role="request"
>>>      esgf:activity="cmip5"
>>>     esgf:product="output"
>>>     esgf:institute="MTEST"
>>>     esgf:model="ECHAM6-MPIOM-TR"
>>>     esgf:experiment="amip"/>
>>> 
>>> Cheers,
>>> Stephen.
>>> 
>>> ---
>>> Stephen Pascoe  +44 (0)1235 445980
>>> Centre of Environmental Data Archival
>>> STFC Rutherford Appleton Laboratory, Harwell Oxford, Didcot OX11 0QX, UK
>>> 
>>> 
>>> -- 
>>> Scanned by iCritical.
>>> 
>>> 
>>> _______________________________________________
>>> GO-ESSP-TECH mailing list
>>> GO-ESSP-TECH at ucar.edu
>>> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
>> 
>> 
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Chris Mattmann, Ph.D.
>> Senior Computer Scientist
>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>> Office: 171-266B, Mailstop: 171-246
>> Email: chris.a.mattmann at nasa.gov
>> WWW:   http://sunset.usc.edu/~mattmann/
>> Phone: +1 (818) 354-8810
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Adjunct Assistant Professor, Computer Science Department
>> University of Southern California, Los Angeles, CA 90089 USA
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> 
>> _______________________________________________
>> GO-ESSP-TECH mailing list
>> GO-ESSP-TECH at ucar.edu
>> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech
> 
> -- 
> Scanned by iCritical.
> _______________________________________________
> GO-ESSP-TECH mailing list
> GO-ESSP-TECH at ucar.edu
> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech

--
Dr. Christopher Lynnes     NASA/GSFC, Code 610.2    phone: 301-614-5185