[Go-essp-tech] Interfacing to esgcet database (was Minutes: 1/6/10 Output Variable Telecon)

[Bob Drach]

Hi Stephen,

I agree that top-down coordination probably isn't necessary at this  
point, especially given that the database access is read-only  
(correct?). However there is no guarantee that the schema will not  
change in such a way that your extra ORM classes break. It might be  
worth considering specification of an API that would be stable across  
future schema migrations, and could be shared among  METAFOR  
applications.

Bob

On Jan 8, 2010, at 1:24 AM, <stephen.pascoe at stfc.ac.uk> wrote:

>
> Is either Hans' or Mark's work intended to be part of the METAFOR  
> tools
> or are they primarily internal development for DKRZ & IPSL?  I am
> assuming we will be deploying METAFOR tools to generate CIM from the
> NetCDF.
>
> For our part we are designing an interface to the esgcet database as
> part of our CMIP5 data ingest system.  This is a different use case to
> producing CIM.  Our plan is to use the esgcet.model package's  
> SQLAlchemy
> interface and add extra ORM classes that reference our own schema,  
> thus
> using SQLALchemy as a schema integration tool.
>
> At this stage top-down coordination probably isn't benificial but I
> suggest we should keep on our separate tracks but keep our code in the
> open so that any obvious overlaps are seen early.  I will post our SVN
> URL once we start coding.
>
> Cheers,
> Stephen.
>
> ---
> Stephen Pascoe  +44 (0)1235 445980
> British Atmospheric Data Centre
> Rutherford Appleton Laboratory
>
> -----Original Message-----
> From: metafor-bounces at lists.enes.org
> [mailto:metafor-bounces at lists.enes.org] On Behalf Of Hans Ramthun
> Sent: 07 January 2010 15:09
> To: Mark Morgan
> Cc: go-essp-tech at ucar.edu; curator at ucar.edu; Sylvia Murphy; Metafor
> List; Karl Taylor
> Subject: Re: [metafor] [Go-essp-tech] Minutes: 1/6/10 Output Variable
> Telecon
>
> Hallo Mark,
>
> Yes of course there are several points of collaboration. I'm happy not
> to dig alone through the mountain...
>
> I published my very first steps in the Metafor svn:
> http://*metaforclimate.eu/trac/browser/cimTOOLS/branches/ESG-DB- 
> access-TE
> ST
> The first test was with the python library 'psycopg2'.
>
> Cheers
> Hans
>
>
> Mark Morgan wrote:
>> Hans
>>
>> Here at IPSL we are in the process of beginning to do exactly the  
>> same
>> thing: coding a python library to interface with the ESG publisher
>> postgres database in order to act as a prelimnary step in the Metafor
>> CIM publishing process.
>>
>> Perhaps this can be a point of collaboration either in the  
>> development
>
>> or testing (or both)?
>>
>> Regards
>>
>> Mark
>>
>> Hans Ramthun wrote:
>>> Hallo,
>>>
>>> Bryan Lawrence wrote:
>>>
>>>> ...
>>>>
>>>>> Right now, we would have to parse the thredds catalog as you  
>>>>> say to
>
>>>>> get the list of variables.  No code exists to convert the data
>>>>> collection names into a list of variable names.  Code does  
>>>>> exist to
>
>>>>> create the list of data collections.
>>>>>
>>>> I think we all need a bit of code for that. We should write it
>>>> standalone so we can all use the logic if not the code ...
>>>> hopefully this is on the agenda for Hans and/or Stephen.
>>>>
>>> On my agenda is to access the ESG publisher postgres database with
>>> SQLAlchemy and to retrieve the appropriate data to create CIMRecords
>>> (dataObject). This should led to a stand alone python program. I
>>> think Stephen wants to go the same way.
>>>
>>>
>>>> Cheers
>>>> Bryan
>>>>
>>> Cheers
>>> Hans
>>>
>>>
>>
>
>
> --
> Hans Ramthun
> Tel.: +49.40.460.094.112
> Email: hans.ramthun at zmaw.de
>
> DKRZ-Hamburg / Max-Planck-Institut fuer Meteorologie
> Abteilung Modelle und Daten     http://*www.*mad.zmaw.de/
> Bundesstr. 45a
> D-20146 Hamburg                 Germany
>
> _______________________________________________
> metafor mailing list
> metafor at lists.enes.org
> https://*lists.enes.org/mailman/listinfo/metafor
> -- 
> Scanned by iCritical.
> _______________________________________________
> GO-ESSP-TECH mailing list
> GO-ESSP-TECH at ucar.edu
> http://*mailman.ucar.edu/mailman/listinfo/go-essp-tech
>