[Go-essp-tech] Status update on metadata pipeline

[Sylvia Murphy]

Hi all,

This a quick update on the metadata pipeline.

***STATUS***
* The metadata pipeline is working.   We can download files from the Metafor harvest and display the results in the ESG gateway trackback.  The items we can  currently harvest and display include basic properties, technical properties, and scientific properties. We have tested this capability with the CIM sample from MOHC (the Hadgem2-es model) dated 1/24.

* An upgrade to the Curator Trackback was completed after ESG release 1.2 and will be part of the ESG 1.3 release.  The main upgrade is changing the left navigation bar so that the simulation name does not precede every item.  This change makes the navigation bar a lot less crowded and easier to use.

Caveats:
    **Metafor's CIM XML output format continues to change.  This has the potential for breaking the pipeline until harvester code can be modified to accommodate the changes.    Although Metafor's  CIM 1.5 schema (the theoretical basis for Metafor's work) has been finalized, the changes still needs to be reflected in the questionnaire and its output XML.  Metafor plans to redeploy the questionnaire on 4/14, bringing it into compliance with CIM 1.5.  At that point they anticipate only small changes in the XML output (specifically grids) going forward.  

    ** Validation in Metafor's questionnaire has been broken, which has prevented new CIM XML files from being exported.  This means that the Curator team has not had new files to test with.  As part of the redeployment mentioned above, Metafor plans to turn off validation briefly in order to export from the questionnaire XML files for MOHC's four completed model simulations.  Our goal is to get these working and display them as examples for upcoming demos.

***NEAR TERM WORK***
* ESG plans a Curator sprint at the end of month (25 April - 8 May).  The NCAR ESG team and myself will be responsible for the completion of the outstanding items.   This sprint will end up in ESG release 1.3 or 1.4 depending upon the results of this week's ESG sprint and the release schedule for 1.3.

The following are our proposed priorities for the sprint, in order of importance:

1. Make sure that the new CIM samples from MOHC can be displayed properly.

2. Enable experiment information to be displayed.  An OWL file containing all of the CMIP5 experiments and their requirements needs to be prepared first.  This will be used by both the data and metadata side of ESG.  On the data side this will be used as the experiment search facets.  On the metadata side this will be used to display the experiment description and experimental requirements associated with each simulation.  We were hoping to write software to do this but are now investigating manual options.  To the extent possible the owl file will be completed before the sprint.

2. Enable the following items to be harvested from the CIM XML.  These items represent additions to the existing functionality and are not required for a basic level of operation.  The order below is the proposed priority - please comment if you would like to change this.
 ** Inputs/couplings (this was previously working but the location of the information has changed)
 ** Grids
 ** Model modifications
 ** Ensemble information

3. Review the user experience and make sure that any additional items that need to be completed (error pages, explanations, acknowledgments) are there.

DEMONSTRATIONS
* The Curator Trackback will be demonstrated in Brussels (by Metafor) on May 6th and during the GO-ESSP meeting in North Carolina (May 10-12).  Reviews of the portal will occur on May 2nd during the regular Curator telco and on May 3rd during the regular Metafor telco.  We expect that participants in these demonstrations will be able to search for the four MOHC simulation and get a walk through of the resulting display (e.g. basic properties, technical properties, scientific properties etc).     
***********************************
Sylvia Murphy
NESII/CIRES/NOAA Earth System Research Laboratory
325 Broadway, Boulder CO 80305
Email: sylvia.murphy at noaa.gov
Phone: 303-497-7753
Fax: 303-497-7649







-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/go-essp-tech/attachments/20110412/03ea24c3/attachment.html