[Go-essp-tech] status metadata from/to CIM for quality control in CMIP5

Mon Nov 1 02:36:32 MDT 2010

  Hi Bryan,

I'd like to sum up the status of our metadata exchange for CMIP5 quality 
contols.

1. get CIM simulationRun for CMIP5 experiment:

I have still not understood how this will be provided: as AtomFeed 
including all described simulationRuns or by a tool underlying the CIM 
portal (ask for simulationRun of DRS experiment) providing the needed 
simulationRun? I could need an example or at least the description of my 
interface.

2. get contact out of CIM simulationRun:

During QC L2 and L3 checks it might be necessary to get in touch with 
the data provider. I recall that we agreed on using the "contact" 
(role=contact) in the simulationRun description. There is still the 
field "AuthorList" in the form which gets lost afterwards but is 
mandatory for the user. The XML export includes only the comment 
. If this information cannot be stored 
in CIM then delete it from the form and we have to ask for the list of 
authors of the DOI during QC L3. And for the contact I asked a couple of 
times to make the "email" field in the parties form mandatory. 
Unfortunately, it is not.

Where do I get the email address for my contact? Will you check and if 
necessary add that during QC L2 for metadata? If not, is there a list of 
contacts including email addresses available that I could use instead of 
CIM metadata?

I used an xml export of a MPI-M example to start with the development of 
my cross-checks of QC L3. I have started to document the QC L3 checks at 
https://redmine.dkrz.de/collaboration/projects/cmip5-qc/wiki/Qc_level_3_checks. 
The majority of the checks test if the information extracted from the 
TDS and ingested in CERA is still equal to the actual metadata in the 
TDS. The overlap for information of TDS and CIM simulationRun is rather 
small.

3. add quality information to CIM

qctool: Bryan, I am not allowed to open requests in the metafor request 
tracker. So, I report it here. A report is stored without the sections 
conformance result and quantitative result. These information is simply 
lost.

I am prepared to export the QC L2 result out of the QC DB. One for 
exporting the QC L3 result from CERA will follow. I could need the XML 
scheme for the report, since I cannot export it from the qctool web site 
(see above).

4. update CIM information for QC L3

It is still not clear to me, how metadata changed by the data author 
during the QC L3 process, which is changed in CERA, will be reported to CIM.

Meantime we set up a view in CERA providing information for quality 
status and DOI, e.g. for our test case:

http://cera-www.dkrz.de/ceraMeta/SelectGeneric?&q=*%20from%20cera2.v_qc_Status%20where%20entry_name%3D%27CMIP5%20output%20MPI-M%20ECHAM6-MPIOM-LR%20rcp45%27&sep=,&fo=x

(select * from cera2.v_qc_Status where entry_name='CMIP5 output MPI-M 
ECHAM6-MPIOM-LR rcp45';)

ENTRY_NAME = DRS experiment name (separated by blanks)
ACCESS_SPEC = DOI link if available
SPECIFICATION = quality flags

We can change that if necessary.

That's our status for the QC-CIM metadata exchanges.

Best wishes,
Martina

On 10/08/2010 05:11 PM, Bryan Lawrence wrote:
> hi Folks
>
> Dominic has deployed the qctool for entering quality information at
> http://qc.cmip5.ceda.ac.uk  (just one more "c" than the questionnaire).
>
> The front page is open, to get further, you use the same access criteria
> you use for access to the questionnaire.
>
> This tool is for *recording* information. It has an atom feed, so other
> tools can use the information, and we expect humans and (other) tools to
> *generate* the information. Currently one would have to enter such
> information manually (although it does allow upload of or links to plots
> and logfiles). A future version will allow one to post an xml upload.
> Future versions will also allow qctool instances deployed to federate
> with each other.
>
> (On access control: in the next few weeks I am hopeful that the ESG
> openid federation will "go-live", which means that an ESG access control
> token should allow access to either the questionnaire or the qctool).
>
> The qctool has had *no* user testing. It might break horribly.  It is
> being backed up daily ... but ...  It's much simpler than the
> questionnaire, so hopefully we will fix things in much more real time,
> and have little problem persisting information (and, in the short term,
> should we fail, it's less likely to be a problem to re-enter ...
> particularly given it is trivial to get xml versions of the material
> entered from the tool).
>
> (I expect it to change even over the next few days, but these changes
> should just be adding functionality and improving look-n-feel, things
> like a title etc :-) ... but we wanted it it out NOW, since we said we
> would :-).
>
> Feedback on usage gratefully accepted (to the metafor list and/or
> metafor trac site using keyword qctool).
>
> Building this has exposed some interesting issues, both with the CIM UML
> (ConCIM) with the CIM serialisation (ApCIM) that we will explore with
> Allyn at the upcoming metafor services meeting.
>
> Cheers
> Bryan
>
>
> Bryan Lawrence
> Director of Environmental Archival and Associated Research
> (NCAS/British Atmospheric Data Centre and NCEO/NERC NEODC)
> STFC, Rutherford Appleton Laboratory
> Phone +44 1235 445012; Fax ... 5848;
> Web: home.badc.rl.ac.uk/lawrence
> _______________________________________________
> GO-ESSP-TECH mailing list
> GO-ESSP-TECH at ucar.edu
> http://mailman.ucar.edu/mailman/listinfo/go-essp-tech