[Go-essp-tech] QC status

[Williams, Dean N.]

Hi Martina,

    Thanks for sending out an update on the QC status and how it is being
coordinated with the vital components of the federation.


> a short status of the QC implementation:
> 
> QC documentation (http://purl.org/org/cmip5/qc):
> We use this redmine pages for supporting errors or problems with the QC
> tool as well. And we set up a subproject for discussion and technical QC
> information within ESGF. Both parts need a registration. So, all people
> involved in the QC L2 checks please register there.
We need coordination between the Publisher's QC L1 check and QC L2 check to
see if there are any overlaps and more importantly any gaps. Has this been
done already? Bob and Jeff from PCMDI will need to be involved here.
> 
> For the construction of data links on the DOI resolving page we use the
> information about your TDS root directory from
> http://esgf.org/wiki/Cmip5Status#TDS_Specific
> 
> QC checks L2:
> BADC is intensively testing the qc tool on their UKMO data. Within that
> tests some issues to be fixed came up. Parts of the results are ingested
> and accessible from the central QCDB.
> status:
> * last changes of QC tool and QC wrapper: We will fix a version by end
> of February and move the bug report from personal emails to the redmine
> pages (https://redmine.dkrz.de/collaboration/projects/qc-internal -
> registration required).
> * criteria for assignment of QC L2 are in preparation (see the QC tool
> exception list: http://www.leuchtturm-atlas.de/SCR/qc2list.html)
Jeff Painter is out this week, but I know he has been trying to test the QC
tool on test data here at PCMDI. When he returns, I'll get an update from
him.
> 
> QC checks L3:
> This is in the final stage of adaptation.
> * GUI to support the author approval step: first release for testing
> within WDCC on 26.01.
> * export of results from QCDB and cross- and double-checks are ready for
> the current metafor examples.
> 
> QC and metafor:
> We agreed on
> 1. using the contact and author list information from the questionnaire
> simulations accessed via atomfeed.
> 2. reporting all QC results to metafor and from there via atomFeed to
> the gateways. QC Level and DOI information have to reach the gateway.
> 3. setting up the DOI resolving page with basic information on the
> DOI-ed data at BADC/within metafor
> Unfortunately, we have no timeline with metafor for that issues.
Yes, this seems much further out and not in the February time frame.
> 
> status:
> 1. author list information have not found a place in the CIM document.
> We are going to ask the main contact for that author list for the data
> publication in the QC L3 process. The main contact might not include an
> email address. The available examples does not.
> 2. A QC questionnaire with atomFeed is set up, but a tool for automatic
> ingest of QC result information into the CIM repository is missing.
> 3. There is no example or to be used URL known to us. For the link to
> additional metadata information that should point directly to the DRS
> experiment within metafor we have no construction, yet.
> 
> getting around:
> 1. list of contacts at the modelling centers in the internal part of the
> QC pages: Could you or Karl send such a list to me?
Karl, can you send this list to Martina?

> 2. WDCC's database CERA can provide all quality information, since we
> need them during QC L3. The relevant piece of information for the
> gateways are QC Level and DOI (if existing). We have already set up a
> view to provide that information.
> 3. We have started to set up a "mirror" of the DOI resolving page at
> WDCC filled with data out of CERA. The link to data and metadata have
> still to be added.
> 
> requirement:
> We could need a table with the information, which modelling center
> delivers its data to which of the ESGF partners. E.g. the TDS2CIM tool
> could be prevented from scanning the same experiment in all three TDS
> after replication checking for exchanged or new data. It would be
> helpful for the QC workflow as well, especially in the case of data
> exchanges during the QC process or revised data after DOI publication.
> Dean, could you or Karl set up such a table in the ESGF wiki or elsewhere?
This is a good suggestion. We will work on this table between PCMDI, BADC,
DKRZ, and ANU. We will post it on the ESGF website.

Best regards,
    Dean
> 
> Best wishes,
> Martina
>