[Go-essp-tech] Fwd: Expected number of variables for which quality control will be needed

Thu Jan 6 09:16:08 MST 2011

Hi Martina, Bryan, Martin, Karl, and .....

this discussion stream fits a discussion Karl raised shortly before 
Christmas when Martina and Frank are captured in San Francisco due to 
snow air traffic shut down in Europe. Karl had been asking for 
interpretation of QC-L2 output interpretation. This is connected to two 
more questions:
the complexity of QC-L2 checks (What can be achieved in an acceptable 
period of time?) and
the definition of criteria for the assignment of the data flag "QC-L2 
passed".

The complexity discussion of QC-L2 checks has just been started on this 
thread and should be continued. But I would like to see also a 
discussion about the criteria for assignment of "QC-L2 passed" to CMIP5 
data. The problem I see is that QC-L2 does not give a clear yes/no or 
white/black decision (as for the CMOR-2 compliance). QC-L2 disseminate 
also grey colour results and we have to deal with these grey colours. 
How much grey colour we can accept without loosing too much quality?

I think we have to discuss precisely what tests can be achieved in QC-L2 
with respect to work load and benefit and develop guidelines to weight 
the results.

Since I am not continuously available for the next weeks I  suggest that 
Martina and Frank own this discussion for DKRZ as Stephan and Estani do 
for the technical part.

With respect to our upcoming GO-ESSP meeting May 2011 in Asheville I 
think we could present our CMIP5 quality control management and discuss 
it in a slightly wider community.

So far for the moment my ideas to this thread on QC in CMIP5.

Best wishes, Michael

---------------
Dr. Michael Lautenschlager

German Climate Computing Centre (DKRZ)
World Data Center Climate (WDCC)
ADDRESS: Bundesstrasse 45a, D-20146 Hamburg, Germany
PHONE:   +4940-460094-118
E-Mail:  lautenschlager at dkrz.de

URL:    http://www.dkrz.de/
         http://www.wdc-climate.de/

Geschäftsführer: Prof. Dr. Thomas Ludwig
Sitz der Gesellschaft: Hamburg
Amtsgericht Hamburg HRB 39784

Am 06.01.2011 12:13, schrieb Martina Stockhause:
>    Hi, Bryan,
>
> I expect that
> - existing errors are found and documented by the QC tool
> - errors are analysed and catagorised by the select-Script (part of the
> wrapper package) : We are still working on this catagorisation and
> evaluation of errors.
>
> The plots are an additional help if the person running the QC is in
> doubt and for documentation of QC results. If there were errors in the
> data  not found by the QC tool, I would expect that they were visible
> within the first few plots.
>
> During QC L3 checks we will double-check the QC L2 results using
> logfiles and plots for spot checking.
>
> Best wishes,
> Martina
>
>
> On 01/05/2011 06:11 PM, Bryan Lawrence wrote:
>> Hi Karl, Martina
>>
>> So are we really expecting PCMDI to look at more than 10,000 plots per
>> day as part of QC level 2?
>>
>> (Martin's numbers are on an internal wiki, but a back of the envelope
>> calculation goes something like:
>>    - one plot per atomic data set,
>>    - o(10^6) atomic datasets
>>    - 100 days
>>    - o(10^4) per day at PCMDI given BADC and DKRZ doing a negligible
>> amount cf PCMDI under current plans)
>>
>> Cheers
>> Bryan
>>
>> ---- Original Message ----
>>> From: "Juckes, Martin (STFC,RAL,SSTD)"<martin.juckes at stfc.ac.uk>
>>> To: "Lawrence, Bryan (STFC,RAL,SSTD)"<bryan.lawrence at stfc.ac.uk>,
>> badc<badc-internal at zonda.badc.rl.ac.uk>
>>> CC: "Pepler, Sam (STFC,RAL,SSTD)"<sam.pepler at stfc.ac.uk>
>>> Subject:Expected number of variables for which quality control will be
>> needed
>>> Following the discussion in the CMIP5 meeting this morning, I've put
>>> some estimates of numbers of variables in
>>> http://proj.badc.rl.ac.uk/badc/wiki/Ar5Cmip5/MOHCCmip5/VolumePredict
>>> ions
>>>
>>> It looks as though we will have up to 40,000 from UKMO (including
>>> HiGEM). If we want to do this in around 100 working days (leaving
>>> some room for repeats), we have 400 plots to verify per day. This
>>> much is, I think, manageable. If we expand it to, say, a third of
>>> the CMIP5 experiment, we will have 4,000 per day, which looks
>>> problematic -- but if it has to be done we could probably keep the
>>> plot inspection time quite small.
>>>
>>> Cheers,
>>> Martin
>>>
>>>
>>> -----Original Message-----
>>> From: Bryan Lawrence [mailto:bryan.lawrence at stfc.ac.uk]
>>> Sent: 05 January 2011 15:51
>>> To: badc
>>> Cc: Pepler, Sam (STFC,RAL,SSTD); Juckes, Martin (STFC,RAL,SSTD)
>>> Subject: cmip5 meeting summary
>>>
>>> Hi Folks
>>>
>>> I've posted a short (2 page) doc suitable for public viewing of where
>>> we are at with cmip5 support on the MIRP website at
>>> http://proj.badc.rl.ac.uk/mirp/wiki/CMIP5status
>>>
>>> If I can't make future meetings, could Sam or Martin please ensure
>>> that a version of this document is updated for future meetings.
>>>
>>> Cheers
>>> Bryan
>>>
>>>
>>> --
>>> Bryan Lawrence
>>> Director of Environmental Archival and Associated Research
>>> (NCAS/British Atmospheric Data Centre and NCEO/NERC NEODC)
>>> STFC, Rutherford Appleton Laboratory
>>> Phone +44 1235 445012; Fax ... 5848;
>>> Web: home.badc.rl.ac.uk/lawrence
>>> --
>>> Scanned by iCritical.
>> --
>> Bryan Lawrence
>> Director of Environmental Archival and Associated Research
>> (NCAS/British Atmospheric Data Centre and NCEO/NERC NEODC)
>> STFC, Rutherford Appleton Laboratory
>> Phone +44 1235 445012; Fax ... 5848;
>> Web: home.badc.rl.ac.uk/lawrence
>