<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Hi, Philip,<br>
<br>
I'd like to clarify, what we do in QC checks for level 2:<br>
<br>
We calculate global statistical values in order to check them
against thresholds or if the variation (min/max) of a parameter
stays in a certain reliable range. More information on the checks is
given in the document Martin sent around yesterday:
CMIP5-AR5-QualityControl-100820.pdf.<br>
<br>
The ranges are set currently to ( min_t - ave(min)_t ) >
magnitude(min_t) * 10^5<br>
ave(min)_t : mean of all minima until the current time step, where
minimum is the minimum over the whole field. The same ranges are set
for the maximum.<br>
<br>
We are aware that this is quite imprecise, but is able to find
runaway values.<br>
<br>
Best wishes,<br>
Martina<br>
<font size="2"><font face="Calibri"><br>
</font></font>
<meta http-equiv="CONTENT-TYPE" content="text/html;
charset=ISO-8859-1">
<title></title>
<meta name="GENERATOR" content="OpenOffice.org 3.2 (Unix)">
<style type="text/css">
<!--
@page { margin: 0.79in }
P { margin-bottom: 0.08in }
--></style><br>
<br>
On 01/07/2011 11:22 AM, Bentley, Philip wrote:
<blockquote
cite="mid:E51EDFEBF10BE44BB4BDAF5FC2F024B9036365B8@EXXMAIL02.desktop.frd.metoffice.com"
type="cite">
<pre wrap="">Hi Martin,
It's good to hear that our initial tranche of CMIP5 data 'appears' to
get the green light. Since we produced the data using the latest (at the
time) release of CMOR-2 I think, as a project, we'd be in deep doo-doos
if it hadn't passed :-)
As regards the 4 QC checks you mention below, the first three would
appear to be fairly cheap operations. However, I'm aware from my own
recent experience of checking our data outputs that computing the
min/max bounds for a given atomic dataset (one variable spread over
numerous netcdf files) is a cpu- and time-expensive operation.
So I was curious to know why this particular check needs to be done?
Since most of the min/max bounds have been removed from the MIP tables,
what bounds would be used in the QC L2 checks? Perhaps it's the case
that this check is simply skipped if no min/max bounds are defined in
the MIP tables?
Regards,
Phil
</pre>
<blockquote type="cite">
<pre wrap="">
I hope we can get some conclusion on what constitutes passed
QC L2 before the Asheville meeting -- we have some UKMO data
which appears to pass all the tests -- but it is hard to be
sure because of the complexity of the output from the quality
control code. We would like to declare this data as passed,
so that we can get onto the next problem (replicating it to
other centres).
The quality control document (attached) lists 4 objective
tests under QC level 2:
(1) Number of records in each file consistent with metadata
(2) Regular time steps
(3) Metadata consistent with data request
(4) Minimum and maximum checked against specified ranges (or
a default based on the mean).
An email from Martina this morning implied that subjective
tests would only come in at the QC L3 stage or if there is
some doubt about the objective tests.
I'm not clear why the process is so complex when the document
only specifies a small number of objective tests -- though
there clearly is complexity of the workflow (ensuring that
results for tests of all files are recorded and retrievable).
Regards,
Martin
</pre>
</blockquote>
<pre wrap="">_______________________________________________
GO-ESSP-TECH mailing list
<a class="moz-txt-link-abbreviated" href="mailto:GO-ESSP-TECH@ucar.edu">GO-ESSP-TECH@ucar.edu</a>
<a class="moz-txt-link-freetext" href="http://mailman.ucar.edu/mailman/listinfo/go-essp-tech">http://mailman.ucar.edu/mailman/listinfo/go-essp-tech</a>
</pre>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
----------- DKRZ / Data Management -----------
Martina Stockhause
Deutsches Klimarechenzentrum
Bundesstr. 45a
D-20146 Hamburg
Germany
phone: +49-40-460094-122
FAX: +49-40-460094-106
e-mail: <a class="moz-txt-link-abbreviated" href="mailto:martina.stockhause@zmaw.de">martina.stockhause@zmaw.de</a>
----------------------------------------------
</pre>
</body>
</html>