<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=Content-Type content="text/html; charset=us-ascii">
<meta name=Generator content="Microsoft Word 12 (filtered medium)">
<!--[if !mso]>
<style>
v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style>
<![endif]-->
<style>
<!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
@font-face
{font-family:"Times New\000D\000A Roman";
panose-1:0 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Times New\000D\000A Roma";
panose-1:0 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Times New\000D\000A Ro";
panose-1:0 0 0 0 0 0 0 0 0 0;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";
color:black;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:12.0pt;
font-family:"Times New Roman","serif";
color:black;}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0cm;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";
color:black;}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:Consolas;
color:black;}
span.EmailStyle19
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle21
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
-->
</style>
<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body bgcolor=white lang=EN-GB link=blue vlink=purple>
<div class=WordSection1>
<p class=MsoNormal><font size=2 color="#1f497d" face=Calibri><span
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Hello
Karl,<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 color="#1f497d" face=Calibri><span
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 color="#1f497d" face=Calibri><span
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Its
true that there is no explicit requirement for complete CF-compliance in the QC
level 2 documentation.<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 color="#1f497d" face=Calibri><span
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 color="#1f497d" face=Calibri><span
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>I’ll
assume for the time being that there is no option of changing the CF 1.4
conformance document, so that CMOR v2.2 output will remain non-compliant even
if the CF checker is adjusted to pass it with a warning.<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 color="#1f497d" face=Calibri><span
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 color="#1f497d" face=Calibri><span
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>In
terms of what is easiest for users, I believe that having accurate metadata in
the file is crucial. I can see Jonathan’s point that uniformity across
the archive is also important, but I would rate that as a lower priority. Given
our general commitment to promoting CF compliant data, I think that it would be
a big mistake to keep producing non-compliant data now that we have identified
the problem. Especially when the metadata in the files explicitly states that
it is CF compliant.<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 color="#1f497d" face=Calibri><span
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 color="#1f497d" face=Calibri><span
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>It
appears that the volume of CMOR v2.2 data is too large for easy correction, so
perhaps clearly documenting the departure from the CF convention is the best
option – though this is not ideal as there is nothing in the files which
will tell users where to find the relevant information.<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 color="#1f497d" face=Calibri><span
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 color="#1f497d" face=Calibri><span
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>I
also think that the departure from CF compliance should be documented in the QC
metadata – but that can be dealt with later.<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 color="#1f497d" face=Calibri><span
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 color="#1f497d" face=Calibri><span
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Regards,<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 color="#1f497d" face=Calibri><span
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Martin<o:p></o:p></span></font></p>
<div style='border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt'>
<div>
<div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm'>
<p class=MsoNormal><b><font size=2 color=black face=Tahoma><span lang=EN-US
style='font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext;
font-weight:bold'>From:</span></font></b><font size=2 color=black face=Tahoma><span
lang=EN-US style='font-size:10.0pt;font-family:"Tahoma","sans-serif";
color:windowtext'> Karl Taylor [mailto:taylor13@llnl.gov] <br>
<b><span style='font-weight:bold'>Sent:</span></b> 01 November 2010 22:13<br>
<b><span style='font-weight:bold'>To:</span></b> Juckes, Martin (STFC,RAL,SSTD)<br>
<b><span style='font-weight:bold'>Cc:</span></b>
jamie.kettleborough@metoffice.gov.uk; Lawrence, Bryan (STFC,RAL,SSTD);
lautenschlager@dkrz.de; philip.bentley@metoffice.gov.uk; V.Balaji@noaa.gov; go-essp-tech@ucar.edu;
cmor@lists.llnl.gov; Kyle.Olivo@noaa.gov; Doutriaux, Charles<br>
<b><span style='font-weight:bold'>Subject:</span></b> Re: [Go-essp-tech] CMOR
and cell_measures issues<o:p></o:p></span></font></p>
</div>
</div>
<p class=MsoNormal><font size=3 color=black face="Times New Roman"><span
style='font-size:12.0pt'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=3 color=black face="Times New Roman"><span
style='font-size:12.0pt'>Couldn't we simply say that we certify that CMIP5 data
conforms to the CF 1.4 standard except that the cell_measures variables
may be found in an external file, rather than the referencing file.
That way the data will pass the CMIP5 QC checks which don't include
requiring the cell_measures variables to be found in the referencing
file. I think the decision between cell_measures and
ext_cell_measures should be based on which one will be most useful to the
users. In CMIP5, users should be able to find the cell areas even without
cell_measures, so I'm not sure this decision is all that critical.<br>
<br>
regards,<br>
Karl<br>
<br>
<br>
<br>
On 11/1/10 2:30 PM, <a href="mailto:martin.juckes@stfc.ac.uk">martin.juckes@stfc.ac.uk</a>
wrote: <o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 color="#1f497d" face=Calibri><span
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Hello
All,</span></font><o:p></o:p></p>
<p class=MsoNormal><font size=2 color="#1f497d" face=Calibri><span
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span></font><o:p></o:p></p>
<p class=MsoNormal><font size=2 color="#1f497d" face=Calibri><span
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Sorry
to be repetitive, but I want to repeat a question I raised earlier today
(Monday in the UK) and hasn’t been answered yet: will the proposed change
to the CF checker be matched to a change to the conformance document so that
the CF 1.4 conformance no longer demands that variables named in cell_measures
be in the same file? </span></font><o:p></o:p></p>
<p class=MsoNormal><font size=2 color="#1f497d" face=Calibri><span
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span></font><o:p></o:p></p>
<p class=MsoNormal><font size=2 color="#1f497d" face=Calibri><span
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>I’ve
also copied Bryan and Michael in again, so to get a quality control perspective
– as it worries me that an agreement made in a rush might not meet the
expectations of the quality control we have committed to,</span></font><o:p></o:p></p>
<p class=MsoNormal><font size=2 color="#1f497d" face=Calibri><span
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span></font><o:p></o:p></p>
<p class=MsoNormal><font size=2 color="#1f497d" face=Calibri><span
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Regards,</span></font><o:p></o:p></p>
<p class=MsoNormal><font size=2 color="#1f497d" face=Calibri><span
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Martin
</span></font><o:p></o:p></p>
<p class=MsoNormal><font size=2 color="#1f497d" face=Calibri><span
style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span></font><o:p></o:p></p>
<div style='border:none;border-left:solid windowtext 1.5pt;padding:0cm 0cm 0cm 4.0pt;
border-color:-moz-use-text-color -moz-use-text-color -moz-use-text-color blue'>
<div>
<div style='border:none;border-top:solid windowtext 1.0pt;padding:3.0pt 0cm 0cm 0cm;
border-color:-moz-use-text-color -moz-use-text-color'>
<p class=MsoNormal><b><font size=2 color=black face=Tahoma><span lang=EN-US
style='font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext;
font-weight:bold'>From:</span></font></b><font size=2 color=black face=Tahoma><span
lang=EN-US style='font-size:10.0pt;font-family:"Tahoma","sans-serif";
color:windowtext'> Karl Taylor [<a href="mailto:taylor13@llnl.gov">mailto:taylor13@llnl.gov</a>]
<br>
<b><span style='font-weight:bold'>Sent:</span></b> 01 November 2010 18:19<br>
<b><span style='font-weight:bold'>To:</span></b> Kettleborough, Jamie<br>
<b><span style='font-weight:bold'>Cc:</span></b> Bentley, Philip; V. Balaji;
Juckes, Martin (STFC,RAL,SSTD); <a href="mailto:go-essp-tech@ucar.edu">go-essp-tech@ucar.edu</a>;
<a href="mailto:cmor@lists.llnl.gov">cmor@lists.llnl.gov</a>; <a
href="mailto:Kyle.Olivo@noaa.gov">Kyle.Olivo@noaa.gov</a>; Doutriaux, Charles<br>
<b><span style='font-weight:bold'>Subject:</span></b> Re: [Go-essp-tech] CMOR
and cell_measures issues</span></font><o:p></o:p></p>
</div>
</div>
<p class=MsoNormal><font size=3 color=black
face="Times New Roman"><span style='font-size:12.0pt;
font-family:"Times New Roman","serif"'> </span></font><o:p></o:p></p>
<p class=MsoNormal><font size=3 color=black
face="Times New Roman"><span style='font-size:12.0pt;
font-family:"Times New Roman","serif"'>Hi Jamie,<br>
<br>
I'm arguing that given that cell_measures (or ext_cell_measures) will *not*
appear in files containing fields most likely to be carried on a mesh
different from the "primary" mesh (because I've removed those from
the requested output table, and hence the CMOR tables), I think it is better to
*assume* the remaining variables are on the "primary" mesh. I
would be surprise if more than 1% of the variables written will have
cell_measures pointing to an incorrect area field. If it does, I assume
the area variable will have different latxlon dimensions than the variable
itself, so it will be difficult for a user to mistakenly apply the areas.<br>
<br>
So rather than advocate completeness over correctness, I'd say I'm advocating
"almost perfect" versus "perfect".<br>
<br>
If the number of offending cases is much larger than I'm imagining, please let
me know.<br>
<br>
Best regards,<br>
Karl<br>
<br>
On 11/1/10 10:09 AM, Kettleborough, Jamie wrote: </span></font><o:p></o:p></p>
<p class=MsoNormal><font size=2 color=blue face=Arial><span style='font-size:
10.0pt;font-family:"Arial","sans-serif";color:blue'>Hello Karl,</span></font><o:p></o:p></p>
<p class=MsoNormal><font size=3 color=black
face="Times New Roman"><span style='font-size:12.0pt;
font-family:"Times New Roman","serif"'> </span></font><o:p></o:p></p>
<p class=MsoNormal><font size=2 color=blue face=Arial><span style='font-size:
10.0pt;font-family:"Arial","sans-serif";color:blue'>thanks for this
reply. Putting aside the issue of whether this is really
ext_cell_measures or cell_measures then I think, given the resources we have
locally, we have to make a choice of correctness vs completeness. The
reason we are tempted to turn off ext_cell_measures is it is the least effort
way we can see of submitting data that is correct. I think you are
suggesting going for completness - even if we risk submitting some data with
ext_cell_measures that is incorrect. Obviously this is *my*
interpretation of what you are saying. Yes we can go for both correctness
and completeness, but this will take us some effort - we need an exta component
in our system that can recognise which cell areas to assign to
which variables (with minimum error) - and we (like everyone) have lots of
demands on our effort at the moment - and we have to make judgements about
where to prioritise. (This isn't supposed to be a sob story - just trying
to explain why we are tempted...)</span></font><o:p></o:p></p>
<p class=MsoNormal><font size=3 color=black
face="Times New Roman"><span style='font-size:12.0pt;
font-family:"Times New Roman","serif"'> </span></font><o:p></o:p></p>
<p class=MsoNormal><font size=2 color=blue face=Arial><span style='font-size:
10.0pt;font-family:"Arial","sans-serif";color:blue'>Would you recommend
'completeness' over 'correctness' - have I interpreted you correctly?
What are the options for correcting incorrect meta-data once data is ingested
into ESG?</span></font><o:p></o:p></p>
<p class=MsoNormal><font size=3 color=black
face="Times New Roman"><span style='font-size:12.0pt;
font-family:"Times New Roman","serif"'> </span></font><o:p></o:p></p>
<p class=MsoNormal><font size=2 color=blue face=Arial><span style='font-size:
10.0pt;font-family:"Arial","sans-serif";color:blue'>Jamie</span></font><o:p></o:p></p>
<div>
<p class=MsoNormal><font size=3 color=black
face="Times New Roma"><span style='font-size:12.0pt;
font-family:"Times New Roma","serif"'> </span></font><o:p></o:p></p>
</div>
<p class=MsoNormal><font size=3 color=black
face="Times New Roman"><span style='font-size:12.0pt;
font-family:"Times New Roman","serif"'> </span></font><o:p></o:p></p>
<blockquote style='border:none;border-left:solid windowtext 1.5pt;padding:0cm 0cm 0cm 4.0pt;
margin-left:3.75pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt;
border-color:-moz-use-text-color -moz-use-text-color -moz-use-text-color blue'>
<div class=MsoNormal align=center style='text-align:center'><font size=3
color=black face="Times New Roman"><span lang=EN-US style='font-size:12.0pt'>
<hr size=2 width="100%" align=center>
</span></font></div>
<p class=MsoNormal style='margin-bottom:12.0pt'><b><font size=2 color=black
face=Tahoma><span lang=EN-US style='font-size:10.0pt;font-family:"Tahoma","sans-serif";
font-weight:bold'>From:</span></font></b><font size=2 face=Tahoma><span
lang=EN-US style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> Karl
Taylor [<a href="mailto:taylor13@llnl.gov">mailto:taylor13@llnl.gov</a>] <br>
<b><span style='font-weight:bold'>Sent:</span></b> 29 October 2010 21:36<br>
<b><span style='font-weight:bold'>To:</span></b> Kettleborough, Jamie<br>
<b><span style='font-weight:bold'>Cc:</span></b> Bentley, Philip; V. Balaji; <a
href="mailto:martin.juckes@stfc.ac.uk">martin.juckes@stfc.ac.uk</a>; <a
href="mailto:go-essp-tech@ucar.edu">go-essp-tech@ucar.edu</a>; <a
href="mailto:cmor@lists.llnl.gov">cmor@lists.llnl.gov</a>; <a
href="mailto:Kyle.Olivo@noaa.gov">Kyle.Olivo@noaa.gov</a>; Doutriaux, Charles<br>
<b><span style='font-weight:bold'>Subject:</span></b> Re: [Go-essp-tech] CMOR
and cell_measures issues</span></font><o:p></o:p></p>
<p class=MsoNormal><font size=3 color=black
face="Times New Roma"><span style='font-size:12.0pt;
font-family:"Times New Roma","serif"'>Dear Jamie and
Charles (a couple of questions for you),<br>
<br>
<br>
</span></font><o:p></o:p></p>
<p class=MsoNormal><font size=2 color=blue face=Arial><span style='font-size:
10.0pt;font-family:"Arial","sans-serif";color:blue'>Hello Karl,</span></font><o:p></o:p></p>
<p class=MsoNormal><font size=3 color=black
face="Times New Roma"><span style='font-size:12.0pt;
font-family:"Times New Roma","serif"'> </span></font><o:p></o:p></p>
<p class=MsoNormal><font size=2 color=blue face=Arial><span style='font-size:
10.0pt;font-family:"Arial","sans-serif";color:blue'>I think the recommended way
to 'turn off' ext_cell_measures is to make a call to
cmor.set_variable_attribute(varid, 'ext_cell_measures', ''). Is that
right? We are very tempted to do this for all variables - so basically
overriding the MIP tables. How big a problem do you think this will be
for data users - our grid is pretty straight forward and users can calculate
cell_areas from the latitudes.</span></font><o:p></o:p></p>
<p class=MsoNormal><font size=3 color=black
face="Times New Roma"><span style='font-size:12.0pt;
font-family:"Times New Roma","serif"'><br>
Yes, if the cell areas stored in areacella are not appropriate for a particular
field, and the requested output tables say that ext_cell_measure includes
areacella, then you should call the set attribute function to reset
ext_cell_measures="". Isn't that right Charles?<br>
<br>
Why are you tempted to turn off the ext_cell_measures for all variables?
Then your output won't conform to the CMIP5 requirements.<br>
<br>
In the latest CMOR tables, I have removed ext_cell_measures from all the
variables that we don't expect always to be on the standard mesh (i.e., on the
grid for which areacella is correct). This includes velocities and
transports and closely related fields, which are sometimes staggered relative
to areacella. I would still be interested in hearing a clear explanation
for why there are additional fields carried on a completely different grid. <br>
<br>
If users must compute the cell areas for only your grid, and for all others
they simply read the areacella field in, then you are creating a special case
that is completely unnecessary.<br>
<br>
<br>
</span></font><o:p></o:p></p>
<p class=MsoNormal><font size=3 color=black
face="Times New Roma"><span style='font-size:12.0pt;
font-family:"Times New Roma","serif"'> </span></font><o:p></o:p></p>
<p class=MsoNormal><font size=2 color=blue face=Arial><span style='font-size:
10.0pt;font-family:"Arial","sans-serif";color:blue'>That aside, doesn't
the approach of providing alternative grid areas need more discussion?</span></font><o:p></o:p></p>
<p class=MsoNormal><font size=3 color=black
face="Times New Roma"><span style='font-size:12.0pt;
font-family:"Times New Roma","serif"'> </span></font><o:p></o:p></p>
<p class=MsoNormal><font size=2 color=blue face=Arial><span style='font-size:
10.0pt;font-family:"Arial","sans-serif";color:blue'> 1. how should we
produce these. The most natural approach I can think of is to modify the fx
MIP tables to add in areacellb (or whatever we choose to call it) and then
output through CMOR - this will maximise the chance of consistency between
different grid area files for any one model. </span></font><o:p></o:p></p>
<p class=MsoNormal><font size=3 color=black
face="Times New Roma"><span style='font-size:12.0pt;
font-family:"Times New Roma","serif"'> </span></font><o:p></o:p></p>
<p class=MsoNormal><font size=2 color=blue face=Arial><span style='font-size:
10.0pt;font-family:"Arial","sans-serif";color:blue'> 2. how should we
reference these additional areas from a variable.? I could call
cmor.set_variable_attribute(varid, 'ext_cell_measures', 'areacellb') - but in
the tests I've done on CMOR 2.4 this only does half the job: it puts the
appropriate ext_call_measures attribute into the file, but does nothing with associatedFiles.</span></font><o:p></o:p></p>
<p class=MsoNormal><font size=3 color=black
face="Times New Roma"><span style='font-size:12.0pt;
font-family:"Times New Roma","serif"'>I don't think it
is a high priority to standardize this immediately. We will want CMOR to
place the fields in the subdirectory fx, so I need to check with Charles
whether this requires the variable to appear in table fx. If not, I would
probably build an entirely new table similar to fx, but with only the
additional variables. This way you won't have to modify your table if a
new fx table comes out. As for referencing these additional area
variables, I think if you include area: <area_name> in the
ext_cell_measures attribute, then if CMOR isn't already doing this, Charles can
modify construction of associated_files to include something following the
template "<area_name>:
<area_name>_fx_IPSL-CM5_historical_r0i0p0.nc" What do
you think, Charles?<br>
<br>
<br>
</span></font><o:p></o:p></p>
<div>
<p class=MsoNormal><font size=3 color=black
face="Times New Ro"><span style='font-size:12.0pt;
font-family:"Times New Ro","serif"'> </span></font><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><font size=2 color=blue face=Arial><span style='font-size:
10.0pt;font-family:"Arial","sans-serif";color:blue'>Clearly these may have been
things you were going to cover - but ran out of time to, in which case sorry.</span></font><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><font size=3 color=black
face="Times New Ro"><span style='font-size:12.0pt;
font-family:"Times New Ro","serif"'> </span></font><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><font size=2 color=blue face=Arial><span style='font-size:
10.0pt;font-family:"Arial","sans-serif";color:blue'>I think another scenario
that still needs some thought is one where a data provider has submitted data
and published it in ESG. They then realise they made a mistake - they
should have turned ext_cell_measures off, but didn't (or visa-versa). What
happens in this case? (We have kind of done this in that we have send
data with incorrect cell_measures to the BADC - but have caught the
issue before ingestion into ESG - I don't believe we will always be this
lucky). You'll probably see through why I'm asking this
question about meta-data updates again now, so I may as well be explicit... If
we choose to turn off ext_cell_measures for all our diagnostics on this initial
submission - what are our options for recovering from this if we later
found the decision to submit without ext_cell_measures was making our data
hard to use?</span></font><o:p></o:p></p>
</div>
<p class=MsoNormal><font size=3 color=black
face="Times New Roma"><span style='font-size:12.0pt;
font-family:"Times New Roma","serif"'><br>
Please don't turn off ext_cell_measures (in general). I think you
could easily write a script to remove the cell_measures attribute using netCDF
tools, but adding it would require rewriting the entire file.<br>
<br>
Best regards,<br>
Karl<br>
<br>
<br>
</span></font><o:p></o:p></p>
<div>
<p class=MsoNormal><font size=3 color=black
face="Times New Ro"><span style='font-size:12.0pt;
font-family:"Times New Ro","serif"'> </span></font><o:p></o:p></p>
</div>
<div>
<p class=MsoNormal><font size=2 color=blue face=Arial><span style='font-size:
10.0pt;font-family:"Arial","sans-serif";color:blue'>Jamie</span></font><o:p></o:p></p>
</div>
<blockquote style='border:none;border-left:solid windowtext 1.5pt;padding:0cm 0cm 0cm 4.0pt;
margin-left:3.75pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt;
border-color:-moz-use-text-color -moz-use-text-color -moz-use-text-color blue'>
<div class=MsoNormal align=center style='text-align:center'><font size=3
color=black face="Times New Ro"><span lang=EN-US
style='font-size:12.0pt;font-family:"Times New Ro","serif"'>
<hr size=2 width="100%" align=center>
</span></font></div>
<p class=MsoNormal style='margin-bottom:12.0pt'><b><font size=2 color=black
face=Tahoma><span lang=EN-US style='font-size:10.0pt;font-family:"Tahoma","sans-serif";
font-weight:bold'>From:</span></font></b><font size=2 face=Tahoma><span
lang=EN-US style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> Karl
Taylor [<a href="mailto:taylor13@llnl.gov">mailto:taylor13@llnl.gov</a>] <br>
<b><span style='font-weight:bold'>Sent:</span></b> 29 October 2010 02:15<br>
<b><span style='font-weight:bold'>To:</span></b> Bentley, Philip<br>
<b><span style='font-weight:bold'>Cc:</span></b> V. Balaji; <a
href="mailto:martin.juckes@stfc.ac.uk">martin.juckes@stfc.ac.uk</a>; <a
href="mailto:go-essp-tech@ucar.edu">go-essp-tech@ucar.edu</a>; <a
href="mailto:cmor@lists.llnl.gov">cmor@lists.llnl.gov</a>; <a
href="mailto:Kyle.Olivo@noaa.gov">Kyle.Olivo@noaa.gov</a>; Doutriaux, Charles;
Kettleborough, Jamie<br>
<b><span style='font-weight:bold'>Subject:</span></b> Re: [Go-essp-tech] CMOR
and cell_measures issues</span></font><o:p></o:p></p>
<p class=MsoNormal><font size=3 color=black
face="Times New Ro"><span style='font-size:12.0pt;
font-family:"Times New Ro","serif"'>Dear all,<br>
<br>
I meant to try to address all the stuff in this discussion, but won't have time
today. This email is just to say that I think we should insist that the
cell_area files (areacella and areacello) be placed in the archive, even if
there are also gridspec files. The ext_cell_measures attribute should
also be included for fields that are on the "standard" grid (i.e.,
the one with the cell areas stored in areacella or areacello). If there
are other fields for which the standard areas are inappropriate and where your
scientists think it is important to provide cell areas, then I recommend that
you create specially named variables and place them in the "fx"
subdirectories. For variables not on the "standard" grid
(i.e., the grid of areacella or areacello), you should "turn off" the
ext_cell_measures attribute.<br>
<br>
I don't expect most groups to produce gridspec files, so most analysts will be
looking for areas in the areacella and areacello variables, not the gridspec
files. This is why you should write the areacella and areacello files
even if you also write the gridspec files.<br>
<br>
Also, could you please explain why you prefer not to duplicate the
"fx" fields in each experiment's directory tree. <br>
<br>
Best regards,<br>
Karl<br>
<br>
On 10/25/10 7:12 AM, Bentley, Philip wrote: </span></font><o:p></o:p></p>
<pre><font size=2 color=black face="Courier New"><span style='font-size:10.0pt'>Hi Balaji,<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'> <o:p></o:p></span></font></pre>
<blockquote style='margin-top:5.0pt;margin-bottom:5.0pt'><pre><font size=2
color=black face="Courier New"><span style='font-size:10.0pt'>Phil, I'm very impressed that Had will have gridspec files, <o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>is this a done deal? I've been so pessimistic about this that <o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>I was wondering if even we should do one ourselves.<o:p></o:p></span></font></pre></blockquote>
<pre><font size=2 color=black face="Courier New"><span style='font-size:10.0pt'>Nope, not a done deal yet :-(<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'> <o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>In line with the CMIP5 expt design doc, we don't really need to provide<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>gridspec files since all our model output is on either regular or<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>uniform grids (i.e. simple cartesian product of lat & long).<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'> <o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>However, this whole cell_measures business prompted me to revisit the<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>gridspec tools and output, which reminded me that the gridspec netcdf<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>files include a cell area variable. Which in turn means we wouldn't need<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>to provide a separate file (or files) for cell areas. Hence we could<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>drop the ext_cell_measures attribute from our CMIP5 output files.<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'> <o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>Using the gridspec tools may be a quick and easy way for us to provide<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>cell area info if we need to.<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'> <o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>Caveat: from a quick glance it looks like the netcdf files produced by<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>the gridspec tools are not CF compliant. Is this is an issue? Presumably<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>it is if we want all the data in the CMIP5 archive to be CF compliant.<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>(NB: it could be I'm not running with the very latest version of the<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>tools - but I couldn't see a more recent version on the gfdl web site).<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'> <o:p></o:p></span></font></pre>
<blockquote style='margin-top:5.0pt;margin-bottom:5.0pt'><pre><font size=2
color=black face="Courier New"><span style='font-size:10.0pt'>You know of course that gridspec says you can supply<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'> <o:p></o:p></span></font></pre>
<blockquote style='margin-top:5.0pt;margin-bottom:5.0pt'><pre><font size=2
color=black face="Courier New"><span style='font-size:10.0pt'>gridspec_fx_HadGEM2-ES_atm_pgrid.nc<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>gridspec_fx_HadGEM2-ES_atm_ugrid.nc<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>gridspec_fx_HadGEM2-ES_atm_vgrid.nc<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>gridspec_fx_HadGEM2-ES_atm_uvgrid.nc<o:p></o:p></span></font></pre></blockquote>
<pre><font size=2 color=black face="Courier New"><span style='font-size:10.0pt'>as one single supergrid...<o:p></o:p></span></font></pre></blockquote>
<pre><font size=2 color=black face="Courier New"><span style='font-size:10.0pt'>If I could figure out how to output all 7 or 8 atm/ocn (sub-)grids to a<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>single netcdf file I would, but the available documentation (e.g. for<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>make_hgrid) isn't clear on this point. Sorry, that's probably just me<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>being dumb! But if there is updated documentation then please point me<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>to it. If necessary I could concatenate variables afterwards using NCO<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>tools.<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'> <o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>Right now I'm trying to figure out how to create a gridspec file for our<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>HadGEM2 ocean model, which uses a stretched (i.e. tartan/plaid) grid:<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>longitudes are evenly spaced, latitudes vary from 1 deg to 1/3 deg.<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>(Looks like I need to use the --my_grid_file option to supply the<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>lat/long coords).<o:p></o:p></span></font></pre>
<blockquote style='margin-top:5.0pt;margin-bottom:5.0pt'><pre><font size=2
color=black face="Courier New"><span style='font-size:10.0pt'>But if you're doing gridspec at all, I will concede this <o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>point:-). Let's both do these separate gridspecs for now.<o:p></o:p></span></font></pre></blockquote>
<pre><font size=2 color=black face="Courier New"><span style='font-size:10.0pt'>Works for me.<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'> <o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>I think we're suffering from 'early-adopter syndrome' :-/<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'> <o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>Cheers,<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>Phil<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'> <o:p></o:p></span></font></pre>
<blockquote style='margin-top:5.0pt;margin-bottom:5.0pt'><pre><font size=2
color=black face="Courier New"><span style='font-size:10.0pt'>Bentley, Philip writes:<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'> <o:p></o:p></span></font></pre>
<blockquote style='margin-top:5.0pt;margin-bottom:5.0pt'><pre><font size=2
color=black face="Courier New"><span style='font-size:10.0pt'>Hi Karl,<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'> <o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>A somewhat belated follow-up question in connection with <o:p></o:p></span></font></pre></blockquote>
<pre><font size=2 color=black face="Courier New"><span style='font-size:10.0pt'>this proposal <o:p></o:p></span></font></pre>
<blockquote style='margin-top:5.0pt;margin-bottom:5.0pt'><pre><font size=2
color=black face="Courier New"><span style='font-size:10.0pt'>(and with some slight overlap with Jamie's email which <o:p></o:p></span></font></pre></blockquote>
<pre><font size=2 color=black face="Courier New"><span style='font-size:10.0pt'>crossed on the <o:p></o:p></span></font></pre>
<blockquote style='margin-top:5.0pt;margin-bottom:5.0pt'><pre><font size=2
color=black face="Courier New"><span style='font-size:10.0pt'>ether)...<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'> <o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>As things stand the files named in the 'associated_files' attribute <o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>appear thus (using our RCP 4.5 simulation as an example):<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'> <o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>"... gridspecFile: gridspec_fx_HadGEM2-ES_rcp45_r0i0p0.nc areacella:<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>areacella_fx_HadGEM2-ES_rcp45_r0i0p0.nc"<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'> <o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>Are the <expt_id>_<rip> parts (i.e. 'rcp45_r0i0p0.nc' ) actually <o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>required? AFAIK, our gridspec/cellarea files will not <o:p></o:p></span></font></pre></blockquote>
<pre><font size=2 color=black face="Courier New"><span style='font-size:10.0pt'>change from one <o:p></o:p></span></font></pre>
<blockquote style='margin-top:5.0pt;margin-bottom:5.0pt'><pre><font size=2
color=black face="Courier New"><span style='font-size:10.0pt'>simulation to the next using the same model (HadGEM2-ES in <o:p></o:p></span></font></pre></blockquote>
<pre><font size=2 color=black face="Courier New"><span style='font-size:10.0pt'>this case).<o:p></o:p></span></font></pre>
<blockquote style='margin-top:5.0pt;margin-bottom:5.0pt'><pre><font size=2
color=black face="Courier New"><span style='font-size:10.0pt'>Since, like most centers, we will be running large numbers of <o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>simulations using the same model, it looks like we would need to <o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>create numerous duplicates of the gridspec/cellarea files - <o:p></o:p></span></font></pre></blockquote>
<pre><font size=2 color=black face="Courier New"><span style='font-size:10.0pt'>or lots of <o:p></o:p></span></font></pre>
<blockquote style='margin-top:5.0pt;margin-bottom:5.0pt'><pre><font size=2
color=black face="Courier New"><span style='font-size:10.0pt'>symlinks<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>- in order to for these references to make sense. Unless you are <o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>planning to manage that on our behalf somehow...?<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'> <o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>I think our 4 gridspec files for the HadGEM2 atm grids are <o:p></o:p></span></font></pre></blockquote>
<pre><font size=2 color=black face="Courier New"><span style='font-size:10.0pt'>likely to <o:p></o:p></span></font></pre>
<blockquote style='margin-top:5.0pt;margin-bottom:5.0pt'><pre><font size=2
color=black face="Courier New"><span style='font-size:10.0pt'>be called something like...<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'> <o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>gridspec_fx_HadGEM2-ES_atm_pgrid.nc<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>gridspec_fx_HadGEM2-ES_atm_ugrid.nc<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>gridspec_fx_HadGEM2-ES_atm_vgrid.nc<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>gridspec_fx_HadGEM2-ES_atm_uvgrid.nc<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'> <o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>So without any simulation-specific info. (There would also be files <o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>for the ocean grids)<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'> <o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>As it happens the gridspec files contain grid cell areas, <o:p></o:p></span></font></pre></blockquote>
<pre><font size=2 color=black face="Courier New"><span style='font-size:10.0pt'>so I'm now <o:p></o:p></span></font></pre>
<blockquote style='margin-top:5.0pt;margin-bottom:5.0pt'><pre><font size=2
color=black face="Courier New"><span style='font-size:10.0pt'>wondering if we'd even supply both?<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'> <o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>I'd be interested to hear your thoughts on this. I may be <o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>mis-understanding something/everything :-)<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'> <o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>Regards<o:p></o:p></span></font></pre><pre><font
size=2 color=black face="Courier New"><span style='font-size:10.0pt'>Phil<o:p></o:p></span></font></pre></blockquote>
</blockquote>
<pre><font size=2 color=black face="Courier New"><span style='font-size:10.0pt'> <o:p></o:p></span></font></pre></blockquote>
</blockquote>
</div>
<p class=MsoNormal><font size=3 color=black face="Times New Roman"><span
style='font-size:12.0pt'><o:p> </o:p></span></font></p>
<p><font size=3 color=black face="Times New Roman"><span style='font-size:12.0pt'>--
<br>
Scanned by iCritical. <o:p></o:p></span></font></p>
<p class=MsoNormal><font size=3 color=black face="Times New Roman"><span
style='font-size:12.0pt'><o:p> </o:p></span></font></p>
</div>
</div>
<br><p>--
<BR>Scanned by iCritical.
</p>
<br></body>
</html>