<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 12 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";
color:black;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body bgcolor="white" lang="EN-GB" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Hi Karl,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Sorry I missed your February email.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Extending the use of the "-suffix" part of temporal subset to include averaging looks reasonable. The geographic subset section is rather complex and I worry
that it will be difficult to implement unambiguous parsers for it. This may not matter provided we can always interpret it as an opaque string in filenames of the form: "c1_c2_...cn_[temporal-subset]_[geospatial-info].nc". My specific concerns about parsing
are below. <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Also, more generally, I wonder whether we are repeating too much information from the CF metadata in the filename. I think the temporal subset is already
pushing to the limit what can be effectively represent in a filename and this could push it too far. Filenames within a dataset should be unique but maybe we could let data providers decide how they are labelled?
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">If we continue to add detailed syntax to the filename it would greatly help to have a formal grammar in BNF notation (http://en.wikipedia.org/wiki/Backus%E2%80%93Naur_Form).<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Section 2.4 Geographic subsets<o:p></o:p></span></b></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">As described the format is "g[-XXXX][-YYYY]" where both XXXX and YYYY are optional and YYYY = "[yyy][-zzz]". XXXX can be omitted when YYYY is present as in
the example "g-ocn-areaavg".<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">I foresee problems in writing parsers that disambiguate the case "g-XXXX" from "g-YYYY" particularly in the case where XXXX is a named region. If we wanted
to extend the valid vocabulary of YYYY we would have to check for clashes with all named regions used in XXXX. This would seam like a hostage to fortune, particularly if users start defining their own regions.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Similarly how do we disambiguate these cases:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">g-XXXX-yyy<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">g-yyy-zzz<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">With a sufficiently complex parser we can differentiate these because yyy and zzz are from controlled vocabularies but writing a generic parser that forsees
extensions to these vocabularies will be tricky and error-prone. <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:Consolas;color:#1F497D">---<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:Consolas;color:#1F497D">Stephen Pascoe +44 (0)1235 445980<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:Consolas;color:#1F497D">Centre of Environmental Data Archival<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:Consolas;color:#1F497D">STFC Rutherford Appleton Laboratory, Harwell Oxford, Didcot OX11 0QX, UK<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext">From:</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext"> Karl Taylor [mailto:taylor13@llnl.gov]
<br>
<b>Sent:</b> 06 June 2012 21:58<br>
<b>To:</b> Kettleborough, Jamie; V. Balaji; Steve Hankin; Juckes, Martin (STFC,RAL,RALSP); Lawrence, Bryan (STFC,RAL,RALSP); Pascoe, Stephen (STFC,RAL,RALSP); go-essp-tech@ucar.edu<br>
<b>Subject:</b> Re: [Go-essp-tech] DRS corrections and extensions<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Dear all,<br>
<br>
In February I asked for comments on my proposal to extend the DRS to include information about spatio-temporal subsets or means. I heard from Jamie, but no one else. I respond to Jamie below, but I also would like your input specifically about:<br>
<br>
1. Is this method of describing spatio-temporal subsets acceptable?<br>
2. Is it worth taking this step if we don't say anything about other "processed" output? For example how to describe "regridded" data or multi-model means.<br>
<br>
I've attached the proposed version of the DRS, which differs from the one I sent in January only in a couple mods made in response to Jamie.<br>
<br>
Best regards,<br>
Karl<br>
<br>
On 2/13/12 6:47 AM, Kettleborough, Jamie wrote: <o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:blue">Hello Karl,</span><o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:blue">this will be terse as I have time to review, but not to necessarily get the words right - hope I don't say anything too bad because of this.</span><o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:blue">1. section 2.3, Not sure 'output' should be mentioned under 'product'. I don't think 'output' ever makes it to publication level, so does not need to appear in
a publication level id. I know cmor produces it, but I think that's kind of historical isn't it, rather than necessary? Maybe its too late for details like this?</span><o:p></o:p></p>
<p class="MsoNormal">It's true that in the end the CMIP5 output should not remain as "output", but be assigned to "output1" or "output2". Nevertheless, I don't think there is any harm in keeping it in the DRS.
<br>
<br>
<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:blue">2. section 2.3 version number: to be consistent with what we really have in CMIP5 I think you need to note that v1, v2 are also present, though any *new* versions
should use vYYYYMMDD.</span><o:p></o:p></p>
<p class="MsoNormal">I have modified the text to indicate that software cannot rely on the version number reflecting a date.<br>
<br>
<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:blue">3. section 2.3 version: I wonder if you need to say more (maybe not here, but if not where?) about what triggers a new version. I think its
</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:blue"> a. anything that changes the content of a file already published and</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:blue"> b. the addition or deletion of files from any publication data set. </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:blue"> Pure 'data management' meta data changes (addition of checksums, move to new URL's) need not trigger a new version.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:blue"> Do you also need to say there is no guarantee that old versions will be kept (unless they have a DOI).</span><o:p></o:p></p>
<p class="MsoNormal">I've added some of this information now to the document.<br>
<br>
<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:blue">4. section 2.4 Temporal Subsets or means: I don't understand the 'avg' example, or if I do I don't know if its right (but the point is relatively minor). I think
the example you quote as one 6 month mean field in it. This is based on 1 day means. I think its a little anomalous to keep the frequency as 'day' in this case. That's not quite consistent with the definition (and I think all other uses) of frequency.
Strictly speaking frequency should be 6mon no? (I may have misunderstood).</span><o:p></o:p></p>
<p class="MsoNormal">I think you're right. I'm not sure why I thought this was the right way to do it. I've changed the example,
<br>
<br>
<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:blue">5. section 3.5. Does this need clarifying? I think the current wording is potentially confusing, I think it should say something like:</span><o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:blue">'URLs referencing the data files will have a site dependent prefix (that may change due to site-specific data management tasks) followed by the directory structure.
This directory structure should (but may not) follow the recommendations of section 3.3'</span><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I've modified the text as suggested.<br>
<br>
<o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:blue">6. I've noticed that the thredds catalogs also expose a thing called the file_id, e.g</span><o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:blue"><property name="file_id" value="cmip5.output1.CNRM-CERFACS.CNRM-CM5.rcp45.mon.ocean.Omon.r1i1p1.vo_Omon_CNRM-CM5_rcp45_r1i1p1_203601-204512.nc"/></span><o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:blue">I don't know if they need a mention as being anything important (we don't use them as they don't give any version info).</span><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">We've already given 5 use cases, which I think is enough. The DRS is used in a number of other ways.<br>
<br>
<o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:blue">Hope this is useful,</span><o:p></o:p></p>
<p class="MsoNormal">Yes thanks very much!<br>
Karl<br>
<br>
<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:blue">Jamie</span><o:p></o:p></p>
<blockquote style="border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt;margin-left:3.75pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<p class="MsoNormal"><o:p> </o:p></p>
<div class="MsoNormal" align="center" style="text-align:center"><span lang="EN-US">
<hr size="2" width="100%" align="center">
</span></div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">
<a href="mailto:go-essp-tech-bounces@ucar.edu">go-essp-tech-bounces@ucar.edu</a> [<a href="mailto:go-essp-tech-bounces@ucar.edu">mailto:go-essp-tech-bounces@ucar.edu</a>]
<b>On Behalf Of </b>Karl Taylor<br>
<b>Sent:</b> 10 February 2012 01:32<br>
<b>To:</b> V. Balaji; Steve Hankin; Martin Juckes; Bryan Lawrence; Stephen Pascoe;
<a href="mailto:go-essp-tech@ucar.edu">go-essp-tech@ucar.edu</a><br>
<b>Subject:</b> [Go-essp-tech] DRS corrections and extensions</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal">Dear all,<br>
<br>
Attached is my attempt to make the DRS consistent with CMIP5 (in describing the precision of "time instants"), but primarily to extend it to a more complete treatment of spatio-temporal subsets or means. I've also corrected a few typos.<br>
<br>
Comments most welcome. In particular could someone recheck sections 3.3-3.5 (which haven't been changed by me) to see if they remain consistent with CMIP5?<br>
<br>
thanks and best regards,<br>
Karl<o:p></o:p></p>
</blockquote>
</div>
<br><p>--
<BR>Scanned by iCritical.
</p>
<br></body>
</html>