<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<meta name="Generator" content="Microsoft Word 12 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:Helvetica;
panose-1:2 11 6 4 2 2 2 2 2 4;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";
color:black;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0cm;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";
color:black;}
span.apple-style-span
{mso-style-name:apple-style-span;}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:Consolas;
color:black;}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body bgcolor="white" lang="EN-GB" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Hi all,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">This subject is full of gray areas :-(. I would say keeping the tracking_id the same is ok as it is an indication that the contents of the NetCDF hasn't changed.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Practical matters for CMIP5 aside, I've been thinking about how we could create an unambiguous manifest of a dataset-version. I.e. containing enough information
to uniquely identify it's contents without any extraneous information that might change with dataset location, available services, etc. . I came to the conclusion there are 2 possible solutions: either a it's a sorted list of (filename, checksum) pairs or
it's just a sorted list of checksums. The difference is whether filenames are "part of the dataset". My instinct is that you can't decouple filenames from the dataset. Users expect filenames to be meaningful and in some contexts information inside files
could refer to filenames within the dataset (e.g. gridspec files). This is how every other contents-based versioning/packaging system I know of works: git, BagIt, BitTorrent<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">So, that's a long way of saying a new version would be necessary, on theoretical grounds as well as pragmatic ones.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Cheers,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Stephen.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:Consolas;color:#1F497D">---<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:Consolas;color:#1F497D">Stephen Pascoe +44 (0)1235 445980<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:Consolas;color:#1F497D">Centre of Environmental Data Archival<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:Consolas;color:#1F497D">STFC Rutherford Appleton Laboratory, Harwell Oxford, Didcot OX11 0QX, UK<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext">From:</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext"> go-essp-tech-bounces@ucar.edu
[mailto:go-essp-tech-bounces@ucar.edu] <b>On Behalf Of </b>Kettleborough, Jamie<br>
<b>Sent:</b> 15 February 2012 10:20<br>
<b>To:</b> Estanislao Gonzalez; go-essp-tech@ucar.edu<br>
<b>Subject:</b> Re: [Go-essp-tech] Incorrect file names?<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:blue">Thanks Estani,</span><o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:blue">do you have any thoughts on the tracking_id? Should this be left as is (I think what you say below implies it should).</span><o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:blue">Jamie</span><o:p></o:p></p>
<blockquote style="border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt;margin-left:3.75pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<p class="MsoNormal"><o:p> </o:p></p>
<div class="MsoNormal" align="center" style="text-align:center"><span lang="EN-US">
<hr size="2" width="100%" align="center">
</span></div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> go-essp-tech-bounces@ucar.edu [mailto:go-essp-tech-bounces@ucar.edu]
<b>On Behalf Of </b>Estanislao Gonzalez<br>
<b>Sent:</b> 15 February 2012 10:06<br>
<b>To:</b> go-essp-tech@ucar.edu<br>
<b>Subject:</b> Re: [Go-essp-tech] Incorrect file names?</span><span lang="EN-US"><o:p></o:p></span></p>
<p class="MsoNormal">Hi,<br>
<br>
it's required a new version to be published, so that its publication will signal that something has changed. If not, then it won't be picked up by other services, e.g. replica services, as it would assume the data hasn't been changed at all.<br>
In our particular case (DKRZ) we will see files haven't been changed, provided that the checksums are properly published, and will just link them to the older ones, allowing users to get both versions (i.e. users that are already downloading files will be able
to keep downloading them, and not have to start everything anew because of this renaming)<br>
(I can't see if the checksums are there because the node is not accessible at this time)<br>
<br>
We could use that information to infer what happened and display it under the history information.<br>
<br>
Maintaining the same version provides no benefit for the publisher at all and creates the same confusion to the user (which will see that files are missing).<br>
<br>
Just my 2c,<br>
Estani<br>
<br>
Am 15.02.2012 10:29, schrieb Kettleborough, Jamie: <o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:blue">Hello,</span><o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:blue">sorry, a but of a side track, but maybe useful. I know this is an unusual case - but it is another example of an understandable slip that can be made when producing
data. When Laura republishes these should it be under a new publication data set version or not? I think the only thing that is changing is the filename - is that right? I don't think this warrants a new publication data set version, but could be wrong.</span><o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:blue">Jamie</span><o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="2" width="100%" align="center">
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">
<a href="mailto:go-essp-tech-bounces@ucar.edu">go-essp-tech-bounces@ucar.edu</a> [<a href="mailto:go-essp-tech-bounces@ucar.edu">mailto:go-essp-tech-bounces@ucar.edu</a>]
<b>On Behalf Of </b>Laura Carriere<br>
<b>Sent:</b> 14 February 2012 19:43<br>
<b>To:</b> Jennifer Adams<br>
<b>Cc:</b> <a href="mailto:go-essp-tech@ucar.edu">go-essp-tech@ucar.edu</a><br>
<b>Subject:</b> Re: [Go-essp-tech] Incorrect file names?</span><o:p></o:p></p>
<blockquote style="border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt;margin-left:3.75pt;margin-top:5.0pt;margin-right:0cm;margin-bottom:5.0pt">
<p class="MsoNormal"><br>
Ah, now I see your reply. Since you have a solution to your immediate problem, I will not rush the republish but will have it done as soon as it's convenient. Thanks.<br>
<br>
Laura.<br>
<br>
On 2/14/2012 2:32 PM, Jennifer Adams wrote: <o:p></o:p></p>
<p class="MsoNormal">Oh dear. Rather than renaming the files, I used a set of symlinks to solve my immediate problem, but I still find this a bit troubling. I will have to check all GEOS-5 data files I grab from now on. I asked Larry Marx to check if any of
COLA's CMIP5 data at NASA have 8-digit date stamps, and he found everything to be correct with only YYYYMM date strings.
<o:p></o:p></p>
<div>
<p class="MsoNormal">--Jennifer<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">On Feb 14, 2012, at 1:29 PM, Laura Carriere wrote:<o:p></o:p></p>
</div>
<p class="MsoNormal"><br>
<br>
<o:p></o:p></p>
<div>
<p class="MsoNormal"><br>
Quick answer on my way to a meeting - CMOR2 was used for this and at least one other dataset that we have (from COLA) that also has the yyyymmdd format. I'll ask a few other questions after my meeting but that's the short answer.<br>
<br>
Laura.<br>
<br>
On 2/14/2012 1:21 PM, Karl Taylor wrote: <o:p></o:p></p>
<p class="MsoNormal">Dear Novice (with clearly more knowledge than most so-called experts),<br>
<br>
I'm copying a contact for the GEOS-5 model who may be able to provide some information on this. I can't explain why the monthly file names are inconsistent with what CMOR2 puts out. Maybe CMOR2 wasn't used. The DRS document doesn't absolutely forbid including
more precision than necessary in specifying the time-periods, so I don't think we can force them to rename their files. That being said, my hope was everyone would use CMOR, so the file names would all follow the same template.<br>
<br>
Karl<br>
<br>
On 2/13/12 9:30 AM, Jennifer Adams wrote: <o:p></o:p></p>
<div>
<p class="MsoNormal">Dear Experts, <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Here is a dataset:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><a href="http://pcmdi3.llnl.gov/esgcet/dataset/cmip5.output1.NASA-GMAO.GEOS-5.decadal1960.mon.atmos.Amon.r1i1p1.html">http://pcmdi3.llnl.gov/esgcet/dataset/cmip5.output1.NASA-GMAO.GEOS-5.decadal1960.mon.atmos.Amon.r1i1p1.html</a><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">And here is the file name template for all the variables in this dataset: <o:p></o:p></p>
</div>
<div>
<div>
<p class="MsoNormal"><varname>_Amon_GEOS-5_decadal1960_r1i1p1_19610116-19701216.nc <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">My script to generate a GrADS descriptor for this file barked because the MONTHLY data file has time stamps in the YYYYMMDD format. <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">If I have read the DRS document correctly, this a not a correct file name. <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Shouldn't I be able to assume that monthly files will have only YYYYMM date strings? <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">--Jennifer<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt">--<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt">Jennifer M. Adams<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt">IGES/COLA<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt">4041 Powder Mill Road, Suite 302<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt">Calverton, MD 20705<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt"><a href="mailto:jma@cola.iges.org">jma@cola.iges.org</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt"><o:p> </o:p></span></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal"><br>
<br>
<br>
<o:p></o:p></p>
<pre>-- <o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre> Laura Carriere, SAIC <a href="mailto:laura.carriere@nasa.gov">laura.carriere@nasa.gov</a><o:p></o:p></pre>
<pre> NCCS, Code 606.2 301 614-5064<o:p></o:p></pre>
</div>
<p class="MsoNormal">_______________________________________________<br>
GO-ESSP-TECH mailing list<br>
<a href="mailto:GO-ESSP-TECH@ucar.edu">GO-ESSP-TECH@ucar.edu</a><br>
<a href="http://mailman.ucar.edu/mailman/listinfo/go-essp-tech">http://mailman.ucar.edu/mailman/listinfo/go-essp-tech</a><o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Helvetica","sans-serif"">--<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Helvetica","sans-serif"">Jennifer M. Adams<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Helvetica","sans-serif"">IGES/COLA<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Helvetica","sans-serif"">4041 Powder Mill Road, Suite 302<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Helvetica","sans-serif"">Calverton, MD 20705<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Helvetica","sans-serif""><a href="mailto:jma@cola.iges.org">jma@cola.iges.org</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Helvetica","sans-serif""><o:p> </o:p></span></p>
</div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Helvetica","sans-serif""><br>
<br>
</span><o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</div>
<p class="MsoNormal"><br>
<br>
<br>
<o:p></o:p></p>
<pre>-- <o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre> Laura Carriere, SAIC <a href="mailto:laura.carriere@nasa.gov">laura.carriere@nasa.gov</a><o:p></o:p></pre>
<pre> NCCS, Code 606.2 301 614-5064<o:p></o:p></pre>
</blockquote>
<p class="MsoNormal"><br>
<br>
<br>
<o:p></o:p></p>
<pre>_______________________________________________<o:p></o:p></pre>
<pre>GO-ESSP-TECH mailing list<o:p></o:p></pre>
<pre><a href="mailto:GO-ESSP-TECH@ucar.edu">GO-ESSP-TECH@ucar.edu</a><o:p></o:p></pre>
<pre><a href="http://mailman.ucar.edu/mailman/listinfo/go-essp-tech">http://mailman.ucar.edu/mailman/listinfo/go-essp-tech</a><o:p></o:p></pre>
<p class="MsoNormal"><br>
<br>
<br>
<o:p></o:p></p>
<pre>-- <o:p></o:p></pre>
<pre>Estanislao Gonzalez<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>Max-Planck-Institut für Meteorologie (MPI-M)<o:p></o:p></pre>
<pre>Deutsches Klimarechenzentrum (DKRZ) - German Climate Computing Centre<o:p></o:p></pre>
<pre>Room 108 - Bundesstrasse 45a, D-20146 Hamburg, Germany<o:p></o:p></pre>
<pre><o:p> </o:p></pre>
<pre>Phone: +49 (40) 46 00 94-126<o:p></o:p></pre>
<pre>E-Mail: <a href="mailto:gonzalez@dkrz.de">gonzalez@dkrz.de</a> <o:p></o:p></pre>
</blockquote>
</div>
<br><p>--
<BR>Scanned by iCritical.
</p>
<br></body>
</html>