<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Hello Martin,<br>
<br>
I totally agree as you may already know :-)<br>
<br>
I'd like to add another good point for this: bandwidth.<br>
The archive is too large to just expect people to download
everything all over again just to find out it hasn't change.<br>
Tools are built around this too. By providing a checksum, it's no
need to download something you "know" it hasn't change.<br>
<br>
For us, as archive sites, I don't think we will really archive
anything without a checksum, I know I won't.<br>
<br>
And furthermore, checksums allows us to perform a version diff even
without having any other information,e.g. check the history tab at
WDCC:<br>
Version with added and renamed files:
<a class="moz-txt-link-freetext" href="http://ipcc-ar5.dkrz.de/dataset/cmip5.output1.NCC.NorESM1-M.sstClim.mon.land.Lmon.r1i1p1.html">http://ipcc-ar5.dkrz.de/dataset/cmip5.output1.NCC.NorESM1-M.sstClim.mon.land.Lmon.r1i1p1.html</a><br>
and some deleted files:
<a class="moz-txt-link-freetext" href="http://ipcc-ar5.dkrz.de/dataset/cmip5.output2.MPI-M.MPI-ESM-LR.rcp26.mon.ocean.Omon.r1i1p1.html">http://ipcc-ar5.dkrz.de/dataset/cmip5.output2.MPI-M.MPI-ESM-LR.rcp26.mon.ocean.Omon.r1i1p1.html</a><br>
<br>
This is only possible with checksums, and I think this info would be
quite helpful for users to know.<br>
<br>
Thanks,<br>
Estani<br>
Am 27.10.2011 12:13, schrieb <a class="moz-txt-link-abbreviated" href="mailto:martin.juckes@stfc.ac.uk:">martin.juckes@stfc.ac.uk:</a>
<blockquote
cite="mid:E21FBC3F00D7304687CB46529F9676D71C7F1FAA@EXCHMBX01.fed.cclrc.ac.uk"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=ISO-8859-1">
<meta name="Generator" content="Microsoft Word 12 (filtered
medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div class="WordSection1">
<p class="MsoNormal">Hello,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Yesterday I ran a few tests transferring a
2Gb file from CSIRO to a server at Reading in the UK using
wget over http. I ran the wget command 4 times, and each time
got a file of the correct size and incorrect checksum. Wget
was using multiple automatic retries. I then throttled back
the transfer rate to 400Kbytes/s and got the file transferred
in one go, and with the correct checksum. It just took a
little longer.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">My tentative conclusions are that users
cannot access the data reliably if we do not provide
checksums, and that download scripts which do not verify
checksums are not good enough for an archive of this size,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Cheers,<o:p></o:p></p>
<p class="MsoNormal">Martin<o:p></o:p></p>
</div>
<br>
<p>-- <br>
Scanned by iCritical.
</p>
<br>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
GO-ESSP-TECH mailing list
<a class="moz-txt-link-abbreviated" href="mailto:GO-ESSP-TECH@ucar.edu">GO-ESSP-TECH@ucar.edu</a>
<a class="moz-txt-link-freetext" href="http://mailman.ucar.edu/mailman/listinfo/go-essp-tech">http://mailman.ucar.edu/mailman/listinfo/go-essp-tech</a>
</pre>
</blockquote>
<br>
<br>
<pre class="moz-signature" cols="72">--
Estanislao Gonzalez
Max-Planck-Institut für Meteorologie (MPI-M)
Deutsches Klimarechenzentrum (DKRZ) - German Climate Computing Centre
Room 108 - Bundesstrasse 45a, D-20146 Hamburg, Germany
Phone: +49 (40) 46 00 94-126
E-Mail: <a class="moz-txt-link-abbreviated" href="mailto:gonzalez@dkrz.de">gonzalez@dkrz.de</a> </pre>
</body>
</html>