<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
I ran into this problem even with the 0.5 data. I was able to get around it using mget rather than wget or curl. I have a bash script that generates filenames for all need and then I make a simple call</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<br>
</div>
<div>
<p style="color: rgb(0, 0, 0); font-family: Monaco; font-size: 10px; margin: 0px;">
mget $gribfile1 $gribfile2 $gribfile3 $gribfile4 $gribfile5 $gribfile6 $gribfile</p>
<p style="color: rgb(0, 0, 0); font-family: Monaco; font-size: 10px; margin: 0px;">
7 $gribfile8 $gribfile9 $gribfile10 $gribfile11 $gribfile12 $gribfile13 $gribfil</p>
<p style="color: rgb(0, 0, 0); font-family: Monaco; font-size: 10px; margin: 0px;">
e14 $gribfile15 $gribfile16 $gribfile17</p>
<p style="color: rgb(0, 0, 0); font-family: Monaco; font-size: 10px; margin: 0px;">
<br>
</p>
<p style="margin: 0px;"><font face="Calibri,sans-serif" size="3"> I’ve never been able to find out why wget and curl don’t reliably work, but mget seems to always work – error recovery perhaps. Good luck.</font></p>
<p style="margin: 0px;"><font face="Calibri,sans-serif" size="3"><br>
</font></p>
<p style="margin: 0px;"><font face="Calibri,sans-serif" size="3">D</font><font face="Calibri,sans-serif" size="3">ave</font></p>
<p style="margin: 0px;"><font face="Calibri,sans-serif" size="3"><br>
</font></p>
</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<br>
</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<div><font class="Apple-style-span" color="rgb(0, 0, 0)"><font class="Apple-style-span" face="Calibri">Dr. David J Smith</font></font></div>
<div><font class="Apple-style-span" color="rgb(0, 0, 0)"><font class="Apple-style-span" face="Calibri">College of Science and Mathematics</font></font></div>
<div><font class="Apple-style-span" color="rgb(0, 0, 0)"><font class="Apple-style-span" face="Calibri">University of the Virgin Islands</font></font></div>
<div><font class="Apple-style-span" color="rgb(0, 0, 0)"><font class="Apple-style-span" face="Calibri">St. Thomas, VI 00802</font></font></div>
<div><font class="Apple-style-span" color="rgb(0, 0, 0)"><font class="Apple-style-span" face="Calibri">340 693-1257</font></font></div>
<div><font class="Apple-style-span" color="rgb(0, 0, 0)"><font class="Apple-style-span" face="Calibri">dsmith@uvi.edu</font></font></div>
</div>
</div>
<div style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<br>
</div>
<span id="OLK_SRC_BODY_SECTION" style="color: rgb(0, 0, 0); font-family: Calibri, sans-serif; font-size: 14px;">
<div style="font-family:Calibri; font-size:11pt; text-align:left; color:black; BORDER-BOTTOM: medium none; BORDER-LEFT: medium none; PADDING-BOTTOM: 0in; PADDING-LEFT: 0in; PADDING-RIGHT: 0in; BORDER-TOP: #b5c4df 1pt solid; BORDER-RIGHT: medium none; PADDING-TOP: 3pt">
<span style="font-weight:bold">From: </span>Micha Silver <<a href="mailto:micha@arava.co.il">micha@arava.co.il</a>><br>
<span style="font-weight:bold">Date: </span>Sat, 24 Oct 2015 14:53:58 +0300<br>
<span style="font-weight:bold">To: </span>Bart Brashers <<a href="mailto:bbrashers@ramboll.com">bbrashers@ramboll.com</a>><br>
<span style="font-weight:bold">Cc: </span><<a href="mailto:Wrf-users@ucar.edu">Wrf-users@ucar.edu</a>>, erick fredj <<a href="mailto:erick.fredj@gmail.com">erick.fredj@gmail.com</a>><br>
<span style="font-weight:bold">Subject: </span>Re: [Wrf-users] GFS file size problem<br>
</div>
<div><br>
</div>
<blockquote id="MAC_OUTLOOK_ATTRIBUTION_BLOCKQUOTE" style="BORDER-LEFT: #b5c4df 5 solid; PADDING:0 0 0 5; MARGIN:0 0 0 5;">
<div style="direction: ltr;"><style type="text/css">body p { margin-bottom: 0cm; margin-top: 0pt; } </style>
<div style="direction: ltr;" bidimailui-charset-is-forced="true" smarttemplateinserted="true" text="#000000" bgcolor="#FFFFFF">
<br>
Hello Bart:<br>
Thanks<br>
It seems with "curl -l" I am getting only the file names, not sizes.<br>
??<br>
Best,<br>
Micha<br>
<br>
<div class="moz-cite-prefix">On 23/10/15 20:51, Bart Brashers wrote:<br>
</div>
<blockquote cite="mid:0A8C91C760228A498C126E51126EDB750192BFDE3D@WCEXS2.ENVIRONCORP.INT" type="cite">
<meta name="Generator" content="Microsoft Word 15 (filtered
medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Verdana;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;
color:black;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman",serif;
color:black;}
p.computer, li.computer, div.computer
{mso-style-name:computer;
margin:0in;
margin-bottom:.0001pt;
font-size:9.0pt;
font-family:"Courier New";
color:black;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);">Does “curl -l
<a class="moz-txt-link-freetext" href="http://...”">http://...”</a> return the same filesizes as wget? Curl might be more honest…<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);"><o:p> </o:p></span></p>
<p class="MsoNormal" style="line-height:11.0pt"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(50, 62, 79);" lang="SV">Bart<o:p></o:p></span></p>
<p class="MsoNormal" style="line-height:11.0pt"><span style="font-size:8.0pt;color:#1F497D" lang="SV">________________________________________<o:p></o:p></span></p>
<p class="MsoNormal" style="line-height:11.0pt"><b><span style="font-size: 9pt; font-family: Verdana, sans-serif;" lang="SV">Bart Brashers,
</span></b><span style="font-size: 8pt; font-family: Verdana, sans-serif;" lang="SV">Ph.D.</span><b><span style="font-size: 9pt; font-family: Verdana, sans-serif;" lang="SV"><o:p></o:p></span></b></p>
<p class="MsoNormal" style="line-height:11.0pt"><span style="font-size: 8pt; font-family: Verdana, sans-serif;">Senior Manager, Air Quality Meteorology<o:p></o:p></span></p>
<p class="MsoNormal" style="line-height:11.0pt"><span style="font-size: 8pt; font-family: Verdana, sans-serif;"><o:p> </o:p></span></p>
<p class="MsoNormal" style="line-height:11.0pt"><span style="font-size: 8pt; font-family: Verdana, sans-serif;">D +1 425 4121812<o:p></o:p></span></p>
<p class="MsoNormal" style="line-height:11.0pt"><span style="font-size: 8pt; font-family: Verdana, sans-serif;">M +1 206 5502606<o:p></o:p></span></p>
<p class="MsoNormal" style="line-height:11.0pt"><span style="font-size: 8pt; font-family: Verdana, sans-serif; color: rgb(0, 158, 227);" lang="EN-GB"><a moz-do-not-send="true" href="mailto:bbrashers@ramboll.com"><span style="color:#0563C1">bbrashers@ramboll.com</span></a>
</span><span style="font-size: 8pt; font-family: Verdana, sans-serif; color: red;" lang="EN-GB">-- Please note my new email address</span><span style="font-size: 8pt; font-family: Verdana, sans-serif;"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:1.0in"><span style="font-size: 9pt; font-family: Verdana, sans-serif; color: red;"><o:p> </o:p></span></p>
<p class="MsoNormal" style="line-height:11.0pt"><span style="font-size: 8pt; font-family: Verdana, sans-serif;">Ramboll Environ<o:p></o:p></span></p>
<p class="MsoNormal" style="line-height:11.0pt"><span style="font-size: 8pt; font-family: Verdana, sans-serif;">19020 33rd Avenue West<br>
Suite 310<br>
Lynnwood, WA 98036<br>
USA</span><span style="font-size:8.0pt;color:#1F497D" lang="EN-GB"><o:p></o:p></span></p>
<p class="MsoNormal" style="line-height:11.0pt"><span style="font-size: 8pt; font-family: Verdana, sans-serif; color: rgb(0, 158, 227);"><a moz-do-not-send="true" href="http://www.ramboll-environ.com/" title="www.ramboll-environ.com"><span style="color:#009EE3" lang="EN-GB"></span></a><a class="moz-txt-link-abbreviated" href="http://www.ramboll-environ.com">www.ramboll-environ.com</a></span><span style="font-size: 8pt; font-family: Verdana, sans-serif;" lang="EN-GB"><o:p></o:p></span></p>
<p class="MsoNormal" style="line-height:11.0pt"><span style="font-size: 8pt; font-family: Verdana, sans-serif;"><o:p> </o:p></span></p>
<p class="MsoNormal" style="line-height:11.0pt"><span style="font-size: 8pt; font-family: Verdana, sans-serif;"><img id="Picture_x0020_1" src="cid:part3.09030102.00070800@arava.co.il" alt="Ramboll_Environ_Logo" height="19" width="152" border="0"></span><span style="font-size: 8pt; font-family: Verdana, sans-serif;"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: rgb(31, 73, 125);"><o:p> </o:p></span></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0in
0in 0in 4.0pt">
<div>
<div style="border:none;border-top:solid #E1E1E1
1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: windowtext;">From:</span></b><span style="font-size: 11pt; font-family: Calibri, sans-serif; color: windowtext;">
<a class="moz-txt-link-abbreviated" href="mailto:wrf-users-bounces@ucar.edu">wrf-users-bounces@ucar.edu</a> [<a class="moz-txt-link-freetext" href="mailto:wrf-users-bounces@ucar.edu">mailto:wrf-users-bounces@ucar.edu</a>]
<b>On Behalf Of </b>Micha Silver<br>
<b>Sent:</b> Friday, October 23, 2015 10:29 AM<br>
<b>To:</b> <a class="moz-txt-link-abbreviated" href="mailto:Wrf-users@ucar.edu">Wrf-users@ucar.edu</a><br>
<b>Subject:</b> [Wrf-users] GFS file size problem<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Hello all:<br>
<br>
We are running WRF-Hydro in an operational flood forecasting system. We download GFS data twice daily from
<a moz-do-not-send="true" href="http://www.ftp.ncep.noaa.gov/data/nccf/com/gfs/prod/">
http://www.ftp.ncep.noaa.gov/data/nccf/com/gfs/prod/</a> using wget. In order to be sure that we are getting the full data files, we first get the file sizes with the wget options: --spider --server-response. We save the file sizes before download, then compare
the sizes after download. If there's any differences, we restart the download.<br>
<br>
This mostly works well. When downloading the 0.5 degree data it works perfectly. However when getting the 0.25 data we have noticed that occasionally the sizes reported by the NCEP server appear to not be the actual file sizes. This mismatch does not occur
consistently, but rather only about once in two weeks or so. When this happens, then even waiting a few hours, and restarting the simulation run does not help. Our wget --spider and comparison of downloaded file sizes fails again. Then at the next GFS cycle
everything goes back to normal.<br>
<br>
Has anyone seen this behavior of wget receiving incorrect file sizes? Is there some other recommended way to ensure that the downloaded GFS data are the final, complete files.<br>
<br>
many thanks,<br>
Micha <o:p></o:p></p>
</div>
</div>
<br>
<hr size="1" color="#004250">
This message contains information that may be confidential, privileged or otherwise protected by law from disclosure. It is intended for the exclusive use of the Addressee(s). Unless you are the addressee or authorized agent of the addressee, you may not review,
copy, distribute or disclose to anyone the message or any information contained within. If you have received this message in error, please contact the sender by electronic reply to
<a class="moz-txt-link-abbreviated" href="mailto:email@ramboll.com">email@ramboll.com</a> and immediately delete all copies of the message.
<br>
<br>
This mail was received via Mail-SeCure System.<br>
</blockquote>
<br>
</div>
</div>
_______________________________________________ Wrf-users mailing list <a href="mailto:Wrf-users@ucar.edu">
Wrf-users@ucar.edu</a> <a href="http://mailman.ucar.edu/mailman/listinfo/wrf-users">
http://mailman.ucar.edu/mailman/listinfo/wrf-users</a> </blockquote>
</span>
</body>
</html>