<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Hi Karl,<br>
<br>
I'm currently testing the replication via wget and got an average of
350 Mbps (around 40MB/s) with 7 parallel streams in the last 14
hours. I've already downloaded 2TB.<br>
<br>
So I guess the problem is across the Atlantic. I'd suggest to start
many more parallel streams.<br>
<br>
In any case I think this might be caused by some problems/congestion
in intermediate networks. The network throughput to ECMWF *might*
not be comparable because of the network architecture, e.g.
intermediate proxies, firewalls, etc. But a 10x speed up might be of
no use anyway as downloading 1TB in 2 Months seems hardly to be an
option...<br>
<br>
When we all have the data replicated worldwide though, it will be
possible to download from all 3 servers at the same time, which will
more than double the speed from this side of the ocean, and will be
a considerable speed up from yours, although probably not as much
(don't nail me on this, I'll deny it vehemently <- this is not my
writing :-)<br>
<br>
Anyway, this being said, the rest should be answered by BADC
directly.<br>
<br>
Thanks,<br>
Estani<br>
<br>
Am 03.03.2011 02:26, schrieb Karl Taylor:<br>
<span style="white-space: pre;">> this should have been sent to
<a class="moz-txt-link-abbreviated" href="mailto:go-essp-tech@ucar.edu">go-essp-tech@ucar.edu</a><br>
> <br>
> sorry about that.<br>
> <br>
> Karl<br>
> <br>
> -------- Original Message -------- Subject: download rates
Date:<br>
> Wed, 2 Mar 2011 16:43:42 -0800 From: Karl Taylor<br>
> <a class="moz-txt-link-rfc2396E" href="mailto:taylor13@llnl.gov"><taylor13@llnl.gov></a> To: <a class="moz-txt-link-abbreviated" href="mailto:esg-support@earthsystemgrid.org">esg-support@earthsystemgrid.org</a>
Grid<br>
> <a class="moz-txt-link-rfc2396E" href="mailto:esg-support@earthsystemgrid.org"><esg-support@earthsystemgrid.org></a><br>
> <br>
> <br>
> <br>
> Hi all,<br>
> <br>
> When I download a 25 MB CMIP5 file from BADC using the wget
method: *<br>
> it took 35 sec to get started * then it downloaded at an
average rate<br>
> of 1.1 MB/min<br>
> <br>
> When I download the same file using the point and click
method: * it<br>
> took 20 sec to get started * then it downloaded at an average
rate of<br>
> 1.4 MB/min<br>
> <br>
> When I download 4 files simultaneously (in parallel) using
the click<br>
> method, I get somewhat slower download rate per file, so
somewhat<br>
> less than 4 times the data transfer rate I would have gotten<br>
> downloading them in series.<br>
> <br>
> Note that at 1 MB/min, I could download 1 GB in about 17
hours or 1<br>
> TB (i.e., about 1/1000 of the entire archive) in about 2
years.<br>
> <br>
> Any way to speed this up some? (A colleague here downloads
files at<br>
> about 10 times this rate from ECMWF.)<br>
> <br>
> Note that I haven't checked how fast download rates are from
nodes<br>
> published to our (PCMDI's) gateway (since they are not yet
publicly<br>
> available).<br>
> <br>
> Best regards, Karl<br>
> <br>
> <br>
> <br>
> <br>
> _______________________________________________ GO-ESSP-TECH
mailing<br>
> list <a class="moz-txt-link-abbreviated" href="mailto:GO-ESSP-TECH@ucar.edu">GO-ESSP-TECH@ucar.edu</a> <br>
> <a class="moz-txt-link-freetext" href="http://mailman.ucar.edu/mailman/listinfo/go-essp-tech">http://mailman.ucar.edu/mailman/listinfo/go-essp-tech</a></span><br>
<br>
<br>
-- <br>
Estanislao Gonzalez<br>
<br>
Max-Planck-Institut für Meteorologie (MPI-M)<br>
Deutsches Klimarechenzentrum (DKRZ) - German Climate Computing
Centre<br>
Room 108 - Bundesstrasse 45a, D-20146 Hamburg, Germany<br>
<br>
Phone: +49 (40) 46 00 94-126<br>
E-Mail: <a class="moz-txt-link-abbreviated" href="mailto:estanislao.gonzalez@zmaw.de">estanislao.gonzalez@zmaw.de</a> <br>
</body>
</html>