<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#000000">
Hi Rachana,<br>
<br>
yes the the only data that can be downloaded at the moment is the
one published at BADC and is http only... <br>
well actually that I think it again, if the user sends a certificate
it means it got redirected to an SSL end point, right? Is he then
redirected back to the original or is the connection being made via
SSL? I'm not sure now...<br>
<br>
Thanks,<br>
Estani<br>
<br>
Am 03.03.2011 17:52, schrieb Rachana Ananthakrishnan:
<blockquote
cite="mid:0153FEE9-2276-4047-A134-9FFCF7C6723F@mcs.anl.gov"
type="cite">Estani,
<div><br>
</div>
<div>Are all these numbers using a HTTP based download?</div>
<div><br>
</div>
<div>Rachana</div>
<div><br>
</div>
<div>
<div>
<div>On Mar 3, 2011, at 5:13 AM, Estanislao Gonzalez wrote:</div>
<br class="Apple-interchange-newline">
<blockquote type="cite">
<div bgcolor="#ffffff" text="#000000"> Hi,<br>
<br>
by Michael suggestion I started a page in the <a
moz-do-not-send="true" href="http://esgf.org">esgf.org</a>
wiki were we could write our measurement regarding CMIP5
federation.<br>
It is not intended for the technical staff, so don't
complain about it being too simplified :-)<br>
On the contrary, it is intended to be a very first and
rough approximation to what's happening in the cloud by
allowing everyone in 5 sec to write a measurement down.<br>
<br>
Everyone has write access to this and if we see the
necessity we could evolve it into something more detailed,
but which will certainly leave the lay users outside.<br>
<br>
And the link:<br>
<a moz-do-not-send="true" class="moz-txt-link-freetext"
href="http://esgf.org/wiki/Cmip5Status/Datarates">http://esgf.org/wiki/Cmip5Status/Datarates</a><br>
<br>
Does someone else find this useful? Any comments?<br>
<br>
Thanks,<br>
Estani<br>
<br>
Am 03.03.2011 09:04, schrieb Estanislao Gonzalez:
<blockquote cite="mid:4D6F4B8A.4080509@zmaw.de"
type="cite"> Hi Karl,<br>
<br>
I'm currently testing the replication via wget and got
an average of 350 Mbps (around 40MB/s) with 7 parallel
streams in the last 14 hours. I've already downloaded
2TB.<br>
<br>
So I guess the problem is across the Atlantic. I'd
suggest to start many more parallel streams.<br>
<br>
In any case I think this might be caused by some
problems/congestion in intermediate networks. The
network throughput to ECMWF *might* not be comparable
because of the network architecture, e.g. intermediate
proxies, firewalls, etc. But a 10x speed up might be of
no use anyway as downloading 1TB in 2 Months seems
hardly to be an option...<br>
<br>
When we all have the data replicated worldwide though,
it will be possible to download from all 3 servers at
the same time, which will more than double the speed
from this side of the ocean, and will be a considerable
speed up from yours, although probably not as much
(don't nail me on this, I'll deny it vehemently <-
this is not my writing :-)<br>
<br>
Anyway, this being said, the rest should be answered by
BADC directly.<br>
<br>
Thanks,<br>
Estani<br>
<br>
Am 03.03.2011 02:26, schrieb Karl Taylor:<br>
<span style="white-space: pre;">> this should have
been sent to<br>
<br>
<a moz-do-not-send="true"
class="moz-txt-link-abbreviated"
href="mailto:go-essp-tech@ucar.edu">go-essp-tech@ucar.edu</a><br>
<br>
<br>
<br>
> <br>
<br>
<br>
<br>
> sorry about that.<br>
<br>
<br>
<br>
> <br>
<br>
<br>
<br>
> Karl<br>
<br>
<br>
<br>
> <br>
<br>
<br>
<br>
> -------- Original Message -------- Subject:
download rates<br>
<br>
Date:<br>
<br>
<br>
<br>
> Wed, 2 Mar 2011 16:43:42 -0800 From: Karl Taylor<br>
<br>
<br>
<br>
> <a moz-do-not-send="true"
class="moz-txt-link-rfc2396E"
href="mailto:taylor13@llnl.gov"><taylor13@llnl.gov></a><br>
To: <a moz-do-not-send="true"
class="moz-txt-link-abbreviated"
href="mailto:esg-support@earthsystemgrid.org">esg-support@earthsystemgrid.org</a><br>
<br>
Grid<br>
<br>
<br>
<br>
> <a moz-do-not-send="true"
class="moz-txt-link-rfc2396E"
href="mailto:esg-support@earthsystemgrid.org"><esg-support@earthsystemgrid.org></a><br>
<br>
<br>
<br>
> <br>
<br>
<br>
<br>
> <br>
<br>
<br>
<br>
> <br>
<br>
<br>
<br>
> Hi all,<br>
<br>
<br>
<br>
> <br>
<br>
<br>
<br>
> When I download a 25 MB CMIP5 file from BADC
using the wget<br>
<br>
method: *<br>
<br>
<br>
<br>
> it took 35 sec to get started * then it
downloaded at an<br>
<br>
average rate<br>
<br>
<br>
<br>
> of 1.1 MB/min<br>
<br>
<br>
<br>
> <br>
<br>
<br>
<br>
> When I download the same file using the point and
click<br>
<br>
method: * it<br>
<br>
<br>
<br>
> took 20 sec to get started * then it downloaded
at an<br>
average<br>
<br>
rate of<br>
<br>
<br>
<br>
> 1.4 MB/min<br>
<br>
<br>
<br>
> <br>
<br>
<br>
<br>
> When I download 4 files simultaneously (in
parallel) using<br>
<br>
the click<br>
<br>
<br>
<br>
> method, I get somewhat slower download rate per
file, so<br>
<br>
somewhat<br>
<br>
<br>
<br>
> less than 4 times the data transfer rate I would
have<br>
gotten<br>
<br>
<br>
<br>
> downloading them in series.<br>
<br>
<br>
<br>
> <br>
<br>
<br>
<br>
> Note that at 1 MB/min, I could download 1 GB in
about 17<br>
<br>
hours or 1<br>
<br>
<br>
<br>
> TB (i.e., about 1/1000 of the entire archive) in
about 2<br>
<br>
years.<br>
<br>
<br>
<br>
> <br>
<br>
<br>
<br>
> Any way to speed this up some? (A colleague here
downloads<br>
<br>
files at<br>
<br>
<br>
<br>
> about 10 times this rate from ECMWF.)<br>
<br>
<br>
<br>
> <br>
<br>
<br>
<br>
> Note that I haven't checked how fast download
rates are<br>
from<br>
<br>
nodes<br>
<br>
<br>
<br>
> published to our (PCMDI's) gateway (since they
are not yet<br>
<br>
publicly<br>
<br>
<br>
<br>
> available).<br>
<br>
<br>
<br>
> <br>
<br>
<br>
<br>
> Best regards, Karl<br>
<br>
<br>
<br>
> <br>
<br>
<br>
<br>
> <br>
<br>
<br>
<br>
> <br>
<br>
<br>
<br>
> <br>
<br>
<br>
<br>
> _______________________________________________<br>
GO-ESSP-TECH<br>
<br>
mailing<br>
<br>
<br>
<br>
> list <a moz-do-not-send="true"
class="moz-txt-link-abbreviated"
href="mailto:GO-ESSP-TECH@ucar.edu">GO-ESSP-TECH@ucar.edu</a><br>
<br>
<br>
<br>
<br>
> <a moz-do-not-send="true"
class="moz-txt-link-freetext"
href="http://mailman.ucar.edu/mailman/listinfo/go-essp-tech">http://mailman.ucar.edu/mailman/listinfo/go-essp-tech</a></span><br>
<br>
<br>
-- <br>
Estanislao Gonzalez<br>
<br>
Max-Planck-Institut für Meteorologie (MPI-M)<br>
Deutsches Klimarechenzentrum (DKRZ) - German Climate
Computing Centre<br>
Room 108 - Bundesstrasse 45a, D-20146 Hamburg, Germany<br>
<br>
Phone: +49 (40) 46 00 94-126<br>
E-Mail: <a moz-do-not-send="true"
class="moz-txt-link-abbreviated"
href="mailto:estanislao.gonzalez@zmaw.de">estanislao.gonzalez@zmaw.de</a>
<br>
</blockquote>
<br>
<br>
<pre class="moz-signature" cols="72">--
Estanislao Gonzalez
Max-Planck-Institut für Meteorologie (MPI-M)
Deutsches Klimarechenzentrum (DKRZ) - German Climate Computing Centre
Room 108 - Bundesstrasse 45a, D-20146 Hamburg, Germany
Phone: +49 (40) 46 00 94-126
E-Mail: <a moz-do-not-send="true" class="moz-txt-link-abbreviated" href="mailto:estanislao.gonzalez@zmaw.de">estanislao.gonzalez@zmaw.de</a> </pre>
</div>
_______________________________________________<br>
GO-ESSP-TECH mailing list<br>
<a moz-do-not-send="true"
href="mailto:GO-ESSP-TECH@ucar.edu">GO-ESSP-TECH@ucar.edu</a><br>
<a class="moz-txt-link-freetext" href="http://mailman.ucar.edu/mailman/listinfo/go-essp-tech">http://mailman.ucar.edu/mailman/listinfo/go-essp-tech</a><br>
</blockquote>
</div>
<br>
<div>
<span class="Apple-style-span" style="border-collapse:
separate; color: rgb(0, 0, 0); font-family: Helvetica;
font-style: normal; font-variant: normal; font-weight:
normal; letter-spacing: normal; line-height: normal;
orphans: 2; text-indent: 0px; text-transform: none;
white-space: normal; widows: 2; word-spacing: 0px;
font-size: medium;"><span class="Apple-style-span"
style="border-collapse: separate; color: rgb(0, 0, 0);
font-family: Helvetica; font-style: normal; font-variant:
normal; font-weight: normal; letter-spacing: normal;
line-height: normal; orphans: 2; text-indent: 0px;
text-transform: none; white-space: normal; widows: 2;
word-spacing: 0px; font-size: medium;">
<div style="word-wrap: break-word;"><span
class="Apple-style-span" style="border-collapse:
separate; color: rgb(0, 0, 0); font-family: Helvetica;
font-style: normal; font-variant: normal; font-weight:
normal; letter-spacing: normal; line-height: normal;
orphans: 2; text-indent: 0px; text-transform: none;
white-space: normal; widows: 2; word-spacing: 0px;
font-size: medium;">
<div style="word-wrap: break-word;"><span
class="Apple-style-span" style="border-collapse:
separate; color: rgb(0, 0, 0); font-family:
Helvetica; font-size: medium; font-style: normal;
font-variant: normal; font-weight: normal;
letter-spacing: normal; line-height: normal;
orphans: 2; text-indent: 0px; text-transform:
none; white-space: normal; widows: 2;
word-spacing: 0px;">
<div style="word-wrap: break-word;">
<div>Rachana Ananthakrishnan</div>
<div>Argonne National Lab | University of
Chicago</div>
</div>
</span></div>
</span></div>
</span></span>
</div>
<br>
</div>
</blockquote>
<br>
<br>
<pre class="moz-signature" cols="72">--
Estanislao Gonzalez
Max-Planck-Institut für Meteorologie (MPI-M)
Deutsches Klimarechenzentrum (DKRZ) - German Climate Computing Centre
Room 108 - Bundesstrasse 45a, D-20146 Hamburg, Germany
Phone: +49 (40) 46 00 94-126
E-Mail: <a class="moz-txt-link-abbreviated" href="mailto:estanislao.gonzalez@zmaw.de">estanislao.gonzalez@zmaw.de</a> </pre>
</body>
</html>