[Go-essp-tech] CMIP5 data archive size estimate
Pauline Mak
pauline.mak at arcs.org.au
Sun Dec 13 17:26:21 MST 2009
Hi all,
Bryan Lawrence wrote:
> hi Stephen
>
> Hmm. I just spoke to you and suggested I thought you were wrong, but I did my calculation again, and it was probably me that was wrong.
>
> I think the number of different outputs requested is of o(500) (*)
> I think the number of experiments is of o(50)
> The number of modelling centres is of o(20)
> The number of ensembles is of o(3)
> Number out=500x50x20x3=1.5E6.
>
> So, what's a factor of two between friends :-)
>
> But, this also implies, 1PB/2 million= 0.5 GB per atomic dataset. We know/think that gridftp doesn't like small files ... is this big enough? Does the BDM aggregate things to faster?
>
I was under the impression that the new implementation of the client (at
least for Java, see http://dev.globus.org/wiki/CoG_JGlobus_1.5.0) that
improves transfers for many small files. Although, I'm not entirely
sure what this means!
Cheers,
-Pauline.
--
Pauline Mak
Assistant Manager, ARCS Data Services
Ph: +61 3 6226 7518
Mob: +61 411 638 196
Email: pauline.mak at arcs.org.au
Jabber: pauline.mak at arcs.org.au
Calendar: http://tinyurl.com/pmak-arcs-calendar
http://www.arcs.org.au/
TPAC
Email: pauline.mak at utas.edu.au
http://www.tpac.org.au/
More information about the GO-ESSP-TECH
mailing list