[Go-essp-tech] CMIP5 data archive size estimate

[Pauline Mak]

Hi all,

Bryan Lawrence wrote:
> hi Stephen
> 
> Hmm. I just spoke to you and suggested I thought you were wrong, but I did my calculation again, and it was probably me that was wrong.
> 
> I think the number of different outputs requested is of o(500) (*)
> I think the number of experiments is of o(50)
> The number of modelling centres is of o(20)
> The number of ensembles is of o(3)
> Number out=500x50x20x3=1.5E6.
> 
> So, what's a factor of two between friends :-)
> 
> But, this also implies, 1PB/2 million= 0.5 GB per atomic dataset. We know/think that gridftp doesn't like small files ... is this big enough? Does the BDM aggregate things to faster?
> 

I was under the impression that the new implementation of the client (at 
least for Java, see http://dev.globus.org/wiki/CoG_JGlobus_1.5.0) that 
improves transfers for many small files.  Although, I'm not entirely 
sure what this means!

Cheers,

-Pauline.

-- 
Pauline Mak

Assistant Manager, ARCS Data Services
Ph:  +61 3 6226 7518
Mob: +61 411 638 196
Email: pauline.mak at arcs.org.au
Jabber: pauline.mak at arcs.org.au
Calendar: http://tinyurl.com/pmak-arcs-calendar
http://www.arcs.org.au/

TPAC
Email: pauline.mak at utas.edu.au
http://www.tpac.org.au/