[Wrf-users] cluster interconnect - Infiniband vs 10 gigabit ethernet

[Ólafur Rögnvaldsson]

I agree with Aaron, it's the low latency of the InfiniBand that makes it a more suitable inter-node fabric than 1/10G Ethernet.

Kind regards, Ólafur.

> Mike,
> 
> Its been my experience that latency is the critical issue when running 
> wrf in parallel, not raw speed.  I would spec out the latency of 10GigE, 
> which I think is about the same as GigE, and compare it to Infiniband.
> I think youll find infiniband is better in this case.
> 
> Aaron
> 
> Zulauf, Michael wrote:
>> Hi all - quick question.  Does anyone have data or experience comparing
>> the performance and scaling of WRF with cluster interconnects utilizing
>> 10 gigabit ethernet (10GigE) vs Infiniband?
>> 
>> We're looking to expand and update our computing resources, and we'd
>> originally spec'd it with Infiniband.  The nodes will have either dual
>> quad- or hex-core Nehalem type processors.  There's been some IT
>> pushback, suggesting that we should go with 10GigE.  I don't have any
>> direct experience with 10GigE, but my experience with 1GigE shows that
>> Infiniband scales far better.
>> 
>> I've seen what I'd consider marketing material on the web that suggests
>> that 10GigE is comparable to Infiniband, but they don't specifically
>> mention WRF.  On the other hand, I've seen other sites that suggest an
>> Infiniband interconnect is far superior.  Again, these don't
>> specifically mention WRF.  I know the particular application in use is
>> critical when deciding these things, and that WRF is a pretty demanding
>> application when it comes to the interconnect.
>> 
>> My suspicion is that Infiniband is still significantly superior, but if
>> I'm going to be able to make any headway with IT, then I'll probably
>> need some type of numbers to back up my arguments.
>> 
>> Can anyone help?
>> 
>> Thanks,
>> Mike
>> 
>> 
> _______________________________________________
> Wrf-users mailing list
> Wrf-users at ucar.edu
> http://mailman.ucar.edu/mailman/listinfo/wrf-users

--
Kveðja/regards,
Ólafur Rögnvaldsson
Reiknistofa í veðurfræði - Belgingur
www.riv.is - www.belgingur.is