[Wrf-users] cluster interconnect - Infiniband vs 10 gigabit ethernet

Ólafur Rögnvaldsson or at belgingur.is
Thu Apr 14 15:00:45 MDT 2011

I agree with Aaron, it's the low latency of the InfiniBand that makes it a more suitable inter-node fabric than 1/10G Ethernet.

Kind regards, Ólafur.

> Mike,
> Its been my experience that latency is the critical issue when running 
> wrf in parallel, not raw speed.  I would spec out the latency of 10GigE, 
> which I think is about the same as GigE, and compare it to Infiniband.
> I think youll find infiniband is better in this case.
> Aaron
> Zulauf, Michael wrote:
>> Hi all - quick question.  Does anyone have data or experience comparing
>> the performance and scaling of WRF with cluster interconnects utilizing
>> 10 gigabit ethernet (10GigE) vs Infiniband?
>> We're looking to expand and update our computing resources, and we'd
>> originally spec'd it with Infiniband.  The nodes will have either dual
>> quad- or hex-core Nehalem type processors.  There's been some IT
>> pushback, suggesting that we should go with 10GigE.  I don't have any
>> direct experience with 10GigE, but my experience with 1GigE shows that
>> Infiniband scales far better.
>> I've seen what I'd consider marketing material on the web that suggests
>> that 10GigE is comparable to Infiniband, but they don't specifically
>> mention WRF.  On the other hand, I've seen other sites that suggest an
>> Infiniband interconnect is far superior.  Again, these don't
>> specifically mention WRF.  I know the particular application in use is
>> critical when deciding these things, and that WRF is a pretty demanding
>> application when it comes to the interconnect.
>> My suspicion is that Infiniband is still significantly superior, but if
>> I'm going to be able to make any headway with IT, then I'll probably
>> need some type of numbers to back up my arguments.
>> Can anyone help?
>> Thanks,
>> Mike
> _______________________________________________
> Wrf-users mailing list
> Wrf-users at ucar.edu
> http://mailman.ucar.edu/mailman/listinfo/wrf-users

Ólafur Rögnvaldsson
Reiknistofa í veðurfræði - Belgingur
www.riv.is - www.belgingur.is

More information about the Wrf-users mailing list