[Wrf-users] cluster interconnect - Infiniband vs 10 gigabit ethernet

[Aaron Sims]

Mike,

Its been my experience that latency is the critical issue when running 
wrf in parallel, not raw speed.  I would spec out the latency of 10GigE, 
which I think is about the same as GigE, and compare it to Infiniband.
I think youll find infiniband is better in this case.

Aaron

Zulauf, Michael wrote:
> Hi all - quick question.  Does anyone have data or experience comparing
> the performance and scaling of WRF with cluster interconnects utilizing
> 10 gigabit ethernet (10GigE) vs Infiniband?
>
> We're looking to expand and update our computing resources, and we'd
> originally spec'd it with Infiniband.  The nodes will have either dual
> quad- or hex-core Nehalem type processors.  There's been some IT
> pushback, suggesting that we should go with 10GigE.  I don't have any
> direct experience with 10GigE, but my experience with 1GigE shows that
> Infiniband scales far better.
>
> I've seen what I'd consider marketing material on the web that suggests
> that 10GigE is comparable to Infiniband, but they don't specifically
> mention WRF.  On the other hand, I've seen other sites that suggest an
> Infiniband interconnect is far superior.  Again, these don't
> specifically mention WRF.  I know the particular application in use is
> critical when deciding these things, and that WRF is a pretty demanding
> application when it comes to the interconnect.
>
> My suspicion is that Infiniband is still significantly superior, but if
> I'm going to be able to make any headway with IT, then I'll probably
> need some type of numbers to back up my arguments.
>
> Can anyone help?
>
> Thanks,
> Mike
>
>