[Wrf-users] cluster interconnect - Infiniband vs 10 gigabit ethernet

Aaron Sims apsims at ncsu.edu
Thu Apr 14 12:32:47 MDT 2011


Its been my experience that latency is the critical issue when running 
wrf in parallel, not raw speed.  I would spec out the latency of 10GigE, 
which I think is about the same as GigE, and compare it to Infiniband.
I think youll find infiniband is better in this case.


Zulauf, Michael wrote:
> Hi all - quick question.  Does anyone have data or experience comparing
> the performance and scaling of WRF with cluster interconnects utilizing
> 10 gigabit ethernet (10GigE) vs Infiniband?
> We're looking to expand and update our computing resources, and we'd
> originally spec'd it with Infiniband.  The nodes will have either dual
> quad- or hex-core Nehalem type processors.  There's been some IT
> pushback, suggesting that we should go with 10GigE.  I don't have any
> direct experience with 10GigE, but my experience with 1GigE shows that
> Infiniband scales far better.
> I've seen what I'd consider marketing material on the web that suggests
> that 10GigE is comparable to Infiniband, but they don't specifically
> mention WRF.  On the other hand, I've seen other sites that suggest an
> Infiniband interconnect is far superior.  Again, these don't
> specifically mention WRF.  I know the particular application in use is
> critical when deciding these things, and that WRF is a pretty demanding
> application when it comes to the interconnect.
> My suspicion is that Infiniband is still significantly superior, but if
> I'm going to be able to make any headway with IT, then I'll probably
> need some type of numbers to back up my arguments.
> Can anyone help?
> Thanks,
> Mike

More information about the Wrf-users mailing list