[Wrf-users] cluster interconnect - Infiniband vs 10 gigabit ethernet

[mmkamal at sciborg.uwaterloo.ca]

Hi Mike,


Yes, interconnect is a big issue while running WRF on a HPC cluster.  
InfiniBand increases WRF performance by up to 115% for cluster size of  
24 nodes. I have used GigE since last one year but when I get the  
opportunity to use Infiniband it really surprised me. You can take a  
look on the following two resources for further details.


http://www.hpcadvisorycouncil.com/pdf/081016a%20WRF%20Model%20analysis.pdf

http://www.linuxclustersinstitute.org/conferences/archive/2009/PDF/Shainer_64557.pdf


Thanks
Kamal






Quoting "Zulauf, Michael" <Michael.Zulauf at iberdrolaren.com>:

> Hi all - quick question. Does anyone have data or experience comparing
> the performance and scaling of WRF with cluster interconnects utilizing
> 10 gigabit ethernet (10GigE) vs Infiniband?
>
> We're looking to expand and update our computing resources, and we'd
> originally spec'd it with Infiniband. The nodes will have either dual
> quad- or hex-core Nehalem type processors. There's been some IT
> pushback, suggesting that we should go with 10GigE. I don't have any
> direct experience with 10GigE, but my experience with 1GigE shows that
> Infiniband scales far better.
>
> I've seen what I'd consider marketing material on the web that suggests
> that 10GigE is comparable to Infiniband, but they don't specifically
> mention WRF. On the other hand, I've seen other sites that suggest an
> Infiniband interconnect is far superior. Again, these don't
> specifically mention WRF. I know the particular application in use is
> critical when deciding these things, and that WRF is a pretty demanding
> application when it comes to the interconnect.
>
> My suspicion is that Infiniband is still significantly superior, but if
> I'm going to be able to make any headway with IT, then I'll probably
> need some type of numbers to back up my arguments.
>
> Can anyone help?
>
> Thanks,
> Mike
>
> --
>
> PLEASE NOTE - NEW E-MAIL ADDRESS:
> michael.zulauf at iberdrolaren.com
>
> Mike Zulauf
> Meteorologist, Lead Senior
> Wind Asset Management
> Iberdrola Renewables
> 1125 NW Couch, Suite 700
> Portland, OR 97209
> Office: 503-478-6304 Cell: 503-913-0403
>
>
> Please be advised that email addresses for Iberdrola Renewables  
> personnel have changed to first.last at iberdrolaREN.com effective Aug.  
> 16, 2010. Please make a note. Thank you.
>
> This message is intended for the exclusive attention of the  
> recipient(s) indicated. Any information contained herein is strictly  
> confidential and privileged. If you are not the intended recipient,  
> please notify us by return e-mail and delete this message from your  
> computer system. Any unauthorized use, reproduction, alteration,  
> filing or sending of this message and/or any attached files may lead  
> to legal action being taken against the party(ies) responsible for  
> said unauthorized use. Any opinion expressed herein is solely that  
> of the author(s) and does not necessarily represent the opinion of  
> the Company. The sender does not guarantee the integrity, speed or  
> safety of this message, and does not accept responsibility for any  
> possible damage arising from the interception, incorporation of  
> viruses, or any other damage as a result of manipulation.
>
>
> _______________________________________________
> Wrf-users mailing list
> Wrf-users at ucar.edu
> http://mailman.ucar.edu/mailman/listinfo/wrf-users
>