[Wrf-users] Wrf-users Digest, Vol 76, Issue 6

[Hein Zelle]

Dear WRF Help,

thanks for the additional comments, good points.

> Just a comment on Hein's comment: there may be another reason he
> sees the > bottleneck at 64+ CPUs. For a given domain size, the
> performance will eventually flat out simply because there isn't
> enough computations to scale up.  There is a balance between
> computation and communication (via halos for decomposed domains). If
> one uses nests, then the performance might be limited by the
> smallest domain.

We verified that on the older xeons it was absolutely memory-bandwidth
related (big scaling problems already going from 8 to 16 to 32 cores,
and they disappear with newer nehalems).  However, we have also seen
the "smallest domain" limitation.  If your smallest domain can't be
divided up efficiently over the cpu's, you don't gain anything.

This is why we often optimize historical simulations differently:
- use a relatively small inner domain (e.g. 60x60 grid points)
- use a maximum of 8 or 16 cores per simulation
- run multiple simulations in parallel, each doing a historical year.

That's why we often have several parallel simulations at one time.
This will actually cause bottlenecks on the storage server.  We
haven't run into network problems (YET!) since we switched to
infiniband.

Jonas, can you report back if you find out about the SSD performance?

Kind regards,
	Hein Zelle


-- 

Dr. Hein Zelle
Advisor Meteorology & Oceanography

Tel:    +31 (0)527-242299
Fax:    +31 (0)527-242016
Email:  hein.zelle at bmtargoss.com
Web:    www.bmtargoss.com

BMT ARGOSS
P.O. Box 61, 8325 ZH Vollenhove
Voorsterweg 28, 8316 PT Marknesse
The Netherlands

----Confidentiality Notice & Disclaimer---- 

The contents of this e-mail and any attachments are intended for the
use of the mail addressee(s) shown. If you are not that person, you
are not allowed to read it, to take any action based upon it or to
copy it, forward, distribute or disclose the contents of it and you
should please delete it from your system. BMT ARGOSS does not accept
liability for any errors or omissions in the context of this e-mail or
its attachments which arise as a result of internet transmission, nor
accept liability for statements which are those of the author and
clearly not made on behalf of BMT ARGOSS.