[Wrf-users] How wrf processes nests

[Gustafson, William I]

Dave,

WRF will sequentially work through the nests for a given timestep. Most likely WRF uses some sort of tree following algorithm (I'm guessing on this detail, so don't quote me on it without checking the code). WRF's communication patterns are currently unable to process two domains simultaneously on different sets of processors. Instead, all the processors look at one domain at a time and step through each domain until there are not more to process for a given timestep. Even if the domains both have the same resolution and mother, they would be treated independently in this stepping procedure.

There is a fair amount of overhead to do the interpolation and communication between nests. So, depending on the size and spacing of your nests, it may be more efficient to run one larger fine domain than two smaller ones. But, I have never tried to optimize this sort of situation. Maybe somebody else has tested this sort of thing.

-Bill
___________________________________________________________________

William I. Gustafson Jr., Ph.D.
Scientist
ATMOSPHERIC SCIENCES AND GLOBAL CHANGE DIVISION

Pacific Northwest National Laboratory
P.O. 999, MSIN K9-30
Richland, WA  99352
Tel: 509-372-6110
William.Gustafson at pnnl.gov<mailto:William.Gustafson at pnnl.gov>
http://www.pnnl.gov/atmospheric/staff/staff_info.asp?staff_num=5716
http://www.researcherid.com/rid/A-7732-2008



From: David Smith <dsmith at uvi.edu<mailto:dsmith at uvi.edu>>
Date: Thursday, June 9, 2016 at 9:33 AM
To: William Gustafson <william.gustafson at pnnl.gov<mailto:william.gustafson at pnnl.gov>>, wrf-users <wrf-users at ucar.edu<mailto:wrf-users at ucar.edu>>
Subject: Re: [Wrf-users] How wrf processes nests

   Thanks Bill, but let me pursue this just a bit to be sure I’m not missing something.  I understand how the coarse and fine nests are processed.  I am asking how the two equal, but separate, 0.333 km fine nests are handled when their “time” comes.  Are you saying that there is no (significant) difference whether wrf runs each of the fine nests sequentially (each getting all the allotted processors) or simultaneously (sharing the total number of processors)?
  Consider a simplistic example.  For a sequential run of the two fine nests, if I make the x and y dimensions of each fine nest divisible by 10 and 5 respectively, then I know wrf can create 50 subgrids for an efficient run on 50 processors.  I would want each of the fine grids to have this same structure.
   But if they will be running simultaneously, then wouldn’t I want the total number of subgrids for the two nests combined to be 50?  (My two fine nests are not the same size.) Of course, I am assuming that wrf makes some reasonable choice for the sub grids – maybe I need to better understand how it does that as well.
 Thanks!

Dave

Dr. David J Smith
College of Science and Mathematics
University of the Virgin Islands
St. Thomas,  VI  00802
340 693-1257
dsmith at uvi.edu<mailto:dsmith at uvi.edu>

From: "Gustafson, William I" <William.Gustafson at pnnl.gov<mailto:William.Gustafson at pnnl.gov>>
Date: Wed, 8 Jun 2016 20:25:35 +0000
To: Dave Smith <dsmith at uvi.edu<mailto:dsmith at uvi.edu>>, wrf-users <wrf-users at ucar.edu<mailto:wrf-users at ucar.edu>>
Subject: Re: [Wrf-users] How wrf processes nests

Dave,

The grids all share the same processors and are kept in sync at the end of each coarse-domain timestep. So, the outer nest runs on the processors first, then that nest waits while the finer nest is calculated. Then, the fine nest waits while the coarse nest calculates, and on-and-on it goes. So, within a timestep, each nest is run sequentially. However, if one sees thinks of this in terms of the processor usage, they are simultaneous.

-Bill
___________________________________________________________________

William I. Gustafson Jr., Ph.D.
Scientist
ATMOSPHERIC SCIENCES AND GLOBAL CHANGE DIVISION

Pacific Northwest National Laboratory
P.O. 999, MSIN K9-30
Richland, WA  99352
Tel: 509-372-6110
William.Gustafson at pnnl.gov<mailto:William.Gustafson at pnnl.gov>
http://www.pnnl.gov/atmospheric/staff/staff_info.asp?staff_num=5716
http://www.researcherid.com/rid/A-7732-2008



From: <wrf-users-bounces at ucar.edu<mailto:wrf-users-bounces at ucar.edu>> on behalf of David Smith <dsmith at uvi.edu<mailto:dsmith at uvi.edu>>
Date: Wednesday, June 8, 2016 at 12:55 PM
To: wrf-users <wrf-users at ucar.edu<mailto:wrf-users at ucar.edu>>
Subject: [Wrf-users] How wrf processes nests

Hi,

  I am running wrf using mpirun on a 10 node cluster with 12 processors each – 120 total.  I have two fine nests (0.333 km) within a 1.0 km coarser nest.  I am trying to make a judicious choice for the dimensions of the fine nests, to minimize runtime.  (Making one fine nest would double the area.)  Will wrf run the two fine grids sequentially or simultaneously?  I would think this info would be in the users guide, but I’m not finding it.  Thanks!

Dave Smith

Dr. David J Smith
College of Science and Mathematics
University of the Virgin Islands
St. Thomas,  VI  00802
340 693-1257
dsmith at uvi.edu<mailto:dsmith at uvi.edu>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/wrf-users/attachments/20160609/8e672974/attachment.html