[Tgcmgroup] processor binding on bluefire

[Ben Foster]

Hi tgcmgroup:

Please add "processor binding" to your TIEGCM/TIMEGCM jobs 
running on the IBM/AIX bluefire machine. See attached instructions.
Doing this can improve model performance by as much as 20%, 
varying with model resolution, and the number of nodes and 
processors used. It is important that we do this for more 
efficient use of the machine.

Instructions are also available at:
http://www/hao.ucar.edu/modeling/tgcm/doc/add_proc_binding.html
(or click on "Add Processor Binding" in the "What's New" section
of the TGCM website).

--Ben

----------------------------------------------------------------------
-
Ben Foster		      	High Altitude Observatory (HAO)
foster at ucar.edu			phone: 303-497-1595  fax: 303-497-1589  
Nat. Center for Atmos. Res.     P.O. Box 3000 Boulder CO 80307 USA
----------------------------------------------------------------------
-
-------------- next part --------------

Procedure to add processor binding to tiegcm/timegcm jobs run on bluefire:

This procedure can improve model performance by as much as 20%,
varying with the number of processors and nodes used.  See also 
http://www.cisl.ucar.edu/docs/bluefire/be_quickstart.html#binding

--------------------------------------------------------------------------
Step 1:
In the ibm job file (e.g., $TGCMROOT/tiegcm1.9/scripts/tiegcm-ibm.job), 
replace this line:

  mpirun.lsf $execdir/$model < $namelist >&! $output || \

with these two lines:

  setenv TARGET_CPU_LIST "-1"
  mpirun.lsf /usr/local/bin/launch $execdir/$model < $namelist >&! $output || \

Be sure to include the backslash at the end of the second line.
--------------------------------------------------------------------------
Step 2:
In the Makefile template (e.g., $TGCMROOT/tiegcm1.9/scripts/Makefile),
replace this line:

  LDFLAGS    += -bloadmap:loadmap -q64

with these two lines:

  LDFLAGS    += -bloadmap:loadmap -q64 $(OPTIM)
  LDFLAGS    += -bdatapsize:64K -bstackpsize:64K -btextpsize:64K

--------------------------------------------------------------------------
Notes:

The following files at both HAO and CISL have been modified for processor 
binding (models tiegcm1.9 and timegcm1-2dev7):<br>

  $TGCMROOT/tiegcm1.9/scripts/tiegcm-ibm.job
  $TGCMROOT/tiegcm1.9/scripts/Makefile.job

  $TGCMROOT/timegcm1-2dev7/scripts/timegcm-ibm.job
  $TGCMROOT/timegcm1-2dev7/scripts/Makefile.job

If your tiegcm1.9 or timegcm1-2dev7 job script allows
the default $utildir (i.e., "set utildir" is commented), then 
the job script, when submitted, will use the corresponding
Makefile above, so you will not have to change your Makefile.<br> 

If you change a Makefile in your *aix exec directory (i.e., one that
was already made by tgcm_config), you should rename it Makefile.new,
and add "-f Makefile.new" to the gmake command in your job script.

--------------------------------------------------------------------------
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/tgcmgroup/attachments/20081121/2cd056bf/attachment.html