[Wrf-users] da_wrfvar.exe failing on "large" domains

Don Morton morton at arsc.edu
Tue Sep 22 16:19:15 MDT 2009

I'm trying to run da_wrfvar.exe on a 1050x1050x75 grid point domain,
at 3km resolution.  This strikes me as a "large" domain, but not
really unreasonably large.  Inevitably, even with 256 cores, I get the
following ABEND message:

taskid: 0 hostname: nid00318
  Ntasks in X            16 , ntasks in Y            16
  Parent domain
  ids,ide,jds,jde             1         1050            1         1050
  ims,ime,jms,jme            -4           73           -4           73
  ips,ipe,jps,jpe             1           66            1           66
 DYNAMICS OPTION: Eulerian Mass Coordinate
    alloc_space_field: domain             1 ,     454678064  bytes allocated
0: ALLOCATE: 18446744072020333888 bytes requested; not enough memory

Although I can believe the figure for the alloc_space_field, I'm just
a little suspicious of the number of bytes requested for Task 0 - if
I've read this correctly, it comes out to 18.4 Exabytes! :)

Although I'm not sure, I believe the ims, ime,jms,jme values are the
start/stop dimensions of a subdomain in a given task, and if that's
the case, I'm suspicious about the negative start value.

I'll look into the code, but I'd like to first pose the question of
whether anybody has used da_wrfvar.exe for domains this big and/or if
anybody knows of inherent limitations that might prevent me from doing


Don Morton
Arctic Region Supercomputing Center

More information about the Wrf-users mailing list