[Wrf-users] da_wrfvar.exe failing on "large" domains

[Don Morton]

I'm trying to run da_wrfvar.exe on a 1050x1050x75 grid point domain,
at 3km resolution.  This strikes me as a "large" domain, but not
really unreasonably large.  Inevitably, even with 256 cores, I get the
following ABEND message:

taskid: 0 hostname: nid00318
  Ntasks in X            16 , ntasks in Y            16
  *************************************
  Parent domain
  ids,ide,jds,jde             1         1050            1         1050
  ims,ime,jms,jme            -4           73           -4           73
  ips,ipe,jps,jpe             1           66            1           66
  *************************************
 DYNAMICS OPTION: Eulerian Mass Coordinate
    alloc_space_field: domain             1 ,     454678064  bytes allocated
 WRF NUMBER OF TILES =   1
0: ALLOCATE: 18446744072020333888 bytes requested; not enough memory


Although I can believe the figure for the alloc_space_field, I'm just
a little suspicious of the number of bytes requested for Task 0 - if
I've read this correctly, it comes out to 18.4 Exabytes! :)

Although I'm not sure, I believe the ims, ime,jms,jme values are the
start/stop dimensions of a subdomain in a given task, and if that's
the case, I'm suspicious about the negative start value.

I'll look into the code, but I'd like to first pose the question of
whether anybody has used da_wrfvar.exe for domains this big and/or if
anybody knows of inherent limitations that might prevent me from doing
so.

Thanks,

Don Morton
-- 
Arctic Region Supercomputing Center
http://www.arsc.edu/~morton/