[Dart-users] warning about WRF, DART, and the NCAR machine 'lightning'

Nancy Collins nancy at ucar.edu
Thu Apr 3 08:52:48 MDT 2008


hi all --

Recently, assimilations with DART/WRF running on
the NCAR supercomputer 'lightning' have been found
to generate non-reproducible behavior during the advance
of the WRF model. This appears to happen on roughly
1 percent of WRF advances.  DART assimilation steps
occurring subsequent to deviant WRF advances have
demonstrated unexpectedly high sensitivity and extremely
large innovations. This DART behavior is unexplained but
is believed to be due to the deviant WRF priors.

Analyzing these problems has turned out to be extremely
difficult as is often the case with sporadic system failures.
Since we have had no reports of this same behavior
on other platforms, and since lightning is scheduled for
decommissioning in Sept 2008, we are not looking into
this issue any further. The specific runs that generated bad
behavior on lightning have been rerun on the IBM machine
'blueice' with no indication of problems. Therefore, we recommend
that  users of DART, and DART/WRF in particular, exercise
extreme caution when using lightning.

If anyone has a case where they see inexplicably
large innovations or non-reproducible results from
identical runs on other computing platforms, please
let someone on the DART team know.

thanks -
nancy




More information about the Dart-users mailing list