[Dart-dev] DART/branches Revision: 12435

[dart at ucar.edu]

nancy at ucar.edu
2018-03-07 13:27:47 -0700 (Wed, 07 Mar 2018)
267
fix the ensemble manager so it compiles.  insert the changes
i made to the mpi utils for passing in an externally generated
mpi communicator (e.g. if running with cesm or another external
program which has already initialized mpi and made us a communicator
to use.)




Modified: DART/branches/rma_distrib/assimilation_code/modules/utilities/ensemble_manager_mod.f90
===================================================================
--- DART/branches/rma_distrib/assimilation_code/modules/utilities/ensemble_manager_mod.f90	2018-03-07 18:44:16 UTC (rev 12434)
+++ DART/branches/rma_distrib/assimilation_code/modules/utilities/ensemble_manager_mod.f90	2018-03-07 20:27:47 UTC (rev 12435)
@@ -114,8 +114,8 @@
 !> (we could make the option of redefining i8 to be i4 in types_mod.f90)
 !> we assume that no task will have the memory to allocate a full i8 buffer,
 !> so my_num_vars will remain expressible as an i4.
-
->
+  
+   ! integer :: num_pes??
    integer(i8)                  :: num_vars 
    integer                      :: num_copies, my_num_copies, my_num_vars
    integer,        allocatable  :: my_copies(:)
@@ -759,7 +759,8 @@
 
 integer :: num_per_pe_below, num_left_over, i
 
-ens_handle%num_pes = task_count()
+!ens_handle%num_pes = task_count()
+num_pes = task_count()
 
 ! Option 1: Maximum separation for both vars and copies
 ! Compute the total number of copies I'll get for var complete

Modified: DART/branches/rma_distrib/assimilation_code/modules/utilities/mpi_utilities_mod.f90
===================================================================
--- DART/branches/rma_distrib/assimilation_code/modules/utilities/mpi_utilities_mod.f90	2018-03-07 18:44:16 UTC (rev 12434)
+++ DART/branches/rma_distrib/assimilation_code/modules/utilities/mpi_utilities_mod.f90	2018-03-07 20:27:47 UTC (rev 12435)
@@ -7,7 +7,7 @@
 
 !> A collection of interfaces to the MPI (Message Passing Interface)
 !> multi-processor communication library routines.
-!>
+!> 
 !> All MPI routines are called from here.  There is a companion
 !> file which has the same module name and entry points but all
 !> routines are stubs.  This allows a single-task version of the
@@ -22,8 +22,7 @@
                           set_output, set_tasknum, initialize_utilities,     &
                           finalize_utilities,                                &
                           nmlfileunit, do_output, do_nml_file, do_nml_term,  &
-                          find_namelist_in_file, check_namelist_read
-
+                             find_namelist_in_file, check_namelist_read
 use time_manager_mod, only : time_type, get_time, set_time
 
 ! BUILD TIP 1
@@ -116,13 +115,15 @@
 character(len=32 ), parameter :: revision = "$Revision$"
 character(len=128), parameter :: revdate  = "$Date$"
 
-logical :: module_initialized   = .false.
+logical :: module_initialized = .false.
 
-character(len = 129) :: saved_progname = ''
-character(len = 129) :: shell_name = ''   ! if needed, add ksh, tcsh, bash, etc
+character(len = 256) :: saved_progname = ''
+character(len = 256) :: shell_name = ''   ! if needed, add ksh, tcsh, bash, etc
 integer :: head_task = 0         ! def 0, but N-1 if reverse_task_layout true
 logical :: print4status = .true. ! minimal messages for async4 handshake
 
+logical :: given_communicator = .false.   ! if communicator passed in, use it
+
 character(len = 256) :: errstring, errstring1
 
 ! for broadcasts, pack small messages into larger ones.  remember that the
@@ -131,17 +132,21 @@
 integer, parameter :: PACKLIMIT1 = 8
 integer, parameter :: PACKLIMIT2 = 512
 
-! also for broadcasts, make sure message size is not too large.  if so,
-! split a single request into two or more broadcasts.  i know 2G is really
-! 2 * 1024 * 1024 * 1024, but err on the conservative side here.
+! also for broadcasts, make sure message size is not too large.  the MPI
+! standard has a max limit on the number of items in a single message
+! (independent of item byte count).  if larger, split the request into
+! multiples, each less than the limit.  the current standard is 2G, but 
+! be a little conservative and use 2 billion and not 2**31.
 integer, parameter :: BCAST_MAXSIZE = 2 * 1000 * 1000 * 1000
 
-! option for simple send/recvs to limit max single message size.
-! split a single request into two or more broadcasts.  i know 2G is really
-! 2 * 1024 * 1024 * 1024, but err on the conservative side here.
+! for send/receive make sure message size is not too large.  the MPI
+! standard has a max limit on the number of items in a single message
+! (independent of item byte count).  if larger, split the request into
+! multiples, each less than the limit.  the current standard is 2G, but
+! be a little conservative and use 2 billion and not 2**31.
 integer, parameter :: SNDRCV_MAXSIZE = 2 * 1000 * 1000 * 1000
 
-! this turns on trace messages for most MPI communications
+! this turns on trace messages for most MPI communications 
 logical :: verbose        = .false.   ! very very very verbose, use with care
 logical :: async2_verbose = .false.   ! messages only for system() in async2
 logical :: async4_verbose = .false.   ! messages only for block/restart async4
@@ -180,11 +185,19 @@
 ! mpi cover routines
 !-----------------------------------------------------------------------------