[mpas-developers] Restructuring high-level MPAS driver

[Michael Duda]

Hi, All.

I've updated the tar file at
http://www.mmm.ucar.edu/people/duda/files/mpas/mpas_newdriver.tar.gz
to include changes that move all restart file-related code into the 
core-specific main modules. If anyone has any other comments,
concerns, or suggestions, I'd be glad to hear them as always;
if the changes look acceptable, I'd propose to commit them to
the repository trunk.

Cheers,
Michael


On Thu, Oct 28, 2010 at 11:04:45AM -0600, Michael Duda wrote:
> Hi, Mark.
> 
> Thanks very much for taking look through the changes, and for the
> excellent suggestions. I think the idea to pass just the domain
> type into mpas_init() not only resolves the problem of block loops
> within block loops, but makes the subdriver that much cleaner!
> 
> I've updated the tar file at
> http://www.mmm.ucar.edu/people/duda/files/mpas/mpas_newdriver.tar.gz
> with your suggested changes, and I'll take a look at the restart file
> issue next. As you and Todd have mentioned, I think it is a safe
> assumption for now that every core will produce an output file, though
> not necessarily a restart file; in any case, we're certainly free to
> change our minds in future and move more I/O control into the cores.
> 
> Cheers,
> Michael
> 
> 
> On Wed, Oct 27, 2010 at 04:24:24PM -0600, Mark Petersen wrote:
> > Michael,
> > 
> > I like the simplicity of mpas.F, and I like the direction of these 
> > revisions.
> > 
> > I did find a problem with the mpas_init call.  You have this within a 
> > block loop in subroutine init, but subroutine setup_sw_test_case and my 
> > version of mpas_init_domain have block loops, so that would be block loops 
> > within block loops.  My addition of mpas_init_domain last week was exactly 
> > because I create derived grid variables that need a halo update, so needs 
> > to be outside of a block loop.  A subroutine should not have both domain 
> > and block arguments, like
> >          call mpas_init(domain, block, block% mesh, dt)
> > since domain contains the blocks.
> > 
> > I think the cleanest solution is to change, in module_subdriver.F
> > 
> >       !
> >       ! Initialize core
> >       !
> >       dt = config_dt
> >       block => domain % blocklist
> >       do while (associated(block))
> >          call mpas_init(domain, block, block% mesh, dt)
> >          block=> block% next
> >       end do
> > 
> > to simply
> > 
> >       !
> >       ! Initialize core
> >       !
> >       call mpas_init(domain)
> > 
> > and have mpas_init in module_core be something like
> > 
> >    subroutine mpas_init(domain)
> > 
> >       implicit none
> > 
> >       type (domain_type), intent(inout) :: domain
> >       type (block_type), intent(inout) :: block
> > 
> >       if (.not. config_do_restart) call setup_sw_test_case(domain)
> > 
> >       !
> >       ! Initialize core
> >       !
> >       dt = config_dt
> >       block => domain % blocklist
> >       do while (associated(block))
> >          call mpas_init_block(block, block% mesh, dt)
> >          block=> block% next
> >       end do
> > 
> >    end subroutine mpas_init
> > 
> > rename the former
> >    subroutine mpas_init(domain, block, mesh, dt)
> > in module_core.F to be
> >    subroutine mpas_init_block(block, mesh, dt)
> > and delete my newly added subroutine mpas_init_domain.  All the code I had 
> > put in mpas_init_domain can now be in the new mpas_init, and I can have 
> > halo updates at the end of mpas_init.
> > 
> > I think it is safe to assume that there will always be output generated, 
> > but not necessarily restart files.
> > 
> > Thanks for all your work on this!
> > 
> > Mark
> > 
> > 
> > 
> > 
> > 
> > On Tue, 26 Oct 2010, Michael Duda wrote:
> > 
> > >Hi, Developers.
> > >
> > >I've been working to restructure the higher levels of the MPAS software
> > >(driver and subdriver) to enable us to separate the test case
> > >initialization from the model proper, among other things, and I've
> > >settled on what I think is a set of changes that will take us in the
> > >proper direction.
> > >
> > >Currently, in the current top-level code (mpas.F and module_subdriver.F),
> > >the setup_test_cases() routine is called directly, which precludes the
> > >separation of this from the model. I view the initialization code that
> > >is currently in module_test_cases.F as something of its own "core" that
> > >uses the MPAS software infrastructure in the same way that the sw,
> > >nhyd_atmos, and ocean cores use this infrastructure. Consequently, I'd
> > >like to generalize both mpas.F and module_subdriver.F by removing any
> > >core-specific code, where the definition of "core" now includes
> > >initialization, post-processing, etc. Under this generalization, the
> > >content of mpas.F would be a main program that simply calls init(),
> > >run(), and finalize() routines in the subdriver module, and the
> > >subdriver module's init() implementation would be responsible for
> > >initializing the framework (infrastructure) and then the core; to
> > >further abstract the details of the infrastructure and core, I'm also
> > >proposing that we provide main modules for each of these, which would
> > >implement their own init, run, and finalize routines. Calls to
> > >setup_test_cases() could still exist within either the init() or run()
> > >routines implemented by the core's main module, if at all; however,
> > >with a separated initialization, the initialization would exist as
> > >its own core, whose run() routine would perform the work currently
> > >done in setup_test_cases().
> > >
> > >I've placed a tar file to illustrate what these changes look like
> > >in http://www.mmm.ucar.edu/people/duda/files/mpas/mpas_newdriver.tar.gz.
> > >This code is an svn working copy, so it should be possible to 'svn
> > >status' and 'svn diff' to see what files have changed. During the
> > >restructuring, I found that the mpas_query() routine is no longer
> > >needed, since number of time levels can be determined for fields purely
> > >in the registry, so I also removed code related to this, too.
> > >
> > >One side effect in the currently reorganization is that both the output
> > >and restart files are always opened at the start of MPAS execution,
> > >regardless of whether a core intends to write a restart file at all.
> > >This raises the question of whether control over output (and perhpas
> > >even input) streams should be given to the individual MPAS cores. Would
> > >assuming that a core always writes at least an output stream, but may
> > >not write a restart stream be reasonable for now? If so, we could
> > >probably relocate code to open/write/close the restart file down to the
> > >core-specific code, for example.
> > >
> > >On the upside, the restructuring that I've proposed has very little
> > >impact on the actual solver code in the MPAS cores; and, I've found
> > >that the high-level code is lightweight enough that it should be easy
> > >to make further changes as we move forward with future development.
> > >
> > >I apologize for the lengthy e-mail, but if anyone has any comments,
> > >particularly concerning other requirements besides separating
> > >initialization from model integration, I'd be very glad to hear them.
> > >Assuming no objections that cannot be remedied, I'd like to commit
> > >the proposed changes to the repository trunk fairly soon.
> > >
> > >Cheers,
> > >Michael
> > >_______________________________________________
> > >mpas-developers mailing list
> > >mpas-developers at mailman.ucar.edu
> > >http://mailman.ucar.edu/mailman/listinfo/mpas-developers
> > >