[mpas-developers] Restructuring high-level MPAS driver

[Mark Petersen]

Yes, it looks good.  I think mpas.F deserves its own slide in some future 
talk on mpas:

program mpas
    use subdriver
    implicit none

    call init()
    call run()
    call finalize()
    stop

end program mpas

What could be simpler?  Only some details remain.

Mark



On Thu, 28 Oct 2010, Michael Duda wrote:

> Hi, All.
>
> I've updated the tar file at
> http://www.mmm.ucar.edu/people/duda/files/mpas/mpas_newdriver.tar.gz
> to include changes that move all restart file-related code into the
> core-specific main modules. If anyone has any other comments,
> concerns, or suggestions, I'd be glad to hear them as always;
> if the changes look acceptable, I'd propose to commit them to
> the repository trunk.
>
> Cheers,
> Michael
>
>
> On Thu, Oct 28, 2010 at 11:04:45AM -0600, Michael Duda wrote:
>> Hi, Mark.
>>
>> Thanks very much for taking look through the changes, and for the
>> excellent suggestions. I think the idea to pass just the domain
>> type into mpas_init() not only resolves the problem of block loops
>> within block loops, but makes the subdriver that much cleaner!
>>
>> I've updated the tar file at
>> http://www.mmm.ucar.edu/people/duda/files/mpas/mpas_newdriver.tar.gz
>> with your suggested changes, and I'll take a look at the restart file
>> issue next. As you and Todd have mentioned, I think it is a safe
>> assumption for now that every core will produce an output file, though
>> not necessarily a restart file; in any case, we're certainly free to
>> change our minds in future and move more I/O control into the cores.
>>
>> Cheers,
>> Michael
>>
>>
>> On Wed, Oct 27, 2010 at 04:24:24PM -0600, Mark Petersen wrote:
>>> Michael,
>>>
>>> I like the simplicity of mpas.F, and I like the direction of these
>>> revisions.
>>>
>>> I did find a problem with the mpas_init call.  You have this within a
>>> block loop in subroutine init, but subroutine setup_sw_test_case and my
>>> version of mpas_init_domain have block loops, so that would be block loops
>>> within block loops.  My addition of mpas_init_domain last week was exactly
>>> because I create derived grid variables that need a halo update, so needs
>>> to be outside of a block loop.  A subroutine should not have both domain
>>> and block arguments, like
>>>          call mpas_init(domain, block, block% mesh, dt)
>>> since domain contains the blocks.
>>>
>>> I think the cleanest solution is to change, in module_subdriver.F
>>>
>>>       !
>>>       ! Initialize core
>>>       !
>>>       dt = config_dt
>>>       block => domain % blocklist
>>>       do while (associated(block))
>>>          call mpas_init(domain, block, block% mesh, dt)
>>>          block=> block% next
>>>       end do
>>>
>>> to simply
>>>
>>>       !
>>>       ! Initialize core
>>>       !
>>>       call mpas_init(domain)
>>>
>>> and have mpas_init in module_core be something like
>>>
>>>    subroutine mpas_init(domain)
>>>
>>>       implicit none
>>>
>>>       type (domain_type), intent(inout) :: domain
>>>       type (block_type), intent(inout) :: block
>>>
>>>       if (.not. config_do_restart) call setup_sw_test_case(domain)
>>>
>>>       !
>>>       ! Initialize core
>>>       !
>>>       dt = config_dt
>>>       block => domain % blocklist
>>>       do while (associated(block))
>>>          call mpas_init_block(block, block% mesh, dt)
>>>          block=> block% next
>>>       end do
>>>
>>>    end subroutine mpas_init
>>>
>>> rename the former
>>>    subroutine mpas_init(domain, block, mesh, dt)
>>> in module_core.F to be
>>>    subroutine mpas_init_block(block, mesh, dt)
>>> and delete my newly added subroutine mpas_init_domain.  All the code I had
>>> put in mpas_init_domain can now be in the new mpas_init, and I can have
>>> halo updates at the end of mpas_init.
>>>
>>> I think it is safe to assume that there will always be output generated,
>>> but not necessarily restart files.
>>>
>>> Thanks for all your work on this!
>>>
>>> Mark
>>>
>>>
>>>
>>>
>>>
>>> On Tue, 26 Oct 2010, Michael Duda wrote:
>>>
>>>> Hi, Developers.
>>>>
>>>> I've been working to restructure the higher levels of the MPAS software
>>>> (driver and subdriver) to enable us to separate the test case
>>>> initialization from the model proper, among other things, and I've
>>>> settled on what I think is a set of changes that will take us in the
>>>> proper direction.
>>>>
>>>> Currently, in the current top-level code (mpas.F and module_subdriver.F),
>>>> the setup_test_cases() routine is called directly, which precludes the
>>>> separation of this from the model. I view the initialization code that
>>>> is currently in module_test_cases.F as something of its own "core" that
>>>> uses the MPAS software infrastructure in the same way that the sw,
>>>> nhyd_atmos, and ocean cores use this infrastructure. Consequently, I'd
>>>> like to generalize both mpas.F and module_subdriver.F by removing any
>>>> core-specific code, where the definition of "core" now includes
>>>> initialization, post-processing, etc. Under this generalization, the
>>>> content of mpas.F would be a main program that simply calls init(),
>>>> run(), and finalize() routines in the subdriver module, and the
>>>> subdriver module's init() implementation would be responsible for
>>>> initializing the framework (infrastructure) and then the core; to
>>>> further abstract the details of the infrastructure and core, I'm also
>>>> proposing that we provide main modules for each of these, which would
>>>> implement their own init, run, and finalize routines. Calls to
>>>> setup_test_cases() could still exist within either the init() or run()
>>>> routines implemented by the core's main module, if at all; however,
>>>> with a separated initialization, the initialization would exist as
>>>> its own core, whose run() routine would perform the work currently
>>>> done in setup_test_cases().
>>>>
>>>> I've placed a tar file to illustrate what these changes look like
>>>> in http://www.mmm.ucar.edu/people/duda/files/mpas/mpas_newdriver.tar.gz.
>>>> This code is an svn working copy, so it should be possible to 'svn
>>>> status' and 'svn diff' to see what files have changed. During the
>>>> restructuring, I found that the mpas_query() routine is no longer
>>>> needed, since number of time levels can be determined for fields purely
>>>> in the registry, so I also removed code related to this, too.
>>>>
>>>> One side effect in the currently reorganization is that both the output
>>>> and restart files are always opened at the start of MPAS execution,
>>>> regardless of whether a core intends to write a restart file at all.
>>>> This raises the question of whether control over output (and perhpas
>>>> even input) streams should be given to the individual MPAS cores. Would
>>>> assuming that a core always writes at least an output stream, but may
>>>> not write a restart stream be reasonable for now? If so, we could
>>>> probably relocate code to open/write/close the restart file down to the
>>>> core-specific code, for example.
>>>>
>>>> On the upside, the restructuring that I've proposed has very little
>>>> impact on the actual solver code in the MPAS cores; and, I've found
>>>> that the high-level code is lightweight enough that it should be easy
>>>> to make further changes as we move forward with future development.
>>>>
>>>> I apologize for the lengthy e-mail, but if anyone has any comments,
>>>> particularly concerning other requirements besides separating
>>>> initialization from model integration, I'd be very glad to hear them.
>>>> Assuming no objections that cannot be remedied, I'd like to commit
>>>> the proposed changes to the repository trunk fairly soon.
>>>>
>>>> Cheers,
>>>> Michael
>>>> _______________________________________________
>>>> mpas-developers mailing list
>>>> mpas-developers at mailman.ucar.edu
>>>> http://mailman.ucar.edu/mailman/listinfo/mpas-developers
>>>>
> _______________________________________________
> mpas-developers mailing list
> mpas-developers at mailman.ucar.edu
> http://mailman.ucar.edu/mailman/listinfo/mpas-developers
>