[mpas-developers] Provenance in output files

[Doug Jacobsen]

Hey Michael,

Thanks for the comments. I had originally thought about putting the
variables in a new module that was something like "mpas_model_variables.F",
but I didn't see the value in doing that when we had mpas_constant.F
already. I had also thought about putting them in the domain structure, so
maybe I'll give that a try and see how it works out.

I think it's perfectly fine to change version to modelVersion. The only
real value I saw with having the core_name attribute separate was that it
would be easier to determine the core. Also, in the case you describe with
the history attribute overflowing, this would prevent that from removing
the core name from the output files. For example, if someone used several
nco operations on the input file before running it, it might not work with
the fixed sizes I declared.

I don't know if this is possible, but something that might be better would
be to copy the history attribute from the input file, and then create a new
history attribute that is the command used to run MPAS. As you said, this
would be more appropriate with the definition of the history attribute. I
don't think we should go down the route of using Fortran 2003 features,
since it could introduce portability issues that we probably don't want to
deal with. Especially since the history attribute is not necessarily
important by itself (what we add to it at least).

We can talk some more about how to migrate history information from the old
mesh to the new mesh. I think one way where the history attribute might be
useful is if we could somehow input some run information (testcase
name/resolution/etc) into it when running.

Thanks!
Doug


On Wed, Apr 3, 2013 at 4:26 PM, Michael Duda <duda at ucar.edu> wrote:

> Hi, Doug.
>
> I'm testing out the branch now, and I have a few initial comments.
>
> I noticed that the variables to hold the version, model name, and core
> name have been added to the mpas_constants.F module. This module is
> currently used for physical constants, so I'd suggest we move these
> attribute variables elsewhere. Currently, we keep the attributes
> on_a_sphere and sphere_radius in the mesh type; but perhaps a better
> place for modelName, coreName, version, and history would be inside the
> domain type. If at some point in the future we add the ability to
> instantiate multiple models in the same MPAS executable, I think it
> would make sense to have attributes like modelName in the individual
> instances of the domain_type, which is supposed to encapsulate the
> complete computational state of the model.
>
> Regarding the naming of these attributes, perhaps we could rename
> 'version' to 'model_version' to be less ambiguous. Also, given that the
> run command in the history attribute already includes the name of the
> specific executable, is there real value in including the attribute
> 'core_name'?
>
> As you pointed out, dealing with the history string can get messy.
> Declaring local variables as StrKIND*2 and StrKIND*4 should work in most
> instances, but we might want to give a little more thought to whether
> this couldn't be handled in a more robust way. For example in cycling
> experiments or for longer runs that periodically restart, the history
> attribute could easily overflow with repetitions of "mpirun -n 128
> nhyd_atmos_model.exe". Fortran 2003 does support allocatable character
> strings, and recent versions of ifort and pgf90 have implemented support
> for this, but correct support in gfortran appears to require version
> 4.8.0; and, I've not yet checked the BG/Q and xlf compilers. This also
> raises the question of whether running the model constitutes
> "modifications to the original data" (as the CF conventions describe the
> "history attribute").
>
> Thanks for all of your work on this so far, and I'll let you know how it
> goes with my testing.
>
> Cheers,
> Michael
>
>
> On Tue, Apr 02, 2013 at 02:49:48PM -0600, Doug Jacobsen wrote:
> > Hi Everyone,
> >
> > I spent part of this morning trying to add some provenance information to
> > the output files that MPAS creates. Currently I have it adding model
> name,
> > core name, and a version string to each output file, and adding the
> command
> > that was used to run the model to the history attribute.
> >
> > You can preview the changes in the following branch.
> > branches/history_attribute
> >
> > The current way to define this information is to modify Registry.xml with
> > the following syntax.
> > <registry model="mpas" core="core_name" version="version_string">
> > ...
> > </registry>
> >
> > Then when the model is built, there are parameters created in
> > src/framework/mpas_constants.F named
> > modelName
> > coreName
> > version
> > history
> >
> > History is read in from the input grid upon runtime, and written to all
> > output files with the additional information describing how MPAS was run.
> > Currently it's hard coded to be "mpirun -n ${PROCS} ${CORE}_model.exe"
> > since we require MPI at build time.
> >
> > If the model, core, and version strings are missing from Registry.xml
> > currently it replaces their values with "MISSING", and writes those to
> the
> > output files so there is no negative consequences.
> >
> > I've tested this branch with the ocean model and it doesn't change
> results
> > in the current trunk. I also tested building with the shallow water core,
> > when the Registry.xml information is missing.
> >
> > I would like to merge this branch back onto the trunk within the next few
> > days unless anyone has any objections, as I think this additional
> > information will be important to have in the future.
> >
> > Any comments/questions are welcome.
> >
> > Thanks
> > Doug
>
> > _______________________________________________
> > mpas-developers mailing list
> > mpas-developers at mailman.ucar.edu
> > http://mailman.ucar.edu/mailman/listinfo/mpas-developers
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/mpas-developers/attachments/20130403/5a804c4f/attachment.html