[mpas-developers] Provenance in output files

Doug Jacobsen jacobsen.douglas at gmail.com
Tue Apr 9 12:45:52 MDT 2013


Hi again everyone,

I wanted to resurrect this discussion and attempt to get the
history_attribute branch merged onto the trunk within the next week or so.

I think I addressed most of Michael's concerns with my last few comments,
with the exception of the history attribute.

If anyone has anymore questions/comments please let me know. But if not
I'll plan to merge this on Tuesday April 16th.

Thanks,
Doug


On Wed, Apr 3, 2013 at 7:02 PM, Doug Jacobsen <jacobsen.douglas at gmail.com>wrote:

> Hey Michael,
>
> Thanks for the comments. I had originally thought about putting the
> variables in a new module that was something like "mpas_model_variables.F",
> but I didn't see the value in doing that when we had mpas_constant.F
> already. I had also thought about putting them in the domain structure, so
> maybe I'll give that a try and see how it works out.
>
> I think it's perfectly fine to change version to modelVersion. The only
> real value I saw with having the core_name attribute separate was that it
> would be easier to determine the core. Also, in the case you describe with
> the history attribute overflowing, this would prevent that from removing
> the core name from the output files. For example, if someone used several
> nco operations on the input file before running it, it might not work with
> the fixed sizes I declared.
>
> I don't know if this is possible, but something that might be better would
> be to copy the history attribute from the input file, and then create a new
> history attribute that is the command used to run MPAS. As you said, this
> would be more appropriate with the definition of the history attribute. I
> don't think we should go down the route of using Fortran 2003 features,
> since it could introduce portability issues that we probably don't want to
> deal with. Especially since the history attribute is not necessarily
> important by itself (what we add to it at least).
>
> We can talk some more about how to migrate history information from the
> old mesh to the new mesh. I think one way where the history attribute might
> be useful is if we could somehow input some run information (testcase
> name/resolution/etc) into it when running.
>
> Thanks!
> Doug
>
>
> On Wed, Apr 3, 2013 at 4:26 PM, Michael Duda <duda at ucar.edu> wrote:
>
>> Hi, Doug.
>>
>> I'm testing out the branch now, and I have a few initial comments.
>>
>> I noticed that the variables to hold the version, model name, and core
>> name have been added to the mpas_constants.F module. This module is
>> currently used for physical constants, so I'd suggest we move these
>> attribute variables elsewhere. Currently, we keep the attributes
>> on_a_sphere and sphere_radius in the mesh type; but perhaps a better
>> place for modelName, coreName, version, and history would be inside the
>> domain type. If at some point in the future we add the ability to
>> instantiate multiple models in the same MPAS executable, I think it
>> would make sense to have attributes like modelName in the individual
>> instances of the domain_type, which is supposed to encapsulate the
>> complete computational state of the model.
>>
>> Regarding the naming of these attributes, perhaps we could rename
>> 'version' to 'model_version' to be less ambiguous. Also, given that the
>> run command in the history attribute already includes the name of the
>> specific executable, is there real value in including the attribute
>> 'core_name'?
>>
>> As you pointed out, dealing with the history string can get messy.
>> Declaring local variables as StrKIND*2 and StrKIND*4 should work in most
>> instances, but we might want to give a little more thought to whether
>> this couldn't be handled in a more robust way. For example in cycling
>> experiments or for longer runs that periodically restart, the history
>> attribute could easily overflow with repetitions of "mpirun -n 128
>> nhyd_atmos_model.exe". Fortran 2003 does support allocatable character
>> strings, and recent versions of ifort and pgf90 have implemented support
>> for this, but correct support in gfortran appears to require version
>> 4.8.0; and, I've not yet checked the BG/Q and xlf compilers. This also
>> raises the question of whether running the model constitutes
>> "modifications to the original data" (as the CF conventions describe the
>> "history attribute").
>>
>> Thanks for all of your work on this so far, and I'll let you know how it
>> goes with my testing.
>>
>> Cheers,
>> Michael
>>
>>
>> On Tue, Apr 02, 2013 at 02:49:48PM -0600, Doug Jacobsen wrote:
>> > Hi Everyone,
>> >
>> > I spent part of this morning trying to add some provenance information
>> to
>> > the output files that MPAS creates. Currently I have it adding model
>> name,
>> > core name, and a version string to each output file, and adding the
>> command
>> > that was used to run the model to the history attribute.
>> >
>> > You can preview the changes in the following branch.
>> > branches/history_attribute
>> >
>> > The current way to define this information is to modify Registry.xml
>> with
>> > the following syntax.
>> > <registry model="mpas" core="core_name" version="version_string">
>> > ...
>> > </registry>
>> >
>> > Then when the model is built, there are parameters created in
>> > src/framework/mpas_constants.F named
>> > modelName
>> > coreName
>> > version
>> > history
>> >
>> > History is read in from the input grid upon runtime, and written to all
>> > output files with the additional information describing how MPAS was
>> run.
>> > Currently it's hard coded to be "mpirun -n ${PROCS} ${CORE}_model.exe"
>> > since we require MPI at build time.
>> >
>> > If the model, core, and version strings are missing from Registry.xml
>> > currently it replaces their values with "MISSING", and writes those to
>> the
>> > output files so there is no negative consequences.
>> >
>> > I've tested this branch with the ocean model and it doesn't change
>> results
>> > in the current trunk. I also tested building with the shallow water
>> core,
>> > when the Registry.xml information is missing.
>> >
>> > I would like to merge this branch back onto the trunk within the next
>> few
>> > days unless anyone has any objections, as I think this additional
>> > information will be important to have in the future.
>> >
>> > Any comments/questions are welcome.
>> >
>> > Thanks
>> > Doug
>>
>> > _______________________________________________
>> > mpas-developers mailing list
>> > mpas-developers at mailman.ucar.edu
>> > http://mailman.ucar.edu/mailman/listinfo/mpas-developers
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/mpas-developers/attachments/20130409/e41d5727/attachment.html 


More information about the mpas-developers mailing list