[mpas-developers] config_write_initial_output option
Mark Petersen
mpetersen at lanl.gov
Fri May 11 16:35:42 MDT 2012
Michael,
Thanks for your work on this. What you are suggesting is more
comprehensive than my changes. I agree that appending output after a
restart, as if it were a continuous run, is a nice feature.
Please commit whatever changes you like to my branch. I will go ahead
and test it on the ocean core when I see your commits.
Mark
On 05/11/12 15:27, Michael Duda wrote:
> Hi, Mark.
>
> Although the branch does avoid writing the initial time for a cold-start run
> with all frames in the same output file, it appears that the output filename has
> the timestamp of the first frame in the file added to it; whereas with
> config_write_initial_output=.true. we might get an output file "output.nc", with
> the option set to false, the output file is named, e.g.,
> "output.0000-01-01_06.00.00.nc". I suspect this is done to avoid filename
> conflicts when restarting a run, which is where I think what I had in mind
> differs from the current behavior.
>
> Consider the example where we write history every 6h and write 4 frames per
> outfile (in other words, separate output files for each day) with files
>
> output.0000-01-01_00.00.00.nc
> output.0000-01-02_00.00.00.nc
> output.0000-01-03_00.00.00.nc
> ...
>
> that contain times
>
> "0000-01-01_00:00:00 ",
> "0000-01-01_06:00:00 ",
> "0000-01-01_12:00:00 ",
> "0000-01-01_18:00:00 " ;
>
> "0000-01-02_00:00:00 ",
> "0000-01-02_06:00:00 ",
> "0000-01-02_12:00:00 ",
> "0000-01-02_18:00:00 " ;
>
> "0000-01-03_00:00:00 ",
> "0000-01-03_06:00:00 ",
> "0000-01-03_12:00:00 ",
> "0000-01-03_18:00:00 " ;
>
> ...
>
> respectively. It may typically happen that we run for an integral number of
> days, in which we end up with an output file at the final time that has just a
> single output time, e.g.,
>
> output.0000-01-05_00.00.00.nc
>
> with the output frame at 0000-01-05_00:00:00 only. Ideally (in my opinion), we'd
> like to be able to restart the model from 0000-01-05_00:00:00 and avoid writing
> the intial time, but still write the times 0000-01-05_06:00:00, 0000-01-05_12:00:00,
> and 0000-01-05_18:00:00 to the file output.0000-01-05_00.00.00.nc. Basically,
> the restarted run would have output files structured in an identical way to a
> single run for the complete simulation period.
>
> To support this behavior, I've modified the IO layers to support appending to
> existing files, and to seek through files to find the frame number that we need
> to begin writing at. If it sounds alright to you, I can commit the changes that
> I have to support appending to existing output files to the branch. The only
> remaining work then would be in the code where we open and close files (the same
> parts that you've modified); here, I think we'd need logic to determine whether
> we want to append to an output file or to overwrite it (considering whether it
> is a restart run, whether the file already exists, and the number of frames
> in each file). I'd volunteer to make the additional changes in the branch, if
> you'd like.
>
> Michael
>
>
> On Thu, May 10, 2012 at 03:51:34PM -0600, Mark Petersen wrote:
>> Michael,
>>
>> Thanks for your thoughts.
>>
>> I tested the ocean core using config_write_initial_output=.false. with
>> both single and multiple frames per output file, and both worked as
>> expected. In both cases,
>> mpas_output_state_init
>> is called the first time within a
>> if(output_frame == 1) then
>> branch in mpas_*_mpas_core.F. Is calling the initialization routine
>> your concern for multiple frames?
>>
>> Your more general implementation of time frames sounds good. It is
>> mostly a question of timing. If you are thinking of making these
>> changes in the next week or two, I'm happy to wait and you can use my
>> branch if you like. If it is a longer term improvement, I prefer to
>> merge this config_write_initial_output flag to the trunk so I don't need
>> to maintain a branch from which to run our large simulations.
>>
>> Thanks,
>> Mark
>>
>>
>> On 05/10/12 14:24, Michael Duda wrote:
>>> Mark,
>>>
>>> I think this would be a useful option, though as currently implemented
>>> in the branch, I'm not sure that it would work as expected when there
>>> are multiple history frames written to the same file. I've also been
>>> looking at implementing this, and I have some changes to the IO layer to
>>> support appending time frames to an existing file that I think might be
>>> useful for a completely general implementation of the
>>> write_initial_output functionality. I can send these changes to you to
>>> take a look at, or I can merge them into the branch; just let me know.
>>>
>>> Michael
>>>
>>>
>>> On Thu, May 10, 2012 at 01:06:15PM -0600, Mark Petersen wrote:
>>>> MPAS Developers,
>>>>
>>>> I added the flag config_write_initial_output to all cores in the branch
>>>> ocean_projects/first_output_optional. It simply controls if an output
>>>> file is written upon start-up. The ocean core needs this for our large
>>>> runs, to save i/o and for simply time-averaged variables. Because the
>>>> first file initialized in driver/mpas_subdriver.F, it affects all
>>>> cores. In addition, it seems like a useful flag for everyone.
>>>>
>>>> I tested by compiling all four cores, and running with the ocean core.
>>>> Could someone run a little test with the atmospheric cores, switching
>>>> this flag on and off, and make sure it works as expected? When I hear
>>>> back, I will merge the branch to the trunk.
>>>>
>>>> Thanks,
>>>> Mark
>>>> _______________________________________________
>>>> mpas-developers mailing list
>>>> mpas-developers at mailman.ucar.edu
>>>> http://mailman.ucar.edu/mailman/listinfo/mpas-developers
More information about the mpas-developers
mailing list