[mpas-developers] mpas output file naming conventions

Jones, Philip W pwjones at lanl.gov
Thu Sep 8 11:09:05 MDT 2011


Michael,

I think we'll want a more general capability, including renaming the root based on experiment name and having one frame in each file with a simulation date stamp.  While some of this could be handled via post-processing, it can be difficult to manage within a queue/script structure, especially if you're doing more than one restart per submission and need to keep them separate. It's much easier to do on the fly during the simulation.

Phil


On 9/8/11 10:54 AM, "Michael Duda" <duda at ucar.edu> wrote:

Hi, All.

In the trunk code, we recently added an option to split output across
multiple files at the same time we added the time manager code. The new
namelist option config_frames_per_outfile defaults to 0, in which case
all output frames are written to a single output file whose name is
simply the name specified by config_output_name. However, if
config_frames_per_outfile is set to a positive integer n, at most n
frames will be written to an output file, and the (n+1)st frame will
cause a new output file to be created; obviously, this requires that we
provided a mechanism to automatically generate unique names for output
files, and the approach we've taken is to insert the time stamp of the
first frame in the file into the file's name, e.g.,
output.2011-09-08_00:00:00.nc. If it would be desirable to keep all
output frames for a run in a single file, one can simply set
config_frames_per_outfile to a very large integer -- larger than the
number of output frames that will be generated during the run -- and the
first output file will still have a time stamp embedded in its name. It
sounds like this functionality would be of help in the scenarios that
Sara described; would it simply be a matter of porting these changes
from the trunk into, e.g., the mpas_cam_coupling branch?

Cheers,
Michael


On Thu, Sep 08, 2011 at 10:39:21AM -0600, Todd Ringler wrote:
>
> Hi Everyone,
>
> More broadly, the issue that Sara raises is a common headache in climate modeling. A simulations might be composed of a dozen or more queue submissions that pick up where the last submission ended.
>
> Sara offered one possible remedy. I am guessing Mat has also come up with solutions and that Phil has had to address this more broadly within the CESM. Can others offer up ideas on how we should build in support for production simulations that need a large number of submissions/restarts to complete?
>
> Cheers,
> Todd
>
>
> On Sep 8, 2011, at 10:22 AM, Sara A. Rauscher wrote:
>
> > Hi,
> >
> > I mentioned this to Todd and he suggested I post my user experience to the developer team.
> >
> > I've been running cam-mpas, and revision 983 brought up a small run-time hassle. Is there any way to automatically put a unique name on mpas output files instead of generic names like "output.nc", "ocean.nc" without editing namelist.input every time? For example, they could be time stamped by model time (YYYY-MM-DD-HH as in CESM)  or number of time steps) or the real-world time at which the job is running.
> >
> > It's not a big deal during testing, but in production runs it starts to be a hassle. I have set my run script to capture the date from the latest CAM restart file and sed that into namelist.input so that the output files have a unique and meaningful name. It works fine but errors can still happen. In production runs, with one misnamed file, or writing over an output file, things get messy fast.
> >
> > thanks
> > cheers
> > sara
> >
> >> -------- Original Message --------
> >> Subject:   [mpas-developers] /home/subversion/mpas/model revision 983
> >> Date:      Wed, 7 Sep 2011 11:00:48 -0600 (MDT)
> >> From:      mpas-developers at ucar.edu
> >> Reply-To:  mpas-developers at ucar.edu
> >> To:        mpas-developers at ucar.edu
> >>
> >> ringler at lanl.gov 2011-09-07 11:00:48 -0600 (Wed, 07 Sep 2011)
> >>
> >> add default naming for I/O files
> >> Modified: branches/ocean_projects/timesplitting_mrp/namelist.input.ocean
> >> ===================================================================
> >> --- branches/ocean_projects/timesplitting_mrp/namelist.input.ocean 2011-09-07 15:53:33 UTC (rev 982)
> >> +++ branches/ocean_projects/timesplitting_mrp/namelist.input.ocean 2011-09-07 17:00:48 UTC (rev 983)
> >> @@ -8,6 +8,9 @@
> >>     config_stats_interval = 100
> >>  /
> >>  &io
> >> +   config_input_name = 'ocean.nc'
> >> +   config_output_name = 'output.nc'
> >> +   config_restart_name = 'restart.nc'
> >>  /
> >>  &restart
> >>     config_restart_interval = 10000000
> >>
> >>
> >> <Attached Message Part.txt>
> >
> > _______________________________________________
> > mpas-developers mailing list
> > mpas-developers at mailman.ucar.edu
> > http://mailman.ucar.edu/mailman/listinfo/mpas-developers
>

> _______________________________________________
> mpas-developers mailing list
> mpas-developers at mailman.ucar.edu
> http://mailman.ucar.edu/mailman/listinfo/mpas-developers

_______________________________________________
mpas-developers mailing list
mpas-developers at mailman.ucar.edu
http://mailman.ucar.edu/mailman/listinfo/mpas-developers



---
Correspondence/TSPA/DUSA EARTH
------------------------------------------------------------
Philip Jones                                pwjones at lanl.gov
Climate, Ocean and Sea Ice Modeling
Los Alamos National Laboratory
T-3 MS B216                                 Ph: 505-500-2699
PO Box 1663                                Fax: 505-665-5926
Los Alamos, NM 87545-1663



-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/mpas-developers/attachments/20110908/9500fd2f/attachment.html 


More information about the mpas-developers mailing list