<HTML>
<HEAD>
<TITLE>Re: [mpas-developers] mpas output file naming conventions</TITLE>
</HEAD>
<BODY>
<FONT FACE="Calibri, Verdana, Helvetica, Arial"><SPAN STYLE='font-size:12pt'><BR>
Michael,<BR>
<BR>
I think we’ll want a more general capability, including renaming the root based on experiment name and having one frame in each file with a simulation date stamp. While some of this could be handled via post-processing, it can be difficult to manage within a queue/script structure, especially if you’re doing more than one restart per submission and need to keep them separate. It’s much easier to do on the fly during the simulation.<BR>
<BR>
Phil<BR>
<BR>
<BR>
On 9/8/11 10:54 AM, "Michael Duda" <<a href="duda@ucar.edu">duda@ucar.edu</a>> wrote:<BR>
<BR>
</SPAN></FONT><BLOCKQUOTE><FONT FACE="Calibri, Verdana, Helvetica, Arial"><SPAN STYLE='font-size:12pt'>Hi, All.<BR>
<BR>
In the trunk code, we recently added an option to split output across<BR>
multiple files at the same time we added the time manager code. The new<BR>
namelist option config_frames_per_outfile defaults to 0, in which case<BR>
all output frames are written to a single output file whose name is<BR>
simply the name specified by config_output_name. However, if<BR>
config_frames_per_outfile is set to a positive integer n, at most n<BR>
frames will be written to an output file, and the (n+1)st frame will<BR>
cause a new output file to be created; obviously, this requires that we<BR>
provided a mechanism to automatically generate unique names for output<BR>
files, and the approach we've taken is to insert the time stamp of the<BR>
first frame in the file into the file's name, e.g.,<BR>
output.2011-09-08_00:00:00.nc. If it would be desirable to keep all<BR>
output frames for a run in a single file, one can simply set<BR>
config_frames_per_outfile to a very large integer -- larger than the<BR>
number of output frames that will be generated during the run -- and the<BR>
first output file will still have a time stamp embedded in its name. It<BR>
sounds like this functionality would be of help in the scenarios that<BR>
Sara described; would it simply be a matter of porting these changes<BR>
from the trunk into, e.g., the mpas_cam_coupling branch?<BR>
<BR>
Cheers,<BR>
Michael<BR>
<BR>
<BR>
On Thu, Sep 08, 2011 at 10:39:21AM -0600, Todd Ringler wrote:<BR>
><BR>
> Hi Everyone,<BR>
><BR>
> More broadly, the issue that Sara raises is a common headache in climate modeling. A simulations might be composed of a dozen or more queue submissions that pick up where the last submission ended.<BR>
><BR>
> Sara offered one possible remedy. I am guessing Mat has also come up with solutions and that Phil has had to address this more broadly within the CESM. Can others offer up ideas on how we should build in support for production simulations that need a large number of submissions/restarts to complete?<BR>
><BR>
> Cheers,<BR>
> Todd<BR>
><BR>
><BR>
> On Sep 8, 2011, at 10:22 AM, Sara A. Rauscher wrote:<BR>
><BR>
> > Hi,<BR>
> ><BR>
> > I mentioned this to Todd and he suggested I post my user experience to the developer team.<BR>
> ><BR>
> > I've been running cam-mpas, and revision 983 brought up a small run-time hassle. Is there any way to automatically put a unique name on mpas output files instead of generic names like "output.nc", "ocean.nc" without editing namelist.input every time? For example, they could be time stamped by model time (YYYY-MM-DD-HH as in CESM) or number of time steps) or the real-world time at which the job is running.<BR>
> ><BR>
> > It's not a big deal during testing, but in production runs it starts to be a hassle. I have set my run script to capture the date from the latest CAM restart file and sed that into namelist.input so that the output files have a unique and meaningful name. It works fine but errors can still happen. In production runs, with one misnamed file, or writing over an output file, things get messy fast.<BR>
> ><BR>
> > thanks<BR>
> > cheers<BR>
> > sara<BR>
> ><BR>
> >> -------- Original Message --------<BR>
> >> Subject: [mpas-developers] /home/subversion/mpas/model revision 983<BR>
> >> Date: Wed, 7 Sep 2011 11:00:48 -0600 (MDT)<BR>
> >> From: <a href="mpas-developers@ucar.edu">mpas-developers@ucar.edu</a><BR>
> >> Reply-To: <a href="mpas-developers@ucar.edu">mpas-developers@ucar.edu</a><BR>
> >> To: <a href="mpas-developers@ucar.edu">mpas-developers@ucar.edu</a><BR>
> >><BR>
> >> <a href="ringler@lanl.gov">ringler@lanl.gov</a> 2011-09-07 11:00:48 -0600 (Wed, 07 Sep 2011)<BR>
> >><BR>
> >> add default naming for I/O files<BR>
> >> Modified: branches/ocean_projects/timesplitting_mrp/namelist.input.ocean<BR>
> >> ===================================================================<BR>
> >> --- branches/ocean_projects/timesplitting_mrp/namelist.input.ocean 2011-09-07 15:53:33 UTC (rev 982)<BR>
> >> +++ branches/ocean_projects/timesplitting_mrp/namelist.input.ocean 2011-09-07 17:00:48 UTC (rev 983)<BR>
> >> @@ -8,6 +8,9 @@<BR>
> >> config_stats_interval = 100<BR>
> >> /<BR>
> >> &io<BR>
> >> + config_input_name = 'ocean.nc'<BR>
> >> + config_output_name = 'output.nc'<BR>
> >> + config_restart_name = 'restart.nc'<BR>
> >> /<BR>
> >> &restart<BR>
> >> config_restart_interval = 10000000<BR>
> >><BR>
> >><BR>
> >> <Attached Message Part.txt><BR>
> ><BR>
> > _______________________________________________<BR>
> > mpas-developers mailing list<BR>
> > <a href="mpas-developers@mailman.ucar.edu">mpas-developers@mailman.ucar.edu</a><BR>
> > <a href="http://mailman.ucar.edu/mailman/listinfo/mpas-developers">http://mailman.ucar.edu/mailman/listinfo/mpas-developers</a><BR>
><BR>
<BR>
> _______________________________________________<BR>
> mpas-developers mailing list<BR>
> <a href="mpas-developers@mailman.ucar.edu">mpas-developers@mailman.ucar.edu</a><BR>
> <a href="http://mailman.ucar.edu/mailman/listinfo/mpas-developers">http://mailman.ucar.edu/mailman/listinfo/mpas-developers</a><BR>
<BR>
_______________________________________________<BR>
mpas-developers mailing list<BR>
<a href="mpas-developers@mailman.ucar.edu">mpas-developers@mailman.ucar.edu</a><BR>
<a href="http://mailman.ucar.edu/mailman/listinfo/mpas-developers">http://mailman.ucar.edu/mailman/listinfo/mpas-developers</a><BR>
<BR>
</SPAN></FONT></BLOCKQUOTE><FONT FACE="Calibri, Verdana, Helvetica, Arial"><SPAN STYLE='font-size:12pt'><BR>
</SPAN></FONT><SPAN STYLE='font-size:12pt'><FONT FACE="Courier, Courier New"><BR>
---<BR>
Correspondence/TSPA/DUSA EARTH<BR>
------------------------------------------------------------<BR>
Philip Jones <a href="pwjones@lanl.gov">pwjones@lanl.gov</a><BR>
Climate, Ocean and Sea Ice Modeling<BR>
Los Alamos National Laboratory<BR>
T-3 MS B216 Ph: 505-500-2699<BR>
PO Box 1663 Fax: 505-665-5926<BR>
Los Alamos, NM 87545-1663<BR>
</FONT></SPAN><FONT SIZE="1"><FONT FACE="Helvetica, Verdana, Arial"><SPAN STYLE='font-size:9pt'><BR>
<BR>
</SPAN></FONT></FONT><FONT FACE="Calibri, Verdana, Helvetica, Arial"><SPAN STYLE='font-size:12pt'><BR>
</SPAN></FONT>
</BODY>
</HTML>