<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"News Gothic MT";
panose-1:2 11 5 4 2 2 3 2 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="EN-AU" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">Hi Joe<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">I’m late to this party but I have done this kind of thing in the past a lot, and Adam’s correct – you need to predefine the variable and its dimensions BEFORE you write the actual data, so it’ll
look something like this (adjust actual names as necessary):<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal">wrfpost = addfile(dir_out+file_out,"c") ;create new netCDF file<o:p></o:p></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">dimNam = (/”time”,”lat”,”lon”/)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">dimSiz = (/-1,nlat,nlon/)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">filedimdef(wrfpost, dimNam, dimSiz,(/True,False,False/))<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">dList = [/time, lat, lon/] ;I’,m lazy so I loop through all the dimension defs<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">do I = 0, 2<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">filevardef(wrfpost, dimNam(i),typeof(dList[i]),dimNam(i))<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">filevarattdef(wrfpost, dimNam(i), dList[i])<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">end do<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">slpAtt = </span>wrf_user_getvar(wrfout,"slp",0)/) ;get attributes, vartype from one of the files<span style="mso-fareast-language:EN-US"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">filevardef(wrfpost, “slp”,typeof(slpAtt), dimNams)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">filevarattdef(wrfpost, “slp”, slpAtt)<br>
<br>
<o:p></o:p></span></p>
<p class="MsoNormal">wrfpost->slp(0,:,:)=(/slpAtt/) ;might as well use it since you read it!<span style="mso-fareast-language:EN-US"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal">do n = 1, nTime-1<br>
wrfpost->slp(n,:,:)=(/wrf_user_getvar(wrfout,"slp",n)/) <br>
end do<o:p></o:p></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">Will<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span style="font-size:12.0pt;color:black">From: </span></b><span style="font-size:12.0pt;color:black">ncl-talk <ncl-talk-bounces@mailman.ucar.edu> on behalf of Adam Phillips via ncl-talk <ncl-talk@mailman.ucar.edu><br>
<b>Reply to: </b>Adam Phillips <asphilli@ucar.edu><br>
<b>Date: </b>Friday, 23 April 2021 at 2:24 am<br>
<b>To: </b>Joseph Zambon <jbzambon@ncsu.edu><br>
<b>Cc: </b>ncl-talk <ncl-talk@mailman.ucar.edu><br>
<b>Subject: </b>Re: [ncl-talk] Updating NetCDF File frame by frame vs all at once<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Hi Joe,<o:p></o:p></p>
<div>
<p class="MsoNormal">I think what is happening is that when you use this coding:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> wrfpost = addfile(dir_out+file_out,"c") ;create new netCDF file<o:p></o:p></p>
</div>
<div>
<div>
<p class="MsoNormal">do n = 0, nTime-1<br>
print(n)<br>
wrfpost->slp(n,:,:)=(/wrf_user_getvar(wrfout,"slp",n)/) <- Errors out<br>
end do<o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">the slp array is not preallocated space on the output .nc file, so NCL doesn't know to write the entire 3D slp array the first time you write to the slp array. The possible way around this is to create the .nc file the efficient way in
NCL, where arrays are preallocated. See method #2 here:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><a href="http://www.ncl.ucar.edu/Applications/write_netcdf.shtml">http://www.ncl.ucar.edu/Applications/write_netcdf.shtml<br>
</a><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><a href="http://www.ncl.ucar.edu/Applications/write_netcdf.shtml"><br>
</a><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">I have not created a .nc file this way, writing partially to an array like what I am proposing you do, but I do not see any reason why it wouldn't work. As your script is rather long, I would recommend trying a write and run a short test
script attempting to do this offline, before implementing it into your script.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Hope that helps!<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Adam<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">On Wed, Apr 21, 2021 at 12:55 PM Joseph Zambon via ncl-talk <<a href="mailto:ncl-talk@mailman.ucar.edu">ncl-talk@mailman.ucar.edu</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<p class="MsoNormal">NCL-Talk,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal">I'm attempting to modify the <a href="https://sundowner.colorado.edu/wrfout_to_cf/release_notes.html" target="_blank">wrfout_to_cf.ncl</a> script by Matt Higgins to loop over time. The existing script calculates
variables like SLP by ingesting the entire dataset and then spitting out the result. For smaller domains / fewer frames per outfile, this works fine. For my outfile, I have 601 timesteps and a domain of 1664x1544x50 meaning each 4-D variable takes up ~288GB,
which is way too much to work with efficiently in memory.<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I'm trying to re-structure the code so that it loops over time. E.g. at each timestep for SLP it will calculate SLP from 3-D fields (x,y,z) and then output a 2-D array (x,y) to save to a NetCDF. The problem comes when I try to save to
NetCDF, I get an error...<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">fatal:["Execute.c":8637]:Execute: Error occurred at or near line 461 in file wrfout_to_cf_timefix.ncl<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">when code is executed as...<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black"> do n = 0, nTime-1<br>
print(n)<br>
;slp(n,:,:)=(/wrf_user_getvar(wrfout,"slp",n)/)<br>
wrfpost->slp(n,:,:)=(/wrf_user_getvar(wrfout,"slp",n)/) <- Errors out<br>
end do<br>
;wrfpost->slp =slp</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">wrfpost = output file.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">slp = netcdf variable (601,1664,1544)<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">n = timestep<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">However, when I (less efficiently) run over the loop of 601 timesteps and save them to a 3-D SLP variable, then dump them into the NetCDF file, it works fine...<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"> do n = 0, nTime-1<br>
print(n)<br>
slp(n,:,:)=(/wrf_user_getvar(wrfout,"slp",n)/)<br>
;wrfpost->slp(n,:,:)=(/wrf_user_getvar(wrfout,"slp",n)/)<br>
end do<br>
wrfpost->slp =slp <- Works fine<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">AFAIK, the dimsizes are lined up correctly. Ideally, I would execute this code over each timestep, save the result to the NetCDF file, then rerun so at any point I only have 3-D/2-D and not 4-D/3-D arrays to handle to save memory. I know
I could ncks the wrfout file into smaller chunks but I'd rather figure out what's going on than work around it.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">The entire script is attached, for the sake of simplicity ignore the other 4-D variables. Once I have SLP figured out, the others should fall into place.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Please let me know what I'm doing that's stupid.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Thanks!<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">-Joe<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><br>
<br>
<o:p></o:p></p>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"News Gothic MT",sans-serif;color:black">Joseph B. Zambon, Ph.D.</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"News Gothic MT",sans-serif;color:black">Research Assistant Professor</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"News Gothic MT",sans-serif;color:black">Ocean Observing and Modeling Group</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"News Gothic MT",sans-serif;color:black">Department of Marine, Earth and Atmospheric Sciences</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"News Gothic MT",sans-serif;color:black">North Carolina State University</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"News Gothic MT",sans-serif;color:black">Jordan Hall rm 4140</span><span style="color:black"><o:p></o:p></span></p>
</div>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"News Gothic MT",sans-serif;color:black">919.515.7928</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black"><a href="mailto:jbzambon@ncsu.edu" target="_blank"><span style="font-size:10.5pt;font-family:"News Gothic MT",sans-serif">jbzambon@ncsu.edu</span></a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black"><a href="http://www.joezambon.com" target="_blank"><span style="font-size:10.5pt;font-family:"News Gothic MT",sans-serif">www.joezambon.com</span></a><o:p></o:p></span></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal">_______________________________________________<br>
ncl-talk mailing list<br>
<a href="mailto:ncl-talk@mailman.ucar.edu" target="_blank">ncl-talk@mailman.ucar.edu</a><br>
List instructions, subscriber options, unsubscribe:<br>
<a href="https://mailman.ucar.edu/mailman/listinfo/ncl-talk" target="_blank">https://mailman.ucar.edu/mailman/listinfo/ncl-talk</a><o:p></o:p></p>
</blockquote>
</div>
<p class="MsoNormal"><br clear="all">
<o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal">-- <o:p></o:p></p>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal"><span style="color:#888888">Adam Phillips </span><o:p></o:p></p>
</div>
<p class="MsoNormal"><span style="color:#888888">Associate Scientist, Climate and Global Dynamics Laboratory, NCAR</span><o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal"><span style="color:#888888"><a href="http://www.cgd.ucar.edu/staff/asphilli/" target="_blank">www.cgd.ucar.edu/staff/asphilli/</a> 303-497-1726
</span><o:p></o:p></p>
</div>
<div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<p style="font-size:10pt; line-height:10pt; font-family: Calibri,sans-serif;"><br>
<br>
This email is confidential, and is for the intended recipient only. Access, disclosure, copying, distribution, or reliance on any of it by anyone outside the intended recipient organisation is prohibited and may be a criminal offence. Please delete if obtained
in error and email confirmation to the sender. The views expressed in this email are not necessarily the views of the University of Tasmania, unless clearly intended otherwise.</p>
</body>
</html>