<div dir="ltr">This is great! Thank you so much for your help.<div>Regards</div><div>Archana</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Jan 19, 2016 at 12:02 PM, Mary Haley <span dir="ltr"><<a href="mailto:haley@ucar.edu" target="_blank">haley@ucar.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_default" style="font-size:small">Archana,</div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">I've been working on this for awhile now, and have something that runs, but is still not terribly fast. The simple fact is that you are trying to regrid to a rather large variable, and across 24 time steps.</div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">There are parts of your script that I'm puzzled by. One is the fact that your data is ordered lat x lon x time rather than time x lat x lon. This adds an unnecessary additional step of having to reorder your data, because the regridding routine is expecting time x lat x lon. Second, I'm not sure why you are defining "time" as an unlimited dimension on your file, but then you have named what appears to be the time dimension as "ZDim" instead.</div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">Also, it bears repeating that do loops are inefficient in scripting languages like NCL, Python, and IDL, and you want to keep things out of a do loop if possible. In your script, you are redefining the "T2_array" and "SWDOWN_array" variables every time in the loop, which is expensive. You are also regridding one time step at a time inside the loop, which means you are opening and closing a rather large NetCDF file that contains the weights file multiple times. I understand that you may be doing this to save memory, but I think you might be able to do this better by managing your memory better.</div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">Having said all this, I realize that ESMF_regrid_with_weights is part of the problem, because it's the routine that is reading the necessary variables off the NetCDF weights file in order to do the regridding. Since you had this inside the do loop, this means you are reading the same variables off the file again and again.</div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">To prevent having to open the weights file multiple times, I created a function that reads the necessary data off the weights file and stores it in a "list" variable. I then wrote a second function that uses this list to apply the variables in a sparse matrix calculation for the regridding. This sped things up and I am now able to get a regridded variable written to a new NetCDF file.</div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">Please try the attached script. You may need to modify the directory paths, but otherwise it should be runnable as-is.</div><span class="HOEnZb"><font color="#888888"><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">--Mary</div><div class="gmail_default" style="font-size:small"><br></div></font></span></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Jan 12, 2016 at 1:18 PM, Archana Dayalu <span dir="ltr"><<a href="mailto:adayalu@seas.harvard.edu" target="_blank">adayalu@seas.harvard.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Sorry, a couple more things:<div>1) To answer your question, Mary: I am using the setfileoption command to change file type.</div><div>2) I just re-ran the whole script a different way (i.e., I broke it up by common days of a month rather than entire months ... so instead of 12 independent loops of 28-31 iterations, I did 31 independent loops of 4-12 iterations). So far it seems to be completing successfully which indicates that it was a problem with too many calls to the ESMF regrid with weights function? But I don't understand why that would be so (if it were the issue).</div><div>Regards</div><span><font color="#888888"><div>Archana</div></font></span></div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Jan 12, 2016 at 3:04 PM, Archana Dayalu <span dir="ltr"><<a href="mailto:adayalu@seas.harvard.edu" target="_blank">adayalu@seas.harvard.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi Mary,<div>I just uploaded my code + data to the ftp site. The files I have uploaded are:</div><div>1) 31 files of type "<a href="http://aux2vprm_d01_2006-01-dd_hrlyavg.nc" target="_blank">aux2vprm_d01_2006-01-dd_hrlyavg.nc</a>" (where dd=two digit day in january), 2.5MB each</div><div>2) 1 weights file "<a href="http://T2SWDOWN_to_MODIS_bilinear.faster.nc" target="_blank">T2SWDOWN_to_MODIS_bilinear.faster.nc</a>" (this is kind of big ... ~6GB)</div><div>3) 1 file with the script "regrid_using_weights_T2SWDOWN_MOD_noplots.ncl" (edit the header of file to specify directories; all else should not require editing).</div><div>Other requested information: </div><div>ncl -V = 6.2.1<br></div><div>uname -a = Linux <a href="http://rclogin11.rc.fas.harvard.edu" target="_blank">rclogin11.rc.fas.harvard.edu</a> 2.6.32-431.17.1.el6.x86_64 #1 SMP Wed May 7 23:32:49 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux</div><div>Thanks very much! Please let me know if you have any trouble with the code/files. </div><div>Regards</div><div>Archana</div><div class="gmail_extra"><div><div><br><div class="gmail_quote">On Tue, Jan 12, 2016 at 11:29 AM, Mary Haley <span dir="ltr"><<a href="mailto:haley@ucar.edu" target="_blank">haley@ucar.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_default" style="font-size:small">Are you by any chance using setfileoption to change the write type to "NetCDF4"? </div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">I'm wondering if there's an issue with going into NetCDF4 mode, that may cause a problem if you stay in that mode when reading your weights file.</div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">Can you provide us with the full code and all the files needed to run it? You can use our ftp if the files are not too large:<br></div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default"><a href="http://www.ncl.ucar.edu/report_bug.shtml#HowToFTP" target="_blank">http://www.ncl.ucar.edu/report_bug.shtml#HowToFTP</a><span><font color="#888888"><br></font></span></div><span><font color="#888888"><div class="gmail_default"><br></div><div class="gmail_default" style="font-size:small"> </div><div class="gmail_default" style="font-size:small">--Mary</div><div class="gmail_default" style="font-size:small"><br></div></font></span></div><div class="gmail_extra"><br><div class="gmail_quote"><span>On Mon, Jan 11, 2016 at 2:09 PM, Archana Dayalu <span dir="ltr"><<a href="mailto:adayalu@seas.harvard.edu" target="_blank">adayalu@seas.harvard.edu</a>></span> wrote:<br></span><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div><div dir="ltr">Hi there,<div>I am using the ESMF tools to regrid a bunch of wrf-grid files to rectangular grid. I am running each month independently (running the script in 12 independent batches, where each batch corresponds to a month) but the days in a given month are done serially in that loop. My steps, as a quick summary, are:</div><div>(1) generate the weights once, valid for all variables for all files for the year. Then, for each of 365 daily temperature and radiation files:</div><div>(2) extract the daily T2 (Kelvin), SWDOWN variables which I previously defined as a 3D array (XDim,YDim,23)</div><div>(3) apply the weights to hourly T2, SWDOWN output for a year. This step involves a loop ... and the relevant part includes:</div><div>do hh=0,23</div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div>T2_regrid = ESMF_regrid_with_weights(T2,wgt_file_name,Opt)</div></blockquote><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div>SWDOWN_regrid = ESMF_regrid_with_weights(SWDOWN,wgt_file_name,Opt)</div><div>T2_array(:,:,hh) = T2_regrid</div></blockquote><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div>SWDOWN_array(:,:,hh) = SWDOWN_regrid</div></blockquote><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div>; I also have it print out min max for each variable for each hour regridded</div></blockquote><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div>end do</div></blockquote><div>(4) Write out the two regridded arrays to a daily netcdf file using method 1, Netcdf4, with file compression level set to 4</div><div><br></div><div>The problem: The script fails toward the end (e.g, around the 27th file of a month) with the following error message and begins to input nonsense Temperature and radiation values into the array and states the weights file cannot be opened (it does exist). When I re-run the failed time-period as a one-time regridding, it regrids just fine. Is ncl unhappy because of the high I/O? It doesn't fail at the netcdf generation step ... it seems to fail at the regridding step itself. Thanks for any help! The error output follows ... </div><div><br></div><div><div>Variable: hh</div><div>Type: integer</div><div>Total Size: 4 bytes</div><div> 1 values</div><div>Number of Dimensions: 1</div><div>Dimensions and sizes: [1]</div><div>Coordinates:</div><div>(0) 15</div><div>fatal:["NclNewHDF5.c":3787]:NclNewHDF5.c: Cannot open file: </n/regal/wofsy_lab/adayalu/<a href="http://T2SWDOWN_to_MODIS_bilinear.faster.nc" target="_blank">T2SWDOWN_to_MODIS_bilinear.faster.nc</a>></div><div>fatal:["NclAdvancedFile.c":2667]:_NclFileFillHLFS: Could not open (/n/regal/wofsy_lab/adayalu/<a href="http://T2SWDOWN_to_MODIS_bilinear.faster.nc" target="_blank">T2SWDOWN_to_MODIS_bilinear.faster.nc</a>)</div><div>(0) ESMF_regrid_with_weights: cannot open weight file '/n/regal/wofsy_lab/adayalu/<a href="http://T2SWDOWN_to_MODIS_bilinear.faster.nc" target="_blank">T2SWDOWN_to_MODIS_bilinear.faster.nc</a>'.</div><div>(0)</div><div>(0) TEMP at 2 M: min=239.284 max=299.333</div><div>(0)</div><div>(0) min=9.96921e+36 max=9.96921e+36</div><div>warning:VarVarWrite: rhs has no dimension name or coordinate variable, deleting name of lhs dimension number (0) and destroying coordinate var, use "(/../)" if this is not desired outcome</div><div>warning:VarVarWrite: rhs has no dimension name or coordinate variable, deleting name of lhs dimension number (1) and destroying coordinate var, use "(/../)" if this is not desired outcome</div><div>warning:["Execute.c":8578]:Execute: Error occurred at or near line 64 in file regrid_using_weights_T2SWDOWN_MOD_noplots.ncl</div><div><br></div><div>HDF5-DIAG: Error detected in HDF5 (1.8.13) thread 0:</div></div><div><div> #000: H5D.c line 461 in H5Dget_space(): not a dataset</div><div> major: Invalid arguments to routine</div><div> minor: Inappropriate type</div></div><div>(...then a series of similar error messages are output .... and then it continues .... and nonsense is output):</div><div><br></div><div><div>Variable: hh</div><div>Type: integer</div><div>Total Size: 4 bytes</div><div> 1 values</div><div>Number of Dimensions: 1</div><div>Dimensions and sizes: [1]</div><div>Coordinates:</div><div>(0) 17</div><div>(0)</div><div>(0) TEMP at 2 M: min=0.0114805 max=299.333</div><div>(0)</div><div>(0) DOWNWARD SHORT WAVE FLUX AT GROUND SURFACE: min=0 max=2.90338e-31</div></div><div><br></div><div><div><div> </div>
</div></div></div>
<br></div></div><span>_______________________________________________<br>
ncl-talk mailing list<br>
<a href="mailto:ncl-talk@ucar.edu" target="_blank">ncl-talk@ucar.edu</a><br>
List instructions, subscriber options, unsubscribe:<br>
<a href="http://mailman.ucar.edu/mailman/listinfo/ncl-talk" rel="noreferrer" target="_blank">http://mailman.ucar.edu/mailman/listinfo/ncl-talk</a><br>
<br></span></blockquote></div><br></div>
</blockquote></div><br><br clear="all"><div><br></div></div></div><span><font color="#888888">-- <br><div>____________________________________________________<br><b><font size="2"><span style="color:rgb(0,0,0)">Archana Dayalu</span></font></b><br><font style="color:rgb(0,0,0)" size="2">Graduate Student<br></font><font color="#888888"><font size="2"><span style="color:rgb(0,0,0)">Dept. of Earth and Planetary Sciences</span><br style="color:rgb(0,0,0)"><span style="color:rgb(0,0,0)">Harvard University</span><br style="color:rgb(0,0,0)"><span style="color:rgb(0,0,0)">24 Oxford Street #402</span><br style="color:rgb(0,0,0)"><span style="color:rgb(0,0,0)">Cambridge, MA 02138</span><br style="color:rgb(0,0,0)"><a style="color:rgb(0,0,153)" value="+16174966361">(617) 384-8206</a></font><br>
</font> </div>
</font></span></div></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div>____________________________________________________<br><b><font size="2"><span style="color:rgb(0,0,0)">Archana Dayalu</span></font></b><br><font style="color:rgb(0,0,0)" size="2">Graduate Student<br></font><font color="#888888"><font size="2"><span style="color:rgb(0,0,0)">Dept. of Earth and Planetary Sciences</span><br style="color:rgb(0,0,0)"><span style="color:rgb(0,0,0)">Harvard University</span><br style="color:rgb(0,0,0)"><span style="color:rgb(0,0,0)">24 Oxford Street #402</span><br style="color:rgb(0,0,0)"><span style="color:rgb(0,0,0)">Cambridge, MA 02138</span><br style="color:rgb(0,0,0)"><a style="color:rgb(0,0,153)" value="+16174966361">(617) 384-8206</a></font><br>
</font> </div>
</div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature">____________________________________________________<br><b><font size="2"><span style="color:rgb(0,0,0)">Archana Dayalu</span></font></b><br><font style="color:rgb(0,0,0)" size="2">Graduate Student<br></font><font color="#888888"><font size="2"><span style="color:rgb(0,0,0)">Dept. of Earth and Planetary Sciences</span><br style="color:rgb(0,0,0)"><span style="color:rgb(0,0,0)">Harvard University</span><br style="color:rgb(0,0,0)"><span style="color:rgb(0,0,0)">24 Oxford Street #402</span><br style="color:rgb(0,0,0)"><span style="color:rgb(0,0,0)">Cambridge, MA 02138</span><br style="color:rgb(0,0,0)"><a style="color:rgb(0,0,153)" value="+16174966361">(617) 384-8206</a></font><br>
</font> </div>
</div>