[ncl-talk] A Fatal Error Question: is this a bug in NCL?
Barry Lynn
barry.h.lynn at gmail.com
Wed Jul 19 12:09:00 MDT 2017
Hi Alan, Mary et al:
Thank you.
I feel like I have just climbed Everest. It was great that I had help
along the way.
Here is the program that includes the calculation of time using Alan's
cd_inv_string function, and *does it in a function as well.*
There are two warnings:
They are coming from the call to each of these functions -- I am not sure
why.
xy_start = get_ij_ncl(lat2d,lon2d,lat_beg,lon_beg)
xy_end = get_ij_ncl(lat2d,lon2d,lat_end,lon_end)
warning:Argument 2 of the current function or procedure was coerced to the
appropriate type and thus will not change if the function or procedure
modifies its value
warning:Argument 3 of the current function or procedure was coerced to the
appropriate type and thus will not change if the function or procedure
modifies its value
P.S: I have attached the get_ij.f program linked with WRAPIT get_ij.f
On Wed, Jul 19, 2017 at 6:16 PM, Alan Brammer <abrammer at albany.edu> wrote:
> with regard to accessing the time.
>
>
> initial_time = cd_inv_string( slp at initial_time, "%N/%D/%Y (%H:%M)”)
> print(cd_calendar(initial_time, 0)) ;; make sure nothing went wrong.
>
> forecast_time = initial_time + slp at forecast_time ;; assumes units are
> hours for both variables and that forecast_time is numeric not string
> forecast_time at units = initial_time at units
>
>
>
> http://www.ncl.ucar.edu/Document/Functions/User_contributed/cd_inv_string.
> shtml
> n.b. cd_inv_string is 6.4.0+ and needs to be loaded at the top of the
> script.
>
> I can’t access bitbucket on the network I’m on at the moment but I think
> this was my latest version of the function before it was adopted by ncl.
> Using 6.4.0 would be recommended though.
>
> https://bitbucket.org/a1brammer/ncl_functions/raw/78dea35d95946425e9054cdc9286ae3db88d9585/cd_inv_string.ncl
>
>
>
>
>
>
>
>
> On Jul 19, 2017, at 11:02 AM, Barry Lynn <barry.h.lynn at gmail.com> wrote:
>
> Hi Mary:
>
> Thank you for the pointers:
>
> I assign strings in both res1_list and res2_list. The first prints the
> variable type at the top of the map, while the latter does not.
>
> in res1_list
>
> res at gsnLeftString = "GEFS Temp (C), MSLP (mb)"
>
> res at gsnRightString = ""
>
>
> in res2_list, it does not.
>
> res at gsnLeftString = "GEFS Hum (%), SLP (mb)"
>
> res at gsnRightString = ""
>
> even though I copy: copy_VarCoords(rh,rh_ave)
>
> So, this is a bit of a mystery.
>
> Getting back to my time question:
>
> I think I can do it by reading the initial time from any of the initial
> variables, and then adding the forecast time.
>
> However, I need some help on how to read the variable information
>
> forecast_time : 12
>
> initial_time : 07/19/2017 (00:00)
>
> from printVarSummary(slp).
>
>
> I need the string initial_time and forecast_time. *I am not sure how to
> access these variables within the printVarSummary.*
>
> Thanks,
>
> P.S. If we solve this problem, I "promise" to create a function!
>
>
>
>
>
>
>
>
>
> On Wed, Jul 19, 2017 at 5:43 PM, Mary Haley <haley at ucar.edu> wrote:
>
>> Hi Barry,
>>
>> Good work on the script!
>>
>> If you don't get long_name and units subtitles on your plot, it's likely
>> because your variable doesn't have any metadata attached to it.
>>
>> You should do a printVarSummary of the variable, which I assume is
>> "rh_ave". This will tell you if you have any metadata.
>>
>> If the metadata is not there, then you need to look at the variables
>> being used to do the calculation. I do see that you have:
>>
>> rh_ave = rh
>>
>> so you would need to check if "rh" has metadata, because if it does, it
>> should have been copied to "rh_ave".
>>
>> Calculations cause metadata to *not* be copied to the variable on the
>> left of the "=", but they won't remove any metadata that already exists.
>>
>> I can't answer your question about the times, because I don't know what
>> your gefs_time.file
>> ASCII file looks like, but I don't see any harm in what you're doing
>> if it works. Maybe Bill can weigh in about how to calculate forecast time
>> off a WRF file.
>>
>> However, I think you can turn all the code that is used to calculate the
>> new time array into a function, and move the function call to outside the
>> do loop.
>>
>> --Mary
>>
>>
>> On Wed, Jul 19, 2017 at 7:39 AM, Barry Lynn <barry.h.lynn at gmail.com>
>> wrote:
>>
>>> Hello:
>>>
>>> My code still runs out of memory, but not nearly as fast. This is
>>> helpful by itself.
>>>
>>> I have included two version of the code for users who desire to make
>>> GEFS mean map plots. You have to create GEFS/GEFS_XX directories where XX
>>> is the GEFS member.
>>>
>>> 1) Plotting sea level values of mean GEFS temperatures, humidity, winds,
>>> and sea-level pressure.
>>>
>>> 2) 700 mb mean GEFS temperatures, humidity, winds, and height.
>>>
>>> I have modified the code sent by Mary to skip over missing files. This
>>> step goes way back to a previous ncl-talk (thanks to Guido who gave a
>>> suggestion how to skip over missing files). Without it, the code can fail
>>> if a file is missing.
>>>
>>> I am not sure why the second map does not show a variable title, like
>>> the first. I haven't had a chance to check timings, yet.
>>>
>>> I don't see an easy way to take the initial time in the forecast
>>> variable "summary," include the forecast time in hours, and get the real
>>> time. So, I still calculate times externally. I have included
>>> "variable.file" in case someone has a suggestion. We would need to grab
>>> the actual time, and then find the new time by adding the number of hours.
>>>
>>> Barry
>>>
>>> On Tue, Jul 18, 2017 at 6:28 PM, Mary Haley <haley at ucar.edu> wrote:
>>>
>>>> Hi Barry,
>>>>
>>>> We could provide you with a profiling version of NCL, in which you run
>>>> your script like normal, but then ncl produces a supplemental file with
>>>> information about how long certain code segments are taking. It breaks it
>>>> down by percentage of time spent in various sections.
>>>>
>>>>
>>>> This profiling information is a bit clunky because you have to run a
>>>> script on it and then view it from a browser, but these extra steps may be
>>>> worth it in order to find out where the code is slow.
>>>>
>>>> To see a sample of how the profiler works, go to:
>>>>
>>>> http://www.ncl.ucar.edu/Document/Tools/Profiler/multiple_plots/
>>>>
>>>> and click on any one of the *.xml (the "multiple_plots_ncl__multiple_
>>>> plots.xml
>>>> <http://www.ncl.ucar.edu/Document/Tools/Profiler/multiple_plots/multiple_plots_ncl__multiple_plots.xml>"
>>>> is the main one and a good place to start)
>>>> . Look for lines that are colored
>>>>
>>>> in red, as these indicate where more time is being spent.
>>>>
>>>> You can also use "get_cpu_time()" to track the timings yourself. For
>>>> example, if you want to time how long it takes to create each set of panel
>>>> plots, then you would have code that looks like this:
>>>>
>>>> start_time = get_cpu_time()
>>>> plot(0) = gsn_csm_contour_map(wks,t_ave,res1)
>>>> plot(1) = gsn_csm_contour_map(wks,rh_ave,res2)
>>>> plotB = gsn_csm_vector(wks,u_ave_new(::4,::4), \
>>>> v_ave_new(::4,::4),vecres)
>>>> plotD = gsn_csm_contour(wks,z_ave_new,res3)
>>>> overlay(plot(0),plotB)
>>>> overlay(plot(1),plotD)
>>>> gsn_panel(wks,plot,(/2,1/),resP) ; now draw as one plot
>>>> end_time = get_cpu_time()
>>>> print("Elapsed time for plotting = " + (end_time-start_time)
>>>>
>>>> If you plan to do a lot of these timings, then I recommend a function.
>>>> :-)
>>>>
>>>> procedure print_elapsed_time(stime,etime,title)
>>>> begin
>>>> print("==================================================")
>>>> print("Elapsed time for " + title + ": " + (etime-stime))
>>>> print("==================================================")
>>>> end
>>>>
>>>> Then your code would be:
>>>>
>>>> start_plot_time = get_cpu_time()
>>>> plot(0) = gsn_csm_contour_map(wks,t_ave,res1)
>>>> plot(1) = gsn_csm_contour_map(wks,rh_ave,res2)
>>>> plotB = gsn_csm_vector(wks,u_ave_new(::4,::4), \
>>>> v_ave_new(::4,::4),vecres)
>>>> plotD = gsn_csm_contour(wks,z_ave_new,res3)
>>>> overlay(plot(0),plotB)
>>>> overlay(plot(1),plotD)
>>>> gsn_panel(wks,plot,(/2,1/),resP) ; now draw as one plot
>>>> end_plot_time = get_cpu_time()
>>>> print_elapsed time(start_plot_time,end_plot_time,"Plotting")
>>>>
>>>> Note that I changed the name from "start_time/end_time to
>>>> start_plot_time/end_plot_time. This is in case you do a bunch of these,
>>>> then you can print all the timings at the end, and it also helps keep track
>>>> of what timings these are associated with.
>>>>
>>>> start_read_time = get_cpu_time()
>>>> .... Code that calls addfile and reads data
>>>> end_read_time = get_cpu_time()
>>>> ...
>>>> start_ave_time = get_cpu_time()
>>>> .... code that calls average_subset function a bunch of times
>>>> end_ave_time = get_cpu_time()
>>>> ...
>>>> start_plot_time = get_cpu_time()
>>>> ,,, code that creates and draws plots ...
>>>> end_plot_time = get_cpu_time()
>>>>
>>>> print_elapsed time(start_read_time,end_read_time,"Reading data")
>>>> print_elapsed time(start_ave_time,end_ave_time,"Averages")
>>>> print_elapsed time(start_plot_time,end_plot_time,"Plotting")
>>>>
>>>> Of course, you don't have to put all the elapsed timing prints at the
>>>> end. You can print them out right when the code is done, so you
>>>> immediately have the information. It just depends on what you are
>>>> interested in.
>>>>
>>>> As a side, and Dave or Rick may want to correct me on my understanding
>>>> on this, but I think one thing that can make your code slow is that you
>>>> have a lot of print statements. Every time you do something like this:
>>>>
>>>> print("x = " + x)
>>>>
>>>> versus this:
>>>>
>>>> print(x)
>>>>
>>>> I believe a string gets stored somewhere in memory. If you have a lot
>>>> of these, then NCL will get noticeably slower and slower, especially if you
>>>> are doing this inside do loops with a lot of iterations. I don't know if
>>>> you really have enough print statements for this to be a problem, but it's
>>>> worth making a mental note of this issue.
>>>>
>>>> --Mary
>>>>
>>>>
>>>> On Mon, Jul 17, 2017 at 8:06 PM, Barry Lynn <barry.h.lynn at gmail.com>
>>>> wrote:
>>>>
>>>>> Dear Mary:
>>>>>
>>>>> Thank you. Very helpful!
>>>>>
>>>>> I had noticed that my code was running very, very slowly, but being
>>>>> relatively "new" to NCL programming was not sure why. (Also, it was, as
>>>>> you noted, running out of memory since I "dialed back" the RAM on my
>>>>> account.)
>>>>>
>>>>> I will try your code and compare, and get back to the list about the
>>>>> timing differences.
>>>>>
>>>>> Barry
>>>>>
>>>>> On Tue, Jul 18, 2017 at 12:26 AM, Mary Haley <haley at ucar.edu> wrote:
>>>>>
>>>>>> Hi Barry,
>>>>>>
>>>>>> Rick is out this afternoon.
>>>>>>
>>>>>> Part of what Rick is suggesting is that if you are done with a
>>>>>> variable, then you should delete it in order to free up memory for later
>>>>>> calculations.
>>>>>>
>>>>>> If I may provide some honest comments about your script overall: if
>>>>>> you use some simple "clean coding practices", this will make it *much*
>>>>>> easier for you to debug your own code, and it will certainly make it easier
>>>>>> for other people to debug it. It will also help you see where you might
>>>>>> have some memory issues.
>>>>>>
>>>>>>
>>>>>> Here are some suggestions for improving your script:
>>>>>>
>>>>>> *[1] Use "delete" to remove variables you don't need any more:*
>>>>>>
>>>>>> t_ave=t_ave-273.16 ; convert to C
>>>>>>
>>>>>> z_ave=z_ave/10. ; convert to dam
>>>>>>
>>>>>> printMinMax(z_ave,False)
>>>>>> u = u * 3.6
>>>>>> v = v * 3.6
>>>>>> copy_VarCoords(u,u_ave) ; copy coord
>>>>>> vars to speed
>>>>>> copy_VarCoords(v,v_ave) ; copy coord
>>>>>> vars to speed
>>>>>> copy_VarCoords(z,z_ave) ; copy coord
>>>>>> vars to speed
>>>>>> copy_VarCoords(t,t_ave) ; copy coord
>>>>>> vars to speed
>>>>>> copy_VarCoords(rh,rh_ave) ; copy
>>>>>> coord vars to speed
>>>>>>
>>>>>> If you no longer need the "u", "v", "z", etc variables, then delete
>>>>>> them:
>>>>>>
>>>>>> t_ave=t_ave-273.16 ; convert to C
>>>>>>
>>>>>> z_ave=z_ave/10. ; convert to dam
>>>>>>
>>>>>> printMinMax(z_ave,False)
>>>>>> u = u * 3.6
>>>>>> v = v * 3.6
>>>>>> copy_VarCoords(u,u_ave) ; copy coord
>>>>>> vars to speed
>>>>>> copy_VarCoords(v,v_ave) ; copy coord
>>>>>> vars to speed
>>>>>> copy_VarCoords(z,z_ave) ; copy coord
>>>>>> vars to speed
>>>>>> copy_VarCoords(t,t_ave)
>>>>>> copy_VarCoords(rh,rh_ave)
>>>>>> delete([/u,v,z,t,rh/]) ; no longer
>>>>>> needed
>>>>>>
>>>>>> *[2] You are reading several variables and not doing anything with
>>>>>> them but printing them out. Reading them in uses memory. *
>>>>>>
>>>>>> If all you need to do is print them, then print them directly. An
>>>>>> example:
>>>>>>
>>>>>> lv0 = fin->lv_ISBL0
>>>>>> print("lv0 = " + lv0)
>>>>>> lv8 = fin->lv_ISBL8
>>>>>> print("lv8 = " + lv8)
>>>>>> lv10 = fin->lv_ISBL10
>>>>>> print("lv10 = " + lv10)
>>>>>>
>>>>>> Do this instead, to save memory:
>>>>>>
>>>>>> print("lv0 = " + fin->lv_ISBL0)
>>>>>> print("lv8 = " + fin->lv_ISBL8)
>>>>>> print("lv10 = " + fin->lv_ISBL10)
>>>>>>
>>>>>>
>>>>>> *[3] Avoid putting unchanged code inside do loops.*
>>>>>>
>>>>>> Your script has two do loops with a bunch of code inside of them that
>>>>>> doesn't change.
>>>>>>
>>>>>> Do loops can be slow, so it's important to keep as much code
>>>>>> *outside* of a do loop as possible.
>>>>>>
>>>>>> As an example, these are your two outside do loops:
>>>>>>
>>>>>> do n=0,n_files -1,1
>>>>>> do i_dir = 0,n_dirWRF-1
>>>>>>
>>>>>> Inside these two loops, you have code that looks like this:
>>>>>>
>>>>>> res1 = True
>>>>>> res1 at mpDataBaseVersion = "Ncarg4_1" ; choose more
>>>>>> recent database
>>>>>> . . .
>>>>>> res1 at mpNationalLineThicknessF = 3.0
>>>>>> res1 at mpGeophysicalLineThicknessF = 2.0
>>>>>> . . .
>>>>>>
>>>>>> res1 at tiMainString = time_var
>>>>>> res1 at gsnDraw = False ; don't draw
>>>>>>
>>>>>> res1 at cnLevelSelectionMode = "ExplicitLevels" ; set explicit
>>>>>> contour levels
>>>>>> . . .
>>>>>> res1 at cnInfoLabelOn = False ; turn off
>>>>>> contour info label
>>>>>> res1 at lbAutoManage = False ; control label bar
>>>>>> res1 at lbOrientation
>>>>>> = "Vertical"
>>>>>> . . .
>>>>>>
>>>>>> res1 at lbTopMarginF = 0.001
>>>>>> res2 = True
>>>>>> res2 = True
>>>>>> . . .
>>>>>> res2 at mpNationalLineThicknessF = 3.0
>>>>>> res2 at mpGeophysicalLineThicknessF = 2.0
>>>>>> res2 at tiMainString = time_var
>>>>>> res2 at gsnDraw = False ; don't draw
>>>>>>
>>>>>> . . .
>>>>>> res2 at lbOrientation = "Vertical"
>>>>>> res2 at pmLabelBarSide = "Right"
>>>>>> res2 at lbLabelAutoStride = True
>>>>>> res2 at pmLabelBarWidthF = 0.20
>>>>>> res2 at pmLabelBarHeightF = 0.7
>>>>>> . . .
>>>>>>
>>>>>> None of this code changes inside the do loops, but yet it is getting
>>>>>> set again and again for every iteration of the two do loops. In order to
>>>>>> speed things up, move as much code as you can to outside the do loop, and
>>>>>> only keep things that actually change inside the do loop.
>>>>>>
>>>>>> I recommend creating functions to set the various resource lists:
>>>>>> set_res1_list, set_res2_list, etc. This may not necessarily save memory,
>>>>>> but it makes your code much cleaner.
>>>>>>
>>>>>> *[4] Use functions for cleaner code.*
>>>>>>
>>>>>> You have code that is repeated in multiple locations. This can cause
>>>>>> extra memory usage. If you turn these repeated code segments into NCL
>>>>>> functions, then this can save some memory.
>>>>>>
>>>>>> An example, you have:
>>>>>>
>>>>>> i_loc = 1
>>>>>> j_loc = 1
>>>>>> GET_IJ::get_ij(lat2d,lon2d,lat_beg,lon_beg,i_loc,j_loc,nj,ni)
>>>>>> x_start = i_loc
>>>>>> y_start = j_loc
>>>>>>
>>>>>> Granted, this doesn't use a lot of memory, but every time you copy
>>>>>> these 5 lines, it makes your script harder to read, and you will slowly use
>>>>>> more memory if you do this kind of thing frequently.
>>>>>>
>>>>>> Turn the above five lines into a function:
>>>>>>
>>>>>> function get_ij_ncl(lat2d,lon2d,lat_beg,lon_beg)
>>>>>> local nj, ni
>>>>>> begin
>>>>>> i_loc = 1
>>>>>> j_loc = 1
>>>>>> nj = dimsizes(lat2d(:,0))
>>>>>> ni = dimsizes(lat2d(0,:))
>>>>>> GET_IJ::get_ij(lat2d,lon2d,lat_beg,lon_beg,i_loc,j_loc,nj,ni)
>>>>>> return([/i_loc,j_loc/])
>>>>>> end
>>>>>>
>>>>>> and then call the function with:
>>>>>>
>>>>>> xy_start = get_ij_ncl(lat2d,lon2d,lat_beg,lon_beg)
>>>>>>
>>>>>> You will then need to change "x_start" to xy_start(0) and "y_start"
>>>>>> to xy_start(1)
>>>>>>
>>>>>> *[5] Another place where a function would be helpful:*
>>>>>>
>>>>>> u_ave_new = u_ave(xy_start(1):xy_end(1),xy_start(0):xy_end(0))
>>>>>> v_ave_new = v_ave(xy_start(1):xy_end(1),xy_start(0):xy_end(0))
>>>>>> lat2d_new = lat2d(xy_start(1):xy_end(1),xy_start(0):xy_end(0))
>>>>>> lon2d_new = lon2d(xy_start(1):xy_end(1),xy_start(0):xy_end(0))
>>>>>> ; plotB = gsn_csm_vector(wks,u_ave_new,v_ave_new,vecres)
>>>>>>
>>>>>> ;;;opy_VarCoords(dumb_ave,u_ave_new)
>>>>>>
>>>>>> ; copy_VarCoords(dumb_ave,v_ave_new)
>>>>>>
>>>>>> u_ave_new!0 = "lat"
>>>>>> u_ave_new!1 = "lon"
>>>>>> u_ave_new&lat= lat(xy_start(1):xy_end(1))
>>>>>> u_ave_new&lon= lon(xy_start(0):xy_end(0))
>>>>>> v_ave_new!0 = "lat"
>>>>>> v_ave_new!1 = "lon"
>>>>>> v_ave_new&lat= lat(xy_start(1):xy_end(1))
>>>>>> v_ave_new&lon= lon(xy_start(0):xy_end(0))
>>>>>>
>>>>>> Create a function to do the subsetting and attaching of the metadata:
>>>>>>
>>>>>> function average_subset(x,lat,lon,xy_start,xy_end)
>>>>>> begin
>>>>>> x_ave_new = x(xy_start(1):xy_end(1),xy_start(0):xy_end(0))
>>>>>> x_ave_new!0 = "lat"
>>>>>> x_ave_new!1 = "lon"
>>>>>> x_ave_new&lat= lat(xy_start(1):xy_end(1))
>>>>>> x_ave_new&lon= lon(xy_start(0):xy_end(0))
>>>>>> return(x_ave_new)
>>>>>> end
>>>>>>
>>>>>> so now you can replace the above code with two lines:
>>>>>>
>>>>>> u_ave_new = average_subset(u_ave,lat,lon,xy_start,xy_end)
>>>>>> v_ave_new = average_subset(v_ave,lat,lon,xy_start,xy_end)
>>>>>>
>>>>>> *[6] You created "lat2d_new" and "lon2d_new" but never use them.
>>>>>> Again, this is using up memory for no purpose. Remove those two lines.*
>>>>>>
>>>>>> *[7] If you are propagating a smaller array to a larger array, as is
>>>>>> the case here:*
>>>>>>
>>>>>> lat2d = z
>>>>>> lon2d = z
>>>>>> do j = 0,nj-1
>>>>>> do i = 0,ni-1
>>>>>> lat2d(j,i)=lat(j)
>>>>>> lon2d(j,i)=lon(i)
>>>>>> end do
>>>>>> end do
>>>>>>
>>>>>> then use conform_dims instead of a do loop:
>>>>>>
>>>>>> lat2d = conform_dims(dims2d,lat,0)
>>>>>> lon2d = conform_dims(dims2d,lon,1)
>>>>>>
>>>>>> Note that I no longer need the "lat2d = z" or "lon2d = z" code.
>>>>>>
>>>>>> [8] You have this code, and I'm not sure why:
>>>>>>
>>>>>> a = addfile(filename,"r")
>>>>>> fin = a
>>>>>>
>>>>>> This seems like an unnecessary copy of "a". Why not simply do:
>>>>>>
>>>>>> fin = addfile(filename,"r")
>>>>>>
>>>>>> *[9] The "gsn_define_colormap" call should not be inside the do loop.
>>>>>> Call this right after you call gsn_open_wks.*
>>>>>>
>>>>>> *[10] There's some random code I don't understand. For example:*
>>>>>>
>>>>>> copy_VarCoords(z,lat2d)
>>>>>> copy_VarCoords(z,lon2d)
>>>>>>
>>>>>> lat2d and lon2d don't need to have z's coordinate arrays attached to
>>>>>> them for any reason that I can see, so I think you can remove those two
>>>>>> lines.
>>>>>>
>>>>>> *[11] Every time you have a do loop or an if statement, indent all
>>>>>> the code inside *consistently* so you can more easily see what part of the
>>>>>> code you're in.*
>>>>>>
>>>>>> *[12] Don't make extra copies of variables unless you really need to.*
>>>>>>
>>>>>> For example, you have:
>>>>>>
>>>>>> filename = all_files(n)
>>>>>>
>>>>>> There's really no need to create "filename" since you can just use
>>>>>> "all_files(n)".
>>>>>> I've tried to clean up your code to show you what I'm talking about.
>>>>>> The attached script likely won't run because I don't have your data to test
>>>>>> it. But, hopefully you can see what I'm doing and mimic this in your code.
>>>>>>
>>>>>> --Mary
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Mon, Jul 17, 2017 at 11:40 AM, Barry Lynn <barry.h.lynn at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi Rick:
>>>>>>>
>>>>>>> Thank you for your suggestion.
>>>>>>>
>>>>>>> Could you please give me some guidelines on how to delete resources.
>>>>>>>
>>>>>>> I have included the script. If you can provide an example, I can
>>>>>>> take it from there.
>>>>>>>
>>>>>>> Thank you,
>>>>>>>
>>>>>>> Barry
>>>>>>>
>>>>>>> On Mon, Jul 17, 2017 at 5:22 PM, Rick Brownrigg <brownrig at ucar.edu>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi Barry,
>>>>>>>>
>>>>>>>> On Linux, an errno=12 is an "out of memory" error. I doubt that
>>>>>>>> its related directly to the systemfunc() call, but rather due to memory
>>>>>>>> being consumed by your script in previous iterations of the loop. You
>>>>>>>> might check if there are variables that can be freed (i.e., deleted())
>>>>>>>> after each iteration.
>>>>>>>>
>>>>>>>> HTH...
>>>>>>>> Rick
>>>>>>>>
>>>>>>>> On Sun, Jul 16, 2017 at 8:56 PM, Barry Lynn <barry.h.lynn at gmail.com
>>>>>>>> > wrote:
>>>>>>>>
>>>>>>>>> Hi:
>>>>>>>>>
>>>>>>>>> I have a program that does multiple loops.
>>>>>>>>>
>>>>>>>>> I had this error:
>>>>>>>>>
>>>>>>>>> 0) i_dir = 6
>>>>>>>>>
>>>>>>>>> fatal:systemfunc: cannot create child process:[errno=12]
>>>>>>>>>
>>>>>>>>> fatal:["Execute.c":8640]:Execute: Error occurred at or near line
>>>>>>>>> 84 in file ./plot_loop_700mb.ncl
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> This occurred when reading the file from the sixth directory of 20.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Here was the read from the 5th directory:
>>>>>>>>>
>>>>>>>>> (0) i_dir = 5
>>>>>>>>>
>>>>>>>>> (0) all_files(n) = /home/cust100021_vol1/barry/cu
>>>>>>>>> st100021_vol2/GEFS/GEFS_05/gep05.t00z.pgrb2b.0p50.f084.grb
>>>>>>>>>
>>>>>>>>> Here is a listing of both files:
>>>>>>>>>
>>>>>>>>> They are both present, but the second attempted
>>>>>>>>> allocation/definition of the file fails with the read that follows:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> [barry at cust100021-login1 GEFS]$ ls -ltr
>>>>>>>>> /home/cust100021_vol1/barry/cust100021_vol2/GEFS/GEFS_05/gep
>>>>>>>>> 05.t00z.pgrb2b.0p50.f084.grb
>>>>>>>>>
>>>>>>>>> -rw-r--r-- 1 barry cust100021 78985320 Jul 16 04:58
>>>>>>>>> /home/cust100021_vol1/barry/cust100021_vol2/GEFS/GEFS_05/gep
>>>>>>>>> 05.t00z.pgrb2b.0p50.f084.grb
>>>>>>>>>
>>>>>>>>> [barry at cust100021-login1 GEFS]$ ls -ltr
>>>>>>>>> /home/cust100021_vol1/barry/cust100021_vol2/GEFS/GEFS_06/gep
>>>>>>>>> 06.t00z.pgrb2b.0p50.f084.grb
>>>>>>>>>
>>>>>>>>> -rw-r--r-- 1 barry cust100021 80466702 Jul 16 04:58
>>>>>>>>> /home/cust100021_vol1/barry/cust100021_vol2/GEFS/GEFS_06/gep
>>>>>>>>> 06.t00z.pgrb2b.0p50.f084.grb
>>>>>>>>>
>>>>>>>>> Here is the read:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> print("i_dir = " + i_dir)
>>>>>>>>>
>>>>>>>>> diri = dirWRF(i_dir) + "/"
>>>>>>>>>
>>>>>>>>> ; define individual file read
>>>>>>>>>
>>>>>>>>> all_files = systemfunc("ls " + diri + "ge*pgrb2b*grb")
>>>>>>>>>
>>>>>>>>> print("all_files(n) = " + all_files(n))
>>>>>>>>>
>>>>>>>>> filename = all_files(n)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Is the problem that I make this allocation multiple times, each
>>>>>>>>> directory, each file? Perhaps I should read all files at once into a two
>>>>>>>>> dimensional filename array?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Barry H. Lynn, Ph.D
>>>>>>>>> Senior Lecturer,
>>>>>>>>> The Institute of the Earth Science,
>>>>>>>>> The Hebrew University of Jerusalem,
>>>>>>>>> Givat Ram, Jerusalem 91904, Israel
>>>>>>>>> Tel: 972 547 231 170
>>>>>>>>> Fax: (972)-25662581
>>>>>>>>>
>>>>>>>>> C.E.O, Weather It Is, LTD
>>>>>>>>> Weather and Climate Focus
>>>>>>>>> http://weather-it-is.com
>>>>>>>>> Jerusalem, Israel
>>>>>>>>> Local: 02 930 9525
>>>>>>>>> Cell: 054 7 231 170
>>>>>>>>> Int-IS: x972 2 930 9525
>>>>>>>>> US 914 432 3108 <(914)%20432-3108>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> ncl-talk mailing list
>>>>>>>>> ncl-talk at ucar.edu
>>>>>>>>> List instructions, subscriber options, unsubscribe:
>>>>>>>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Barry H. Lynn, Ph.D
>>>>>>> Senior Lecturer,
>>>>>>> The Institute of the Earth Science,
>>>>>>> The Hebrew University of Jerusalem,
>>>>>>> Givat Ram, Jerusalem 91904, Israel
>>>>>>> Tel: 972 547 231 170
>>>>>>> Fax: (972)-25662581
>>>>>>>
>>>>>>> C.E.O, Weather It Is, LTD
>>>>>>> Weather and Climate Focus
>>>>>>> http://weather-it-is.com
>>>>>>> Jerusalem, Israel
>>>>>>> Local: 02 930 9525
>>>>>>> Cell: 054 7 231 170
>>>>>>> Int-IS: x972 2 930 9525
>>>>>>> US 914 432 3108 <(914)%20432-3108>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> ncl-talk mailing list
>>>>>>> ncl-talk at ucar.edu
>>>>>>> List instructions, subscriber options, unsubscribe:
>>>>>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Barry H. Lynn, Ph.D
>>>>> Senior Lecturer,
>>>>> The Institute of the Earth Science,
>>>>> The Hebrew University of Jerusalem,
>>>>> Givat Ram, Jerusalem 91904, Israel
>>>>> Tel: 972 547 231 170
>>>>> Fax: (972)-25662581
>>>>>
>>>>> C.E.O, Weather It Is, LTD
>>>>> Weather and Climate Focus
>>>>> http://weather-it-is.com
>>>>> Jerusalem, Israel
>>>>> Local: 02 930 9525
>>>>> Cell: 054 7 231 170
>>>>> Int-IS: x972 2 930 9525
>>>>> US 914 432 3108 <(914)%20432-3108>
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Barry H. Lynn, Ph.D
>>> Senior Lecturer,
>>> The Institute of the Earth Science,
>>> The Hebrew University of Jerusalem,
>>> Givat Ram, Jerusalem 91904, Israel
>>> Tel: 972 547 231 170
>>> Fax: (972)-25662581
>>>
>>> C.E.O, Weather It Is, LTD
>>> Weather and Climate Focus
>>> http://weather-it-is.com
>>> Jerusalem, Israel
>>> Local: 02 930 9525
>>> Cell: 054 7 231 170
>>> Int-IS: x972 2 930 9525
>>> US 914 432 3108 <(914)%20432-3108>
>>>
>>
>>
>
>
> --
> Barry H. Lynn, Ph.D
> Senior Lecturer,
> The Institute of the Earth Science,
> The Hebrew University of Jerusalem,
> Givat Ram, Jerusalem 91904, Israel
> Tel: 972 547 231 170
> Fax: (972)-25662581
>
> C.E.O, Weather It Is, LTD
> Weather and Climate Focus
> http://weather-it-is.com
> Jerusalem, Israel
> Local: 02 930 9525
> Cell: 054 7 231 170
> Int-IS: x972 2 930 9525
> US 914 432 3108 <(914)%20432-3108>
> <variable.file>_______________________________________________
> ncl-talk mailing list
> ncl-talk at ucar.edu
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
>
>
--
Barry H. Lynn, Ph.D
Senior Lecturer,
The Institute of the Earth Science,
The Hebrew University of Jerusalem,
Givat Ram, Jerusalem 91904, Israel
Tel: 972 547 231 170
Fax: (972)-25662581
C.E.O, Weather It Is, LTD
Weather and Climate Focus
http://weather-it-is.com
Jerusalem, Israel
Local: 02 930 9525
Cell: 054 7 231 170
Int-IS: x972 2 930 9525
US 914 432 3108
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20170719/73dbce72/attachment.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: plot_loop_surface.ncl
Type: application/octet-stream
Size: 14241 bytes
Desc: not available
Url : http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20170719/73dbce72/attachment.obj
-------------- next part --------------
A non-text attachment was scrubbed...
Name: get_ij.f
Type: application/octet-stream
Size: 931 bytes
Desc: not available
Url : http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20170719/73dbce72/attachment-0001.obj
More information about the ncl-talk
mailing list