[ncl-talk] killed occurence

Sat Dec 23 03:32:09 MST 2017

 Dear MarryThank you again for your help and your code. "main_mod.ncl" is awesome, and it fixes my problems.However, my ncl version is 6.4.0. In another server with ncl 6.3.0 the code (mian.ncl) isn't killed. Your code works great and fast, and my problem has been solved with "main_mod.ncl".
SincerelyEhsan
    On Friday, December 22, 2017, 9:27:44 PM GMT+3:30, Mary Haley <haley at ucar.edu> wrote:  

 Ehsan,
I had no trouble running your script with NCL V6.4.0.  What version of NCL are you using?  You can type "ncl -V" for this information.
As I said in an earlier email, you can make the writing of the ASCII files faster if you don't use a double do loop.  For example, this code:
    do i=0, dimsizes(lat_NW)-1      do j=0, dimsizes(time)-1        write_table("GPM_R24_Interpolated_slow.txt","a",[/time(j),lat_NW(i),lon_NW(i),GPM_fod(j,i)/], \                     " %8i %7.4f %7.4f %5.1f")      end do    end do
can be replaced with:
    ntim = dimsizes(time)    nlat = dimsizes(lat_NW)    dims = (/nlat,ntim/)    time1d = ndtooned(conform_dims(dims,time,1))    lat1d  = ndtooned(conform_dims(dims,lat_NW,0))    lon1d  = ndtooned(conform_dims(dims,lon_NW,0))    write_table("GPM_R24_Interpolated_fast.txt","a",[/time1d,lat1d,lon1d,transpose(GPM_fod)/], \                 " %8i %7.4f %7.4f %5.1f")
Note that because you are writing the time dimension out first and then the lat/lon dimension, I had to transpose the GPM_fod array before writing it.
Also, the faster way does require the use of more memory, because I'm now creating extra arrays to hold all the time, lat, and lon arrays in a long 1D arrays.
However, the speed up for writing the two files without do loops was significant, mainly for the first double do loop.
Original method with double do loops:

 Elapsed time for do loop = 299.03 CPU seconds

New method with no do loops:

 Elapsed time for do loop = 0.702266 CPU seconds

I've attached a modified version of your main.ncl script so you can see what I changed. Note that I changed the named of the output file names to have either the word "slow" or "fast" as part of them, so you can compare the two files. See the FASTER_WAY variable which I added to allow you to switch between the two methods.  You probably want to remove these "if" statements once you verify the code is working the way you expect. 
Please try this new script to see if it fixes your issue.

--Mary
On Wed, Dec 20, 2017 at 12:46 PM, Ehsan Taghizadeh via ncl-talk <ncl-talk at ucar.edu> wrote:

Dear MaryThank you so much for your help. As you suggested I attached input files (GPM_IMERGV04DL_20170401_ 20170430.rar) and script file (GPM_Interpolating.rar). The script works well until first, but it goes slow gradually and finally it killed.About your following comment:  write_table("file2.txt","w", [/j,x,i,y,k/], "string_%03i %8.2f %4.0i %8.1f     string_%03i")About I think it isn't possible for me, because of different indexes of variables:
        write_table("GPM_R24_Grid.txt" ,"a",[/GPM_fnamesc(i),lat_GPM( j),lon_GPM(k), R24h_GPM(k,j)/], " %s %7.4f %7.4f %5.1f").

SincerelyEhsan Taghizadeh    On Tuesday, December 19, 2017, 8:10:12 PM GMT+3:30, Mary Haley <haley at ucar.edu> wrote:  

 Dear Ehsan,
If you are asking us to look at your script and improve it for you, it helps to provide more details about what you expect the script to do and where it appears to be failing.
You included the script and some of the files, but not the NetCDF files, so there's no way for us to actually run the script to see what the problem is.
Did you try debugging your script to see where it might be failing? It would help to add more print statements, especially at the top of the do loops, so you can see how many iterations of the loops it completes before it fails.
Nested do loops shouldn't cause your script to fail, but they could cause your script to be much slower than needed.
I see that you are trying to write data to a file called "GPM_R24_Grid.txt" and that you have a double do loop for writing individual rows to the file. You can write more than one row of data to the file without having to loop across each value, and this could greatly improve the speed of your script. 
Here's a very simple example:
;---Generate some dummy integer and float data.   npts = 100   i    = ispan(1,npts,1)   j    = generate_unique_indices(npts)   k    = generate_unique_indices(npts)   x    = random_uniform(-10,10,npts)   y    = random_uniform(0,1000.,npts)
  write_table("file2.txt","w",[/ j,x,i,y,k/], "string_%03i %8.2f %4.0i %8.1f     string_%03i")
For more examples of using write_table to write multiple lines of text to an ASCII file, see this examples page:
http://www.ncl.ucar.edu/ Applications/write_csv.shtml
This page is for writing CSV files, but these examples can be used for any kind of ASCII file.
If you continue to have problems with your script, please respond back to ncl-talk at ucar.edu and provide more information, like where you think the script is failing, and what you are trying to accomplish with it.
Thanks,
--Mary

On Tue, Dec 19, 2017 at 1:56 AM, Ehsan Taghizadeh via ncl-talk <ncl-talk at ucar.edu> wrote:

Hi,I try to run attached script (main.nc), however each time it counters killed, without any error.I've searched google for that and it seems because of nested loops. Is it true? Could you please help to make this script more efficient? Also I attached some part of printVarSummary.
SincerelyEhsan
______________________________ _________________
ncl-talk mailing list
ncl-talk at ucar.edu
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/ mailman/listinfo/ncl-talk

______________________________ _________________
ncl-talk mailing list
ncl-talk at ucar.edu
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/ mailman/listinfo/ncl-talk

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20171223/e6461cf3/attachment.html>