[ncl-talk] killed occurence
Mary Haley
haley at ucar.edu
Fri Dec 22 10:57:41 MST 2017
Ehsan,
I had no trouble running your script with NCL V6.4.0. What version of NCL
are you using? You can type "ncl -V" for this information.
As I said in an earlier email, you can make the writing of the ASCII files
faster if you don't use a double do loop. For example, this code:
do i=0, dimsizes(lat_NW)-1
do j=0, dimsizes(time)-1
write_table("GPM_R24_Interpolated_slow.txt","a",[/time(j),lat_NW(i),lon_NW(i),GPM_fod(j,i)/],
\
" %8i %7.4f %7.4f %5.1f")
end do
end do
can be replaced with:
ntim = dimsizes(time)
nlat = dimsizes(lat_NW)
dims = (/nlat,ntim/)
time1d = ndtooned(conform_dims(dims,time,1))
lat1d = ndtooned(conform_dims(dims,lat_NW,0))
lon1d = ndtooned(conform_dims(dims,lon_NW,0))
write_table("GPM_R24_Interpolated_fast.txt","a",[/time1d,lat1d,lon1d,transpose(GPM_fod)/],
\
" %8i %7.4f %7.4f %5.1f")
Note that because you are writing the time dimension out first and then the
lat/lon dimension, I had to transpose the GPM_fod array before writing it.
Also, the faster way does require the use of more memory, because I'm now
creating extra arrays to hold all the time, lat, and lon arrays in a long
1D arrays.
However, the speed up for writing the two files without do loops was
significant, mainly for the first double do loop.
Original method with double do loops:
Elapsed time for do loop = *299.03* CPU seconds
New method with no do loops:
Elapsed time for do loop = *0.702266* CPU seconds
I've attached a modified version of your main.ncl script so you can see
what I changed. Note that I changed the named of the output file names to
have either the word "slow" or "fast" as part of them, so you can compare
the two files. See the FASTER_WAY variable which I added to allow you to
switch between the two methods. You probably want to remove these "if"
statements once you verify the code is working the way you expect.
Please try this new script to see if it fixes your issue.
--Mary
On Wed, Dec 20, 2017 at 12:46 PM, Ehsan Taghizadeh via ncl-talk <
ncl-talk at ucar.edu> wrote:
>
> Dear Mary
> Thank you so much for your help. As you suggested I attached input files
> (GPM_IMERGV04DL_20170401_20170430.rar) and script file
> (GPM_Interpolating.rar). The script works well until first, but it goes
> slow gradually and finally it killed.
> About your following comment:
> write_table("file2.txt","w",[/j,x,i,y,k/], "string_%03i %8.2f %4.0i
> %8.1f string_%03i")
> About I think it isn't possible for me, because of different indexes of
> variables:
> write_table("GPM_R24_Grid.txt","a",[/GPM_fnamesc(i),lat_GPM(j)
> ,lon_GPM(k), R24h_GPM(k,j)/], " %s %7.4f %7.4f %5.1f").
>
> Sincerely
> Ehsan Taghizadeh
> On Tuesday, December 19, 2017, 8:10:12 PM GMT+3:30, Mary Haley <
> haley at ucar.edu> wrote:
>
>
> Dear Ehsan,
>
> If you are asking us to look at your script and improve it for you, it
> helps to provide more details about what you expect the script to do and
> where it appears to be failing.
>
> You included the script and some of the files, but not the NetCDF files,
> so there's no way for us to actually run the script to see what the problem
> is.
>
> Did you try debugging your script to see where it might be failing? It
> would help to add more print statements, especially at the top of the do
> loops, so you can see how many iterations of the loops it completes before
> it fails.
>
> Nested do loops shouldn't cause your script to fail, but they could cause
> your script to be much slower than needed.
>
> I see that you are trying to write data to a file called
> "GPM_R24_Grid.txt" and that you have a double do loop
> for writing individual rows to the file. You can write more than one row
> of data to the file without having to loop across each value, and this
> could greatly improve the speed of your script.
>
> Here's a very simple example:
>
> ;---Generate some dummy integer and float data.
> npts = 100
> i = ispan(1,npts,1)
> j = generate_unique_indices(npts)
> k = generate_unique_indices(npts)
> x = random_uniform(-10,10,npts)
> y = random_uniform(0,1000.,npts)
>
> write_table("file2.txt","w",[/j,x,i,y,k/], "string_%03i %8.2f %4.0i
> %8.1f string_%03i")
>
> For more examples of using write_table to write multiple lines of text to
> an ASCII file, see this examples page:
>
> http://www.ncl.ucar.edu/Applications/write_csv.shtml
>
> This page is for writing CSV files, but these examples can be used for any
> kind of ASCII file.
>
> If you continue to have problems with your script, please respond back to
> ncl-talk at ucar.edu and provide more information, like where you think the
> script is failing, and what you are trying to accomplish with it.
>
> Thanks,
>
> --Mary
>
>
>
>
>
>
>
>
>
> On Tue, Dec 19, 2017 at 1:56 AM, Ehsan Taghizadeh via ncl-talk <
> ncl-talk at ucar.edu> wrote:
>
> Hi,
> I try to run attached script (main.nc), however each time it counters
> killed, without any error.
> I've searched google for that and it seems because of nested loops. Is it
> true? Could you please help to make this script more efficient? Also I
> attached some part of printVarSummary.
>
> *Sincerely*
> *Ehsan*
>
> ______________________________ _________________
> ncl-talk mailing list
> ncl-talk at ucar.edu
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/ mailman/listinfo/ncl-talk
> <http://mailman.ucar.edu/mailman/listinfo/ncl-talk>
>
>
>
> _______________________________________________
> ncl-talk mailing list
> ncl-talk at ucar.edu
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20171222/3e88ce6a/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: main_mod.ncl
Type: application/octet-stream
Size: 9116 bytes
Desc: not available
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20171222/3e88ce6a/attachment-0001.obj>
More information about the ncl-talk
mailing list