[ncl-talk] The slow down when addfile is used in do loop

[rupingmo at gmail.com]

Hi Jon,

I do have the latest version of NCL (v6.2.1). The code is running fast to the point where the previous run was terminated, then begins to slow down. The delete function (in my case, delete(dat)) at the end of each iteration doesn't seem to help.

Ruping
Sent from my BlackBerry device on the Rogers Wireless Network

-----Original Message-----
From: Jon Meyer <jonathan.meyer at aggiemail.usu.edu>
Date: Fri, 12 Sep 2014 16:03:19 
To: Ruping Mo<rupingmo at gmail.com>; ncl-talk at ucar.edu<ncl-talk at ucar.edu>
Subject: Re: [ncl-talk] The slow down when addfile is used in do loop

What version of NCL are you using? Version 6.2 fixed a minor memory leak
associated when .grb files are opened. When numerous .grb files are opened
sequentially, the small memory leak can cause your memory footprint to grow
with time...even if you are using the 'delete' function after each loop
iteration.

If your code is running fast at first, using the delete function at the end
of each iteration will likely solve your issue.

Jon

On Fri, Sep 12, 2014 at 2:31 PM, Ruping Mo <rupingmo at gmail.com> wrote:

> Hi NCLers,
>
> I have an archive of large grib files. Each grib file contains a month of
> multiple daily global variables. I use the following script to extract a
> small part of one variable. The script, which uses addfile in a large loop,
> runs very fast for the first few months, then begins to slow down
> substantially. I let it run for a few years, then terminate the job
> (Ctrl-Z) and re-run the same script. Surprisingly, the new job runs very
> fast to the month when the previous job was terminated. I wonder why it
> behaves this way, and what I can do to avoid the slow down problem.
>
> Have a nice day!
>
> Ruping
>
> The script:
>
> ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
> begin
>  nday = 12784     ; number of days from Jan 1979 to Dec 2013
>  temp = new((/nday, 9, 9/), float)
> ;;;;
>  t0 = 0
>  do yr = 1979, 2013
>    syr = tostring(yr)
>    do mo = 1, 12
>     smo = tostring(mo)
>     print(syr + "-" + smo)
>     fname = syr + "/" + syr + "-" + smo + ".grib"
>     dat = addfile(fname, "r")
>     mday = getfilevardimsizes(dat, "initial_time0") - 1
>     i1 = i0 + mday
>     temp(i0:i1, :, :) = (/ dat->2T_GDS0_SFC(:, 80:88, 146:154) /)
> ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; 2T_GDS0_SFC(mday, 241, 480)
>     i0 = i1 + 1
>    end do
>  end do
> end
>
> ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
>
> _______________________________________________
> ncl-talk mailing list
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20140912/49701059/attachment.html