[ncl-talk] loop memory issue

[Mary Haley]

I think the problem might be with the two calls to generate_sample_indices
inside the double do loop.

If I'm reading your code correctly, ix and iy will be size 317 and 720
elements respectively.

The generate_sample_indices function is actually an NCL function (and not a
built-in function) that creates an array of random numbers and then sorts
them.  This means you are creating two new arrays every time inside this do
loop,and never freeing up the space.  Also, the sorting may cause the code
to slow down.

As a start, you might try deleting ix and iy after each iteration in the
loop.

   do while(i.lt.n)
     j = 0

     do while(j.lt.m) ; loop through time

       ; randomise in space
       ix = generate_sample_indices( dms(0), 0 )
       iy = generate_sample_indices( dms(1), 0 )
       rand_HS(:,:,j) = rand_HS(ix, iy, j)

       j=j+1

       delete([/ix,iy/])
     end do

You might also consider whether you need to do this every time inside the
do loop.

Let ncl-talk know if you continue to have problems with this code even
after adding the delete command.

--Mary


On Sun, Feb 8, 2015 at 6:25 AM, David Craig <dcdavemail at gmail.com> wrote:

> Hi,
>
> I have a loop in NCL that randomises an array and preforms some
> operations on the result. However, it seems to be using too much memory
> and I don't understand why as all variables are preallocated. Does
> anyone know what the problem is, code is below.
>
> thanks,
>
> David
>
>
>    n = 1000
>    rand_HS = HS_reorder(:,:,1:)     ; size 317x720x240
>    dms = dimsizes(rand_HS)
>    m = dms(2)
>    ; preallocate array for maximum coefficient values
>    max_rand_ccr = new(n,float)
>    i = 0
>
>    do while(i.lt.n)
>      j = 0
>
>      do while(j.lt.m) ; loop through time
>
>        ; randomise in space
>        ix = generate_sample_indices( dms(0), 0 )
>        iy  = generate_sample_indices( dms(1), 0 )
>        rand_HS(:,:,j) = rand_HS(ix, iy, j)
>
>        j=j+1
>      end do
>
>      ;calculate max correlation coefficient
>      max_rand_ccr(i) = max(abs(escorc(data2,rand_HS)))
>      i=i+1
>    end do
> _______________________________________________
> ncl-talk mailing list
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20150208/00072bcf/attachment.html