[ncl-talk] loop memory issue

[David Craig]

Hi,

thanks for your input, deleting ix, iy didn't help as they are 
overwritten every loop. However, changing the second line from,

rand_HS = HS_reorder(:,:,1:)

to,

rand_HS = (/HS_reorder(:,:,1:)/)

did the trick. I guess the variable HS_reorder was growing in every loop 
for some reason but I am not clear why forcing it to an array fixed the 
issue.

David



On 08/02/15 23:41, Mary Haley wrote:
> I think the problem might be with the two calls to 
> generate_sample_indices inside the double do loop.
>
> If I'm reading your code correctly, ix and iy will be size 317 and 720 
> elements respectively.
>
> The generate_sample_indices function is actually an NCL function (and 
> not a built-in function) that creates an array of random numbers and 
> then sorts them.  This means you are creating two new arrays every 
> time inside this do loop,and never freeing up the space.  Also, the 
> sorting may cause the code to slow down.
>
> As a start, you might try deleting ix and iy after each iteration in 
> the loop.
>
>    do while(i.lt.n)
>      j = 0
>
>      do while(j.lt.m) ; loop through time
>
>        ; randomise in space
>        ix = generate_sample_indices( dms(0), 0 )
>        iy = generate_sample_indices( dms(1), 0 )
>  rand_HS(:,:,j) = rand_HS(ix, iy, j)
>
>        j=j+1
>
>        delete([/ix,iy/])
>      end do
>
> You might also consider whether you need to do this every time inside 
> the do loop.
>
> Let ncl-talk know if you continue to have problems with this code even 
> after adding the delete command.
>
> --Mary
>
>
> On Sun, Feb 8, 2015 at 6:25 AM, David Craig <dcdavemail at gmail.com 
> <mailto:dcdavemail at gmail.com>> wrote:
>
>     Hi,
>
>     I have a loop in NCL that randomises an array and preforms some
>     operations on the result. However, it seems to be using too much
>     memory
>     and I don't understand why as all variables are preallocated. Does
>     anyone know what the problem is, code is below.
>
>     thanks,
>
>     David
>
>
>        n = 1000
>        rand_HS = HS_reorder(:,:,1:)     ; size 317x720x240
>        dms = dimsizes(rand_HS)
>        m = dms(2)
>        ; preallocate array for maximum coefficient values
>        max_rand_ccr = new(n,float)
>        i = 0
>
>        do while(i.lt.n)
>          j = 0
>
>          do while(j.lt.m) ; loop through time
>
>            ; randomise in space
>            ix = generate_sample_indices( dms(0), 0 )
>            iy  = generate_sample_indices( dms(1), 0 )
>            rand_HS(:,:,j) = rand_HS(ix, iy, j)
>
>            j=j+1
>          end do
>
>          ;calculate max correlation coefficient
>          max_rand_ccr(i) = max(abs(escorc(data2,rand_HS)))
>          i=i+1
>        end do
>     _______________________________________________
>     ncl-talk mailing list
>     List instructions, subscriber options, unsubscribe:
>     http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20150209/78bac1b2/attachment.html