[ncl-talk] delete() is not clearing memory?

[Mary Haley]

Well, when I just now ran it, I got numbers more close to your original
numbers:

NCL memory:     75.77     basline memory

NCL memory:    457.35     X created
NCL memory:     75.89     X deleted

NCL memory:    457.25     X created
NCL memory:    838.68     X reassigned
NCL memory:    457.37     X deleted

Then, I just tried it on yellowstone a couple of times, and got very
different results:

*1st try:*

NCL memory:    3355.09     basline memory

NCL memory:    1698.17     X created
NCL memory:    1698.17     X deleted

NCL memory:    1698.17     X created
NCL memory:    841.36     X reassigned
NCL memory:    563.56     X deleted

*2nd try:*

NCL memory:    1421.56     basline memory

NCL memory:    1521.56     X created
NCL memory:    1521.56     X deleted

NCL memory:    1561.56     X created
NCL memory:    1621.56     X reassigned
NCL memory:    1633.56     X deleted

I'm wondering if the "ps" command is the correct one to use.  I will file a
ticket on this and see if we can get another way to profile this.

--Mary


On Fri, Oct 30, 2015 at 5:35 AM, Walter Hannah <walter at hannahlab.org> wrote:

> Mary,
>
> I'm currently using version 6.3.0, but I just tested it with 6.2.1 on my
> machine, as well as on Yellowstone, and got a similar result.
>
> The numbers were off by a fraction of a MB, but essentially the same.
>
> How different were your numbers?
>
> Walter
>
> On Thu, Oct 29, 2015 at 2:23 PM, Mary Haley <haley at ucar.edu> wrote:
>
>> Walter,
>>
>> Thanks for sending a little script that shows the issue.  I'm not seeing
>> the same large difference of results as you, but I do see some slight
>> increase.  If I run the script multiple times, I get very different
>> results, so I'm not sure exactly what kind of memory usage is being
>> reported by the "ps" command you provided.
>>
>> What version of NCL are you using?
>>
>> --Mary
>>
>>
>> On Wed, Oct 28, 2015 at 11:22 AM, Walter Hannah <walter at hannahlab.org>
>> wrote:
>>
>>> I'm having lots of "malloc" errors lately, and I realized that when I
>>> use the "delete" function, NCL doesn't free up as much memory as it
>>> created!!!
>>>
>>> After doing some sleuthing I figured out that the problem happens after
>>> I use the reassingment operator ":=" to reduce a variable to a subset of
>>> itself like this:
>>>
>>> X := X(0,:)
>>>
>>>
>>> I've come up with a stand alone NCL script (see below) that anyone
>>> should be able to run on a linux machine to reproduce the problem. It
>>> prints out the memory usage from the "ps" command.
>>>
>>> procedure printMem (msg)
>>> local rss,cmd
>>> begin
>>>     cmd = "ps --no-headers  -o 'rss' -C ncl"
>>>     rss = max(toint(systemfunc(cmd))) / 1024.
>>>     print("NCL memory:    "+sprintf("%6.2f",rss)+"     "+msg)
>>> end
>>>
>>> begin
>>>     dims = (/10000,100,100/)
>>>
>>>     print("")
>>>     printMem("basline memory")
>>>     print("")
>>>     ;-----------------------------------------------------
>>>     ;-----------------------------------------------------
>>>     X = new( dims ,float)
>>>     printMem("X created")
>>>     delete(X)
>>>     printMem("X deleted")
>>>     print("")
>>>     ;-----------------------------------------------------
>>>     ;-----------------------------------------------------
>>>     X = new( dims ,float)
>>>     printMem("X created")
>>>     X := X
>>>     printMem("X reassigned")
>>>     delete(X)
>>>     printMem("X deleted")
>>>     print("")
>>>     ;-----------------------------------------------------
>>>     ;-----------------------------------------------------
>>> end
>>>
>>>
>>> The output of the script on my linux machine is:
>>>
>>> (0)
>>> (0) NCL memory:     74.38     basline memory
>>> (0)
>>> (0) NCL memory:    456.11     X created
>>> (0) NCL memory:     74.66     X deleted
>>> (0)
>>> (0) NCL memory:    456.13     X created
>>> (0) NCL memory:    837.44     X reassigned
>>> (0) NCL memory:    456.14     X deleted
>>> (0)
>>>
>>>
>>> So, after using doing the reassignment it seems that a copy of the
>>> variable is left in memory and never cleared.
>>>
>>> I can work around it for now, but I wanted to ask if anyone knew about
>>> this, and also can it be fixed?
>>>
>>> Thanks,
>>> Walter Hannah
>>>
>>> _______________________________________________
>>> ncl-talk mailing list
>>> ncl-talk at ucar.edu
>>> List instructions, subscriber options, unsubscribe:
>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20151030/3d004d84/attachment.html