<div dir="ltr"><div class="gmail_default" style="font-size:small">Well, when I just now ran it, I got numbers more close to your original numbers:</div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style=""><div class="gmail_default" style="">NCL memory: 75.77 basline memory</div><div class="gmail_default" style=""><br></div><div class="gmail_default" style="">NCL memory: 457.35 X created</div><div class="gmail_default" style="">NCL memory: 75.89 X deleted</div><div class="gmail_default" style=""><br></div><div class="gmail_default" style="">NCL memory: 457.25 X created</div><div class="gmail_default" style="">NCL memory: 838.68 X reassigned</div><div class="gmail_default" style="">NCL memory: 457.37 X deleted</div><div class="gmail_default" style=""><br></div><div class="gmail_default" style="">Then, I just tried it on yellowstone a couple of times, and got very different results:</div><div class="gmail_default" style=""><br></div><div class="gmail_default" style=""><b>1st try:</b></div><div class="gmail_default" style=""><br></div><div class="gmail_default" style=""><div class="gmail_default">NCL memory: 3355.09 basline memory</div><div class="gmail_default"><br></div><div class="gmail_default">NCL memory: 1698.17 X created</div><div class="gmail_default">NCL memory: 1698.17 X deleted</div><div class="gmail_default"><br></div><div class="gmail_default">NCL memory: 1698.17 X created</div><div class="gmail_default">NCL memory: 841.36 X reassigned</div><div class="gmail_default">NCL memory: 563.56 X deleted</div><div class="gmail_default"><br></div><div class="gmail_default"><div class="gmail_default"><b>2nd try:</b></div><div><b><br></b></div></div><div class="gmail_default"><div class="gmail_default">NCL memory: 1421.56 basline memory</div><div class="gmail_default"><br></div><div class="gmail_default">NCL memory: 1521.56 X created</div><div class="gmail_default">NCL memory: 1521.56 X deleted</div><div class="gmail_default"><br></div><div class="gmail_default">NCL memory: 1561.56 X created</div><div class="gmail_default">NCL memory: 1621.56 X reassigned</div><div class="gmail_default">NCL memory: 1633.56 X deleted</div></div></div></div><div class="gmail_extra"><br></div><div class="gmail_extra"><div class="gmail_default" style="font-size:small">I'm wondering if the "ps" command is the correct one to use. I will file a ticket on this and see if we can get another way to profile this.</div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">--Mary</div><div class="gmail_default" style="font-size:small"><br></div><br><div class="gmail_quote">On Fri, Oct 30, 2015 at 5:35 AM, Walter Hannah <span dir="ltr"><<a href="mailto:walter@hannahlab.org" target="_blank">walter@hannahlab.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div>Mary, </div><div><br></div>I'm currently using version 6.3.0, but I just tested it with 6.2.1 on my machine, as well as on Yellowstone, and got a similar result. <div><br></div><div>The numbers were off by a fraction of a MB, but essentially the same.<div><br></div><div>How different were your numbers?</div><span class=""><font color="#888888"><div><br></div><div>Walter</div></font></span></div></div><div class=""><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Oct 29, 2015 at 2:23 PM, Mary Haley <span dir="ltr"><<a href="mailto:haley@ucar.edu" target="_blank">haley@ucar.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div style="font-size:small">Walter,</div><div style="font-size:small"><br></div><div style="font-size:small">Thanks for sending a little script that shows the issue. I'm not seeing the same large difference of results as you, but I do see some slight increase. If I run the script multiple times, I get very different results, so I'm not sure exactly what kind of memory usage is being reported by the "ps" command you provided.</div><div style="font-size:small"><br></div><div style="font-size:small">What version of NCL are you using?</div><span><font color="#888888"><div style="font-size:small"><br></div><div style="font-size:small">--Mary</div><div style="font-size:small"><br></div></font></span></div><div class="gmail_extra"><br><div class="gmail_quote"><span>On Wed, Oct 28, 2015 at 11:22 AM, Walter Hannah <span dir="ltr"><<a href="mailto:walter@hannahlab.org" target="_blank">walter@hannahlab.org</a>></span> wrote:<br></span><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div><div><div dir="ltr">I'm having lots of "malloc" errors lately, and I realized that when I use the "delete" function, NCL doesn't free up as much memory as it created!!!<div><br></div><div>After doing some sleuthing I figured out that the problem happens after I use the reassingment operator ":=" to reduce a variable to a subset of itself like this:</div><div><br></div><div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><font face="monospace, monospace">X := X(0,:)</font></blockquote></div><div><br></div><div>I've come up with a stand alone NCL script (see below) that anyone should be able to run on a linux machine to reproduce the problem. It prints out the memory usage from the "ps" command. </div><div><br></div><div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div><font face="monospace, monospace">procedure printMem (msg)</font></div><div><div><font face="monospace, monospace">local rss,cmd</font></div><div><font face="monospace, monospace">begin</font></div><div><font face="monospace, monospace"> cmd = "ps --no-headers -o 'rss' -C ncl"</font></div><div><font face="monospace, monospace"> rss = max(toint(systemfunc(cmd))) / 1024.</font></div></div><div><font face="monospace, monospace"> print("NCL memory: "+sprintf("%6.2f",rss)+" "+msg)</font></div><div><font face="monospace, monospace">end</font></div><div><font face="monospace, monospace"><br></font></div><div><font face="monospace, monospace">begin</font></div><div><font face="monospace, monospace"> dims = (/10000,100,100/)</font></div><div><font face="monospace, monospace"><br></font></div><div><font face="monospace, monospace"> print("")</font></div><div><font face="monospace, monospace"> printMem("basline memory")</font></div><div><font face="monospace, monospace"> print("")</font></div><div><font face="monospace, monospace"> ;-----------------------------------------------------</font></div><div><font face="monospace, monospace"> ;-----------------------------------------------------</font></div><div><font face="monospace, monospace"> X = new( dims ,float)</font></div><div><font face="monospace, monospace"> printMem("X created")</font></div><div><font face="monospace, monospace"> delete(X)</font></div><div><font face="monospace, monospace"> printMem("X deleted")</font></div><div><font face="monospace, monospace"> print("")</font></div><div><font face="monospace, monospace"> ;-----------------------------------------------------</font></div><div><font face="monospace, monospace"> ;-----------------------------------------------------</font></div><div><font face="monospace, monospace"> X = new( dims ,float)</font></div><div><font face="monospace, monospace"> printMem("X created")</font></div><div><font face="monospace, monospace"> X := X</font></div><div><font face="monospace, monospace"> printMem("X reassigned")</font></div><div><font face="monospace, monospace"> delete(X)</font></div><div><font face="monospace, monospace"> printMem("X deleted")</font></div><div><font face="monospace, monospace"> print("")</font></div><div><font face="monospace, monospace"> ;-----------------------------------------------------</font></div><div><font face="monospace, monospace"> ;-----------------------------------------------------</font></div><div><font face="monospace, monospace">end </font></div></blockquote></div><div><br></div><div>The output of the script on my linux machine is:</div><div><br></div><div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div><font face="monospace, monospace">(0)</font></div><div><font face="monospace, monospace">(0)<span style="white-space:pre-wrap"> </span>NCL memory: 74.38 basline memory</font></div><div><font face="monospace, monospace">(0)</font></div><div><font face="monospace, monospace">(0)<span style="white-space:pre-wrap"> </span>NCL memory: 456.11 X created</font></div><div><font face="monospace, monospace">(0)<span style="white-space:pre-wrap"> </span>NCL memory: 74.66 X deleted</font></div><div><font face="monospace, monospace">(0)</font></div><div><font face="monospace, monospace">(0)<span style="white-space:pre-wrap"> </span>NCL memory: 456.13 X created</font></div><div><font face="monospace, monospace">(0)<span style="white-space:pre-wrap"> </span>NCL memory: 837.44 X reassigned</font></div><div><font face="monospace, monospace">(0)<span style="white-space:pre-wrap"> </span>NCL memory: 456.14 X deleted</font></div><div><font face="monospace, monospace">(0)</font></div></blockquote></div><div><br></div><div>So, after using doing the reassignment it seems that a copy of the variable is left in memory and never cleared.</div><div><br></div><div>I can work around it for now, but I wanted to ask if anyone knew about this, and also can it be fixed?</div><div><br></div><div>Thanks,</div><div>Walter Hannah</div></div>
<br></div></div><span>_______________________________________________<br>
ncl-talk mailing list<br>
<a href="mailto:ncl-talk@ucar.edu" target="_blank">ncl-talk@ucar.edu</a><br>
List instructions, subscriber options, unsubscribe:<br>
<a href="http://mailman.ucar.edu/mailman/listinfo/ncl-talk" rel="noreferrer" target="_blank">http://mailman.ucar.edu/mailman/listinfo/ncl-talk</a><br>
<br></span></blockquote></div><br></div>
</blockquote></div><br></div>
</div></div></blockquote></div><br></div></div>