<div dir="ltr"><div>3 Tb is a very large data set, difficult to contain all in memory at the same time. I recommend using serial methods to perform your analysis along one or two dimensions, one step at a time. You can implement serial methods in NCL or any other scientific programming language. In NCL, you can use the full complement of library functions to assist your serial methods.</div><div><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Oct 8, 2020 at 12:08 PM Chathurika via ncl-talk <<a href="mailto:ncl-talk@mailman.ucar.edu">ncl-talk@mailman.ucar.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><br>
<p>
<span style="vertical-align:inherit"><span style="vertical-align:inherit"><span style="vertical-align:inherit"><span style="vertical-align:inherit">Hello,</span></span></span></span>
</p>
<p>
<span style="vertical-align:inherit"><span style="vertical-align:inherit"><br>
</span></span>
</p>
<p>
<span style="vertical-align:inherit"><span style="vertical-align:inherit"><span style="vertical-align:inherit"><span style="vertical-align:inherit">I want to handle large data set (more than 3TB).However, I am getting the error number 12 due to the failure of memory allocation. I read lot of Q&A for this issue, however there is no proper answer. I have to analyze this large data set. So please let me know how can I solve this memory allocation issue. I hope there is someone who already has a solution for this.</span></span></span></span></p>
<p>
<br>
</p>
<p>
<span style="vertical-align:inherit"><span style="vertical-align:inherit">Thank you so much and best regards</span></span></p>
<span>
<hr align="left" style="margin:0.5em 0px;width:10em;height:1px;background-color:rgb(153,153,153);border:none">
<blockquote style="margin:0px 0px 0px 40px;padding:0px;font-family:"Helvetica Neue",Helvetica,Arial,sans-serif;font-size:small;font-variant-ligatures:normal;border:none"><span style="font-size:x-small;font-style:italic">Wickramage Chathurika Hemamali</span><br></blockquote><blockquote style="margin:0px 0px 0px 40px;padding:0px;font-family:"Helvetica Neue",Helvetica,Arial,sans-serif;font-variant-ligatures:normal;border:none"><div>
<span style="font-style:italic"><span style="font-size:x-small"><span style="vertical-align:inherit"><span style="vertical-align:inherit">Msc in Physical Oceanography</span></span></span></span>
</div>
<div>
<div id="gmail-m_239189906115319016ydpc6aa4ef1yui_3_16_0_ym19_1_1496286059659_2251">
<span style="font-style:italic"><span style="font-size:x-small"><span style="vertical-align:inherit"><span style="vertical-align:inherit">State Key Laboratory of Tropical Oceanography</span></span></span></span>
</div>
</div>
<div>
<div id="gmail-m_239189906115319016ydpc6aa4ef1yui_3_16_0_ym19_1_1496286059659_2252">
<span style="font-style:italic"><span style="font-size:x-small"><span style="vertical-align:inherit"><span style="vertical-align:inherit">South China Sea Institute of Oceanology</span></span></span></span>
</div>
<div id="gmail-m_239189906115319016ydpc6aa4ef1yui_3_16_0_ym19_1_1496286059659_2252">
<span style="font-style:italic"><span style="font-size:x-small"><span style="vertical-align:inherit"><span style="vertical-align:inherit">University of Chinese Academy of Science</span></span></span></span>
</div>
</div>
<div>
<div id="gmail-m_239189906115319016ydpc6aa4ef1yui_3_16_0_ym19_1_1496286059659_2264">
<span style="font-style:italic"><span style="font-size:x-small"><span style="vertical-align:inherit"><span style="vertical-align:inherit">China</span></span></span></span>
</div>
</div>
<div id="gmail-m_239189906115319016ydpc6aa4ef1yui_3_16_0_ym19_1_1496286059659_2264" style="font-size:small">
<span style="font-style:italic;font-size:medium"><br>
</span>
</div>
<div id="gmail-m_239189906115319016ydpc6aa4ef1yui_3_16_0_ym19_1_1496286059659_2264" style="font-size:small">
<i><span style="font-size:xx-small"><span style="vertical-align:inherit"><span style="vertical-align:inherit">Specialized in Oceanography and Marine Geology (Bachelor)</span></span></span></i>
</div>
<div id="gmail-m_239189906115319016ydpc6aa4ef1yui_3_16_0_ym19_1_1496286059659_2264" style="font-size:small">
<i><span style="font-size:xx-small"><span style="vertical-align:inherit"><span style="vertical-align:inherit">University of Ruhuna</span></span></span></i>
</div>
<div id="gmail-m_239189906115319016ydpc6aa4ef1yui_3_16_0_ym19_1_1496286059659_2264" style="font-size:small">
<i><span style="font-size:xx-small"><span style="vertical-align:inherit"><span style="vertical-align:inherit">Matara</span></span></span></i>
</div>
<div id="gmail-m_239189906115319016ydpc6aa4ef1yui_3_16_0_ym19_1_1496286059659_2264" style="font-size:small">
<i><span style="font-size:xx-small"><span style="vertical-align:inherit"><span style="vertical-align:inherit">Sri Lanka</span></span></span></i>
</div>
<div id="gmail-m_239189906115319016ydpc6aa4ef1yui_3_16_0_ym19_1_1496286059659_2264" style="font-size:small">
<span style="font-style:italic;font-size:medium"><br>
</span>
</div>
<div id="gmail-m_239189906115319016ydpc6aa4ef1yui_3_16_0_ym19_1_1496286059659_2264" style="font-size:small">
<span style="font-style:italic;font-size:medium"><span style="vertical-align:inherit"><span style="vertical-align:inherit">Email: </span></span><a><span style="vertical-align:inherit"><span style="vertical-align:inherit">wickramagechathurika@rocketmail.com</span></span></a></span>
</div>
<div id="gmail-m_239189906115319016ydpc6aa4ef1yui_3_16_0_ym19_1_1496286059659_2264" style="font-size:small">
<span style="font-style:italic;font-size:medium"><a><span style="vertical-align:inherit"><span style="vertical-align:inherit">chatu@scsio.ac.cn</span></span></a></span></div></blockquote></span></blockquote></div></div>