<div dir="ltr"><div class="gmail_default" style="font-size:small">This is likely due you to running out of memory, since you are now trying to read all the data at once. You may have to read the variable in sections. I can help with this, but I need to know more about the variable you are trying to read.</div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">How big is the variable on *one* of the files, and what type is it? That is, what are the dimension sizes of the array? You can use "ncl_filedump" to get this information. </div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">Is this variable the same size on all 107 files?</div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">--Mary</div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Aug 24, 2015 at 10:35 AM, Mariama Barr - Dallas <span dir="ltr"><<a href="mailto:bama4@umbc.edu" target="_blank">bama4@umbc.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="HOEnZb"><div class="h5"><div dir="ltr"><br><div class="gmail_quote">---------- Forwarded message ----------<br>From: <b class="gmail_sendername">Mariama Barr - Dallas</b> <span dir="ltr"><<a href="mailto:bama4@umbc.edu" target="_blank">bama4@umbc.edu</a>></span><br>Date: Mon, Aug 24, 2015 at 12:31 PM<br>Subject: Re: [ncl-install] Code/loop problems<br>To: Mary Haley <<a href="mailto:haley@ucar.edu" target="_blank">haley@ucar.edu</a>>, <a href="mailto:ncl-talk@ucar.edu" target="_blank">ncl-talk@ucar.edu</a><br><br><br><div dir="ltr">When I try to use the addfiles function for 107 HDF5 files that are approx. 60,000 Kb large, I get a segmentation fault: failure to allocate memory for NclHDF5datatype..why is this?<div><div><br><div class="gmail_extra"><br><div class="gmail_quote">On Sun, Aug 23, 2015 at 11:20 PM, Mary Haley <span dir="ltr"><<a href="mailto:haley@ucar.edu" target="_blank">haley@ucar.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div style="font-size:small"><span style="font-size:12.8000001907349px">Mariama,</span></div><div style="font-size:small"><span style="font-size:12.8000001907349px"><br></span></div><div style="font-size:small"><span style="font-size:12.8000001907349px">In the future, please post NCL questions to ncl-talk, and not ncl-install:</span></div><div style="font-size:small"><span style="font-size:12.8000001907349px"><br></span></div><div><span style="font-size:12.8000001907349px"><a href="http://www.ncl.ucar.edu/Support/email_lists.shtml" target="_blank">http://www.ncl.ucar.edu/Support/email_lists.shtml</a></span><br></div><div style="font-size:small"><span style="font-size:12.8000001907349px"><br></span></div><div><span style="font-size:12.8000001907349px">Without knowing anything about your data, it's almost impossible to tell you how to fix your code so you can correctly "gather" all the data. We need to know what structure the data is on each file.</span></div><div><span style="font-size:12.8000001907349px"><br></span></div><div><span style="font-size:12.8000001907349px">Depending on your data, you might be able to use "addfiles" instead of "addfile", as it allows you to read in a series of files with a single call.You can then read data off all the files in either "join" or "cat" mode. See examples 1 and 2 on the "addfiles" function page:</span></div><div><span style="font-size:12.8000001907349px"><br></span></div><div><span style="font-size:12.8000001907349px"><a href="http://www.ncl.ucar.edu/Document/Functions/Built-in/addfiles.shtml" target="_blank">http://www.ncl.ucar.edu/Document/Functions/Built-in/addfiles.shtml</a></span><br></div><div><span style="font-size:12.8000001907349px"><br></span></div><div><span style="font-size:12.8000001907349px">Your code might look something like this:</span></div><div><span style="font-size:12.8000001907349px"><br></span></div><div><span style="font-size:12.8000001907349px"><font face="monospace, monospace"> files_0 = addfiles(names,"r") ; Open all the HDF5 files in one call.</font></span></div><div><span style="font-size:12.8000001907349px"><font face="monospace, monospace"> longs = files_0[:]->retrieval_longitude ; Read retrieval_longitude across all files</font></span></div><div><div><span style="font-size:12.8000001907349px"><font face="monospace, monospace"> lats = files_0[:]->retrieval_latitude ; ...</font></span></div><div><span style="font-size:12.8000001907349px"><font face="monospace, monospace"> data = files_0[:]->xco2</font></span></div></div><div><br></div><div><span style="font-size:12.8000001907349px"><br></span></div><div><span style="font-size:12.8000001907349px">Also, you don't need to redirect your "find" output to an ASCII file, and then read in that ASCII file to get the list of names. You should be able to do something like this:</span></div><div><span style="font-size:12.8000001907349px"><br></span></div><div><span style="font-size:12.8000001907349px"><font face="monospace, monospace">function get_file_names()</font></span></div><div><span style="font-size:12.8000001907349px"><font face="monospace, monospace">begin</font></span></div><div><span style="font-size:12.8000001907349px"><font face="monospace, monospace"> names = systemfunc("find /home/Bama4/OCO2_DATA/ -name '*.h5' -print')</font></span></div><div><span style="font-size:12.8000001907349px"><font face="monospace, monospace"> return names</font></span></div><div><span style="font-size:12.8000001907349px"><font face="monospace, monospace">end</font></span></div><div><span style="font-size:12.8000001907349px"><br></span></div><div><span style="font-size:12.8000001907349px">Please look at your data files carefully, and determine whether "addfiles" might work for you. If you continue to have questions, post them to ncl-talk, but include more information about the structure of your data. You can either use "ncl_filedump" to look at single files from the UNIX command line:</span></div><div><span style="font-size:12.8000001907349px"><br></span></div><div><span style="font-size:12.8000001907349px"><font face="monospace, monospace"> ncl_filedump xxxxx.h5</font></span></div><div><span style="font-size:12.8000001907349px"><br></span></div><div><span style="font-size:12.8000001907349px">or "printVarSummary" to examine the contents of a variable after you read it off a file:</span></div><div><span style="font-size:12.8000001907349px"><br></span></div><div><div><span style="font-size:12.8000001907349px"><font face="monospace, monospace"> data = files_0->xco2</font></span></div><div><span style="font-size:12.8000001907349px"><font face="monospace, monospace"> printVarSummary(data)</font></span></div><div><span style="font-size:12.8000001907349px"><font face="arial, helvetica, sans-serif"><br></font></span></div><div><span style="font-size:12.8000001907349px"><font face="arial, helvetica, sans-serif">--Mary</font></span></div><div><span style="font-size:12.8000001907349px"><font face="monospace, monospace"><br></font></span></div><div></div></div></div><div class="gmail_extra"><br><div class="gmail_quote"><div><div>On Sat, Aug 22, 2015 at 5:09 PM, Mariama Barr - Dallas <span dir="ltr"><<a href="mailto:bama4@umbc.edu" target="_blank">bama4@umbc.edu</a>></span> wrote:<br></div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div><div dir="ltr">The following code is giving me problems, Im trying to gather all of the data into three arrays: longs, lats, and data. It keeps saying that the arrays are not the same dimension when I try array_append_record, and when I try copy_MetaVar for the arrays it just gives me an empty array. Is there an easier way to gather up all of the latitudes, longitudes, and data from each file into three arrays so I can in the future display it on one map?<div><img src="cid:ii_idnoh57n0_14f57ac343510480" width="392" height="454"><br><span><font color="#888888"><br clear="all"><div><br></div>-- <br><div><div dir="ltr"><div>Mariama Barr-Dallas</div>
<div>Meyerhoff Scholar/ NSA Scholar M23<br>Computer Science Major, History Major</div>
<div>University of Maryland Baltimore County<br>Class of 2015</div></div></div>
</font></span></div></div>
<br></div></div>_______________________________________________<br>
ncl-install mailing list<br>
List instructions, subscriber options, unsubscribe:<br>
<a href="http://mailman.ucar.edu/mailman/listinfo/ncl-install" rel="noreferrer" target="_blank">http://mailman.ucar.edu/mailman/listinfo/ncl-install</a><br>
<br></blockquote></div><br></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div><div dir="ltr"><div>Mariama Barr-Dallas</div>
<div>Meyerhoff Scholar/ NSA Scholar M23<br>Computer Science Major, History Major</div>
<div>University of Maryland Baltimore County<br>Class of 2015</div></div></div>
</div></div></div></div>
</div><br><br clear="all"><div><br></div>-- <br><div><div dir="ltr"><div>Mariama Barr-Dallas</div>
<div>Meyerhoff Scholar/ NSA Scholar M23<br>Computer Science Major, History Major</div>
<div>University of Maryland Baltimore County<br>Class of 2015</div></div></div>
</div>
</div></div><br>_______________________________________________<br>
ncl-talk mailing list<br>
<a href="mailto:ncl-talk@ucar.edu">ncl-talk@ucar.edu</a><br>
List instructions, subscriber options, unsubscribe:<br>
<a href="http://mailman.ucar.edu/mailman/listinfo/ncl-talk" rel="noreferrer" target="_blank">http://mailman.ucar.edu/mailman/listinfo/ncl-talk</a><br>
<br></blockquote></div><br></div></div>