[ncl-talk] Fwd: [ncl-install] Code/loop problems

[Mary Haley]

This is likely due you to running out of memory, since you are now trying
to read all the data at once. You may have to read the variable in
sections. I can help with this, but I need to know more about the variable
you are trying to read.

How big is the variable on *one* of the files, and what type is it?  That
is, what are the dimension sizes of the array? You can use "ncl_filedump"
to get this information.

Is this variable the same size on all 107 files?

--Mary

On Mon, Aug 24, 2015 at 10:35 AM, Mariama Barr - Dallas <bama4 at umbc.edu>
wrote:

>
> ---------- Forwarded message ----------
> From: Mariama Barr - Dallas <bama4 at umbc.edu>
> Date: Mon, Aug 24, 2015 at 12:31 PM
> Subject: Re: [ncl-install] Code/loop problems
> To: Mary Haley <haley at ucar.edu>, ncl-talk at ucar.edu
>
>
> When I try to use the addfiles function for 107 HDF5 files that are
> approx. 60,000 Kb large, I get a segmentation fault: failure to allocate
> memory for NclHDF5datatype..why is this?
>
>
> On Sun, Aug 23, 2015 at 11:20 PM, Mary Haley <haley at ucar.edu> wrote:
>
>> Mariama,
>>
>> In the future, please post NCL questions to ncl-talk, and not ncl-install:
>>
>> http://www.ncl.ucar.edu/Support/email_lists.shtml
>>
>> Without knowing anything about your data, it's almost impossible to tell
>> you how to fix your code so you can correctly "gather" all the data. We
>> need to know what structure the data is on each file.
>>
>> Depending on your data, you might be able to use "addfiles" instead of
>> "addfile", as it allows you to read in a series of files with a single
>> call.You can then read data off all the files in either "join" or "cat"
>> mode.  See examples 1 and 2 on the "addfiles" function page:
>>
>> http://www.ncl.ucar.edu/Document/Functions/Built-in/addfiles.shtml
>>
>> Your code might look something like this:
>>
>>   files_0 = addfiles(names,"r")    ; Open all the HDF5 files in one call.
>>   longs = files_0[:]->retrieval_longitude   ; Read retrieval_longitude
>> across all files
>>   lats  = files_0[:]->retrieval_latitude    ; ...
>>   data  = files_0[:]->xco2
>>
>>
>> Also, you don't need to redirect your "find" output to an ASCII file, and
>> then read in that ASCII file to get the list of names. You should be able
>> to do something like this:
>>
>> function get_file_names()
>> begin
>>   names = systemfunc("find /home/Bama4/OCO2_DATA/ -name '*.h5' -print')
>>   return names
>> end
>>
>> Please look at your data files carefully, and determine whether
>> "addfiles" might work for you. If you continue to have questions, post them
>> to ncl-talk, but include more information about the structure of your
>> data.  You can either use "ncl_filedump" to look at single files from the
>> UNIX command line:
>>
>>     ncl_filedump xxxxx.h5
>>
>> or "printVarSummary" to examine the contents of a variable after you read
>> it off a file:
>>
>>    data = files_0->xco2
>>    printVarSummary(data)
>>
>> --Mary
>>
>>
>> On Sat, Aug 22, 2015 at 5:09 PM, Mariama Barr - Dallas <bama4 at umbc.edu>
>> wrote:
>>
>>> The following code is giving me problems, Im trying to gather all of the
>>> data into three arrays: longs, lats, and data. It  keeps saying that the
>>> arrays are not the same dimension when I try array_append_record, and when
>>> I try copy_MetaVar for the arrays it just gives me an empty array. Is there
>>> an easier way to gather up all of the latitudes, longitudes, and data from
>>> each file into three arrays so I can in the future display it  on one map?
>>>
>>> ​
>>>
>>> --
>>> Mariama Barr-Dallas
>>> Meyerhoff Scholar/ NSA Scholar M23
>>> Computer Science Major, History Major
>>> University of Maryland Baltimore County
>>> Class of 2015
>>>
>>> _______________________________________________
>>> ncl-install mailing list
>>> List instructions, subscriber options, unsubscribe:
>>> http://mailman.ucar.edu/mailman/listinfo/ncl-install
>>>
>>>
>>
>
>
> --
> Mariama Barr-Dallas
> Meyerhoff Scholar/ NSA Scholar M23
> Computer Science Major, History Major
> University of Maryland Baltimore County
> Class of 2015
>
>
>
> --
> Mariama Barr-Dallas
> Meyerhoff Scholar/ NSA Scholar M23
> Computer Science Major, History Major
> University of Maryland Baltimore County
> Class of 2015
>
> _______________________________________________
> ncl-talk mailing list
> ncl-talk at ucar.edu
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20150824/df7cb4d0/attachment.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: code.PNG
Type: image/png
Size: 58715 bytes
Desc: not available
Url : http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20150824/df7cb4d0/attachment.png