[ncl-talk] Fwd: Deal with a lot of netcdf files

[Rabah Hachelaf]

I solved my problem by doing a loop over years to read wrfoutputs

Thanks to all for you suggestions.

2015-06-25 23:43 GMT-04:00 Lam Hoang <lamhpvn at gmail.com>:

> Try to combine them to monthly to reduce number of files to less than 512
> to continue
> Or
> Set ulimit -n xxx where xxx >= 24x365x5
>
> On Thursday, June 25, 2015, Rabah Hachelaf <hachelaf at sca.uqam.ca> wrote:
>
>> 24 X 365 X 5 (24 files per day over 5 years)
>>
>>
>> 2015-06-25 3:42 GMT-04:00 Lam Hoang <lamhpvn at gmail.com>:
>>
>>> How man files do you have?
>>>
>>>
>>> On Thursday, June 25, 2015, Rabah Hachelaf <hachelaf at sca.uqam.ca> wrote:
>>>
>>>> ulimit -n 512 does't not solve the problem
>>>>
>>>> Any other advises ??
>>>>
>>>> 2015-06-24 23:47 GMT-04:00 Lam Hoang <lamhpvn at gmail.com>:
>>>>
>>>>> You should check the open files limit by using
>>>>> ulimit -a
>>>>>
>>>>> You will find parameter: open files (-n) 256 with 256 is maximum open
>>>>> files.
>>>>> You can change this parameter with command:
>>>>> ulimit -n 512
>>>>>
>>>>> Hope this help.
>>>>> Lam
>>>>>
>>>>> On Thu, Jun 25, 2015 at 1:05 PM, Rabah Hachelaf <hachelaf at sca.uqam.ca>
>>>>> wrote:
>>>>>
>>>>>>
>>>>>> Hello NCL users,
>>>>>>
>>>>>> I have netcdf wrf outputs every 1h for 5 years,
>>>>>> i use those commands to read files :
>>>>>>
>>>>>> files = systemfunc("ls wrf_1h/wrfout_d02_*") + ".nc"
>>>>>> a  = addfiles (files, "r")
>>>>>> ListSetType (a, "cat ")
>>>>>>
>>>>>> but i get a Segmentation fault error,
>>>>>> when reduce the number of read files my script works correctly.
>>>>>>
>>>>>> Any suggestion about this issue.
>>>>>>
>>>>>> Thank you
>>>>>> --
>>>>>> ------------------------------
>>>>>> Best regards,
>>>>>> Rabah Hachelaf
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> ncl-talk mailing list
>>>>>> ncl-talk at ucar.edu
>>>>>> List instructions, subscriber options, unsubscribe:
>>>>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Lam Hoang
>>>>> Viet Nam National Center of Hydro-Meteorology Forecasting
>>>>> National Hydro-Meteorological Services of Vietnam
>>>>>
>>>>> PhD Candidate
>>>>> School of Earth, Atmosphere and Environment
>>>>> Monash University, VIC 3800
>>>>> Mob: +84 12 1616 5258 (VN)
>>>>>          +61 4 5251 8668 (AUS)
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> ------------------------------
>>>> Cordialement,
>>>> Best regards,
>>>> Rabah Hachelaf
>>>>
>>>
>>>
>>> --
>>> Lam Hoang
>>> Viet Nam National Center of Hydro-Meteorology Forecasting
>>> National Hydro-Meteorological Services of Vietnam
>>>
>>> PhD Candidate
>>> School of Earth, Atmosphere and Environment
>>> Monash University, VIC 3800
>>> Mob: +84 12 1616 5258 (VN)
>>>          +61 4 5251 8668 (AUS)
>>>
>>>
>>>
>>
>>
>> --
>> ------------------------------
>> Cordialement,
>> Best regards,
>> Rabah Hachelaf
>>
>
>
> --
> Lam Hoang
> Viet Nam National Center of Hydro-Meteorology Forecasting
> National Hydro-Meteorological Services of Vietnam
>
> PhD Candidate
> School of Earth, Atmosphere and Environment
> Monash University, VIC 3800
> Mob: +84 12 1616 5258 (VN)
>          +61 4 5251 8668 (AUS)
>
>
>


-- 
------------------------------
Cordialement,
Best regards,
Rabah Hachelaf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20150627/c456a160/attachment.html