[ncl-talk] Averaging over time dimension

Dennis Shea shea at ucar.edu
Thu Mar 8 14:39:46 MST 2018 

In addition to the CDO, the netCDF operators (NCO;
http://nco.sourceforge.net/ ) could perform the averaging. Specifically,
http://nco.sourceforge.net/nco.html#ncra

   %> ncra ERA_Plev.nc  ERA_Plev.Means.nc

The above will average all variables on the file with a 'time' (record)
dimension. There are assorted options that allow such actions as subsetting.

The CDO and NCO are invoked from the command line. They are designed to do
a specific task in a computational and memory efficient manner.
Also, there is no 'programming' involved!

NCL is a general purpose language. As a result, it does have some overhead.

Good luck

On Thu, Mar 8, 2018 at 12:52 PM, Soma Roy <somaroy892 at gmail.com> wrote:

> Thank you Adam Phillips.
> I will check this. Another way I think, using CDO first 'split' time steps
> or 'timmean' entire file then perform the operations over different
> variables in NCL.
>
> Soma
>
> On Mar 9, 2018 01:15, "Adam Phillips" <asphilli at ucar.edu> wrote:
>
>> Hi Soma,
>> Please remember to always respond to the ncl-talk email list when
>> responding. That way others can assist and can also see the answers to
>> queries.
>>
>> Your emailed reply stated:
>> > How much space is then required to process all the variables in single
>> run? I am using NCL in my laptop (i5 with 8GB ram)and free space in that
>> drive is 86 GB.
>>
>> I cannot answer that question precisely. Your machine will be using part
>> of its' memory for core functionality and other tasks. What you can do is
>> to try to conserve memory in your script as best as you can. The sample
>> coding snippets I sent you will assist in that task. Try what I suggested
>> and let ncl-talk know if you have further issues. Note that beyond what I
>> stated in my original reply, another avenue you might have to take to save
>> memory is to read in the variables in  different scripts.
>> Adam
>>
>>
>> On Thu, Mar 8, 2018 at 8:35 AM, Soma Roy <somaroy892 at gmail.com> wrote:
>>
>>> Hi,
>>>
>>> I am trying to access multiple variables from a file and need to average
>>> over time for each variables.
>>>
>>> I am running the following codes:
>>>
>>>    fn  = "ERA_Plev.nc"                      ; define filename
>>>    in  = addfile(fn,"r")                         ; open netcdf file
>>>    printVarSummary(in)
>>>
>>> T=in->t       ; temperature
>>> U=in->u      ; U wind
>>> V=in->v      ; V wind
>>>
>>> printVarSummary(T)
>>> printVarSummary(U)
>>> printVarSummary(V)
>>>
>>> ;T1 = dim_avg_n(T,0)
>>> ;T at units = "K"
>>> end
>>>
>>> Summary of the file is like below;
>>>
>>> Variable: in
>>> Type: file
>>> File path: ERA_Plev.nc
>>> Number of global attributes:     2
>>> Number of dimensions:    4
>>> Number of variables:     18
>>> fatal:NclMalloc Failed:[errno=12]
>>> Segmentation fault (core dumped)
>>>
>>> When I am calling only Temperature using the codes below;
>>>
>>> fn  = "ERA_Plev.nc"               ; define filename
>>> in  = addfile(fn,"r")                         ; open netcdf file
>>>    printVarSummary(in)
>>>
>>> T=in->t     ; temperature
>>> ;U=in->u     ; U wind
>>> ;V=in->v     ; V wind
>>>
>>> printVarSummary(T)
>>> ;printVarSummary(U)
>>> ;printVarSummary(V)
>>>
>>> ;T1 = dim_avg_n(T,0)
>>> ;T at units = "K"
>>> end
>>>
>>> It shows
>>>
>>> Variable: T
>>> Type: short
>>> Total Size: 1042566000 bytes
>>>             521283000 values
>>> Number of Dimensions: 4
>>> Dimensions and sizes:   [time | 120] x [level | 25] x [latitude | 721] x
>>> [longitude | 241]
>>> Coordinates:
>>>             time: [994176..994890]
>>>             level: [50..125]
>>>             latitude: [90..-90]
>>>             longitude: [60..120]
>>> Number Of Attributes: 7
>>>   scale_factor :        0.00218919045579553
>>>   add_offset :  252.4797479951041
>>>   _FillValue :  -32767
>>>   missing_value :       -32767
>>>   units :       K
>>>   long_name :   Temperature
>>>   standard_name :       air_temperature
>>>
>>>
>>> I have to call several other variables from this file (14 GB) and I need
>>> to average over time such that 'Dimensions' of each variables will be like
>>> below;
>>>
>>> [time | 1]* [level | 25] x [latitude | 721] x [longitude | 241]
>>>
>>> Which function I have to use in this case?
>>>
>>>
>>> When I am trying to call  other variables from the file I am getting the
>>> error message as;
>>>
>>> Variable: in
>>> Type: file
>>> File path: ERA_Plev.nc
>>> Number of global attributes:     2
>>> Number of dimensions:    4
>>> Number of variables:     18
>>> fatal:NclMalloc Failed:[errno=12]
>>> Segmentation fault (core dumped)
>>>
>>> Please kindly inform me how to resolve this issue.
>>>
>>>
>>> Thank you,
>>> Soma
>>>
>>>
>>>
>>> _______________________________________________
>>> ncl-talk mailing list
>>> ncl-talk at ucar.edu
>>> List instructions, subscriber options, unsubscribe:
>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>>
>>>
>>
>>
>> --
>> Adam Phillips
>> Associate Scientist,  Climate and Global Dynamics Laboratory, NCAR
>> www.cgd.ucar.edu/staff/asphilli/   303-497-1726 <(303)%20497-1726>
>>
>> <http://www.cgd.ucar.edu/staff/asphilli>
>>
>
> _______________________________________________
> ncl-talk mailing list
> ncl-talk at ucar.edu
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20180308/0583cc06/attachment.html>

More information about the ncl-talk mailing list