[ncl-talk] "sum" and "avg" functions without ignoring missing values

[Alan Brammer]

Rashed,

How about the dim_avg_wgt_n() or dim_sum_wgt_n() variants.
They have a flag to deal with missing values, so you ca
​n ​
just use a wgt of 1 for all values
​ ​
but still use the missing value flag
​​
.


http://www.ncl.ucar.edu/Document/Functions/Contributed/dim_sum_wgt_n_
Wrap.shtml
A scalar: (a) opt=0 means compute the weighted sum only if all values are
not missing; (b) opt=1 means compute the weighted sum of all non-missing
values; (c) opt>1 means to return the weighted sum only if the number of
non-missing values is greater-than or equal to opt.

E.g.

a = (/(/-9.,2,-9/),(/1,-9.,2./)/)
a at _FillValue = -9.
a_dims = dimsizes(a)
dim=0
wgt = new( a_dims(dim), float)
wgt = 1
c = dim_sum_wgt_n(a,wgt,0, dim)
print(c)

Variable: c
Type: float
Total Size: 12 bytes
            3 values
Number of Dimensions: 1
Dimensions and sizes: [3]
Coordinates:
Number Of Attributes: 1
  _FillValue : -9
(0) -9
(1) -9
(2) -9




Good luck,
​Alan​


On Fri, Nov 24, 2017 at 3:07 PM, Dave Allured - NOAA Affiliate <
dave.allured at noaa.gov> wrote:

> Oops, I missed something.  dim_cumsum returns an array of the same shape
> as the input.  To get the desired "final sum" array of rank n-1, you must
> take the last slice of the function result array.   For dim_cumsum_n, you
> must similarly take the last slice along dimension number n.
>
>    Let a be dimensioned (3,4,5).  Then:
>
>    nz = dimsizes(a(0,0,:))
>
>    opt = 0
>    b3d = dim_cumsum (a, opt)   ; dimensions (3,4,5)
>    b   = b3d(:,:,nz-1)         ; dimensions (3,4)
>
> --Dave
>
>
> On Fri, Nov 24, 2017 at 12:52 PM, Rashed Mahmood <rashidcomsis at gmail.com>
> wrote:
>
>> Thanks Dave, that is exactly what I was looking for...
>>
>>
>> On Fri, Nov 24, 2017 at 11:46 AM, Dave Allured - NOAA Affiliate <
>> dave.allured at noaa.gov> wrote:
>>
>>> Rasheed,
>>>
>>> I think dim_cumsum and dim_cumsum_n with opt=0 are intended for this
>>> purpose.  There is no corresponding function for "avg".  However, simply
>>> dividing the result array from dim_cumsum by N should give the correct
>>> array result, with no extra "if" or "where" statement.
>>>
>>> --Dave
>>>
>>>
>>> On Fri, Nov 24, 2017 at 12:29 PM, Rashed Mahmood <rashidcomsis at gmail.com
>>> > wrote:
>>>
>>>> Hi Arne,
>>>>
>>>> Thanks for trying to answer the question, however, the things you
>>>> mentioned would add confusion and possibly wrong results.
>>>> Actually I know how to achieve what I want using a do loop. For
>>>> example, let us say we have variable a = (ntimes,nlevel,nlat,nlon) and we
>>>> want to do sum over time domain which can easily be done using:
>>>> (dim_sum_n(a,0)), Now the results would be sum over all non-missing values
>>>> of time1,time2,time2...timeN, which is correct.
>>>>
>>>> But if I want to do sum ONLY when all of the values of
>>>> time1,time2,time3...timeN are non-missing then we can do this:
>>>>
>>>>  dimz = dimsizes(a)
>>>>  b      = new(/dimz(1),dimz(2),dimz(3),float)
>>>>  b at _FillValue = a at _FillValue
>>>>
>>>>   b(:,:,:) = 0.
>>>>   do n=0,dimz(0)-1
>>>>        b = b+a(n,:,:,:)
>>>>   end do
>>>>
>>>> I just wanted to avoid the loop and use the "sum" function for
>>>> efficiency. I hope this clarifies the question a bit more.
>>>>
>>>> Cheers,
>>>> Rashed
>>>>
>>>>
>>>> On Fri, Nov 24, 2017 at 3:01 AM, Arne Melsom <arne.melsom at met.no>
>>>> wrote:
>>>>
>>>>> Hi!
>>>>> Since 'music piano' replied, I thought I might chime in ;)
>>>>> I'm pretty sure that no option(s) to functions allow you to do what
>>>>> you want.
>>>>> So you'll need to write alternative functions yourself (perhaps not
>>>>> what you consider 'an easy way'...)
>>>>> You may use something like
>>>>>
>>>>> aDim = dimsizes(a)
>>>>> orgFill = a at _FillValue
>>>>> newFill = -min(abs(a)) - 1    ; make sure newFill is out of range and
>>>>> <0
>>>>> if (newFill .eq. orgFill) then  ; newFill must be different from
>>>>> orgFill
>>>>>   newFill = newFill - 1
>>>>> end if
>>>>> newVar = ndtooned(a)
>>>>> newVar at _FillValue = newFill
>>>>> indx = ind(ismissing(newVar))
>>>>> newVar(indx) = orgFill
>>>>> newa = onedtond(newVar,aDim)
>>>>>
>>>>> ...and, if you later want to restore:
>>>>> a = newa
>>>>> a at _FillValue = orgFill
>>>>>
>>>>> Hope this helps/clarifies!
>>>>> Best regards,
>>>>> Arne
>>>>>
>>>>>
>>>>> 2017-11-24 1:49 GMT+01:00 Rashed Mahmood <rashidcomsis at gmail.com>:
>>>>>
>>>>>> Hello all,
>>>>>> I am just wondering if there is an easy way to tell the "sum" and
>>>>>> "avg" functions NOT ignore the missing values. Below lines clarify my
>>>>>> question where I would like to see same values for c and d:
>>>>>>
>>>>>> ncl 0> a = (/(/-9.,2,-9./),(/1,-9.,2./)/)
>>>>>> ncl 1> a at _FillValue = -9.
>>>>>> ncl 2> c = (a(0,:)+a(1,:))
>>>>>> ncl 3> d = dim_sum_n(a,0)
>>>>>> ncl 4> print("  "+c+ " ... "+d)
>>>>>> (0)       -9 ... 1
>>>>>> (1)       -9 ... 2
>>>>>> (2)       -9 ... 2
>>>>>>
>>>>>> Thanks,
>>>>>> Rashed
>>>>>>
>>>>>
> _______________________________________________
> ncl-talk mailing list
> ncl-talk at ucar.edu
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20171124/3ad93cee/attachment.html>