[ncl-talk] "sum" and "avg" functions without ignoring missing values

[Rashed Mahmood]

Hi Alan,
Yes both Dave's and your suggestion work.

Thanks,
Rashed

On Fri, Nov 24, 2017 at 5:25 PM, Alan Brammer <abrammer at albany.edu> wrote:

> Rashed,
>
> How about the dim_avg_wgt_n() or dim_sum_wgt_n() variants.
> They have a flag to deal with missing values, so you ca
> ​n ​
> just use a wgt of 1 for all values
> ​ ​
> but still use the missing value flag
> ​​
> .
>
>
> http://www.ncl.ucar.edu/Document/Functions/Contributed/dim_
> sum_wgt_n_Wrap.shtml
> A scalar: (a) opt=0 means compute the weighted sum only if all values are
> not missing; (b) opt=1 means compute the weighted sum of all non-missing
> values; (c) opt>1 means to return the weighted sum only if the number of
> non-missing values is greater-than or equal to opt.
>
> E.g.
>
> a = (/(/-9.,2,-9/),(/1,-9.,2./)/)
> a at _FillValue = -9.
> a_dims = dimsizes(a)
> dim=0
> wgt = new( a_dims(dim), float)
> wgt = 1
> c = dim_sum_wgt_n(a,wgt,0, dim)
> print(c)
>
> Variable: c
> Type: float
> Total Size: 12 bytes
>             3 values
> Number of Dimensions: 1
> Dimensions and sizes: [3]
> Coordinates:
> Number Of Attributes: 1
>   _FillValue : -9
> (0) -9
> (1) -9
> (2) -9
>
>
>
>
> Good luck,
> ​Alan​
>
>
> On Fri, Nov 24, 2017 at 3:07 PM, Dave Allured - NOAA Affiliate <
> dave.allured at noaa.gov> wrote:
>
>> Oops, I missed something.  dim_cumsum returns an array of the same shape
>> as the input.  To get the desired "final sum" array of rank n-1, you must
>> take the last slice of the function result array.   For dim_cumsum_n, you
>> must similarly take the last slice along dimension number n.
>>
>>    Let a be dimensioned (3,4,5).  Then:
>>
>>    nz = dimsizes(a(0,0,:))
>>
>>    opt = 0
>>    b3d = dim_cumsum (a, opt)   ; dimensions (3,4,5)
>>    b   = b3d(:,:,nz-1)         ; dimensions (3,4)
>>
>> --Dave
>>
>>
>> On Fri, Nov 24, 2017 at 12:52 PM, Rashed Mahmood <rashidcomsis at gmail.com>
>> wrote:
>>
>>> Thanks Dave, that is exactly what I was looking for...
>>>
>>>
>>> On Fri, Nov 24, 2017 at 11:46 AM, Dave Allured - NOAA Affiliate <
>>> dave.allured at noaa.gov> wrote:
>>>
>>>> Rasheed,
>>>>
>>>> I think dim_cumsum and dim_cumsum_n with opt=0 are intended for this
>>>> purpose.  There is no corresponding function for "avg".  However, simply
>>>> dividing the result array from dim_cumsum by N should give the correct
>>>> array result, with no extra "if" or "where" statement.
>>>>
>>>> --Dave
>>>>
>>>>
>>>> On Fri, Nov 24, 2017 at 12:29 PM, Rashed Mahmood <
>>>> rashidcomsis at gmail.com> wrote:
>>>>
>>>>> Hi Arne,
>>>>>
>>>>> Thanks for trying to answer the question, however, the things you
>>>>> mentioned would add confusion and possibly wrong results.
>>>>> Actually I know how to achieve what I want using a do loop. For
>>>>> example, let us say we have variable a = (ntimes,nlevel,nlat,nlon) and we
>>>>> want to do sum over time domain which can easily be done using:
>>>>> (dim_sum_n(a,0)), Now the results would be sum over all non-missing values
>>>>> of time1,time2,time2...timeN, which is correct.
>>>>>
>>>>> But if I want to do sum ONLY when all of the values of
>>>>> time1,time2,time3...timeN are non-missing then we can do this:
>>>>>
>>>>>  dimz = dimsizes(a)
>>>>>  b      = new(/dimz(1),dimz(2),dimz(3),float)
>>>>>  b at _FillValue = a at _FillValue
>>>>>
>>>>>   b(:,:,:) = 0.
>>>>>   do n=0,dimz(0)-1
>>>>>        b = b+a(n,:,:,:)
>>>>>   end do
>>>>>
>>>>> I just wanted to avoid the loop and use the "sum" function for
>>>>> efficiency. I hope this clarifies the question a bit more.
>>>>>
>>>>> Cheers,
>>>>> Rashed
>>>>>
>>>>>
>>>>> On Fri, Nov 24, 2017 at 3:01 AM, Arne Melsom <arne.melsom at met.no>
>>>>> wrote:
>>>>>
>>>>>> Hi!
>>>>>> Since 'music piano' replied, I thought I might chime in ;)
>>>>>> I'm pretty sure that no option(s) to functions allow you to do what
>>>>>> you want.
>>>>>> So you'll need to write alternative functions yourself (perhaps not
>>>>>> what you consider 'an easy way'...)
>>>>>> You may use something like
>>>>>>
>>>>>> aDim = dimsizes(a)
>>>>>> orgFill = a at _FillValue
>>>>>> newFill = -min(abs(a)) - 1    ; make sure newFill is out of range and
>>>>>> <0
>>>>>> if (newFill .eq. orgFill) then  ; newFill must be different from
>>>>>> orgFill
>>>>>>   newFill = newFill - 1
>>>>>> end if
>>>>>> newVar = ndtooned(a)
>>>>>> newVar at _FillValue = newFill
>>>>>> indx = ind(ismissing(newVar))
>>>>>> newVar(indx) = orgFill
>>>>>> newa = onedtond(newVar,aDim)
>>>>>>
>>>>>> ...and, if you later want to restore:
>>>>>> a = newa
>>>>>> a at _FillValue = orgFill
>>>>>>
>>>>>> Hope this helps/clarifies!
>>>>>> Best regards,
>>>>>> Arne
>>>>>>
>>>>>>
>>>>>> 2017-11-24 1:49 GMT+01:00 Rashed Mahmood <rashidcomsis at gmail.com>:
>>>>>>
>>>>>>> Hello all,
>>>>>>> I am just wondering if there is an easy way to tell the "sum" and
>>>>>>> "avg" functions NOT ignore the missing values. Below lines clarify my
>>>>>>> question where I would like to see same values for c and d:
>>>>>>>
>>>>>>> ncl 0> a = (/(/-9.,2,-9./),(/1,-9.,2./)/)
>>>>>>> ncl 1> a at _FillValue = -9.
>>>>>>> ncl 2> c = (a(0,:)+a(1,:))
>>>>>>> ncl 3> d = dim_sum_n(a,0)
>>>>>>> ncl 4> print("  "+c+ " ... "+d)
>>>>>>> (0)       -9 ... 1
>>>>>>> (1)       -9 ... 2
>>>>>>> (2)       -9 ... 2
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Rashed
>>>>>>>
>>>>>>
>> _______________________________________________
>> ncl-talk mailing list
>> ncl-talk at ucar.edu
>> List instructions, subscriber options, unsubscribe:
>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20171124/c011fc0a/attachment.html>