[ncl-talk] problem removing missing values from obs dataset

[Dave Allured - NOAA Affiliate]

That was not quite right.  My apologies.  Please see documentation for the
*where* function.  It is actually legal for the conditional mask to contain
missing values.  But when the mask contains missing values, then missing
values are returned in the output array, in the corresponding positions.
This is the behavior that you saw originally.

The remedy is the same as before, use *where (ismissing ())* as shown below.


On Thu, Feb 27, 2020 at 5:11 PM Dave Allured - NOAA Affiliate <
dave.allured at noaa.gov> wrote:

> This is not valid:
>
>    *VAR = where (VAR.eq.VAR at _FillValue, 0., VAR)*
>
> ... because the array result of the *".eq."* operator, with fill values
> present, can contain three values:  True, False, and fill value.  The
> *"where"* function must receive a conditional mask (first argument) that
> contains only *True* and *False*.
>
> This construction will form the desired missing value mask, and do what
> you intended:
>
>    *VAR = where (ismissing (VAR), 0., VAR)*
>
> Next, if *VAR at missing_value* is different than the original
> *VAR at _FillValue*, then you should keep the second statement the way you
> had it, to change *"missing_values"* to zero.
>
>
> On Thu, Feb 27, 2020 at 4:50 PM Adam Herrington via ncl-talk <
> ncl-talk at ucar.edu> wrote:
>
>> Hi All,
>>
>> I have a variable that I am trying to remove the missing values from.
>> Here are the attributes from printVarSummary call:
>>
>> Variable: VAR
>> Type: float
>> Total Size: 8220672 bytes
>>             2055168 values
>> Number of Dimensions: 3
>> Dimensions and sizes: [time | 223] x [lat | 32] x [lon | 288]
>> Coordinates:
>>             time: [9504..10170]
>>             lat: [-14.60732984293194..14.60732984293194]
>>             lon: [   0..358.75]
>> Number Of Attributes: 9
>>   remap : remapped via ESMF_regrid_with_weights: Bilinear
>>   _FillValue : -9999.9
>>   missing_value : -9999.9
>>   time_statistic : instantaneous
>>   units :
>>   long_name : precipitation
>>   comments : Unknown1 variable comment
>>   delta_t : 0000-00-00 03:00:00
>>   avg_period : 0000-00-00 01:00:00
>>
>> So there are two missing value attributes. To set all missing values to
>> zero, I try:
>>
>>   VAR = where(VAR.eq.VAR at _FillValue,0.,VAR)
>>   VAR = where(VAR.eq.VAR at missing_value,0.,VAR)
>>
>> But then subsequently looking for missing values: nMsg =
>> num(ismissing(VAR)), it says there are still all the missing values:
>>
>> (0) missing values present: not allowed: nMsg=17913
>>
>> Does anyone see why this is occurring, why my where statements are not
>> resulting in num(ismissing(VAR))=0?
>>
>> Thanks,
>>
>> Adam
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20200227/3c13c759/attachment.html>