Dear Dennis and Dave,<div>thanks for pointing me to the relevant function. It works! </div><div><br></div><div>I wasn't aware of the difference between NaN ans "_FillValue". I naively thought they referred to the very same thing. This is all good to know.</div><div><br></div><div>Cheers,</div><div>Giorgio</div><div><div>
<br>
<blockquote>
----Messaggio originale----
<br> Da: shea@ucar.edu
<br> Data: 10-giu-2020 3.38
<br> A: "Dave Allured - NOAA Affiliate"<dave.allured@noaa.gov>
<br> Cc: "Giorgio Graffino"<g.graffino@tim.it>, "Ncl-talk"<ncl-talk@ucar.edu>
<br> Ogg: Re: [ncl-talk] Problem on averaging variable
<br>
<br>
<div dir="ltr">
<div>
As previously noted, NCL recognizes "
<b><a href="https://www.unidata.ucar.edu/software/netcdf/docs/attribute_conventions.html">_FillValue</a></b>"
<br>
</div>
<div>
<br>
</div>
<div>
It is the user responsibility change NaNs to _FillValue
</div>
<div>
<br>
</div>
<div>
See:
<a href="http://www.ncl.ucar.edu/Document/Functions/Built-in/replace_ieeenan.shtml"><b>replace_ieeenan</b></a>
</div>
<div>
It is easy to use:
</div>
<div>
<b><br></b>
</div>
<div>
x = f->.....
<b><br></b>
</div>
<div>
<pre> if (<a href="http://www.ncl.ucar.edu/Document/Functions/Built-in/any.shtml"><strong>any</strong></a>(<a href="http://www.ncl.ucar.edu/Document/Functions/Built-in/isnan_ieee.shtml"><strong>isnan_ieee</strong></a>(x))) then
if(.not.<a href="http://www.ncl.ucar.edu/Document/Functions/Built-in/isatt.shtml"><strong>isatt</strong></a>(x,"_FillValue")) then
x@_FillValue = <a href="http://www.ncl.ucar.edu/Document/Functions/Built-in/default_fillvalue.shtml"><strong>default_fillvalue</strong></a>(<a href="http://www.ncl.ucar.edu/Document/Functions/Built-in/typeof.shtml"><strong>typeof</strong></a>(x))
end if
<strong>replace_ieeenan</strong> (x, x@_FillValue, 0)
end if</pre>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">
On Tue, Jun 9, 2020 at 4:09 PM Dave Allured - NOAA Affiliate via ncl-talk <
<a href="javascript:handleMailto('mailto:ncl-talk@mailman.ucar.edu');return false;">ncl-talk@mailman.ucar.edu</a>> wrote:
<br>
</div>
<blockquote class="gmail_quote" style="margin: 0.0px 0.0px 0.0px 0.8ex;border-left: 1.0px solid rgb(204,204,204);padding-left: 1.0ex;">
<div dir="ltr">
<div dir="ltr">
<div>
It looks like your input data contains NaN's. Otherwise you would probably never see this.
<br>
<br>> average_op_ncl : dim_avg_n over dimension(s): lon
<br>> (0) min=nan max=nan
<br>
<br>In NCL, NaN's ara NaN's, not missing values. Support for NaN's is not clean. If it was, we would not be having this conversation. ;-)
</div>
<div>
<br>
</div>
<div>
There is more than one way to fix this. Best practice IMO is to convert all NaN's to missing values after reading input data, and before any other calculations (except maybe for subsetting). Please review NCL docs for detecting, counting, and replacing NaN's. In particular, see the isnan_ieee function.
</div>
<div>
<br>
</div>
<div>
Depending on how much you care, you might also want to take a quick look at the spatial and temporal distributions of NaN's in that model's file set. NaN's are an anomaly. They may or may not indicate a significant quality problem in that data, something you might want to know about before further efforts.
</div>
<div>
<br>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">
On Tue, Jun 9, 2020 at 3:41 PM Giorgio Graffino via ncl-talk <
<a href="javascript:handleMailto('mailto:ncl-talk@mailman.ucar.edu');return false;">ncl-talk@mailman.ucar.edu</a>> wrote:
<br>
</div>
<blockquote class="gmail_quote" style="margin: 0.0px 0.0px 0.0px 0.8ex;border-left: 1.0px solid rgb(204,204,204);padding-left: 1.0ex;">
<font face="tahoma, arial, helvetica, sans-serif"><span style="font-size: 12.0px;">Dear NCL users,</span></font>
<div style="font-family: tahoma , arial , helvetica , sans-serif;font-size: 12.0px;">
I have a weird issue on computing a simple avarage of a variable.
</div>
<div style="font-family: tahoma , arial , helvetica , sans-serif;font-size: 12.0px;">
<br>
</div>
<div style="font-family: tahoma , arial , helvetica , sans-serif;font-size: 12.0px;">
I'm analysing a large subset of CMIP models to obtain density plots. However, there is a particular model I can't analyse: when I compute the average, all values in the variable becomes missing values. Here is the relevant snippet
</div>
<div style="font-family: tahoma , arial , helvetica , sans-serif;font-size: 12.0px;">
<br>
</div>
<div>
<div>
<font face="tahoma, arial, helvetica, sans-serif"><span style="font-size: 12.0px;"><i> printVarSummary(rho_9s_mask)</i></span></font>
</div>
<div>
<font face="tahoma, arial, helvetica, sans-serif"><span style="font-size: 12.0px;"><i> printMinMax(rho_9s_mask,False) </i></span></font>
</div>
<div>
<font face="tahoma, arial, helvetica, sans-serif"><span style="font-size: 12.0px;"><i><br></i></span></font>
</div>
<div>
<font face="tahoma, arial, helvetica, sans-serif"><span style="font-size: 12.0px;"><i>;; Create cross sections</i></span></font>
</div>
<div>
<span style="font-size: 12.0px;font-family: tahoma , arial , helvetica , sans-serif;"><i> rho_9s_mask_ave = dim_avg_n_Wrap(rho_9s_mask,2)</i></span>
</div>
<div>
<span style="font-size: 12.0px;font-family: tahoma , arial , helvetica , sans-serif;"><i> copy_VarCoords_1(rho_9s_mask,rho_9s_mask_ave)</i></span>
</div>
<div>
<span style="font-size: 12.0px;font-family: tahoma , arial , helvetica , sans-serif;"><i> </i></span>
</div>
<div>
<font face="tahoma, arial, helvetica, sans-serif"><span style="font-size: 12.0px;"><i> printVarSummary(rho_9s_mask_ave)</i></span></font>
</div>
<div>
<font face="tahoma, arial, helvetica, sans-serif"><span style="font-size: 12.0px;"><i> printMinMax(rho_9s_mask_ave,False) </i></span></font>
</div>
</div>
<div style="font-family: tahoma , arial , helvetica , sans-serif;font-size: 12.0px;">
<br>
</div>
<div style="font-family: tahoma , arial , helvetica , sans-serif;font-size: 12.0px;">
and here is what I obtain
</div>
<div style="font-family: tahoma , arial , helvetica , sans-serif;font-size: 12.0px;">
<br>
</div>
<div style="font-family: tahoma , arial , helvetica , sans-serif;font-size: 12.0px;">
<span style="font-family: monospace;"><span style="background-color: rgb(255,255,255);">Variable: rho_9s_mask </span><br>Type: float <br>Total Size: 14493600 bytes <br> 3623400 values <br>Number of Dimensions: 3 <br>Dimensions and sizes: [time | 165] x [lev | 61] x [lon | 360] <br>Coordinates: <br> time: [181.25..60081.25] <br> lev: [ 1..6525] <br> lon: [ 0..358.9818] <br>Number Of Attributes: 1 <br> _FillValue : 1e+20 <br>(0) min=1020.5 max=1027.72 <br><br>Variable: rho_9s_mask_ave <br>Type: float <br>Total Size: 40260 bytes <br> 10065 values <br>Number of Dimensions: 2 <br>Dimensions and sizes: [time | 165] x [lev | 61] <br>Coordinates: <br> time: [181.25..60081.25] <br> lev: [ 1..6525] <br>Number Of Attributes: 2 <br> _FillValue : 1e+20 <br> average_op_ncl : dim_avg_n over dimension(s): lon <br>(0) min=nan max=nan<br><br></span>
</div>
<div style="font-family: tahoma , arial , helvetica , sans-serif;font-size: 12.0px;">
This works for all models but one. I tried to use a different function for the average, but nothing changes. I also tried to compute the average on other dimensions: the problem occurs again when averaging on the lev coordinate, but it doesn't when averaging on the time coordinate. It seems that there is a problem in the coordinate, but everything looks alright in the variable before the averaging.
</div>
<div style="font-family: tahoma , arial , helvetica , sans-serif;font-size: 12.0px;">
<br>
</div>
<div style="font-family: tahoma , arial , helvetica , sans-serif;font-size: 12.0px;">
Do you have any guess of what's wrong or any suggestion to help me?
</div>
<div style="font-family: tahoma , arial , helvetica , sans-serif;font-size: 12.0px;">
<br>
</div>
<div style="font-family: tahoma , arial , helvetica , sans-serif;font-size: 12.0px;">
Thanks a lot,
</div>
<div style="font-family: tahoma , arial , helvetica , sans-serif;font-size: 12.0px;">
Giorgio
</div>
</blockquote>
</div>
</div>
</div> _______________________________________________
<br> ncl-talk mailing list
<br>
<a href="javascript:handleMailto('mailto:ncl-talk@mailman.ucar.edu');return false;">ncl-talk@mailman.ucar.edu</a>
<br> List instructions, subscriber options, unsubscribe:
<br>
<a href="https://mailman.ucar.edu/mailman/listinfo/ncl-talk" rel="noreferrer">https://mailman.ucar.edu/mailman/listinfo/ncl-talk</a>
</blockquote>
</div>
<br>
</blockquote>
<br></div></div>