<div dir="ltr"><div class="gmail_default" style="font-family:arial,sans-serif;font-size:small">Hi Dave:</div><div class="gmail_default" style="font-family:arial,sans-serif;font-size:small">Many thanks for the email you sent me last Friday. You were completely right when you wrote that the data set was originally written as unsigned bytes. This definitely agreed with the min and max values of the Reflectivity data that I was getting. I think that I was slowly getting there but you definitely helped me speed up the process. Always nice to start Monday with something accomplished over the weekend.</div><div class="gmail_default" style="font-family:arial,sans-serif;font-size:small"><br></div><div class="gmail_default" style="font-family:arial,sans-serif;font-size:small">Instead of applying the different steps that you suggested, I simply used the function <font color="#0000ff">tounsigned </font><font color="#000000">before the function </font><font color="#0000ff">byte2flt. </font><font color="#000000">This led to getting the correct min and max of for the radar reflectivity, and reflectivity maps consistent with the publication I was using as a reference.</font></div><div class="gmail_default" style="font-family:arial,sans-serif;font-size:small"><font color="#000000"><br></font></div><div class="gmail_default" style="font-family:arial,sans-serif;font-size:small"><font color="#000000">Really appreciate your help. Thanks again,</font></div><div class="gmail_default" style="font-family:arial,sans-serif;font-size:small"><font color="#000000">Laura</font></div><div class="gmail_default" style="font-family:arial,sans-serif;font-size:small"><font color="#000000"><br></font></div><div class="gmail_default" style="font-family:arial,sans-serif;font-size:small"><font color="#000000"><br></font></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Jul 14, 2023 at 6:51 PM Dave Allured - NOAA Affiliate <<a href="mailto:dave.allured@noaa.gov">dave.allured@noaa.gov</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>A long time, ago, there was ambiguity within Netcdf and NCL, as to whether type "byte" was signed or unsigned. Eventually everyone settled on "signed". NCL is attempting to interpret this byte array as signed, but there are some clues that this file may have been originally written as unsigned bytes. That would cause NCL to misinterpret the data. This would also be consistent with your symptoms.</div><div dir="ltr"><br></div><div>Unsigned maps into signed like this. 0 to 127 map straight across to 0 to 127. 128 to 255 maps to -128 to -1. To unscramble, try this custom formula (untested). It is important to unscramble BEFORE unpacking with scale and offset.</div><div><br></div><div><span style="font-family:arial,sans-serif">d</span>BZ_b = f->Reflectivity ! read from file<br></div><div>dbz2 = toshort (dBZ_b) ! convert to a type that can hold full positive range</div><div>dbz3 = where ( (dbz2 .lt. 0), (dbz2 + 256), dbz2) ! decode negatives into positives</div><div>copy_VarMeta (dBZ_b, dbz3) ! restore original metadata</div><div>dbz4 = short2flt (dbz3) ! unpack with scale and offset</div><div>printMinMax (dbz4,0)</div><div><br></div><div>If that doesn't come out right, then use printVarSummary and printMinMax at every step, to see what is going on. Another diagnostic would be to count discrete original values, and examine their distribution curve.</div><div><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Jul 14, 2023 at 5:57 PM Laura Fowler via ncl-talk <<a href="mailto:ncl-talk@mailman.ucar.edu" target="_blank">ncl-talk@mailman.ucar.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div style="font-family:arial,sans-serif;font-size:small">Hello:</div><div style="font-family:arial,sans-serif;font-size:small"><br></div><div style="font-family:arial,sans-serif;font-size:small">I am trying to read Nexrad Reflectivity data from a netCDF file in which Reflectivity has a "byte" format and is defined as:</div><div style="font-family:arial,sans-serif">
<p style="font-size:14px;margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:Courier;color:rgb(0,0,0)"><span style="color:rgb(0,0,255);font-variant-ligatures:no-common-ligatures;font-size:small">radar-data/20120611> ncdump -h 20120611T1900Z.nc</span></p><p style="font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-variant-east-asian:normal;font-kerning:auto;font-feature-settings:normal;font-weight:normal;font-stretch:normal;line-height:normal;font-family:Courier;margin:0px;background-color:rgb(255,255,255)"><span style="font-variant-ligatures:no-common-ligatures"><font color="#0000ff"><span style="white-space:pre-wrap"> </span>byte Reflectivity(Altitude, Latitude, Longitude) ;</font></span></p><p style="font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-variant-east-asian:normal;font-kerning:auto;font-feature-settings:normal;font-weight:normal;font-stretch:normal;line-height:normal;font-family:Courier;margin:0px;background-color:rgb(255,255,255)"><span style="font-variant-ligatures:no-common-ligatures"><font color="#0000ff"><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>Reflectivity:long_name = "Radar Reflectivity" ;</font></span></p><p style="font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-variant-east-asian:normal;font-kerning:auto;font-feature-settings:normal;font-weight:normal;font-stretch:normal;line-height:normal;font-family:Courier;margin:0px;background-color:rgb(255,255,255)"><span style="font-variant-ligatures:no-common-ligatures"><font color="#0000ff"><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>Reflectivity:units = "dBZ" ;</font></span></p><p style="font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-variant-east-asian:normal;font-kerning:auto;font-feature-settings:normal;font-weight:normal;font-stretch:normal;line-height:normal;font-family:Courier;margin:0px;background-color:rgb(255,255,255)"><span style="font-variant-ligatures:no-common-ligatures"><font color="#0000ff"><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>Reflectivity:scale_factor = 0.5f ;</font></span></p><p style="font-style:normal;font-variant-ligatures:normal;font-variant-caps:normal;font-variant-east-asian:normal;font-kerning:auto;font-feature-settings:normal;font-weight:normal;font-stretch:normal;line-height:normal;font-family:Courier;margin:0px;background-color:rgb(255,255,255)"><span style="font-variant-ligatures:no-common-ligatures"><font color="#0000ff"><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>Reflectivity:add_offset = -33.f ;</font></span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:Courier"><span style="font-variant-ligatures:no-common-ligatures"><font color="#0000ff">
</font></span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:Courier"><span style="font-variant-ligatures:no-common-ligatures"><font color="#0000ff"><span style="white-space:pre-wrap"> </span><span style="white-space:pre-wrap"> </span>Reflectivity:missing_value = 0b ;</font></span></p></div><div style="font-family:arial,sans-serif;font-size:small"><br></div><div style="font-family:arial,sans-serif;font-size:small">A printVarSummary of the dBZ_b variable that points to Reflectivity shows:</div><div style="font-family:arial,sans-serif">
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:Courier"><span style="font-variant-ligatures:no-common-ligatures"><font color="#0000ff">Variable: dBZ_b</font></span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:Courier"><span style="font-variant-ligatures:no-common-ligatures"><font color="#0000ff">Type: byte</font></span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:Courier"><span style="font-variant-ligatures:no-common-ligatures"><font color="#0000ff">Total Size: 10064208 bytes</font></span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:Courier"><span style="font-variant-ligatures:no-common-ligatures"><font color="#0000ff"><span> </span>10064208 values</font></span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:Courier"><span style="font-variant-ligatures:no-common-ligatures"><font color="#0000ff">Number of Dimensions: 3</font></span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:Courier"><span style="font-variant-ligatures:no-common-ligatures"><font color="#0000ff">Dimensions and sizes:<span style="white-space:pre-wrap"> </span>[Altitude | 24] x [Latitude | 539] x [Longitude | 778]</font></span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:Courier"><span style="font-variant-ligatures:no-common-ligatures"><font color="#0000ff">Coordinates:<span> </span></font></span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:Courier"><span style="font-variant-ligatures:no-common-ligatures"><font color="#0000ff"><span> </span>Altitude: [ 1..24]</font></span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:Courier"><span style="font-variant-ligatures:no-common-ligatures"><font color="#0000ff"><span> </span>Latitude: [30.94..41.7]</font></span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:Courier"><span style="font-variant-ligatures:no-common-ligatures"><font color="#0000ff"><span> </span>Longitude: [260.12..275.66]</font></span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:Courier"><span style="font-variant-ligatures:no-common-ligatures"><font color="#0000ff">Number Of Attributes: 6</font></span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:Courier"><span style="font-variant-ligatures:no-common-ligatures"><font color="#0000ff"><span> </span>long_name :<span style="white-space:pre-wrap"> </span>Radar Reflectivity</font></span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:Courier"><span style="font-variant-ligatures:no-common-ligatures"><font color="#0000ff"><span> </span>units :<span style="white-space:pre-wrap"> </span>dBZ</font></span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:Courier"><span style="font-variant-ligatures:no-common-ligatures"><font color="#0000ff"><span> </span>scale_factor :<span style="white-space:pre-wrap"> </span>0.5</font></span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:Courier"><span style="font-variant-ligatures:no-common-ligatures"><font color="#0000ff"><span> </span>add_offset :<span style="white-space:pre-wrap"> </span>-33</font></span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:Courier"><span style="font-variant-ligatures:no-common-ligatures"><font color="#0000ff"><span> </span>missing_value :<span style="white-space:pre-wrap"> </span>0</font></span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:Courier"><span style="font-variant-ligatures:no-common-ligatures"><font color="#0000ff"><span> </span>_FillValue :<span style="white-space:pre-wrap"> </span>0</font></span></p></div><div style="font-family:arial,sans-serif"><font color="#0000ff"><br></font></div><div style="font-family:arial,sans-serif;font-size:small">After reading the section on "Coercion of types", and for now ignoring scale_factor and add_offset, I simply used the tofloat function to output the max and min values of dBZ_b. I got:</div><div>
<p style="font-family:Courier;margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal"><span style="font-variant-ligatures:no-common-ligatures"><font color="#0000ff">(0)<span style="white-space:pre-wrap"> </span>--- max dBZ_b = 127</font></span></p>
<p style="font-family:Courier;margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal"><span style="font-variant-ligatures:no-common-ligatures"><font color="#0000ff">(0)<span style="white-space:pre-wrap"> </span>--- min dBZ_b = -128</font></span></p><p style="font-family:Courier;margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal"><span style="font-variant-ligatures:no-common-ligatures"><font color="#0000ff"><br></font></span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal"><span style="font-variant-ligatures:no-common-ligatures"><font face="arial, sans-serif" color="#000000">Finally, using scale_factor and add_offset, I used the function byte2flt to convert dBZ_b to a float and output the max and min value of dBZ_s, or:</font></span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal"><span style="font-variant-ligatures:no-common-ligatures">
</span></p><p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:Courier;color:rgb(0,0,0)"><span style="color:rgb(0,0,255);font-variant-ligatures:no-common-ligatures">dBZ_s = byte2flt(dBZ_b)</span><br></p></div><div style="font-family:arial,sans-serif">
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:Courier"><span style="font-variant-ligatures:no-common-ligatures"><font color="#0000ff">(0)<span style="white-space:pre-wrap"> </span>--- max dBZ_s = 30.5</font></span></p>
<p style="margin:0px;font-variant-numeric:normal;font-variant-east-asian:normal;font-variant-alternates:normal;font-kerning:auto;font-feature-settings:normal;font-stretch:normal;line-height:normal;font-family:Courier"><span style="font-variant-ligatures:no-common-ligatures"><font color="#0000ff">(0)<span style="white-space:pre-wrap"> </span>--- min dBZ_s = -97</font></span></p></div><div><br></div><div><div style="font-family:arial,sans-serif;font-size:small">The maximum value of dBZ_s is much lower than I expected (the maximum value of dBZ_s should actually be 58.5 dBZ, as derived by an IDL script used to plot the nexrad Reflectivity data for a peer-review publication).</div><div style="font-family:arial,sans-serif;font-size:small"><br></div><div style="font-family:arial,sans-serif"><span style="font-size:small">Can somebody help me figure out why I am getting values that are too low relative to what I am expecting. I also tried to bypass the byte2flt function and use the scale_factor and add_offset variables </span><font color="#0000ff">(<span style="font-family:courier">x_float = x_byte*scale + offset</span>)</font>, but to no avail.</div><div style="font-family:arial,sans-serif;font-size:small"><br></div><div style="font-family:arial,sans-serif">If you interested to help me out, my working directory on cheyenne is <font color="#0000ff"><span style="font-variant-ligatures:no-common-ligatures;font-family:Courier">/glade/u/home/laura/fromYunYao/radar-data/20120611; the data file is </span><span style="font-variant-ligatures:no-common-ligatures;font-family:Courier">20120611T1900Z.nc; the ncl script is read.ncl; and the main idl script is <a href="http://nexrad.pro" target="_blank">nexrad.pro</a>.</span></font></div>
<div style="font-family:arial,sans-serif;font-size:small"><br></div><div style="font-family:arial,sans-serif;font-size:small">Many thanks,</div><div style="font-family:arial,sans-serif;font-size:small">Laura</div></div><span class="gmail_signature_prefix">-- </span><br><div dir="ltr" class="gmail_signature"><div dir="ltr"><font face="arial, helvetica, sans-serif" color="#0000ff" style="background-color:rgb(255,255,255)">!-------------------------------------------------------------------------------------------------------------</font><div><font face="arial, helvetica, sans-serif" color="#0000ff" style="background-color:rgb(255,255,255)">Laura D. Fowler </font></div><div><font face="arial, helvetica, sans-serif" color="#0000ff" style="background-color:rgb(255,255,255)">Mesoscale and Microscale Meteorology Division (MMM) </font></div><div><font face="arial, helvetica, sans-serif" color="#0000ff" style="background-color:rgb(255,255,255)">National Center for Atmospheric Research<br></font></div><div><font face="arial, helvetica, sans-serif" color="#0000ff" style="background-color:rgb(255,255,255)">P.O. Box 3000, Boulder CO 80307-3000</font></div><div><font face="arial, helvetica, sans-serif" color="#0000ff" style="background-color:rgb(255,255,255)"><br></font></div><div><font face="arial, helvetica, sans-serif" color="#0000ff" style="background-color:rgb(255,255,255)">e-mail: <a href="mailto:laura@ucar.edu" target="_blank">laura@ucar.edu</a></font></div><div><font face="arial, helvetica, sans-serif" color="#0000ff" style="background-color:rgb(255,255,255)">phone: 303-497-1628</font></div><div><font face="arial, helvetica, sans-serif" color="#0000ff" style="background-color:rgb(255,255,255)">!-------------------------------------------------------------------------------------------------------------</font></div></div></div></div>
</blockquote></div></div>
</blockquote></div><br clear="all"><div><br></div><span class="gmail_signature_prefix">-- </span><br><div dir="ltr" class="gmail_signature"><div dir="ltr"><font face="arial, helvetica, sans-serif" color="#0000ff" style="background-color:rgb(255,255,255)">!-------------------------------------------------------------------------------------------------------------</font><div><font face="arial, helvetica, sans-serif" color="#0000ff" style="background-color:rgb(255,255,255)">Laura D. Fowler </font></div><div><font face="arial, helvetica, sans-serif" color="#0000ff" style="background-color:rgb(255,255,255)">Mesoscale and Microscale Meteorology Division (MMM) </font></div><div><font face="arial, helvetica, sans-serif" color="#0000ff" style="background-color:rgb(255,255,255)">National Center for Atmospheric Research<br></font></div><div><font face="arial, helvetica, sans-serif" color="#0000ff" style="background-color:rgb(255,255,255)">P.O. Box 3000, Boulder CO 80307-3000</font></div><div><font face="arial, helvetica, sans-serif" color="#0000ff" style="background-color:rgb(255,255,255)"><br></font></div><div><font face="arial, helvetica, sans-serif" color="#0000ff" style="background-color:rgb(255,255,255)">e-mail: <a href="mailto:laura@ucar.edu" target="_blank">laura@ucar.edu</a></font></div><div><font face="arial, helvetica, sans-serif" color="#0000ff" style="background-color:rgb(255,255,255)">phone: 303-497-1628</font></div><div><font face="arial, helvetica, sans-serif" color="#0000ff" style="background-color:rgb(255,255,255)"><br></font></div><div><font face="arial, helvetica, sans-serif" color="#0000ff" style="background-color:rgb(255,255,255)">!-------------------------------------------------------------------------------------------------------------</font><br></div></div></div>