[ncl-talk] reading binary data

[Dennis Shea]

The NCL equivalent of what you sent is:

  nlat   = 180
  mlon = 360
  lat    = fspan(-89.5,89.5,nlat)
  lon  = fspan(-179.5, 179.5,mlon)

  fili = "2015161_highQc_aot550_eps.dat"  ; total size 259200 (=
4*nlat*mlon)

  var1f = fbindirread(fili,0,(/nlat,mlon/),"float")
           ; or .... these are equivalent in this case
  var1c = cbinread(fili,(/nlat,mlon/),"float")

============
No error messages!! Everything looks great!

...... NOT .....

The golden rule of data processing is ***Look at your data***.

============

Did you try to print the min and max values in IDL?
Likely, they would be 'nice' numbers.

Did you try to print the min and max values in NCL?
If you did, you would see

(0)    min=nan   max=nan

=============

IDL (Matlab too!) recognizes NaN (nan) as values to be ignored. AKA
'missing values'

NCL recognizes the netCDF _FillValue attribute. This can *not* be a NaN.

You *must* change the NaNs to a real number for NCL.
Please read the following.

http://www.ncl.ucar.edu/Document/Functions/Built-in/isnan_ieee.shtml
http://www.ncl.ucar.edu/Document/Functions/Built-in/replace_ieeenan.shtml

Add the following at the end of the above. This is one example of how to
*Look at your data* !

  nNaN   = num(isnan_ieee(var1f))                              ; # of NaN
values
  pcNaN = (tofloat(nNaN)/(nlat*mlon))*100                  ; % NaN
  print("nNaN="+nNaN+"  pcNaN="+pcNaN+"%")

(0)    nNaN=33395  pcNaN=51.5355%                        ; 51.5%  ... more
than half

 printMinMax(var1f,0)         ; BEFORE
(0)    min=nan   max=nan

  if (any(isnan_ieee(var1f))) then
      value = 1.e20
      replace_ieeenan (var1f, value, 0)
      var1f at _FillValue = value
  end if

  printMinMax(var1f,0)       ; AFTER

min=-0.0499198   max=3.96129

========
The _FillValue attribute is fundamental to NCL.
Please read the documentation at:

http://www.ncl.ucar.edu/Document/Manuals/

Mini-Reference Manual

The official Reference manual

http://www.ncl.ucar.edu/Document/Manuals/Ref_Manual/NclVariables.shtml#MissingValueAttribute

Also, the two DKRZ Manuals at the above site.

=================

Further, the NCL plot code you sent in thye initial email whicj I think you
cut and pasted from some example will likely not yield a nice plot.

Likely, you will have to use raster mode.

res at cnFillMode       = "RasterFill"       ; Raster Mode



http://www.ncl.ucar.edu/Applications/raster.shtml







On Wed, May 11, 2016 at 9:52 AM, Debasish Hazra <debasish.hazra5 at gmail.com>
wrote:

> Thanks Dennis. But it still does no produce any figures. I have attached
> the script of ncl. Found the following instruction of how to read that in
> iDL :
>
>    res = 1.0                 ; 1.0 degree spatial resolution
>    nlon = ROUND(360./res)    ; number of columns (-180,180)
>    nlat = ROUND(180./res)    ; number of rows    (-90, 90)
>    lonFirst = -180+0.5*res   ; west boundary (first column)
>    latFirst = -90+0.5*res    ; south boundary (first row)
>
>    dailyGriddedAod550 = FLTARR(nlon, nlat)
>
>    date = '2015161'   ; string in YYYYDDD format
>
>    fileName = date+'_highQc_aot550_eps.dat'
>    OPENR, lun, /GET_LUN, fileName
>    READU, lun, dailyGriddedAod550
>    CLOSE, lun
>    FREE_LUN, lun
>
> Thanks.
> Debasish
>
> On Tue, May 10, 2016 at 4:21 PM, Dennis Shea <shea at ucar.edu> wrote:
>
>> Did you look at the size of your file??
>>
>> 259200  2015161_highQc_aot550_eps.dat
>> ^^^^^^^
>>
>> float or integer are 4 bytes
>>
>> 4*180*360 = 259200     so **one** float or integer variable
>>
>> You are using
>>
>>    setfileoption("bin","readbyteorder","littleendian")
>>
>>    var1 =
>> cbinread("./EPS_HQ/2015161_highQc_aot550_eps.dat",(/2,180,360/),"float")
>>    var2 =
>> cbinread("./EPS_HQ/2015161_highQc_aot550_eps.dat",(/2,180,360/),"integer")
>>
>> For two integer or float variables the size should be:  259200 + 259200 =
>> 518400
>>
>> At the very least, you should have been getting an NCL error message
>> about wrong size.
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> On Tue, May 10, 2016 at 12:32 PM, Debasish Hazra <
>> debasish.hazra5 at gmail.com> wrote:
>>
>>> Hi,
>>>
>>> I am trying to read binary data with ncl (v6.3) but not sure whether it
>>> is reading it right. Also, it does not produce any plots. Attached are the
>>> script presently working on and sample data. Any help on this is well
>>> appreciated.
>>>
>>> Thanks.
>>> Debasish.
>>>
>>> _______________________________________________
>>> ncl-talk mailing list
>>> ncl-talk at ucar.edu
>>> List instructions, subscriber options, unsubscribe:
>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>>
>>>
>>
>
> _______________________________________________
> ncl-talk mailing list
> ncl-talk at ucar.edu
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20160511/c3d2fa25/attachment.html