[ncl-talk] reading binary data

[Debasish Hazra]

Many thanks Dennis, it worked.

Deba.

On Wed, May 11, 2016 at 1:37 PM, Dennis Shea <shea at ucar.edu> wrote:

> The NCL equivalent of what you sent is:
>
>   nlat   = 180
>   mlon = 360
>   lat    = fspan(-89.5,89.5,nlat)
>   lon  = fspan(-179.5, 179.5,mlon)
>
>   fili = "2015161_highQc_aot550_eps.dat"  ; total size 259200 (=
> 4*nlat*mlon)
>
>   var1f = fbindirread(fili,0,(/nlat,mlon/),"float")
>            ; or .... these are equivalent in this case
>   var1c = cbinread(fili,(/nlat,mlon/),"float")
>
> ============
> No error messages!! Everything looks great!
>
> ...... NOT .....
>
> The golden rule of data processing is ***Look at your data***.
>
> ============
>
> Did you try to print the min and max values in IDL?
> Likely, they would be 'nice' numbers.
>
> Did you try to print the min and max values in NCL?
> If you did, you would see
>
> (0)    min=nan   max=nan
>
> =============
>
> IDL (Matlab too!) recognizes NaN (nan) as values to be ignored. AKA
> 'missing values'
>
> NCL recognizes the netCDF _FillValue attribute. This can *not* be a NaN.
>
> You *must* change the NaNs to a real number for NCL.
> Please read the following.
>
> http://www.ncl.ucar.edu/Document/Functions/Built-in/isnan_ieee.shtml
> http://www.ncl.ucar.edu/Document/Functions/Built-in/replace_ieeenan.shtml
>
> Add the following at the end of the above. This is one example of how to
> *Look at your data* !
>
>   nNaN   = num(isnan_ieee(var1f))                              ; # of NaN
> values
>   pcNaN = (tofloat(nNaN)/(nlat*mlon))*100                  ; % NaN
>   print("nNaN="+nNaN+"  pcNaN="+pcNaN+"%")
>
> (0)    nNaN=33395  pcNaN=51.5355%                        ; 51.5%  ... more
> than half
>
>  printMinMax(var1f,0)         ; BEFORE
> (0)    min=nan   max=nan
>
>   if (any(isnan_ieee(var1f))) then
>       value = 1.e20
>       replace_ieeenan (var1f, value, 0)
>       var1f at _FillValue = value
>   end if
>
>   printMinMax(var1f,0)       ; AFTER
>
> min=-0.0499198   max=3.96129
>
> ========
> The _FillValue attribute is fundamental to NCL.
> Please read the documentation at:
>
> http://www.ncl.ucar.edu/Document/Manuals/
>
> Mini-Reference Manual
>
> The official Reference manual
>
>
> http://www.ncl.ucar.edu/Document/Manuals/Ref_Manual/NclVariables.shtml#MissingValueAttribute
>
> Also, the two DKRZ Manuals at the above site.
>
> =================
>
> Further, the NCL plot code you sent in thye initial email whicj I think
> you cut and pasted from some example will likely not yield a nice plot.
>
> Likely, you will have to use raster mode.
>
> res at cnFillMode       = "RasterFill"       ; Raster Mode
>
>
>
> http://www.ncl.ucar.edu/Applications/raster.shtml
>
>
>
>
>
>
>
> On Wed, May 11, 2016 at 9:52 AM, Debasish Hazra <debasish.hazra5 at gmail.com
> > wrote:
>
>> Thanks Dennis. But it still does no produce any figures. I have attached
>> the script of ncl. Found the following instruction of how to read that in
>> iDL :
>>
>>    res = 1.0                 ; 1.0 degree spatial resolution
>>    nlon = ROUND(360./res)    ; number of columns (-180,180)
>>    nlat = ROUND(180./res)    ; number of rows    (-90, 90)
>>    lonFirst = -180+0.5*res   ; west boundary (first column)
>>    latFirst = -90+0.5*res    ; south boundary (first row)
>>
>>    dailyGriddedAod550 = FLTARR(nlon, nlat)
>>
>>    date = '2015161'   ; string in YYYYDDD format
>>
>>    fileName = date+'_highQc_aot550_eps.dat'
>>    OPENR, lun, /GET_LUN, fileName
>>    READU, lun, dailyGriddedAod550
>>    CLOSE, lun
>>    FREE_LUN, lun
>>
>> Thanks.
>> Debasish
>>
>> On Tue, May 10, 2016 at 4:21 PM, Dennis Shea <shea at ucar.edu> wrote:
>>
>>> Did you look at the size of your file??
>>>
>>> 259200  2015161_highQc_aot550_eps.dat
>>> ^^^^^^^
>>>
>>> float or integer are 4 bytes
>>>
>>> 4*180*360 = 259200     so **one** float or integer variable
>>>
>>> You are using
>>>
>>>    setfileoption("bin","readbyteorder","littleendian")
>>>
>>>    var1 =
>>> cbinread("./EPS_HQ/2015161_highQc_aot550_eps.dat",(/2,180,360/),"float")
>>>    var2 =
>>> cbinread("./EPS_HQ/2015161_highQc_aot550_eps.dat",(/2,180,360/),"integer")
>>>
>>> For two integer or float variables the size should be:  259200 + 259200
>>> = 518400
>>>
>>> At the very least, you should have been getting an NCL error message
>>> about wrong size.
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> On Tue, May 10, 2016 at 12:32 PM, Debasish Hazra <
>>> debasish.hazra5 at gmail.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> I am trying to read binary data with ncl (v6.3) but not sure whether it
>>>> is reading it right. Also, it does not produce any plots. Attached are the
>>>> script presently working on and sample data. Any help on this is well
>>>> appreciated.
>>>>
>>>> Thanks.
>>>> Debasish.
>>>>
>>>> _______________________________________________
>>>> ncl-talk mailing list
>>>> ncl-talk at ucar.edu
>>>> List instructions, subscriber options, unsubscribe:
>>>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>>>
>>>>
>>>
>>
>> _______________________________________________
>> ncl-talk mailing list
>> ncl-talk at ucar.edu
>> List instructions, subscriber options, unsubscribe:
>> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20160513/1e62e0f8/attachment.html