[ncl-talk] binary file

[Dave Allured - NOAA Affiliate]

Debasish,

(1)  Since you already know the dimensions of the 2-D array, save yourself
some trouble and read the file directly into a new 2-D array by inserting
the actual dimensions into the third argument of *fbindirread*.  Replace
"-1" with (/nlat,mlon/), and remove the *onedtond* conversion.

(2)  You have a plain binary file with no metadata.  NCL does not know the
grid coordinates until you add them by one method or another.  This is why
we prefer Netcdf files with metadata conveniently included.

For one way to add coordinates, please see example 2 on the *Read binary
data* example page.  In particular, notice the way that *fspan* is used to
generate linear coordinate arrays.  You do not need to write a new Netcdf
file like the example shows, but add lat and lon coordinate arrays before
going to NCL plot functions.  Do you know what the coordinates are supposed
to be?

Use *printVarSummary* and *printMinMax* to evaluate your data array at
different processing stages.  Use *printMinMax* to check that your data to
plot is in the expected numeric range.  Also, missing value handling can be
added if you think you might have missing values in the array.


On Fri, Oct 9, 2020 at 3:21 PM Debasish Hazra <debasish.hazra5 at gmail.com>
wrote:

> Thanks Dave. When I use fbinrecread as shown below , I am still getting
> warning and generated figure does not show any value.
> setfileoption("bin","readbyteorder","littleendian")
>  d1 = fbindirread("./Test/modelout.dat",0,-1,"float")
>
>            printVarSummary(d1)
>            print(d1(0:100))
>             nlat  = 384
>            mlon  = 384
>            UNDEF = -999.0
>
>            N1d   = dimsizes(d1)
>             M1d   = N1d-(nlat*mlon)
>            print("N1d="+N1d+"  M1d="+M1d)
>
>            x     = onedtond(d1(1:), (/nlat,mlon/))
>            x at _FillValue = UNDEF                     ;
>            printVarSummary(x)
>           printMinMax(x,0)
>
>     Variable: d1
> Type: float
> Total Size: 589832 bytes
>             147458 values
> Number of Dimensions: 1
> Dimensions and sizes:   [147458]
> Coordinates:
>
> Variable: d1 (subsection)
> Type: float
> Total Size: 404 bytes
>             101 values
> Number of Dimensions: 1
> Dimensions and sizes:   [101]
> Coordinates:
> (0)     8.265195e-40
> (1)      0
> .....[SNIP]
>
>
> *(0)     N1d=147458  M1d=2warning:onedtond : output dimension sizes have
> fewer elements than input, some data not copied*
> Variable: x
> Type: float
> Total Size: 589824 bytes
>             147456 values
> Number of Dimensions: 2
> Dimensions and sizes:   [384] x [384]
> Coordinates:
> Number Of Attributes: 1
>   _FillValue :  -999
> (0)     min=0   max=6.12579e-07,
>
> On Fri, Oct 9, 2020 at 2:55 PM Dave Allured - NOAA Affiliate <
> dave.allured at noaa.gov> wrote:
>
>> That looks like fortran unformatted sequential format, not flat binary.
>> Confirm by seeing if the first four bytes are 00 00 09 00, the same as the
>> last four bytes that you showed.  This would be the record size as a
>> little-endian 4-byte integer, which would be correct for this format.  This
>> would also explain the 8-byte difference.
>>
>> Read in NCL with fbinrecread, not cbinread.  Please see the function docs
>> and follow the prototype.
>>
>>
>> On Fri, Oct 9, 2020 at 12:34 PM Rick Brownrigg via ncl-talk <
>> ncl-talk at mailman.ucar.edu> wrote:
>>
>>> Well I don't think you want to read it as double, particularly not if
>>> you are certain the dimensions are 384x384. I'm wondering if you can safely
>>> ignore that last 8 bytes (and ignore the warning).  A quick plot would
>>> reveal whether the data seem reasonable or definitely misaligned.
>>>
>>> RB
>>>
>>> On Fri, Oct 9, 2020 at 12:14 PM Debasish Hazra <
>>> debasish.hazra5 at gmail.com> wrote:
>>>
>>>> Thanks Rick.Should I read it as double?
>>>>
>>>> On Fri, Oct 9, 2020 at 1:39 PM Rick Brownrigg <brownrig at ucar.edu>
>>>> wrote:
>>>>
>>>>> Well, the math works out: 4x384x384=589824.  Further, the last 8 bytes
>>>>> of that file do not look like valid floating point values. They are:
>>>>>
>>>>>  00 00 00 00 00 00 09 00
>>>>>
>>>>> Do you have reason to believe the data is not being read correctly, or
>>>>> are you mostly concerned about the warning message?
>>>>>
>>>>> BTW - the data are mostly zeros.
>>>>>
>>>>> Rick
>>>>>
>>>>>
>>>>> On Fri, Oct 9, 2020 at 11:09 AM Debasish Hazra via ncl-talk <
>>>>> ncl-talk at mailman.ucar.edu> wrote:
>>>>>
>>>>>> Hi,
>>>>>> I am trying to read binary file (attached in the email) through NCL
>>>>>> and file size is 589832 bytes. I am reading it as float but it coming out
>>>>>> to be 589824. I am not sure what needs to be done to read it.
>>>>>>
>>>>>>  setfileoption("bin","readbyteorder","littleendian")
>>>>>>  d1 = cbinread("./Test/modelout.dat",(/4,384,384/),"float")
>>>>>>
>>>>>> warning:cbinread: The size implied by the dimension arrays is less
>>>>>> that the size of the file.
>>>>>>  Only the first 589824 contiguous bytes of the file will be read in.
>>>>>> Note dimensions and values may not be aligned properly
>>>>>>
>>>>>> Any help on this is appreciated.
>>>>>> Thanks
>>>>>> Debasish
>>>>>>
>>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mailman.ucar.edu/pipermail/ncl-talk/attachments/20201009/879e2086/attachment.html>