<div dir="ltr">Thanks Gus. Mary and myself both tried "endian" options, and presently trying with <span lang="EN"><p>"setfileoption("bin","readbyteorder","bigendian") option which seems to produce reasonable minimum and maximum of data values. However, as Mary mentioned large number of values are constant whcih is bit strange. </p></span><div>You mentioned about "double" and I think input is in "<span style="color:rgb(34,34,34);text-transform:none;text-indent:0px;letter-spacing:normal;font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-weight:normal;word-spacing:0px;float:none;display:inline;white-space:normal;background-color:rgb(255,255,255)">double precision floating point data and it is 8 bytes".</span></div><div><span style="color:rgb(34,34,34);text-transform:none;text-indent:0px;letter-spacing:normal;font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-weight:normal;word-spacing:0px;float:none;display:inline;white-space:normal;background-color:rgb(255,255,255)"><br></span></div><div><span style="color:rgb(34,34,34);text-transform:none;text-indent:0px;letter-spacing:normal;font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-weight:normal;word-spacing:0px;float:none;display:inline;white-space:normal;background-color:rgb(255,255,255)">Thanks.</span></div><div><span style="color:rgb(34,34,34);text-transform:none;text-indent:0px;letter-spacing:normal;font-family:arial,sans-serif;font-size:12.8px;font-style:normal;font-weight:normal;word-spacing:0px;float:none;display:inline;white-space:normal;background-color:rgb(255,255,255)">Debasish</span></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Mar 30, 2017 at 4:06 PM, Gus Correa <span dir="ltr"><<a href="mailto:gus@ldeo.columbia.edu" target="_blank">gus@ldeo.columbia.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi Mary, Debasish<br>
<br>
Could it be a little-endian vs. big-endian issue?<br>
I don't know IDL (I should! My boss uses it! :) )<br>
but their "read_binary" default endianness is "native" (like NCL).<br>
I.e., the endianness of the data on the file depends on the<br>
machine it was created (and data_type=5 is indeed double precision).<br>
<br>
Maybe using setfileoption('bin',"<wbr>ReadByteOrder","BigEndian"),<br>
and trying also "LittleEndian" if not lucky with "Big"<br>
(who knows where the file was written ....),<br>
then cbinread/fbindirread with datatype "double" would help?<br>
Just a guess, and you probably tried the endianness thing already ...<br>
<br>
Best,<br>
Gus Correa<br>
<div><div class="h5"><br>
On 03/30/2017 02:54 PM, Mary Haley wrote:<br>
> Hi Debasish,<br>
><br>
> Dennis guess that maybe the "read_binary" function in IDL was meant to<br>
> read files created by "write_binary" but I didn't see a function with<br>
> that name. However, is it possible that this is some kind of special IDL<br>
> file and not a flat C binary file?<br>
><br>
> In your IDL script, you have:<br>
><br>
> fdata=read_binary('viirs_<wbr>meandbdi_gridded_statis2013'+<wbr>day+'.dat',data_type=5,data_<wbr>dims=[1440,720])<br>
><br>
> If you read the documentation for "read_binary", it states that<br>
> "data_type=5" is double.<br>
><br>
> In your NCL script, you are reading the data as an unsigned integer.<br>
><br>
> I tried reading your data as a double, but I get what looks like<br>
> nonsensical values:<br>
><br>
> min=-1.642556686681977e+308 max=6.633924105807938e+307<br>
><br>
> You are right that the unsigned integer values look reasonable, but only<br>
> after you multiply them by 1e-9.<br>
><br>
> When I look at your unsigned values, I see that<br>
> 517,484<br>
> of your values are equal to the same number: 6.3615e-05, while only<br>
> 1,831<br>
> values are equal to something else.<br>
> This seems a bit suspicious to me, and is likely the source of the<br>
> problem.<br>
><br>
> I modified your script to plot red markers where the values are all<br>
> equal to 6.3615e-05, and black markers everywhere else. Does this look<br>
> correct?<br>
><br>
> I have a feeling that there's something more to the "read_binary"<br>
> function that we need to know in order to read the file correctly. As I<br>
> think I mentioned before: perhaps each byte of data represents something<br>
> different, and you need to use something like dim_gbits to pick off values.<br>
><br>
> In your IDL script, is there anything you have to do additionally to the<br>
> data before you plot it? Can you check the IDL script to see if you are<br>
> getting a lot of values equal to the same constant value that NCL is?<br>
><br>
> --Mary<br>
><br>
><br>
><br>
> On Thu, Mar 30, 2017 at 8:36 AM, Debasish Hazra<br>
</div></div><span>> <<a href="mailto:debasish.hazra5@gmail.com">debasish.hazra5@gmail.com</a> <mailto:<a href="mailto:debasish.hazra5@gmail.com">debasish.hazra5@gmail.<wbr>com</a>>> wrote:<br>
><br>
> Mary,<br>
><br>
> Thanks.Taking your suggestion and reading that as 2 * 720 * 1440 and<br>
> assuming input as C binary file, I am getting min=1.4e-08<br>
> max=4.29371 , which is reasonble. Attached is the new script. Any<br>
> suggestions.<br>
><br>
> Debasish<br>
><br>
> On Wed, Mar 29, 2017 at 5:28 PM, Mary Haley <<a href="mailto:haley@ucar.edu">haley@ucar.edu</a><br>
</span><div><div class="h5">> <mailto:<a href="mailto:haley@ucar.edu">haley@ucar.edu</a>>> wrote:<br>
><br>
> Hi Debasish,<br>
><br>
> Kevin and I took a look at this. For starters, there *is* an<br>
> error message coming out of your script:<br>
><br>
> warning:cbinread: The size implied by the dimension arrays is<br>
> greater that the size of the file.<br>
> The default _FillValue for the specified type will be filled in.<br>
> Note dimensions and values may not be aligned properly<br>
><br>
> If you look at the size of the file, it doesn't match with the<br>
> dimensions you're requesting:<br>
><br>
> Size of file = 8294400 bytes<br>
><br>
> Size of dimensions = 5 * 720 * 1440 * 4 (for a uint) = 20736000<br>
><br>
> If this is truly a C binary file, it looks like it only has 2 *<br>
> 720 * 1440 * 4 bytes.<br>
><br>
> This doesn't really change the results, however, because you<br>
> still get two strange looking plots.<br>
><br>
> We tried several different things:<br>
><br>
> 1) reading the data as ubyte, int, and ushort<br>
> 2) reversing the array to 1440 x 720 x 2<br>
> 3) reading the data as little endian<br>
> 4) plotting the data as a simple contour plot to take out the<br>
> map component.<br>
><br>
> Nothing we did produced more information about the file, or<br>
> produced better plots.<br>
><br>
> Is there some documentation on this file to understand how it<br>
> was written? For example, are you sure the "uint" type is<br>
> correct? Are you sure the dimension sizes are correct? Why are<br>
> the values so large? Is it possible that this is "packed" data,<br>
> and that you need to use a function like dim_gbits to pick off<br>
> individual bits of information?<br>
><br>
> If you can find a C or Fortran code that was used to create this<br>
> file, then it should be fairly straightforward to figure out how<br>
> to read it.<br>
><br>
> --Mary<br>
><br>
><br>
><br>
><br>
><br>
><br>
> On Wed, Mar 29, 2017 at 2:18 PM, Debasish Hazra<br>
</div></div>> <<a href="mailto:debasish.hazra5@gmail.com">debasish.hazra5@gmail.com</a> <mailto:<a href="mailto:debasish.hazra5@gmail.com">debasish.hazra5@gmail.<wbr>com</a>>><br>
<span>> wrote:<br>
><br>
> Hi,<br>
><br>
> I am trying to read a binary file with the attached code,<br>
> but getting all empty fields in the figure with no apparent<br>
> error message. Uploaded the data file in the ftp server<br>
> "viirs_meandbdi_gridded_<wbr>statis2015048.dat". Any help with<br>
> this is appreciated.<br>
><br>
> Thanks.<br>
> Debasish<br>
><br>
> On Wed, Mar 22, 2017 at 10:33 AM, Debasish Hazra<br>
> <<a href="mailto:debasish.hazra5@gmail.com">debasish.hazra5@gmail.com</a><br>
</span><span>> <mailto:<a href="mailto:debasish.hazra5@gmail.com">debasish.hazra5@gmail.<wbr>com</a>>> wrote:<br>
><br>
> Hi,<br>
><br>
> I am trying to read a binary file with the attached<br>
> code, but getting all empty fields in the figure with<br>
> no apparent error message. Uploaded the data file in<br>
> the ftp server<br>
> "viirs_meandbdi_gridded_<wbr>statis2015002.dat". Any help<br>
> with this is appreciated.<br>
><br>
> Thanks.<br>
> Debasish.<br>
><br>
><br>
><br>
><br>
> ______________________________<wbr>_________________<br>
> ncl-talk mailing list<br>
</span>> <a href="mailto:ncl-talk@ucar.edu">ncl-talk@ucar.edu</a> <mailto:<a href="mailto:ncl-talk@ucar.edu">ncl-talk@ucar.edu</a>><br>
<span>> List instructions, subscriber options, unsubscribe:<br>
> <a href="http://mailman.ucar.edu/mailman/listinfo/ncl-talk" target="_blank" rel="noreferrer">http://mailman.ucar.edu/<wbr>mailman/listinfo/ncl-talk</a><br>
</span>> <<a href="http://mailman.ucar.edu/mailman/listinfo/ncl-talk" target="_blank" rel="noreferrer">http://mailman.ucar.edu/<wbr>mailman/listinfo/ncl-talk</a>><br>
<div class="HOEnZb"><div class="h5">><br>
><br>
><br>
><br>
><br>
><br>
> ______________________________<wbr>_________________<br>
> ncl-talk mailing list<br>
> <a href="mailto:ncl-talk@ucar.edu">ncl-talk@ucar.edu</a><br>
> List instructions, subscriber options, unsubscribe:<br>
> <a href="http://mailman.ucar.edu/mailman/listinfo/ncl-talk" target="_blank" rel="noreferrer">http://mailman.ucar.edu/<wbr>mailman/listinfo/ncl-talk</a><br>
><br>
<br>
______________________________<wbr>_________________<br>
ncl-talk mailing list<br>
<a href="mailto:ncl-talk@ucar.edu">ncl-talk@ucar.edu</a><br>
List instructions, subscriber options, unsubscribe:<br>
<a href="http://mailman.ucar.edu/mailman/listinfo/ncl-talk" target="_blank" rel="noreferrer">http://mailman.ucar.edu/<wbr>mailman/listinfo/ncl-talk</a><br>
</div></div></blockquote></div><br></div>