<div dir="ltr"><div class="gmail_default" style="font-size:small">Hi Debasish,</div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">Dennis guess that maybe the "read_binary" function in IDL was meant to read files created by "write_binary" but I didn't see a function with that name. However, is it possible that this is some kind of special IDL file and not a flat C binary file?</div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">In your IDL script, you have:</div><div class="gmail_default" style="font-size:small">
<p class="m_-2903204906668778697m_1278222147089946618gmail-p1"><span class="m_-2903204906668778697m_1278222147089946618gmail-s1"><font face="monospace, monospace">fdata=read_binary('viirs_meand<wbr>bdi_gridded_statis2013'+day+'.<wbr>dat',data_type=5,data_dims=[<wbr>1440,720])</font></span></p><p class="m_-2903204906668778697m_1278222147089946618gmail-p1"><span class="m_-2903204906668778697m_1278222147089946618gmail-s1">If you read the documentation for "read_binary", it states that "data_type=5" is double. </span></p><p class="m_-2903204906668778697m_1278222147089946618gmail-p1">In your NCL script, you are reading the data as an unsigned integer.</p><p class="m_-2903204906668778697m_1278222147089946618gmail-p1">I tried reading your data as a double, but I get what looks like nonsensical values:</p><p class="m_-2903204906668778697m_1278222147089946618gmail-p1">
</p><p class="m_-2903204906668778697m_1278222147089946618gmail-p1"><span class="m_-2903204906668778697m_1278222147089946618gmail-s1"><font face="monospace, monospace"><span class="m_-2903204906668778697m_1278222147089946618gmail-Apple-tab-span"> </span>min=-1.642556686681977e+308 max=6.633924105807938e+307</font></span></p></div><div class="gmail_default" style="font-size:small">You are right that the unsigned integer values look reasonable, but only after you multiply them by 1e-9. </div><div class="gmail_default" style="font-size:small"><br></div>When I look at your unsigned values, I see that <div class="gmail_default" style="font-size:small;display:inline">517,484 </div>of your values are equal to the same number: 6.3615e-05, while only<div class="gmail_default" style="font-size:small;display:inline"> 1,831</div> values are equal to something else.<div class="gmail_default" style="font-size:small;display:inline"> This seems a bit suspicious to me, and is likely the source of the problem.</div><div><br></div><div><div class="gmail_default" style="font-size:small">I modified your script to plot red markers where the values are all equal to 6.3615e-05, and black markers everywhere else. Does this look correct?</div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">I have a feeling that there's something more to the "read_binary" function that we need to know in order to read the file correctly. As I think I mentioned before: perhaps each byte of data represents something different, and you need to use something like dim_gbits to pick off values.</div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">In your IDL script, is there anything you have to do additionally to the data before you plot it? Can you check the IDL script to see if you are getting a lot of values equal to the same constant value that NCL is?</div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">--Mary</div><br><div class="gmail_default" style="font-size:small"><br></div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Mar 30, 2017 at 8:36 AM, Debasish Hazra <span dir="ltr"><<a href="mailto:debasish.hazra5@gmail.com" target="_blank">debasish.hazra5@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>Mary,</div><div><br></div><div>Thanks.Taking your suggestion and reading that as 2 * 720 * 1440 and assuming input as C binary file, I am getting min=1.4e-08 max=4.29371 , which is reasonble. Attached is the new script. Any suggestions.</div><span class="HOEnZb"><font color="#888888"><div><br></div><div>Debasish</div></font></span></div><div class="gmail_extra"><br><div class="gmail_quote"><span class="">On Wed, Mar 29, 2017 at 5:28 PM, Mary Haley <span dir="ltr"><<a href="mailto:haley@ucar.edu" target="_blank">haley@ucar.edu</a>></span> wrote:<br></span><div><div class="h5"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div class="gmail_default" style="font-size:small">Hi Debasish,</div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">Kevin and I took a look at this. For starters, there *is* an error message coming out of your script:</div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default"><div class="gmail_default" style="font-size:small"><font face="monospace, monospace">warning:cbinread: The size implied by the dimension arrays is greater that the size of the file.</font></div><div class="gmail_default" style="font-size:small"><font face="monospace, monospace"> The default _FillValue for the specified type will be filled in.</font></div><div class="gmail_default" style="font-size:small"><font face="monospace, monospace"> Note dimensions and values may not be aligned properly</font></div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">If you look at the size of the file, it doesn't match with the dimensions you're requesting:</div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">Size of file = 8294400 bytes</div><br>Size of dimensions = 5 * 720 * 1440 * 4 (for a uint) = 20736000</div><div class="gmail_default"><br></div><div class="gmail_default">If this is truly a C binary file, it looks like it only has 2 * 720 * 1440 * 4 bytes.</div><div class="gmail_default"><br></div><div class="gmail_default">This doesn't really change the results, however, because you still get two strange looking plots.</div><div class="gmail_default"><br></div><div class="gmail_default">We tried several different things:</div><div class="gmail_default"><br></div><div class="gmail_default">1) reading the data as ubyte, int, and ushort</div><div class="gmail_default">2) reversing the array to 1440 x 720 x 2</div><div class="gmail_default">3) reading the data as little endian</div><div class="gmail_default">4) plotting the data as a simple contour plot to take out the map component.</div><div class="gmail_default"><br></div><div class="gmail_default">Nothing we did produced more information about the file, or produced better plots.</div><div class="gmail_default"><br></div><div class="gmail_default">Is there some documentation on this file to understand how it was written? For example, are you sure the "uint" type is correct? Are you sure the dimension sizes are correct? Why are the values so large? Is it possible that this is "packed" data, and that you need to use a function like dim_gbits to pick off individual bits of information?</div><div class="gmail_default"><br></div><div class="gmail_default">If you can find a C or Fortran code that was used to create this file, then it should be fairly straightforward to figure out how to read it.</div><div class="gmail_default"><br></div><div class="gmail_default">--Mary</div><div class="gmail_default"><br></div><div class="gmail_default"><br></div><div class="gmail_default"><br></div><div class="gmail_default"><br></div><div class="gmail_default"><br></div></div><div class="gmail_extra"><br><div class="gmail_quote"><span>On Wed, Mar 29, 2017 at 2:18 PM, Debasish Hazra <span dir="ltr"><<a href="mailto:debasish.hazra5@gmail.com" target="_blank">debasish.hazra5@gmail.com</a>></span> wrote:<br></span><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;padding-left:1ex;border-left-color:rgb(204,204,204);border-left-width:1px;border-left-style:solid"><span><div dir="ltr"><div>Hi,</div><div><br></div><div><div>I am trying to read a binary file with the attached code, but getting all empty fields in the figure with no apparent error message. Uploaded the data file in the ftp server "viirs_meandbdi_gridded_statis<wbr>2015048.dat". Any help with this is appreciated.</div><div><br></div><div>Thanks.</div><span class="m_-2690639159146125938m_-2697389384942006388HOEnZb"><font color="#888888"><div>Debasish</div></font></span></div></div><div class="m_-2690639159146125938m_-2697389384942006388HOEnZb"><div class="m_-2690639159146125938m_-2697389384942006388h5"><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Mar 22, 2017 at 10:33 AM, Debasish Hazra <span dir="ltr"><<a href="mailto:debasish.hazra5@gmail.com" target="_blank">debasish.hazra5@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;padding-left:1ex;border-left-color:rgb(204,204,204);border-left-width:1px;border-left-style:solid"><div dir="ltr"><div>Hi,</div><div><br></div><div>I am trying to read a binary file with the attached code, but getting all empty fields in the figure with no apparent error message. Uploaded the data file in the ftp server "viirs_meandbdi_gridded_statis<wbr>2015002.dat". Any help with this is appreciated.</div><div><br></div><div>Thanks.</div><span class="m_-2690639159146125938m_-2697389384942006388m_3158856751608814800HOEnZb"><font color="#888888"><div>Debasish.</div><div><br></div><div><br></div></font></span></div>
</blockquote></div><br></div>
</div></div><br></span>______________________________<wbr>_________________<br>
ncl-talk mailing list<br>
<a href="mailto:ncl-talk@ucar.edu" target="_blank">ncl-talk@ucar.edu</a><br>
List instructions, subscriber options, unsubscribe:<br>
<a href="http://mailman.ucar.edu/mailman/listinfo/ncl-talk" rel="noreferrer" target="_blank">http://mailman.ucar.edu/mailma<wbr>n/listinfo/ncl-talk</a><br>
<br></blockquote></div><br></div>
</blockquote></div></div></div><br></div>
</blockquote></div><br></div>