[ncl-talk] Writing binary characters while invoking WriteByteOrder

[Dave Allured - NOAA Affiliate]

Kay and NCL,

I can confirm that fbinrecwrite fails to correctly write an array of
characters in big endian mode.  I also found the same failure for types
byte and ubyte.  So I think this is a bug in NCL that needs fixing.

Here is a reduced test case.  Use any of x1, x2, or x3 in the write
statement.  I get valid little-endian output files when the setfileoption
statement is commented out.  I get the same good and bad results on both
Linux and Mac systems; both are little-endian X86 type systems.

  f = "test7.bin"
  x1 = tochar ("name7")
  x2 = tobyte (x1)
  x3 = toubyte (x1)
  setfileoption ("bin", "WriteByteOrder", "BigEndian")
  fbinrecwrite (f, 0, x1)

--Dave


On Thu, Jul 7, 2016 at 4:31 AM, Kay Shelton <kay.shelton at gmail.com> wrote:

> Hello ncl-talkers,
>
> I have a little bit of a conundrum with trying to write a binary file for
> a colleague that is destined to be used in TELEMAC-2D. The binary file that
> I am trying to write needs to include characters, integers and floats, and
> will be used on a machine that is Big Endian. I am creating the file on a
> Linux machine that is Little Endian, hence when I write the binary file I
> am using "setfileoption("bin","WriteByteOrder","BigEndian")". This all
> works fine for integers and floats, but the characters are not correctly
> represented in the output binary.
>
> I have written a short piece of code that demonstrates the problem
> succinctly:
>
> --------------------------------------------------------
> begin
>
> endianness = isbigendian()
> words = "blah_de_blah_de_blah"
> words_char = new(strlen(words),character)
> words_char = tochar(" ")
> words_char(0:strlen(words)-1) = tochar(words)
> outfil = "temp.bin"
>
> if (.not.endianness) then
>   print("Endianness is LittleEndian")
>
> ; run and look at results, then uncomment the next two lines and re-run
> ;  setfileoption("bin","ReadByteOrder","BigEndian")
> ;  setfileoption("bin","WriteByteOrder","BigEndian")
>
>   if (fileexists(outfil)) then
>     system("rm -f "+outfil)
>   end if
>   fbinrecwrite (outfil,0, (/ words_char /))
>   new_words = fbinrecread(outfil, 0, -1, "character")
>   print("WORDS (original): $$ "+words+" $$")
>   print("WORDS (original, char to str): $$ "+tostring(words_char)+" $$")
>   print("NEW_WORDS (char to str): $$ "+tostring(new_words)+" $$")
> else
>   print("Already BigEndian")
> end if
>
> end
> --------------------------------------------------------
>
> I have tested this on two completely different Linux machines running NCL
> 6.3.0 (both 64-bit Ubuntu) and also on one of them using NCL 6.2.1. All
> give the same result.
>
> Running the script first time with the two setfileoption lines commented
> out (i.e. native LittleEndian binary):
> (0)     Endianness is LittleEndian
> (0)     WORDS (original): $$ blah_de_blah_de_blah $$
> (0)     WORDS (original, char to str): $$ blah_de_blah_de_blah $$
> (0)     NEW_WORDS (char to str): $$ blah_de_blah_de_blah $$
>
> Running with the two setfileoption lines un-commented:
> (0)     Endianness is LittleEndian
> (0)     WORDS (original): $$ blah_de_blah_de_blah $$
> (0)     WORDS (original, char to str): $$ blah_de_blah_de_blah $$
> (0)     NEW_WORDS (char to str): $$  $$
>
> As you can see with the BigEndian option the characters are not correctly
> written to the file.
>
> Is this correct behaviour? Have I missed something I should have included
> to get this to work? (This is entirely possible and I apologise for wasting
> your time if this is a simple user-error.)
>
> I have searched the ncl-talk archives and have not found this problem
> encountered by others, so it may well be an issue exclusive to my machines.
> If it is, I can work around it by wrapping in some fortran to handle the
> reading and writing of the binary correctly. I would prefer to avoid such a
> work around as the size of the arrays in the binary files I have to read
> and write will vary, and I would prefer not to have to hard-code the array
> sizes when NCL makes calls to the fortran code.
>
> [Note: on the machine running NCL 6.2.1 I also get the following warning
> message on both attempts:
> warning:fbinrecwrite: end of file reached before record number, writing
> record as last record in file]
>
> Many thanks for your help,
> Kay
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20160708/51abdad0/attachment.html