[ncl-talk] Writing binary characters while invoking WriteByteOrder

[Dave Allured - NOAA Affiliate]

Thanks, Dennis.

Kay, here is a workaround to write characters in big endian mode.  This
uses unsigned shorts to circumvent the current problem with single-byte
data types.

  f = "test8.bin"
  str = "abcdef"
  ubytes  = toubyte (tochar (str))
  ushorts = toushort (256 * ubytes(0::2) + ubytes(1::2))
  setfileoption ("bin", "WriteByteOrder", "BigEndian")
  fbinrecwrite (f, 0, ushorts)

This trick requires an even number of characters in the string to be
written.  HTH.

--Dave


On Fri, Jul 8, 2016 at 4:08 PM, Dennis Shea <shea at ucar.edu> wrote:

> FYI: A JIRA ticket has been opened: NCL_2462
>
>
> On Fri, Jul 8, 2016 at 3:10 PM, Dave Allured - NOAA Affiliate <
> dave.allured at noaa.gov> wrote:
>
>> Kay and NCL,
>>
>> I can confirm that fbinrecwrite fails to correctly write an array of
>> characters in big endian mode.  I also found the same failure for types
>> byte and ubyte.  So I think this is a bug in NCL that needs fixing.
>>
>> Here is a reduced test case.  Use any of x1, x2, or x3 in the write
>> statement.  I get valid little-endian output files when the setfileoption
>> statement is commented out.  I get the same good and bad results on both
>> Linux and Mac systems; both are little-endian X86 type systems.
>>
>>   f = "test7.bin"
>>   x1 = tochar ("name7")
>>   x2 = tobyte (x1)
>>   x3 = toubyte (x1)
>>   setfileoption ("bin", "WriteByteOrder", "BigEndian")
>>   fbinrecwrite (f, 0, x1)
>>
>> --Dave
>>
>>
>> On Thu, Jul 7, 2016 at 4:31 AM, Kay Shelton <kay.shelton at gmail.com>
>> wrote:
>>
>>> Hello ncl-talkers,
>>>
>>> I have a little bit of a conundrum with trying to write a binary file
>>> for a colleague that is destined to be used in TELEMAC-2D. The binary file
>>> that I am trying to write needs to include characters, integers and floats,
>>> and will be used on a machine that is Big Endian. I am creating the file on
>>> a Linux machine that is Little Endian, hence when I write the binary file I
>>> am using "setfileoption("bin","WriteByteOrder","BigEndian")". This all
>>> works fine for integers and floats, but the characters are not correctly
>>> represented in the output binary.
>>>
>>> I have written a short piece of code that demonstrates the problem
>>> succinctly:
>>>
>>> --------------------------------------------------------
>>> begin
>>>
>>> endianness = isbigendian()
>>> words = "blah_de_blah_de_blah"
>>> words_char = new(strlen(words),character)
>>> words_char = tochar(" ")
>>> words_char(0:strlen(words)-1) = tochar(words)
>>> outfil = "temp.bin"
>>>
>>> if (.not.endianness) then
>>>   print("Endianness is LittleEndian")
>>>
>>> ; run and look at results, then uncomment the next two lines and re-run
>>> ;  setfileoption("bin","ReadByteOrder","BigEndian")
>>> ;  setfileoption("bin","WriteByteOrder","BigEndian")
>>>
>>>   if (fileexists(outfil)) then
>>>     system("rm -f "+outfil)
>>>   end if
>>>   fbinrecwrite (outfil,0, (/ words_char /))
>>>   new_words = fbinrecread(outfil, 0, -1, "character")
>>>   print("WORDS (original): $$ "+words+" $$")
>>>   print("WORDS (original, char to str): $$ "+tostring(words_char)+" $$")
>>>   print("NEW_WORDS (char to str): $$ "+tostring(new_words)+" $$")
>>> else
>>>   print("Already BigEndian")
>>> end if
>>>
>>> end
>>> --------------------------------------------------------
>>>
>>> I have tested this on two completely different Linux machines running
>>> NCL 6.3.0 (both 64-bit Ubuntu) and also on one of them using NCL 6.2.1. All
>>> give the same result.
>>>
>>> Running the script first time with the two setfileoption lines commented
>>> out (i.e. native LittleEndian binary):
>>> (0)     Endianness is LittleEndian
>>> (0)     WORDS (original): $$ blah_de_blah_de_blah $$
>>> (0)     WORDS (original, char to str): $$ blah_de_blah_de_blah $$
>>> (0)     NEW_WORDS (char to str): $$ blah_de_blah_de_blah $$
>>>
>>> Running with the two setfileoption lines un-commented:
>>> (0)     Endianness is LittleEndian
>>> (0)     WORDS (original): $$ blah_de_blah_de_blah $$
>>> (0)     WORDS (original, char to str): $$ blah_de_blah_de_blah $$
>>> (0)     NEW_WORDS (char to str): $$  $$
>>>
>>> As you can see with the BigEndian option the characters are not
>>> correctly written to the file.
>>>
>>> Is this correct behaviour? Have I missed something I should have
>>> included to get this to work? (This is entirely possible and I apologise
>>> for wasting your time if this is a simple user-error.)
>>>
>>> I have searched the ncl-talk archives and have not found this problem
>>> encountered by others, so it may well be an issue exclusive to my machines.
>>> If it is, I can work around it by wrapping in some fortran to handle the
>>> reading and writing of the binary correctly. I would prefer to avoid such a
>>> work around as the size of the arrays in the binary files I have to read
>>> and write will vary, and I would prefer not to have to hard-code the array
>>> sizes when NCL makes calls to the fortran code.
>>>
>>> [Note: on the machine running NCL 6.2.1 I also get the following warning
>>> message on both attempts:
>>> warning:fbinrecwrite: end of file reached before record number, writing
>>> record as last record in file]
>>>
>>> Many thanks for your help,
>>> Kay
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20160708/6f5973a9/attachment.html