[ncl-talk] error due to write more than 2 Gb values to a file

[Dennis Shea]

Hi Debasish,


Please read the description at:
https://www.ncl.ucar.edu/Document/Functions/Built-in/fbinwrite.shtml

Specifically:

"By default, the record marker size at the beginning and end of Fortran
sequential files is assumed to be 4 bytes. In V6.1.1 and later, you can
change this to 8 bytes using the "RecordMarkerSize" option in the
*setfileoption*
<https://www.ncl.ucar.edu/Document/Functions/Built-in/setfileoption.shtml>
procedure."

[1] fortran's default integer is 4-bytes (signed). The range is
-2,147,483,648 to 2,147,483,647
[2] by default, fortran sequential records are preceded and postpended by a
'hidden' 4-byte value indicating the size of the record.
              [hidden 4-byte value][data_record][hidden 4-byte value]
[3] The max value for a signed 4-byte integer is 2,147,483,647  (think 2GB).

So, in a sense, this is a fortran based issue.

---
NCL's setfileoption allows the hidden record marker to be set to 8-bytes

     setfileoption("bin", "RecordMarkerSize", 8)
     fbinwrite(file_name, foo)                                 ; foo is
variable

This can be read via NCL but you must specify that an 8-byte record
separator is needed.

     setfileoption("bin", "RecordMarkerSize", 8)
     foo = fbinread(file_name, rec_dims, "float")


*---*
If reading the binary file in a fortran code, it is likely that either (i)
a compiler level option will have to be used so that it recognizes that
8-byte separator is being used or (ii) a language extension function
(similar to 'setfileoption') must be used.

Cheers

On Sat, Mar 5, 2016 at 6:40 PM, Debasish Pai Mazumder <debasish at ucar.edu>
wrote:

> Hi All,
>
> I am trying to write more than 2 Gb values to a binary file and I
> received following error message
> fatal:fbinwrite: cannot write more than 2 Gb values to a file.
>
> Is there any way to create this binary file without splitting in to two
> files.
>
> with regards
> -Debasish
>
>
>
> _______________________________________________
> ncl-talk mailing list
> ncl-talk at ucar.edu
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20160307/d904d2e8/attachment.html