[ncl-talk] error due to write more than 2 Gb values to a file

[Debasish Pai Mazumder]

Hi Dennis,

Thanks for your response. I am trying to write a variable with dimension 
(year = 16, hour = 368, lev = 19, lat = 66, lon = 80). Total file size I 
am trying to write is 2.3 GB. I have tried the setfileoption but getting 
same error message (fatal:fbinwrite: cannot write more than 2 Gb values 
to a file).

outfilename = diri + "CESM_ASO_Tcor_" + mk + "_1990010100Z-2005123118Z.bin"
system("/bin/rm -f "+ outfilename)
setfileoption("bin", "RecordMarkerSize", 8)
fbinwrite(outfilename, (/Tcor/))

with regards
-Deb

On 3/7/16 10:40 AM, Dennis Shea wrote:
> Hi Debasish,
>
>
> Please read the description at:
> https://www.ncl.ucar.edu/Document/Functions/Built-in/fbinwrite.shtml
>
> Specifically:
>
> "By default, the record marker size at the beginning and end of 
> Fortran sequential files is assumed to be 4 bytes. In V6.1.1 and 
> later, you can change this to 8 bytes using the "RecordMarkerSize" 
> option in the *setfileoption* 
> <https://www.ncl.ucar.edu/Document/Functions/Built-in/setfileoption.shtml> 
> procedure."
>
> [1] fortran's default integer is 4-bytes (signed). The range is 
> -2,147,483,648 to 2,147,483,647
> [2] by default, fortran sequential records are preceded and postpended 
> by a 'hidden' 4-byte value indicating the size of the record.
>               [hidden 4-byte value][data_record][hidden 4-byte value]
> [3] The max value for a signed 4-byte integer is 2,147,483,647  (think 
> 2GB).
>
> So, in a sense, this is a fortran based issue.
>
> ---
> NCL's setfileoption allows the hidden record marker to be set to 8-bytes
>
>      setfileoption("bin", "RecordMarkerSize", 8)*
> *
> ** fbinwrite(file_name, foo)**; foo is variable*
> *
>
> This can be read via NCL but you must specify that an 8-byte record 
> separator is needed.
>
>      setfileoption("bin", "RecordMarkerSize", 8)*
> *** foo = fbinread(file_name, rec_dims, "float")*
>
> ---
> *
> If reading the binary file in a fortran code, it is likely that either 
> (i) a compiler level option will have to be used so that it recognizes 
> that 8-byte separator is being used or (ii) a language extension 
> function (similar to 'setfileoption') must be used.
>
> Cheers
>
> On Sat, Mar 5, 2016 at 6:40 PM, Debasish Pai Mazumder 
> <debasish at ucar.edu <mailto:debasish at ucar.edu>> wrote:
>
>     Hi All,
>
>     I am trying to write more than 2 Gb values to a binary file and I
>     received following error message
>     fatal:fbinwrite: cannot write more than 2 Gb values to a file.
>
>     Is there any way to create this binary file without splitting in
>     to two
>     files.
>
>     with regards
>     -Debasish
>
>
>
>     _______________________________________________
>     ncl-talk mailing list
>     ncl-talk at ucar.edu <mailto:ncl-talk at ucar.edu>
>     List instructions, subscriber options, unsubscribe:
>     http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20160307/04cc0337/attachment.html