<div dir="ltr"><div class="gmail_default" style="font-size:small">Debasish,</div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small">I looked at the "fbinwrite" code, and it has a test in it that explicitly doesn't allow you to write more than 2 GB to a file:</div><div class="gmail_default" style="font-size:small"><br></div><div class="gmail_default" style="font-size:small"><div class="gmail_default"> if(totalsize > INT_MAX) { </div><div class="gmail_default"> NhlPError(NhlFATAL,NhlEUNKNOWN,"fbinwrite: cannot write more than 2 Gb values to a file."); </div><div class="gmail_default"> return(NhlFATAL); </div><div class="gmail_default"> } </div><div class="gmail_default"><br></div><div class="gmail_default">So, Dennis' suggestion is not going to work unfortunately.</div><div class="gmail_default"><br></div><div class="gmail_default">Also, I found an older ncl-talk message on a similar issue, and there's a statement, "Apparently, this is a gcc (or more precisely a gfortran) issue, which internally, each written block has a limit of 2G."</div><div class="gmail_default"><br></div><div class="gmail_default">You can see the full post and a possible work-around here:</div><div class="gmail_default"><br></div><div class="gmail_default"><a href="https://www.ncl.ucar.edu/Support/talk_archives/2014/0209.html">https://www.ncl.ucar.edu/Support/talk_archives/2014/0209.html</a></div><div class="gmail_default"><br></div><div class="gmail_default">I've added your email to a ticket that we already had open on "fbindirwrite" (NCL-1889). I marked it for a 6.4.0 release, but I can't guarantee this. I'm not sure how hard the fix will be to allow all of our binary write functions to write larger than 2 GB.</div><div class="gmail_default"><br></div><div class="gmail_default">--Mary</div><div class="gmail_default"><br></div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Mar 7, 2016 at 12:03 PM, Debasish Pai Mazumder <span dir="ltr"><<a href="mailto:debasish@ucar.edu" target="_blank">debasish@ucar.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
Hi Dennis,<br>
<br>
Thanks for your response. I am trying to write a variable with
dimension (year = 16, hour = 368, lev = 19, lat = 66, lon = 80).
Total file size I am trying to write is 2.3 GB. I have tried the
setfileoption but getting same error message (fatal:fbinwrite:
cannot write more than 2 Gb values to a file).<br>
<br>
outfilename = diri + "CESM_ASO_Tcor_" + mk +
"_1990010100Z-2005123118Z.bin"<br>
system("/bin/rm -f "+ outfilename)<span class=""><span><br>
setfileoption("bin", "RecordMarkerSize", 8)<br>
</span></span> fbinwrite(outfilename, (/Tcor/))<br>
<br>
with regards<br>
-Deb<div><div class="h5"><br>
<br>
<div>On 3/7/16 10:40 AM, Dennis Shea wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>Hi Debasish,<br>
<br>
</div>
<div><br>
</div>
</div>
Please read the description at:<br>
<a href="https://www.ncl.ucar.edu/Document/Functions/Built-in/fbinwrite.shtml" target="_blank">https://www.ncl.ucar.edu/Document/Functions/Built-in/fbinwrite.shtml</a><br>
<br>
</div>
Specifically:<br>
<br>
"By default, the record marker size at the
beginning and end of Fortran
sequential files is assumed to be 4 bytes. In
V6.1.1 and later, you
can change this to 8 bytes using the
"RecordMarkerSize" option in the
<a href="https://www.ncl.ucar.edu/Document/Functions/Built-in/setfileoption.shtml" target="_blank"><strong>setfileoption</strong></a>
procedure."<br>
<br>
</div>
[1] fortran's default integer is 4-bytes (signed).
The range is -2,147,483,648 to 2,147,483,647</div>
[2] by default, fortran sequential records are
preceded and postpended by a 'hidden' 4-byte value
indicating the size of the record.<br>
</div>
[hidden 4-byte
value][data_record][hidden 4-byte value]</div>
[3] The max value for a signed 4-byte integer is
2,147,483,647 (think 2GB).<br>
<br>
</div>
So, in a sense, this is a fortran based issue.<br>
<br>
---<br>
</div>
NCL's setfileoption allows the hidden record marker to be
set to 8-bytes<br>
<br>
</div>
setfileoption("bin", "RecordMarkerSize", 8)<b><br>
</b></div>
<b> </b> fbinwrite(file_name, foo)<b>
</b>; foo is variable<b><br>
</b>
<div>
<div><br>
<div>
<div>This can be read via NCL but you must specify that an
8-byte record separator is needed. </div>
<div><br>
<div>
<div>
<div> setfileoption("bin", "RecordMarkerSize",
8)<b><br>
</b><b> </b> foo = fbinread(file_name,
rec_dims, "float")<b><br>
<br>
---<br>
</b></div>
<div>If reading the binary file in a fortran code,
it is likely that either (i) a compiler level
option will have to be used so that it recognizes
that 8-byte separator is being used or (ii) a
language extension function (similar to
'setfileoption') must be used.<br>
<br>
</div>
<div>Cheers<br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Sat, Mar 5, 2016 at 6:40 PM,
Debasish Pai Mazumder <span dir="ltr"><<a href="mailto:debasish@ucar.edu" target="_blank"></a><a href="mailto:debasish@ucar.edu" target="_blank">debasish@ucar.edu</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi All,<br>
<br>
I am trying to write more than 2 Gb values to a binary file
and I<br>
received following error message<br>
fatal:fbinwrite: cannot write more than 2 Gb values to a
file.<br>
<br>
Is there any way to create this binary file without
splitting in to two<br>
files.<br>
<br>
with regards<br>
-Debasish<br>
<br>
<br>
<br>
_______________________________________________<br>
ncl-talk mailing list<br>
<a href="mailto:ncl-talk@ucar.edu" target="_blank">ncl-talk@ucar.edu</a><br>
List instructions, subscriber options, unsubscribe:<br>
<a href="http://mailman.ucar.edu/mailman/listinfo/ncl-talk" rel="noreferrer" target="_blank">http://mailman.ucar.edu/mailman/listinfo/ncl-talk</a><br>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
</div></div></div>
<br>_______________________________________________<br>
ncl-talk mailing list<br>
<a href="mailto:ncl-talk@ucar.edu">ncl-talk@ucar.edu</a><br>
List instructions, subscriber options, unsubscribe:<br>
<a href="http://mailman.ucar.edu/mailman/listinfo/ncl-talk" rel="noreferrer" target="_blank">http://mailman.ucar.edu/mailman/listinfo/ncl-talk</a><br>
<br></blockquote></div><br></div>