[ncl-talk] issue when writing netCDF4

[Mary Haley]

Hi Laura,

Using "delete" is the correct way to close the file, and is in practice a
good idea. I have to be honest, though, and say it's not something that I
went out of my way to recommend that users do this, because NCL is supposed
to close files for you.

Since you are writing rather large files, and potentially looping across
several variables, it would help to use the efficient method for creating a
NetCDF file.  I'm running a version of your script now to see how much this
makes a difference. I'll report back here once I have some results.

--Mary




On Mon, Oct 30, 2017 at 7:04 AM, Laura Fowler <laura at ucar.edu> wrote:

> Hi Mary:
>
> Thanks for replying to this e-mail. The arrays are actually filled
> with _FillValues when I try to read the individual arrays. An ncdump
> would show "_/". I think that it is because the netcdf array I am not
> properly closing the netcdf array before the do loop goes to the next
> array, and so on. Of course, this does not happen all the time either.
>
> Over the week-end, it looks like adding the line "delete(ncdf)" at the
> bottom of the loop works. I am not sure if it is the actual proper way
> to close the file though as Rick Browing suggested to use filesystem?
>
> As for the getenv, you can use the following:
>
> setenv dir "./../"
> setenv diri "./../run.20151129_to_20151205/"
>
> setenv diro "./run.20151129_to_20151205/20151129/"
> setenv file_i  centeredPacific.history_3d.r4.2015-11-29_00.00.00.nc
> unbuffer ncl test.ncl >& logfile.20151129.out
>
> from the directory
> /glade2/scratch2/laura/MPAS.PacificOcean/run.centeredPacificOceanMesh.15-
> 3km.20151129-20151231/3Dfields
>
> I also use "unbuffer" which was the best way Davide (at cisl) found to
> force cheyenne to dump the logfile as test.ncl runs? Is this right?
>
>
> Many thanks for your help,
> Laura
>
>
>
>
> On Sat, Oct 28, 2017 at 9:49 AM, Mary Haley <haley at ucar.edu> wrote:
> > Hi Laura,
> >
> > By "empty" do you mean that the NetCDF file itself doesn't have much
> written
> > to it, or that it does have values written to it, but the values are
> equal
> > to _FillValue?
> >
> > And when you say it sometimes happens and sometimes it doesn't, is this
> from
> > running the exact same script without any changes?
> >
> > In order to run this script, it looks like I need to know the values of
> some
> > environment variables:
> >
> > dir = getenv("dir")
> > diri = getenv("diri")
> > diro = getenv("diro")
> > file_i = getenv("file_i")
> >
> > Thanks,
> >
> > --Mary
> >
> >
> > On Fri, Oct 27, 2017 at 2:02 PM, Laura Fowler <laura at ucar.edu> wrote:
> >>
> >> Hello:
> >>
> >> I am not understanding why the content of a netCDF4 file remains
> >> empty, but one in a while. I wrote an ncl script that reads an array
> >> from a history file  (in a cdf5 format), interpolates the data on
> >> fixed pressure levels, and writes the interpolated data to a separate
> >> netCDF4 file. It happens that my output netCDF4 file is empty (the
> >> output data is all missing), but not all the times. I do print the max
> >> value of the interpolated data before writing the data to the output
> >> file, so that I know that the output data is not garbage. I am really
> >> puzzled because sometimes it works and sometimes it does. It does not
> >> matter if I run the script in a batch job or interactively.
> >>
> >> The only error I got once is:
> >>
> >> (0) itime = 22 max diag = 1 max diag_pFixed = 1
> >> (0) itime = 23 max diag = 1 max diag_pFixed = 1
> >> (0) max diag_pFixed = 1
> >> (0) fname = qc_cu
> >>
> >>
> >> fatal:NclMalloc Failed:[errno=12]
> >> ncl: putget.c:6833: getNCvx_float_float: Assertion `value != ((void*)0)'
> >> failed.
> >>
> >>
> >> Has anybody got this error message before? If interested, my script is
> in
> >>
> >> /glade2/scratch2/laura/MPAS.PacificOcean/run.
> centeredPacificOceanMesh.15-3km.20151129-20151231/3Dfields/test/ncl
> >>
> >>
> >> Thanks,
> >> Laura
> >>
> >>
> >>
> >>
> >> --
> >>
> >> !-----------------------------------------------------------
> --------------------------------------------------
> >> Laura D. Fowler
> >> Mesoscale and Microscale Meteorology Division (MMM)
> >> National Center for Atmospheric Research
> >> P.O. Box 3000, Boulder CO 80307-3000
> >>
> >> e-mail: laura at ucar.edu
> >> phone: 303-497-1628
> >>
> >>
> >> !-----------------------------------------------------------
> --------------------------------------------------
> >> _______________________________________________
> >> ncl-talk mailing list
> >> ncl-talk at ucar.edu
> >> List instructions, subscriber options, unsubscribe:
> >> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
> >
> >
>
>
>
> --
> !-----------------------------------------------------------
> --------------------------------------------------
> Laura D. Fowler
> Mesoscale and Microscale Meteorology Division (MMM)
> National Center for Atmospheric Research
> P.O. Box 3000, Boulder CO 80307-3000
>
> e-mail: laura at ucar.edu
> phone: 303-497-1628
>
> !-----------------------------------------------------------
> --------------------------------------------------
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20171030/3586f808/attachment.html>