[ncl-talk] variable metadata getting overwritten when variable is written to a NetCDF4 file

[Keith Lindsay]

Dave,

With 6.3.0 on both yellowstone and my desktop linux box, I get

 Copyright (C) 1995-2015 - All Rights Reserved
 University Corporation for Atmospheric Research
 NCAR Command Language Version 6.3.0
 The use of this software is governed by a License Agreement.
 See http://www.ncl.ucar.edu/ for more details.
netcdf file_var_metadata_clobber {
variables:
float testvar ;
testvar:long_name = "long_name" ;
testvar:units = "units" ;
}
netcdf file_var_metadata_clobber {
variables:
float testvar ;
testvar:_FillValue = -99.f ;
}

Based on your output, it seems like this behavior is fixed in 6.3.1.

Keith

On Tue, Nov 3, 2015 at 12:14 AM, David Brown <dbrown at ucar.edu> wrote:

> Well yes that would definitely be a bug, But that is not what I was
> seeing with your script (I ran it only on my Mac with my 6.3.1
> development version).
>
> Here is the output of your test script on my system:
>
>  ncl file_var_metadata_clobber.ncl
>  Copyright (C) 1995-2015 - All Rights Reserved
>  University Corporation for Atmospheric Research
>  NCAR Command Language Version 6.3.1
>  The use of this software is governed by a License Agreement.
>  See http://www.ncl.ucar.edu/ for more details.
>
> netcdf file_var_metadata_clobber {
> variables:
> float testvar ;
> testvar:long_name = "long_name" ;
> testvar:units = "units" ;
> }
>
> netcdf file_var_metadata_clobber {
> variables:
> float testvar ;
> testvar:long_name = "long_name" ;
> testvar:units = "units" ;
> testvar:_FillValue = -99.f ;
> }
>
> The copy & paste seems to have removed the indentation, but otherwise
> this seems like the expected output to me.
>
> Is the script giving different results somewhere else?
>  -dave
>
>
>
> On Mon, Nov 2, 2015 at 7:28 PM, Keith Lindsay <klindsay at ucar.edu> wrote:
> > Dave,
> >
> > Thanks for your description of the subtleties of _FillValue and the
> > value-only assignment. Please note though that not only is the _FillValue
> > attribute getting added, the variable's pre-existing attributes are
> getting
> > lost. Do you consider this latter aspect a bug or feature? It is this
> that I
> > am more concerned about. Also, this removal of the pre-existing
> attributes
> > happens with NetCDF4 (for ncl version 6.3.0, but not 6.2.1) and not with
> > NetCDF3.
> >
> > Keith
> >
> > On Mon, Nov 2, 2015 at 5:13 PM, David Brown <dbrown at ucar.edu> wrote:
> >>
> >> Hi Keith,
> >> I think that what you are seeing is actually documented (although
> >> poorly apparently) behavior. At least, if, as I am assuming from your
> >> test script, the only attribute that gets passed through the
> >> value-only assignment is the _FillValue. The NCL Reference Manual says
> >> this about value-only assignment:
> >>
> >> "No dimension names, coordinate variables or attributes other than
> >> _FillValue are assigned. "
> >>
> >> In fact the _FillValue is a special attribute that even values (as
> >> opposed to variables) in NCL carry around. And it is propagated even
> >> through value-only assignment. That is because it is considered an
> >> essential component of the value.
> >>
> >> If in fact you are seeing other attributes get propagated let us know.
> >> Your test script only showed the _FillValue propagation as far as I
> >> could tell.
> >>  -dave
> >>
> >>
> >>
> >>
> >>
> >>
> >> On Mon, Nov 2, 2015 at 9:03 AM, Keith Lindsay <klindsay at ucar.edu>
> wrote:
> >> > Hi,
> >> > Thanks Dennis. For other users that run into this behavior, a
> workaround
> >> > to
> >> > avoid it is to ensure that the variable on the rhs of the problematic
> >> > assignment does not have the _FillValue attribute. I.e.
> >> >       if (isatt(tmpvar, "_FillValue")) then
> >> >         delete(tmpvar at _FillValue)
> >> >       end if
> >> > Keith
> >> >
> >> >
> >> > On Mon, Nov 2, 2015 at 8:44 AM, Dennis Shea <shea at ucar.edu> wrote:
> >> >>
> >> >> A JIRA ticket [  NCL-2302 ] has been opened.  THX
> >> >>
> >> >> On Mon, Nov 2, 2015 at 8:06 AM, Keith Lindsay <klindsay at ucar.edu>
> >> >> wrote:
> >> >> > Hi,
> >> >> >
> >> >> > I have come across an instance where I'm writing a variable to a
> file
> >> >> > and
> >> >> > the variable metadata in the file is getting overwritten, even
> though
> >> >> > I'm
> >> >> > using (/ ... /) on the right-hand side of the write to file
> >> >> > statement.
> >> >> > That
> >> >> > is, the statement
> >> >> > f->$varname$ = (/ tmpvar /)
> >> >> > is overwriting the metadata for the file variable $varname$ in the
> >> >> > file
> >> >> > pointed to by f. The attached script demonstrates the problem. This
> >> >> > seems
> >> >> > like an ncl bug to me.
> >> >> >
> >> >> > Details of my script that seem to matter are:
> >> >> > 1) the file is NetCDF4
> >> >> > 2) the file variable whose metadata is getting overwritten has been
> >> >> > defined
> >> >> > in the file but not written to yet
> >> >> > 3) the file variable whose metadata is getting overwritten does not
> >> >> > have
> >> >> > the
> >> >> > _FillValue attribute
> >> >> > 4) the variable on rhs of write to file statement has _FillValue
> >> >> > attribute
> >> >> >
> >> >> > The problem happens with ncl 6.3.0 on yellowstone. It does not
> happen
> >> >> > with
> >> >> > ncl 6.2.1, though with 6.2.1 the _FillValue gets added to the file
> >> >> > variable's metadata, which is unexpected to me.
> >> >> >
> >> >> > Keith
> >> >> >
> >> >> > _______________________________________________
> >> >> > ncl-talk mailing list
> >> >> > ncl-talk at ucar.edu
> >> >> > List instructions, subscriber options, unsubscribe:
> >> >> > http://mailman.ucar.edu/mailman/listinfo/ncl-talk
> >> >> >
> >> >
> >> >
> >> >
> >> > _______________________________________________
> >> > ncl-talk mailing list
> >> > ncl-talk at ucar.edu
> >> > List instructions, subscriber options, unsubscribe:
> >> > http://mailman.ucar.edu/mailman/listinfo/ncl-talk
> >> >
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20151103/f2dfcafe/attachment.html