[ncl-talk] variable metadata getting overwritten when variable is written to a NetCDF4 file

[David Brown]

Yes, that appears to be the case. I just ran the script with the NCL
6.3.0 and then with the development version on yellowstone.
6.3.0 gave your results and the dev version appeared to be correct
with the same results I get on the mac.
So mystery solved. But I guess you will need to use the dev version if
this is an important issue for you.
 -dave

On Tue, Nov 3, 2015 at 6:06 AM, Keith Lindsay <klindsay at ucar.edu> wrote:
> Dave,
>
> With 6.3.0 on both yellowstone and my desktop linux box, I get
>
>  Copyright (C) 1995-2015 - All Rights Reserved
>  University Corporation for Atmospheric Research
>  NCAR Command Language Version 6.3.0
>  The use of this software is governed by a License Agreement.
>  See http://www.ncl.ucar.edu/ for more details.
> netcdf file_var_metadata_clobber {
> variables:
> float testvar ;
> testvar:long_name = "long_name" ;
> testvar:units = "units" ;
> }
> netcdf file_var_metadata_clobber {
> variables:
> float testvar ;
> testvar:_FillValue = -99.f ;
> }
>
> Based on your output, it seems like this behavior is fixed in 6.3.1.
>
> Keith
>
> On Tue, Nov 3, 2015 at 12:14 AM, David Brown <dbrown at ucar.edu> wrote:
>>
>> Well yes that would definitely be a bug, But that is not what I was
>> seeing with your script (I ran it only on my Mac with my 6.3.1
>> development version).
>>
>> Here is the output of your test script on my system:
>>
>>  ncl file_var_metadata_clobber.ncl
>>  Copyright (C) 1995-2015 - All Rights Reserved
>>  University Corporation for Atmospheric Research
>>  NCAR Command Language Version 6.3.1
>>  The use of this software is governed by a License Agreement.
>>  See http://www.ncl.ucar.edu/ for more details.
>>
>> netcdf file_var_metadata_clobber {
>> variables:
>> float testvar ;
>> testvar:long_name = "long_name" ;
>> testvar:units = "units" ;
>> }
>>
>> netcdf file_var_metadata_clobber {
>> variables:
>> float testvar ;
>> testvar:long_name = "long_name" ;
>> testvar:units = "units" ;
>> testvar:_FillValue = -99.f ;
>> }
>>
>> The copy & paste seems to have removed the indentation, but otherwise
>> this seems like the expected output to me.
>>
>> Is the script giving different results somewhere else?
>>  -dave
>>
>>
>>
>> On Mon, Nov 2, 2015 at 7:28 PM, Keith Lindsay <klindsay at ucar.edu> wrote:
>> > Dave,
>> >
>> > Thanks for your description of the subtleties of _FillValue and the
>> > value-only assignment. Please note though that not only is the
>> > _FillValue
>> > attribute getting added, the variable's pre-existing attributes are
>> > getting
>> > lost. Do you consider this latter aspect a bug or feature? It is this
>> > that I
>> > am more concerned about. Also, this removal of the pre-existing
>> > attributes
>> > happens with NetCDF4 (for ncl version 6.3.0, but not 6.2.1) and not with
>> > NetCDF3.
>> >
>> > Keith
>> >
>> > On Mon, Nov 2, 2015 at 5:13 PM, David Brown <dbrown at ucar.edu> wrote:
>> >>
>> >> Hi Keith,
>> >> I think that what you are seeing is actually documented (although
>> >> poorly apparently) behavior. At least, if, as I am assuming from your
>> >> test script, the only attribute that gets passed through the
>> >> value-only assignment is the _FillValue. The NCL Reference Manual says
>> >> this about value-only assignment:
>> >>
>> >> "No dimension names, coordinate variables or attributes other than
>> >> _FillValue are assigned. "
>> >>
>> >> In fact the _FillValue is a special attribute that even values (as
>> >> opposed to variables) in NCL carry around. And it is propagated even
>> >> through value-only assignment. That is because it is considered an
>> >> essential component of the value.
>> >>
>> >> If in fact you are seeing other attributes get propagated let us know.
>> >> Your test script only showed the _FillValue propagation as far as I
>> >> could tell.
>> >>  -dave
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> On Mon, Nov 2, 2015 at 9:03 AM, Keith Lindsay <klindsay at ucar.edu>
>> >> wrote:
>> >> > Hi,
>> >> > Thanks Dennis. For other users that run into this behavior, a
>> >> > workaround
>> >> > to
>> >> > avoid it is to ensure that the variable on the rhs of the problematic
>> >> > assignment does not have the _FillValue attribute. I.e.
>> >> >       if (isatt(tmpvar, "_FillValue")) then
>> >> >         delete(tmpvar at _FillValue)
>> >> >       end if
>> >> > Keith
>> >> >
>> >> >
>> >> > On Mon, Nov 2, 2015 at 8:44 AM, Dennis Shea <shea at ucar.edu> wrote:
>> >> >>
>> >> >> A JIRA ticket [  NCL-2302 ] has been opened.  THX
>> >> >>
>> >> >> On Mon, Nov 2, 2015 at 8:06 AM, Keith Lindsay <klindsay at ucar.edu>
>> >> >> wrote:
>> >> >> > Hi,
>> >> >> >
>> >> >> > I have come across an instance where I'm writing a variable to a
>> >> >> > file
>> >> >> > and
>> >> >> > the variable metadata in the file is getting overwritten, even
>> >> >> > though
>> >> >> > I'm
>> >> >> > using (/ ... /) on the right-hand side of the write to file
>> >> >> > statement.
>> >> >> > That
>> >> >> > is, the statement
>> >> >> > f->$varname$ = (/ tmpvar /)
>> >> >> > is overwriting the metadata for the file variable $varname$ in the
>> >> >> > file
>> >> >> > pointed to by f. The attached script demonstrates the problem.
>> >> >> > This
>> >> >> > seems
>> >> >> > like an ncl bug to me.
>> >> >> >
>> >> >> > Details of my script that seem to matter are:
>> >> >> > 1) the file is NetCDF4
>> >> >> > 2) the file variable whose metadata is getting overwritten has
>> >> >> > been
>> >> >> > defined
>> >> >> > in the file but not written to yet
>> >> >> > 3) the file variable whose metadata is getting overwritten does
>> >> >> > not
>> >> >> > have
>> >> >> > the
>> >> >> > _FillValue attribute
>> >> >> > 4) the variable on rhs of write to file statement has _FillValue
>> >> >> > attribute
>> >> >> >
>> >> >> > The problem happens with ncl 6.3.0 on yellowstone. It does not
>> >> >> > happen
>> >> >> > with
>> >> >> > ncl 6.2.1, though with 6.2.1 the _FillValue gets added to the file
>> >> >> > variable's metadata, which is unexpected to me.
>> >> >> >
>> >> >> > Keith
>> >> >> >
>> >> >> > _______________________________________________
>> >> >> > ncl-talk mailing list
>> >> >> > ncl-talk at ucar.edu
>> >> >> > List instructions, subscriber options, unsubscribe:
>> >> >> > http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>> >> >> >
>> >> >
>> >> >
>> >> >
>> >> > _______________________________________________
>> >> > ncl-talk mailing list
>> >> > ncl-talk at ucar.edu
>> >> > List instructions, subscriber options, unsubscribe:
>> >> > http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>> >> >
>> >
>> >
>
>