[ncl-talk] convert the double missing_value to float one
Dennis Shea
shea at ucar.edu
Tue Jun 20 07:35:02 MDT 2017
I am Matlab ignorant so I can't answer what Matlab does/does-not do..
NCL has some 'netCDF rules' built into the language:
ncl 0> x := fspan(1,10,10) ; float
ncl 1> x at _FillValue = -999d ; double
fatal:Type Mismatch: The type of missing value could not be converted to
type of variable (x)
fatal:["Execute.c":8573]:Execute: Error occurred at or near line 1
NCL will 'help' in some cases
ncl 5> x at _FillValue = -999 ; integer ... here NCL silently 'promotes'
the integer to a float
ncl 6> print(x)
ncl 7> print(typeof(x at _FillValue)) ; float
====
ncdump -h MPI-ESM-LR-CLM4.8.17_dry_power.nc
[snip]
float pr(time, latitude, longitude) ;
pr:_FillValue = -999.f ;
[snip]
Don't be misled by the 'f' ... extension (-999.f).
f = addfile("MPI-ESM-LR-CLM4.8.17_dry_power","r")
pr = f->pr
print(typeof(pr)) ; float
print( typeof(pr at _FillValue)) ; I speculate this will be type double.
; Note the := syntax
pr at _FillValue := totype( pr at _FillValue, typeof(pr) ) ; MH's suggestion
print( typeof(pr at _FillValue))
[snip]
====
You can send the file to
ftp ftp.cgd.ucar.edu
anonymous
your_email
cd incoming
put .....
quit
Then notify ncl-talk after a successful file transfer.
You can attach a clean version of your script to the email.
Cheers
D
On Tue, Jun 20, 2017 at 1:32 AM, Beáta Szabó-Takács <szabo.b at czechglobe.cz>
wrote:
> Dear Dennis,
>
>
>
> You are right I agree with you. To be honest I created the netCDF files in
> Matlab. The missing values were originally NaN values but I converted them
> to single -999 by:
>
> data(isnan(data))=single(-999);
>
> pr = permute(data,[2,3,1]);
>
> pr = single(pr);
>
>
>
> and I defined -999 as _FillValue by:
>
> netcdf.putAtt(ncid1,3,'_FillValue', cast(-999, class(pr)));
>
>
>
> I checked the netCDF files with ncdump -h which denoted that _FillValue
> and pr value are both float:
>
> $ ncdump -h MPI-ESM-LR-CLM4.8.17_dry_power.nc
>
> netcdf MPI-ESM-LR-CLM4.8.17_dry_power {
>
> dimensions:
>
> longitude = 464 ;
>
> latitude = 201 ;
>
> time = 5479 ;
>
> variables:
>
> float longitude(longitude) ;
>
> longitude:long_name = "longitude" ;
>
> longitude:standard_name = "lon" ;
>
> longitude:units = "degrees_east" ;
>
> float latitude(latitude) ;
>
> latitude:long_name = "latitude" ;
>
> latitude:standard_name = "lat" ;
>
> latitude:units = "degrees_north" ;
>
> double time(time) ;
>
> time:long_name = "time" ;
>
> time:standard_name = "time" ;
>
> time:units = "days since 1949-12-1 00:00:00" ;
>
> float pr(time, latitude, longitude) ;
>
> pr:_FillValue = -999.f ;
>
> pr:long_name = "precipitation" ;
>
> pr:standard_name = "precipitation" ;
>
> pr:units = "mm" ;
>
> pr:cell_methods = "time: mean" ;
>
> }
>
> When I tried to calculate the multi year seasonal mean values with CDO I
> got the following error message:
>
> cdo yseasmean MPI-ESM-LR-CLM4.8.17_dry_power.nc MPI-ESM-LR-CLM4.8.17_dry_
> power_sm.nc
>
> cdf_put_vara_double : ncid = 131072 varid = 3 val0 = -999.000000
>
> cdf_put_vara_double : varname = pr
>
>
>
> Error (cdf_put_vara_double) : NetCDF: Numeric conversion not representable
>
> I could carry out this operator without error message if I use –b 64
> option but due to this option the values become double precision. I do not
> understand why CDO considers _Fill Value to be double?
>
>
>
> Beata
>
>
>
>
>
> *From:* Dennis Shea [mailto:shea at ucar.edu]
> *Sent:* Monday, June 19, 2017 6:39 PM
> *To:* Mary Haley <haley at ucar.edu>
> *Cc:* Beáta Szabó-Takács <szabo.b at czechglobe.cz>; ncl-talk (
> ncl-talk at ucar.edu) <ncl-talk at ucar.edu>
> *Subject:* Re: [ncl-talk] convert the double missing_value to float one
>
>
>
> A comment about netCDF variables. Since NCL variables follow the netCDF
> variable model, it holds for NCL variables also.
>
> By rule, the type of the _FillValue should match the type of the variable
> with which it is associated.
>
> short variable must have a short _Fillvalue
>
> integer variable ... integer _FillValue
>
> float variable ... float _FillValue
>
> double variable ... double _FillValue
>
> ---
>
>
>
>
> On Mon, Jun 19, 2017 at 9:38 AM, Mary Haley <haley at ucar.edu> wrote:
>
> Beata,
>
>
>
> When going from a 'higher' type to a 'lower' type (double to float, for
> example), you need to use the reassignment operator (':=') to force the
> conversion:
>
>
>
> p8 at missing_value := doubletofloat(p8 at missing_value)
>
> p8 at _FillValue := p8 at missing_value
>
>
>
> It might be better to use the "totype" function:
>
>
>
> p8 at _FillValue := totype(p8 at _FillValue,typeof(p8))
>
> p8 at missing_value := totype(p8 at missing_value,typeof(p8))
>
>
>
> --Mary
>
>
>
>
>
>
>
> On Mon, Jun 19, 2017 at 1:04 AM, Beáta Szabó-Takács <szabo.b at czechglobe.cz>
> wrote:
>
> Dear NCL Users,
>
>
>
> I would like to create a taylor_metrics_table where the missing values are
> denoted by -999. This part of script is:
>
>
>
> season = (/ "DJF","JJA" /)
>
> nSeason = dimsizes(season)
>
>
>
> table = new ( (/nCase,nSeason,nSource/), typeof(ratio) )
>
> table(0,:,:) = (/CA_bias, CA_biasn/)
>
> table(1,:,:) = (/CB_bias, CB_biasn/)
>
> table(2,:,:) = (/CC_bias, CC_biasn/)
>
> table(3,:,:) = (/CD_bias, CD_biasn/)
>
>
>
>
>
> tt_opt = True
>
> tt_opt at tableTitle = "Bias(%)"
>
> tt_opt at pltType= "png" ; "eps" [default], "pdf", "ps"
>
> ; "png", "gif" [if you have
> ImageMajik 'convert']
>
> ; tt_opt at color0 = "palegreen2"
>
> ; tt_opt at color1 = "tomato2"
>
> tt_opt at color0 = "white"
>
> tt_opt at color1 = "white"
>
>
>
>
>
> taylor_metrics_table("taylor_bias_dry", source, case ,season, table,
> tt_opt)
>
>
>
> I have some netcdf files where the precipitation (pr) values are float
> data type, but the _FillValue and missing_value are double. I tried convert
> these data to float with:
>
>
>
> ncap2 -s 'pr=float(pr)' CM5A-MR_RCA4_dry_power_sm.nc
> CM5A-MR_RCA4_dry_power_sm2.nc
>
>
>
> It converted the _FillValue to float but the missing_value are still
> double:
>
> netcdf CM5A-MR_RCA4_dry_power_sm2 {
>
> dimensions:
>
> time = UNLIMITED ; // (4 currently)
>
> latitude = 201 ;
>
> longitude = 464 ;
>
> variables:
>
> float pr(time, latitude, longitude) ;
>
> pr:_FillValue = -999.f ;
>
> pr:cell_methods = "time: mean" ;
>
> pr:long_name = "precipitation" ;
>
> pr:missing_value = -999. ;
>
> pr:standard_name = "precipitation_flux" ;
>
> pr:units = "mm" ;
>
> float longitude(longitude) ;
>
> longitude:standard_name = "longitude" ;
>
> longitude:long_name = "longitude" ;
>
> longitude:units = "degrees_east" ;
>
> longitude:axis = "X" ;
>
> float latitude(latitude) ;
>
> latitude:standard_name = "latitude" ;
>
> latitude:long_name = "latitude" ;
>
> latitude:units = "degrees_north" ;
>
> latitude:axis = "Y" ;
>
> double time(time) ;
>
> time:standard_name = "time" ;
>
> time:long_name = "time" ;
>
> time:units = "days since 1949-12-1 00:00:00" ;
>
> time:calendar = "standard" ;
>
> time:axis = "T" ;
>
>
>
> The resulted table is attached. In the table the missing values are
> denoted by double values in CD_biasn despite the fact that the printed
> CD_biasn contains _FillValues:
>
>
>
> Variable: CD_biasn
>
> Type: float
>
> Total Size: 20 bytes
>
> 5 values
>
> Number of Dimensions: 1
>
> Dimensions and sizes: [5]
>
> Coordinates:
>
> Number Of Attributes: 1
>
> _FillValue : -999
>
> (0) -999
>
> (1) -4.541727
>
> (2) 12.45617
>
> (3) 10.47692
>
> (4) -999
>
>
>
> I also tried to convert the missing values to float type by ncl
> doubletofloat function but it does not work:
>
>
>
> p8 at missing_value=doubletofloat(p8 at missing_value)
>
> p8 at _FillValue = p8 at missing_value
>
> delete(p8 at missing_value)
>
>
>
> fatal:Type Mismatch: The type of missing value could not be converted to
> type of variable (p8)
>
>
>
> p8_out = addfile("p8.nc","c")
>
> p8_out ->p8 = p8
>
>
>
>
>
> ncdump -h p8.nc
>
> netcdf p8 {
>
> dimensions:
>
> time = 4 ;
>
> latitude = 201 ;
>
> longitude = 464 ;
>
> variables:
>
> float p8(time, latitude, longitude) ;
>
> p8:_FillValue = -999.f ;
>
> p8:cell_methods = "time: mean" ;
>
> p8:long_name = "precipitation" ;
>
> p8:missing_value = -999. ;
>
> p8:standard_name = "precipitation_flux" ;
>
> p8:units = "mm" ;
>
> double time(time) ;
>
> time:standard_name = "time" ;
>
> time:long_name = "time" ;
>
> time:units = "days since 1949-12-1 00:00:00" ;
>
> time:calendar = "standard" ;
>
> time:axis = "T" ;
>
> float latitude(latitude) ;
>
> latitude:standard_name = "latitude" ;
>
> latitude:long_name = "latitude" ;
>
> latitude:units = "degrees_north" ;
>
> latitude:axis = "Y" ;
>
> float longitude(longitude) ;
>
> longitude:standard_name = "longitude" ;
>
> longitude:long_name = "longitude" ;
>
> longitude:units = "degrees_east" ;
>
> longitude:axis = "X" ;
>
> }
>
>
>
>
>
> Can someone suggest me a solution?
>
> Thank you for your help in advance!
>
> Kind regards,
>
> Beata
>
>
>
>
>
> _______________________________________________
> ncl-talk mailing list
> ncl-talk at ucar.edu
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
>
>
>
> _______________________________________________
> ncl-talk mailing list
> ncl-talk at ucar.edu
> List instructions, subscriber options, unsubscribe:
> http://mailman.ucar.edu/mailman/listinfo/ncl-talk
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20170620/1d4333d8/attachment.html
More information about the ncl-talk
mailing list