[ncl-talk] Computing coefficient of determination with gridded datasets

Sat Jan 13 11:13:57 MST 2018

Hi Rick,

Thanks for the answer, I thought also about the differences between missing values and fill values. I tested your suggestion, and the result was the same. Since I am dealing with doubles, I standardized both (missing values and fill values) to -9999. Quick and dirty, but in order to make sure both attributes are the same, I performed:

cveg_HWSD_b_avg_3d at _FillValue = -9999
cveg_HWSD_b_avg_3d at missing_value = -9999
lpjml_hadgem2_es_avg_mask_3d at _FillValue = -9999
lpjml_hadgem2_es_avg_mask_3d at missing_value = -9999
cveg_HWSD_b_avg_3d at _FillValue = cveg_HWSD_b_avg_3d at missing_value
lpjml_hadgem2_es_avg_mask_3d at _FillValue = lpjml_hadgem2_es_avg_mask_3d at missing_value
delete(cveg_HWSD_b_avg_3d at missing_value)      ; not necessary, just cleaning up
delete(lpjml_hadgem2_es_avg_mask_3d at missing_value)      ; not necessary, just cleaning up

This resulted in:

Variable: cveg_HWSD_b_avg_3d
Type: double
Total Size: 106560 bytes
            13320 values
Number of Dimensions: 3
Dimensions and sizes:        [time | 1] x [lat | 37] x [lon | 360]
Coordinates:
            time: [1..1]
            lat: [79.5..43.5]
            lon: [-179.5..179.5]
Number Of Attributes: 5
  average_op_ncl :              dim_avg_n over dimension(s): time
  units :   g C m-2
  long_name :       C stored in Cveg
  percentile :           50
  _FillValue :          -9999
(0)           C stored in Cveg: min=0.1382737806910972   max=12.84650588106167

Variable: lpjml_hadgem2_es_avg_mask_3d
Type: double
Total Size: 106560 bytes
            13320 values
Number of Dimensions: 3
Dimensions and sizes:        [time | 1] x [lat | 37] x [lon | 360]
Coordinates:
            time: [1..1]
            lat: [79.5..43.5]
            lon: [-179.5..179.5]
Number Of Attributes: 5
  units :   kg m-2
  long_name :       Carbon Mass in Vegetation
  standard_name :              vegetation_carbon_content
  average_op_ncl :              dim_avg_n over dimension(s): time
  _FillValue :          -9999
(0)           Carbon Mass in Vegetation: min=3.493304393487051e-05   max=15.10204410552979

And then the escorc_n function [lpjml_r = escorc_n(cveg_HWSD_b_avg_3d,lpjml_hadgem2_es_avg_mask_3d, 0, 0)], which again has resulted in min=-9999   max=-9999. The issue is clearly in the escorc_n function.

Best,
Efren

From: Rick Brownrigg <brownrig at ucar.edu>
Date: Saturday, 13 January 2018 at 16.49
To: Efren Lopez Blanco <elb at bios.au.dk>
Cc: "ncl-talk at ucar.edu" <ncl-talk at ucar.edu>
Subject: Re: [ncl-talk] Computing coefficient of determination with gridded datasets

Pure speculation on my part, but I notice that both variables have different values for their missing_value vs. _FillValue attributes. I wonder which value actually occurs in the data? I *think* if both are present as attributes, NCL will honor the _FillValue value, and then values of missing_value get treated as data (?)  Something to perhaps check out.
Rick

On Sat, Jan 13, 2018 at 3:21 AM, Efren Lopez Blanco via ncl-talk <ncl-talk at ucar.edu<mailto:ncl-talk at ucar.edu>> wrote:
Hi there,

I would like to compute the coefficient of determination (r squared) between 2 gridded datasets of the form (time,lat,lon). In order to do so, I aim to apply the escorc_n function. The two datasets I am using look like:

Variable: cveg_HWSD_b_avg_3d
Type: double
Total Size: 106560 bytes
            13320 values
Number of Dimensions: 3
Dimensions and sizes:        [time | 1] x [lat | 37] x [lon | 360]
Coordinates:
            time: [1..1]
            lat: [79.5..43.5]
            lon: [-179.5..179.5]
Number Of Attributes: 6
  average_op_ncl :              dim_avg_n over dimension(s): time
  missing_value :  -9999
  units :   g C m-2
  long_name :       C stored in Cveg
  percentile :           50
  _FillValue :          1.000000020040877e+20
(0)           C stored in Cveg: min=0.1382737806910972   max=12.84650588106167

Variable: lpjml_hadgem2_es_avg_mask_3d
Type: double
Total Size: 106560 bytes
            13320 values
Number of Dimensions: 3
Dimensions and sizes:        [time | 1] x [lat | 37] x [lon | 360]
Coordinates:
            time: [1..1]
            lat: [79.5..43.5]
            lon: [-179.5..179.5]
Number Of Attributes: 6
  units :   kg m-2
  long_name :       Carbon Mass in Vegetation
  standard_name :              vegetation_carbon_content
  average_op_ncl :              dim_avg_n over dimension(s): time
  missing_value :  1e+20
  _FillValue :          1.000000020040877e+20
(0)           Carbon Mass in Vegetation: min=3.493304393487051e-05   max=15.10204410552979

Then I apply the function:

lpjml_r = escorc_n(cveg_HWSD_b_avg_3d,lpjml_hadgem2_es_avg_mask_3d, 0, 0)   ; r(nlat,mlon)

However, I’ve noticed the output only contain NAs:

Variable: lpjml_r
Type: double
Total Size: 106560 bytes
            13320 values
Number of Dimensions: 2
Dimensions and sizes:        [lat | 37] x [lon | 360]
Coordinates:
            lat: [79.5..43.5]
            lon: [-179.5..179.5]
Number Of Attributes: 6
  missing_value :  -9999
  average_op_ncl :              dim_avg_n over dimension(s): time
  units :   g C m-2
  long_name :       C stored in Cveg
  percentile :           50
  _FillValue :          1.000000020040877e+20
(0)           C stored in Cveg: min=1.000000020040877e+20   max=1.000000020040877e+20

Am I doing something wrong? I’ve also tried to convert from double to float.

Best,
Efren

_______________________________________________
ncl-talk mailing list
ncl-talk at ucar.edu<mailto:ncl-talk at ucar.edu>
List instructions, subscriber options, unsubscribe:
http://mailman.ucar.edu/mailman/listinfo/ncl-talk

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.ucar.edu/pipermail/ncl-talk/attachments/20180113/225258b2/attachment.html>