[Met_help] [rt.rap.ucar.edu #92453] History for define criteria in configuration file
John Halley Gotway via RT
met_help at ucar.edu
Fri Oct 4 13:39:44 MDT 2019
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
To Whom It may Concern:
I have a question to select observed AOD >= 0 for grid_stat. I am unsure
which way to do it and need to clarify what MET grid-stat will do based on
its design OR according to configuration file & option specified.
The GOES-16 AOD gridded nc file is produced by regrid_data_plane with
fill_value=-9999. When performing grid_stat, will those grids with
fill-value automatically be excluded from the statistic calculation?
If not, also going forward if a observed var has values < 0., then how do I
use only Observed AOD >=0 ?
(A) In my current configuration, obs field is already specified as
obs = {
wind_thresh = [ NA ];
field = [
{
file_type = NETCDF_MET; // NetCDF created by another MET tool
name = "AOD";
level = [ "L0" ];
cat_thresh = [ >=0.1, >=0.2, >=0.3, >=0.4, >=0.5, >=0.6, >=0.8,
>=1.0, >=1.5, >=2.0, >=2.5, >=3.0, >=3.5, >=4.0, >=4.5, >=5.0 ];
}
];
};
cat_thresh is for categorized plot such as CSI. Should I add >= 0.0 in
cat_threshold to generate the statistic, then when doing plot such as RMSE
or time series in Metviewer I selected obs_thresh >=0 [part of fixed value]
?
(B) Binbin uses
poly = [ "${OBSV_FILE} {name= \"TCTD"; level = \"L0\";} >=0" ]; to masking
positive obs field
I already has masking defined as below (stat for each regions) , how do I
append "${OBSV_FILE} {name= \"AOD"; level = \"L0\";} >=0" in the statement
of poly so grid_stat produces multiple stat for each regions coupled
observed AOD > 0. criteria ? i.e.,
for i in $area_mask do grid_stat for
region $i && AOD >=0;
mask = {
grid = [ "FULL" ] ;
poly = [ "CONUS_mask.nc",
"EAST_mask.nc",
"WEST_mask.nc",
"APL_mask.nc",
"ATC_mask.nc",
"CAM_mask.nc",
"CAR_mask.nc",
"ECA_mask.nc",
"GLF_mask.nc",
"GMC_mask.nc",
"GRB_mask.nc",
"LMV_mask.nc",
"MDW_mask.nc",
"MEX_mask.nc",
"NAK_mask.nc",
"NAO_mask.nc",
"NEC_mask.nc",
"NMT_mask.nc",
"NPL_mask.nc",
"NPO_mask.nc",
"NSA_mask.nc",
"NWC_mask.nc",
"PRI_mask.nc",
"SAK_mask.nc",
"SAO_mask.nc",
"SEC_mask.nc",
"SMT_mask.nc",
"SPL_mask.nc",
"SPO_mask.nc",
"SWC_mask.nc",
"SWD_mask.nc",
"WCA_mask.nc"];
};
Thank you for your help.
Ho-Chun
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: define criteria in configuration file
From: John Halley Gotway
Time: Fri Oct 04 10:31:10 2019
Ho-Chun,
I see that you have questions about ignoring observed AOD values of 0.
There are two good configuration options for you to consider. And
which
option you choose depends on how you want to handle those observed AOD
values of 0.
(1) The first option is to completely exclude all observed AOD values
of
0. You can do this by replacing all observed AOD values of 0 with bad
data. In MET, this is called "censoring" the data which is controlled
by
the "censor_thresh" and "censor_val" configuration options. In the
example
below, I removed the "file_type" option which shouldn't be necessary
when
reading the output of regrid_data_plane:
obs = {
wind_thresh = [ NA ];
field = [
{
name = "AOD";
level = [ "L0" ];
censor_thresh = [ <=0 ];
censor_value = [ -9999 ];
cat_thresh = [ >=0.1, >=0.2, >=0.3, >=0.4, >=0.5, >=0.6,
>=0.8,
>=1.0, >=1.5, >=2.0, >=2.5, >=3.0, >=3.5, >=4.0, >=4.5, >=5.0 ];
}
];
};
With this option, observed AOD values of 0 will be reset to bad data
and
those points will be entirely excluded from the analysis.
(2) The second choice uses the "cnt_thresh" configuration option
instead.
So we leave the observed AOD values of 0 as-is when creating matched
FCST/OBS pairs. We apply the "cat_thresh" thresholds to define 2x2
contingency tables and derived corresponding statistics, like PODY,
CSI,
and GSS. But we also add in a list of "cnt_thresh" thresholds which
control which matched pair values are used to compute continuous
statistics
(i.e. output CNT line type) and partial sums (i.e. SL1L2 line type).
obs = {
wind_thresh = [ NA ];
field = [
{
name = "AOD";
level = [ "L0" ];
cnt_thresh = [ NA, >0 ];
cat_thresh = [ >=0.1, >=0.2, >=0.3, >=0.4, >=0.5, >=0.6,
>=0.8,
>=1.0, >=1.5, >=2.0, >=2.5, >=3.0, >=3.5, >=4.0, >=4.5, >=5.0 ];
}
];
};
In the example listed above, instead of getting one output CNT line,
you'll
get two:
- 1 with all matched pairs include, when cnt_thresh = NA.
- 1 with only those matched pairs where the OBS AOD value is >0.
Hope that helps clarify the options.
Thanks,
John
On Fri, Oct 4, 2019 at 6:37 AM Ho-Chun Huang - NOAA Affiliate via RT <
met_help at ucar.edu> wrote:
>
> Fri Oct 04 06:37:08 2019: Request 92453 was acted upon.
> Transaction: Ticket created by ho-chun.huang at noaa.gov
> Queue: met_help
> Subject: define criteria in configuration file
> Owner: Nobody
> Requestors: ho-chun.huang at noaa.gov
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92453 >
>
>
> To Whom It may Concern:
>
> I have a question to select observed AOD >= 0 for grid_stat. I am
unsure
> which way to do it and need to clarify what MET grid-stat will do
based on
> its design OR according to configuration file & option specified.
>
> The GOES-16 AOD gridded nc file is produced by regrid_data_plane
with
> fill_value=-9999. When performing grid_stat, will those grids with
> fill-value automatically be excluded from the statistic calculation?
>
> If not, also going forward if a observed var has values < 0., then
how do I
> use only Observed AOD >=0 ?
>
> (A) In my current configuration, obs field is already specified as
>
> obs = {
> wind_thresh = [ NA ];
>
> field = [
> {
> file_type = NETCDF_MET; // NetCDF created by another MET
tool
> name = "AOD";
> level = [ "L0" ];
> cat_thresh = [ >=0.1, >=0.2, >=0.3, >=0.4, >=0.5, >=0.6,
>=0.8,
> >=1.0, >=1.5, >=2.0, >=2.5, >=3.0, >=3.5, >=4.0, >=4.5, >=5.0 ];
> }
> ];
>
> };
>
> cat_thresh is for categorized plot such as CSI. Should I add >= 0.0
in
> cat_threshold to generate the statistic, then when doing plot such
as RMSE
> or time series in Metviewer I selected obs_thresh >=0 [part of fixed
value]
> ?
>
> (B) Binbin uses
> poly = [ "${OBSV_FILE} {name= \"TCTD"; level = \"L0\";} >=0" ]; to
masking
> positive obs field
>
> I already has masking defined as below (stat for each regions) , how
do I
> append "${OBSV_FILE} {name= \"AOD"; level = \"L0\";} >=0" in the
statement
> of poly so grid_stat produces multiple stat for each regions coupled
> observed AOD > 0. criteria ? i.e.,
>
> for i in $area_mask do grid_stat for
> region $i && AOD >=0;
>
> mask = {
> grid = [ "FULL" ] ;
> poly = [ "CONUS_mask.nc",
> "EAST_mask.nc",
> "WEST_mask.nc",
> "APL_mask.nc",
> "ATC_mask.nc",
> "CAM_mask.nc",
> "CAR_mask.nc",
> "ECA_mask.nc",
> "GLF_mask.nc",
> "GMC_mask.nc",
> "GRB_mask.nc",
> "LMV_mask.nc",
> "MDW_mask.nc",
> "MEX_mask.nc",
> "NAK_mask.nc",
> "NAO_mask.nc",
> "NEC_mask.nc",
> "NMT_mask.nc",
> "NPL_mask.nc",
> "NPO_mask.nc",
> "NSA_mask.nc",
> "NWC_mask.nc",
> "PRI_mask.nc",
> "SAK_mask.nc",
> "SAO_mask.nc",
> "SEC_mask.nc",
> "SMT_mask.nc",
> "SPL_mask.nc",
> "SPO_mask.nc",
> "SWC_mask.nc",
> "SWD_mask.nc",
> "WCA_mask.nc"];
> };
>
> Thank you for your help.
>
> Ho-Chun
>
>
------------------------------------------------
Subject: define criteria in configuration file
From: Ho-Chun Huang - NOAA Affiliate
Time: Fri Oct 04 12:54:30 2019
Hi, John:
I pick option (1) but got error message of
ERROR : VarInfo::set_dict() -> The number of censor thresholds in
"censor_thresh" (1) must match the number of replacement values in
"censor_val" (0).
Can you figure out the error? using -9999. for -9999? or with "..."?
Ho-Chun
On Fri, Oct 4, 2019 at 12:31 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Ho-Chun,
>
> I see that you have questions about ignoring observed AOD values of
0.
> There are two good configuration options for you to consider. And
which
> option you choose depends on how you want to handle those observed
AOD
> values of 0.
>
> (1) The first option is to completely exclude all observed AOD
values of
> 0. You can do this by replacing all observed AOD values of 0 with
bad
> data. In MET, this is called "censoring" the data which is
controlled by
> the "censor_thresh" and "censor_val" configuration options. In the
example
> below, I removed the "file_type" option which shouldn't be necessary
when
> reading the output of regrid_data_plane:
>
> obs = {
> wind_thresh = [ NA ];
> field = [
> {
> name = "AOD";
> level = [ "L0" ];
> censor_thresh = [ <=0 ];
> censor_value = [ -9999 ];
> cat_thresh = [ >=0.1, >=0.2, >=0.3, >=0.4, >=0.5, >=0.6,
>=0.8,
> >=1.0, >=1.5, >=2.0, >=2.5, >=3.0, >=3.5, >=4.0, >=4.5, >=5.0 ];
> }
> ];
> };
>
> With this option, observed AOD values of 0 will be reset to bad data
and
> those points will be entirely excluded from the analysis.
>
> (2) The second choice uses the "cnt_thresh" configuration option
instead.
> So we leave the observed AOD values of 0 as-is when creating matched
> FCST/OBS pairs. We apply the "cat_thresh" thresholds to define 2x2
> contingency tables and derived corresponding statistics, like PODY,
CSI,
> and GSS. But we also add in a list of "cnt_thresh" thresholds which
> control which matched pair values are used to compute continuous
statistics
> (i.e. output CNT line type) and partial sums (i.e. SL1L2 line type).
>
> obs = {
> wind_thresh = [ NA ];
> field = [
> {
> name = "AOD";
> level = [ "L0" ];
> cnt_thresh = [ NA, >0 ];
> cat_thresh = [ >=0.1, >=0.2, >=0.3, >=0.4, >=0.5, >=0.6,
>=0.8,
> >=1.0, >=1.5, >=2.0, >=2.5, >=3.0, >=3.5, >=4.0, >=4.5, >=5.0 ];
> }
> ];
> };
>
> In the example listed above, instead of getting one output CNT line,
you'll
> get two:
> - 1 with all matched pairs include, when cnt_thresh = NA.
> - 1 with only those matched pairs where the OBS AOD value is >0.
>
> Hope that helps clarify the options.
>
> Thanks,
> John
>
> On Fri, Oct 4, 2019 at 6:37 AM Ho-Chun Huang - NOAA Affiliate via RT
<
> met_help at ucar.edu> wrote:
>
> >
> > Fri Oct 04 06:37:08 2019: Request 92453 was acted upon.
> > Transaction: Ticket created by ho-chun.huang at noaa.gov
> > Queue: met_help
> > Subject: define criteria in configuration file
> > Owner: Nobody
> > Requestors: ho-chun.huang at noaa.gov
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92453 >
> >
> >
> > To Whom It may Concern:
> >
> > I have a question to select observed AOD >= 0 for grid_stat. I am
unsure
> > which way to do it and need to clarify what MET grid-stat will do
based
> on
> > its design OR according to configuration file & option specified.
> >
> > The GOES-16 AOD gridded nc file is produced by regrid_data_plane
with
> > fill_value=-9999. When performing grid_stat, will those grids with
> > fill-value automatically be excluded from the statistic
calculation?
> >
> > If not, also going forward if a observed var has values < 0., then
how
> do I
> > use only Observed AOD >=0 ?
> >
> > (A) In my current configuration, obs field is already specified as
> >
> > obs = {
> > wind_thresh = [ NA ];
> >
> > field = [
> > {
> > file_type = NETCDF_MET; // NetCDF created by another MET
tool
> > name = "AOD";
> > level = [ "L0" ];
> > cat_thresh = [ >=0.1, >=0.2, >=0.3, >=0.4, >=0.5, >=0.6,
>=0.8,
> > >=1.0, >=1.5, >=2.0, >=2.5, >=3.0, >=3.5, >=4.0, >=4.5, >=5.0 ];
> > }
> > ];
> >
> > };
> >
> > cat_thresh is for categorized plot such as CSI. Should I add >=
0.0 in
> > cat_threshold to generate the statistic, then when doing plot such
as
> RMSE
> > or time series in Metviewer I selected obs_thresh >=0 [part of
fixed
> value]
> > ?
> >
> > (B) Binbin uses
> > poly = [ "${OBSV_FILE} {name= \"TCTD"; level = \"L0\";} >=0" ];
to
> masking
> > positive obs field
> >
> > I already has masking defined as below (stat for each regions) ,
how do I
> > append "${OBSV_FILE} {name= \"AOD"; level = \"L0\";} >=0" in the
> statement
> > of poly so grid_stat produces multiple stat for each regions
coupled
> > observed AOD > 0. criteria ? i.e.,
> >
> > for i in $area_mask do grid_stat for
> > region $i && AOD >=0;
> >
> > mask = {
> > grid = [ "FULL" ] ;
> > poly = [ "CONUS_mask.nc",
> > "EAST_mask.nc",
> > "WEST_mask.nc",
> > "APL_mask.nc",
> > "ATC_mask.nc",
> > "CAM_mask.nc",
> > "CAR_mask.nc",
> > "ECA_mask.nc",
> > "GLF_mask.nc",
> > "GMC_mask.nc",
> > "GRB_mask.nc",
> > "LMV_mask.nc",
> > "MDW_mask.nc",
> > "MEX_mask.nc",
> > "NAK_mask.nc",
> > "NAO_mask.nc",
> > "NEC_mask.nc",
> > "NMT_mask.nc",
> > "NPL_mask.nc",
> > "NPO_mask.nc",
> > "NSA_mask.nc",
> > "NWC_mask.nc",
> > "PRI_mask.nc",
> > "SAK_mask.nc",
> > "SAO_mask.nc",
> > "SEC_mask.nc",
> > "SMT_mask.nc",
> > "SPL_mask.nc",
> > "SPO_mask.nc",
> > "SWC_mask.nc",
> > "SWD_mask.nc",
> > "WCA_mask.nc"];
> > };
> >
> > Thank you for your help.
> >
> > Ho-Chun
> >
> >
>
>
------------------------------------------------
Subject: define criteria in configuration file
From: Ho-Chun Huang - NOAA Affiliate
Time: Fri Oct 04 13:00:54 2019
Hi, John:
I found out from user guide that should use "censor_val = [ ... ];"
Ho-Chun
On Fri, Oct 4, 2019 at 2:54 PM Ho-Chun Huang - NOAA Affiliate <
ho-chun.huang at noaa.gov> wrote:
> Hi, John:
>
> I pick option (1) but got error message of
>
> ERROR : VarInfo::set_dict() -> The number of censor thresholds in
> "censor_thresh" (1) must match the number of replacement values in
> "censor_val" (0).
>
> Can you figure out the error? using -9999. for -9999? or with
"..."?
>
> Ho-Chun
>
> On Fri, Oct 4, 2019 at 12:31 PM John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
>> Ho-Chun,
>>
>> I see that you have questions about ignoring observed AOD values of
0.
>> There are two good configuration options for you to consider. And
which
>> option you choose depends on how you want to handle those observed
AOD
>> values of 0.
>>
>> (1) The first option is to completely exclude all observed AOD
values of
>> 0. You can do this by replacing all observed AOD values of 0 with
bad
>> data. In MET, this is called "censoring" the data which is
controlled by
>> the "censor_thresh" and "censor_val" configuration options. In the
>> example
>> below, I removed the "file_type" option which shouldn't be
necessary when
>> reading the output of regrid_data_plane:
>>
>> obs = {
>> wind_thresh = [ NA ];
>> field = [
>> {
>> name = "AOD";
>> level = [ "L0" ];
>> censor_thresh = [ <=0 ];
>> censor_value = [ -9999 ];
>> cat_thresh = [ >=0.1, >=0.2, >=0.3, >=0.4, >=0.5, >=0.6,
>=0.8,
>> >=1.0, >=1.5, >=2.0, >=2.5, >=3.0, >=3.5, >=4.0, >=4.5, >=5.0 ];
>> }
>> ];
>> };
>>
>> With this option, observed AOD values of 0 will be reset to bad
data and
>> those points will be entirely excluded from the analysis.
>>
>> (2) The second choice uses the "cnt_thresh" configuration option
instead.
>> So we leave the observed AOD values of 0 as-is when creating
matched
>> FCST/OBS pairs. We apply the "cat_thresh" thresholds to define 2x2
>> contingency tables and derived corresponding statistics, like PODY,
CSI,
>> and GSS. But we also add in a list of "cnt_thresh" thresholds
which
>> control which matched pair values are used to compute continuous
>> statistics
>> (i.e. output CNT line type) and partial sums (i.e. SL1L2 line
type).
>>
>> obs = {
>> wind_thresh = [ NA ];
>> field = [
>> {
>> name = "AOD";
>> level = [ "L0" ];
>> cnt_thresh = [ NA, >0 ];
>> cat_thresh = [ >=0.1, >=0.2, >=0.3, >=0.4, >=0.5, >=0.6,
>=0.8,
>> >=1.0, >=1.5, >=2.0, >=2.5, >=3.0, >=3.5, >=4.0, >=4.5, >=5.0 ];
>> }
>> ];
>> };
>>
>> In the example listed above, instead of getting one output CNT
line,
>> you'll
>> get two:
>> - 1 with all matched pairs include, when cnt_thresh = NA.
>> - 1 with only those matched pairs where the OBS AOD value is >0.
>>
>> Hope that helps clarify the options.
>>
>> Thanks,
>> John
>>
>> On Fri, Oct 4, 2019 at 6:37 AM Ho-Chun Huang - NOAA Affiliate via
RT <
>> met_help at ucar.edu> wrote:
>>
>> >
>> > Fri Oct 04 06:37:08 2019: Request 92453 was acted upon.
>> > Transaction: Ticket created by ho-chun.huang at noaa.gov
>> > Queue: met_help
>> > Subject: define criteria in configuration file
>> > Owner: Nobody
>> > Requestors: ho-chun.huang at noaa.gov
>> > Status: new
>> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92453 >
>> >
>> >
>> > To Whom It may Concern:
>> >
>> > I have a question to select observed AOD >= 0 for grid_stat. I
am
>> unsure
>> > which way to do it and need to clarify what MET grid-stat will do
based
>> on
>> > its design OR according to configuration file & option specified.
>> >
>> > The GOES-16 AOD gridded nc file is produced by regrid_data_plane
with
>> > fill_value=-9999. When performing grid_stat, will those grids
with
>> > fill-value automatically be excluded from the statistic
calculation?
>> >
>> > If not, also going forward if a observed var has values < 0.,
then how
>> do I
>> > use only Observed AOD >=0 ?
>> >
>> > (A) In my current configuration, obs field is already specified
as
>> >
>> > obs = {
>> > wind_thresh = [ NA ];
>> >
>> > field = [
>> > {
>> > file_type = NETCDF_MET; // NetCDF created by another MET
tool
>> > name = "AOD";
>> > level = [ "L0" ];
>> > cat_thresh = [ >=0.1, >=0.2, >=0.3, >=0.4, >=0.5, >=0.6,
>=0.8,
>> > >=1.0, >=1.5, >=2.0, >=2.5, >=3.0, >=3.5, >=4.0, >=4.5, >=5.0 ];
>> > }
>> > ];
>> >
>> > };
>> >
>> > cat_thresh is for categorized plot such as CSI. Should I add >=
0.0 in
>> > cat_threshold to generate the statistic, then when doing plot
such as
>> RMSE
>> > or time series in Metviewer I selected obs_thresh >=0 [part of
fixed
>> value]
>> > ?
>> >
>> > (B) Binbin uses
>> > poly = [ "${OBSV_FILE} {name= \"TCTD"; level = \"L0\";} >=0" ];
to
>> masking
>> > positive obs field
>> >
>> > I already has masking defined as below (stat for each regions) ,
how do
>> I
>> > append "${OBSV_FILE} {name= \"AOD"; level = \"L0\";} >=0" in the
>> statement
>> > of poly so grid_stat produces multiple stat for each regions
coupled
>> > observed AOD > 0. criteria ? i.e.,
>> >
>> > for i in $area_mask do grid_stat for
>> > region $i && AOD >=0;
>> >
>> > mask = {
>> > grid = [ "FULL" ] ;
>> > poly = [ "CONUS_mask.nc",
>> > "EAST_mask.nc",
>> > "WEST_mask.nc",
>> > "APL_mask.nc",
>> > "ATC_mask.nc",
>> > "CAM_mask.nc",
>> > "CAR_mask.nc",
>> > "ECA_mask.nc",
>> > "GLF_mask.nc",
>> > "GMC_mask.nc",
>> > "GRB_mask.nc",
>> > "LMV_mask.nc",
>> > "MDW_mask.nc",
>> > "MEX_mask.nc",
>> > "NAK_mask.nc",
>> > "NAO_mask.nc",
>> > "NEC_mask.nc",
>> > "NMT_mask.nc",
>> > "NPL_mask.nc",
>> > "NPO_mask.nc",
>> > "NSA_mask.nc",
>> > "NWC_mask.nc",
>> > "PRI_mask.nc",
>> > "SAK_mask.nc",
>> > "SAO_mask.nc",
>> > "SEC_mask.nc",
>> > "SMT_mask.nc",
>> > "SPL_mask.nc",
>> > "SPO_mask.nc",
>> > "SWC_mask.nc",
>> > "SWD_mask.nc",
>> > "WCA_mask.nc"];
>> > };
>> >
>> > Thank you for your help.
>> >
>> > Ho-Chun
>> >
>> >
>>
>>
------------------------------------------------
Subject: define criteria in configuration file
From: Ho-Chun Huang - NOAA Affiliate
Time: Fri Oct 04 13:13:21 2019
Hi, John:
Thank you for your help. The verification is running. Please close
the
ticket.
Ho-Chun
On Fri, Oct 4, 2019 at 3:00 PM Ho-Chun Huang - NOAA Affiliate <
ho-chun.huang at noaa.gov> wrote:
> Hi, John:
>
> I found out from user guide that should use "censor_val = [ ... ];"
>
> Ho-Chun
>
> On Fri, Oct 4, 2019 at 2:54 PM Ho-Chun Huang - NOAA Affiliate <
> ho-chun.huang at noaa.gov> wrote:
>
>> Hi, John:
>>
>> I pick option (1) but got error message of
>>
>> ERROR : VarInfo::set_dict() -> The number of censor thresholds in
>> "censor_thresh" (1) must match the number of replacement values in
>> "censor_val" (0).
>>
>> Can you figure out the error? using -9999. for -9999? or with
"..."?
>>
>> Ho-Chun
>>
>> On Fri, Oct 4, 2019 at 12:31 PM John Halley Gotway via RT <
>> met_help at ucar.edu> wrote:
>>
>>> Ho-Chun,
>>>
>>> I see that you have questions about ignoring observed AOD values
of 0.
>>> There are two good configuration options for you to consider. And
which
>>> option you choose depends on how you want to handle those observed
AOD
>>> values of 0.
>>>
>>> (1) The first option is to completely exclude all observed AOD
values of
>>> 0. You can do this by replacing all observed AOD values of 0 with
bad
>>> data. In MET, this is called "censoring" the data which is
controlled by
>>> the "censor_thresh" and "censor_val" configuration options. In
the
>>> example
>>> below, I removed the "file_type" option which shouldn't be
necessary when
>>> reading the output of regrid_data_plane:
>>>
>>> obs = {
>>> wind_thresh = [ NA ];
>>> field = [
>>> {
>>> name = "AOD";
>>> level = [ "L0" ];
>>> censor_thresh = [ <=0 ];
>>> censor_value = [ -9999 ];
>>> cat_thresh = [ >=0.1, >=0.2, >=0.3, >=0.4, >=0.5, >=0.6,
>=0.8,
>>> >=1.0, >=1.5, >=2.0, >=2.5, >=3.0, >=3.5, >=4.0, >=4.5, >=5.0 ];
>>> }
>>> ];
>>> };
>>>
>>> With this option, observed AOD values of 0 will be reset to bad
data and
>>> those points will be entirely excluded from the analysis.
>>>
>>> (2) The second choice uses the "cnt_thresh" configuration option
instead.
>>> So we leave the observed AOD values of 0 as-is when creating
matched
>>> FCST/OBS pairs. We apply the "cat_thresh" thresholds to define
2x2
>>> contingency tables and derived corresponding statistics, like
PODY, CSI,
>>> and GSS. But we also add in a list of "cnt_thresh" thresholds
which
>>> control which matched pair values are used to compute continuous
>>> statistics
>>> (i.e. output CNT line type) and partial sums (i.e. SL1L2 line
type).
>>>
>>> obs = {
>>> wind_thresh = [ NA ];
>>> field = [
>>> {
>>> name = "AOD";
>>> level = [ "L0" ];
>>> cnt_thresh = [ NA, >0 ];
>>> cat_thresh = [ >=0.1, >=0.2, >=0.3, >=0.4, >=0.5, >=0.6,
>=0.8,
>>> >=1.0, >=1.5, >=2.0, >=2.5, >=3.0, >=3.5, >=4.0, >=4.5, >=5.0 ];
>>> }
>>> ];
>>> };
>>>
>>> In the example listed above, instead of getting one output CNT
line,
>>> you'll
>>> get two:
>>> - 1 with all matched pairs include, when cnt_thresh = NA.
>>> - 1 with only those matched pairs where the OBS AOD value is >0.
>>>
>>> Hope that helps clarify the options.
>>>
>>> Thanks,
>>> John
>>>
>>> On Fri, Oct 4, 2019 at 6:37 AM Ho-Chun Huang - NOAA Affiliate via
RT <
>>> met_help at ucar.edu> wrote:
>>>
>>> >
>>> > Fri Oct 04 06:37:08 2019: Request 92453 was acted upon.
>>> > Transaction: Ticket created by ho-chun.huang at noaa.gov
>>> > Queue: met_help
>>> > Subject: define criteria in configuration file
>>> > Owner: Nobody
>>> > Requestors: ho-chun.huang at noaa.gov
>>> > Status: new
>>> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=92453
>>> >
>>> >
>>> >
>>> > To Whom It may Concern:
>>> >
>>> > I have a question to select observed AOD >= 0 for grid_stat. I
am
>>> unsure
>>> > which way to do it and need to clarify what MET grid-stat will
do
>>> based on
>>> > its design OR according to configuration file & option
specified.
>>> >
>>> > The GOES-16 AOD gridded nc file is produced by regrid_data_plane
with
>>> > fill_value=-9999. When performing grid_stat, will those grids
with
>>> > fill-value automatically be excluded from the statistic
calculation?
>>> >
>>> > If not, also going forward if a observed var has values < 0.,
then how
>>> do I
>>> > use only Observed AOD >=0 ?
>>> >
>>> > (A) In my current configuration, obs field is already specified
as
>>> >
>>> > obs = {
>>> > wind_thresh = [ NA ];
>>> >
>>> > field = [
>>> > {
>>> > file_type = NETCDF_MET; // NetCDF created by another MET
tool
>>> > name = "AOD";
>>> > level = [ "L0" ];
>>> > cat_thresh = [ >=0.1, >=0.2, >=0.3, >=0.4, >=0.5, >=0.6,
>=0.8,
>>> > >=1.0, >=1.5, >=2.0, >=2.5, >=3.0, >=3.5, >=4.0, >=4.5, >=5.0 ];
>>> > }
>>> > ];
>>> >
>>> > };
>>> >
>>> > cat_thresh is for categorized plot such as CSI. Should I add >=
0.0 in
>>> > cat_threshold to generate the statistic, then when doing plot
such as
>>> RMSE
>>> > or time series in Metviewer I selected obs_thresh >=0 [part of
fixed
>>> value]
>>> > ?
>>> >
>>> > (B) Binbin uses
>>> > poly = [ "${OBSV_FILE} {name= \"TCTD"; level = \"L0\";} >=0" ];
to
>>> masking
>>> > positive obs field
>>> >
>>> > I already has masking defined as below (stat for each regions) ,
how
>>> do I
>>> > append "${OBSV_FILE} {name= \"AOD"; level = \"L0\";} >=0" in the
>>> statement
>>> > of poly so grid_stat produces multiple stat for each regions
coupled
>>> > observed AOD > 0. criteria ? i.e.,
>>> >
>>> > for i in $area_mask do grid_stat for
>>> > region $i && AOD >=0;
>>> >
>>> > mask = {
>>> > grid = [ "FULL" ] ;
>>> > poly = [ "CONUS_mask.nc",
>>> > "EAST_mask.nc",
>>> > "WEST_mask.nc",
>>> > "APL_mask.nc",
>>> > "ATC_mask.nc",
>>> > "CAM_mask.nc",
>>> > "CAR_mask.nc",
>>> > "ECA_mask.nc",
>>> > "GLF_mask.nc",
>>> > "GMC_mask.nc",
>>> > "GRB_mask.nc",
>>> > "LMV_mask.nc",
>>> > "MDW_mask.nc",
>>> > "MEX_mask.nc",
>>> > "NAK_mask.nc",
>>> > "NAO_mask.nc",
>>> > "NEC_mask.nc",
>>> > "NMT_mask.nc",
>>> > "NPL_mask.nc",
>>> > "NPO_mask.nc",
>>> > "NSA_mask.nc",
>>> > "NWC_mask.nc",
>>> > "PRI_mask.nc",
>>> > "SAK_mask.nc",
>>> > "SAO_mask.nc",
>>> > "SEC_mask.nc",
>>> > "SMT_mask.nc",
>>> > "SPL_mask.nc",
>>> > "SPO_mask.nc",
>>> > "SWC_mask.nc",
>>> > "SWD_mask.nc",
>>> > "WCA_mask.nc"];
>>> > };
>>> >
>>> > Thank you for your help.
>>> >
>>> > Ho-Chun
>>> >
>>> >
>>>
>>>
------------------------------------------------
More information about the Met_help
mailing list