[Met_help] [rt.rap.ucar.edu #96913] History for wind_thresh
John Halley Gotway via RT
met_help at ucar.edu
Mon Oct 19 19:27:07 MDT 2020
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hello,
Based on MET documentation, "wind_thresh" refers to the wind speed
thresholds. Why is it added to the default "PointStatConfig" file but there
is no "wind" variable at all?
Here are the lines from the default "PointStatConfig"
wind_thresh = [ NA ];
wind_logic = UNION;
What are the two lines used for?
Thank you.
Binyu
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: wind_thresh
From: John Halley Gotway
Time: Thu Oct 01 13:45:32 2020
Binyu,
The wind_thresh and wind_logic options are listed in the Point-Stat
config
file simply to provide default values. They are only applied in the
computation of VL1L2, VAL1L2, and VCNT output line types. If you're
not
processing winds, and therefore not requesting those output line
types,
then they are not used.
John
On Thu, Oct 1, 2020 at 12:25 PM binyu.wang at noaa.gov via RT <
met_help at ucar.edu> wrote:
>
> Thu Oct 01 12:24:34 2020: Request 96913 was acted upon.
> Transaction: Ticket created by binyu.wang at noaa.gov
> Queue: met_help
> Subject: wind_thresh
> Owner: Nobody
> Requestors: binyu.wang at noaa.gov
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96913 >
>
>
> Hello,
>
> Based on MET documentation, "wind_thresh" refers to the wind speed
> thresholds. Why is it added to the default "PointStatConfig" file
but there
> is no "wind" variable at all?
>
> Here are the lines from the default "PointStatConfig"
> wind_thresh = [ NA ];
> wind_logic = UNION;
>
> What are the two lines used for?
>
> Thank you.
> Binyu
>
>
------------------------------------------------
Subject: wind_thresh
From: binyu.wang at noaa.gov
Time: Thu Oct 01 13:59:08 2020
Thank you. John
There is another thing I want to confirm:
I am wondering what is the difference of ME and ME_OERR as lised on
the
page
https://dtcenter.github.io/MET/Users_Guide/ensemble-stat.html
eg:
If I have 21 ensemble members, 1 control and 20 perturb members. Based
on
the definition: ME is "The Root Mean Square Error of the ensemble mean
(unperturbed or supplied)"
Does that mean only the CONTROL member is used for ME, and the 20
perturb
is only used for ME_OERR?
Thank you.
Binyu
On Thu, Oct 1, 2020 at 3:50 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Binyu,
>
> The wind_thresh and wind_logic options are listed in the Point-Stat
config
> file simply to provide default values. They are only applied in the
> computation of VL1L2, VAL1L2, and VCNT output line types. If you're
not
> processing winds, and therefore not requesting those output line
types,
> then they are not used.
>
> John
>
> On Thu, Oct 1, 2020 at 12:25 PM binyu.wang at noaa.gov via RT <
> met_help at ucar.edu> wrote:
>
> >
> > Thu Oct 01 12:24:34 2020: Request 96913 was acted upon.
> > Transaction: Ticket created by binyu.wang at noaa.gov
> > Queue: met_help
> > Subject: wind_thresh
> > Owner: Nobody
> > Requestors: binyu.wang at noaa.gov
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96913 >
> >
> >
> > Hello,
> >
> > Based on MET documentation, "wind_thresh" refers to the wind speed
> > thresholds. Why is it added to the default "PointStatConfig" file
but
> there
> > is no "wind" variable at all?
> >
> > Here are the lines from the default "PointStatConfig"
> > wind_thresh = [ NA ];
> > wind_logic = UNION;
> >
> > What are the two lines used for?
> >
> > Thank you.
> > Binyu
> >
> >
>
>
------------------------------------------------
Subject: wind_thresh
From: John Halley Gotway
Time: Thu Oct 01 14:15:40 2020
Binyu,
I see you have a question about the ME and ME_OERR columns in the ECNT
line
type, described here:
https://dtcenter.github.io/MET/Users_Guide/ensemble-stat.html#id2
Ensemble-Stat includes the ability to account for observation
uncertainty.
This logic is based on some assumption about the distribution of the
errors
in your observation values, such as perhaps a known instrument error.
For
example, let's say you know that the error of your temperature
observations
follow a normal distribution with a standard deviation of 0.5 degrees.
You
can configure Ensemble-Stat to use this information in the
verification step.
When obs error parameters are supplied, Ensemble-Stat randomly
perturbs the
ensemble member forecast values within that range prior to computing
the
ensemble mean.
The ME column is the mean error you get from the raw ensemble member
values, with no obs error perturbation applied.
The ME_OERR column is the mean error you get from perturbed ensemble
member
values.
I would only recommend using this logic if you have some confidence in
your
estimate of the observation errors.
John
On Thu, Oct 1, 2020 at 1:59 PM binyu.wang at noaa.gov via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96913 >
>
> Thank you. John
> There is another thing I want to confirm:
> I am wondering what is the difference of ME and ME_OERR as lised on
the
> page
> https://dtcenter.github.io/MET/Users_Guide/ensemble-stat.html
>
> eg:
> If I have 21 ensemble members, 1 control and 20 perturb members.
Based on
> the definition: ME is "The Root Mean Square Error of the ensemble
mean
> (unperturbed or supplied)"
> Does that mean only the CONTROL member is used for ME, and the 20
perturb
> is only used for ME_OERR?
>
> Thank you.
> Binyu
>
> On Thu, Oct 1, 2020 at 3:50 PM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > Binyu,
> >
> > The wind_thresh and wind_logic options are listed in the Point-
Stat
> config
> > file simply to provide default values. They are only applied in
the
> > computation of VL1L2, VAL1L2, and VCNT output line types. If
you're not
> > processing winds, and therefore not requesting those output line
types,
> > then they are not used.
> >
> > John
> >
> > On Thu, Oct 1, 2020 at 12:25 PM binyu.wang at noaa.gov via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > Thu Oct 01 12:24:34 2020: Request 96913 was acted upon.
> > > Transaction: Ticket created by binyu.wang at noaa.gov
> > > Queue: met_help
> > > Subject: wind_thresh
> > > Owner: Nobody
> > > Requestors: binyu.wang at noaa.gov
> > > Status: new
> > > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96913
> >
> > >
> > >
> > > Hello,
> > >
> > > Based on MET documentation, "wind_thresh" refers to the wind
speed
> > > thresholds. Why is it added to the default "PointStatConfig"
file but
> > there
> > > is no "wind" variable at all?
> > >
> > > Here are the lines from the default "PointStatConfig"
> > > wind_thresh = [ NA ];
> > > wind_logic = UNION;
> > >
> > > What are the two lines used for?
> > >
> > > Thank you.
> > > Binyu
> > >
> > >
> >
> >
>
>
------------------------------------------------
Subject: wind_thresh
From: binyu.wang at noaa.gov
Time: Thu Oct 01 15:17:57 2020
Got it. So the "unperturbed " is not referred to the control mmber.
Since I don't have any obs. related error information, I should just
use ME
(not MET_OERR), as indicated in the formula below, f is the mean of
all the
ensemble members.
[image: image.png]
If you look at the config file at:
/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/MET/grid2grid/verf_met/parm/verf_g2g_grid_stat_regn_config_Raikoke
If I want to get ME (or RMSE) using a different threshold, like both
FCST
and OBS greater than 0.1 (or 0.01, 0r 1), is there any easy way to do
that?
Can I just set censor_thresh = [0.1] once and it will apply for
both
obs. and FCST or I have to set that twice separately under both FCST
and
obs? Do I have to change the config each time for a different
threshold or
is there any way I can get ME for a different threshold with only once
config file?
Thank you
Binyu
On Thu, Oct 1, 2020 at 4:20 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Binyu,
>
> I see you have a question about the ME and ME_OERR columns in the
ECNT line
> type, described here:
> https://dtcenter.github.io/MET/Users_Guide/ensemble-stat.html#id2
>
> Ensemble-Stat includes the ability to account for observation
uncertainty.
> This logic is based on some assumption about the distribution of the
errors
> in your observation values, such as perhaps a known instrument
error. For
> example, let's say you know that the error of your temperature
observations
> follow a normal distribution with a standard deviation of 0.5
degrees. You
> can configure Ensemble-Stat to use this information in the
> verification step.
>
> When obs error parameters are supplied, Ensemble-Stat randomly
perturbs the
> ensemble member forecast values within that range prior to computing
the
> ensemble mean.
>
> The ME column is the mean error you get from the raw ensemble member
> values, with no obs error perturbation applied.
> The ME_OERR column is the mean error you get from perturbed ensemble
member
> values.
>
> I would only recommend using this logic if you have some confidence
in your
> estimate of the observation errors.
>
> John
>
> On Thu, Oct 1, 2020 at 1:59 PM binyu.wang at noaa.gov via RT <
> met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96913 >
> >
> > Thank you. John
> > There is another thing I want to confirm:
> > I am wondering what is the difference of ME and ME_OERR as lised
on the
> > page
> > https://dtcenter.github.io/MET/Users_Guide/ensemble-stat.html
> >
> > eg:
> > If I have 21 ensemble members, 1 control and 20 perturb members.
Based on
> > the definition: ME is "The Root Mean Square Error of the ensemble
mean
> > (unperturbed or supplied)"
> > Does that mean only the CONTROL member is used for ME, and the 20
perturb
> > is only used for ME_OERR?
> >
> > Thank you.
> > Binyu
> >
> > On Thu, Oct 1, 2020 at 3:50 PM John Halley Gotway via RT <
> > met_help at ucar.edu>
> > wrote:
> >
> > > Binyu,
> > >
> > > The wind_thresh and wind_logic options are listed in the Point-
Stat
> > config
> > > file simply to provide default values. They are only applied in
the
> > > computation of VL1L2, VAL1L2, and VCNT output line types. If
you're not
> > > processing winds, and therefore not requesting those output line
types,
> > > then they are not used.
> > >
> > > John
> > >
> > > On Thu, Oct 1, 2020 at 12:25 PM binyu.wang at noaa.gov via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > Thu Oct 01 12:24:34 2020: Request 96913 was acted upon.
> > > > Transaction: Ticket created by binyu.wang at noaa.gov
> > > > Queue: met_help
> > > > Subject: wind_thresh
> > > > Owner: Nobody
> > > > Requestors: binyu.wang at noaa.gov
> > > > Status: new
> > > > Ticket <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96913
> > >
> > > >
> > > >
> > > > Hello,
> > > >
> > > > Based on MET documentation, "wind_thresh" refers to the wind
speed
> > > > thresholds. Why is it added to the default "PointStatConfig"
file but
> > > there
> > > > is no "wind" variable at all?
> > > >
> > > > Here are the lines from the default "PointStatConfig"
> > > > wind_thresh = [ NA ];
> > > > wind_logic = UNION;
> > > >
> > > > What are the two lines used for?
> > > >
> > > > Thank you.
> > > > Binyu
> > > >
> > > >
> > >
> > >
> >
> >
>
>
------------------------------------------------
Subject: wind_thresh
From: binyu.wang at noaa.gov
Time: Thu Oct 01 21:47:26 2020
Actually, this remind me another thing:
When you look at the config file in my previous email, you can see the
two
lines below:
censor_thresh=[<0.1];
censor_val=[-9999]; desc="GE0.1";
Which I learned from you, but it is for ensemble verification. So the
question is: I have to use the above two lines for ensemble, but if it
is
not ensemble, I can just use "fcst_thresh=[>0.1]" is that correct?
Thank you.
Binyu
On Thu, Oct 1, 2020 at 5:17 PM Binyu Wang - NOAA Affiliate <
binyu.wang at noaa.gov> wrote:
> Got it. So the "unperturbed " is not referred to the control mmber.
> Since I don't have any obs. related error information, I should just
use ME
> (not MET_OERR), as indicated in the formula below, f is the mean of
all the
> ensemble members.
> [image: image.png]
>
> If you look at the config file at:
>
>
/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/MET/grid2grid/verf_met/parm/verf_g2g_grid_stat_regn_config_Raikoke
> If I want to get ME (or RMSE) using a different threshold, like both
FCST
> and OBS greater than 0.1 (or 0.01, 0r 1), is there any easy way to
do that?
> Can I just set censor_thresh = [0.1] once and it will apply for
both
> obs. and FCST or I have to set that twice separately under both FCST
and
> obs? Do I have to change the config each time for a different
threshold or
> is there any way I can get ME for a different threshold with only
once
> config file?
>
> Thank you
> Binyu
>
>
>
> On Thu, Oct 1, 2020 at 4:20 PM John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
>> Binyu,
>>
>> I see you have a question about the ME and ME_OERR columns in the
ECNT
>> line
>> type, described here:
>> https://dtcenter.github.io/MET/Users_Guide/ensemble-stat.html#id2
>>
>> Ensemble-Stat includes the ability to account for observation
uncertainty.
>> This logic is based on some assumption about the distribution of
the
>> errors
>> in your observation values, such as perhaps a known instrument
error. For
>> example, let's say you know that the error of your temperature
>> observations
>> follow a normal distribution with a standard deviation of 0.5
degrees. You
>> can configure Ensemble-Stat to use this information in the
>> verification step.
>>
>> When obs error parameters are supplied, Ensemble-Stat randomly
perturbs
>> the
>> ensemble member forecast values within that range prior to
computing the
>> ensemble mean.
>>
>> The ME column is the mean error you get from the raw ensemble
member
>> values, with no obs error perturbation applied.
>> The ME_OERR column is the mean error you get from perturbed
ensemble
>> member
>> values.
>>
>> I would only recommend using this logic if you have some confidence
in
>> your
>> estimate of the observation errors.
>>
>> John
>>
>> On Thu, Oct 1, 2020 at 1:59 PM binyu.wang at noaa.gov via RT <
>> met_help at ucar.edu>
>> wrote:
>>
>> >
>> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96913 >
>> >
>> > Thank you. John
>> > There is another thing I want to confirm:
>> > I am wondering what is the difference of ME and ME_OERR as lised
on the
>> > page
>> > https://dtcenter.github.io/MET/Users_Guide/ensemble-stat.html
>> >
>> > eg:
>> > If I have 21 ensemble members, 1 control and 20 perturb members.
Based
>> on
>> > the definition: ME is "The Root Mean Square Error of the ensemble
mean
>> > (unperturbed or supplied)"
>> > Does that mean only the CONTROL member is used for ME, and the 20
>> perturb
>> > is only used for ME_OERR?
>> >
>> > Thank you.
>> > Binyu
>> >
>> > On Thu, Oct 1, 2020 at 3:50 PM John Halley Gotway via RT <
>> > met_help at ucar.edu>
>> > wrote:
>> >
>> > > Binyu,
>> > >
>> > > The wind_thresh and wind_logic options are listed in the Point-
Stat
>> > config
>> > > file simply to provide default values. They are only applied in
the
>> > > computation of VL1L2, VAL1L2, and VCNT output line types. If
you're
>> not
>> > > processing winds, and therefore not requesting those output
line
>> types,
>> > > then they are not used.
>> > >
>> > > John
>> > >
>> > > On Thu, Oct 1, 2020 at 12:25 PM binyu.wang at noaa.gov via RT <
>> > > met_help at ucar.edu> wrote:
>> > >
>> > > >
>> > > > Thu Oct 01 12:24:34 2020: Request 96913 was acted upon.
>> > > > Transaction: Ticket created by binyu.wang at noaa.gov
>> > > > Queue: met_help
>> > > > Subject: wind_thresh
>> > > > Owner: Nobody
>> > > > Requestors: binyu.wang at noaa.gov
>> > > > Status: new
>> > > > Ticket <URL:
>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96913
>> > >
>> > > >
>> > > >
>> > > > Hello,
>> > > >
>> > > > Based on MET documentation, "wind_thresh" refers to the wind
speed
>> > > > thresholds. Why is it added to the default "PointStatConfig"
file
>> but
>> > > there
>> > > > is no "wind" variable at all?
>> > > >
>> > > > Here are the lines from the default "PointStatConfig"
>> > > > wind_thresh = [ NA ];
>> > > > wind_logic = UNION;
>> > > >
>> > > > What are the two lines used for?
>> > > >
>> > > > Thank you.
>> > > > Binyu
>> > > >
>> > > >
>> > >
>> > >
>> >
>> >
>>
>>
------------------------------------------------
Subject: wind_thresh
From: John Halley Gotway
Time: Fri Oct 02 09:39:25 2020
Binyu,
No, that is not correct.
The only tool that supports the use of "fcst_thresh" is the STAT-
Analysis
tool, which is used to summarize and aggregate results across one or
more
runs of the other "stat" tools.
I suspect you're actually referring to the "cnt_thresh" filtering
option in
Point-Stat and Grid-Stat. What you're trying to do is called
"conditional
verification"... where you compute statistics but only using forecast
or
observation values that fall within a specific range.
In Point-Stat and Grid-Stat, you can set "cnt_thresh = [ NA, >0.1,
>0.2,
>0.3 ];"
And instead of getting 1 output CNT line, you'd get 4:
- One for all input values since cnt_thresh = NA; results in all pairs
being used.
- One only for values >0.1... one for >0.2... and one for >0.3.
We called it "cnt_thresh" because it controls which points are used
when
applying continuous verification methods. It affects the output of the
CNT,
SL1L2, and SAL1L2 line types.
You're trying to do the same thing in Ensemble-Stat, but it has no
existing
option to filter the pairs in this way. So instead, you're using data
censoring logic (censor_thresh and censor_val) to approximate this
logic.
Thanks,
John
On Thu, Oct 1, 2020 at 9:47 PM binyu.wang at noaa.gov via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96913 >
>
> Actually, this remind me another thing:
>
> When you look at the config file in my previous email, you can see
the two
> lines below:
> censor_thresh=[<0.1];
> censor_val=[-9999]; desc="GE0.1";
> Which I learned from you, but it is for ensemble verification. So
the
> question is: I have to use the above two lines for ensemble, but if
it is
> not ensemble, I can just use "fcst_thresh=[>0.1]" is that correct?
>
> Thank you.
> Binyu
>
>
> On Thu, Oct 1, 2020 at 5:17 PM Binyu Wang - NOAA Affiliate <
> binyu.wang at noaa.gov> wrote:
>
> > Got it. So the "unperturbed " is not referred to the control
mmber.
> > Since I don't have any obs. related error information, I should
just use
> ME
> > (not MET_OERR), as indicated in the formula below, f is the mean
of all
> the
> > ensemble members.
> > [image: image.png]
> >
> > If you look at the config file at:
> >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/MET/grid2grid/verf_met/parm/verf_g2g_grid_stat_regn_config_Raikoke
> > If I want to get ME (or RMSE) using a different threshold, like
both FCST
> > and OBS greater than 0.1 (or 0.01, 0r 1), is there any easy way to
do
> that?
> > Can I just set censor_thresh = [0.1] once and it will apply for
both
> > obs. and FCST or I have to set that twice separately under both
FCST and
> > obs? Do I have to change the config each time for a different
threshold
> or
> > is there any way I can get ME for a different threshold with only
once
> > config file?
> >
> > Thank you
> > Binyu
> >
> >
> >
> > On Thu, Oct 1, 2020 at 4:20 PM John Halley Gotway via RT <
> > met_help at ucar.edu> wrote:
> >
> >> Binyu,
> >>
> >> I see you have a question about the ME and ME_OERR columns in the
ECNT
> >> line
> >> type, described here:
> >> https://dtcenter.github.io/MET/Users_Guide/ensemble-stat.html#id2
> >>
> >> Ensemble-Stat includes the ability to account for observation
> uncertainty.
> >> This logic is based on some assumption about the distribution of
the
> >> errors
> >> in your observation values, such as perhaps a known instrument
error.
> For
> >> example, let's say you know that the error of your temperature
> >> observations
> >> follow a normal distribution with a standard deviation of 0.5
degrees.
> You
> >> can configure Ensemble-Stat to use this information in the
> >> verification step.
> >>
> >> When obs error parameters are supplied, Ensemble-Stat randomly
perturbs
> >> the
> >> ensemble member forecast values within that range prior to
computing the
> >> ensemble mean.
> >>
> >> The ME column is the mean error you get from the raw ensemble
member
> >> values, with no obs error perturbation applied.
> >> The ME_OERR column is the mean error you get from perturbed
ensemble
> >> member
> >> values.
> >>
> >> I would only recommend using this logic if you have some
confidence in
> >> your
> >> estimate of the observation errors.
> >>
> >> John
> >>
> >> On Thu, Oct 1, 2020 at 1:59 PM binyu.wang at noaa.gov via RT <
> >> met_help at ucar.edu>
> >> wrote:
> >>
> >> >
> >> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96913 >
> >> >
> >> > Thank you. John
> >> > There is another thing I want to confirm:
> >> > I am wondering what is the difference of ME and ME_OERR as
lised on
> the
> >> > page
> >> > https://dtcenter.github.io/MET/Users_Guide/ensemble-stat.html
> >> >
> >> > eg:
> >> > If I have 21 ensemble members, 1 control and 20 perturb
members. Based
> >> on
> >> > the definition: ME is "The Root Mean Square Error of the
ensemble mean
> >> > (unperturbed or supplied)"
> >> > Does that mean only the CONTROL member is used for ME, and the
20
> >> perturb
> >> > is only used for ME_OERR?
> >> >
> >> > Thank you.
> >> > Binyu
> >> >
> >> > On Thu, Oct 1, 2020 at 3:50 PM John Halley Gotway via RT <
> >> > met_help at ucar.edu>
> >> > wrote:
> >> >
> >> > > Binyu,
> >> > >
> >> > > The wind_thresh and wind_logic options are listed in the
Point-Stat
> >> > config
> >> > > file simply to provide default values. They are only applied
in the
> >> > > computation of VL1L2, VAL1L2, and VCNT output line types. If
you're
> >> not
> >> > > processing winds, and therefore not requesting those output
line
> >> types,
> >> > > then they are not used.
> >> > >
> >> > > John
> >> > >
> >> > > On Thu, Oct 1, 2020 at 12:25 PM binyu.wang at noaa.gov via RT <
> >> > > met_help at ucar.edu> wrote:
> >> > >
> >> > > >
> >> > > > Thu Oct 01 12:24:34 2020: Request 96913 was acted upon.
> >> > > > Transaction: Ticket created by binyu.wang at noaa.gov
> >> > > > Queue: met_help
> >> > > > Subject: wind_thresh
> >> > > > Owner: Nobody
> >> > > > Requestors: binyu.wang at noaa.gov
> >> > > > Status: new
> >> > > > Ticket <URL:
> >> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96913
> >> > >
> >> > > >
> >> > > >
> >> > > > Hello,
> >> > > >
> >> > > > Based on MET documentation, "wind_thresh" refers to the
wind speed
> >> > > > thresholds. Why is it added to the default
"PointStatConfig" file
> >> but
> >> > > there
> >> > > > is no "wind" variable at all?
> >> > > >
> >> > > > Here are the lines from the default "PointStatConfig"
> >> > > > wind_thresh = [ NA ];
> >> > > > wind_logic = UNION;
> >> > > >
> >> > > > What are the two lines used for?
> >> > > >
> >> > > > Thank you.
> >> > > > Binyu
> >> > > >
> >> > > >
> >> > >
> >> > >
> >> >
> >> >
> >>
> >>
>
>
------------------------------------------------
Subject: wind_thresh
From: binyu.wang at noaa.gov
Time: Fri Oct 02 13:27:57 2020
Thank you, John. Just to confirm "fcst_thresh" is the STAT-Analysis
tool
only, how about "obs_thresh"? Obs_thresh can only be used in "
STAT-Analysis" and "Ensemble_stat"?
Binyu
On Fri, Oct 2, 2020 at 11:39 AM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Binyu,
>
> No, that is not correct.
>
> The only tool that supports the use of "fcst_thresh" is the STAT-
Analysis
> tool, which is used to summarize and aggregate results across one or
more
> runs of the other "stat" tools.
>
> I suspect you're actually referring to the "cnt_thresh" filtering
option in
> Point-Stat and Grid-Stat. What you're trying to do is called
"conditional
> verification"... where you compute statistics but only using
forecast or
> observation values that fall within a specific range.
>
> In Point-Stat and Grid-Stat, you can set "cnt_thresh = [ NA, >0.1,
>0.2,
> >0.3 ];"
> And instead of getting 1 output CNT line, you'd get 4:
> - One for all input values since cnt_thresh = NA; results in all
pairs
> being used.
> - One only for values >0.1... one for >0.2... and one for >0.3.
> We called it "cnt_thresh" because it controls which points are used
when
> applying continuous verification methods. It affects the output of
the CNT,
> SL1L2, and SAL1L2 line types.
>
> You're trying to do the same thing in Ensemble-Stat, but it has no
existing
> option to filter the pairs in this way. So instead, you're using
data
> censoring logic (censor_thresh and censor_val) to approximate this
logic.
>
> Thanks,
> John
>
> On Thu, Oct 1, 2020 at 9:47 PM binyu.wang at noaa.gov via RT <
> met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96913 >
> >
> > Actually, this remind me another thing:
> >
> > When you look at the config file in my previous email, you can see
the
> two
> > lines below:
> > censor_thresh=[<0.1];
> > censor_val=[-9999]; desc="GE0.1";
> > Which I learned from you, but it is for ensemble verification. So
the
> > question is: I have to use the above two lines for ensemble, but
if it is
> > not ensemble, I can just use "fcst_thresh=[>0.1]" is that
correct?
> >
> > Thank you.
> > Binyu
> >
> >
> > On Thu, Oct 1, 2020 at 5:17 PM Binyu Wang - NOAA Affiliate <
> > binyu.wang at noaa.gov> wrote:
> >
> > > Got it. So the "unperturbed " is not referred to the control
mmber.
> > > Since I don't have any obs. related error information, I should
just
> use
> > ME
> > > (not MET_OERR), as indicated in the formula below, f is the mean
of all
> > the
> > > ensemble members.
> > > [image: image.png]
> > >
> > > If you look at the config file at:
> > >
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/MET/grid2grid/verf_met/parm/verf_g2g_grid_stat_regn_config_Raikoke
> > > If I want to get ME (or RMSE) using a different threshold, like
both
> FCST
> > > and OBS greater than 0.1 (or 0.01, 0r 1), is there any easy way
to do
> > that?
> > > Can I just set censor_thresh = [0.1] once and it will apply
for both
> > > obs. and FCST or I have to set that twice separately under both
FCST
> and
> > > obs? Do I have to change the config each time for a different
threshold
> > or
> > > is there any way I can get ME for a different threshold with
only once
> > > config file?
> > >
> > > Thank you
> > > Binyu
> > >
> > >
> > >
> > > On Thu, Oct 1, 2020 at 4:20 PM John Halley Gotway via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > >> Binyu,
> > >>
> > >> I see you have a question about the ME and ME_OERR columns in
the ECNT
> > >> line
> > >> type, described here:
> > >> https://dtcenter.github.io/MET/Users_Guide/ensemble-
stat.html#id2
> > >>
> > >> Ensemble-Stat includes the ability to account for observation
> > uncertainty.
> > >> This logic is based on some assumption about the distribution
of the
> > >> errors
> > >> in your observation values, such as perhaps a known instrument
error.
> > For
> > >> example, let's say you know that the error of your temperature
> > >> observations
> > >> follow a normal distribution with a standard deviation of 0.5
degrees.
> > You
> > >> can configure Ensemble-Stat to use this information in the
> > >> verification step.
> > >>
> > >> When obs error parameters are supplied, Ensemble-Stat randomly
> perturbs
> > >> the
> > >> ensemble member forecast values within that range prior to
computing
> the
> > >> ensemble mean.
> > >>
> > >> The ME column is the mean error you get from the raw ensemble
member
> > >> values, with no obs error perturbation applied.
> > >> The ME_OERR column is the mean error you get from perturbed
ensemble
> > >> member
> > >> values.
> > >>
> > >> I would only recommend using this logic if you have some
confidence in
> > >> your
> > >> estimate of the observation errors.
> > >>
> > >> John
> > >>
> > >> On Thu, Oct 1, 2020 at 1:59 PM binyu.wang at noaa.gov via RT <
> > >> met_help at ucar.edu>
> > >> wrote:
> > >>
> > >> >
> > >> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96913
>
> > >> >
> > >> > Thank you. John
> > >> > There is another thing I want to confirm:
> > >> > I am wondering what is the difference of ME and ME_OERR as
lised on
> > the
> > >> > page
> > >> > https://dtcenter.github.io/MET/Users_Guide/ensemble-stat.html
> > >> >
> > >> > eg:
> > >> > If I have 21 ensemble members, 1 control and 20 perturb
members.
> Based
> > >> on
> > >> > the definition: ME is "The Root Mean Square Error of the
ensemble
> mean
> > >> > (unperturbed or supplied)"
> > >> > Does that mean only the CONTROL member is used for ME, and
the 20
> > >> perturb
> > >> > is only used for ME_OERR?
> > >> >
> > >> > Thank you.
> > >> > Binyu
> > >> >
> > >> > On Thu, Oct 1, 2020 at 3:50 PM John Halley Gotway via RT <
> > >> > met_help at ucar.edu>
> > >> > wrote:
> > >> >
> > >> > > Binyu,
> > >> > >
> > >> > > The wind_thresh and wind_logic options are listed in the
> Point-Stat
> > >> > config
> > >> > > file simply to provide default values. They are only
applied in
> the
> > >> > > computation of VL1L2, VAL1L2, and VCNT output line types.
If
> you're
> > >> not
> > >> > > processing winds, and therefore not requesting those output
line
> > >> types,
> > >> > > then they are not used.
> > >> > >
> > >> > > John
> > >> > >
> > >> > > On Thu, Oct 1, 2020 at 12:25 PM binyu.wang at noaa.gov via RT
<
> > >> > > met_help at ucar.edu> wrote:
> > >> > >
> > >> > > >
> > >> > > > Thu Oct 01 12:24:34 2020: Request 96913 was acted upon.
> > >> > > > Transaction: Ticket created by binyu.wang at noaa.gov
> > >> > > > Queue: met_help
> > >> > > > Subject: wind_thresh
> > >> > > > Owner: Nobody
> > >> > > > Requestors: binyu.wang at noaa.gov
> > >> > > > Status: new
> > >> > > > Ticket <URL:
> > >> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96913
> > >> > >
> > >> > > >
> > >> > > >
> > >> > > > Hello,
> > >> > > >
> > >> > > > Based on MET documentation, "wind_thresh" refers to the
wind
> speed
> > >> > > > thresholds. Why is it added to the default
"PointStatConfig"
> file
> > >> but
> > >> > > there
> > >> > > > is no "wind" variable at all?
> > >> > > >
> > >> > > > Here are the lines from the default "PointStatConfig"
> > >> > > > wind_thresh = [ NA ];
> > >> > > > wind_logic = UNION;
> > >> > > >
> > >> > > > What are the two lines used for?
> > >> > > >
> > >> > > > Thank you.
> > >> > > > Binyu
> > >> > > >
> > >> > > >
> > >> > >
> > >> > >
> > >> >
> > >> >
> > >>
> > >>
> >
> >
>
>
------------------------------------------------
Subject: wind_thresh
From: John Halley Gotway
Time: Mon Oct 05 11:10:52 2020
Binyu,
To summarize...
- In Point-Stat and Grid-Stat, use "cnt_thresh" in the fcst or obs
dictionaries along with "cnt_logic" to filter which matched pairs are
included in the CNT, SL1L2, and SAL1L2 line types.
- In Point-Stat and Grid-Stat, use "wind_thresh" in the fcst or obs
dictionaries along with "wind_logic" to filter which U/V matched pairs
are
included in the VCNT, VL1L2, and VAL1L2 line types.
- In Ensemble-Stat, use "obs_thresh" in the obs dictionary to filter
which
observations are included in the output.
- In Stat-Analysis, use "-fcst_thresh", "-obs_thresh" options to
filter
which input lines are included in the analysis. This is string
matching to
select the input based on the contents of the FCST_THRESH and
OBS_THRESH
columns.
- In Stat-Analysis, use the "-column_thresh" option to define a
filtering
threshold for a numeric input column. For example, "-line_type MPR
-column_thresh FCST gt0" will only use MPR lines where the FCST column
contains a value greater than 0.
- Throughout the MET tools, you can use the "censor_thresh" and
"censor_val" options to "censor" the input data. And censoring means,
changing their values. You have been using this to set some of the
values
to bad data (-9999).
Thanks,
John
On Fri, Oct 2, 2020 at 1:28 PM binyu.wang at noaa.gov via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96913 >
>
> Thank you, John. Just to confirm "fcst_thresh" is the STAT-
Analysis tool
> only, how about "obs_thresh"? Obs_thresh can only be used in "
> STAT-Analysis" and "Ensemble_stat"?
>
> Binyu
>
>
> On Fri, Oct 2, 2020 at 11:39 AM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > Binyu,
> >
> > No, that is not correct.
> >
> > The only tool that supports the use of "fcst_thresh" is the STAT-
Analysis
> > tool, which is used to summarize and aggregate results across one
or more
> > runs of the other "stat" tools.
> >
> > I suspect you're actually referring to the "cnt_thresh" filtering
option
> in
> > Point-Stat and Grid-Stat. What you're trying to do is called
"conditional
> > verification"... where you compute statistics but only using
forecast or
> > observation values that fall within a specific range.
> >
> > In Point-Stat and Grid-Stat, you can set "cnt_thresh = [ NA, >0.1,
>0.2,
> > >0.3 ];"
> > And instead of getting 1 output CNT line, you'd get 4:
> > - One for all input values since cnt_thresh = NA; results in all
pairs
> > being used.
> > - One only for values >0.1... one for >0.2... and one for >0.3.
> > We called it "cnt_thresh" because it controls which points are
used when
> > applying continuous verification methods. It affects the output of
the
> CNT,
> > SL1L2, and SAL1L2 line types.
> >
> > You're trying to do the same thing in Ensemble-Stat, but it has no
> existing
> > option to filter the pairs in this way. So instead, you're using
data
> > censoring logic (censor_thresh and censor_val) to approximate this
logic.
> >
> > Thanks,
> > John
> >
> > On Thu, Oct 1, 2020 at 9:47 PM binyu.wang at noaa.gov via RT <
> > met_help at ucar.edu>
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96913 >
> > >
> > > Actually, this remind me another thing:
> > >
> > > When you look at the config file in my previous email, you can
see the
> > two
> > > lines below:
> > > censor_thresh=[<0.1];
> > > censor_val=[-9999]; desc="GE0.1";
> > > Which I learned from you, but it is for ensemble verification.
So the
> > > question is: I have to use the above two lines for ensemble, but
if it
> is
> > > not ensemble, I can just use "fcst_thresh=[>0.1]" is that
correct?
> > >
> > > Thank you.
> > > Binyu
> > >
> > >
> > > On Thu, Oct 1, 2020 at 5:17 PM Binyu Wang - NOAA Affiliate <
> > > binyu.wang at noaa.gov> wrote:
> > >
> > > > Got it. So the "unperturbed " is not referred to the control
mmber.
> > > > Since I don't have any obs. related error information, I
should just
> > use
> > > ME
> > > > (not MET_OERR), as indicated in the formula below, f is the
mean of
> all
> > > the
> > > > ensemble members.
> > > > [image: image.png]
> > > >
> > > > If you look at the config file at:
> > > >
> > > >
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/MET/grid2grid/verf_met/parm/verf_g2g_grid_stat_regn_config_Raikoke
> > > > If I want to get ME (or RMSE) using a different threshold,
like both
> > FCST
> > > > and OBS greater than 0.1 (or 0.01, 0r 1), is there any easy
way to do
> > > that?
> > > > Can I just set censor_thresh = [0.1] once and it will apply
for
> both
> > > > obs. and FCST or I have to set that twice separately under
both FCST
> > and
> > > > obs? Do I have to change the config each time for a different
> threshold
> > > or
> > > > is there any way I can get ME for a different threshold with
only
> once
> > > > config file?
> > > >
> > > > Thank you
> > > > Binyu
> > > >
> > > >
> > > >
> > > > On Thu, Oct 1, 2020 at 4:20 PM John Halley Gotway via RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > >> Binyu,
> > > >>
> > > >> I see you have a question about the ME and ME_OERR columns in
the
> ECNT
> > > >> line
> > > >> type, described here:
> > > >> https://dtcenter.github.io/MET/Users_Guide/ensemble-
stat.html#id2
> > > >>
> > > >> Ensemble-Stat includes the ability to account for observation
> > > uncertainty.
> > > >> This logic is based on some assumption about the distribution
of the
> > > >> errors
> > > >> in your observation values, such as perhaps a known
instrument
> error.
> > > For
> > > >> example, let's say you know that the error of your
temperature
> > > >> observations
> > > >> follow a normal distribution with a standard deviation of 0.5
> degrees.
> > > You
> > > >> can configure Ensemble-Stat to use this information in the
> > > >> verification step.
> > > >>
> > > >> When obs error parameters are supplied, Ensemble-Stat
randomly
> > perturbs
> > > >> the
> > > >> ensemble member forecast values within that range prior to
computing
> > the
> > > >> ensemble mean.
> > > >>
> > > >> The ME column is the mean error you get from the raw ensemble
member
> > > >> values, with no obs error perturbation applied.
> > > >> The ME_OERR column is the mean error you get from perturbed
ensemble
> > > >> member
> > > >> values.
> > > >>
> > > >> I would only recommend using this logic if you have some
confidence
> in
> > > >> your
> > > >> estimate of the observation errors.
> > > >>
> > > >> John
> > > >>
> > > >> On Thu, Oct 1, 2020 at 1:59 PM binyu.wang at noaa.gov via RT <
> > > >> met_help at ucar.edu>
> > > >> wrote:
> > > >>
> > > >> >
> > > >> > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96913 >
> > > >> >
> > > >> > Thank you. John
> > > >> > There is another thing I want to confirm:
> > > >> > I am wondering what is the difference of ME and ME_OERR as
lised
> on
> > > the
> > > >> > page
> > > >> > https://dtcenter.github.io/MET/Users_Guide/ensemble-
stat.html
> > > >> >
> > > >> > eg:
> > > >> > If I have 21 ensemble members, 1 control and 20 perturb
members.
> > Based
> > > >> on
> > > >> > the definition: ME is "The Root Mean Square Error of the
ensemble
> > mean
> > > >> > (unperturbed or supplied)"
> > > >> > Does that mean only the CONTROL member is used for ME, and
the 20
> > > >> perturb
> > > >> > is only used for ME_OERR?
> > > >> >
> > > >> > Thank you.
> > > >> > Binyu
> > > >> >
> > > >> > On Thu, Oct 1, 2020 at 3:50 PM John Halley Gotway via RT <
> > > >> > met_help at ucar.edu>
> > > >> > wrote:
> > > >> >
> > > >> > > Binyu,
> > > >> > >
> > > >> > > The wind_thresh and wind_logic options are listed in the
> > Point-Stat
> > > >> > config
> > > >> > > file simply to provide default values. They are only
applied in
> > the
> > > >> > > computation of VL1L2, VAL1L2, and VCNT output line types.
If
> > you're
> > > >> not
> > > >> > > processing winds, and therefore not requesting those
output line
> > > >> types,
> > > >> > > then they are not used.
> > > >> > >
> > > >> > > John
> > > >> > >
> > > >> > > On Thu, Oct 1, 2020 at 12:25 PM binyu.wang at noaa.gov via
RT <
> > > >> > > met_help at ucar.edu> wrote:
> > > >> > >
> > > >> > > >
> > > >> > > > Thu Oct 01 12:24:34 2020: Request 96913 was acted upon.
> > > >> > > > Transaction: Ticket created by binyu.wang at noaa.gov
> > > >> > > > Queue: met_help
> > > >> > > > Subject: wind_thresh
> > > >> > > > Owner: Nobody
> > > >> > > > Requestors: binyu.wang at noaa.gov
> > > >> > > > Status: new
> > > >> > > > Ticket <URL:
> > > >> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96913
> > > >> > >
> > > >> > > >
> > > >> > > >
> > > >> > > > Hello,
> > > >> > > >
> > > >> > > > Based on MET documentation, "wind_thresh" refers to the
wind
> > speed
> > > >> > > > thresholds. Why is it added to the default
"PointStatConfig"
> > file
> > > >> but
> > > >> > > there
> > > >> > > > is no "wind" variable at all?
> > > >> > > >
> > > >> > > > Here are the lines from the default "PointStatConfig"
> > > >> > > > wind_thresh = [ NA ];
> > > >> > > > wind_logic = UNION;
> > > >> > > >
> > > >> > > > What are the two lines used for?
> > > >> > > >
> > > >> > > > Thank you.
> > > >> > > > Binyu
> > > >> > > >
> > > >> > > >
> > > >> > >
> > > >> > >
> > > >> >
> > > >> >
> > > >>
> > > >>
> > >
> > >
> >
> >
>
>
------------------------------------------------
Subject: wind_thresh
From: binyu.wang at noaa.gov
Time: Mon Oct 05 11:45:59 2020
Thank you very much, John.
Binyu
On Mon, Oct 5, 2020 at 1:11 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Binyu,
>
> To summarize...
>
> - In Point-Stat and Grid-Stat, use "cnt_thresh" in the fcst or obs
> dictionaries along with "cnt_logic" to filter which matched pairs
are
> included in the CNT, SL1L2, and SAL1L2 line types.
>
> - In Point-Stat and Grid-Stat, use "wind_thresh" in the fcst or obs
> dictionaries along with "wind_logic" to filter which U/V matched
pairs are
> included in the VCNT, VL1L2, and VAL1L2 line types.
>
> - In Ensemble-Stat, use "obs_thresh" in the obs dictionary to filter
which
> observations are included in the output.
>
> - In Stat-Analysis, use "-fcst_thresh", "-obs_thresh" options to
filter
> which input lines are included in the analysis. This is string
matching to
> select the input based on the contents of the FCST_THRESH and
OBS_THRESH
> columns.
>
> - In Stat-Analysis, use the "-column_thresh" option to define a
filtering
> threshold for a numeric input column. For example, "-line_type MPR
> -column_thresh FCST gt0" will only use MPR lines where the FCST
column
> contains a value greater than 0.
>
> - Throughout the MET tools, you can use the "censor_thresh" and
> "censor_val" options to "censor" the input data. And censoring
means,
> changing their values. You have been using this to set some of the
values
> to bad data (-9999).
>
> Thanks,
> John
>
> On Fri, Oct 2, 2020 at 1:28 PM binyu.wang at noaa.gov via RT <
> met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96913 >
> >
> > Thank you, John. Just to confirm "fcst_thresh" is the STAT-
Analysis
> tool
> > only, how about "obs_thresh"? Obs_thresh can only be used in "
> > STAT-Analysis" and "Ensemble_stat"?
> >
> > Binyu
> >
> >
> > On Fri, Oct 2, 2020 at 11:39 AM John Halley Gotway via RT <
> > met_help at ucar.edu>
> > wrote:
> >
> > > Binyu,
> > >
> > > No, that is not correct.
> > >
> > > The only tool that supports the use of "fcst_thresh" is the
> STAT-Analysis
> > > tool, which is used to summarize and aggregate results across
one or
> more
> > > runs of the other "stat" tools.
> > >
> > > I suspect you're actually referring to the "cnt_thresh"
filtering
> option
> > in
> > > Point-Stat and Grid-Stat. What you're trying to do is called
> "conditional
> > > verification"... where you compute statistics but only using
forecast
> or
> > > observation values that fall within a specific range.
> > >
> > > In Point-Stat and Grid-Stat, you can set "cnt_thresh = [ NA,
>0.1,
> >0.2,
> > > >0.3 ];"
> > > And instead of getting 1 output CNT line, you'd get 4:
> > > - One for all input values since cnt_thresh = NA; results in all
pairs
> > > being used.
> > > - One only for values >0.1... one for >0.2... and one for >0.3.
> > > We called it "cnt_thresh" because it controls which points are
used
> when
> > > applying continuous verification methods. It affects the output
of the
> > CNT,
> > > SL1L2, and SAL1L2 line types.
> > >
> > > You're trying to do the same thing in Ensemble-Stat, but it has
no
> > existing
> > > option to filter the pairs in this way. So instead, you're using
data
> > > censoring logic (censor_thresh and censor_val) to approximate
this
> logic.
> > >
> > > Thanks,
> > > John
> > >
> > > On Thu, Oct 1, 2020 at 9:47 PM binyu.wang at noaa.gov via RT <
> > > met_help at ucar.edu>
> > > wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96913
>
> > > >
> > > > Actually, this remind me another thing:
> > > >
> > > > When you look at the config file in my previous email, you can
see
> the
> > > two
> > > > lines below:
> > > > censor_thresh=[<0.1];
> > > > censor_val=[-9999]; desc="GE0.1";
> > > > Which I learned from you, but it is for ensemble verification.
So the
> > > > question is: I have to use the above two lines for ensemble,
but if
> it
> > is
> > > > not ensemble, I can just use "fcst_thresh=[>0.1]" is that
correct?
> > > >
> > > > Thank you.
> > > > Binyu
> > > >
> > > >
> > > > On Thu, Oct 1, 2020 at 5:17 PM Binyu Wang - NOAA Affiliate <
> > > > binyu.wang at noaa.gov> wrote:
> > > >
> > > > > Got it. So the "unperturbed " is not referred to the
control
> mmber.
> > > > > Since I don't have any obs. related error information, I
should
> just
> > > use
> > > > ME
> > > > > (not MET_OERR), as indicated in the formula below, f is the
mean of
> > all
> > > > the
> > > > > ensemble members.
> > > > > [image: image.png]
> > > > >
> > > > > If you look at the config file at:
> > > > >
> > > > >
> > > >
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/MET/grid2grid/verf_met/parm/verf_g2g_grid_stat_regn_config_Raikoke
> > > > > If I want to get ME (or RMSE) using a different threshold,
like
> both
> > > FCST
> > > > > and OBS greater than 0.1 (or 0.01, 0r 1), is there any easy
way to
> do
> > > > that?
> > > > > Can I just set censor_thresh = [0.1] once and it will
apply for
> > both
> > > > > obs. and FCST or I have to set that twice separately under
both
> FCST
> > > and
> > > > > obs? Do I have to change the config each time for a
different
> > threshold
> > > > or
> > > > > is there any way I can get ME for a different threshold with
only
> > once
> > > > > config file?
> > > > >
> > > > > Thank you
> > > > > Binyu
> > > > >
> > > > >
> > > > >
> > > > > On Thu, Oct 1, 2020 at 4:20 PM John Halley Gotway via RT <
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > >> Binyu,
> > > > >>
> > > > >> I see you have a question about the ME and ME_OERR columns
in the
> > ECNT
> > > > >> line
> > > > >> type, described here:
> > > > >> https://dtcenter.github.io/MET/Users_Guide/ensemble-
stat.html#id2
> > > > >>
> > > > >> Ensemble-Stat includes the ability to account for
observation
> > > > uncertainty.
> > > > >> This logic is based on some assumption about the
distribution of
> the
> > > > >> errors
> > > > >> in your observation values, such as perhaps a known
instrument
> > error.
> > > > For
> > > > >> example, let's say you know that the error of your
temperature
> > > > >> observations
> > > > >> follow a normal distribution with a standard deviation of
0.5
> > degrees.
> > > > You
> > > > >> can configure Ensemble-Stat to use this information in the
> > > > >> verification step.
> > > > >>
> > > > >> When obs error parameters are supplied, Ensemble-Stat
randomly
> > > perturbs
> > > > >> the
> > > > >> ensemble member forecast values within that range prior to
> computing
> > > the
> > > > >> ensemble mean.
> > > > >>
> > > > >> The ME column is the mean error you get from the raw
ensemble
> member
> > > > >> values, with no obs error perturbation applied.
> > > > >> The ME_OERR column is the mean error you get from perturbed
> ensemble
> > > > >> member
> > > > >> values.
> > > > >>
> > > > >> I would only recommend using this logic if you have some
> confidence
> > in
> > > > >> your
> > > > >> estimate of the observation errors.
> > > > >>
> > > > >> John
> > > > >>
> > > > >> On Thu, Oct 1, 2020 at 1:59 PM binyu.wang at noaa.gov via RT <
> > > > >> met_help at ucar.edu>
> > > > >> wrote:
> > > > >>
> > > > >> >
> > > > >> > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96913 >
> > > > >> >
> > > > >> > Thank you. John
> > > > >> > There is another thing I want to confirm:
> > > > >> > I am wondering what is the difference of ME and ME_OERR
as lised
> > on
> > > > the
> > > > >> > page
> > > > >> > https://dtcenter.github.io/MET/Users_Guide/ensemble-
stat.html
> > > > >> >
> > > > >> > eg:
> > > > >> > If I have 21 ensemble members, 1 control and 20 perturb
members.
> > > Based
> > > > >> on
> > > > >> > the definition: ME is "The Root Mean Square Error of the
> ensemble
> > > mean
> > > > >> > (unperturbed or supplied)"
> > > > >> > Does that mean only the CONTROL member is used for ME,
and the
> 20
> > > > >> perturb
> > > > >> > is only used for ME_OERR?
> > > > >> >
> > > > >> > Thank you.
> > > > >> > Binyu
> > > > >> >
> > > > >> > On Thu, Oct 1, 2020 at 3:50 PM John Halley Gotway via RT
<
> > > > >> > met_help at ucar.edu>
> > > > >> > wrote:
> > > > >> >
> > > > >> > > Binyu,
> > > > >> > >
> > > > >> > > The wind_thresh and wind_logic options are listed in
the
> > > Point-Stat
> > > > >> > config
> > > > >> > > file simply to provide default values. They are only
applied
> in
> > > the
> > > > >> > > computation of VL1L2, VAL1L2, and VCNT output line
types. If
> > > you're
> > > > >> not
> > > > >> > > processing winds, and therefore not requesting those
output
> line
> > > > >> types,
> > > > >> > > then they are not used.
> > > > >> > >
> > > > >> > > John
> > > > >> > >
> > > > >> > > On Thu, Oct 1, 2020 at 12:25 PM binyu.wang at noaa.gov via
RT <
> > > > >> > > met_help at ucar.edu> wrote:
> > > > >> > >
> > > > >> > > >
> > > > >> > > > Thu Oct 01 12:24:34 2020: Request 96913 was acted
upon.
> > > > >> > > > Transaction: Ticket created by binyu.wang at noaa.gov
> > > > >> > > > Queue: met_help
> > > > >> > > > Subject: wind_thresh
> > > > >> > > > Owner: Nobody
> > > > >> > > > Requestors: binyu.wang at noaa.gov
> > > > >> > > > Status: new
> > > > >> > > > Ticket <URL:
> > > > >> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96913
> > > > >> > >
> > > > >> > > >
> > > > >> > > >
> > > > >> > > > Hello,
> > > > >> > > >
> > > > >> > > > Based on MET documentation, "wind_thresh" refers to
the wind
> > > speed
> > > > >> > > > thresholds. Why is it added to the default
"PointStatConfig"
> > > file
> > > > >> but
> > > > >> > > there
> > > > >> > > > is no "wind" variable at all?
> > > > >> > > >
> > > > >> > > > Here are the lines from the default
"PointStatConfig"
> > > > >> > > > wind_thresh = [ NA ];
> > > > >> > > > wind_logic = UNION;
> > > > >> > > >
> > > > >> > > > What are the two lines used for?
> > > > >> > > >
> > > > >> > > > Thank you.
> > > > >> > > > Binyu
> > > > >> > > >
> > > > >> > > >
> > > > >> > >
> > > > >> > >
> > > > >> >
> > > > >> >
> > > > >>
> > > > >>
> > > >
> > > >
> > >
> > >
> >
> >
>
>
------------------------------------------------
Subject: wind_thresh
From: binyu.wang at noaa.gov
Time: Fri Oct 16 13:36:40 2020
Hello John,
I am looking for a tool that can be used to make plots with prebufr
data so
I can find where there are observations. I happen to find the link "
https://mailman.ucar.edu/pipermail/met_help/2008-June/000275.html" and
seems you already had the tool. Do you mind sharing with me?
Thank you very much!
Binyu
On Mon, Oct 5, 2020 at 1:45 PM Binyu Wang - NOAA Affiliate <
binyu.wang at noaa.gov> wrote:
> Thank you very much, John.
>
> Binyu
>
> On Mon, Oct 5, 2020 at 1:11 PM John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
>> Binyu,
>>
>> To summarize...
>>
>> - In Point-Stat and Grid-Stat, use "cnt_thresh" in the fcst or obs
>> dictionaries along with "cnt_logic" to filter which matched pairs
are
>> included in the CNT, SL1L2, and SAL1L2 line types.
>>
>> - In Point-Stat and Grid-Stat, use "wind_thresh" in the fcst or obs
>> dictionaries along with "wind_logic" to filter which U/V matched
pairs are
>> included in the VCNT, VL1L2, and VAL1L2 line types.
>>
>> - In Ensemble-Stat, use "obs_thresh" in the obs dictionary to
filter which
>> observations are included in the output.
>>
>> - In Stat-Analysis, use "-fcst_thresh", "-obs_thresh" options to
filter
>> which input lines are included in the analysis. This is string
matching to
>> select the input based on the contents of the FCST_THRESH and
OBS_THRESH
>> columns.
>>
>> - In Stat-Analysis, use the "-column_thresh" option to define a
filtering
>> threshold for a numeric input column. For example, "-line_type MPR
>> -column_thresh FCST gt0" will only use MPR lines where the FCST
column
>> contains a value greater than 0.
>>
>> - Throughout the MET tools, you can use the "censor_thresh" and
>> "censor_val" options to "censor" the input data. And censoring
means,
>> changing their values. You have been using this to set some of the
values
>> to bad data (-9999).
>>
>> Thanks,
>> John
>>
>> On Fri, Oct 2, 2020 at 1:28 PM binyu.wang at noaa.gov via RT <
>> met_help at ucar.edu>
>> wrote:
>>
>> >
>> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96913 >
>> >
>> > Thank you, John. Just to confirm "fcst_thresh" is the STAT-
Analysis
>> tool
>> > only, how about "obs_thresh"? Obs_thresh can only be used in "
>> > STAT-Analysis" and "Ensemble_stat"?
>> >
>> > Binyu
>> >
>> >
>> > On Fri, Oct 2, 2020 at 11:39 AM John Halley Gotway via RT <
>> > met_help at ucar.edu>
>> > wrote:
>> >
>> > > Binyu,
>> > >
>> > > No, that is not correct.
>> > >
>> > > The only tool that supports the use of "fcst_thresh" is the
>> STAT-Analysis
>> > > tool, which is used to summarize and aggregate results across
one or
>> more
>> > > runs of the other "stat" tools.
>> > >
>> > > I suspect you're actually referring to the "cnt_thresh"
filtering
>> option
>> > in
>> > > Point-Stat and Grid-Stat. What you're trying to do is called
>> "conditional
>> > > verification"... where you compute statistics but only using
forecast
>> or
>> > > observation values that fall within a specific range.
>> > >
>> > > In Point-Stat and Grid-Stat, you can set "cnt_thresh = [ NA,
>0.1,
>> >0.2,
>> > > >0.3 ];"
>> > > And instead of getting 1 output CNT line, you'd get 4:
>> > > - One for all input values since cnt_thresh = NA; results in
all pairs
>> > > being used.
>> > > - One only for values >0.1... one for >0.2... and one for >0.3.
>> > > We called it "cnt_thresh" because it controls which points are
used
>> when
>> > > applying continuous verification methods. It affects the output
of the
>> > CNT,
>> > > SL1L2, and SAL1L2 line types.
>> > >
>> > > You're trying to do the same thing in Ensemble-Stat, but it has
no
>> > existing
>> > > option to filter the pairs in this way. So instead, you're
using data
>> > > censoring logic (censor_thresh and censor_val) to approximate
this
>> logic.
>> > >
>> > > Thanks,
>> > > John
>> > >
>> > > On Thu, Oct 1, 2020 at 9:47 PM binyu.wang at noaa.gov via RT <
>> > > met_help at ucar.edu>
>> > > wrote:
>> > >
>> > > >
>> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96913
>
>> > > >
>> > > > Actually, this remind me another thing:
>> > > >
>> > > > When you look at the config file in my previous email, you
can see
>> the
>> > > two
>> > > > lines below:
>> > > > censor_thresh=[<0.1];
>> > > > censor_val=[-9999]; desc="GE0.1";
>> > > > Which I learned from you, but it is for ensemble
verification. So
>> the
>> > > > question is: I have to use the above two lines for ensemble,
but if
>> it
>> > is
>> > > > not ensemble, I can just use "fcst_thresh=[>0.1]" is that
correct?
>> > > >
>> > > > Thank you.
>> > > > Binyu
>> > > >
>> > > >
>> > > > On Thu, Oct 1, 2020 at 5:17 PM Binyu Wang - NOAA Affiliate <
>> > > > binyu.wang at noaa.gov> wrote:
>> > > >
>> > > > > Got it. So the "unperturbed " is not referred to the
control
>> mmber.
>> > > > > Since I don't have any obs. related error information, I
should
>> just
>> > > use
>> > > > ME
>> > > > > (not MET_OERR), as indicated in the formula below, f is the
mean
>> of
>> > all
>> > > > the
>> > > > > ensemble members.
>> > > > > [image: image.png]
>> > > > >
>> > > > > If you look at the config file at:
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/MET/grid2grid/verf_met/parm/verf_g2g_grid_stat_regn_config_Raikoke
>> > > > > If I want to get ME (or RMSE) using a different threshold,
like
>> both
>> > > FCST
>> > > > > and OBS greater than 0.1 (or 0.01, 0r 1), is there any easy
way
>> to do
>> > > > that?
>> > > > > Can I just set censor_thresh = [0.1] once and it will
apply for
>> > both
>> > > > > obs. and FCST or I have to set that twice separately under
both
>> FCST
>> > > and
>> > > > > obs? Do I have to change the config each time for a
different
>> > threshold
>> > > > or
>> > > > > is there any way I can get ME for a different threshold
with only
>> > once
>> > > > > config file?
>> > > > >
>> > > > > Thank you
>> > > > > Binyu
>> > > > >
>> > > > >
>> > > > >
>> > > > > On Thu, Oct 1, 2020 at 4:20 PM John Halley Gotway via RT <
>> > > > > met_help at ucar.edu> wrote:
>> > > > >
>> > > > >> Binyu,
>> > > > >>
>> > > > >> I see you have a question about the ME and ME_OERR columns
in the
>> > ECNT
>> > > > >> line
>> > > > >> type, described here:
>> > > > >>
>> https://dtcenter.github.io/MET/Users_Guide/ensemble-stat.html#id2
>> > > > >>
>> > > > >> Ensemble-Stat includes the ability to account for
observation
>> > > > uncertainty.
>> > > > >> This logic is based on some assumption about the
distribution of
>> the
>> > > > >> errors
>> > > > >> in your observation values, such as perhaps a known
instrument
>> > error.
>> > > > For
>> > > > >> example, let's say you know that the error of your
temperature
>> > > > >> observations
>> > > > >> follow a normal distribution with a standard deviation of
0.5
>> > degrees.
>> > > > You
>> > > > >> can configure Ensemble-Stat to use this information in the
>> > > > >> verification step.
>> > > > >>
>> > > > >> When obs error parameters are supplied, Ensemble-Stat
randomly
>> > > perturbs
>> > > > >> the
>> > > > >> ensemble member forecast values within that range prior to
>> computing
>> > > the
>> > > > >> ensemble mean.
>> > > > >>
>> > > > >> The ME column is the mean error you get from the raw
ensemble
>> member
>> > > > >> values, with no obs error perturbation applied.
>> > > > >> The ME_OERR column is the mean error you get from
perturbed
>> ensemble
>> > > > >> member
>> > > > >> values.
>> > > > >>
>> > > > >> I would only recommend using this logic if you have some
>> confidence
>> > in
>> > > > >> your
>> > > > >> estimate of the observation errors.
>> > > > >>
>> > > > >> John
>> > > > >>
>> > > > >> On Thu, Oct 1, 2020 at 1:59 PM binyu.wang at noaa.gov via RT
<
>> > > > >> met_help at ucar.edu>
>> > > > >> wrote:
>> > > > >>
>> > > > >> >
>> > > > >> > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96913
>> >
>> > > > >> >
>> > > > >> > Thank you. John
>> > > > >> > There is another thing I want to confirm:
>> > > > >> > I am wondering what is the difference of ME and ME_OERR
as
>> lised
>> > on
>> > > > the
>> > > > >> > page
>> > > > >> > https://dtcenter.github.io/MET/Users_Guide/ensemble-
stat.html
>> > > > >> >
>> > > > >> > eg:
>> > > > >> > If I have 21 ensemble members, 1 control and 20 perturb
>> members.
>> > > Based
>> > > > >> on
>> > > > >> > the definition: ME is "The Root Mean Square Error of the
>> ensemble
>> > > mean
>> > > > >> > (unperturbed or supplied)"
>> > > > >> > Does that mean only the CONTROL member is used for ME,
and the
>> 20
>> > > > >> perturb
>> > > > >> > is only used for ME_OERR?
>> > > > >> >
>> > > > >> > Thank you.
>> > > > >> > Binyu
>> > > > >> >
>> > > > >> > On Thu, Oct 1, 2020 at 3:50 PM John Halley Gotway via RT
<
>> > > > >> > met_help at ucar.edu>
>> > > > >> > wrote:
>> > > > >> >
>> > > > >> > > Binyu,
>> > > > >> > >
>> > > > >> > > The wind_thresh and wind_logic options are listed in
the
>> > > Point-Stat
>> > > > >> > config
>> > > > >> > > file simply to provide default values. They are only
applied
>> in
>> > > the
>> > > > >> > > computation of VL1L2, VAL1L2, and VCNT output line
types. If
>> > > you're
>> > > > >> not
>> > > > >> > > processing winds, and therefore not requesting those
output
>> line
>> > > > >> types,
>> > > > >> > > then they are not used.
>> > > > >> > >
>> > > > >> > > John
>> > > > >> > >
>> > > > >> > > On Thu, Oct 1, 2020 at 12:25 PM binyu.wang at noaa.gov
via RT <
>> > > > >> > > met_help at ucar.edu> wrote:
>> > > > >> > >
>> > > > >> > > >
>> > > > >> > > > Thu Oct 01 12:24:34 2020: Request 96913 was acted
upon.
>> > > > >> > > > Transaction: Ticket created by binyu.wang at noaa.gov
>> > > > >> > > > Queue: met_help
>> > > > >> > > > Subject: wind_thresh
>> > > > >> > > > Owner: Nobody
>> > > > >> > > > Requestors: binyu.wang at noaa.gov
>> > > > >> > > > Status: new
>> > > > >> > > > Ticket <URL:
>> > > > >> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96913
>> > > > >> > >
>> > > > >> > > >
>> > > > >> > > >
>> > > > >> > > > Hello,
>> > > > >> > > >
>> > > > >> > > > Based on MET documentation, "wind_thresh" refers to
the
>> wind
>> > > speed
>> > > > >> > > > thresholds. Why is it added to the default
>> "PointStatConfig"
>> > > file
>> > > > >> but
>> > > > >> > > there
>> > > > >> > > > is no "wind" variable at all?
>> > > > >> > > >
>> > > > >> > > > Here are the lines from the default
"PointStatConfig"
>> > > > >> > > > wind_thresh = [ NA ];
>> > > > >> > > > wind_logic = UNION;
>> > > > >> > > >
>> > > > >> > > > What are the two lines used for?
>> > > > >> > > >
>> > > > >> > > > Thank you.
>> > > > >> > > > Binyu
>> > > > >> > > >
>> > > > >> > > >
>> > > > >> > >
>> > > > >> > >
>> > > > >> >
>> > > > >> >
>> > > > >>
>> > > > >>
>> > > >
>> > > >
>> > >
>> > >
>> >
>> >
>>
>>
------------------------------------------------
Subject: wind_thresh
From: John Halley Gotway
Time: Fri Oct 16 13:52:03 2020
Binyu,
Several tools in MET pre-process point observations and write to a
common
output NetCDF file format. They are ascii2nc, madis2nc, lidar2nc, and
pb2nc. The location of the observations in those NetCDF files can be
plotted using the plot_point_obs tool. And that lives in the same
directory
where the other MET executables live.
Here’s the section from the users guide about that:
https://dtcenter.github.io/MET/latest/Users_Guide/plotting.html#plot-
point-obs-usage
And on this page of the online tutorial, we run plot_point_obs on the
output of pb2nc:
https://dtcenter.org/metplus-practical-session-guide-version-3-
0/session-2-grid-obs/met-tool-pb2nc/pb2nc-tool-output
Hopefully those examples will help get you going.
Thanks
John
On Fri, Oct 16, 2020 at 1:37 PM binyu.wang at noaa.gov via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96913 >
>
> Hello John,
>
> I am looking for a tool that can be used to make plots with prebufr
data so
> I can find where there are observations. I happen to find the link "
> https://mailman.ucar.edu/pipermail/met_help/2008-June/000275.html"
and
> seems you already had the tool. Do you mind sharing with me?
>
> Thank you very much!
> Binyu
>
> On Mon, Oct 5, 2020 at 1:45 PM Binyu Wang - NOAA Affiliate <
> binyu.wang at noaa.gov> wrote:
>
> > Thank you very much, John.
> >
> > Binyu
> >
> > On Mon, Oct 5, 2020 at 1:11 PM John Halley Gotway via RT <
> > met_help at ucar.edu> wrote:
> >
> >> Binyu,
> >>
> >> To summarize...
> >>
> >> - In Point-Stat and Grid-Stat, use "cnt_thresh" in the fcst or
obs
> >> dictionaries along with "cnt_logic" to filter which matched pairs
are
> >> included in the CNT, SL1L2, and SAL1L2 line types.
> >>
> >> - In Point-Stat and Grid-Stat, use "wind_thresh" in the fcst or
obs
> >> dictionaries along with "wind_logic" to filter which U/V matched
pairs
> are
> >> included in the VCNT, VL1L2, and VAL1L2 line types.
> >>
> >> - In Ensemble-Stat, use "obs_thresh" in the obs dictionary to
filter
> which
> >> observations are included in the output.
> >>
> >> - In Stat-Analysis, use "-fcst_thresh", "-obs_thresh" options to
filter
> >> which input lines are included in the analysis. This is string
matching
> to
> >> select the input based on the contents of the FCST_THRESH and
OBS_THRESH
> >> columns.
> >>
> >> - In Stat-Analysis, use the "-column_thresh" option to define a
> filtering
> >> threshold for a numeric input column. For example, "-line_type
MPR
> >> -column_thresh FCST gt0" will only use MPR lines where the FCST
column
> >> contains a value greater than 0.
> >>
> >> - Throughout the MET tools, you can use the "censor_thresh" and
> >> "censor_val" options to "censor" the input data. And censoring
means,
> >> changing their values. You have been using this to set some of
the
> values
> >> to bad data (-9999).
> >>
> >> Thanks,
> >> John
> >>
> >> On Fri, Oct 2, 2020 at 1:28 PM binyu.wang at noaa.gov via RT <
> >> met_help at ucar.edu>
> >> wrote:
> >>
> >> >
> >> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96913 >
> >> >
> >> > Thank you, John. Just to confirm "fcst_thresh" is the STAT-
Analysis
> >> tool
> >> > only, how about "obs_thresh"? Obs_thresh can only be used in "
> >> > STAT-Analysis" and "Ensemble_stat"?
> >> >
> >> > Binyu
> >> >
> >> >
> >> > On Fri, Oct 2, 2020 at 11:39 AM John Halley Gotway via RT <
> >> > met_help at ucar.edu>
> >> > wrote:
> >> >
> >> > > Binyu,
> >> > >
> >> > > No, that is not correct.
> >> > >
> >> > > The only tool that supports the use of "fcst_thresh" is the
> >> STAT-Analysis
> >> > > tool, which is used to summarize and aggregate results across
one or
> >> more
> >> > > runs of the other "stat" tools.
> >> > >
> >> > > I suspect you're actually referring to the "cnt_thresh"
filtering
> >> option
> >> > in
> >> > > Point-Stat and Grid-Stat. What you're trying to do is called
> >> "conditional
> >> > > verification"... where you compute statistics but only using
> forecast
> >> or
> >> > > observation values that fall within a specific range.
> >> > >
> >> > > In Point-Stat and Grid-Stat, you can set "cnt_thresh = [ NA,
>0.1,
> >> >0.2,
> >> > > >0.3 ];"
> >> > > And instead of getting 1 output CNT line, you'd get 4:
> >> > > - One for all input values since cnt_thresh = NA; results in
all
> pairs
> >> > > being used.
> >> > > - One only for values >0.1... one for >0.2... and one for
>0.3.
> >> > > We called it "cnt_thresh" because it controls which points
are used
> >> when
> >> > > applying continuous verification methods. It affects the
output of
> the
> >> > CNT,
> >> > > SL1L2, and SAL1L2 line types.
> >> > >
> >> > > You're trying to do the same thing in Ensemble-Stat, but it
has no
> >> > existing
> >> > > option to filter the pairs in this way. So instead, you're
using
> data
> >> > > censoring logic (censor_thresh and censor_val) to approximate
this
> >> logic.
> >> > >
> >> > > Thanks,
> >> > > John
> >> > >
> >> > > On Thu, Oct 1, 2020 at 9:47 PM binyu.wang at noaa.gov via RT <
> >> > > met_help at ucar.edu>
> >> > > wrote:
> >> > >
> >> > > >
> >> > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96913 >
> >> > > >
> >> > > > Actually, this remind me another thing:
> >> > > >
> >> > > > When you look at the config file in my previous email, you
can see
> >> the
> >> > > two
> >> > > > lines below:
> >> > > > censor_thresh=[<0.1];
> >> > > > censor_val=[-9999]; desc="GE0.1";
> >> > > > Which I learned from you, but it is for ensemble
verification. So
> >> the
> >> > > > question is: I have to use the above two lines for
ensemble, but
> if
> >> it
> >> > is
> >> > > > not ensemble, I can just use "fcst_thresh=[>0.1]" is that
> correct?
> >> > > >
> >> > > > Thank you.
> >> > > > Binyu
> >> > > >
> >> > > >
> >> > > > On Thu, Oct 1, 2020 at 5:17 PM Binyu Wang - NOAA Affiliate
<
> >> > > > binyu.wang at noaa.gov> wrote:
> >> > > >
> >> > > > > Got it. So the "unperturbed " is not referred to the
control
> >> mmber.
> >> > > > > Since I don't have any obs. related error information, I
should
> >> just
> >> > > use
> >> > > > ME
> >> > > > > (not MET_OERR), as indicated in the formula below, f is
the mean
> >> of
> >> > all
> >> > > > the
> >> > > > > ensemble members.
> >> > > > > [image: image.png]
> >> > > > >
> >> > > > > If you look at the config file at:
> >> > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
>
/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/MET/grid2grid/verf_met/parm/verf_g2g_grid_stat_regn_config_Raikoke
> >> > > > > If I want to get ME (or RMSE) using a different
threshold, like
> >> both
> >> > > FCST
> >> > > > > and OBS greater than 0.1 (or 0.01, 0r 1), is there any
easy way
> >> to do
> >> > > > that?
> >> > > > > Can I just set censor_thresh = [0.1] once and it will
apply
> for
> >> > both
> >> > > > > obs. and FCST or I have to set that twice separately
under both
> >> FCST
> >> > > and
> >> > > > > obs? Do I have to change the config each time for a
different
> >> > threshold
> >> > > > or
> >> > > > > is there any way I can get ME for a different threshold
with
> only
> >> > once
> >> > > > > config file?
> >> > > > >
> >> > > > > Thank you
> >> > > > > Binyu
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > > On Thu, Oct 1, 2020 at 4:20 PM John Halley Gotway via RT
<
> >> > > > > met_help at ucar.edu> wrote:
> >> > > > >
> >> > > > >> Binyu,
> >> > > > >>
> >> > > > >> I see you have a question about the ME and ME_OERR
columns in
> the
> >> > ECNT
> >> > > > >> line
> >> > > > >> type, described here:
> >> > > > >>
> >> https://dtcenter.github.io/MET/Users_Guide/ensemble-stat.html#id2
> >> > > > >>
> >> > > > >> Ensemble-Stat includes the ability to account for
observation
> >> > > > uncertainty.
> >> > > > >> This logic is based on some assumption about the
distribution
> of
> >> the
> >> > > > >> errors
> >> > > > >> in your observation values, such as perhaps a known
instrument
> >> > error.
> >> > > > For
> >> > > > >> example, let's say you know that the error of your
temperature
> >> > > > >> observations
> >> > > > >> follow a normal distribution with a standard deviation
of 0.5
> >> > degrees.
> >> > > > You
> >> > > > >> can configure Ensemble-Stat to use this information in
the
> >> > > > >> verification step.
> >> > > > >>
> >> > > > >> When obs error parameters are supplied, Ensemble-Stat
randomly
> >> > > perturbs
> >> > > > >> the
> >> > > > >> ensemble member forecast values within that range prior
to
> >> computing
> >> > > the
> >> > > > >> ensemble mean.
> >> > > > >>
> >> > > > >> The ME column is the mean error you get from the raw
ensemble
> >> member
> >> > > > >> values, with no obs error perturbation applied.
> >> > > > >> The ME_OERR column is the mean error you get from
perturbed
> >> ensemble
> >> > > > >> member
> >> > > > >> values.
> >> > > > >>
> >> > > > >> I would only recommend using this logic if you have some
> >> confidence
> >> > in
> >> > > > >> your
> >> > > > >> estimate of the observation errors.
> >> > > > >>
> >> > > > >> John
> >> > > > >>
> >> > > > >> On Thu, Oct 1, 2020 at 1:59 PM binyu.wang at noaa.gov via
RT <
> >> > > > >> met_help at ucar.edu>
> >> > > > >> wrote:
> >> > > > >>
> >> > > > >> >
> >> > > > >> > <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96913
> >> >
> >> > > > >> >
> >> > > > >> > Thank you. John
> >> > > > >> > There is another thing I want to confirm:
> >> > > > >> > I am wondering what is the difference of ME and
ME_OERR as
> >> lised
> >> > on
> >> > > > the
> >> > > > >> > page
> >> > > > >> >
> https://dtcenter.github.io/MET/Users_Guide/ensemble-stat.html
> >> > > > >> >
> >> > > > >> > eg:
> >> > > > >> > If I have 21 ensemble members, 1 control and 20
perturb
> >> members.
> >> > > Based
> >> > > > >> on
> >> > > > >> > the definition: ME is "The Root Mean Square Error of
the
> >> ensemble
> >> > > mean
> >> > > > >> > (unperturbed or supplied)"
> >> > > > >> > Does that mean only the CONTROL member is used for ME,
and
> the
> >> 20
> >> > > > >> perturb
> >> > > > >> > is only used for ME_OERR?
> >> > > > >> >
> >> > > > >> > Thank you.
> >> > > > >> > Binyu
> >> > > > >> >
> >> > > > >> > On Thu, Oct 1, 2020 at 3:50 PM John Halley Gotway via
RT <
> >> > > > >> > met_help at ucar.edu>
> >> > > > >> > wrote:
> >> > > > >> >
> >> > > > >> > > Binyu,
> >> > > > >> > >
> >> > > > >> > > The wind_thresh and wind_logic options are listed in
the
> >> > > Point-Stat
> >> > > > >> > config
> >> > > > >> > > file simply to provide default values. They are only
> applied
> >> in
> >> > > the
> >> > > > >> > > computation of VL1L2, VAL1L2, and VCNT output line
types.
> If
> >> > > you're
> >> > > > >> not
> >> > > > >> > > processing winds, and therefore not requesting those
output
> >> line
> >> > > > >> types,
> >> > > > >> > > then they are not used.
> >> > > > >> > >
> >> > > > >> > > John
> >> > > > >> > >
> >> > > > >> > > On Thu, Oct 1, 2020 at 12:25 PM binyu.wang at noaa.gov
via
> RT <
> >> > > > >> > > met_help at ucar.edu> wrote:
> >> > > > >> > >
> >> > > > >> > > >
> >> > > > >> > > > Thu Oct 01 12:24:34 2020: Request 96913 was acted
upon.
> >> > > > >> > > > Transaction: Ticket created by binyu.wang at noaa.gov
> >> > > > >> > > > Queue: met_help
> >> > > > >> > > > Subject: wind_thresh
> >> > > > >> > > > Owner: Nobody
> >> > > > >> > > > Requestors: binyu.wang at noaa.gov
> >> > > > >> > > > Status: new
> >> > > > >> > > > Ticket <URL:
> >> > > > >> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96913
> >> > > > >> > >
> >> > > > >> > > >
> >> > > > >> > > >
> >> > > > >> > > > Hello,
> >> > > > >> > > >
> >> > > > >> > > > Based on MET documentation, "wind_thresh" refers
to the
> >> wind
> >> > > speed
> >> > > > >> > > > thresholds. Why is it added to the default
> >> "PointStatConfig"
> >> > > file
> >> > > > >> but
> >> > > > >> > > there
> >> > > > >> > > > is no "wind" variable at all?
> >> > > > >> > > >
> >> > > > >> > > > Here are the lines from the default
"PointStatConfig"
> >> > > > >> > > > wind_thresh = [ NA ];
> >> > > > >> > > > wind_logic = UNION;
> >> > > > >> > > >
> >> > > > >> > > > What are the two lines used for?
> >> > > > >> > > >
> >> > > > >> > > > Thank you.
> >> > > > >> > > > Binyu
> >> > > > >> > > >
> >> > > > >> > > >
> >> > > > >> > >
> >> > > > >> > >
> >> > > > >> >
> >> > > > >> >
> >> > > > >>
> >> > > > >>
> >> > > >
> >> > > >
> >> > >
> >> > >
> >> >
> >> >
> >>
> >>
>
>
------------------------------------------------
Subject: wind_thresh
From: binyu.wang at noaa.gov
Time: Fri Oct 16 14:27:08 2020
Thank you, that is very helpful. But there are two things I want to
know:
1. How to zoom the plot up to a particular region of the world?
2. For the layer argument, eg. if I want to find overvation for wind
at
250mb, is this correct?
% plot_point_obs prepbufr.nc prepbufr.ps -v p250 -obs_var UGRD?
Binyu
On Fri, Oct 16, 2020 at 3:52 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Binyu,
>
> Several tools in MET pre-process point observations and write to a
common
> output NetCDF file format. They are ascii2nc, madis2nc, lidar2nc,
and
> pb2nc. The location of the observations in those NetCDF files can be
> plotted using the plot_point_obs tool. And that lives in the same
directory
> where the other MET executables live.
>
> Here’s the section from the users guide about that:
>
>
>
https://dtcenter.github.io/MET/latest/Users_Guide/plotting.html#plot-
point-obs-usage
>
> And on this page of the online tutorial, we run plot_point_obs on
the
> output of pb2nc:
>
>
> https://dtcenter.org/metplus-practical-session-guide-version-3-
0/session-2-grid-obs/met-tool-pb2nc/pb2nc-tool-output
>
> Hopefully those examples will help get you going.
>
> Thanks
> John
>
> On Fri, Oct 16, 2020 at 1:37 PM binyu.wang at noaa.gov via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96913 >
> >
> > Hello John,
> >
> > I am looking for a tool that can be used to make plots with
prebufr data
> so
> > I can find where there are observations. I happen to find the link
"
> > https://mailman.ucar.edu/pipermail/met_help/2008-June/000275.html"
and
> > seems you already had the tool. Do you mind sharing with me?
> >
> > Thank you very much!
> > Binyu
> >
> > On Mon, Oct 5, 2020 at 1:45 PM Binyu Wang - NOAA Affiliate <
> > binyu.wang at noaa.gov> wrote:
> >
> > > Thank you very much, John.
> > >
> > > Binyu
> > >
> > > On Mon, Oct 5, 2020 at 1:11 PM John Halley Gotway via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > >> Binyu,
> > >>
> > >> To summarize...
> > >>
> > >> - In Point-Stat and Grid-Stat, use "cnt_thresh" in the fcst or
obs
> > >> dictionaries along with "cnt_logic" to filter which matched
pairs are
> > >> included in the CNT, SL1L2, and SAL1L2 line types.
> > >>
> > >> - In Point-Stat and Grid-Stat, use "wind_thresh" in the fcst or
obs
> > >> dictionaries along with "wind_logic" to filter which U/V
matched pairs
> > are
> > >> included in the VCNT, VL1L2, and VAL1L2 line types.
> > >>
> > >> - In Ensemble-Stat, use "obs_thresh" in the obs dictionary to
filter
> > which
> > >> observations are included in the output.
> > >>
> > >> - In Stat-Analysis, use "-fcst_thresh", "-obs_thresh" options
to
> filter
> > >> which input lines are included in the analysis. This is string
> matching
> > to
> > >> select the input based on the contents of the FCST_THRESH and
> OBS_THRESH
> > >> columns.
> > >>
> > >> - In Stat-Analysis, use the "-column_thresh" option to define a
> > filtering
> > >> threshold for a numeric input column. For example, "-line_type
MPR
> > >> -column_thresh FCST gt0" will only use MPR lines where the FCST
column
> > >> contains a value greater than 0.
> > >>
> > >> - Throughout the MET tools, you can use the "censor_thresh" and
> > >> "censor_val" options to "censor" the input data. And censoring
means,
> > >> changing their values. You have been using this to set some of
the
> > values
> > >> to bad data (-9999).
> > >>
> > >> Thanks,
> > >> John
> > >>
> > >> On Fri, Oct 2, 2020 at 1:28 PM binyu.wang at noaa.gov via RT <
> > >> met_help at ucar.edu>
> > >> wrote:
> > >>
> > >> >
> > >> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96913
>
> > >> >
> > >> > Thank you, John. Just to confirm "fcst_thresh" is the
> STAT-Analysis
> > >> tool
> > >> > only, how about "obs_thresh"? Obs_thresh can only be used in
"
> > >> > STAT-Analysis" and "Ensemble_stat"?
> > >> >
> > >> > Binyu
> > >> >
> > >> >
> > >> > On Fri, Oct 2, 2020 at 11:39 AM John Halley Gotway via RT <
> > >> > met_help at ucar.edu>
> > >> > wrote:
> > >> >
> > >> > > Binyu,
> > >> > >
> > >> > > No, that is not correct.
> > >> > >
> > >> > > The only tool that supports the use of "fcst_thresh" is the
> > >> STAT-Analysis
> > >> > > tool, which is used to summarize and aggregate results
across one
> or
> > >> more
> > >> > > runs of the other "stat" tools.
> > >> > >
> > >> > > I suspect you're actually referring to the "cnt_thresh"
filtering
> > >> option
> > >> > in
> > >> > > Point-Stat and Grid-Stat. What you're trying to do is
called
> > >> "conditional
> > >> > > verification"... where you compute statistics but only
using
> > forecast
> > >> or
> > >> > > observation values that fall within a specific range.
> > >> > >
> > >> > > In Point-Stat and Grid-Stat, you can set "cnt_thresh = [
NA, >0.1,
> > >> >0.2,
> > >> > > >0.3 ];"
> > >> > > And instead of getting 1 output CNT line, you'd get 4:
> > >> > > - One for all input values since cnt_thresh = NA; results
in all
> > pairs
> > >> > > being used.
> > >> > > - One only for values >0.1... one for >0.2... and one for
>0.3.
> > >> > > We called it "cnt_thresh" because it controls which points
are
> used
> > >> when
> > >> > > applying continuous verification methods. It affects the
output of
> > the
> > >> > CNT,
> > >> > > SL1L2, and SAL1L2 line types.
> > >> > >
> > >> > > You're trying to do the same thing in Ensemble-Stat, but it
has no
> > >> > existing
> > >> > > option to filter the pairs in this way. So instead, you're
using
> > data
> > >> > > censoring logic (censor_thresh and censor_val) to
approximate this
> > >> logic.
> > >> > >
> > >> > > Thanks,
> > >> > > John
> > >> > >
> > >> > > On Thu, Oct 1, 2020 at 9:47 PM binyu.wang at noaa.gov via RT <
> > >> > > met_help at ucar.edu>
> > >> > > wrote:
> > >> > >
> > >> > > >
> > >> > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96913 >
> > >> > > >
> > >> > > > Actually, this remind me another thing:
> > >> > > >
> > >> > > > When you look at the config file in my previous email,
you can
> see
> > >> the
> > >> > > two
> > >> > > > lines below:
> > >> > > > censor_thresh=[<0.1];
> > >> > > > censor_val=[-9999]; desc="GE0.1";
> > >> > > > Which I learned from you, but it is for ensemble
verification.
> So
> > >> the
> > >> > > > question is: I have to use the above two lines for
ensemble, but
> > if
> > >> it
> > >> > is
> > >> > > > not ensemble, I can just use "fcst_thresh=[>0.1]" is
that
> > correct?
> > >> > > >
> > >> > > > Thank you.
> > >> > > > Binyu
> > >> > > >
> > >> > > >
> > >> > > > On Thu, Oct 1, 2020 at 5:17 PM Binyu Wang - NOAA
Affiliate <
> > >> > > > binyu.wang at noaa.gov> wrote:
> > >> > > >
> > >> > > > > Got it. So the "unperturbed " is not referred to the
control
> > >> mmber.
> > >> > > > > Since I don't have any obs. related error information,
I
> should
> > >> just
> > >> > > use
> > >> > > > ME
> > >> > > > > (not MET_OERR), as indicated in the formula below, f is
the
> mean
> > >> of
> > >> > all
> > >> > > > the
> > >> > > > > ensemble members.
> > >> > > > > [image: image.png]
> > >> > > > >
> > >> > > > > If you look at the config file at:
> > >> > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
>
/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/MET/grid2grid/verf_met/parm/verf_g2g_grid_stat_regn_config_Raikoke
> > >> > > > > If I want to get ME (or RMSE) using a different
threshold,
> like
> > >> both
> > >> > > FCST
> > >> > > > > and OBS greater than 0.1 (or 0.01, 0r 1), is there any
easy
> way
> > >> to do
> > >> > > > that?
> > >> > > > > Can I just set censor_thresh = [0.1] once and it
will apply
> > for
> > >> > both
> > >> > > > > obs. and FCST or I have to set that twice separately
under
> both
> > >> FCST
> > >> > > and
> > >> > > > > obs? Do I have to change the config each time for a
different
> > >> > threshold
> > >> > > > or
> > >> > > > > is there any way I can get ME for a different threshold
with
> > only
> > >> > once
> > >> > > > > config file?
> > >> > > > >
> > >> > > > > Thank you
> > >> > > > > Binyu
> > >> > > > >
> > >> > > > >
> > >> > > > >
> > >> > > > > On Thu, Oct 1, 2020 at 4:20 PM John Halley Gotway via
RT <
> > >> > > > > met_help at ucar.edu> wrote:
> > >> > > > >
> > >> > > > >> Binyu,
> > >> > > > >>
> > >> > > > >> I see you have a question about the ME and ME_OERR
columns in
> > the
> > >> > ECNT
> > >> > > > >> line
> > >> > > > >> type, described here:
> > >> > > > >>
> > >> https://dtcenter.github.io/MET/Users_Guide/ensemble-
stat.html#id2
> > >> > > > >>
> > >> > > > >> Ensemble-Stat includes the ability to account for
observation
> > >> > > > uncertainty.
> > >> > > > >> This logic is based on some assumption about the
distribution
> > of
> > >> the
> > >> > > > >> errors
> > >> > > > >> in your observation values, such as perhaps a known
> instrument
> > >> > error.
> > >> > > > For
> > >> > > > >> example, let's say you know that the error of your
> temperature
> > >> > > > >> observations
> > >> > > > >> follow a normal distribution with a standard deviation
of 0.5
> > >> > degrees.
> > >> > > > You
> > >> > > > >> can configure Ensemble-Stat to use this information in
the
> > >> > > > >> verification step.
> > >> > > > >>
> > >> > > > >> When obs error parameters are supplied, Ensemble-Stat
> randomly
> > >> > > perturbs
> > >> > > > >> the
> > >> > > > >> ensemble member forecast values within that range
prior to
> > >> computing
> > >> > > the
> > >> > > > >> ensemble mean.
> > >> > > > >>
> > >> > > > >> The ME column is the mean error you get from the raw
ensemble
> > >> member
> > >> > > > >> values, with no obs error perturbation applied.
> > >> > > > >> The ME_OERR column is the mean error you get from
perturbed
> > >> ensemble
> > >> > > > >> member
> > >> > > > >> values.
> > >> > > > >>
> > >> > > > >> I would only recommend using this logic if you have
some
> > >> confidence
> > >> > in
> > >> > > > >> your
> > >> > > > >> estimate of the observation errors.
> > >> > > > >>
> > >> > > > >> John
> > >> > > > >>
> > >> > > > >> On Thu, Oct 1, 2020 at 1:59 PM binyu.wang at noaa.gov via
RT <
> > >> > > > >> met_help at ucar.edu>
> > >> > > > >> wrote:
> > >> > > > >>
> > >> > > > >> >
> > >> > > > >> > <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96913
> > >> >
> > >> > > > >> >
> > >> > > > >> > Thank you. John
> > >> > > > >> > There is another thing I want to confirm:
> > >> > > > >> > I am wondering what is the difference of ME and
ME_OERR as
> > >> lised
> > >> > on
> > >> > > > the
> > >> > > > >> > page
> > >> > > > >> >
> > https://dtcenter.github.io/MET/Users_Guide/ensemble-stat.html
> > >> > > > >> >
> > >> > > > >> > eg:
> > >> > > > >> > If I have 21 ensemble members, 1 control and 20
perturb
> > >> members.
> > >> > > Based
> > >> > > > >> on
> > >> > > > >> > the definition: ME is "The Root Mean Square Error of
the
> > >> ensemble
> > >> > > mean
> > >> > > > >> > (unperturbed or supplied)"
> > >> > > > >> > Does that mean only the CONTROL member is used for
ME, and
> > the
> > >> 20
> > >> > > > >> perturb
> > >> > > > >> > is only used for ME_OERR?
> > >> > > > >> >
> > >> > > > >> > Thank you.
> > >> > > > >> > Binyu
> > >> > > > >> >
> > >> > > > >> > On Thu, Oct 1, 2020 at 3:50 PM John Halley Gotway
via RT <
> > >> > > > >> > met_help at ucar.edu>
> > >> > > > >> > wrote:
> > >> > > > >> >
> > >> > > > >> > > Binyu,
> > >> > > > >> > >
> > >> > > > >> > > The wind_thresh and wind_logic options are listed
in the
> > >> > > Point-Stat
> > >> > > > >> > config
> > >> > > > >> > > file simply to provide default values. They are
only
> > applied
> > >> in
> > >> > > the
> > >> > > > >> > > computation of VL1L2, VAL1L2, and VCNT output line
types.
> > If
> > >> > > you're
> > >> > > > >> not
> > >> > > > >> > > processing winds, and therefore not requesting
those
> output
> > >> line
> > >> > > > >> types,
> > >> > > > >> > > then they are not used.
> > >> > > > >> > >
> > >> > > > >> > > John
> > >> > > > >> > >
> > >> > > > >> > > On Thu, Oct 1, 2020 at 12:25 PM
binyu.wang at noaa.gov via
> > RT <
> > >> > > > >> > > met_help at ucar.edu> wrote:
> > >> > > > >> > >
> > >> > > > >> > > >
> > >> > > > >> > > > Thu Oct 01 12:24:34 2020: Request 96913 was
acted upon.
> > >> > > > >> > > > Transaction: Ticket created by
binyu.wang at noaa.gov
> > >> > > > >> > > > Queue: met_help
> > >> > > > >> > > > Subject: wind_thresh
> > >> > > > >> > > > Owner: Nobody
> > >> > > > >> > > > Requestors: binyu.wang at noaa.gov
> > >> > > > >> > > > Status: new
> > >> > > > >> > > > Ticket <URL:
> > >> > > > >>
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96913
> > >> > > > >> > >
> > >> > > > >> > > >
> > >> > > > >> > > >
> > >> > > > >> > > > Hello,
> > >> > > > >> > > >
> > >> > > > >> > > > Based on MET documentation, "wind_thresh" refers
to the
> > >> wind
> > >> > > speed
> > >> > > > >> > > > thresholds. Why is it added to the default
> > >> "PointStatConfig"
> > >> > > file
> > >> > > > >> but
> > >> > > > >> > > there
> > >> > > > >> > > > is no "wind" variable at all?
> > >> > > > >> > > >
> > >> > > > >> > > > Here are the lines from the default
"PointStatConfig"
> > >> > > > >> > > > wind_thresh = [ NA ];
> > >> > > > >> > > > wind_logic = UNION;
> > >> > > > >> > > >
> > >> > > > >> > > > What are the two lines used for?
> > >> > > > >> > > >
> > >> > > > >> > > > Thank you.
> > >> > > > >> > > > Binyu
> > >> > > > >> > > >
> > >> > > > >> > > >
> > >> > > > >> > >
> > >> > > > >> > >
> > >> > > > >> >
> > >> > > > >> >
> > >> > > > >>
> > >> > > > >>
> > >> > > >
> > >> > > >
> > >> > >
> > >> > >
> > >> >
> > >> >
> > >>
> > >>
> >
> >
>
>
------------------------------------------------
Subject: wind_thresh
From: John Halley Gotway
Time: Fri Oct 16 14:55:46 2020
Binyu,
By default it plots points on the full earth. If you supply the
“-data_file” command line argument with the path to a gridded data
file
that MET can read... like a GRIB file... it’ll plot the points on that
grid
instead.
While you can filter by message type and variable name, I don’t think
there’s currently a way to filter points by their vertical level. The
“-v”
command line option is used by all of the MET to to specify the
logging
level... or verbosity.
John
On Fri, Oct 16, 2020 at 2:27 PM binyu.wang at noaa.gov via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96913 >
>
> Thank you, that is very helpful. But there are two things I want to
know:
> 1. How to zoom the plot up to a particular region of the world?
> 2. For the layer argument, eg. if I want to find overvation for wind
at
> 250mb, is this correct?
> % plot_point_obs prepbufr.nc prepbufr.ps -v p250 -obs_var UGRD?
>
> Binyu
>
> On Fri, Oct 16, 2020 at 3:52 PM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > Binyu,
> >
> > Several tools in MET pre-process point observations and write to a
common
> > output NetCDF file format. They are ascii2nc, madis2nc, lidar2nc,
and
> > pb2nc. The location of the observations in those NetCDF files can
be
> > plotted using the plot_point_obs tool. And that lives in the same
> directory
> > where the other MET executables live.
> >
> > Here’s the section from the users guide about that:
> >
> >
> >
>
https://dtcenter.github.io/MET/latest/Users_Guide/plotting.html#plot-
point-obs-usage
> >
> > And on this page of the online tutorial, we run plot_point_obs on
the
> > output of pb2nc:
> >
> >
> >
> https://dtcenter.org/metplus-practical-session-guide-version-3-
0/session-2-grid-obs/met-tool-pb2nc/pb2nc-tool-output
> >
> > Hopefully those examples will help get you going.
> >
> > Thanks
> > John
> >
> > On Fri, Oct 16, 2020 at 1:37 PM binyu.wang at noaa.gov via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96913 >
> > >
> > > Hello John,
> > >
> > > I am looking for a tool that can be used to make plots with
prebufr
> data
> > so
> > > I can find where there are observations. I happen to find the
link "
> > > https://mailman.ucar.edu/pipermail/met_help/2008-
June/000275.html" and
> > > seems you already had the tool. Do you mind sharing with me?
> > >
> > > Thank you very much!
> > > Binyu
> > >
> > > On Mon, Oct 5, 2020 at 1:45 PM Binyu Wang - NOAA Affiliate <
> > > binyu.wang at noaa.gov> wrote:
> > >
> > > > Thank you very much, John.
> > > >
> > > > Binyu
> > > >
> > > > On Mon, Oct 5, 2020 at 1:11 PM John Halley Gotway via RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > >> Binyu,
> > > >>
> > > >> To summarize...
> > > >>
> > > >> - In Point-Stat and Grid-Stat, use "cnt_thresh" in the fcst
or obs
> > > >> dictionaries along with "cnt_logic" to filter which matched
pairs
> are
> > > >> included in the CNT, SL1L2, and SAL1L2 line types.
> > > >>
> > > >> - In Point-Stat and Grid-Stat, use "wind_thresh" in the fcst
or obs
> > > >> dictionaries along with "wind_logic" to filter which U/V
matched
> pairs
> > > are
> > > >> included in the VCNT, VL1L2, and VAL1L2 line types.
> > > >>
> > > >> - In Ensemble-Stat, use "obs_thresh" in the obs dictionary to
filter
> > > which
> > > >> observations are included in the output.
> > > >>
> > > >> - In Stat-Analysis, use "-fcst_thresh", "-obs_thresh" options
to
> > filter
> > > >> which input lines are included in the analysis. This is
string
> > matching
> > > to
> > > >> select the input based on the contents of the FCST_THRESH and
> > OBS_THRESH
> > > >> columns.
> > > >>
> > > >> - In Stat-Analysis, use the "-column_thresh" option to define
a
> > > filtering
> > > >> threshold for a numeric input column. For example, "-
line_type MPR
> > > >> -column_thresh FCST gt0" will only use MPR lines where the
FCST
> column
> > > >> contains a value greater than 0.
> > > >>
> > > >> - Throughout the MET tools, you can use the "censor_thresh"
and
> > > >> "censor_val" options to "censor" the input data. And
censoring
> means,
> > > >> changing their values. You have been using this to set some
of the
> > > values
> > > >> to bad data (-9999).
> > > >>
> > > >> Thanks,
> > > >> John
> > > >>
> > > >> On Fri, Oct 2, 2020 at 1:28 PM binyu.wang at noaa.gov via RT <
> > > >> met_help at ucar.edu>
> > > >> wrote:
> > > >>
> > > >> >
> > > >> > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96913 >
> > > >> >
> > > >> > Thank you, John. Just to confirm "fcst_thresh" is the
> > STAT-Analysis
> > > >> tool
> > > >> > only, how about "obs_thresh"? Obs_thresh can only be used
in "
> > > >> > STAT-Analysis" and "Ensemble_stat"?
> > > >> >
> > > >> > Binyu
> > > >> >
> > > >> >
> > > >> > On Fri, Oct 2, 2020 at 11:39 AM John Halley Gotway via RT <
> > > >> > met_help at ucar.edu>
> > > >> > wrote:
> > > >> >
> > > >> > > Binyu,
> > > >> > >
> > > >> > > No, that is not correct.
> > > >> > >
> > > >> > > The only tool that supports the use of "fcst_thresh" is
the
> > > >> STAT-Analysis
> > > >> > > tool, which is used to summarize and aggregate results
across
> one
> > or
> > > >> more
> > > >> > > runs of the other "stat" tools.
> > > >> > >
> > > >> > > I suspect you're actually referring to the "cnt_thresh"
> filtering
> > > >> option
> > > >> > in
> > > >> > > Point-Stat and Grid-Stat. What you're trying to do is
called
> > > >> "conditional
> > > >> > > verification"... where you compute statistics but only
using
> > > forecast
> > > >> or
> > > >> > > observation values that fall within a specific range.
> > > >> > >
> > > >> > > In Point-Stat and Grid-Stat, you can set "cnt_thresh = [
NA,
> >0.1,
> > > >> >0.2,
> > > >> > > >0.3 ];"
> > > >> > > And instead of getting 1 output CNT line, you'd get 4:
> > > >> > > - One for all input values since cnt_thresh = NA; results
in all
> > > pairs
> > > >> > > being used.
> > > >> > > - One only for values >0.1... one for >0.2... and one for
>0.3.
> > > >> > > We called it "cnt_thresh" because it controls which
points are
> > used
> > > >> when
> > > >> > > applying continuous verification methods. It affects the
output
> of
> > > the
> > > >> > CNT,
> > > >> > > SL1L2, and SAL1L2 line types.
> > > >> > >
> > > >> > > You're trying to do the same thing in Ensemble-Stat, but
it has
> no
> > > >> > existing
> > > >> > > option to filter the pairs in this way. So instead,
you're using
> > > data
> > > >> > > censoring logic (censor_thresh and censor_val) to
approximate
> this
> > > >> logic.
> > > >> > >
> > > >> > > Thanks,
> > > >> > > John
> > > >> > >
> > > >> > > On Thu, Oct 1, 2020 at 9:47 PM binyu.wang at noaa.gov via RT
<
> > > >> > > met_help at ucar.edu>
> > > >> > > wrote:
> > > >> > >
> > > >> > > >
> > > >> > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96913
> >
> > > >> > > >
> > > >> > > > Actually, this remind me another thing:
> > > >> > > >
> > > >> > > > When you look at the config file in my previous email,
you can
> > see
> > > >> the
> > > >> > > two
> > > >> > > > lines below:
> > > >> > > > censor_thresh=[<0.1];
> > > >> > > > censor_val=[-9999]; desc="GE0.1";
> > > >> > > > Which I learned from you, but it is for ensemble
verification.
> > So
> > > >> the
> > > >> > > > question is: I have to use the above two lines for
ensemble,
> but
> > > if
> > > >> it
> > > >> > is
> > > >> > > > not ensemble, I can just use "fcst_thresh=[>0.1]" is
that
> > > correct?
> > > >> > > >
> > > >> > > > Thank you.
> > > >> > > > Binyu
> > > >> > > >
> > > >> > > >
> > > >> > > > On Thu, Oct 1, 2020 at 5:17 PM Binyu Wang - NOAA
Affiliate <
> > > >> > > > binyu.wang at noaa.gov> wrote:
> > > >> > > >
> > > >> > > > > Got it. So the "unperturbed " is not referred to the
> control
> > > >> mmber.
> > > >> > > > > Since I don't have any obs. related error
information, I
> > should
> > > >> just
> > > >> > > use
> > > >> > > > ME
> > > >> > > > > (not MET_OERR), as indicated in the formula below, f
is the
> > mean
> > > >> of
> > > >> > all
> > > >> > > > the
> > > >> > > > > ensemble members.
> > > >> > > > > [image: image.png]
> > > >> > > > >
> > > >> > > > > If you look at the config file at:
> > > >> > > > >
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/MET/grid2grid/verf_met/parm/verf_g2g_grid_stat_regn_config_Raikoke
> > > >> > > > > If I want to get ME (or RMSE) using a different
threshold,
> > like
> > > >> both
> > > >> > > FCST
> > > >> > > > > and OBS greater than 0.1 (or 0.01, 0r 1), is there
any easy
> > way
> > > >> to do
> > > >> > > > that?
> > > >> > > > > Can I just set censor_thresh = [0.1] once and it
will
> apply
> > > for
> > > >> > both
> > > >> > > > > obs. and FCST or I have to set that twice separately
under
> > both
> > > >> FCST
> > > >> > > and
> > > >> > > > > obs? Do I have to change the config each time for a
> different
> > > >> > threshold
> > > >> > > > or
> > > >> > > > > is there any way I can get ME for a different
threshold with
> > > only
> > > >> > once
> > > >> > > > > config file?
> > > >> > > > >
> > > >> > > > > Thank you
> > > >> > > > > Binyu
> > > >> > > > >
> > > >> > > > >
> > > >> > > > >
> > > >> > > > > On Thu, Oct 1, 2020 at 4:20 PM John Halley Gotway via
RT <
> > > >> > > > > met_help at ucar.edu> wrote:
> > > >> > > > >
> > > >> > > > >> Binyu,
> > > >> > > > >>
> > > >> > > > >> I see you have a question about the ME and ME_OERR
columns
> in
> > > the
> > > >> > ECNT
> > > >> > > > >> line
> > > >> > > > >> type, described here:
> > > >> > > > >>
> > > >> https://dtcenter.github.io/MET/Users_Guide/ensemble-
stat.html#id2
> > > >> > > > >>
> > > >> > > > >> Ensemble-Stat includes the ability to account for
> observation
> > > >> > > > uncertainty.
> > > >> > > > >> This logic is based on some assumption about the
> distribution
> > > of
> > > >> the
> > > >> > > > >> errors
> > > >> > > > >> in your observation values, such as perhaps a known
> > instrument
> > > >> > error.
> > > >> > > > For
> > > >> > > > >> example, let's say you know that the error of your
> > temperature
> > > >> > > > >> observations
> > > >> > > > >> follow a normal distribution with a standard
deviation of
> 0.5
> > > >> > degrees.
> > > >> > > > You
> > > >> > > > >> can configure Ensemble-Stat to use this information
in the
> > > >> > > > >> verification step.
> > > >> > > > >>
> > > >> > > > >> When obs error parameters are supplied, Ensemble-
Stat
> > randomly
> > > >> > > perturbs
> > > >> > > > >> the
> > > >> > > > >> ensemble member forecast values within that range
prior to
> > > >> computing
> > > >> > > the
> > > >> > > > >> ensemble mean.
> > > >> > > > >>
> > > >> > > > >> The ME column is the mean error you get from the raw
> ensemble
> > > >> member
> > > >> > > > >> values, with no obs error perturbation applied.
> > > >> > > > >> The ME_OERR column is the mean error you get from
perturbed
> > > >> ensemble
> > > >> > > > >> member
> > > >> > > > >> values.
> > > >> > > > >>
> > > >> > > > >> I would only recommend using this logic if you have
some
> > > >> confidence
> > > >> > in
> > > >> > > > >> your
> > > >> > > > >> estimate of the observation errors.
> > > >> > > > >>
> > > >> > > > >> John
> > > >> > > > >>
> > > >> > > > >> On Thu, Oct 1, 2020 at 1:59 PM binyu.wang at noaa.gov
via RT
> <
> > > >> > > > >> met_help at ucar.edu>
> > > >> > > > >> wrote:
> > > >> > > > >>
> > > >> > > > >> >
> > > >> > > > >> > <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96913
> > > >> >
> > > >> > > > >> >
> > > >> > > > >> > Thank you. John
> > > >> > > > >> > There is another thing I want to confirm:
> > > >> > > > >> > I am wondering what is the difference of ME and
ME_OERR
> as
> > > >> lised
> > > >> > on
> > > >> > > > the
> > > >> > > > >> > page
> > > >> > > > >> >
> > > https://dtcenter.github.io/MET/Users_Guide/ensemble-stat.html
> > > >> > > > >> >
> > > >> > > > >> > eg:
> > > >> > > > >> > If I have 21 ensemble members, 1 control and 20
perturb
> > > >> members.
> > > >> > > Based
> > > >> > > > >> on
> > > >> > > > >> > the definition: ME is "The Root Mean Square Error
of the
> > > >> ensemble
> > > >> > > mean
> > > >> > > > >> > (unperturbed or supplied)"
> > > >> > > > >> > Does that mean only the CONTROL member is used for
ME,
> and
> > > the
> > > >> 20
> > > >> > > > >> perturb
> > > >> > > > >> > is only used for ME_OERR?
> > > >> > > > >> >
> > > >> > > > >> > Thank you.
> > > >> > > > >> > Binyu
> > > >> > > > >> >
> > > >> > > > >> > On Thu, Oct 1, 2020 at 3:50 PM John Halley Gotway
via RT
> <
> > > >> > > > >> > met_help at ucar.edu>
> > > >> > > > >> > wrote:
> > > >> > > > >> >
> > > >> > > > >> > > Binyu,
> > > >> > > > >> > >
> > > >> > > > >> > > The wind_thresh and wind_logic options are
listed in
> the
> > > >> > > Point-Stat
> > > >> > > > >> > config
> > > >> > > > >> > > file simply to provide default values. They are
only
> > > applied
> > > >> in
> > > >> > > the
> > > >> > > > >> > > computation of VL1L2, VAL1L2, and VCNT output
line
> types.
> > > If
> > > >> > > you're
> > > >> > > > >> not
> > > >> > > > >> > > processing winds, and therefore not requesting
those
> > output
> > > >> line
> > > >> > > > >> types,
> > > >> > > > >> > > then they are not used.
> > > >> > > > >> > >
> > > >> > > > >> > > John
> > > >> > > > >> > >
> > > >> > > > >> > > On Thu, Oct 1, 2020 at 12:25 PM
binyu.wang at noaa.gov
> via
> > > RT <
> > > >> > > > >> > > met_help at ucar.edu> wrote:
> > > >> > > > >> > >
> > > >> > > > >> > > >
> > > >> > > > >> > > > Thu Oct 01 12:24:34 2020: Request 96913 was
acted
> upon.
> > > >> > > > >> > > > Transaction: Ticket created by
binyu.wang at noaa.gov
> > > >> > > > >> > > > Queue: met_help
> > > >> > > > >> > > > Subject: wind_thresh
> > > >> > > > >> > > > Owner: Nobody
> > > >> > > > >> > > > Requestors: binyu.wang at noaa.gov
> > > >> > > > >> > > > Status: new
> > > >> > > > >> > > > Ticket <URL:
> > > >> > > > >>
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96913
> > > >> > > > >> > >
> > > >> > > > >> > > >
> > > >> > > > >> > > >
> > > >> > > > >> > > > Hello,
> > > >> > > > >> > > >
> > > >> > > > >> > > > Based on MET documentation, "wind_thresh"
refers to
> the
> > > >> wind
> > > >> > > speed
> > > >> > > > >> > > > thresholds. Why is it added to the default
> > > >> "PointStatConfig"
> > > >> > > file
> > > >> > > > >> but
> > > >> > > > >> > > there
> > > >> > > > >> > > > is no "wind" variable at all?
> > > >> > > > >> > > >
> > > >> > > > >> > > > Here are the lines from the default
> "PointStatConfig"
> > > >> > > > >> > > > wind_thresh = [ NA ];
> > > >> > > > >> > > > wind_logic = UNION;
> > > >> > > > >> > > >
> > > >> > > > >> > > > What are the two lines used for?
> > > >> > > > >> > > >
> > > >> > > > >> > > > Thank you.
> > > >> > > > >> > > > Binyu
> > > >> > > > >> > > >
> > > >> > > > >> > > >
> > > >> > > > >> > >
> > > >> > > > >> > >
> > > >> > > > >> >
> > > >> > > > >> >
> > > >> > > > >>
> > > >> > > > >>
> > > >> > > >
> > > >> > > >
> > > >> > >
> > > >> > >
> > > >> >
> > > >> >
> > > >>
> > > >>
> > >
> > >
> >
> >
>
>
------------------------------------------------
Subject: wind_thresh
From: binyu.wang at noaa.gov
Time: Fri Oct 16 15:30:32 2020
Got it, thank you.
Binyu
On Fri, Oct 16, 2020 at 4:55 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Binyu,
>
> By default it plots points on the full earth. If you supply the
> “-data_file” command line argument with the path to a gridded data
file
> that MET can read... like a GRIB file... it’ll plot the points on
that grid
> instead.
>
> While you can filter by message type and variable name, I don’t
think
> there’s currently a way to filter points by their vertical level.
The “-v”
> command line option is used by all of the MET to to specify the
logging
> level... or verbosity.
>
> John
>
> On Fri, Oct 16, 2020 at 2:27 PM binyu.wang at noaa.gov via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96913 >
> >
> > Thank you, that is very helpful. But there are two things I want
to know:
> > 1. How to zoom the plot up to a particular region of the world?
> > 2. For the layer argument, eg. if I want to find overvation for
wind at
> > 250mb, is this correct?
> > % plot_point_obs prepbufr.nc prepbufr.ps -v p250 -obs_var UGRD?
> >
> > Binyu
> >
> > On Fri, Oct 16, 2020 at 3:52 PM John Halley Gotway via RT <
> > met_help at ucar.edu>
> > wrote:
> >
> > > Binyu,
> > >
> > > Several tools in MET pre-process point observations and write to
a
> common
> > > output NetCDF file format. They are ascii2nc, madis2nc,
lidar2nc, and
> > > pb2nc. The location of the observations in those NetCDF files
can be
> > > plotted using the plot_point_obs tool. And that lives in the
same
> > directory
> > > where the other MET executables live.
> > >
> > > Here’s the section from the users guide about that:
> > >
> > >
> > >
> >
>
https://dtcenter.github.io/MET/latest/Users_Guide/plotting.html#plot-
point-obs-usage
> > >
> > > And on this page of the online tutorial, we run plot_point_obs
on the
> > > output of pb2nc:
> > >
> > >
> > >
> >
> https://dtcenter.org/metplus-practical-session-guide-version-3-
0/session-2-grid-obs/met-tool-pb2nc/pb2nc-tool-output
> > >
> > > Hopefully those examples will help get you going.
> > >
> > > Thanks
> > > John
> > >
> > > On Fri, Oct 16, 2020 at 1:37 PM binyu.wang at noaa.gov via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96913
>
> > > >
> > > > Hello John,
> > > >
> > > > I am looking for a tool that can be used to make plots with
prebufr
> > data
> > > so
> > > > I can find where there are observations. I happen to find the
link "
> > > > https://mailman.ucar.edu/pipermail/met_help/2008-
June/000275.html"
> and
> > > > seems you already had the tool. Do you mind sharing with me?
> > > >
> > > > Thank you very much!
> > > > Binyu
> > > >
> > > > On Mon, Oct 5, 2020 at 1:45 PM Binyu Wang - NOAA Affiliate <
> > > > binyu.wang at noaa.gov> wrote:
> > > >
> > > > > Thank you very much, John.
> > > > >
> > > > > Binyu
> > > > >
> > > > > On Mon, Oct 5, 2020 at 1:11 PM John Halley Gotway via RT <
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > >> Binyu,
> > > > >>
> > > > >> To summarize...
> > > > >>
> > > > >> - In Point-Stat and Grid-Stat, use "cnt_thresh" in the fcst
or obs
> > > > >> dictionaries along with "cnt_logic" to filter which matched
pairs
> > are
> > > > >> included in the CNT, SL1L2, and SAL1L2 line types.
> > > > >>
> > > > >> - In Point-Stat and Grid-Stat, use "wind_thresh" in the
fcst or
> obs
> > > > >> dictionaries along with "wind_logic" to filter which U/V
matched
> > pairs
> > > > are
> > > > >> included in the VCNT, VL1L2, and VAL1L2 line types.
> > > > >>
> > > > >> - In Ensemble-Stat, use "obs_thresh" in the obs dictionary
to
> filter
> > > > which
> > > > >> observations are included in the output.
> > > > >>
> > > > >> - In Stat-Analysis, use "-fcst_thresh", "-obs_thresh"
options to
> > > filter
> > > > >> which input lines are included in the analysis. This is
string
> > > matching
> > > > to
> > > > >> select the input based on the contents of the FCST_THRESH
and
> > > OBS_THRESH
> > > > >> columns.
> > > > >>
> > > > >> - In Stat-Analysis, use the "-column_thresh" option to
define a
> > > > filtering
> > > > >> threshold for a numeric input column. For example, "-
line_type MPR
> > > > >> -column_thresh FCST gt0" will only use MPR lines where the
FCST
> > column
> > > > >> contains a value greater than 0.
> > > > >>
> > > > >> - Throughout the MET tools, you can use the "censor_thresh"
and
> > > > >> "censor_val" options to "censor" the input data. And
censoring
> > means,
> > > > >> changing their values. You have been using this to set some
of the
> > > > values
> > > > >> to bad data (-9999).
> > > > >>
> > > > >> Thanks,
> > > > >> John
> > > > >>
> > > > >> On Fri, Oct 2, 2020 at 1:28 PM binyu.wang at noaa.gov via RT <
> > > > >> met_help at ucar.edu>
> > > > >> wrote:
> > > > >>
> > > > >> >
> > > > >> > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96913 >
> > > > >> >
> > > > >> > Thank you, John. Just to confirm "fcst_thresh" is the
> > > STAT-Analysis
> > > > >> tool
> > > > >> > only, how about "obs_thresh"? Obs_thresh can only be
used in "
> > > > >> > STAT-Analysis" and "Ensemble_stat"?
> > > > >> >
> > > > >> > Binyu
> > > > >> >
> > > > >> >
> > > > >> > On Fri, Oct 2, 2020 at 11:39 AM John Halley Gotway via RT
<
> > > > >> > met_help at ucar.edu>
> > > > >> > wrote:
> > > > >> >
> > > > >> > > Binyu,
> > > > >> > >
> > > > >> > > No, that is not correct.
> > > > >> > >
> > > > >> > > The only tool that supports the use of "fcst_thresh" is
the
> > > > >> STAT-Analysis
> > > > >> > > tool, which is used to summarize and aggregate results
across
> > one
> > > or
> > > > >> more
> > > > >> > > runs of the other "stat" tools.
> > > > >> > >
> > > > >> > > I suspect you're actually referring to the "cnt_thresh"
> > filtering
> > > > >> option
> > > > >> > in
> > > > >> > > Point-Stat and Grid-Stat. What you're trying to do is
called
> > > > >> "conditional
> > > > >> > > verification"... where you compute statistics but only
using
> > > > forecast
> > > > >> or
> > > > >> > > observation values that fall within a specific range.
> > > > >> > >
> > > > >> > > In Point-Stat and Grid-Stat, you can set "cnt_thresh =
[ NA,
> > >0.1,
> > > > >> >0.2,
> > > > >> > > >0.3 ];"
> > > > >> > > And instead of getting 1 output CNT line, you'd get 4:
> > > > >> > > - One for all input values since cnt_thresh = NA;
results in
> all
> > > > pairs
> > > > >> > > being used.
> > > > >> > > - One only for values >0.1... one for >0.2... and one
for
> >0.3.
> > > > >> > > We called it "cnt_thresh" because it controls which
points are
> > > used
> > > > >> when
> > > > >> > > applying continuous verification methods. It affects
the
> output
> > of
> > > > the
> > > > >> > CNT,
> > > > >> > > SL1L2, and SAL1L2 line types.
> > > > >> > >
> > > > >> > > You're trying to do the same thing in Ensemble-Stat,
but it
> has
> > no
> > > > >> > existing
> > > > >> > > option to filter the pairs in this way. So instead,
you're
> using
> > > > data
> > > > >> > > censoring logic (censor_thresh and censor_val) to
approximate
> > this
> > > > >> logic.
> > > > >> > >
> > > > >> > > Thanks,
> > > > >> > > John
> > > > >> > >
> > > > >> > > On Thu, Oct 1, 2020 at 9:47 PM binyu.wang at noaa.gov via
RT <
> > > > >> > > met_help at ucar.edu>
> > > > >> > > wrote:
> > > > >> > >
> > > > >> > > >
> > > > >> > > > <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96913
> > >
> > > > >> > > >
> > > > >> > > > Actually, this remind me another thing:
> > > > >> > > >
> > > > >> > > > When you look at the config file in my previous
email, you
> can
> > > see
> > > > >> the
> > > > >> > > two
> > > > >> > > > lines below:
> > > > >> > > > censor_thresh=[<0.1];
> > > > >> > > > censor_val=[-9999]; desc="GE0.1";
> > > > >> > > > Which I learned from you, but it is for ensemble
> verification.
> > > So
> > > > >> the
> > > > >> > > > question is: I have to use the above two lines for
ensemble,
> > but
> > > > if
> > > > >> it
> > > > >> > is
> > > > >> > > > not ensemble, I can just use "fcst_thresh=[>0.1]"
is that
> > > > correct?
> > > > >> > > >
> > > > >> > > > Thank you.
> > > > >> > > > Binyu
> > > > >> > > >
> > > > >> > > >
> > > > >> > > > On Thu, Oct 1, 2020 at 5:17 PM Binyu Wang - NOAA
Affiliate <
> > > > >> > > > binyu.wang at noaa.gov> wrote:
> > > > >> > > >
> > > > >> > > > > Got it. So the "unperturbed " is not referred to
the
> > control
> > > > >> mmber.
> > > > >> > > > > Since I don't have any obs. related error
information, I
> > > should
> > > > >> just
> > > > >> > > use
> > > > >> > > > ME
> > > > >> > > > > (not MET_OERR), as indicated in the formula below,
f is
> the
> > > mean
> > > > >> of
> > > > >> > all
> > > > >> > > > the
> > > > >> > > > > ensemble members.
> > > > >> > > > > [image: image.png]
> > > > >> > > > >
> > > > >> > > > > If you look at the config file at:
> > > > >> > > > >
> > > > >> > > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Binyu.Wang/MET/grid2grid/verf_met/parm/verf_g2g_grid_stat_regn_config_Raikoke
> > > > >> > > > > If I want to get ME (or RMSE) using a different
threshold,
> > > like
> > > > >> both
> > > > >> > > FCST
> > > > >> > > > > and OBS greater than 0.1 (or 0.01, 0r 1), is there
any
> easy
> > > way
> > > > >> to do
> > > > >> > > > that?
> > > > >> > > > > Can I just set censor_thresh = [0.1] once and it
will
> > apply
> > > > for
> > > > >> > both
> > > > >> > > > > obs. and FCST or I have to set that twice
separately under
> > > both
> > > > >> FCST
> > > > >> > > and
> > > > >> > > > > obs? Do I have to change the config each time for a
> > different
> > > > >> > threshold
> > > > >> > > > or
> > > > >> > > > > is there any way I can get ME for a different
threshold
> with
> > > > only
> > > > >> > once
> > > > >> > > > > config file?
> > > > >> > > > >
> > > > >> > > > > Thank you
> > > > >> > > > > Binyu
> > > > >> > > > >
> > > > >> > > > >
> > > > >> > > > >
> > > > >> > > > > On Thu, Oct 1, 2020 at 4:20 PM John Halley Gotway
via RT <
> > > > >> > > > > met_help at ucar.edu> wrote:
> > > > >> > > > >
> > > > >> > > > >> Binyu,
> > > > >> > > > >>
> > > > >> > > > >> I see you have a question about the ME and ME_OERR
> columns
> > in
> > > > the
> > > > >> > ECNT
> > > > >> > > > >> line
> > > > >> > > > >> type, described here:
> > > > >> > > > >>
> > > > >> https://dtcenter.github.io/MET/Users_Guide/ensemble-
stat.html#id2
> > > > >> > > > >>
> > > > >> > > > >> Ensemble-Stat includes the ability to account for
> > observation
> > > > >> > > > uncertainty.
> > > > >> > > > >> This logic is based on some assumption about the
> > distribution
> > > > of
> > > > >> the
> > > > >> > > > >> errors
> > > > >> > > > >> in your observation values, such as perhaps a
known
> > > instrument
> > > > >> > error.
> > > > >> > > > For
> > > > >> > > > >> example, let's say you know that the error of your
> > > temperature
> > > > >> > > > >> observations
> > > > >> > > > >> follow a normal distribution with a standard
deviation of
> > 0.5
> > > > >> > degrees.
> > > > >> > > > You
> > > > >> > > > >> can configure Ensemble-Stat to use this
information in
> the
> > > > >> > > > >> verification step.
> > > > >> > > > >>
> > > > >> > > > >> When obs error parameters are supplied, Ensemble-
Stat
> > > randomly
> > > > >> > > perturbs
> > > > >> > > > >> the
> > > > >> > > > >> ensemble member forecast values within that range
prior
> to
> > > > >> computing
> > > > >> > > the
> > > > >> > > > >> ensemble mean.
> > > > >> > > > >>
> > > > >> > > > >> The ME column is the mean error you get from the
raw
> > ensemble
> > > > >> member
> > > > >> > > > >> values, with no obs error perturbation applied.
> > > > >> > > > >> The ME_OERR column is the mean error you get from
> perturbed
> > > > >> ensemble
> > > > >> > > > >> member
> > > > >> > > > >> values.
> > > > >> > > > >>
> > > > >> > > > >> I would only recommend using this logic if you
have some
> > > > >> confidence
> > > > >> > in
> > > > >> > > > >> your
> > > > >> > > > >> estimate of the observation errors.
> > > > >> > > > >>
> > > > >> > > > >> John
> > > > >> > > > >>
> > > > >> > > > >> On Thu, Oct 1, 2020 at 1:59 PM binyu.wang at noaa.gov
via
> RT
> > <
> > > > >> > > > >> met_help at ucar.edu>
> > > > >> > > > >> wrote:
> > > > >> > > > >>
> > > > >> > > > >> >
> > > > >> > > > >> > <URL:
> > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96913
> > > > >> >
> > > > >> > > > >> >
> > > > >> > > > >> > Thank you. John
> > > > >> > > > >> > There is another thing I want to confirm:
> > > > >> > > > >> > I am wondering what is the difference of ME and
ME_OERR
> > as
> > > > >> lised
> > > > >> > on
> > > > >> > > > the
> > > > >> > > > >> > page
> > > > >> > > > >> >
> > > > https://dtcenter.github.io/MET/Users_Guide/ensemble-stat.html
> > > > >> > > > >> >
> > > > >> > > > >> > eg:
> > > > >> > > > >> > If I have 21 ensemble members, 1 control and 20
perturb
> > > > >> members.
> > > > >> > > Based
> > > > >> > > > >> on
> > > > >> > > > >> > the definition: ME is "The Root Mean Square
Error of
> the
> > > > >> ensemble
> > > > >> > > mean
> > > > >> > > > >> > (unperturbed or supplied)"
> > > > >> > > > >> > Does that mean only the CONTROL member is used
for ME,
> > and
> > > > the
> > > > >> 20
> > > > >> > > > >> perturb
> > > > >> > > > >> > is only used for ME_OERR?
> > > > >> > > > >> >
> > > > >> > > > >> > Thank you.
> > > > >> > > > >> > Binyu
> > > > >> > > > >> >
> > > > >> > > > >> > On Thu, Oct 1, 2020 at 3:50 PM John Halley
Gotway via
> RT
> > <
> > > > >> > > > >> > met_help at ucar.edu>
> > > > >> > > > >> > wrote:
> > > > >> > > > >> >
> > > > >> > > > >> > > Binyu,
> > > > >> > > > >> > >
> > > > >> > > > >> > > The wind_thresh and wind_logic options are
listed in
> > the
> > > > >> > > Point-Stat
> > > > >> > > > >> > config
> > > > >> > > > >> > > file simply to provide default values. They
are only
> > > > applied
> > > > >> in
> > > > >> > > the
> > > > >> > > > >> > > computation of VL1L2, VAL1L2, and VCNT output
line
> > types.
> > > > If
> > > > >> > > you're
> > > > >> > > > >> not
> > > > >> > > > >> > > processing winds, and therefore not requesting
those
> > > output
> > > > >> line
> > > > >> > > > >> types,
> > > > >> > > > >> > > then they are not used.
> > > > >> > > > >> > >
> > > > >> > > > >> > > John
> > > > >> > > > >> > >
> > > > >> > > > >> > > On Thu, Oct 1, 2020 at 12:25 PM
binyu.wang at noaa.gov
> > via
> > > > RT <
> > > > >> > > > >> > > met_help at ucar.edu> wrote:
> > > > >> > > > >> > >
> > > > >> > > > >> > > >
> > > > >> > > > >> > > > Thu Oct 01 12:24:34 2020: Request 96913 was
acted
> > upon.
> > > > >> > > > >> > > > Transaction: Ticket created by
binyu.wang at noaa.gov
> > > > >> > > > >> > > > Queue: met_help
> > > > >> > > > >> > > > Subject: wind_thresh
> > > > >> > > > >> > > > Owner: Nobody
> > > > >> > > > >> > > > Requestors: binyu.wang at noaa.gov
> > > > >> > > > >> > > > Status: new
> > > > >> > > > >> > > > Ticket <URL:
> > > > >> > > > >>
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96913
> > > > >> > > > >> > >
> > > > >> > > > >> > > >
> > > > >> > > > >> > > >
> > > > >> > > > >> > > > Hello,
> > > > >> > > > >> > > >
> > > > >> > > > >> > > > Based on MET documentation, "wind_thresh"
refers to
> > the
> > > > >> wind
> > > > >> > > speed
> > > > >> > > > >> > > > thresholds. Why is it added to the default
> > > > >> "PointStatConfig"
> > > > >> > > file
> > > > >> > > > >> but
> > > > >> > > > >> > > there
> > > > >> > > > >> > > > is no "wind" variable at all?
> > > > >> > > > >> > > >
> > > > >> > > > >> > > > Here are the lines from the default
> > "PointStatConfig"
> > > > >> > > > >> > > > wind_thresh = [ NA ];
> > > > >> > > > >> > > > wind_logic = UNION;
> > > > >> > > > >> > > >
> > > > >> > > > >> > > > What are the two lines used for?
> > > > >> > > > >> > > >
> > > > >> > > > >> > > > Thank you.
> > > > >> > > > >> > > > Binyu
> > > > >> > > > >> > > >
> > > > >> > > > >> > > >
> > > > >> > > > >> > >
> > > > >> > > > >> > >
> > > > >> > > > >> >
> > > > >> > > > >> >
> > > > >> > > > >>
> > > > >> > > > >>
> > > > >> > > >
> > > > >> > > >
> > > > >> > >
> > > > >> > >
> > > > >> >
> > > > >> >
> > > > >>
> > > > >>
> > > >
> > > >
> > >
> > >
> >
> >
>
>
------------------------------------------------
More information about the Met_help
mailing list