[Met_help] [rt.rap.ucar.edu #78584] History for questions about QC for observation
John Halley Gotway via RT
met_help at ucar.edu
Thu Nov 3 16:28:19 MDT 2016
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Dear Mr. or Mrs.,
Thanks in advance for your help.
I'm wondering what methods can be used with MET to screen the observations
in the verification process.
I found obs_quality can help if the flag parameters are in the obs file.
However, if there are no such flags in my obs and I know the definitions of
missing values or other values like -99 assigned with special meaning,
how can I make those info with MET?
Or , for simplification, if I want to use the obs in a range like -10 <
obs<70.
Can I or ,if I can, How can I invoke such a constraint in the MET process.
Your help is appreciated.
Best,
Ting
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: questions about QC for observation
From: John Halley Gotway
Time: Thu Nov 03 14:31:15 2016
Ting,
OK, there are two things I'd like to point out regarding this issue.
First, the NetCDF point observation files that Point-Stat reads
contain a
slot for quality control information. Specifically, the NetCDF output
from
ascii2nc, madis2nc, and pb2nc contain a variable named "obs_qty" which
is a
string containing quality control info.
If you're running ascii2nc, it's your job to format the input ascii
data
the expected way. If you have quality control info, it should go in
column
10, named "QC_String" in the ascii2nc usage statement. MET does not
do any
QC for you, but it can use the QC strings you've set.
In the Point-Stat config file, you can set the "obs_quality" entry to
a
list of one or more quality control strings to be used. If set, only
observations whose quality control value appears in that list will be
used.
Second, you asked if there's a way to restrict the observations used
by
filtering their values, such as -10<obs<70. The answer is yes, at
least
for continuous statistics, like RMSE and ME. We call this
"conditional
continuous verification" since we are conditioning the analysis on the
input values.
We set this up using the "cnt_thresh" and "cnt_logic" settings in the
Point-Stat config file, such as:
fcst = {
field = [
{ name = "REFC"; level=[ "L0" ]; cnt_thresh = [ NA ]; }
];
}
obs = {
field = [
{ name = "REFC"; level=[ "L0" ]; cnt_thresh = [ ge-10&&le70 ];
cnt_logic = INTERSECTION; }
];
}
Note that in the forecast I'm setting cnt_thresh to NA, meaning that
no
filtering criteria will be applied to the forecast value. But in the
observation I used "ge-10&&le70" to define the desired range of
observation
values.
You can list as many cnt_thresh values as you'd like in the list but
the
number of fcst settings must match the number of obs settings. For
example...
fcst...
cnt_thresh = [ NA, NA, NA, NA, NA ];
obs...
cnt_thresh = [ NA, ge-10, ge10, ge20, ge30 ];
The above settings will compute continuous statistics 5 ways... using
(1)
no filtering (2) obs >= -10 (3) obs >= 10 (4) obs >= 20 and (5) obs >=
30
Hope that helps. Just let me know if you have more questions.
Thanks,
John
On Thu, Nov 3, 2016 at 11:10 AM, Ting Lei - NOAA Affiliate via RT <
met_help at ucar.edu> wrote:
>
> Thu Nov 03 11:10:27 2016: Request 78584 was acted upon.
> Transaction: Ticket created by ting.lei at noaa.gov
> Queue: met_help
> Subject: questions about QC for observation
> Owner: Nobody
> Requestors: ting.lei at noaa.gov
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=78584 >
>
>
> Dear Mr. or Mrs.,
> Thanks in advance for your help.
> I'm wondering what methods can be used with MET to screen the
observations
> in the verification process.
> I found obs_quality can help if the flag parameters are in the obs
file.
> However, if there are no such flags in my obs and I know the
definitions of
> missing values or other values like -99 assigned with special
meaning,
> how can I make those info with MET?
> Or , for simplification, if I want to use the obs in a range like
-10 <
> obs<70.
> Can I or ,if I can, How can I invoke such a constraint in the MET
process.
> Your help is appreciated.
> Best,
> Ting
>
>
------------------------------------------------
Subject: questions about QC for observation
From: Ting Lei - NOAA Affiliate
Time: Thu Nov 03 14:36:41 2016
John,
Thanks a lot.
I need more time to study the first part about QC in your explanation.
However, it is great that the cnt_thresh can be used. It can
definitely
meet my requirement for being now.
Just a clarification is great. This function is working for both
Met-point and Met-grid, right?
Thanks.
Ting
On Thu, Nov 3, 2016 at 4:31 PM, John Halley Gotway via RT
<met_help at ucar.edu
> wrote:
> Ting,
>
> OK, there are two things I'd like to point out regarding this issue.
>
> First, the NetCDF point observation files that Point-Stat reads
contain a
> slot for quality control information. Specifically, the NetCDF
output from
> ascii2nc, madis2nc, and pb2nc contain a variable named "obs_qty"
which is a
> string containing quality control info.
>
> If you're running ascii2nc, it's your job to format the input ascii
data
> the expected way. If you have quality control info, it should go in
column
> 10, named "QC_String" in the ascii2nc usage statement. MET does not
do any
> QC for you, but it can use the QC strings you've set.
>
> In the Point-Stat config file, you can set the "obs_quality" entry
to a
> list of one or more quality control strings to be used. If set,
only
> observations whose quality control value appears in that list will
be used.
>
> Second, you asked if there's a way to restrict the observations used
by
> filtering their values, such as -10<obs<70. The answer is yes, at
least
> for continuous statistics, like RMSE and ME. We call this
"conditional
> continuous verification" since we are conditioning the analysis on
the
> input values.
>
> We set this up using the "cnt_thresh" and "cnt_logic" settings in
the
> Point-Stat config file, such as:
>
> fcst = {
>
> field = [
> { name = "REFC"; level=[ "L0" ]; cnt_thresh = [ NA ]; }
> ];
>
> }
>
> obs = {
>
> field = [
> { name = "REFC"; level=[ "L0" ]; cnt_thresh = [ ge-10&&le70 ];
> cnt_logic = INTERSECTION; }
> ];
>
> }
>
> Note that in the forecast I'm setting cnt_thresh to NA, meaning that
no
> filtering criteria will be applied to the forecast value. But in
the
> observation I used "ge-10&&le70" to define the desired range of
observation
> values.
>
> You can list as many cnt_thresh values as you'd like in the list but
the
> number of fcst settings must match the number of obs settings. For
> example...
>
> fcst...
> cnt_thresh = [ NA, NA, NA, NA, NA ];
>
> obs...
> cnt_thresh = [ NA, ge-10, ge10, ge20, ge30 ];
>
> The above settings will compute continuous statistics 5 ways...
using (1)
> no filtering (2) obs >= -10 (3) obs >= 10 (4) obs >= 20 and (5) obs
>= 30
>
> Hope that helps. Just let me know if you have more questions.
>
> Thanks,
> John
>
>
> On Thu, Nov 3, 2016 at 11:10 AM, Ting Lei - NOAA Affiliate via RT <
> met_help at ucar.edu> wrote:
>
> >
> > Thu Nov 03 11:10:27 2016: Request 78584 was acted upon.
> > Transaction: Ticket created by ting.lei at noaa.gov
> > Queue: met_help
> > Subject: questions about QC for observation
> > Owner: Nobody
> > Requestors: ting.lei at noaa.gov
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=78584 >
> >
> >
> > Dear Mr. or Mrs.,
> > Thanks in advance for your help.
> > I'm wondering what methods can be used with MET to screen the
> observations
> > in the verification process.
> > I found obs_quality can help if the flag parameters are in the obs
file.
> > However, if there are no such flags in my obs and I know the
definitions
> of
> > missing values or other values like -99 assigned with special
meaning,
> > how can I make those info with MET?
> > Or , for simplification, if I want to use the obs in a range like
-10 <
> > obs<70.
> > Can I or ,if I can, How can I invoke such a constraint in the MET
> process.
> > Your help is appreciated.
> > Best,
> > Ting
> >
> >
>
>
------------------------------------------------
Subject: questions about QC for observation
From: John Halley Gotway
Time: Thu Nov 03 15:12:33 2016
Ting,
Yes, the cnt_thresh filtering option works for both Grid-Stat and
Point-Stat.
The fcst -> cnt_thresh option filters the forecast values. The obs ->
cnt_thresh option filters the observation values. And the cnt_logic
option
can be set to UNION, INTERSECTION, or SYMDIFF to specify how the logic
for
how the forecast and observation thresholds should be combined.
For example... suppose you've defined cnt_thresh for temperature as
>273.
cnt_logic = UNION means the forecast **or** observation value is >
273.
cnt_logic = INTERSECTION means the forecast **and** observation values
are
>273.
cnt_logic = SYMDIFF means the that one is >273 and the other is not.
Thanks,
John
On Thu, Nov 3, 2016 at 2:36 PM, Ting Lei - NOAA Affiliate via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=78584 >
>
> John,
> Thanks a lot.
> I need more time to study the first part about QC in your
explanation.
> However, it is great that the cnt_thresh can be used. It can
definitely
> meet my requirement for being now.
> Just a clarification is great. This function is working for both
> Met-point and Met-grid, right?
> Thanks.
> Ting
>
> On Thu, Nov 3, 2016 at 4:31 PM, John Halley Gotway via RT <
> met_help at ucar.edu
> > wrote:
>
> > Ting,
> >
> > OK, there are two things I'd like to point out regarding this
issue.
> >
> > First, the NetCDF point observation files that Point-Stat reads
contain a
> > slot for quality control information. Specifically, the NetCDF
output
> from
> > ascii2nc, madis2nc, and pb2nc contain a variable named "obs_qty"
which
> is a
> > string containing quality control info.
> >
> > If you're running ascii2nc, it's your job to format the input
ascii data
> > the expected way. If you have quality control info, it should go
in
> column
> > 10, named "QC_String" in the ascii2nc usage statement. MET does
not do
> any
> > QC for you, but it can use the QC strings you've set.
> >
> > In the Point-Stat config file, you can set the "obs_quality" entry
to a
> > list of one or more quality control strings to be used. If set,
only
> > observations whose quality control value appears in that list will
be
> used.
> >
> > Second, you asked if there's a way to restrict the observations
used by
> > filtering their values, such as -10<obs<70. The answer is yes, at
least
> > for continuous statistics, like RMSE and ME. We call this
"conditional
> > continuous verification" since we are conditioning the analysis on
the
> > input values.
> >
> > We set this up using the "cnt_thresh" and "cnt_logic" settings in
the
> > Point-Stat config file, such as:
> >
> > fcst = {
> >
> > field = [
> > { name = "REFC"; level=[ "L0" ]; cnt_thresh = [ NA ]; }
> > ];
> >
> > }
> >
> > obs = {
> >
> > field = [
> > { name = "REFC"; level=[ "L0" ]; cnt_thresh = [ ge-10&&le70 ];
> > cnt_logic = INTERSECTION; }
> > ];
> >
> > }
> >
> > Note that in the forecast I'm setting cnt_thresh to NA, meaning
that no
> > filtering criteria will be applied to the forecast value. But in
the
> > observation I used "ge-10&&le70" to define the desired range of
> observation
> > values.
> >
> > You can list as many cnt_thresh values as you'd like in the list
but the
> > number of fcst settings must match the number of obs settings.
For
> > example...
> >
> > fcst...
> > cnt_thresh = [ NA, NA, NA, NA, NA ];
> >
> > obs...
> > cnt_thresh = [ NA, ge-10, ge10, ge20, ge30 ];
> >
> > The above settings will compute continuous statistics 5 ways...
using (1)
> > no filtering (2) obs >= -10 (3) obs >= 10 (4) obs >= 20 and (5)
obs >= 30
> >
> > Hope that helps. Just let me know if you have more questions.
> >
> > Thanks,
> > John
> >
> >
> > On Thu, Nov 3, 2016 at 11:10 AM, Ting Lei - NOAA Affiliate via RT
<
> > met_help at ucar.edu> wrote:
> >
> > >
> > > Thu Nov 03 11:10:27 2016: Request 78584 was acted upon.
> > > Transaction: Ticket created by ting.lei at noaa.gov
> > > Queue: met_help
> > > Subject: questions about QC for observation
> > > Owner: Nobody
> > > Requestors: ting.lei at noaa.gov
> > > Status: new
> > > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=78584
> >
> > >
> > >
> > > Dear Mr. or Mrs.,
> > > Thanks in advance for your help.
> > > I'm wondering what methods can be used with MET to screen the
> > observations
> > > in the verification process.
> > > I found obs_quality can help if the flag parameters are in the
obs
> file.
> > > However, if there are no such flags in my obs and I know the
> definitions
> > of
> > > missing values or other values like -99 assigned with special
meaning,
> > > how can I make those info with MET?
> > > Or , for simplification, if I want to use the obs in a range
like -10 <
> > > obs<70.
> > > Can I or ,if I can, How can I invoke such a constraint in the
MET
> > process.
> > > Your help is appreciated.
> > > Best,
> > > Ting
> > >
> > >
> >
> >
>
>
------------------------------------------------
Subject: questions about QC for observation
From: Ting Lei - NOAA Affiliate
Time: Thu Nov 03 16:14:07 2016
John,
Thanks again for your explanation.
That is great.
My question in this email has been perfectly answered.
Best Wishes,
Ting
On Thu, Nov 3, 2016 at 5:12 PM, John Halley Gotway via RT
<met_help at ucar.edu
> wrote:
> Ting,
>
> Yes, the cnt_thresh filtering option works for both Grid-Stat and
> Point-Stat.
>
> The fcst -> cnt_thresh option filters the forecast values. The obs
->
> cnt_thresh option filters the observation values. And the cnt_logic
option
> can be set to UNION, INTERSECTION, or SYMDIFF to specify how the
logic for
> how the forecast and observation thresholds should be combined.
>
> For example... suppose you've defined cnt_thresh for temperature as
>273.
> cnt_logic = UNION means the forecast **or** observation value is >
273.
> cnt_logic = INTERSECTION means the forecast **and** observation
values are
> >273.
> cnt_logic = SYMDIFF means the that one is >273 and the other is not.
>
> Thanks,
> John
>
> On Thu, Nov 3, 2016 at 2:36 PM, Ting Lei - NOAA Affiliate via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=78584 >
> >
> > John,
> > Thanks a lot.
> > I need more time to study the first part about QC in your
explanation.
> > However, it is great that the cnt_thresh can be used. It can
definitely
> > meet my requirement for being now.
> > Just a clarification is great. This function is working for both
> > Met-point and Met-grid, right?
> > Thanks.
> > Ting
> >
> > On Thu, Nov 3, 2016 at 4:31 PM, John Halley Gotway via RT <
> > met_help at ucar.edu
> > > wrote:
> >
> > > Ting,
> > >
> > > OK, there are two things I'd like to point out regarding this
issue.
> > >
> > > First, the NetCDF point observation files that Point-Stat reads
> contain a
> > > slot for quality control information. Specifically, the NetCDF
output
> > from
> > > ascii2nc, madis2nc, and pb2nc contain a variable named "obs_qty"
which
> > is a
> > > string containing quality control info.
> > >
> > > If you're running ascii2nc, it's your job to format the input
ascii
> data
> > > the expected way. If you have quality control info, it should
go in
> > column
> > > 10, named "QC_String" in the ascii2nc usage statement. MET does
not do
> > any
> > > QC for you, but it can use the QC strings you've set.
> > >
> > > In the Point-Stat config file, you can set the "obs_quality"
entry to a
> > > list of one or more quality control strings to be used. If set,
only
> > > observations whose quality control value appears in that list
will be
> > used.
> > >
> > > Second, you asked if there's a way to restrict the observations
used by
> > > filtering their values, such as -10<obs<70. The answer is yes,
at
> least
> > > for continuous statistics, like RMSE and ME. We call this
"conditional
> > > continuous verification" since we are conditioning the analysis
on the
> > > input values.
> > >
> > > We set this up using the "cnt_thresh" and "cnt_logic" settings
in the
> > > Point-Stat config file, such as:
> > >
> > > fcst = {
> > >
> > > field = [
> > > { name = "REFC"; level=[ "L0" ]; cnt_thresh = [ NA ]; }
> > > ];
> > >
> > > }
> > >
> > > obs = {
> > >
> > > field = [
> > > { name = "REFC"; level=[ "L0" ]; cnt_thresh = [ ge-10&&le70
];
> > > cnt_logic = INTERSECTION; }
> > > ];
> > >
> > > }
> > >
> > > Note that in the forecast I'm setting cnt_thresh to NA, meaning
that no
> > > filtering criteria will be applied to the forecast value. But
in the
> > > observation I used "ge-10&&le70" to define the desired range of
> > observation
> > > values.
> > >
> > > You can list as many cnt_thresh values as you'd like in the list
but
> the
> > > number of fcst settings must match the number of obs settings.
For
> > > example...
> > >
> > > fcst...
> > > cnt_thresh = [ NA, NA, NA, NA, NA ];
> > >
> > > obs...
> > > cnt_thresh = [ NA, ge-10, ge10, ge20, ge30 ];
> > >
> > > The above settings will compute continuous statistics 5 ways...
using
> (1)
> > > no filtering (2) obs >= -10 (3) obs >= 10 (4) obs >= 20 and (5)
obs >=
> 30
> > >
> > > Hope that helps. Just let me know if you have more questions.
> > >
> > > Thanks,
> > > John
> > >
> > >
> > > On Thu, Nov 3, 2016 at 11:10 AM, Ting Lei - NOAA Affiliate via
RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > Thu Nov 03 11:10:27 2016: Request 78584 was acted upon.
> > > > Transaction: Ticket created by ting.lei at noaa.gov
> > > > Queue: met_help
> > > > Subject: questions about QC for observation
> > > > Owner: Nobody
> > > > Requestors: ting.lei at noaa.gov
> > > > Status: new
> > > > Ticket <URL: https://rt.rap.ucar.edu/rt/
> Ticket/Display.html?id=78584
> > >
> > > >
> > > >
> > > > Dear Mr. or Mrs.,
> > > > Thanks in advance for your help.
> > > > I'm wondering what methods can be used with MET to screen the
> > > observations
> > > > in the verification process.
> > > > I found obs_quality can help if the flag parameters are in the
obs
> > file.
> > > > However, if there are no such flags in my obs and I know the
> > definitions
> > > of
> > > > missing values or other values like -99 assigned with special
> meaning,
> > > > how can I make those info with MET?
> > > > Or , for simplification, if I want to use the obs in a range
like
> -10 <
> > > > obs<70.
> > > > Can I or ,if I can, How can I invoke such a constraint in the
MET
> > > process.
> > > > Your help is appreciated.
> > > > Best,
> > > > Ting
> > > >
> > > >
> > >
> > >
> >
> >
>
>
------------------------------------------------
More information about the Met_help
mailing list