[Met_help] [rt.rap.ucar.edu #95099] History for No. YES per forecast probability threshold
John Halley Gotway via RT
met_help at ucar.edu
Thu Apr 30 15:27:02 MDT 2020
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Folks - I was wondering if there's a way to obtain the ni1 or the number of YES per probability threshold as ni1 is defined on Table C.2 page 410 of MET_Users_Guide_v81.12.pdf.
I know that Likelihood(i) = ni1/(SUM ni1) and I'm after the number of cases per probability threshold or ni1. Thank you very much.
Efren A. Serra (Contractor)
Physicist
DeVine Consulting, Inc.
Naval Research Laboratory
Marine Meteorology Division
7 Grace Hopper Ave., STOP 2
Monterey, CA 93943
Code 7542
Mobile: 408-425-5027
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: No. YES per forecast probability threshold
From: Minna Win
Time: Thu Apr 30 11:06:12 2020
Hi Efren,
It looks like you have a verification question that Tara can answer.
I'll
assign this issue to her, but she is currently in a workshop, so
please
allow a few days for a response.
Regards,
Minna
---------------
Minna Win
National Center for Atmospheric Research
Developmental Testbed Center
Phone: 303-497-8423
Fax: 303-497-8401
On Thu, Apr 30, 2020 at 10:13 AM efren.serra.ctr at nrlmry.navy.mil via
RT <
met_help at ucar.edu> wrote:
>
> Thu Apr 30 10:12:57 2020: Request 95099 was acted upon.
> Transaction: Ticket created by efren.serra.ctr at nrlmry.navy.mil
> Queue: met_help
> Subject: No. YES per forecast probability threshold
> Owner: Nobody
> Requestors: efren.serra.ctr at nrlmry.navy.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95099 >
>
>
> Folks - I was wondering if there's a way to obtain the ni1 or the
number
> of YES per probability threshold as ni1 is defined on Table C.2 page
410 of
> MET_Users_Guide_v81.12.pdf.
> I know that Likelihood(i) = ni1/(SUM ni1) and I'm after the number
of
> cases per probability threshold or ni1. Thank you very much.
>
> Efren A. Serra (Contractor)
> Physicist
>
> DeVine Consulting, Inc.
> Naval Research Laboratory
> Marine Meteorology Division
> 7 Grace Hopper Ave., STOP 2
> Monterey, CA 93943
> Code 7542
> Mobile: 408-425-5027
>
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #95099] No. YES per forecast probability threshold
From: efren.serra.ctr at nrlmry.navy.mil
Time: Thu Apr 30 11:28:16 2020
Thanks!
-----Original Message-----
From: Minna Win via RT <met_help at ucar.edu>
Sent: Thursday, April 30, 2020 10:06 AM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #95099] No. YES per forecast probability
threshold
Hi Efren,
It looks like you have a verification question that Tara can answer.
I'll assign this issue to her, but she is currently in a workshop, so
please allow a few days for a response.
Regards,
Minna
---------------
Minna Win
National Center for Atmospheric Research Developmental Testbed Center
Phone: 303-497-8423
Fax: 303-497-8401
On Thu, Apr 30, 2020 at 10:13 AM efren.serra.ctr at nrlmry.navy.mil via
RT < met_help at ucar.edu> wrote:
>
> Thu Apr 30 10:12:57 2020: Request 95099 was acted upon.
> Transaction: Ticket created by efren.serra.ctr at nrlmry.navy.mil
> Queue: met_help
> Subject: No. YES per forecast probability threshold
> Owner: Nobody
> Requestors: efren.serra.ctr at nrlmry.navy.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95099
> >
>
>
> Folks - I was wondering if there's a way to obtain the ni1 or the
> number of YES per probability threshold as ni1 is defined on Table
C.2
> page 410 of MET_Users_Guide_v81.12.pdf.
> I know that Likelihood(i) = ni1/(SUM ni1) and I'm after the number
of
> cases per probability threshold or ni1. Thank you very much.
>
> Efren A. Serra (Contractor)
> Physicist
>
> DeVine Consulting, Inc.
> Naval Research Laboratory
> Marine Meteorology Division
> 7 Grace Hopper Ave., STOP 2
> Monterey, CA 93943
> Code 7542
> Mobile: 408-425-5027
>
>
>
------------------------------------------------
Subject: No. YES per forecast probability threshold
From: Minna Win
Time: Thu Apr 30 11:54:05 2020
Hi Efren,
I incorrectly assigned this to Tara, so I am going to re-assign this
to
John Halley Gotway. He is busy finishing up some tasks that have
tight
deadlines this week, so please allow a few days for a full response.
Thanks for your patience.
Regards,
Minna
---------------
Minna Win
National Center for Atmospheric Research
Developmental Testbed Center
Phone: 303-497-8423
Fax: 303-497-8401
On Thu, Apr 30, 2020 at 11:28 AM efren.serra.ctr at nrlmry.navy.mil via
RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95099 >
>
> Thanks!
>
> -----Original Message-----
> From: Minna Win via RT <met_help at ucar.edu>
> Sent: Thursday, April 30, 2020 10:06 AM
> To: Serra, Mr. Efren, Contractor, Code 7531 <
> efren.serra.ctr at nrlmry.navy.mil>
> Subject: Re: [rt.rap.ucar.edu #95099] No. YES per forecast
probability
> threshold
>
> Hi Efren,
>
> It looks like you have a verification question that Tara can
answer.
> I'll assign this issue to her, but she is currently in a workshop,
so
> please allow a few days for a response.
>
> Regards,
> Minna
> ---------------
> Minna Win
> National Center for Atmospheric Research Developmental Testbed
Center
> Phone: 303-497-8423
> Fax: 303-497-8401
>
>
>
> On Thu, Apr 30, 2020 at 10:13 AM efren.serra.ctr at nrlmry.navy.mil via
RT <
> met_help at ucar.edu> wrote:
>
> >
> > Thu Apr 30 10:12:57 2020: Request 95099 was acted upon.
> > Transaction: Ticket created by efren.serra.ctr at nrlmry.navy.mil
> > Queue: met_help
> > Subject: No. YES per forecast probability threshold
> > Owner: Nobody
> > Requestors: efren.serra.ctr at nrlmry.navy.mil
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95099
> > >
> >
> >
> > Folks - I was wondering if there's a way to obtain the ni1 or the
> > number of YES per probability threshold as ni1 is defined on Table
C.2
> > page 410 of MET_Users_Guide_v81.12.pdf.
> > I know that Likelihood(i) = ni1/(SUM ni1) and I'm after the number
of
> > cases per probability threshold or ni1. Thank you very much.
> >
> > Efren A. Serra (Contractor)
> > Physicist
> >
> > DeVine Consulting, Inc.
> > Naval Research Laboratory
> > Marine Meteorology Division
> > 7 Grace Hopper Ave., STOP 2
> > Monterey, CA 93943
> > Code 7542
> > Mobile: 408-425-5027
> >
> >
> >
>
>
>
>
------------------------------------------------
Subject: No. YES per forecast probability threshold
From: John Halley Gotway
Time: Thu Apr 30 12:11:10 2020
Efren,
MET verifies probability forecasts using an Nx2 probabilistic
contingency
table. The user defines the probability bins by setting the
"cat_thresh"
in the "fcst" dictionary of the configuration file. And the user
defines
the event threshold by setting the "cat_thresh" in the "obs"
dictionary of
the configuration file.
For example, when verifying a probability of precipitation forecast,
you
might set...
fcst = {
...
cat_thresh = [ ==0.1 ];
}
obs = {
...
cat_thresh = [ >0 ];
}
This specifies that we want 10, evenly-spaced, probability bins (0 to
0.1,
0.1 to 0.2, and so on). And defines the event for the observation as
precip amounts greater than 0.
So this yields a 10x2 contingency table. For each matched pair, MET
bins
the forecast probability value to determine the row of the table. And
then
it checks if the event did or did not actually occur in the
observation to
determine the column. It processes all the matched pairs in this way
to
populate the counts for each cell of that 10x2 table. All of the
probabilistic stats that MET reports are based on the counts of this
Nx2
table. And those probabilistic outputs are spread across 4 lines
types
(PCT, PSTD, PJC, and PRC). The contents of each of these line types
is
described in tables 7.10 through 7.13 of the MET User's Guide:
https://dtcenter.org/sites/default/files/community-code/met/docs/user-
guide/MET_Users_Guide_v8.1.2.pdf
You're asking about the raw counts, which are contained in the PCT
line
type. Please see table 7.10 for a description.
Also, be sure you're actually requesting PCT output in Point-Stat
config
file by setting:
output_flag = {
...
pct = BOTH; // or just STAT
...
}
Thanks,
John
On Thu, Apr 30, 2020 at 11:55 AM Minna Win via RT <met_help at ucar.edu>
wrote:
>
> Thu Apr 30 11:54:53 2020: Request 95099 was acted upon.
> Transaction: Given to johnhg (John Halley Gotway) by minnawin
> Queue: met_help
> Subject: No. YES per forecast probability threshold
> Owner: johnhg
> Requestors: efren.serra.ctr at nrlmry.navy.mil
> Status: open
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95099 >
>
>
> This transaction appears to have no content
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #95099] No. YES per forecast probability threshold
From: efren.serra.ctr at nrlmry.navy.mil
Time: Thu Apr 30 12:14:49 2020
Thanks! I do have the pct data file; I think this is what we want.
-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Thursday, April 30, 2020 11:11 AM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #95099] No. YES per forecast probability
threshold
Efren,
MET verifies probability forecasts using an Nx2 probabilistic
contingency table. The user defines the probability bins by setting
the "cat_thresh"
in the "fcst" dictionary of the configuration file. And the user
defines the event threshold by setting the "cat_thresh" in the "obs"
dictionary of the configuration file.
For example, when verifying a probability of precipitation forecast,
you might set...
fcst = {
...
cat_thresh = [ ==0.1 ];
}
obs = {
...
cat_thresh = [ >0 ];
}
This specifies that we want 10, evenly-spaced, probability bins (0 to
0.1,
0.1 to 0.2, and so on). And defines the event for the observation as
precip amounts greater than 0.
So this yields a 10x2 contingency table. For each matched pair, MET
bins the forecast probability value to determine the row of the table.
And then it checks if the event did or did not actually occur in the
observation to determine the column. It processes all the matched
pairs in this way to populate the counts for each cell of that 10x2
table. All of the probabilistic stats that MET reports are based on
the counts of this Nx2 table. And those probabilistic outputs are
spread across 4 lines types (PCT, PSTD, PJC, and PRC). The contents
of each of these line types is described in tables 7.10 through 7.13
of the MET User's Guide:
https://dtcenter.org/sites/default/files/community-code/met/docs/user-
guide/MET_Users_Guide_v8.1.2.pdf
You're asking about the raw counts, which are contained in the PCT
line type. Please see table 7.10 for a description.
Also, be sure you're actually requesting PCT output in Point-Stat
config file by setting:
output_flag = {
...
pct = BOTH; // or just STAT
...
}
Thanks,
John
On Thu, Apr 30, 2020 at 11:55 AM Minna Win via RT <met_help at ucar.edu>
wrote:
>
> Thu Apr 30 11:54:53 2020: Request 95099 was acted upon.
> Transaction: Given to johnhg (John Halley Gotway) by minnawin
> Queue: met_help
> Subject: No. YES per forecast probability threshold
> Owner: johnhg
> Requestors: efren.serra.ctr at nrlmry.navy.mil
> Status: open
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95099
> >
>
>
> This transaction appears to have no content
>
------------------------------------------------
Subject: No. YES per forecast probability threshold
From: Sampson, Mr. Buck
Time: Thu Apr 30 12:16:29 2020
Ah. Many thanks.
Buck
-----Original Message-----
From: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Sent: Thursday, April 30, 2020 11:15 AM
To: met_help at ucar.edu; Sampson, Mr. Buck
<Buck.Sampson at nrlmry.navy.mil>
Subject: RE: [rt.rap.ucar.edu #95099] No. YES per forecast probability
threshold
Thanks! I do have the pct data file; I think this is what we want.
-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Thursday, April 30, 2020 11:11 AM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #95099] No. YES per forecast probability
threshold
Efren,
MET verifies probability forecasts using an Nx2 probabilistic
contingency
table. The user defines the probability bins by setting the
"cat_thresh"
in the "fcst" dictionary of the configuration file. And the user
defines the
event threshold by setting the "cat_thresh" in the "obs" dictionary of
the
configuration file.
For example, when verifying a probability of precipitation forecast,
you might
set...
fcst = {
...
cat_thresh = [ ==0.1 ];
}
obs = {
...
cat_thresh = [ >0 ];
}
This specifies that we want 10, evenly-spaced, probability bins (0 to
0.1,
0.1 to 0.2, and so on). And defines the event for the observation as
precip
amounts greater than 0.
So this yields a 10x2 contingency table. For each matched pair, MET
bins the
forecast probability value to determine the row of the table. And
then it
checks if the event did or did not actually occur in the observation
to
determine the column. It processes all the matched pairs in this way
to
populate the counts for each cell of that 10x2 table. All of the
probabilistic stats that MET reports are based on the counts of this
Nx2
table. And those probabilistic outputs are spread across 4 lines
types (PCT,
PSTD, PJC, and PRC). The contents of each of these line types is
described in
tables 7.10 through 7.13 of the MET User's Guide:
https://dtcenter.org/sites/default/files/community-code/met/docs/user-
guide/MET_Users_Guide_v8.1.2.pdf
You're asking about the raw counts, which are contained in the PCT
line type.
Please see table 7.10 for a description.
Also, be sure you're actually requesting PCT output in Point-Stat
config file
by setting:
output_flag = {
...
pct = BOTH; // or just STAT
...
}
Thanks,
John
On Thu, Apr 30, 2020 at 11:55 AM Minna Win via RT <met_help at ucar.edu>
wrote:
>
> Thu Apr 30 11:54:53 2020: Request 95099 was acted upon.
> Transaction: Given to johnhg (John Halley Gotway) by minnawin
> Queue: met_help
> Subject: No. YES per forecast probability threshold
> Owner: johnhg
> Requestors: efren.serra.ctr at nrlmry.navy.mil
> Status: open
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95099
> >
>
>
> This transaction appears to have no content
>
------------------------------------------------
Subject: No. YES per forecast probability threshold
From: efren.serra.ctr at nrlmry.navy.mil
Time: Thu Apr 30 12:25:07 2020
John - here's a pair of outputs I'm creating from we've been
hammering. In the likelihood plot, I plot as "Not observed" 1-
likelihood[i] for each probability threshold *i*. Is this accurate?
I plot likelihood[i] as "Observed."
-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Thursday, April 30, 2020 11:11 AM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #95099] No. YES per forecast probability
threshold
Efren,
MET verifies probability forecasts using an Nx2 probabilistic
contingency table. The user defines the probability bins by setting
the "cat_thresh"
in the "fcst" dictionary of the configuration file. And the user
defines the event threshold by setting the "cat_thresh" in the "obs"
dictionary of the configuration file.
For example, when verifying a probability of precipitation forecast,
you might set...
fcst = {
...
cat_thresh = [ ==0.1 ];
}
obs = {
...
cat_thresh = [ >0 ];
}
This specifies that we want 10, evenly-spaced, probability bins (0 to
0.1,
0.1 to 0.2, and so on). And defines the event for the observation as
precip amounts greater than 0.
So this yields a 10x2 contingency table. For each matched pair, MET
bins the forecast probability value to determine the row of the table.
And then it checks if the event did or did not actually occur in the
observation to determine the column. It processes all the matched
pairs in this way to populate the counts for each cell of that 10x2
table. All of the probabilistic stats that MET reports are based on
the counts of this Nx2 table. And those probabilistic outputs are
spread across 4 lines types (PCT, PSTD, PJC, and PRC). The contents
of each of these line types is described in tables 7.10 through 7.13
of the MET User's Guide:
https://dtcenter.org/sites/default/files/community-code/met/docs/user-
guide/MET_Users_Guide_v8.1.2.pdf
You're asking about the raw counts, which are contained in the PCT
line type. Please see table 7.10 for a description.
Also, be sure you're actually requesting PCT output in Point-Stat
config file by setting:
output_flag = {
...
pct = BOTH; // or just STAT
...
}
Thanks,
John
On Thu, Apr 30, 2020 at 11:55 AM Minna Win via RT <met_help at ucar.edu>
wrote:
>
> Thu Apr 30 11:54:53 2020: Request 95099 was acted upon.
> Transaction: Given to johnhg (John Halley Gotway) by minnawin
> Queue: met_help
> Subject: No. YES per forecast probability threshold
> Owner: johnhg
> Requestors: efren.serra.ctr at nrlmry.navy.mil
> Status: open
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95099
> >
>
>
> This transaction appears to have no content
>
------------------------------------------------
Subject: No. YES per forecast probability threshold
From: efren.serra.ctr at nrlmry.navy.mil
Time: Thu Apr 30 12:27:00 2020
I updated 2nd x-axis from "No. of Cases" *to* "No. of YES Cases"
-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Thursday, April 30, 2020 11:11 AM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #95099] No. YES per forecast probability
threshold
Efren,
MET verifies probability forecasts using an Nx2 probabilistic
contingency table. The user defines the probability bins by setting
the "cat_thresh"
in the "fcst" dictionary of the configuration file. And the user
defines the event threshold by setting the "cat_thresh" in the "obs"
dictionary of the configuration file.
For example, when verifying a probability of precipitation forecast,
you might set...
fcst = {
...
cat_thresh = [ ==0.1 ];
}
obs = {
...
cat_thresh = [ >0 ];
}
This specifies that we want 10, evenly-spaced, probability bins (0 to
0.1,
0.1 to 0.2, and so on). And defines the event for the observation as
precip amounts greater than 0.
So this yields a 10x2 contingency table. For each matched pair, MET
bins the forecast probability value to determine the row of the table.
And then it checks if the event did or did not actually occur in the
observation to determine the column. It processes all the matched
pairs in this way to populate the counts for each cell of that 10x2
table. All of the probabilistic stats that MET reports are based on
the counts of this Nx2 table. And those probabilistic outputs are
spread across 4 lines types (PCT, PSTD, PJC, and PRC). The contents
of each of these line types is described in tables 7.10 through 7.13
of the MET User's Guide:
https://dtcenter.org/sites/default/files/community-code/met/docs/user-
guide/MET_Users_Guide_v8.1.2.pdf
You're asking about the raw counts, which are contained in the PCT
line type. Please see table 7.10 for a description.
Also, be sure you're actually requesting PCT output in Point-Stat
config file by setting:
output_flag = {
...
pct = BOTH; // or just STAT
...
}
Thanks,
John
On Thu, Apr 30, 2020 at 11:55 AM Minna Win via RT <met_help at ucar.edu>
wrote:
>
> Thu Apr 30 11:54:53 2020: Request 95099 was acted upon.
> Transaction: Given to johnhg (John Halley Gotway) by minnawin
> Queue: met_help
> Subject: No. YES per forecast probability threshold
> Owner: johnhg
> Requestors: efren.serra.ctr at nrlmry.navy.mil
> Status: open
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95099
> >
>
>
> This transaction appears to have no content
>
------------------------------------------------
Subject: No. YES per forecast probability threshold
From: John Halley Gotway
Time: Thu Apr 30 13:46:11 2020
Efren,
I'd recommend one change to the reliability diagram you sent. For
each
probability bin, n to n+1, it looks like you're plotting the dot at x
= n.
For example, your dot at x = 0.5 represents all probability forecasts
from
0.5 to 0.6. I'd recommend plotting that at x = 0.55 instead, the mid-
point
of the probability bin.
In the DTC, we use a database and display system for the output of
MET,
called METviewer. METviewer is currently deployed at NCAR, NOAA/EMC
(via
Amazon web services), NOAA/GSL in Boulder, and a few other places.
After
generating output via MET, we load that data into a METviewer database
and
create plots either via a batch plotting system or by interacting with
a
web GUI. When we last visited NRL, there wasn't much interest in
METviewer
yet, but I just want to remind you about it.
In addition, our team is in the process of breaking METviewer apart
into
smaller components, including the transition from creating plots with
R to
plotting via python. This is still in the early stages, but METviewer
will
ultimately become METdb (to store MET output in a database), METcalcpy
(to
aggregate stats across cases), METplotpy (to visualize the stats),
METviewer (a GUI frontend for the aggregation and plotting), and
METexpress
(a second GUI developed by NOAA/GSL).
To illustrate METviewer, I made the attached reliability diagram using
the
NCAR instance of METviewer:
http://www.dtcenter.org/met/metviewer/metviewer1.jsp
If you'd like to test this out yourself, you could download the
attached
xml file. Go to the website and click "Load XML" in the top-right
corner.
Navigate to the saved XML file and once you've selected it, click the
"Generate Plot" button. When you do, METviewer, will...
(1) Use the selected criteria to construct a database query for PCT
data.
(2) Execute the query and retrieve the PCT data from the database.
(3) Aggregate the PCT data across all the cases present, but
separately for
each line of the plot.
(4) Derive the correct stats for each line.
(5) Create the plot as a png and serve it up via the website.
So there's a whole lot of functionality already in there. And the
reliability diagram is only one of several plot types METviewer can
handle. If think folks at NRL might find it very useful.
Thanks,
John
On Thu, Apr 30, 2020 at 12:27 PM efren.serra.ctr at nrlmry.navy.mil via
RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95099 >
>
> I updated 2nd x-axis from "No. of Cases" *to* "No. of YES Cases"
>
> -----Original Message-----
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Thursday, April 30, 2020 11:11 AM
> To: Serra, Mr. Efren, Contractor, Code 7531 <
> efren.serra.ctr at nrlmry.navy.mil>
> Subject: Re: [rt.rap.ucar.edu #95099] No. YES per forecast
probability
> threshold
>
> Efren,
>
> MET verifies probability forecasts using an Nx2 probabilistic
contingency
> table. The user defines the probability bins by setting the
"cat_thresh"
> in the "fcst" dictionary of the configuration file. And the user
defines
> the event threshold by setting the "cat_thresh" in the "obs"
dictionary of
> the configuration file.
>
> For example, when verifying a probability of precipitation forecast,
you
> might set...
> fcst = {
> ...
> cat_thresh = [ ==0.1 ];
> }
> obs = {
> ...
> cat_thresh = [ >0 ];
> }
>
> This specifies that we want 10, evenly-spaced, probability bins (0
to 0.1,
> 0.1 to 0.2, and so on). And defines the event for the observation
as
> precip amounts greater than 0.
>
> So this yields a 10x2 contingency table. For each matched pair, MET
bins
> the forecast probability value to determine the row of the table.
And then
> it checks if the event did or did not actually occur in the
observation to
> determine the column. It processes all the matched pairs in this
way to
> populate the counts for each cell of that 10x2 table. All of the
> probabilistic stats that MET reports are based on the counts of this
Nx2
> table. And those probabilistic outputs are spread across 4 lines
types
> (PCT, PSTD, PJC, and PRC). The contents of each of these line types
is
> described in tables 7.10 through 7.13 of the MET User's Guide:
>
> https://dtcenter.org/sites/default/files/community-
code/met/docs/user-guide/MET_Users_Guide_v8.1.2.pdf
>
> You're asking about the raw counts, which are contained in the PCT
line
> type. Please see table 7.10 for a description.
>
> Also, be sure you're actually requesting PCT output in Point-Stat
config
> file by setting:
> output_flag = {
> ...
> pct = BOTH; // or just STAT
> ...
> }
>
> Thanks,
> John
>
> On Thu, Apr 30, 2020 at 11:55 AM Minna Win via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > Thu Apr 30 11:54:53 2020: Request 95099 was acted upon.
> > Transaction: Given to johnhg (John Halley Gotway) by minnawin
> > Queue: met_help
> > Subject: No. YES per forecast probability threshold
> > Owner: johnhg
> > Requestors: efren.serra.ctr at nrlmry.navy.mil
> > Status: open
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95099
> > >
> >
> >
> > This transaction appears to have no content
> >
>
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #95099] No. YES per forecast probability threshold
From: efren.serra.ctr at nrlmry.navy.mil
Time: Thu Apr 30 15:02:55 2020
Thank you very much John, this is greatly appreciated.
-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Thursday, April 30, 2020 12:46 PM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #95099] No. YES per forecast probability
threshold
Efren,
I'd recommend one change to the reliability diagram you sent. For
each probability bin, n to n+1, it looks like you're plotting the dot
at x = n.
For example, your dot at x = 0.5 represents all probability forecasts
from
0.5 to 0.6. I'd recommend plotting that at x = 0.55 instead, the mid-
point of the probability bin.
In the DTC, we use a database and display system for the output of
MET, called METviewer. METviewer is currently deployed at NCAR,
NOAA/EMC (via Amazon web services), NOAA/GSL in Boulder, and a few
other places. After generating output via MET, we load that data into
a METviewer database and create plots either via a batch plotting
system or by interacting with a web GUI. When we last visited NRL,
there wasn't much interest in METviewer yet, but I just want to remind
you about it.
In addition, our team is in the process of breaking METviewer apart
into smaller components, including the transition from creating plots
with R to plotting via python. This is still in the early stages, but
METviewer will ultimately become METdb (to store MET output in a
database), METcalcpy (to aggregate stats across cases), METplotpy (to
visualize the stats), METviewer (a GUI frontend for the aggregation
and plotting), and METexpress (a second GUI developed by NOAA/GSL).
To illustrate METviewer, I made the attached reliability diagram using
the NCAR instance of METviewer:
http://www.dtcenter.org/met/metviewer/metviewer1.jsp
If you'd like to test this out yourself, you could download the
attached xml file. Go to the website and click "Load XML" in the top-
right corner.
Navigate to the saved XML file and once you've selected it, click the
"Generate Plot" button. When you do, METviewer, will...
(1) Use the selected criteria to construct a database query for PCT
data.
(2) Execute the query and retrieve the PCT data from the database.
(3) Aggregate the PCT data across all the cases present, but
separately for each line of the plot.
(4) Derive the correct stats for each line.
(5) Create the plot as a png and serve it up via the website.
So there's a whole lot of functionality already in there. And the
reliability diagram is only one of several plot types METviewer can
handle. If think folks at NRL might find it very useful.
Thanks,
John
On Thu, Apr 30, 2020 at 12:27 PM efren.serra.ctr at nrlmry.navy.mil via
RT < met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95099 >
>
> I updated 2nd x-axis from "No. of Cases" *to* "No. of YES Cases"
>
> -----Original Message-----
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Thursday, April 30, 2020 11:11 AM
> To: Serra, Mr. Efren, Contractor, Code 7531 <
> efren.serra.ctr at nrlmry.navy.mil>
> Subject: Re: [rt.rap.ucar.edu #95099] No. YES per forecast
probability
> threshold
>
> Efren,
>
> MET verifies probability forecasts using an Nx2 probabilistic
> contingency table. The user defines the probability bins by setting
the "cat_thresh"
> in the "fcst" dictionary of the configuration file. And the user
> defines the event threshold by setting the "cat_thresh" in the "obs"
> dictionary of the configuration file.
>
> For example, when verifying a probability of precipitation forecast,
> you might set...
> fcst = {
> ...
> cat_thresh = [ ==0.1 ];
> }
> obs = {
> ...
> cat_thresh = [ >0 ];
> }
>
> This specifies that we want 10, evenly-spaced, probability bins (0
to
> 0.1,
> 0.1 to 0.2, and so on). And defines the event for the observation
as
> precip amounts greater than 0.
>
> So this yields a 10x2 contingency table. For each matched pair, MET
> bins the forecast probability value to determine the row of the
table.
> And then it checks if the event did or did not actually occur in the
> observation to determine the column. It processes all the matched
> pairs in this way to populate the counts for each cell of that 10x2
> table. All of the probabilistic stats that MET reports are based on
> the counts of this Nx2 table. And those probabilistic outputs are
> spread across 4 lines types (PCT, PSTD, PJC, and PRC). The contents
> of each of these line types is described in tables 7.10 through 7.13
of the MET User's Guide:
>
> https://dtcenter.org/sites/default/files/community-
code/met/docs/user-
> guide/MET_Users_Guide_v8.1.2.pdf
>
> You're asking about the raw counts, which are contained in the PCT
> line type. Please see table 7.10 for a description.
>
> Also, be sure you're actually requesting PCT output in Point-Stat
> config file by setting:
> output_flag = {
> ...
> pct = BOTH; // or just STAT
> ...
> }
>
> Thanks,
> John
>
> On Thu, Apr 30, 2020 at 11:55 AM Minna Win via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > Thu Apr 30 11:54:53 2020: Request 95099 was acted upon.
> > Transaction: Given to johnhg (John Halley Gotway) by minnawin
> > Queue: met_help
> > Subject: No. YES per forecast probability threshold
> > Owner: johnhg
> > Requestors: efren.serra.ctr at nrlmry.navy.mil
> > Status: open
> > Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95099
> > >
> >
> >
> > This transaction appears to have no content
> >
>
>
>
------------------------------------------------
More information about the Met_help
mailing list