[Met_help] [rt.rap.ucar.edu #39740] History for Re: MODE contingency table statistics questions

Mon Aug 2 16:33:43 MDT 2010

----------------------------------------------------------------
  Initial Request
----------------------------------------------------------------

I am forwarding your email to MET group..

On Jul 30, 2010, at 1:27 PM, Jeffrey Duda wrote:

> Hello,
> I am confused on the definition of the variables forecast mean,  
> frequency bias, accuracy, and base rate in the *_cts.txt files that  
> are output from MODE.  My guess is that forecast mean (for the case  
> of precipitation forecast and observed data sets) is the average  
> precipitation amount per grid point, base rate is the average  
> observed precipitation amount per grid point, and frequency bias is  
> forecast mean / base rate, but that leaves accuracy a mystery to  
> me.  Am I right about these, and what is accuracy?
>
> Jeff Duda
>

wrfhelp
http://www.mmm.ucar.edu/wrf/users/supports/wrfhelp.html

----------------------------------------------------------------
  Complete Ticket History
----------------------------------------------------------------

Subject: Re: [rt.rap.ucar.edu #39740] Re: MODE contingency table statistics questions
From: John Halley Gotway
Time: Mon Aug 02 16:04:34 2010

Jeff,

We received your question about output from MODE via the WRFHelp
folks.  In the future, please send questions about MET directly to
"met_help at ucar.edu".

This output file is discussed on pages 6-19 and 6-20 of the MET User's
Guide
(http://www.dtcenter.org/met/users/docs/users_guide/MET_Users_Guide_v2.0_rev2.pdf).

Please be sure to make a special note of the "FIELD" column which is
set to RAW, FILTER, or OBJECT.  The data in the RAW line gives you the
contingency table information you'd get if you applied the
convolution thresholds directly to the raw fields.  The FILTER line
will contain the same values as the RAW line unless you've used
fcst_raw_thresh or obs_raw_thresh in the config file.  And lastly,
the OBJECT lines give you contingency table information for the fields
of resolved objects.  The object fields are treated as binary images
containing 0's and 1's and are scored like a contingency table.

To answer your specific questions...
Forecast mean, accuracy, base rate, and frequency bias are defined
from the contingency table, as described in the MET User's Guide in
Appendix C (page C-2).

Forecast mean is the percent of grid points at which the event was
forecasted.  In the OBJECT line, an event occurs if that grid point is
part of a forecasted object.

Base rate is the percent of grid points at which the observed event
occurred (i.e. it's part of an object).

Accuracy is the percent of grid points at which the forecast was
correct (correctly yes and correctly no).

Frequency bias is the ratio of the number of forecast events divided
by the number of observed events.

Hopefully that clears it up.

Thanks,
John Halley Gotway
met_help at ucar.edu

RAL HelpDesk {for wrfhelp at ucar.edu} wrote:
> Sun Aug 01 18:43:53 2010: Request 39740 was acted upon.
> Transaction: Ticket created by wrfhelp at ucar.edu
>        Queue: met_help
>      Subject: Re: MODE contingency table statistics questions
>        Owner: Nobody
>   Requestors: wrfhelp at ucar.edu
>       Status: new
>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=39740 >
>
>
> I am forwarding your email to MET group..
>
> On Jul 30, 2010, at 1:27 PM, Jeffrey Duda wrote:
>
>> Hello,
>> I am confused on the definition of the variables forecast mean,
>> frequency bias, accuracy, and base rate in the *_cts.txt files that
>> are output from MODE.  My guess is that forecast mean (for the case
>> of precipitation forecast and observed data sets) is the average
>> precipitation amount per grid point, base rate is the average
>> observed precipitation amount per grid point, and frequency bias is
>> forecast mean / base rate, but that leaves accuracy a mystery to
>> me.  Am I right about these, and what is accuracy?
>>
>> Jeff Duda
>>
>
> wrfhelp
> http://www.mmm.ucar.edu/wrf/users/supports/wrfhelp.html
>
>
>

------------------------------------------------
Subject: Re: MODE contingency table statistics questions
From: jdduda at iastate.edu
Time: Mon Aug 02 16:33:04 2010

John,
Yes, thanks for pointing me to that section of the users guide.  I
hadn't
looked there yet.

Jeff

On Mon, Aug 2, 2010 at 5:04 PM, RAL HelpDesk {for John Halley Gotway}
<
met_help at ucar.edu> wrote:

> Jeff,
>
> We received your question about output from MODE via the WRFHelp
folks.  In
> the future, please send questions about MET directly to
"met_help at ucar.edu
> ".
>
> This output file is discussed on pages 6-19 and 6-20 of the MET
User's
> Guide (
>
http://www.dtcenter.org/met/users/docs/users_guide/MET_Users_Guide_v2.0_rev2.pdf
> ).
>
> Please be sure to make a special note of the "FIELD" column which is
set to
> RAW, FILTER, or OBJECT.  The data in the RAW line gives you the
contingency
> table information you'd get if you applied the
> convolution thresholds directly to the raw fields.  The FILTER line
will
> contain the same values as the RAW line unless you've used
fcst_raw_thresh
> or obs_raw_thresh in the config file.  And lastly,
> the OBJECT lines give you contingency table information for the
fields of
> resolved objects.  The object fields are treated as binary images
containing
> 0's and 1's and are scored like a contingency table.
>
> To answer your specific questions...
> Forecast mean, accuracy, base rate, and frequency bias are defined
from the
> contingency table, as described in the MET User's Guide in Appendix
C (page
> C-2).
>
> Forecast mean is the percent of grid points at which the event was
> forecasted.  In the OBJECT line, an event occurs if that grid point
is part
> of a forecasted object.
>
> Base rate is the percent of grid points at which the observed event
> occurred (i.e. it's part of an object).
>
> Accuracy is the percent of grid points at which the forecast was
correct
> (correctly yes and correctly no).
>
> Frequency bias is the ratio of the number of forecast events divided
by the
> number of observed events.
>
> Hopefully that clears it up.
>
> Thanks,
> John Halley Gotway
> met_help at ucar.edu
>
> RAL HelpDesk {for wrfhelp at ucar.edu} wrote:
> > Sun Aug 01 18:43:53 2010: Request 39740 was acted upon.
> > Transaction: Ticket created by wrfhelp at ucar.edu
> >        Queue: met_help
> >      Subject: Re: MODE contingency table statistics questions
> >        Owner: Nobody
> >   Requestors: wrfhelp at ucar.edu
> >       Status: new
> >  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=39740 >
> >
> >
> > I am forwarding your email to MET group..
> >
> > On Jul 30, 2010, at 1:27 PM, Jeffrey Duda wrote:
> >
> >> Hello,
> >> I am confused on the definition of the variables forecast mean,
> >> frequency bias, accuracy, and base rate in the *_cts.txt files
that
> >> are output from MODE.  My guess is that forecast mean (for the
case
> >> of precipitation forecast and observed data sets) is the average
> >> precipitation amount per grid point, base rate is the average
> >> observed precipitation amount per grid point, and frequency bias
is
> >> forecast mean / base rate, but that leaves accuracy a mystery to
> >> me.  Am I right about these, and what is accuracy?
> >>
> >> Jeff Duda
> >>
> >
> > wrfhelp
> > http://www.mmm.ucar.edu/wrf/users/supports/wrfhelp.html
> >
> >
> >
>
>

--
Jeff Duda
Iowa State University
Meteorology Graduate Student
3134 Agronomy Hall
www.meteor.iastate.edu/~jdduda

------------------------------------------------