[Met_help] onto grid_stat: threat score and bias for precip

[John Halley Gotway]

Jonathan,

Since this question is more about methods and statistics, I'm copying
Tressa Fowler on this message.  She's the statistician who's leading the
MET development.  And she may have something to add.

I believe that people typically analyze precipitation by choosing one or
more threshold values of interest and computing contigency
tables/statistics using those thresholds.

When you run Grid-Stat, the Contingency Table Counts (CTC) lines contains
the raw counts for the threshold(s) you select.  And the Contingency Table
Statistics (CTS) lines contain the statistics that can be derived from
those counts - including CSI (threat score), GSS (equitable threat score),
and BIAS.  If you're looking at multiple forecast times, you could compute
the CTS scores for each time and then plot a time series of them.  Also,
you could aggregate the CTC counts through time using the STAT-Analysis
tool and derive the CTS statistics from those aggregated counts.  That
would give you a measure of "overall" performance through time.  I believe
it's the "stat_job_aggr_cts" job in the STAT-Analysis tool that performs
this function.

Please refer to appendix C of the MET User's Guide for a nice description
of the stats that are included in the output lines.

You may also find the following website of interest:
http://www.bom.gov.au/bmrc/wefor/staff/eee/verif/verif_web_page.html

Also, since you want to compare to MODE, please note that MODE itself does
actually create a file containing contingency table counts and statistics.
 We created this for users in exactly your situation, wondering how does
the output of MODE compare to the output of more traditional statistics? 
However, the MODE output currently contains no confidence intervals for
the stats.  Again, I'll refer you to the user's guide for details.

Hope that helps.

John

> John,
>
>
>
> I'm about to set up a run using grid_stat for accumulated precipitation.
>
>
>
>
> Before I get into the technical motions of running it, I was wondering
> how would one generate the equivalent of an overall threat score and
> precipitation bias using the grid_stat and stat_analysis utilities?  I'd
> like to generate some traditional precip scores and then compare those
> numbers to the mode output and mode_analysis to determine the relative
> quality of my WRF precip forecasts.
>
>
>
> Thanks again for your insight!
> Jonathan
>
>
>
> ***********************************************************
> Jonathan Case, ENSCO, Inc.
> Aerospace Sciences & Engineering Division
> Short-term Prediction Research and Transition Center
> 320 Sparkman Drive, Room 3062
> Huntsville, AL 35805-1912
> Voice: (256) 961-7504   Fax: (256) 961-7788
> Emails: Jonathan.Case-1 at nasa.gov
>
>              case.jonathan at ensco.com
>
> ***********************************************************
>
>
>
>