[Met_help] [rt.rap.ucar.edu #47128] History for METv3.0 Verification Range?

[RAL HelpDesk {for John Halley Gotway}]

----------------------------------------------------------------
  Initial Request
----------------------------------------------------------------

Does MET provide a way to verify a threshold range?  I see in the documentation that fcst_thresh and obs_thresh accept multiple thresholds as in "gt0 le0" when used within double quotes, but it doesn't appear that this represents a range of values.  For example, using "ge5 le10" does not verify the range of values that exist from 5 to 10.  Is there a simple way to verify values that fall within a range (0 to 5, 5 to 10, 10 to 15, etc.)?

Thanks,
Scott



----------------------------------------------------------------
  Complete Ticket History
----------------------------------------------------------------

Subject: Re: [rt.rap.ucar.edu #47128] METv3.0 Verification Range?
From: John Halley Gotway
Time: Thu May 26 10:34:14 2011

Scott,

The answer to your question is yes, MET does offer support for
verifying ranges of thresholds.  However, it may or may not produce
the type of output statistics you were expecting.

When you run Point-Stat or Grid-Stat, you can specify that a range of
thresholds be used.  If you turn on the output for the MCTC and/or the
MCTS line types (multi-category contingency table counts
and statistics) in the "output_flag" parameter in the configuration
file, Point-Stat and Grid-Stat will use those thresholds to define an
NxN contingency table.  And it'll dump the corresponding
counts to an MCTC output line, and the corresponding statistics to an
MCTS output line.

The types of statistics available for a multi-category contingency
table are rather limited - accuracy, hannsen-kuipers, heidke skill
score, and the gerrity score.  Please refer to the MET User's
Guide for more detail on these statistics.  Also, please refer to the
MET test scripts for an example of producing MCTC and MCTS output.
Specifically, look in the 3rd call to grid_stat in the
"METv3.0.1/scripts/test_grid_stat.sh" script.

If you've already run the test scripts, you can see the MCTC and MCTS
output to which I'm referring in these files:
   METv3.0.1/out/grid_stat/grid_stat_APCP_24_240000L_20050808_000000V_mctc.txt
   METv3.0.1/out/grid_stat/grid_stat_APCP_24_240000L_20050808_000000V_mcts.txt

One thing to note is that when you're specifying thresholds for these,
the threshold-types must be consistent.  For example, the following
thresholds would work because they partition the forecast space:
   fcst_thresh[] = [ "gt0.0 gt5.0 gt10.0" ];

However, the following would produce an error because the thresholds
overlap:
   fcst_thresh[] = [ "gt0.0 gt5.0 le10.0" ];

I've copied Tressa Fowler on this message - our resident statistician.
If you have trouble interpreting the multi-category statistics, just
respond to this message, and hopefully Tressa could provide
some guidance.

Hope that helps.

John Halley Gotway
met_help at ucar.edu

On 05/25/2011 11:22 AM, RAL HelpDesk {for Dembek, Scott
Robert[Universities Space Research Association]} wrote:
>
> Wed May 25 11:22:28 2011: Request 47128 was acted upon.
> Transaction: Ticket created by scott.r.dembek at nasa.gov
>        Queue: met_help
>      Subject: METv3.0 Verification Range?
>        Owner: Nobody
>   Requestors: scott.r.dembek at nasa.gov
>       Status: new
>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47128 >
>
>
> Does MET provide a way to verify a threshold range?  I see in the
documentation that fcst_thresh and obs_thresh accept multiple
thresholds as in "gt0 le0" when used within double quotes, but it
doesn't appear that this represents a range of values.  For example,
using "ge5 le10" does not verify the range of values that exist from 5
to 10.  Is there a simple way to verify values that fall within a
range (0 to 5, 5 to 10, 10 to 15, etc.)?
>
> Thanks,
> Scott
>

------------------------------------------------