[Met_help] [rt.rap.ucar.edu #98664] History for FAR and POFD

[Minna Win via RT]

----------------------------------------------------------------
  Initial Request
----------------------------------------------------------------

Good evening,

We are in the process of incorporating FAR and POFD into our analysis.

In metviewer FAR stands for false alarm ratio.  However, some of us are
concerned that it is false alarm rate.  POFD looks as expected.  The
statistics converge to zero as there are less cases.  The divergence
between runs increases at the low-end as there are more cases (second
attachment).  Some of us think that the first plot is false alarm rate
rather than false alarm ratio. We would expect a higher rate of detecting a
false alarm for extreme events.  We believe that POFD and FAR should be
similar.  Can you confirm?


----------------------------------------------------------------
  Complete Ticket History
----------------------------------------------------------------

Subject: FAR and POFD
From: Minna Win
Time: Thu Feb 11 16:26:17 2021

Hello Ed,

Is the information in the MET documentation what you need:

Appendix C:
https://dtcenter.github.io/MET/develop/Users_Guide/appendixC.html#false-
alarm-ratio-far

And the function in MET where far() is defined:
https://github.com/dtcenter/MET/blob/0fe2e42609120df236f74ae23c07f3fa4b930247/met/src/libcode/vx_statistics/contable_stats.cc#L205


Regards,
Minna
---------------
Minna Win
National Center for Atmospheric Research
Developmental Testbed Center
Phone: 303-497-8423
Fax:   303-497-8401
---------------
Pronouns: she/her


On Thu, Feb 11, 2021 at 3:28 PM Edward Strobach - NOAA Affiliate via
RT <
met_help at ucar.edu> wrote:

>
> Thu Feb 11 15:28:18 2021: Request 98664 was acted upon.
> Transaction: Ticket created by edward.strobach at noaa.gov
>        Queue: met_help
>      Subject: FAR and POFD
>        Owner: Nobody
>   Requestors: edward.strobach at noaa.gov
>       Status: new
>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98664 >
>
>
> Good evening,
>
> We are in the process of incorporating FAR and POFD into our
analysis.
>
> In metviewer FAR stands for false alarm ratio.  However, some of us
are
> concerned that it is false alarm rate.  POFD looks as expected.  The
> statistics converge to zero as there are less cases.  The divergence
> between runs increases at the low-end as there are more cases
(second
> attachment).  Some of us think that the first plot is false alarm
rate
> rather than false alarm ratio. We would expect a higher rate of
detecting a
> false alarm for extreme events.  We believe that POFD and FAR should
be
> similar.  Can you confirm?
>
>

------------------------------------------------
Subject: FAR and POFD
From: Tara Jensen
Time: Fri Feb 12 09:50:18 2021

Hi Ed,

If you refer to the Intro to Categorical Statistics presentation Slide
16
in our full in-person tutorial, you'll see that you are correct in
assuming
POFD = False Alarm Rate.  Here's the link to our most recent set of
presentations.  Hopefully this will also help you determine which
statistics to use.

http://www.dtcenter.org/community-code/model-evaluation-tools-met/met-
tutorial-presentations

Cheers, Tara

On Thu, Feb 11, 2021 at 4:26 PM Minna Win via RT <met_help at ucar.edu>
wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98664 >
>
> Hello Ed,
>
> Is the information in the MET documentation what you need:
>
> Appendix C:
>
>
https://dtcenter.github.io/MET/develop/Users_Guide/appendixC.html#false-
alarm-ratio-far
>
> And the function in MET where far() is defined:
>
>
https://github.com/dtcenter/MET/blob/0fe2e42609120df236f74ae23c07f3fa4b930247/met/src/libcode/vx_statistics/contable_stats.cc#L205
>
>
> Regards,
> Minna
> ---------------
> Minna Win
> National Center for Atmospheric Research
> Developmental Testbed Center
> Phone: 303-497-8423
> Fax:   303-497-8401
> ---------------
> Pronouns: she/her
>
>
> On Thu, Feb 11, 2021 at 3:28 PM Edward Strobach - NOAA Affiliate via
RT <
> met_help at ucar.edu> wrote:
>
> >
> > Thu Feb 11 15:28:18 2021: Request 98664 was acted upon.
> > Transaction: Ticket created by edward.strobach at noaa.gov
> >        Queue: met_help
> >      Subject: FAR and POFD
> >        Owner: Nobody
> >   Requestors: edward.strobach at noaa.gov
> >       Status: new
> >  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98664 >
> >
> >
> > Good evening,
> >
> > We are in the process of incorporating FAR and POFD into our
analysis.
> >
> > In metviewer FAR stands for false alarm ratio.  However, some of
us are
> > concerned that it is false alarm rate.  POFD looks as expected.
The
> > statistics converge to zero as there are less cases.  The
divergence
> > between runs increases at the low-end as there are more cases
(second
> > attachment).  Some of us think that the first plot is false alarm
rate
> > rather than false alarm ratio. We would expect a higher rate of
> detecting a
> > false alarm for extreme events.  We believe that POFD and FAR
should be
> > similar.  Can you confirm?
> >
> >
>
>

--
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Tara Jensen
Project Manager II
NCAR RAL and DTC
PO Box 3000, Boulder, Colorado 80307 USA
+1 303-497-8479          jensen at ucar.edu

------------------------------------------------