[Met_help] [rt.rap.ucar.edu #97988] History for MET functionalities

[John Halley Gotway via RT]

----------------------------------------------------------------
  Initial Request
----------------------------------------------------------------

Hi,
May I have some questions about MET functionalities?
1. May I know how to config wind speed verification?
2. Will MET include verification of wind direction in the future?
3. Will MET support ROC curves based on multi-thresholds of observation
data?
    Right now, MET creates ROC curves based on a single obs_thresh.
    I have observation data containing 4 category values, trace, light,
moderate, severe. For each category of these four, I understand MET
supports creating STAT file and ROC curves. My question is: will MET
support creating ROC curves based on all categories, in other words, will
MET support ROC curves on multiple obs_thresh?
Thank you.
Yali


----------------------------------------------------------------
  Complete Ticket History
----------------------------------------------------------------

Subject: MET functionalities
From: Julie Prestopnik
Time: Tue Dec 22 11:01:55 2020

Hi Yali.

I am assigning your ticket to John HG, who I believe can best answer
your
questions.  Please allow a few business days for a response.  Thanks!

Julie

On Tue, Dec 22, 2020 at 10:57 AM Yali Mao - NOAA Affiliate via RT <
met_help at ucar.edu> wrote:

>
> Tue Dec 22 10:57:14 2020: Request 97988 was acted upon.
> Transaction: Ticket created by yali.mao at noaa.gov
>        Queue: met_help
>      Subject: MET functionalities
>        Owner: Nobody
>   Requestors: yali.mao at noaa.gov
>       Status: new
>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97988 >
>
>
> Hi,
> May I have some questions about MET functionalities?
> 1. May I know how to config wind speed verification?
> 2. Will MET include verification of wind direction in the future?
> 3. Will MET support ROC curves based on multi-thresholds of
observation
> data?
>     Right now, MET creates ROC curves based on a single obs_thresh.
>     I have observation data containing 4 category values, trace,
light,
> moderate, severe. For each category of these four, I understand MET
> supports creating STAT file and ROC curves. My question is: will MET
> support creating ROC curves based on all categories, in other words,
will
> MET support ROC curves on multiple obs_thresh?
> Thank you.
> Yali
>
>

--
Julie Prestopnik (she/her)
Software Engineer
National Center for Atmospheric Research
Research Applications Laboratory
Email: jpresto at ucar.edu

My working day may not be your working day.  Please do not feel
obliged to
reply to this email outside of your normal working hours.

------------------------------------------------
Subject: MET functionalities
From: John Halley Gotway
Time: Tue Dec 22 11:18:50 2020

Yali, answers are inline:

May I have some questions about MET functionalities?
1. May I know how to config wind speed verification?

*In the METplus config file, just request that the variable named
"WIND" be
verified. That is the GRIB abbreviation for wind speed. Assuming
you're
using GRIB1 or GRIB2 input, MET will first attempt to read WIND
directly
from the input files. If not present, it'll automatically read the
corresponding UGRD and VGRD records and derive WIND on the fly. Debub
level
3 or 4 log messages will tell you what it's doing.*

2. Will MET include verification of wind direction in the future?

*Some statistics related to wind direction errors are included in the
existing VCNT output line type
(https://dtcenter.github.io/MET/latest/Users_Guide/point-
stat.html#id23
<https://dtcenter.github.io/MET/latest/Users_Guide/point-
stat.html#id23>).
In addition, here is a feature request from NOAA/MDL to verify wind
direction in a categorical way:
https://github.com/dtcenter/MET/issues/1236
<https://github.com/dtcenter/MET/issues/1236>*
*Are these approaches sufficient or do you have additional
verification
metrics for wind direction in mind?*

3. Will MET support ROC curves based on multi-thresholds of
observation
data?
    Right now, MET creates ROC curves based on a single obs_thresh.
    I have observation data containing 4 category values, trace,
light,
moderate, severe. For each category of these four, I understand MET
supports creating STAT file and ROC curves. My question is: will MET
support creating ROC curves based on all categories, in other words,
will
MET support ROC curves on multiple obs_thresh?

*METviewer might already support this, but I don't have much
experience
generating ROC curves in this way. I've attached a screenshot of the
METviewer ROC template. Under the "ROC Calculations" tab, you can
select
"PCT" or "CTC". PCT stands for probabilistic contingency table and is
how
ROC curves are created for probabilistic forecasts. CTC stands for
contingency table and is intended for computing ROC curves from
categorical
data.*

*The question is whether or not the CTC option satisfies your needs.
If
not, we should set up a meeting with Tatiana Burek (the main METviewer
developer), Eric Gilleland (a DTC statistician), Tara Jensen (project
manager), and myself to discuss exactly what additional functionality
is
needed and write it up as a GitHub issue so that the METviewer
developers
know exactly what to change.*

Thanks,
John Halley Gotway

[image: Screen Shot 2020-12-22 at 11.13.53 AM.png]

On Tue, Dec 22, 2020 at 11:02 AM Julie Prestopnik via RT
<met_help at ucar.edu>
wrote:

>
> Tue Dec 22 11:02:02 2020: Request 97988 was acted upon.
> Transaction: Given to johnhg (John Halley Gotway) by jpresto
>        Queue: met_help
>      Subject: MET functionalities
>        Owner: johnhg
>   Requestors: yali.mao at noaa.gov
>       Status: open
>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97988 >
>
>
> This transaction appears to have no content
>

------------------------------------------------