[Met_help] [rt.rap.ucar.edu #98382] History for Fractions Skill Score Computation
John Halley Gotway via RT
met_help at ucar.edu
Wed Feb 10 09:08:35 MST 2021
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hi John,
I know we discussed this topic a couple of years ago dealing with the computation of Fractions Skill Score (FSS). I remember you had made some modifications to make it more flexible in generating the neighborhood fractional fields based on whether the neighborhood probabilities for the forecast or observations were already created out of MET. When I look back at the documentation, I noticed there is a field entry for the nbrhd dictionary that can be set to BOTH, FCST, OBS, or NONE (where BOTH is the default application). If we already have neighborhood ensemble probabilities from a global or convection allowing ensemble, then we would set the field flag to OBS or NONE (depending upon whether we need to compute fractional coverage of an event still for the observations). Would that be the correct interpretation?
Thanks,
Chris
Dr. Christopher J. Melick, DAF Civilian
Meteorologist, Data Sciences and Analytics Flight
16th Weather Squadron (16 WS/WXD)
557th Weather Wing, Offutt AFB, NE
DSN 271-3693; Comm (402) 294-3693
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: RE: [Non-DoD Source] [rt.rap.ucar.edu #98382] AutoReply: Fractions Skill Score Computation
From: christopher.melick at us.af.mil
Time: Mon Jan 25 13:07:09 2021
Hi John,
I think I need to clarify my query. Is there a way to have forecast
ensemble probability input to the grid_stat tool for the neighborhood
dictionary? In that case, the threshold probability was pre-computed
for the grid point but needs to be generated for the neighborhood
(e.g. what would be the NEP in Schwartz and Sobash (2017)). Wasn't
there a way to do neighborhood averaging? I'm wondering if there is
a threshold for both the observations and forecast? It seems live the
coverage threshold is applied to both.
Thanks,
Chris
-----Original Message-----
From: met_help at ucar.edu via RT <met_help at ucar.edu>
Sent: Monday, January 25, 2021 1:00 PM
To: MELICK, CHRISTOPHER J GS-12 USAF ACC 16 WS/WXD
<christopher.melick at us.af.mil>
Subject: [Non-DoD Source] [rt.rap.ucar.edu #98382] AutoReply:
Fractions Skill Score Computation
Greetings,
This message has been automatically generated in response to the
creation of a trouble ticket regarding:
"Fractions Skill Score Computation",
a summary of which appears below.
There is no need to reply to this message right now. Your ticket has
been assigned an ID of [rt.rap.ucar.edu #98382].
Please include the string:
[rt.rap.ucar.edu #98382]
in the subject line of all future correspondence about this issue. To
do so, you may reply to this message.
For more information, please see:
MET Online Tutorial:
https://www.dtcenter.org/met/users/support/online_tutorial/index.php
MET Users Guide:
https://www.dtcenter.org/met/users/docs/overview.php
MET FAQs:
https://www.dtcenter.org/met/users/support/faqs/index.php
MET-Help Email Archive:
http://mailman.ucar.edu/pipermail/met_help
Thank you,
met_help at ucar.edu
-------------------------------------------------------------------------
Hi John,
I know we discussed this topic a couple of years ago dealing with the
computation of Fractions Skill Score (FSS). I remember you had made
some modifications to make it more flexible in generating the
neighborhood fractional fields based on whether the neighborhood
probabilities for the forecast or observations were already created
out of MET. When I look back at the documentation, I noticed there
is a field entry for the nbrhd dictionary that can be set to BOTH,
FCST, OBS, or NONE (where BOTH is the default application). If we
already have neighborhood ensemble probabilities from a global or
convection allowing ensemble, then we would set the field flag to OBS
or NONE (depending upon whether we need to compute fractional coverage
of an event still for the observations). Would that be the correct
interpretation?
Thanks,
Chris
Dr. Christopher J. Melick, DAF Civilian
Meteorologist, Data Sciences and Analytics Flight 16th Weather
Squadron (16 WS/WXD) 557th Weather Wing, Offutt AFB, NE DSN 271-3693;
Comm (402) 294-3693
------------------------------------------------
Subject: Fractions Skill Score Computation
From: John Halley Gotway
Time: Thu Jan 28 13:24:49 2021
Chris,
Sorry for the delay in responding about this issue. I've had a busy
week so
far.
So you're asking about configuration options in the Grid-Stat tool to
skip
over the computation of the fractional coverage field within the tool
and
instead, interpret the input values as being pre-computed fractional
coverage fields. I vaguely remember this detail, but was glad to find
info
about it in the User's Guide:
https://dtcenter.github.io/MET/latest/Users_Guide/grid-stat.html
Search for "nbrhd", you'll find this info about the "nbrhd.field"
config
file option:
The *field* entry is set to *BOTH, FCST, OBS*, or *NONE* to indicate
the
fields to which the fractional coverage derivation logic should be
applied.
This should always to be set to *BOTH* unless you have already
computed the
fractional coverage field(s) with numbers between 0 and 1 outside of
MET.
So are you finding that when you set "field = NONE;" in the nbrhd
dictionary, it is not behaving how you'd expect?
Thanks,
John
On Wed, Jan 27, 2021 at 10:34 AM George McCabe via RT
<met_help at ucar.edu>
wrote:
>
> Wed Jan 27 10:34:48 2021: Request 98382 was acted upon.
> Transaction: Given to johnhg (John Halley Gotway) by mccabe
> Queue: met_help
> Subject: Fractions Skill Score Computation
> Owner: johnhg
> Requestors: christopher.melick at us.af.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98382 >
>
>
> This transaction appears to have no content
>
------------------------------------------------
Subject: Fractions Skill Score Computation
From: christopher.melick at us.af.mil
Time: Thu Jan 28 13:56:29 2021
John,
No worries. I understand.
I don't think we have actually tried setting the field to NONE yet but
I did see it in the documentation. Is there an example that has been
attempted before? I think this is the first time we have done
neighborhood verification on ensemble probabilities in MET. Was some
other functionality placed for MET to do NEP/NMEP type statistics like
FSS in version 9.x? I see something of added capability with the
neighborhood dictionary.
Thanks,
Chris
________________________________
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Thursday, January 28, 2021 2:24 PM
To: MELICK, CHRISTOPHER J GS-12 USAF ACC 16 WS/WXD
<christopher.melick at us.af.mil>
Cc: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXD
<robert.craig.2 at us.af.mil>; LACK, STEVEN A GS-13 USAF ACC 16 WS/WXD
<steven.lack at us.af.mil>; SEDLACEK, WILLIAM T GS-12 USAF ACC 16 WS/WXD
<william.sedlacek at us.af.mil>
Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #98382] Fractions Skill
Score Computation
Chris,
Sorry for the delay in responding about this issue. I've had a busy
week so
far.
So you're asking about configuration options in the Grid-Stat tool to
skip
over the computation of the fractional coverage field within the tool
and
instead, interpret the input values as being pre-computed fractional
coverage fields. I vaguely remember this detail, but was glad to find
info
about it in the User's Guide:
https://dtcenter.github.io/MET/latest/Users_Guide/grid-stat.html
Search for "nbrhd", you'll find this info about the "nbrhd.field"
config
file option:
The *field* entry is set to *BOTH, FCST, OBS*, or *NONE* to indicate
the
fields to which the fractional coverage derivation logic should be
applied.
This should always to be set to *BOTH* unless you have already
computed the
fractional coverage field(s) with numbers between 0 and 1 outside of
MET.
So are you finding that when you set "field = NONE;" in the nbrhd
dictionary, it is not behaving how you'd expect?
Thanks,
John
On Wed, Jan 27, 2021 at 10:34 AM George McCabe via RT
<met_help at ucar.edu>
wrote:
>
> Wed Jan 27 10:34:48 2021: Request 98382 was acted upon.
> Transaction: Given to johnhg (John Halley Gotway) by mccabe
> Queue: met_help
> Subject: Fractions Skill Score Computation
> Owner: johnhg
> Requestors: christopher.melick at us.af.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98382 >
>
>
> This transaction appears to have no content
>
------------------------------------------------
Subject: Fractions Skill Score Computation
From: John Halley Gotway
Time: Thu Jan 28 14:44:10 2021
Chris,
Looks like those changes in regards to NEP and NMEP were added for
version
9.0:
http://dtcenter.org/community-code/model-evaluation-tools-met/met-
version-9-0#notes
Here's the corresponding GitHub issue for that work:
https://github.com/dtcenter/MET/issues/1089
And you can read about the NEP and NMEP Ensemble-Stat configuration
options
here:
https://dtcenter.github.io/MET/latest/Users_Guide/ensemble-
stat.html#practical-information
Hope that helps.
Thanks,
John
On Thu, Jan 28, 2021 at 1:57 PM christopher.melick at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98382 >
>
> John,
>
> No worries. I understand.
>
> I don't think we have actually tried setting the field to NONE yet
but I
> did see it in the documentation. Is there an example that has been
> attempted before? I think this is the first time we have done
neighborhood
> verification on ensemble probabilities in MET. Was some other
> functionality placed for MET to do NEP/NMEP type statistics like FSS
in
> version 9.x? I see something of added capability with the
neighborhood
> dictionary.
>
> Thanks,
> Chris
> ________________________________
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Thursday, January 28, 2021 2:24 PM
> To: MELICK, CHRISTOPHER J GS-12 USAF ACC 16 WS/WXD <
> christopher.melick at us.af.mil>
> Cc: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXD
<robert.craig.2 at us.af.mil>;
> LACK, STEVEN A GS-13 USAF ACC 16 WS/WXD <steven.lack at us.af.mil>;
> SEDLACEK, WILLIAM T GS-12 USAF ACC 16 WS/WXD
<william.sedlacek at us.af.mil>
> Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #98382] Fractions
Skill
> Score Computation
>
> Chris,
>
> Sorry for the delay in responding about this issue. I've had a busy
week so
> far.
>
> So you're asking about configuration options in the Grid-Stat tool
to skip
> over the computation of the fractional coverage field within the
tool and
> instead, interpret the input values as being pre-computed fractional
> coverage fields. I vaguely remember this detail, but was glad to
find info
> about it in the User's Guide:
>
> https://dtcenter.github.io/MET/latest/Users_Guide/grid-stat.html
>
> Search for "nbrhd", you'll find this info about the "nbrhd.field"
config
> file option:
>
> The *field* entry is set to *BOTH, FCST, OBS*, or *NONE* to indicate
the
> fields to which the fractional coverage derivation logic should be
applied.
> This should always to be set to *BOTH* unless you have already
computed the
> fractional coverage field(s) with numbers between 0 and 1 outside of
MET.
>
> So are you finding that when you set "field = NONE;" in the nbrhd
> dictionary, it is not behaving how you'd expect?
>
> Thanks,
> John
>
> On Wed, Jan 27, 2021 at 10:34 AM George McCabe via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > Wed Jan 27 10:34:48 2021: Request 98382 was acted upon.
> > Transaction: Given to johnhg (John Halley Gotway) by mccabe
> > Queue: met_help
> > Subject: Fractions Skill Score Computation
> > Owner: johnhg
> > Requestors: christopher.melick at us.af.mil
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98382 >
> >
> >
> > This transaction appears to have no content
> >
>
>
>
------------------------------------------------
More information about the Met_help
mailing list