[Met_help] [rt.rap.ucar.edu #74421] History for Obtaining Heidke Skill Score through MET
John Halley Gotway via RT
met_help at ucar.edu
Tue Jan 26 12:13:29 MST 2016
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Dear Met Help,
I successfully ran MET after initial issues (timely addressed by John in #74338). However, I need the Heidke Skill Score, and the file contains such column, but with a value of "NA". Is there any way you can help me to get those values from the files I sent before?
Best Regards,
Jose M. Garcia-Rivera, PhD
I.M. Systems Group
This e-mail, including attachments, may include confidential and/or proprietary information, and may be used only by the person or entity to which it is addressed. If the reader of this e-mail is not the intended recipient or his or her authorized agent, the reader is hereby notified that any dissemination, distribution or copying of this e-mail is prohibited. If you have received this e-mail in error, please notify the sender by replying to this message and delete this e-mail immediately.
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Obtaining Heidke Skill Score through MET
From: John Halley Gotway
Time: Wed Dec 16 10:32:19 2015
Jose,
You can find a description of Heidke Skill Score in Appendix C of the
MET
User's Guide:
http://www.dtcenter.org/met/users/docs/users_guide/MET_Users_Guide_v5.1.pdf
HSS can be computed for a 2x2 contingency table and, more generally,
for a
NxN contingency table. That's why HSS shows up in the output CTS (2x2
contingency table statistics) line type and also in the MCTS
(multi-category contingency table statistics) line type.
Since its computation is based on thresholding the forecast and
observation
values, it is affected by that choice of threshold. In the example
you
sent me, you were applying thresholds of 0 and 5 to 2-meter
temperature in
Kelvin. Looking at the 2x2 contingency tables in the output of Grid-
Stat,
all matched pairs count as hits for those choice of thresholds.
That's a
degenerate case which results in an HSS value of NA. We were probably
dividing by 0 somewhere along the way.
Here are the HSS values you get for a better choice of thresholds for
2-meter temperature:
FCST_THRESH HSS
>=273 0
>=278 0.55952
>=293 0.9146
>=298 0.1977
Using those 4 thresholds to define a 5x5 contingency table, you get
the
following HSS value in the MCTS output line: 0.79004
I've attached the grid_stat configuration file with these choice of
thresholds.
Hope that helps.
Thanks,
John
On Wed, Dec 16, 2015 at 9:31 AM, Jose Garcia-Rivera via RT <
met_help at ucar.edu> wrote:
>
> Wed Dec 16 09:31:15 2015: Request 74421 was acted upon.
> Transaction: Ticket created by GarciaJ at imsg.com
> Queue: met_help
> Subject: Obtaining Heidke Skill Score through MET
> Owner: Nobody
> Requestors: GarciaJ at imsg.com
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74421 >
>
>
> Dear Met Help,
>
> I successfully ran MET after initial issues (timely addressed by
John in
> #74338). However, I need the Heidke Skill Score, and the file
contains such
> column, but with a value of "NA". Is there any way you can help me
to get
> those values from the files I sent before?
>
> Best Regards,
> Jose M. Garcia-Rivera, PhD
> I.M. Systems Group
> This e-mail, including attachments, may include confidential and/or
> proprietary information, and may be used only by the person or
entity to
> which it is addressed. If the reader of this e-mail is not the
intended
> recipient or his or her authorized agent, the reader is hereby
notified
> that any dissemination, distribution or copying of this e-mail is
> prohibited. If you have received this e-mail in error, please notify
the
> sender by replying to this message and delete this e-mail
immediately.
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #74421] Obtaining Heidke Skill Score through MET
From: Jose Garcia-Rivera
Time: Thu Dec 17 08:43:36 2015
Dear John,
Thank you so much for the response and help, it is truly appreciated.
I got the HSS, and now understand how to set the thresholds in order
to obtain it. MET will certainly be helpful for our forecast
verification purposes.
Take care,
Jose
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Wednesday, December 16, 2015 12:32 PM
To: Jose Garcia-Rivera <GarciaJ at imsg.com>
Subject: Re: [rt.rap.ucar.edu #74421] Obtaining Heidke Skill Score
through MET
Jose,
You can find a description of Heidke Skill Score in Appendix C of the
MET User's Guide:
http://www.dtcenter.org/met/users/docs/users_guide/MET_Users_Guide_v5.1.pdf
HSS can be computed for a 2x2 contingency table and, more generally,
for a NxN contingency table. That's why HSS shows up in the output
CTS (2x2 contingency table statistics) line type and also in the MCTS
(multi-category contingency table statistics) line type.
Since its computation is based on thresholding the forecast and
observation values, it is affected by that choice of threshold. In
the example you sent me, you were applying thresholds of 0 and 5 to 2-
meter temperature in Kelvin. Looking at the 2x2 contingency tables in
the output of Grid-Stat, all matched pairs count as hits for those
choice of thresholds. That's a degenerate case which results in an
HSS value of NA. We were probably dividing by 0 somewhere along the
way.
Here are the HSS values you get for a better choice of thresholds for
2-meter temperature:
FCST_THRESH HSS
>=273 0
>=278 0.55952
>=293 0.9146
>=298 0.1977
Using those 4 thresholds to define a 5x5 contingency table, you get
the following HSS value in the MCTS output line: 0.79004
I've attached the grid_stat configuration file with these choice of
thresholds.
Hope that helps.
Thanks,
John
On Wed, Dec 16, 2015 at 9:31 AM, Jose Garcia-Rivera via RT <
met_help at ucar.edu> wrote:
>
> Wed Dec 16 09:31:15 2015: Request 74421 was acted upon.
> Transaction: Ticket created by GarciaJ at imsg.com
> Queue: met_help
> Subject: Obtaining Heidke Skill Score through MET
> Owner: Nobody
> Requestors: GarciaJ at imsg.com
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74421
> >
>
>
> Dear Met Help,
>
> I successfully ran MET after initial issues (timely addressed by
John
> in #74338). However, I need the Heidke Skill Score, and the file
> contains such column, but with a value of "NA". Is there any way you
> can help me to get those values from the files I sent before?
>
> Best Regards,
> Jose M. Garcia-Rivera, PhD
> I.M. Systems Group
> This e-mail, including attachments, may include confidential and/or
> proprietary information, and may be used only by the person or
entity
> to which it is addressed. If the reader of this e-mail is not the
> intended recipient or his or her authorized agent, the reader is
> hereby notified that any dissemination, distribution or copying of
> this e-mail is prohibited. If you have received this e-mail in
error,
> please notify the sender by replying to this message and delete this
e-mail immediately.
>
>
This e-mail, including attachments, may include confidential and/or
proprietary information, and may be used only by the person or entity
to which it is addressed. If the reader of this e-mail is not the
intended recipient or his or her authorized agent, the reader is
hereby notified that any dissemination, distribution or copying of
this e-mail is prohibited. If you have received this e-mail in error,
please notify the sender by replying to this message and delete this
e-mail immediately.
------------------------------------------------
Subject: Obtaining Heidke Skill Score through MET
From: John Halley Gotway
Time: Thu Dec 17 09:37:09 2015
Jose,
Great! Glad to hear it. I'll go ahead and resolve this ticket now.
Just let us know what other questions or issues arise in your use of
MET.
Thanks,
John
On Thu, Dec 17, 2015 at 8:43 AM, Jose Garcia-Rivera via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74421 >
>
> Dear John,
>
> Thank you so much for the response and help, it is truly
appreciated. I
> got the HSS, and now understand how to set the thresholds in order
to
> obtain it. MET will certainly be helpful for our forecast
verification
> purposes.
>
> Take care,
> Jose
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Wednesday, December 16, 2015 12:32 PM
> To: Jose Garcia-Rivera <GarciaJ at imsg.com>
> Subject: Re: [rt.rap.ucar.edu #74421] Obtaining Heidke Skill Score
> through MET
>
> Jose,
>
> You can find a description of Heidke Skill Score in Appendix C of
the MET
> User's Guide:
>
>
http://www.dtcenter.org/met/users/docs/users_guide/MET_Users_Guide_v5.1.pdf
>
> HSS can be computed for a 2x2 contingency table and, more generally,
for a
> NxN contingency table. That's why HSS shows up in the output CTS
(2x2
> contingency table statistics) line type and also in the MCTS
> (multi-category contingency table statistics) line type.
>
> Since its computation is based on thresholding the forecast and
> observation values, it is affected by that choice of threshold. In
the
> example you sent me, you were applying thresholds of 0 and 5 to 2-
meter
> temperature in Kelvin. Looking at the 2x2 contingency tables in the
output
> of Grid-Stat, all matched pairs count as hits for those choice of
> thresholds. That's a degenerate case which results in an HSS value
of NA.
> We were probably dividing by 0 somewhere along the way.
>
> Here are the HSS values you get for a better choice of thresholds
for
> 2-meter temperature:
>
> FCST_THRESH HSS
> >=273 0
> >=278 0.55952
> >=293 0.9146
> >=298 0.1977
>
> Using those 4 thresholds to define a 5x5 contingency table, you get
the
> following HSS value in the MCTS output line: 0.79004
>
> I've attached the grid_stat configuration file with these choice of
> thresholds.
>
> Hope that helps.
>
> Thanks,
> John
>
>
>
>
>
> On Wed, Dec 16, 2015 at 9:31 AM, Jose Garcia-Rivera via RT <
> met_help at ucar.edu> wrote:
>
> >
> > Wed Dec 16 09:31:15 2015: Request 74421 was acted upon.
> > Transaction: Ticket created by GarciaJ at imsg.com
> > Queue: met_help
> > Subject: Obtaining Heidke Skill Score through MET
> > Owner: Nobody
> > Requestors: GarciaJ at imsg.com
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=74421
> > >
> >
> >
> > Dear Met Help,
> >
> > I successfully ran MET after initial issues (timely addressed by
John
> > in #74338). However, I need the Heidke Skill Score, and the file
> > contains such column, but with a value of "NA". Is there any way
you
> > can help me to get those values from the files I sent before?
> >
> > Best Regards,
> > Jose M. Garcia-Rivera, PhD
> > I.M. Systems Group
> > This e-mail, including attachments, may include confidential
and/or
> > proprietary information, and may be used only by the person or
entity
> > to which it is addressed. If the reader of this e-mail is not the
> > intended recipient or his or her authorized agent, the reader is
> > hereby notified that any dissemination, distribution or copying of
> > this e-mail is prohibited. If you have received this e-mail in
error,
> > please notify the sender by replying to this message and delete
this
> e-mail immediately.
> >
> >
>
> This e-mail, including attachments, may include confidential and/or
> proprietary information, and may be used only by the person or
entity to
> which it is addressed. If the reader of this e-mail is not the
intended
> recipient or his or her authorized agent, the reader is hereby
notified
> that any dissemination, distribution or copying of this e-mail is
> prohibited. If you have received this e-mail in error, please notify
the
> sender by replying to this message and delete this e-mail
immediately.
>
>
>
------------------------------------------------
More information about the Met_help
mailing list