[Met_help] [rt.rap.ucar.edu #83246] History for how to calculate ensemble evaluation with MET Tools?

John Halley Gotway via RT met_help at ucar.edu
Tue Jul 9 12:04:02 MDT 2019


----------------------------------------------------------------
  Initial Request
----------------------------------------------------------------

Hi All,

    From the ensemble_stat, the output only includes some simple evaluation as rank histogram, ensemble mean.  From the point_stat, we can get more evaluation,  but it is limited to the single member. How can I use the MET-tools to calculate the ensemble evaluations?

  Can anyone give me a guide?

Thank you very much!

Mandy


----------------------------------------------------------------
  Complete Ticket History
----------------------------------------------------------------

Subject: how to calculate ensemble evaluation with MET Tools?
From: John Halley Gotway
Time: Fri Dec 08 14:28:21 2017

Mandy,

Typically, when doing ensemble verification, the flow of our logic is
this...

(1) For a given forecast time, identify the N ensemble members.
(2) Pass the N members and verifying observations (gridded and/or
points)
to Ensemble-Stat.
(3) Ensemble-Stat generates verification statistics such as ranked
histograms (RHIST), pit histograms (PHIST), and spread-skill (SSVAR).
(4) Ensemble-Stat also writes out a gridded NetCDF file containing the
ensemble mean and ensemble relative frequency (i.e. like
probabilities).
(5) Pass each of the N ensemble members through grid_stat or
point_stat to
verify it as a deterministic forecast.
(6) Pass the ensemble mean through grid_stat or point_stat to verify
it as
a deterministic forecast.
(7) Pass the ensemble frequencies through grid_stat or point_stat to
verify
the probability data.

Hope that helps describe the logical flow.

Thanks,
John

On Fri, Dec 8, 2017 at 10:21 AM, Zhang, Mandy via RT
<met_help at ucar.edu>
wrote:

>
> Fri Dec 08 10:21:16 2017: Request 83246 was acted upon.
> Transaction: Ticket created by Man.Zhang-1 at ou.edu
>        Queue: met_help
>      Subject: how to calculate ensemble evaluation with MET Tools?
>        Owner: Nobody
>   Requestors: Man.Zhang-1 at ou.edu
>       Status: new
>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83246 >
>
>
> Hi All,
>
>     From the ensemble_stat, the output only includes some simple
> evaluation as rank histogram, ensemble mean.  From the point_stat,
we can
> get more evaluation,  but it is limited to the single member. How
can I use
> the MET-tools to calculate the ensemble evaluations?
>
>   Can anyone give me a guide?
>
> Thank you very much!
>
> Mandy
>
>

------------------------------------------------
Subject: how to calculate ensemble evaluation with MET Tools?
From: Zhang, Mandy
Time: Fri Dec 08 14:57:50 2017

Hi John,

     Can I use Stat_Ananlysis tool to combine the results of
ensemble_stat and Point_stat to get the verification of probability
data?

     Thank you very much!

Mandy


________________________________
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Friday, December 8, 2017 3:28:21 PM
To: Zhang, Mandy
Subject: Re: [rt.rap.ucar.edu #83246] how to calculate ensemble
evaluation with MET Tools?

Mandy,

Typically, when doing ensemble verification, the flow of our logic is
this...

(1) For a given forecast time, identify the N ensemble members.
(2) Pass the N members and verifying observations (gridded and/or
points)
to Ensemble-Stat.
(3) Ensemble-Stat generates verification statistics such as ranked
histograms (RHIST), pit histograms (PHIST), and spread-skill (SSVAR).
(4) Ensemble-Stat also writes out a gridded NetCDF file containing the
ensemble mean and ensemble relative frequency (i.e. like
probabilities).
(5) Pass each of the N ensemble members through grid_stat or
point_stat to
verify it as a deterministic forecast.
(6) Pass the ensemble mean through grid_stat or point_stat to verify
it as
a deterministic forecast.
(7) Pass the ensemble frequencies through grid_stat or point_stat to
verify
the probability data.

Hope that helps describe the logical flow.

Thanks,
John

On Fri, Dec 8, 2017 at 10:21 AM, Zhang, Mandy via RT
<met_help at ucar.edu>
wrote:

>
> Fri Dec 08 10:21:16 2017: Request 83246 was acted upon.
> Transaction: Ticket created by Man.Zhang-1 at ou.edu
>        Queue: met_help
>      Subject: how to calculate ensemble evaluation with MET Tools?
>        Owner: Nobody
>   Requestors: Man.Zhang-1 at ou.edu
>       Status: new
>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83246 >
>
>
> Hi All,
>
>     From the ensemble_stat, the output only includes some simple
> evaluation as rank histogram, ensemble mean.  From the point_stat,
we can
> get more evaluation,  but it is limited to the single member. How
can I use
> the MET-tools to calculate the ensemble evaluations?
>
>   Can anyone give me a guide?
>
> Thank you very much!
>
> Mandy
>
>


------------------------------------------------
Subject: how to calculate ensemble evaluation with MET Tools?
From: John Halley Gotway
Time: Fri Dec 08 15:03:19 2017

Mandy,

Point-Stat and Grid-Stat are used to verify probability forecasts.
The
probability vx output is stored in PCT line type (probabilistic
contingency
table).  The Ensemble-Stat tool writes a different set of line types.

However, the .stat output file from Point-Stat, Grid-Stat, and
Ensemble-Stat are all readable by the same STAT-Analysis tool.

Hopefully that helps clarify.

Thanks,
John

On Fri, Dec 8, 2017 at 2:57 PM, Zhang, Mandy via RT
<met_help at ucar.edu>
wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83246 >
>
> Hi John,
>
>      Can I use Stat_Ananlysis tool to combine the results of
ensemble_stat
> and Point_stat to get the verification of probability data?
>
>      Thank you very much!
>
> Mandy
>
>
> ________________________________
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Friday, December 8, 2017 3:28:21 PM
> To: Zhang, Mandy
> Subject: Re: [rt.rap.ucar.edu #83246] how to calculate ensemble
> evaluation with MET Tools?
>
> Mandy,
>
> Typically, when doing ensemble verification, the flow of our logic
is
> this...
>
> (1) For a given forecast time, identify the N ensemble members.
> (2) Pass the N members and verifying observations (gridded and/or
points)
> to Ensemble-Stat.
> (3) Ensemble-Stat generates verification statistics such as ranked
> histograms (RHIST), pit histograms (PHIST), and spread-skill
(SSVAR).
> (4) Ensemble-Stat also writes out a gridded NetCDF file containing
the
> ensemble mean and ensemble relative frequency (i.e. like
probabilities).
> (5) Pass each of the N ensemble members through grid_stat or
point_stat to
> verify it as a deterministic forecast.
> (6) Pass the ensemble mean through grid_stat or point_stat to verify
it as
> a deterministic forecast.
> (7) Pass the ensemble frequencies through grid_stat or point_stat to
verify
> the probability data.
>
> Hope that helps describe the logical flow.
>
> Thanks,
> John
>
> On Fri, Dec 8, 2017 at 10:21 AM, Zhang, Mandy via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > Fri Dec 08 10:21:16 2017: Request 83246 was acted upon.
> > Transaction: Ticket created by Man.Zhang-1 at ou.edu
> >        Queue: met_help
> >      Subject: how to calculate ensemble evaluation with MET Tools?
> >        Owner: Nobody
> >   Requestors: Man.Zhang-1 at ou.edu
> >       Status: new
> >  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83246 >
> >
> >
> > Hi All,
> >
> >     From the ensemble_stat, the output only includes some simple
> > evaluation as rank histogram, ensemble mean.  From the point_stat,
we can
> > get more evaluation,  but it is limited to the single member. How
can I
> use
> > the MET-tools to calculate the ensemble evaluations?
> >
> >   Can anyone give me a guide?
> >
> > Thank you very much!
> >
> > Mandy
> >
> >
>
>
>

------------------------------------------------
Subject: how to calculate ensemble evaluation with MET Tools?
From: Zhang, Mandy
Time: Fri Dec 08 15:10:07 2017

John,

        I'll try it, and it may help me understand it.

Thank you very much!

Mandy

________________________________
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Friday, December 8, 2017 4:03:19 PM
To: Zhang, Mandy
Subject: Re: [rt.rap.ucar.edu #83246] how to calculate ensemble
evaluation with MET Tools?

Mandy,

Point-Stat and Grid-Stat are used to verify probability forecasts.
The
probability vx output is stored in PCT line type (probabilistic
contingency
table).  The Ensemble-Stat tool writes a different set of line types.

However, the .stat output file from Point-Stat, Grid-Stat, and
Ensemble-Stat are all readable by the same STAT-Analysis tool.

Hopefully that helps clarify.

Thanks,
John

On Fri, Dec 8, 2017 at 2:57 PM, Zhang, Mandy via RT
<met_help at ucar.edu>
wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83246 >
>
> Hi John,
>
>      Can I use Stat_Ananlysis tool to combine the results of
ensemble_stat
> and Point_stat to get the verification of probability data?
>
>      Thank you very much!
>
> Mandy
>
>
> ________________________________
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Friday, December 8, 2017 3:28:21 PM
> To: Zhang, Mandy
> Subject: Re: [rt.rap.ucar.edu #83246] how to calculate ensemble
> evaluation with MET Tools?
>
> Mandy,
>
> Typically, when doing ensemble verification, the flow of our logic
is
> this...
>
> (1) For a given forecast time, identify the N ensemble members.
> (2) Pass the N members and verifying observations (gridded and/or
points)
> to Ensemble-Stat.
> (3) Ensemble-Stat generates verification statistics such as ranked
> histograms (RHIST), pit histograms (PHIST), and spread-skill
(SSVAR).
> (4) Ensemble-Stat also writes out a gridded NetCDF file containing
the
> ensemble mean and ensemble relative frequency (i.e. like
probabilities).
> (5) Pass each of the N ensemble members through grid_stat or
point_stat to
> verify it as a deterministic forecast.
> (6) Pass the ensemble mean through grid_stat or point_stat to verify
it as
> a deterministic forecast.
> (7) Pass the ensemble frequencies through grid_stat or point_stat to
verify
> the probability data.
>
> Hope that helps describe the logical flow.
>
> Thanks,
> John
>
> On Fri, Dec 8, 2017 at 10:21 AM, Zhang, Mandy via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > Fri Dec 08 10:21:16 2017: Request 83246 was acted upon.
> > Transaction: Ticket created by Man.Zhang-1 at ou.edu
> >        Queue: met_help
> >      Subject: how to calculate ensemble evaluation with MET Tools?
> >        Owner: Nobody
> >   Requestors: Man.Zhang-1 at ou.edu
> >       Status: new
> >  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83246 >
> >
> >
> > Hi All,
> >
> >     From the ensemble_stat, the output only includes some simple
> > evaluation as rank histogram, ensemble mean.  From the point_stat,
we can
> > get more evaluation,  but it is limited to the single member. How
can I
> use
> > the MET-tools to calculate the ensemble evaluations?
> >
> >   Can anyone give me a guide?
> >
> > Thank you very much!
> >
> > Mandy
> >
> >
>
>
>


------------------------------------------------


More information about the Met_help mailing list