[Met_help] [rt.rap.ucar.edu #88227] History for Plotting ensemble spread/error curves

Wed Dec 19 10:38:30 MST 2018

----------------------------------------------------------------
  Initial Request
----------------------------------------------------------------

Hi,

I am interested in creating ensemble spread/error curves with MET and
METviewer.  I loaded statistics for the ensemble mean from point stat into
METviewer, and I see that I have RMSE, OSTDEV, FSTDEV, and ESTDEV.

For ensemble spread, it looks like I needed to also save output from
ensemble stat and load that into METviewer.  I did not save any output that
I generated from ensemble stat, but it looks like SPREAD in the RHIST line
type could be what I am looking for because I want to calculate spread over
a masking region.  I found this in the MET user's guide:

SPREAD
The ensemble spread for a single observation is the standard deviation of
the ensemble member forecast values at that location. When verifying
against point observations, these values are written to the ENS_SPREAD
column of the Observation Rank (ORANK) line type. The ensemble spread
computed over a spatial masking region is the average of the spread values
for all observations falling within the mask. These values are written to
the SPREAD column of the Ranked Histogram (RHIST) line type.

Do I need to re-run ensemble stat to generate values for spread or is it
possible to calculate with the variables I already have from point stat?

Thanks!
Ben

----------------------------------------------------------------
  Complete Ticket History
----------------------------------------------------------------

Subject: Plotting ensemble spread/error curves
From: Julie Prestopnik
Time: Tue Dec 18 11:53:58 2018

Hi Ben.

Unfortunately, I do not know the answer to your question, and John is
out
of the office for a while due to the holidays.  I have copied some
other
members of the MET team in hopes that someone else will know and be
able to
help.  Thank you for your patience while we work toward getting an
answer
to you.

Thanks,
Julie

On Tue, Dec 18, 2018 at 11:27 AM Benjamin Blake - NOAA Affiliate via
RT <
met_help at ucar.edu> wrote:

>
> Tue Dec 18 11:27:26 2018: Request 88227 was acted upon.
> Transaction: Ticket created by benjamin.blake at noaa.gov
>        Queue: met_help
>      Subject: Plotting ensemble spread/error curves
>        Owner: Nobody
>   Requestors: benjamin.blake at noaa.gov
>       Status: new
>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=88227 >
>
>
> Hi,
>
> I am interested in creating ensemble spread/error curves with MET
and
> METviewer.  I loaded statistics for the ensemble mean from point
stat into
> METviewer, and I see that I have RMSE, OSTDEV, FSTDEV, and ESTDEV.
>
> For ensemble spread, it looks like I needed to also save output from
> ensemble stat and load that into METviewer.  I did not save any
output that
> I generated from ensemble stat, but it looks like SPREAD in the
RHIST line
> type could be what I am looking for because I want to calculate
spread over
> a masking region.  I found this in the MET user's guide:
>
> SPREAD
> The ensemble spread for a single observation is the standard
deviation of
> the ensemble member forecast values at that location. When verifying
> against point observations, these values are written to the
ENS_SPREAD
> column of the Observation Rank (ORANK) line type. The ensemble
spread
> computed over a spatial masking region is the average of the spread
values
> for all observations falling within the mask. These values are
written to
> the SPREAD column of the Ranked Histogram (RHIST) line type.
>
> Do I need to re-run ensemble stat to generate values for spread or
is it
> possible to calculate with the variables I already have from point
stat?
>
> Thanks!
> Ben
>
>

------------------------------------------------
Subject: Plotting ensemble spread/error curves
From: Benjamin Blake - NOAA Affiliate
Time: Tue Dec 18 12:31:09 2018

Hi Julie,

No problem, thank you for letting me know!

Ben

On Tue, Dec 18, 2018 at 2:17 PM Julie Prestopnik via RT
<met_help at ucar.edu>
wrote:

> Hi Ben.
>
> Unfortunately, I do not know the answer to your question, and John
is out
> of the office for a while due to the holidays.  I have copied some
other
> members of the MET team in hopes that someone else will know and be
able to
> help.  Thank you for your patience while we work toward getting an
answer
> to you.
>
> Thanks,
> Julie
>
> On Tue, Dec 18, 2018 at 11:27 AM Benjamin Blake - NOAA Affiliate via
RT <
> met_help at ucar.edu> wrote:
>
> >
> > Tue Dec 18 11:27:26 2018: Request 88227 was acted upon.
> > Transaction: Ticket created by benjamin.blake at noaa.gov
> >        Queue: met_help
> >      Subject: Plotting ensemble spread/error curves
> >        Owner: Nobody
> >   Requestors: benjamin.blake at noaa.gov
> >       Status: new
> >  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=88227 >
> >
> >
> > Hi,
> >
> > I am interested in creating ensemble spread/error curves with MET
and
> > METviewer.  I loaded statistics for the ensemble mean from point
stat
> into
> > METviewer, and I see that I have RMSE, OSTDEV, FSTDEV, and ESTDEV.
> >
> > For ensemble spread, it looks like I needed to also save output
from
> > ensemble stat and load that into METviewer.  I did not save any
output
> that
> > I generated from ensemble stat, but it looks like SPREAD in the
RHIST
> line
> > type could be what I am looking for because I want to calculate
spread
> over
> > a masking region.  I found this in the MET user's guide:
> >
> > SPREAD
> > The ensemble spread for a single observation is the standard
deviation of
> > the ensemble member forecast values at that location. When
verifying
> > against point observations, these values are written to the
ENS_SPREAD
> > column of the Observation Rank (ORANK) line type. The ensemble
spread
> > computed over a spatial masking region is the average of the
spread
> values
> > for all observations falling within the mask. These values are
written to
> > the SPREAD column of the Ranked Histogram (RHIST) line type.
> >
> > Do I need to re-run ensemble stat to generate values for spread or
is it
> > possible to calculate with the variables I already have from point
stat?
> >
> > Thanks!
> > Ben
> >
> >
>
>

------------------------------------------------
Subject: Plotting ensemble spread/error curves
From: Tressa Fowler
Time: Tue Dec 18 15:01:00 2018

Hi Ben,

I don't have an answer, but I have a few more questions. Do you only
have
the four variables listed (RMSE, OSTDEV, FSTDEV, and ESTDEV)? If so,
then
you do not have ensemble spread and you will have to do more runs.

The last 3 of these are the standard deviation of the observations, of
the
ensemble mean, and of the error. They are not the ensemble spread
(though
the ESTDEV is the 'error' for your plot). I think John added variables
from
the ensemble stat output to the point stat lines in the last release
to
handle the observation error perturbations. In older versions of met,
you
used to have to save the ensemble stat output). However, they do not
appear
in your list.

I was going to check the latest version of the documentation, but I
don't
have a copy and our web site is down. It seems like there is ORANK
info now
in one of the line types. Do you have a copy to check?

Tressa

On Tue, Dec 18, 2018 at 11:53 AM Julie Prestopnik <jpresto at ucar.edu>
wrote:

> Hi Ben.
>
> Unfortunately, I do not know the answer to your question, and John
is out
> of the office for a while due to the holidays.  I have copied some
other
> members of the MET team in hopes that someone else will know and be
able to
> help.  Thank you for your patience while we work toward getting an
answer
> to you.
>
> Thanks,
> Julie
>
> On Tue, Dec 18, 2018 at 11:27 AM Benjamin Blake - NOAA Affiliate via
RT <
> met_help at ucar.edu> wrote:
>
>>
>> Tue Dec 18 11:27:26 2018: Request 88227 was acted upon.
>> Transaction: Ticket created by benjamin.blake at noaa.gov
>>        Queue: met_help
>>      Subject: Plotting ensemble spread/error curves
>>        Owner: Nobody
>>   Requestors: benjamin.blake at noaa.gov
>>       Status: new
>>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=88227 >
>>
>>
>> Hi,
>>
>> I am interested in creating ensemble spread/error curves with MET
and
>> METviewer.  I loaded statistics for the ensemble mean from point
stat into
>> METviewer, and I see that I have RMSE, OSTDEV, FSTDEV, and ESTDEV.
>>
>> For ensemble spread, it looks like I needed to also save output
from
>> ensemble stat and load that into METviewer.  I did not save any
output
>> that
>> I generated from ensemble stat, but it looks like SPREAD in the
RHIST line
>> type could be what I am looking for because I want to calculate
spread
>> over
>> a masking region.  I found this in the MET user's guide:
>>
>> SPREAD
>> The ensemble spread for a single observation is the standard
deviation of
>> the ensemble member forecast values at that location. When
verifying
>> against point observations, these values are written to the
ENS_SPREAD
>> column of the Observation Rank (ORANK) line type. The ensemble
spread
>> computed over a spatial masking region is the average of the spread
values
>> for all observations falling within the mask. These values are
written to
>> the SPREAD column of the Ranked Histogram (RHIST) line type.
>>
>> Do I need to re-run ensemble stat to generate values for spread or
is it
>> possible to calculate with the variables I already have from point
stat?
>>
>> Thanks!
>> Ben
>>
>>

------------------------------------------------
Subject: Plotting ensemble spread/error curves
From: Benjamin Blake - NOAA Affiliate
Time: Tue Dec 18 15:20:35 2018

Hi Tressa,

I have a copy of the MET v7.0 user's guide which might not be too
different.

I have a bunch of other variables than those 4 but those were just the
ones
I mentioned.  I have all the variables in the cnt, sl1l2, and vl1l2
line
types for point stat.  However, I don't have the orank line type
selected
so I'm guessing I do not have the ensemble spread data or enough
information to calculate it.  I don't see the orank or rhist line
types as
options for point stat, but you can select them for ensemble stat.

Thanks!
Ben

On Tue, Dec 18, 2018 at 5:01 PM Tressa Fowler via RT
<met_help at ucar.edu>
wrote:

> Hi Ben,
>
> I don't have an answer, but I have a few more questions. Do you only
have
> the four variables listed (RMSE, OSTDEV, FSTDEV, and ESTDEV)? If so,
then
> you do not have ensemble spread and you will have to do more runs.
>
> The last 3 of these are the standard deviation of the observations,
of the
> ensemble mean, and of the error. They are not the ensemble spread
(though
> the ESTDEV is the 'error' for your plot). I think John added
variables from
> the ensemble stat output to the point stat lines in the last release
to
> handle the observation error perturbations. In older versions of
met, you
> used to have to save the ensemble stat output). However, they do not
appear
> in your list.
>
> I was going to check the latest version of the documentation, but I
don't
> have a copy and our web site is down. It seems like there is ORANK
info now
> in one of the line types. Do you have a copy to check?
>
> Tressa
>
> On Tue, Dec 18, 2018 at 11:53 AM Julie Prestopnik <jpresto at ucar.edu>
> wrote:
>
> > Hi Ben.
> >
> > Unfortunately, I do not know the answer to your question, and John
is out
> > of the office for a while due to the holidays.  I have copied some
other
> > members of the MET team in hopes that someone else will know and
be able
> to
> > help.  Thank you for your patience while we work toward getting an
answer
> > to you.
> >
> > Thanks,
> > Julie
> >
> > On Tue, Dec 18, 2018 at 11:27 AM Benjamin Blake - NOAA Affiliate
via RT <
> > met_help at ucar.edu> wrote:
> >
> >>
> >> Tue Dec 18 11:27:26 2018: Request 88227 was acted upon.
> >> Transaction: Ticket created by benjamin.blake at noaa.gov
> >>        Queue: met_help
> >>      Subject: Plotting ensemble spread/error curves
> >>        Owner: Nobody
> >>   Requestors: benjamin.blake at noaa.gov
> >>       Status: new
> >>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=88227 >
> >>
> >>
> >> Hi,
> >>
> >> I am interested in creating ensemble spread/error curves with MET
and
> >> METviewer.  I loaded statistics for the ensemble mean from point
stat
> into
> >> METviewer, and I see that I have RMSE, OSTDEV, FSTDEV, and
ESTDEV.
> >>
> >> For ensemble spread, it looks like I needed to also save output
from
> >> ensemble stat and load that into METviewer.  I did not save any
output
> >> that
> >> I generated from ensemble stat, but it looks like SPREAD in the
RHIST
> line
> >> type could be what I am looking for because I want to calculate
spread
> >> over
> >> a masking region.  I found this in the MET user's guide:
> >>
> >> SPREAD
> >> The ensemble spread for a single observation is the standard
deviation
> of
> >> the ensemble member forecast values at that location. When
verifying
> >> against point observations, these values are written to the
ENS_SPREAD
> >> column of the Observation Rank (ORANK) line type. The ensemble
spread
> >> computed over a spatial masking region is the average of the
spread
> values
> >> for all observations falling within the mask. These values are
written
> to
> >> the SPREAD column of the Ranked Histogram (RHIST) line type.
> >>
> >> Do I need to re-run ensemble stat to generate values for spread
or is it
> >> possible to calculate with the variables I already have from
point stat?
> >>
> >> Thanks!
> >> Ben
> >>
> >>
>
>

------------------------------------------------
Subject: Plotting ensemble spread/error curves
From: Tressa Fowler
Time: Tue Dec 18 15:32:59 2018

Hey Ben,

Tara sent me the documentation (totally stupid of me to not have a
copy,
but I just always look it up online . . .)

Anyway, you need the ECNT or SSVAR line types from the ensemble stat
tool.
That tool can read your observations and compare them to the ensemble,
so
you should not need to run point stat at all. Both of those line types
have
the RMSE of the ensemble mean and the average spread of the ensemble
values. These two are your variables for the spread / skill plot.

Let me know if you have more questions.

Happy Holidays!

Tressa

On Tue, Dec 18, 2018 at 3:00 PM Tressa Fowler <tressa at ucar.edu> wrote:

> Hi Ben,
>
> I don't have an answer, but I have a few more questions. Do you only
have
> the four variables listed (RMSE, OSTDEV, FSTDEV, and ESTDEV)? If so,
then
> you do not have ensemble spread and you will have to do more runs.
>
> The last 3 of these are the standard deviation of the observations,
of the
> ensemble mean, and of the error. They are not the ensemble spread
(though
> the ESTDEV is the 'error' for your plot). I think John added
variables from
> the ensemble stat output to the point stat lines in the last release
to
> handle the observation error perturbations. In older versions of
met, you
> used to have to save the ensemble stat output). However, they do not
appear
> in your list.
>
> I was going to check the latest version of the documentation, but I
don't
> have a copy and our web site is down. It seems like there is ORANK
info now
> in one of the line types. Do you have a copy to check?
>
> Tressa
>
> On Tue, Dec 18, 2018 at 11:53 AM Julie Prestopnik <jpresto at ucar.edu>
> wrote:
>
>> Hi Ben.
>>
>> Unfortunately, I do not know the answer to your question, and John
is out
>> of the office for a while due to the holidays.  I have copied some
other
>> members of the MET team in hopes that someone else will know and be
able to
>> help.  Thank you for your patience while we work toward getting an
answer
>> to you.
>>
>> Thanks,
>> Julie
>>
>> On Tue, Dec 18, 2018 at 11:27 AM Benjamin Blake - NOAA Affiliate
via RT <
>> met_help at ucar.edu> wrote:
>>
>>>
>>> Tue Dec 18 11:27:26 2018: Request 88227 was acted upon.
>>> Transaction: Ticket created by benjamin.blake at noaa.gov
>>>        Queue: met_help
>>>      Subject: Plotting ensemble spread/error curves
>>>        Owner: Nobody
>>>   Requestors: benjamin.blake at noaa.gov
>>>       Status: new
>>>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=88227 >
>>>
>>>
>>> Hi,
>>>
>>> I am interested in creating ensemble spread/error curves with MET
and
>>> METviewer.  I loaded statistics for the ensemble mean from point
stat
>>> into
>>> METviewer, and I see that I have RMSE, OSTDEV, FSTDEV, and ESTDEV.
>>>
>>> For ensemble spread, it looks like I needed to also save output
from
>>> ensemble stat and load that into METviewer.  I did not save any
output
>>> that
>>> I generated from ensemble stat, but it looks like SPREAD in the
RHIST
>>> line
>>> type could be what I am looking for because I want to calculate
spread
>>> over
>>> a masking region.  I found this in the MET user's guide:
>>>
>>> SPREAD
>>> The ensemble spread for a single observation is the standard
deviation of
>>> the ensemble member forecast values at that location. When
verifying
>>> against point observations, these values are written to the
ENS_SPREAD
>>> column of the Observation Rank (ORANK) line type. The ensemble
spread
>>> computed over a spatial masking region is the average of the
spread
>>> values
>>> for all observations falling within the mask. These values are
written to
>>> the SPREAD column of the Ranked Histogram (RHIST) line type.
>>>
>>> Do I need to re-run ensemble stat to generate values for spread or
is it
>>> possible to calculate with the variables I already have from point
stat?
>>>
>>> Thanks!
>>> Ben
>>>
>>>

------------------------------------------------
Subject: Plotting ensemble spread/error curves
From: Benjamin Blake - NOAA Affiliate
Time: Wed Dec 19 05:49:16 2018

Hi Tressa,

Ok - thanks for letting me know!  Happy holidays to you too.

Ben

On Tue, Dec 18, 2018 at 5:32 PM Tressa Fowler <tressa at ucar.edu> wrote:

> Hey Ben,
>
> Tara sent me the documentation (totally stupid of me to not have a
copy,
> but I just always look it up online . . .)
>
> Anyway, you need the ECNT or SSVAR line types from the ensemble stat
tool.
> That tool can read your observations and compare them to the
ensemble, so
> you should not need to run point stat at all. Both of those line
types have
> the RMSE of the ensemble mean and the average spread of the ensemble
> values. These two are your variables for the spread / skill plot.
>
> Let me know if you have more questions.
>
> Happy Holidays!
>
> Tressa
>
> On Tue, Dec 18, 2018 at 3:00 PM Tressa Fowler <tressa at ucar.edu>
wrote:
>
>> Hi Ben,
>>
>> I don't have an answer, but I have a few more questions. Do you
only have
>> the four variables listed (RMSE, OSTDEV, FSTDEV, and ESTDEV)? If
so,
>> then you do not have ensemble spread and you will have to do more
runs.
>>
>> The last 3 of these are the standard deviation of the observations,
of
>> the ensemble mean, and of the error. They are not the ensemble
spread
>> (though the ESTDEV is the 'error' for your plot). I think John
added
>> variables from the ensemble stat output to the point stat lines in
the last
>> release to handle the observation error perturbations. In older
versions of
>> met, you used to have to save the ensemble stat output). However,
they do
>> not appear in your list.
>>
>> I was going to check the latest version of the documentation, but I
don't
>> have a copy and our web site is down. It seems like there is ORANK
info now
>> in one of the line types. Do you have a copy to check?
>>
>> Tressa
>>
>> On Tue, Dec 18, 2018 at 11:53 AM Julie Prestopnik
<jpresto at ucar.edu>
>> wrote:
>>
>>> Hi Ben.
>>>
>>> Unfortunately, I do not know the answer to your question, and John
is
>>> out of the office for a while due to the holidays.  I have copied
some
>>> other members of the MET team in hopes that someone else will know
and be
>>> able to help.  Thank you for your patience while we work toward
getting an
>>> answer to you.
>>>
>>> Thanks,
>>> Julie
>>>
>>> On Tue, Dec 18, 2018 at 11:27 AM Benjamin Blake - NOAA Affiliate
via RT <
>>> met_help at ucar.edu> wrote:
>>>
>>>>
>>>> Tue Dec 18 11:27:26 2018: Request 88227 was acted upon.
>>>> Transaction: Ticket created by benjamin.blake at noaa.gov
>>>>        Queue: met_help
>>>>      Subject: Plotting ensemble spread/error curves
>>>>        Owner: Nobody
>>>>   Requestors: benjamin.blake at noaa.gov
>>>>       Status: new
>>>>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=88227 >
>>>>
>>>>
>>>> Hi,
>>>>
>>>> I am interested in creating ensemble spread/error curves with MET
and
>>>> METviewer.  I loaded statistics for the ensemble mean from point
stat
>>>> into
>>>> METviewer, and I see that I have RMSE, OSTDEV, FSTDEV, and
ESTDEV.
>>>>
>>>> For ensemble spread, it looks like I needed to also save output
from
>>>> ensemble stat and load that into METviewer.  I did not save any
output
>>>> that
>>>> I generated from ensemble stat, but it looks like SPREAD in the
RHIST
>>>> line
>>>> type could be what I am looking for because I want to calculate
spread
>>>> over
>>>> a masking region.  I found this in the MET user's guide:
>>>>
>>>> SPREAD
>>>> The ensemble spread for a single observation is the standard
deviation
>>>> of
>>>> the ensemble member forecast values at that location. When
verifying
>>>> against point observations, these values are written to the
ENS_SPREAD
>>>> column of the Observation Rank (ORANK) line type. The ensemble
spread
>>>> computed over a spatial masking region is the average of the
spread
>>>> values
>>>> for all observations falling within the mask. These values are
written
>>>> to
>>>> the SPREAD column of the Ranked Histogram (RHIST) line type.
>>>>
>>>> Do I need to re-run ensemble stat to generate values for spread
or is it
>>>> possible to calculate with the variables I already have from
point stat?
>>>>
>>>> Thanks!
>>>> Ben
>>>>
>>>>

------------------------------------------------