[Met_help] [rt.rap.ucar.edu #88227] History for Plotting ensemble spread/error curves
John Halley Gotway via RT
met_help at ucar.edu
Mon Jan 28 10:26:07 MST 2019
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hi,
I am interested in creating ensemble spread/error curves with MET and
METviewer. I loaded statistics for the ensemble mean from point stat into
METviewer, and I see that I have RMSE, OSTDEV, FSTDEV, and ESTDEV.
For ensemble spread, it looks like I needed to also save output from
ensemble stat and load that into METviewer. I did not save any output that
I generated from ensemble stat, but it looks like SPREAD in the RHIST line
type could be what I am looking for because I want to calculate spread over
a masking region. I found this in the MET user's guide:
SPREAD
The ensemble spread for a single observation is the standard deviation of
the ensemble member forecast values at that location. When verifying
against point observations, these values are written to the ENS_SPREAD
column of the Observation Rank (ORANK) line type. The ensemble spread
computed over a spatial masking region is the average of the spread values
for all observations falling within the mask. These values are written to
the SPREAD column of the Ranked Histogram (RHIST) line type.
Do I need to re-run ensemble stat to generate values for spread or is it
possible to calculate with the variables I already have from point stat?
Thanks!
Ben
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Plotting ensemble spread/error curves
From: Julie Prestopnik
Time: Tue Dec 18 11:53:58 2018
Hi Ben.
Unfortunately, I do not know the answer to your question, and John is
out
of the office for a while due to the holidays. I have copied some
other
members of the MET team in hopes that someone else will know and be
able to
help. Thank you for your patience while we work toward getting an
answer
to you.
Thanks,
Julie
On Tue, Dec 18, 2018 at 11:27 AM Benjamin Blake - NOAA Affiliate via
RT <
met_help at ucar.edu> wrote:
>
> Tue Dec 18 11:27:26 2018: Request 88227 was acted upon.
> Transaction: Ticket created by benjamin.blake at noaa.gov
> Queue: met_help
> Subject: Plotting ensemble spread/error curves
> Owner: Nobody
> Requestors: benjamin.blake at noaa.gov
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=88227 >
>
>
> Hi,
>
> I am interested in creating ensemble spread/error curves with MET
and
> METviewer. I loaded statistics for the ensemble mean from point
stat into
> METviewer, and I see that I have RMSE, OSTDEV, FSTDEV, and ESTDEV.
>
> For ensemble spread, it looks like I needed to also save output from
> ensemble stat and load that into METviewer. I did not save any
output that
> I generated from ensemble stat, but it looks like SPREAD in the
RHIST line
> type could be what I am looking for because I want to calculate
spread over
> a masking region. I found this in the MET user's guide:
>
> SPREAD
> The ensemble spread for a single observation is the standard
deviation of
> the ensemble member forecast values at that location. When verifying
> against point observations, these values are written to the
ENS_SPREAD
> column of the Observation Rank (ORANK) line type. The ensemble
spread
> computed over a spatial masking region is the average of the spread
values
> for all observations falling within the mask. These values are
written to
> the SPREAD column of the Ranked Histogram (RHIST) line type.
>
> Do I need to re-run ensemble stat to generate values for spread or
is it
> possible to calculate with the variables I already have from point
stat?
>
> Thanks!
> Ben
>
>
------------------------------------------------
Subject: Plotting ensemble spread/error curves
From: Benjamin Blake - NOAA Affiliate
Time: Tue Dec 18 12:31:09 2018
Hi Julie,
No problem, thank you for letting me know!
Ben
On Tue, Dec 18, 2018 at 2:17 PM Julie Prestopnik via RT
<met_help at ucar.edu>
wrote:
> Hi Ben.
>
> Unfortunately, I do not know the answer to your question, and John
is out
> of the office for a while due to the holidays. I have copied some
other
> members of the MET team in hopes that someone else will know and be
able to
> help. Thank you for your patience while we work toward getting an
answer
> to you.
>
> Thanks,
> Julie
>
> On Tue, Dec 18, 2018 at 11:27 AM Benjamin Blake - NOAA Affiliate via
RT <
> met_help at ucar.edu> wrote:
>
> >
> > Tue Dec 18 11:27:26 2018: Request 88227 was acted upon.
> > Transaction: Ticket created by benjamin.blake at noaa.gov
> > Queue: met_help
> > Subject: Plotting ensemble spread/error curves
> > Owner: Nobody
> > Requestors: benjamin.blake at noaa.gov
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=88227 >
> >
> >
> > Hi,
> >
> > I am interested in creating ensemble spread/error curves with MET
and
> > METviewer. I loaded statistics for the ensemble mean from point
stat
> into
> > METviewer, and I see that I have RMSE, OSTDEV, FSTDEV, and ESTDEV.
> >
> > For ensemble spread, it looks like I needed to also save output
from
> > ensemble stat and load that into METviewer. I did not save any
output
> that
> > I generated from ensemble stat, but it looks like SPREAD in the
RHIST
> line
> > type could be what I am looking for because I want to calculate
spread
> over
> > a masking region. I found this in the MET user's guide:
> >
> > SPREAD
> > The ensemble spread for a single observation is the standard
deviation of
> > the ensemble member forecast values at that location. When
verifying
> > against point observations, these values are written to the
ENS_SPREAD
> > column of the Observation Rank (ORANK) line type. The ensemble
spread
> > computed over a spatial masking region is the average of the
spread
> values
> > for all observations falling within the mask. These values are
written to
> > the SPREAD column of the Ranked Histogram (RHIST) line type.
> >
> > Do I need to re-run ensemble stat to generate values for spread or
is it
> > possible to calculate with the variables I already have from point
stat?
> >
> > Thanks!
> > Ben
> >
> >
>
>
------------------------------------------------
Subject: Plotting ensemble spread/error curves
From: Tressa Fowler
Time: Tue Dec 18 15:01:00 2018
Hi Ben,
I don't have an answer, but I have a few more questions. Do you only
have
the four variables listed (RMSE, OSTDEV, FSTDEV, and ESTDEV)? If so,
then
you do not have ensemble spread and you will have to do more runs.
The last 3 of these are the standard deviation of the observations, of
the
ensemble mean, and of the error. They are not the ensemble spread
(though
the ESTDEV is the 'error' for your plot). I think John added variables
from
the ensemble stat output to the point stat lines in the last release
to
handle the observation error perturbations. In older versions of met,
you
used to have to save the ensemble stat output). However, they do not
appear
in your list.
I was going to check the latest version of the documentation, but I
don't
have a copy and our web site is down. It seems like there is ORANK
info now
in one of the line types. Do you have a copy to check?
Tressa
On Tue, Dec 18, 2018 at 11:53 AM Julie Prestopnik <jpresto at ucar.edu>
wrote:
> Hi Ben.
>
> Unfortunately, I do not know the answer to your question, and John
is out
> of the office for a while due to the holidays. I have copied some
other
> members of the MET team in hopes that someone else will know and be
able to
> help. Thank you for your patience while we work toward getting an
answer
> to you.
>
> Thanks,
> Julie
>
> On Tue, Dec 18, 2018 at 11:27 AM Benjamin Blake - NOAA Affiliate via
RT <
> met_help at ucar.edu> wrote:
>
>>
>> Tue Dec 18 11:27:26 2018: Request 88227 was acted upon.
>> Transaction: Ticket created by benjamin.blake at noaa.gov
>> Queue: met_help
>> Subject: Plotting ensemble spread/error curves
>> Owner: Nobody
>> Requestors: benjamin.blake at noaa.gov
>> Status: new
>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=88227 >
>>
>>
>> Hi,
>>
>> I am interested in creating ensemble spread/error curves with MET
and
>> METviewer. I loaded statistics for the ensemble mean from point
stat into
>> METviewer, and I see that I have RMSE, OSTDEV, FSTDEV, and ESTDEV.
>>
>> For ensemble spread, it looks like I needed to also save output
from
>> ensemble stat and load that into METviewer. I did not save any
output
>> that
>> I generated from ensemble stat, but it looks like SPREAD in the
RHIST line
>> type could be what I am looking for because I want to calculate
spread
>> over
>> a masking region. I found this in the MET user's guide:
>>
>> SPREAD
>> The ensemble spread for a single observation is the standard
deviation of
>> the ensemble member forecast values at that location. When
verifying
>> against point observations, these values are written to the
ENS_SPREAD
>> column of the Observation Rank (ORANK) line type. The ensemble
spread
>> computed over a spatial masking region is the average of the spread
values
>> for all observations falling within the mask. These values are
written to
>> the SPREAD column of the Ranked Histogram (RHIST) line type.
>>
>> Do I need to re-run ensemble stat to generate values for spread or
is it
>> possible to calculate with the variables I already have from point
stat?
>>
>> Thanks!
>> Ben
>>
>>
------------------------------------------------
Subject: Plotting ensemble spread/error curves
From: Benjamin Blake - NOAA Affiliate
Time: Tue Dec 18 15:20:35 2018
Hi Tressa,
I have a copy of the MET v7.0 user's guide which might not be too
different.
I have a bunch of other variables than those 4 but those were just the
ones
I mentioned. I have all the variables in the cnt, sl1l2, and vl1l2
line
types for point stat. However, I don't have the orank line type
selected
so I'm guessing I do not have the ensemble spread data or enough
information to calculate it. I don't see the orank or rhist line
types as
options for point stat, but you can select them for ensemble stat.
Thanks!
Ben
On Tue, Dec 18, 2018 at 5:01 PM Tressa Fowler via RT
<met_help at ucar.edu>
wrote:
> Hi Ben,
>
> I don't have an answer, but I have a few more questions. Do you only
have
> the four variables listed (RMSE, OSTDEV, FSTDEV, and ESTDEV)? If so,
then
> you do not have ensemble spread and you will have to do more runs.
>
> The last 3 of these are the standard deviation of the observations,
of the
> ensemble mean, and of the error. They are not the ensemble spread
(though
> the ESTDEV is the 'error' for your plot). I think John added
variables from
> the ensemble stat output to the point stat lines in the last release
to
> handle the observation error perturbations. In older versions of
met, you
> used to have to save the ensemble stat output). However, they do not
appear
> in your list.
>
> I was going to check the latest version of the documentation, but I
don't
> have a copy and our web site is down. It seems like there is ORANK
info now
> in one of the line types. Do you have a copy to check?
>
> Tressa
>
> On Tue, Dec 18, 2018 at 11:53 AM Julie Prestopnik <jpresto at ucar.edu>
> wrote:
>
> > Hi Ben.
> >
> > Unfortunately, I do not know the answer to your question, and John
is out
> > of the office for a while due to the holidays. I have copied some
other
> > members of the MET team in hopes that someone else will know and
be able
> to
> > help. Thank you for your patience while we work toward getting an
answer
> > to you.
> >
> > Thanks,
> > Julie
> >
> > On Tue, Dec 18, 2018 at 11:27 AM Benjamin Blake - NOAA Affiliate
via RT <
> > met_help at ucar.edu> wrote:
> >
> >>
> >> Tue Dec 18 11:27:26 2018: Request 88227 was acted upon.
> >> Transaction: Ticket created by benjamin.blake at noaa.gov
> >> Queue: met_help
> >> Subject: Plotting ensemble spread/error curves
> >> Owner: Nobody
> >> Requestors: benjamin.blake at noaa.gov
> >> Status: new
> >> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=88227 >
> >>
> >>
> >> Hi,
> >>
> >> I am interested in creating ensemble spread/error curves with MET
and
> >> METviewer. I loaded statistics for the ensemble mean from point
stat
> into
> >> METviewer, and I see that I have RMSE, OSTDEV, FSTDEV, and
ESTDEV.
> >>
> >> For ensemble spread, it looks like I needed to also save output
from
> >> ensemble stat and load that into METviewer. I did not save any
output
> >> that
> >> I generated from ensemble stat, but it looks like SPREAD in the
RHIST
> line
> >> type could be what I am looking for because I want to calculate
spread
> >> over
> >> a masking region. I found this in the MET user's guide:
> >>
> >> SPREAD
> >> The ensemble spread for a single observation is the standard
deviation
> of
> >> the ensemble member forecast values at that location. When
verifying
> >> against point observations, these values are written to the
ENS_SPREAD
> >> column of the Observation Rank (ORANK) line type. The ensemble
spread
> >> computed over a spatial masking region is the average of the
spread
> values
> >> for all observations falling within the mask. These values are
written
> to
> >> the SPREAD column of the Ranked Histogram (RHIST) line type.
> >>
> >> Do I need to re-run ensemble stat to generate values for spread
or is it
> >> possible to calculate with the variables I already have from
point stat?
> >>
> >> Thanks!
> >> Ben
> >>
> >>
>
>
------------------------------------------------
Subject: Plotting ensemble spread/error curves
From: Tressa Fowler
Time: Tue Dec 18 15:32:59 2018
Hey Ben,
Tara sent me the documentation (totally stupid of me to not have a
copy,
but I just always look it up online . . .)
Anyway, you need the ECNT or SSVAR line types from the ensemble stat
tool.
That tool can read your observations and compare them to the ensemble,
so
you should not need to run point stat at all. Both of those line types
have
the RMSE of the ensemble mean and the average spread of the ensemble
values. These two are your variables for the spread / skill plot.
Let me know if you have more questions.
Happy Holidays!
Tressa
On Tue, Dec 18, 2018 at 3:00 PM Tressa Fowler <tressa at ucar.edu> wrote:
> Hi Ben,
>
> I don't have an answer, but I have a few more questions. Do you only
have
> the four variables listed (RMSE, OSTDEV, FSTDEV, and ESTDEV)? If so,
then
> you do not have ensemble spread and you will have to do more runs.
>
> The last 3 of these are the standard deviation of the observations,
of the
> ensemble mean, and of the error. They are not the ensemble spread
(though
> the ESTDEV is the 'error' for your plot). I think John added
variables from
> the ensemble stat output to the point stat lines in the last release
to
> handle the observation error perturbations. In older versions of
met, you
> used to have to save the ensemble stat output). However, they do not
appear
> in your list.
>
> I was going to check the latest version of the documentation, but I
don't
> have a copy and our web site is down. It seems like there is ORANK
info now
> in one of the line types. Do you have a copy to check?
>
> Tressa
>
> On Tue, Dec 18, 2018 at 11:53 AM Julie Prestopnik <jpresto at ucar.edu>
> wrote:
>
>> Hi Ben.
>>
>> Unfortunately, I do not know the answer to your question, and John
is out
>> of the office for a while due to the holidays. I have copied some
other
>> members of the MET team in hopes that someone else will know and be
able to
>> help. Thank you for your patience while we work toward getting an
answer
>> to you.
>>
>> Thanks,
>> Julie
>>
>> On Tue, Dec 18, 2018 at 11:27 AM Benjamin Blake - NOAA Affiliate
via RT <
>> met_help at ucar.edu> wrote:
>>
>>>
>>> Tue Dec 18 11:27:26 2018: Request 88227 was acted upon.
>>> Transaction: Ticket created by benjamin.blake at noaa.gov
>>> Queue: met_help
>>> Subject: Plotting ensemble spread/error curves
>>> Owner: Nobody
>>> Requestors: benjamin.blake at noaa.gov
>>> Status: new
>>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=88227 >
>>>
>>>
>>> Hi,
>>>
>>> I am interested in creating ensemble spread/error curves with MET
and
>>> METviewer. I loaded statistics for the ensemble mean from point
stat
>>> into
>>> METviewer, and I see that I have RMSE, OSTDEV, FSTDEV, and ESTDEV.
>>>
>>> For ensemble spread, it looks like I needed to also save output
from
>>> ensemble stat and load that into METviewer. I did not save any
output
>>> that
>>> I generated from ensemble stat, but it looks like SPREAD in the
RHIST
>>> line
>>> type could be what I am looking for because I want to calculate
spread
>>> over
>>> a masking region. I found this in the MET user's guide:
>>>
>>> SPREAD
>>> The ensemble spread for a single observation is the standard
deviation of
>>> the ensemble member forecast values at that location. When
verifying
>>> against point observations, these values are written to the
ENS_SPREAD
>>> column of the Observation Rank (ORANK) line type. The ensemble
spread
>>> computed over a spatial masking region is the average of the
spread
>>> values
>>> for all observations falling within the mask. These values are
written to
>>> the SPREAD column of the Ranked Histogram (RHIST) line type.
>>>
>>> Do I need to re-run ensemble stat to generate values for spread or
is it
>>> possible to calculate with the variables I already have from point
stat?
>>>
>>> Thanks!
>>> Ben
>>>
>>>
------------------------------------------------
Subject: Plotting ensemble spread/error curves
From: Benjamin Blake - NOAA Affiliate
Time: Wed Dec 19 05:49:16 2018
Hi Tressa,
Ok - thanks for letting me know! Happy holidays to you too.
Ben
On Tue, Dec 18, 2018 at 5:32 PM Tressa Fowler <tressa at ucar.edu> wrote:
> Hey Ben,
>
> Tara sent me the documentation (totally stupid of me to not have a
copy,
> but I just always look it up online . . .)
>
> Anyway, you need the ECNT or SSVAR line types from the ensemble stat
tool.
> That tool can read your observations and compare them to the
ensemble, so
> you should not need to run point stat at all. Both of those line
types have
> the RMSE of the ensemble mean and the average spread of the ensemble
> values. These two are your variables for the spread / skill plot.
>
> Let me know if you have more questions.
>
> Happy Holidays!
>
> Tressa
>
> On Tue, Dec 18, 2018 at 3:00 PM Tressa Fowler <tressa at ucar.edu>
wrote:
>
>> Hi Ben,
>>
>> I don't have an answer, but I have a few more questions. Do you
only have
>> the four variables listed (RMSE, OSTDEV, FSTDEV, and ESTDEV)? If
so,
>> then you do not have ensemble spread and you will have to do more
runs.
>>
>> The last 3 of these are the standard deviation of the observations,
of
>> the ensemble mean, and of the error. They are not the ensemble
spread
>> (though the ESTDEV is the 'error' for your plot). I think John
added
>> variables from the ensemble stat output to the point stat lines in
the last
>> release to handle the observation error perturbations. In older
versions of
>> met, you used to have to save the ensemble stat output). However,
they do
>> not appear in your list.
>>
>> I was going to check the latest version of the documentation, but I
don't
>> have a copy and our web site is down. It seems like there is ORANK
info now
>> in one of the line types. Do you have a copy to check?
>>
>> Tressa
>>
>> On Tue, Dec 18, 2018 at 11:53 AM Julie Prestopnik
<jpresto at ucar.edu>
>> wrote:
>>
>>> Hi Ben.
>>>
>>> Unfortunately, I do not know the answer to your question, and John
is
>>> out of the office for a while due to the holidays. I have copied
some
>>> other members of the MET team in hopes that someone else will know
and be
>>> able to help. Thank you for your patience while we work toward
getting an
>>> answer to you.
>>>
>>> Thanks,
>>> Julie
>>>
>>> On Tue, Dec 18, 2018 at 11:27 AM Benjamin Blake - NOAA Affiliate
via RT <
>>> met_help at ucar.edu> wrote:
>>>
>>>>
>>>> Tue Dec 18 11:27:26 2018: Request 88227 was acted upon.
>>>> Transaction: Ticket created by benjamin.blake at noaa.gov
>>>> Queue: met_help
>>>> Subject: Plotting ensemble spread/error curves
>>>> Owner: Nobody
>>>> Requestors: benjamin.blake at noaa.gov
>>>> Status: new
>>>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=88227 >
>>>>
>>>>
>>>> Hi,
>>>>
>>>> I am interested in creating ensemble spread/error curves with MET
and
>>>> METviewer. I loaded statistics for the ensemble mean from point
stat
>>>> into
>>>> METviewer, and I see that I have RMSE, OSTDEV, FSTDEV, and
ESTDEV.
>>>>
>>>> For ensemble spread, it looks like I needed to also save output
from
>>>> ensemble stat and load that into METviewer. I did not save any
output
>>>> that
>>>> I generated from ensemble stat, but it looks like SPREAD in the
RHIST
>>>> line
>>>> type could be what I am looking for because I want to calculate
spread
>>>> over
>>>> a masking region. I found this in the MET user's guide:
>>>>
>>>> SPREAD
>>>> The ensemble spread for a single observation is the standard
deviation
>>>> of
>>>> the ensemble member forecast values at that location. When
verifying
>>>> against point observations, these values are written to the
ENS_SPREAD
>>>> column of the Observation Rank (ORANK) line type. The ensemble
spread
>>>> computed over a spatial masking region is the average of the
spread
>>>> values
>>>> for all observations falling within the mask. These values are
written
>>>> to
>>>> the SPREAD column of the Ranked Histogram (RHIST) line type.
>>>>
>>>> Do I need to re-run ensemble stat to generate values for spread
or is it
>>>> possible to calculate with the variables I already have from
point stat?
>>>>
>>>> Thanks!
>>>> Ben
>>>>
>>>>
------------------------------------------------
Subject: Plotting ensemble spread/error curves
From: John Halley Gotway
Time: Thu Jan 03 14:11:04 2019
Hi Ben,
I assume that this issue is on hold during the shutdown. Tressa and
Tara
are correct, you can plot spread/skill information using either output
from
the SSVAR line type or the newer ECNT line type. The SSVAR line type
is
used to create the binned spread/skill type of plots recommended by
Tony
Eckel. The ECNT line type was added to create the more traditional
line
plots of spread and skill preferred by folks at NOAA-GSD.
Basically, the SSVAR line type is used to make line plots that have
the
ensemble spread on the X-axis and skill (RMSE) on the Y-axis. See
attached
ssvar_spread_skill.png plot. With this plot type, curves falling on
the
1-to-1 line are perfect.
The ECNT line type is used to make line plots of spread and skill.
For
example, for a particular ensemble, you could plot the spread and
skill as
separate lines versus forecast hour on the X-axis. In this case,
you're
looking to see if the spread and skill lines are close to eachother.
In
addition, you could add a derived curve where you compute the ratio of
spread divided by skill... and that would ideally be close to the
horizontal line where Y = 1.
I went hunting around on METviewer at the DTC, but since ECNT is still
pretty new, I couldn't find any good examples to show you.
Hopefully that helps clarify.
Thanks,
John
On Wed, Dec 19, 2018 at 5:49 AM Benjamin Blake - NOAA Affiliate via RT
<
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=88227 >
>
> Hi Tressa,
>
> Ok - thanks for letting me know! Happy holidays to you too.
>
> Ben
>
> On Tue, Dec 18, 2018 at 5:32 PM Tressa Fowler <tressa at ucar.edu>
wrote:
>
> > Hey Ben,
> >
> > Tara sent me the documentation (totally stupid of me to not have a
copy,
> > but I just always look it up online . . .)
> >
> > Anyway, you need the ECNT or SSVAR line types from the ensemble
stat
> tool.
> > That tool can read your observations and compare them to the
ensemble, so
> > you should not need to run point stat at all. Both of those line
types
> have
> > the RMSE of the ensemble mean and the average spread of the
ensemble
> > values. These two are your variables for the spread / skill plot.
> >
> > Let me know if you have more questions.
> >
> > Happy Holidays!
> >
> > Tressa
> >
> > On Tue, Dec 18, 2018 at 3:00 PM Tressa Fowler <tressa at ucar.edu>
wrote:
> >
> >> Hi Ben,
> >>
> >> I don't have an answer, but I have a few more questions. Do you
only
> have
> >> the four variables listed (RMSE, OSTDEV, FSTDEV, and ESTDEV)? If
so,
> >> then you do not have ensemble spread and you will have to do more
runs.
> >>
> >> The last 3 of these are the standard deviation of the
observations, of
> >> the ensemble mean, and of the error. They are not the ensemble
spread
> >> (though the ESTDEV is the 'error' for your plot). I think John
added
> >> variables from the ensemble stat output to the point stat lines
in the
> last
> >> release to handle the observation error perturbations. In older
> versions of
> >> met, you used to have to save the ensemble stat output). However,
they
> do
> >> not appear in your list.
> >>
> >> I was going to check the latest version of the documentation, but
I
> don't
> >> have a copy and our web site is down. It seems like there is
ORANK info
> now
> >> in one of the line types. Do you have a copy to check?
> >>
> >> Tressa
> >>
> >> On Tue, Dec 18, 2018 at 11:53 AM Julie Prestopnik
<jpresto at ucar.edu>
> >> wrote:
> >>
> >>> Hi Ben.
> >>>
> >>> Unfortunately, I do not know the answer to your question, and
John is
> >>> out of the office for a while due to the holidays. I have
copied some
> >>> other members of the MET team in hopes that someone else will
know and
> be
> >>> able to help. Thank you for your patience while we work toward
> getting an
> >>> answer to you.
> >>>
> >>> Thanks,
> >>> Julie
> >>>
> >>> On Tue, Dec 18, 2018 at 11:27 AM Benjamin Blake - NOAA Affiliate
via
> RT <
> >>> met_help at ucar.edu> wrote:
> >>>
> >>>>
> >>>> Tue Dec 18 11:27:26 2018: Request 88227 was acted upon.
> >>>> Transaction: Ticket created by benjamin.blake at noaa.gov
> >>>> Queue: met_help
> >>>> Subject: Plotting ensemble spread/error curves
> >>>> Owner: Nobody
> >>>> Requestors: benjamin.blake at noaa.gov
> >>>> Status: new
> >>>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=88227
> >
> >>>>
> >>>>
> >>>> Hi,
> >>>>
> >>>> I am interested in creating ensemble spread/error curves with
MET and
> >>>> METviewer. I loaded statistics for the ensemble mean from
point stat
> >>>> into
> >>>> METviewer, and I see that I have RMSE, OSTDEV, FSTDEV, and
ESTDEV.
> >>>>
> >>>> For ensemble spread, it looks like I needed to also save output
from
> >>>> ensemble stat and load that into METviewer. I did not save any
output
> >>>> that
> >>>> I generated from ensemble stat, but it looks like SPREAD in the
RHIST
> >>>> line
> >>>> type could be what I am looking for because I want to calculate
spread
> >>>> over
> >>>> a masking region. I found this in the MET user's guide:
> >>>>
> >>>> SPREAD
> >>>> The ensemble spread for a single observation is the standard
deviation
> >>>> of
> >>>> the ensemble member forecast values at that location. When
verifying
> >>>> against point observations, these values are written to the
ENS_SPREAD
> >>>> column of the Observation Rank (ORANK) line type. The ensemble
spread
> >>>> computed over a spatial masking region is the average of the
spread
> >>>> values
> >>>> for all observations falling within the mask. These values are
written
> >>>> to
> >>>> the SPREAD column of the Ranked Histogram (RHIST) line type.
> >>>>
> >>>> Do I need to re-run ensemble stat to generate values for spread
or is
> it
> >>>> possible to calculate with the variables I already have from
point
> stat?
> >>>>
> >>>> Thanks!
> >>>> Ben
> >>>>
> >>>>
>
>
------------------------------------------------
Subject: Plotting ensemble spread/error curves
From: Benjamin Blake - NOAA Affiliate
Time: Mon Jan 28 05:56:45 2019
Hi John,
Thank you for the further clarification! I believe line plots of
spread
and skill as a function of forecast hour are what I'm most interested
in,
so it sounds like I would just need to re-run ensemble stat and turn
on the
ECNT line type.
Ben
On Thu, Jan 3, 2019 at 4:11 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Hi Ben,
>
> I assume that this issue is on hold during the shutdown. Tressa and
Tara
> are correct, you can plot spread/skill information using either
output from
> the SSVAR line type or the newer ECNT line type. The SSVAR line
type is
> used to create the binned spread/skill type of plots recommended by
Tony
> Eckel. The ECNT line type was added to create the more traditional
line
> plots of spread and skill preferred by folks at NOAA-GSD.
>
> Basically, the SSVAR line type is used to make line plots that have
the
> ensemble spread on the X-axis and skill (RMSE) on the Y-axis. See
attached
> ssvar_spread_skill.png plot. With this plot type, curves falling on
the
> 1-to-1 line are perfect.
>
> The ECNT line type is used to make line plots of spread and skill.
For
> example, for a particular ensemble, you could plot the spread and
skill as
> separate lines versus forecast hour on the X-axis. In this case,
you're
> looking to see if the spread and skill lines are close to eachother.
In
> addition, you could add a derived curve where you compute the ratio
of
> spread divided by skill... and that would ideally be close to the
> horizontal line where Y = 1.
>
> I went hunting around on METviewer at the DTC, but since ECNT is
still
> pretty new, I couldn't find any good examples to show you.
>
> Hopefully that helps clarify.
>
> Thanks,
> John
>
>
>
>
>
> On Wed, Dec 19, 2018 at 5:49 AM Benjamin Blake - NOAA Affiliate via
RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=88227 >
> >
> > Hi Tressa,
> >
> > Ok - thanks for letting me know! Happy holidays to you too.
> >
> > Ben
> >
> > On Tue, Dec 18, 2018 at 5:32 PM Tressa Fowler <tressa at ucar.edu>
wrote:
> >
> > > Hey Ben,
> > >
> > > Tara sent me the documentation (totally stupid of me to not have
a
> copy,
> > > but I just always look it up online . . .)
> > >
> > > Anyway, you need the ECNT or SSVAR line types from the ensemble
stat
> > tool.
> > > That tool can read your observations and compare them to the
ensemble,
> so
> > > you should not need to run point stat at all. Both of those line
types
> > have
> > > the RMSE of the ensemble mean and the average spread of the
ensemble
> > > values. These two are your variables for the spread / skill
plot.
> > >
> > > Let me know if you have more questions.
> > >
> > > Happy Holidays!
> > >
> > > Tressa
> > >
> > > On Tue, Dec 18, 2018 at 3:00 PM Tressa Fowler <tressa at ucar.edu>
wrote:
> > >
> > >> Hi Ben,
> > >>
> > >> I don't have an answer, but I have a few more questions. Do you
only
> > have
> > >> the four variables listed (RMSE, OSTDEV, FSTDEV, and ESTDEV)?
If so,
> > >> then you do not have ensemble spread and you will have to do
more
> runs.
> > >>
> > >> The last 3 of these are the standard deviation of the
observations, of
> > >> the ensemble mean, and of the error. They are not the ensemble
spread
> > >> (though the ESTDEV is the 'error' for your plot). I think John
added
> > >> variables from the ensemble stat output to the point stat lines
in the
> > last
> > >> release to handle the observation error perturbations. In older
> > versions of
> > >> met, you used to have to save the ensemble stat output).
However, they
> > do
> > >> not appear in your list.
> > >>
> > >> I was going to check the latest version of the documentation,
but I
> > don't
> > >> have a copy and our web site is down. It seems like there is
ORANK
> info
> > now
> > >> in one of the line types. Do you have a copy to check?
> > >>
> > >> Tressa
> > >>
> > >> On Tue, Dec 18, 2018 at 11:53 AM Julie Prestopnik
<jpresto at ucar.edu>
> > >> wrote:
> > >>
> > >>> Hi Ben.
> > >>>
> > >>> Unfortunately, I do not know the answer to your question, and
John is
> > >>> out of the office for a while due to the holidays. I have
copied
> some
> > >>> other members of the MET team in hopes that someone else will
know
> and
> > be
> > >>> able to help. Thank you for your patience while we work
toward
> > getting an
> > >>> answer to you.
> > >>>
> > >>> Thanks,
> > >>> Julie
> > >>>
> > >>> On Tue, Dec 18, 2018 at 11:27 AM Benjamin Blake - NOAA
Affiliate via
> > RT <
> > >>> met_help at ucar.edu> wrote:
> > >>>
> > >>>>
> > >>>> Tue Dec 18 11:27:26 2018: Request 88227 was acted upon.
> > >>>> Transaction: Ticket created by benjamin.blake at noaa.gov
> > >>>> Queue: met_help
> > >>>> Subject: Plotting ensemble spread/error curves
> > >>>> Owner: Nobody
> > >>>> Requestors: benjamin.blake at noaa.gov
> > >>>> Status: new
> > >>>> Ticket <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=88227
> > >
> > >>>>
> > >>>>
> > >>>> Hi,
> > >>>>
> > >>>> I am interested in creating ensemble spread/error curves with
MET
> and
> > >>>> METviewer. I loaded statistics for the ensemble mean from
point
> stat
> > >>>> into
> > >>>> METviewer, and I see that I have RMSE, OSTDEV, FSTDEV, and
ESTDEV.
> > >>>>
> > >>>> For ensemble spread, it looks like I needed to also save
output from
> > >>>> ensemble stat and load that into METviewer. I did not save
any
> output
> > >>>> that
> > >>>> I generated from ensemble stat, but it looks like SPREAD in
the
> RHIST
> > >>>> line
> > >>>> type could be what I am looking for because I want to
calculate
> spread
> > >>>> over
> > >>>> a masking region. I found this in the MET user's guide:
> > >>>>
> > >>>> SPREAD
> > >>>> The ensemble spread for a single observation is the standard
> deviation
> > >>>> of
> > >>>> the ensemble member forecast values at that location. When
verifying
> > >>>> against point observations, these values are written to the
> ENS_SPREAD
> > >>>> column of the Observation Rank (ORANK) line type. The
ensemble
> spread
> > >>>> computed over a spatial masking region is the average of the
spread
> > >>>> values
> > >>>> for all observations falling within the mask. These values
are
> written
> > >>>> to
> > >>>> the SPREAD column of the Ranked Histogram (RHIST) line type.
> > >>>>
> > >>>> Do I need to re-run ensemble stat to generate values for
spread or
> is
> > it
> > >>>> possible to calculate with the variables I already have from
point
> > stat?
> > >>>>
> > >>>> Thanks!
> > >>>> Ben
> > >>>>
> > >>>>
> >
> >
>
>
------------------------------------------------
Subject: Plotting ensemble spread/error curves
From: John Halley Gotway
Time: Mon Jan 28 10:25:23 2019
Ben,
Glad you guys are back in the office. I'll go ahead and resolve this
ticket. But don't hesitate to write if more issues or questions
arise.
Thanks,
John
On Mon, Jan 28, 2019 at 5:57 AM Benjamin Blake - NOAA Affiliate via RT
<
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=88227 >
>
> Hi John,
>
> Thank you for the further clarification! I believe line plots of
spread
> and skill as a function of forecast hour are what I'm most
interested in,
> so it sounds like I would just need to re-run ensemble stat and turn
on the
> ECNT line type.
>
> Ben
>
> On Thu, Jan 3, 2019 at 4:11 PM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > Hi Ben,
> >
> > I assume that this issue is on hold during the shutdown. Tressa
and Tara
> > are correct, you can plot spread/skill information using either
output
> from
> > the SSVAR line type or the newer ECNT line type. The SSVAR line
type is
> > used to create the binned spread/skill type of plots recommended
by Tony
> > Eckel. The ECNT line type was added to create the more
traditional line
> > plots of spread and skill preferred by folks at NOAA-GSD.
> >
> > Basically, the SSVAR line type is used to make line plots that
have the
> > ensemble spread on the X-axis and skill (RMSE) on the Y-axis. See
> attached
> > ssvar_spread_skill.png plot. With this plot type, curves falling
on the
> > 1-to-1 line are perfect.
> >
> > The ECNT line type is used to make line plots of spread and skill.
For
> > example, for a particular ensemble, you could plot the spread and
skill
> as
> > separate lines versus forecast hour on the X-axis. In this case,
you're
> > looking to see if the spread and skill lines are close to
eachother. In
> > addition, you could add a derived curve where you compute the
ratio of
> > spread divided by skill... and that would ideally be close to the
> > horizontal line where Y = 1.
> >
> > I went hunting around on METviewer at the DTC, but since ECNT is
still
> > pretty new, I couldn't find any good examples to show you.
> >
> > Hopefully that helps clarify.
> >
> > Thanks,
> > John
> >
> >
> >
> >
> >
> > On Wed, Dec 19, 2018 at 5:49 AM Benjamin Blake - NOAA Affiliate
via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=88227 >
> > >
> > > Hi Tressa,
> > >
> > > Ok - thanks for letting me know! Happy holidays to you too.
> > >
> > > Ben
> > >
> > > On Tue, Dec 18, 2018 at 5:32 PM Tressa Fowler <tressa at ucar.edu>
wrote:
> > >
> > > > Hey Ben,
> > > >
> > > > Tara sent me the documentation (totally stupid of me to not
have a
> > copy,
> > > > but I just always look it up online . . .)
> > > >
> > > > Anyway, you need the ECNT or SSVAR line types from the
ensemble stat
> > > tool.
> > > > That tool can read your observations and compare them to the
> ensemble,
> > so
> > > > you should not need to run point stat at all. Both of those
line
> types
> > > have
> > > > the RMSE of the ensemble mean and the average spread of the
ensemble
> > > > values. These two are your variables for the spread / skill
plot.
> > > >
> > > > Let me know if you have more questions.
> > > >
> > > > Happy Holidays!
> > > >
> > > > Tressa
> > > >
> > > > On Tue, Dec 18, 2018 at 3:00 PM Tressa Fowler
<tressa at ucar.edu>
> wrote:
> > > >
> > > >> Hi Ben,
> > > >>
> > > >> I don't have an answer, but I have a few more questions. Do
you only
> > > have
> > > >> the four variables listed (RMSE, OSTDEV, FSTDEV, and ESTDEV)?
If so,
> > > >> then you do not have ensemble spread and you will have to do
more
> > runs.
> > > >>
> > > >> The last 3 of these are the standard deviation of the
observations,
> of
> > > >> the ensemble mean, and of the error. They are not the
ensemble
> spread
> > > >> (though the ESTDEV is the 'error' for your plot). I think
John added
> > > >> variables from the ensemble stat output to the point stat
lines in
> the
> > > last
> > > >> release to handle the observation error perturbations. In
older
> > > versions of
> > > >> met, you used to have to save the ensemble stat output).
However,
> they
> > > do
> > > >> not appear in your list.
> > > >>
> > > >> I was going to check the latest version of the documentation,
but I
> > > don't
> > > >> have a copy and our web site is down. It seems like there is
ORANK
> > info
> > > now
> > > >> in one of the line types. Do you have a copy to check?
> > > >>
> > > >> Tressa
> > > >>
> > > >> On Tue, Dec 18, 2018 at 11:53 AM Julie Prestopnik
<jpresto at ucar.edu
> >
> > > >> wrote:
> > > >>
> > > >>> Hi Ben.
> > > >>>
> > > >>> Unfortunately, I do not know the answer to your question,
and John
> is
> > > >>> out of the office for a while due to the holidays. I have
copied
> > some
> > > >>> other members of the MET team in hopes that someone else
will know
> > and
> > > be
> > > >>> able to help. Thank you for your patience while we work
toward
> > > getting an
> > > >>> answer to you.
> > > >>>
> > > >>> Thanks,
> > > >>> Julie
> > > >>>
> > > >>> On Tue, Dec 18, 2018 at 11:27 AM Benjamin Blake - NOAA
Affiliate
> via
> > > RT <
> > > >>> met_help at ucar.edu> wrote:
> > > >>>
> > > >>>>
> > > >>>> Tue Dec 18 11:27:26 2018: Request 88227 was acted upon.
> > > >>>> Transaction: Ticket created by benjamin.blake at noaa.gov
> > > >>>> Queue: met_help
> > > >>>> Subject: Plotting ensemble spread/error curves
> > > >>>> Owner: Nobody
> > > >>>> Requestors: benjamin.blake at noaa.gov
> > > >>>> Status: new
> > > >>>> Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=88227
> > > >
> > > >>>>
> > > >>>>
> > > >>>> Hi,
> > > >>>>
> > > >>>> I am interested in creating ensemble spread/error curves
with MET
> > and
> > > >>>> METviewer. I loaded statistics for the ensemble mean from
point
> > stat
> > > >>>> into
> > > >>>> METviewer, and I see that I have RMSE, OSTDEV, FSTDEV, and
ESTDEV.
> > > >>>>
> > > >>>> For ensemble spread, it looks like I needed to also save
output
> from
> > > >>>> ensemble stat and load that into METviewer. I did not save
any
> > output
> > > >>>> that
> > > >>>> I generated from ensemble stat, but it looks like SPREAD in
the
> > RHIST
> > > >>>> line
> > > >>>> type could be what I am looking for because I want to
calculate
> > spread
> > > >>>> over
> > > >>>> a masking region. I found this in the MET user's guide:
> > > >>>>
> > > >>>> SPREAD
> > > >>>> The ensemble spread for a single observation is the
standard
> > deviation
> > > >>>> of
> > > >>>> the ensemble member forecast values at that location. When
> verifying
> > > >>>> against point observations, these values are written to the
> > ENS_SPREAD
> > > >>>> column of the Observation Rank (ORANK) line type. The
ensemble
> > spread
> > > >>>> computed over a spatial masking region is the average of
the
> spread
> > > >>>> values
> > > >>>> for all observations falling within the mask. These values
are
> > written
> > > >>>> to
> > > >>>> the SPREAD column of the Ranked Histogram (RHIST) line
type.
> > > >>>>
> > > >>>> Do I need to re-run ensemble stat to generate values for
spread or
> > is
> > > it
> > > >>>> possible to calculate with the variables I already have
from point
> > > stat?
> > > >>>>
> > > >>>> Thanks!
> > > >>>> Ben
> > > >>>>
> > > >>>>
> > >
> > >
> >
> >
>
>
------------------------------------------------
More information about the Met_help
mailing list