[Met_help] [rt.rap.ucar.edu #100229] History for Question about RMSE in MET
George McCabe via RT
met_help at ucar.edu
Wed Aug 18 09:02:33 MDT 2021
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hi,
I'm wondering if the RMSE as it is currently calculated in MET is centered
(as, for example, the Anomaly Correlation Coefficient is centered) or if
the centered RMSE can be derived using other MET statistics? If not, would
it be possible to include a centered version of the RMSE among the
continuous statistics offered?
I would find it useful to create versions of the Taylor Diagram–for
example, one that displays the centered Anomaly Correlation Coefficient–and
these versions would require either the centered RMSE or the standard
deviations of both the forecast and observed anomalies. Admittedly I
haven't seen the centered RMSE used in applications other than Taylor
Diagrams.
For context, I'm using MET v9.0.2 and METplus v3.0.2
Best,
-Marcel
--
*___________________________________*
*Marcel G. Caron*
Physical Scientist, I.M. Systems Group, Inc.
Affiliate, Model Evaluation Group
@NOAA/NWS/NCEP/EMC
*5830 University Research Court,*
*College Park, MD 20740*
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Question about RMSE in MET
From: Minna Win
Time: Tue Jun 15 10:44:03 2021
Hi Marcel,
It looks like you have a question about how RMSE is implemented in
MET. I
am going to consult one of our statisticians to get the answer for
you. I
also noticed that you are not using the latest version of MET and
METplus.
There are a number of bug fixes that are in MET 10.0 and METplus 4.0
release, so if possible, can you migrate to a NOAA host that has the
latest
versions?
Regards,
Minna
---------------
Minna Win
Pronouns: she/her
National Center for Atmospheric Research
Developmental Testbed Center
Phone: 303-497-8423
Fax: 303-497-8401
---------------
On Tue, Jun 15, 2021 at 8:03 AM Marcel Caron - NOAA Affiliate via RT <
met_help at ucar.edu> wrote:
>
> Tue Jun 15 08:03:27 2021: Request 100229 was acted upon.
> Transaction: Ticket created by marcel.caron at noaa.gov
> Queue: met_help
> Subject: Question about RMSE in MET
> Owner: Nobody
> Requestors: marcel.caron at noaa.gov
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=100229 >
>
>
> Hi,
>
> I'm wondering if the RMSE as it is currently calculated in MET is
centered
> (as, for example, the Anomaly Correlation Coefficient is centered)
or if
> the centered RMSE can be derived using other MET statistics? If not,
would
> it be possible to include a centered version of the RMSE among the
> continuous statistics offered?
>
> I would find it useful to create versions of the Taylor Diagram–for
> example, one that displays the centered Anomaly Correlation
Coefficient–and
> these versions would require either the centered RMSE or the
standard
> deviations of both the forecast and observed anomalies. Admittedly
I
> haven't seen the centered RMSE used in applications other than
Taylor
> Diagrams.
>
> For context, I'm using MET v9.0.2 and METplus v3.0.2
>
> Best,
> -Marcel
>
> --
> *___________________________________*
> *Marcel G. Caron*
> Physical Scientist, I.M. Systems Group, Inc.
> Affiliate, Model Evaluation Group
> @NOAA/NWS/NCEP/EMC
> *5830 University Research Court,*
> *College Park, MD 20740*
>
>
------------------------------------------------
Subject: Question about RMSE in MET
From: Barbara Brown
Time: Mon Jun 21 16:03:30 2021
On Tue Jun 15 08:03:27 2021, marcel.caron at noaa.gov wrote:
> Hi,
>
> I'm wondering if the RMSE as it is currently calculated in MET is
centered
> (as, for example, the Anomaly Correlation Coefficient is centered)
or if
> the centered RMSE can be derived using other MET statistics? If not,
would
> it be possible to include a centered version of the RMSE among the
> continuous statistics offered?
>
> I would find it useful to create versions of the Taylor Diagram–for
> example, one that displays the centered Anomaly Correlation
Coefficient–and
> these versions would require either the centered RMSE or the
standard
> deviations of both the forecast and observed anomalies. Admittedly
I
> haven't seen the centered RMSE used in applications other than
Taylor
> Diagrams.
>
> For context, I'm using MET v9.0.2 and METplus v3.0.2
>
> Best,
> -Marcel
>
Hi Marcel,
The standard MSE in MET is not centered. However, MET also computes
the "Bias-corrected MSE (BCMSE)", which is equivalent to the centered
MSE described by Taylor (and used to create a Taylor diagram). This
statistic is described in the documentation for point-stat and is also
computed by grid-stat; BCMSE is also defined in Appendix C of the MET
User's Guide.
I also want to note that METviewer includes an option to create Taylor
diagrams, which is described in the METviewer documentation.
Please let us know if you have any additional questions.
Barbara Brown
------------------------------------------------
Subject: Question about RMSE in MET
From: Marcel Caron - NOAA Affiliate
Time: Tue Jun 22 06:31:24 2021
Barbara:
Thanks for the clarification. I'll take a look in the PointStat
documentation for BCMSE.
Best,
-Marcel
On Mon, Jun 21, 2021 at 6:03 PM Barbara Brown via RT
<met_help at ucar.edu>
wrote:
> On Tue Jun 15 08:03:27 2021, marcel.caron at noaa.gov wrote:
> > Hi,
> >
> > I'm wondering if the RMSE as it is currently calculated in MET is
> centered
> > (as, for example, the Anomaly Correlation Coefficient is centered)
or if
> > the centered RMSE can be derived using other MET statistics? If
not,
> would
> > it be possible to include a centered version of the RMSE among the
> > continuous statistics offered?
> >
> > I would find it useful to create versions of the Taylor
Diagram–for
> > example, one that displays the centered Anomaly Correlation
> Coefficient–and
> > these versions would require either the centered RMSE or the
standard
> > deviations of both the forecast and observed anomalies.
Admittedly I
> > haven't seen the centered RMSE used in applications other than
Taylor
> > Diagrams.
> >
> > For context, I'm using MET v9.0.2 and METplus v3.0.2
> >
> > Best,
> > -Marcel
> >
>
> Hi Marcel,
>
> The standard MSE in MET is not centered. However, MET also computes
the
> "Bias-corrected MSE (BCMSE)", which is equivalent to the centered
MSE
> described by Taylor (and used to create a Taylor diagram). This
statistic
> is described in the documentation for point-stat and is also
computed by
> grid-stat; BCMSE is also defined in Appendix C of the MET User's
Guide.
>
> I also want to note that METviewer includes an option to create
Taylor
> diagrams, which is described in the METviewer documentation.
>
> Please let us know if you have any additional questions.
>
> Barbara Brown
>
------------------------------------------------
Subject: Question about RMSE in MET
From: Minna Win
Time: Tue Jun 22 09:14:33 2021
Hi Barb,
Thanks for answering the question. Your response was correctly sent
to the requestor.
Regards,
Minna
------------------------------------------------
Subject: Question about RMSE in MET
From: Barbara Brown
Time: Tue Jun 22 10:04:07 2021
Great! Thanks, Minna.
Barb
---
Barbara Brown, Senior Research Associate
Research Applications Laboratory
NCAR PO Box 3000
Boulder CO 80307-3000 USA
Ph: +1 303 497 8468 FAX: +1 303 497 8401
On Tue, Jun 22, 2021 at 9:14 AM Minna Win via RT <
met_help-comment at rap.ucar.edu> wrote:
> Hi Barb,
>
> Thanks for answering the question. Your response was correctly sent
to
> the requestor.
>
> Regards,
> Minna
>
------------------------------------------------
Subject: Question about RMSE in MET
From: Marcel Caron - NOAA Affiliate
Time: Wed Jun 23 09:43:58 2021
Hi Barbara:
I have one quick follow-up question about the available centered
statistics. The scalar L1L2 stats include the mean forecast and
reference
anomalies (FABAR and OABAR), but not their standard deviations, which
would
be useful for normalizing the BCMSE. Is there a way to calculate the
standard deviation of the anomaly fields using the provided stats?
Best,
-Marcel
On Tue, Jun 22, 2021 at 8:31 AM Marcel Caron - NOAA Affiliate <
marcel.caron at noaa.gov> wrote:
> Barbara:
>
> Thanks for the clarification. I'll take a look in the PointStat
> documentation for BCMSE.
>
> Best,
> -Marcel
>
> On Mon, Jun 21, 2021 at 6:03 PM Barbara Brown via RT
<met_help at ucar.edu>
> wrote:
>
>> On Tue Jun 15 08:03:27 2021, marcel.caron at noaa.gov wrote:
>> > Hi,
>> >
>> > I'm wondering if the RMSE as it is currently calculated in MET is
>> centered
>> > (as, for example, the Anomaly Correlation Coefficient is
centered) or if
>> > the centered RMSE can be derived using other MET statistics? If
not,
>> would
>> > it be possible to include a centered version of the RMSE among
the
>> > continuous statistics offered?
>> >
>> > I would find it useful to create versions of the Taylor
Diagram–for
>> > example, one that displays the centered Anomaly Correlation
>> Coefficient–and
>> > these versions would require either the centered RMSE or the
standard
>> > deviations of both the forecast and observed anomalies.
Admittedly I
>> > haven't seen the centered RMSE used in applications other than
Taylor
>> > Diagrams.
>> >
>> > For context, I'm using MET v9.0.2 and METplus v3.0.2
>> >
>> > Best,
>> > -Marcel
>> >
>>
>> Hi Marcel,
>>
>> The standard MSE in MET is not centered. However, MET also computes
the
>> "Bias-corrected MSE (BCMSE)", which is equivalent to the centered
MSE
>> described by Taylor (and used to create a Taylor diagram). This
statistic
>> is described in the documentation for point-stat and is also
computed by
>> grid-stat; BCMSE is also defined in Appendix C of the MET User's
Guide.
>>
>> I also want to note that METviewer includes an option to create
Taylor
>> diagrams, which is described in the METviewer documentation.
>>
>> Please let us know if you have any additional questions.
>>
>> Barbara Brown
>>
>
------------------------------------------------
Subject: Question about RMSE in MET
From: Barbara Brown
Time: Thu Jul 01 17:53:08 2021
Hi Marcel,
I apologize for the delay in responding to your question. Though I had
expected to be able to respond earlier, I was occupied with a workshop
last week, which prevented me from looking into it, and then out of
the office for a few days.
If I am understanding your question correctly, the statistics you are
asking for are FFABAR and OOABAR (see Table 9.16 in the MET
documentation; and Appendix C, Section 31.3.25). Please let me know
if I am mis-understanding!
Barbara Brown
On Wed Jun 23 09:43:58 2021, marcel.caron at noaa.gov wrote:
> Hi Barbara:
>
> I have one quick follow-up question about the available centered
> statistics. The scalar L1L2 stats include the mean forecast and
> reference
> anomalies (FABAR and OABAR), but not their standard deviations,
which
> would
> be useful for normalizing the BCMSE. Is there a way to calculate
the
> standard deviation of the anomaly fields using the provided stats?
>
> Best,
> -Marcel
>
> On Tue, Jun 22, 2021 at 8:31 AM Marcel Caron - NOAA Affiliate <
> marcel.caron at noaa.gov> wrote:
>
> > Barbara:
> >
> > Thanks for the clarification. I'll take a look in the PointStat
> > documentation for BCMSE.
> >
> > Best,
> > -Marcel
> >
> > On Mon, Jun 21, 2021 at 6:03 PM Barbara Brown via RT
> > <met_help at ucar.edu>
> > wrote:
> >
> >> On Tue Jun 15 08:03:27 2021, marcel.caron at noaa.gov wrote:
> >> > Hi,
> >> >
> >> > I'm wondering if the RMSE as it is currently calculated in MET
is
> >> centered
> >> > (as, for example, the Anomaly Correlation Coefficient is
centered)
> >> > or if
> >> > the centered RMSE can be derived using other MET statistics? If
> >> > not,
> >> would
> >> > it be possible to include a centered version of the RMSE among
the
> >> > continuous statistics offered?
> >> >
> >> > I would find it useful to create versions of the Taylor
> >> > Diagram–for
> >> > example, one that displays the centered Anomaly Correlation
> >> Coefficient–and
> >> > these versions would require either the centered RMSE or the
> >> > standard
> >> > deviations of both the forecast and observed anomalies.
> >> > Admittedly I
> >> > haven't seen the centered RMSE used in applications other than
> >> > Taylor
> >> > Diagrams.
> >> >
> >> > For context, I'm using MET v9.0.2 and METplus v3.0.2
> >> >
> >> > Best,
> >> > -Marcel
> >> >
> >>
> >> Hi Marcel,
> >>
> >> The standard MSE in MET is not centered. However, MET also
computes
> >> the
> >> "Bias-corrected MSE (BCMSE)", which is equivalent to the centered
> >> MSE
> >> described by Taylor (and used to create a Taylor diagram). This
> >> statistic
> >> is described in the documentation for point-stat and is also
> >> computed by
> >> grid-stat; BCMSE is also defined in Appendix C of the MET User's
> >> Guide.
> >>
> >> I also want to note that METviewer includes an option to create
> >> Taylor
> >> diagrams, which is described in the METviewer documentation.
> >>
> >> Please let us know if you have any additional questions.
> >>
> >> Barbara Brown
> >>
> >
------------------------------------------------
Subject: Question about RMSE in MET
From: Marcel Caron - NOAA Affiliate
Time: Thu Jul 01 22:56:39 2021
Hi Barbara:
Not a problem at all, it is a busy time of year.
I think OOABAR and FFABAR are very close to what I'm asking about. Do
you
know if there is a version of OOABAR or FFABAR with the input
variables (f,
o, and c) centered, that is:
[image: image.png]
Or perhaps there is a way to derive this using the statistics already
available?
Best,
-Marcel
On Thu, Jul 1, 2021 at 7:53 PM Barbara Brown via RT
<met_help at ucar.edu>
wrote:
> Hi Marcel,
>
> I apologize for the delay in responding to your question. Though I
had
> expected to be able to respond earlier, I was occupied with a
workshop last
> week, which prevented me from looking into it, and then out of the
office
> for a few days.
>
> If I am understanding your question correctly, the statistics you
are
> asking for are FFABAR and OOABAR (see Table 9.16 in the MET
documentation;
> and Appendix C, Section 31.3.25). Please let me know if I am
> mis-understanding!
>
> Barbara Brown
>
>
> On Wed Jun 23 09:43:58 2021, marcel.caron at noaa.gov wrote:
> > Hi Barbara:
> >
> > I have one quick follow-up question about the available centered
> > statistics. The scalar L1L2 stats include the mean forecast and
> > reference
> > anomalies (FABAR and OABAR), but not their standard deviations,
which
> > would
> > be useful for normalizing the BCMSE. Is there a way to calculate
the
> > standard deviation of the anomaly fields using the provided stats?
> >
> > Best,
> > -Marcel
> >
> > On Tue, Jun 22, 2021 at 8:31 AM Marcel Caron - NOAA Affiliate <
> > marcel.caron at noaa.gov> wrote:
> >
> > > Barbara:
> > >
> > > Thanks for the clarification. I'll take a look in the PointStat
> > > documentation for BCMSE.
> > >
> > > Best,
> > > -Marcel
> > >
> > > On Mon, Jun 21, 2021 at 6:03 PM Barbara Brown via RT
> > > <met_help at ucar.edu>
> > > wrote:
> > >
> > >> On Tue Jun 15 08:03:27 2021, marcel.caron at noaa.gov wrote:
> > >> > Hi,
> > >> >
> > >> > I'm wondering if the RMSE as it is currently calculated in
MET is
> > >> centered
> > >> > (as, for example, the Anomaly Correlation Coefficient is
centered)
> > >> > or if
> > >> > the centered RMSE can be derived using other MET statistics?
If
> > >> > not,
> > >> would
> > >> > it be possible to include a centered version of the RMSE
among the
> > >> > continuous statistics offered?
> > >> >
> > >> > I would find it useful to create versions of the Taylor
> > >> > Diagram–for
> > >> > example, one that displays the centered Anomaly Correlation
> > >> Coefficient–and
> > >> > these versions would require either the centered RMSE or the
> > >> > standard
> > >> > deviations of both the forecast and observed anomalies.
> > >> > Admittedly I
> > >> > haven't seen the centered RMSE used in applications other
than
> > >> > Taylor
> > >> > Diagrams.
> > >> >
> > >> > For context, I'm using MET v9.0.2 and METplus v3.0.2
> > >> >
> > >> > Best,
> > >> > -Marcel
> > >> >
> > >>
> > >> Hi Marcel,
> > >>
> > >> The standard MSE in MET is not centered. However, MET also
computes
> > >> the
> > >> "Bias-corrected MSE (BCMSE)", which is equivalent to the
centered
> > >> MSE
> > >> described by Taylor (and used to create a Taylor diagram). This
> > >> statistic
> > >> is described in the documentation for point-stat and is also
> > >> computed by
> > >> grid-stat; BCMSE is also defined in Appendix C of the MET
User's
> > >> Guide.
> > >>
> > >> I also want to note that METviewer includes an option to create
> > >> Taylor
> > >> diagrams, which is described in the METviewer documentation.
> > >>
> > >> Please let us know if you have any additional questions.
> > >>
> > >> Barbara Brown
> > >>
> > >
>
>
>
>
------------------------------------------------
Subject: Question about RMSE in MET
From: Barbara Brown
Time: Fri Jul 02 15:52:23 2021
Hi Marcel,
I apologize for the delay in responding to your question. Though I had
expected to be able to respond earlier, I was occupied with a workshop
last week, which prevented me from looking into it, and then out of
the office for a few days.
If I am understanding your question correctly, the statistics you are
asking for are FFABAR and OOABAR (see Table 9.16 in the MET
documentation; and Appendix C, Section 31.3.25). Please let me know
if I am mis-understanding!
Barbara Brown
On Wed Jun 23 09:43:58 2021, marcel.caron at noaa.gov wrote:
> Hi Barbara:
>
> I have one quick follow-up question about the available centered
> statistics. The scalar L1L2 stats include the mean forecast and
> reference
> anomalies (FABAR and OABAR), but not their standard deviations,
which
> would
> be useful for normalizing the BCMSE. Is there a way to calculate
the
> standard deviation of the anomaly fields using the provided stats?
>
> Best,
> -Marcel
>
> On Tue, Jun 22, 2021 at 8:31 AM Marcel Caron - NOAA Affiliate <
> marcel.caron at noaa.gov> wrote:
>
> > Barbara:
> >
> > Thanks for the clarification. I'll take a look in the PointStat
> > documentation for BCMSE.
> >
> > Best,
> > -Marcel
> >
> > On Mon, Jun 21, 2021 at 6:03 PM Barbara Brown via RT
> > <met_help at ucar.edu>
> > wrote:
> >
> >> On Tue Jun 15 08:03:27 2021, marcel.caron at noaa.gov wrote:
> >> > Hi,
> >> >
> >> > I'm wondering if the RMSE as it is currently calculated in MET
is
> >> centered
> >> > (as, for example, the Anomaly Correlation Coefficient is
centered)
> >> > or if
> >> > the centered RMSE can be derived using other MET statistics? If
> >> > not,
> >> would
> >> > it be possible to include a centered version of the RMSE among
the
> >> > continuous statistics offered?
> >> >
> >> > I would find it useful to create versions of the Taylor
> >> > Diagram–for
> >> > example, one that displays the centered Anomaly Correlation
> >> Coefficient–and
> >> > these versions would require either the centered RMSE or the
> >> > standard
> >> > deviations of both the forecast and observed anomalies.
> >> > Admittedly I
> >> > haven't seen the centered RMSE used in applications other than
> >> > Taylor
> >> > Diagrams.
> >> >
> >> > For context, I'm using MET v9.0.2 and METplus v3.0.2
> >> >
> >> > Best,
> >> > -Marcel
> >> >
> >>
> >> Hi Marcel,
> >>
> >> The standard MSE in MET is not centered. However, MET also
computes
> >> the
> >> "Bias-corrected MSE (BCMSE)", which is equivalent to the centered
> >> MSE
> >> described by Taylor (and used to create a Taylor diagram). This
> >> statistic
> >> is described in the documentation for point-stat and is also
> >> computed by
> >> grid-stat; BCMSE is also defined in Appendix C of the MET User's
> >> Guide.
> >>
> >> I also want to note that METviewer includes an option to create
> >> Taylor
> >> diagrams, which is described in the METviewer documentation.
> >>
> >> Please let us know if you have any additional questions.
> >>
> >> Barbara Brown
> >>
> >
------------------------------------------------
Subject: Question about RMSE in MET
From: Barbara Brown
Time: Tue Jul 06 13:01:34 2021
Hi Marcel,
While MET does not directly compute these standard deviations, it
turns out that - with a bit of algebra - you can show that your
FAvariance is equivalent to
FFABAR - FABAR**2
Note both of these terms are included in the SAL12 output.
Analogously, you could compute OAvariance from OABAR and OOABAR. I can
send the bit of algebra to you if you are interested in seeing it.
Hope this helps! Let us know if you have further questions.
Barbara
On Wed Jun 23 09:43:58 2021, marcel.caron at noaa.gov wrote:
> Hi Barbara:
>
> I have one quick follow-up question about the available centered
> statistics. The scalar L1L2 stats include the mean forecast and
> reference
> anomalies (FABAR and OABAR), but not their standard deviations,
which
> would
> be useful for normalizing the BCMSE. Is there a way to calculate
the
> standard deviation of the anomaly fields using the provided stats?
>
> Best,
> -Marcel
>
> On Tue, Jun 22, 2021 at 8:31 AM Marcel Caron - NOAA Affiliate <
> marcel.caron at noaa.gov> wrote:
>
> > Barbara:
> >
> > Thanks for the clarification. I'll take a look in the PointStat
> > documentation for BCMSE.
> >
> > Best,
> > -Marcel
> >
> > On Mon, Jun 21, 2021 at 6:03 PM Barbara Brown via RT
> > <met_help at ucar.edu>
> > wrote:
> >
> >> On Tue Jun 15 08:03:27 2021, marcel.caron at noaa.gov wrote:
> >> > Hi,
> >> >
> >> > I'm wondering if the RMSE as it is currently calculated in MET
is
> >> centered
> >> > (as, for example, the Anomaly Correlation Coefficient is
centered)
> >> > or if
> >> > the centered RMSE can be derived using other MET statistics? If
> >> > not,
> >> would
> >> > it be possible to include a centered version of the RMSE among
the
> >> > continuous statistics offered?
> >> >
> >> > I would find it useful to create versions of the Taylor
> >> > Diagram–for
> >> > example, one that displays the centered Anomaly Correlation
> >> Coefficient–and
> >> > these versions would require either the centered RMSE or the
> >> > standard
> >> > deviations of both the forecast and observed anomalies.
> >> > Admittedly I
> >> > haven't seen the centered RMSE used in applications other than
> >> > Taylor
> >> > Diagrams.
> >> >
> >> > For context, I'm using MET v9.0.2 and METplus v3.0.2
> >> >
> >> > Best,
> >> > -Marcel
> >> >
> >>
> >> Hi Marcel,
> >>
> >> The standard MSE in MET is not centered. However, MET also
computes
> >> the
> >> "Bias-corrected MSE (BCMSE)", which is equivalent to the centered
> >> MSE
> >> described by Taylor (and used to create a Taylor diagram). This
> >> statistic
> >> is described in the documentation for point-stat and is also
> >> computed by
> >> grid-stat; BCMSE is also defined in Appendix C of the MET User's
> >> Guide.
> >>
> >> I also want to note that METviewer includes an option to create
> >> Taylor
> >> diagrams, which is described in the METviewer documentation.
> >>
> >> Please let us know if you have any additional questions.
> >>
> >> Barbara Brown
> >>
> >
------------------------------------------------
Subject: Question about RMSE in MET
From: Marcel Caron - NOAA Affiliate
Time: Wed Jul 07 13:20:49 2021
Barbara:
This is just what I was looking for. Thank you! I appreciate your
time
with this.
Although I was able to confirm that this works with sample data, if it
isn't too much trouble for you I would definitely be interested to see
the
algebra.
Best,
-Marcel
On Tue, Jul 6, 2021 at 3:01 PM Barbara Brown via RT
<met_help at ucar.edu>
wrote:
> Hi Marcel,
>
> While MET does not directly compute these standard deviations, it
turns
> out that - with a bit of algebra - you can show that your FAvariance
is
> equivalent to
>
> FFABAR - FABAR**2
>
> Note both of these terms are included in the SAL12 output.
Analogously,
> you could compute OAvariance from OABAR and OOABAR. I can send the
bit of
> algebra to you if you are interested in seeing it.
>
> Hope this helps! Let us know if you have further questions.
>
> Barbara
>
> On Wed Jun 23 09:43:58 2021, marcel.caron at noaa.gov wrote:
> > Hi Barbara:
> >
> > I have one quick follow-up question about the available centered
> > statistics. The scalar L1L2 stats include the mean forecast and
> > reference
> > anomalies (FABAR and OABAR), but not their standard deviations,
which
> > would
> > be useful for normalizing the BCMSE. Is there a way to calculate
the
> > standard deviation of the anomaly fields using the provided stats?
> >
> > Best,
> > -Marcel
> >
> > On Tue, Jun 22, 2021 at 8:31 AM Marcel Caron - NOAA Affiliate <
> > marcel.caron at noaa.gov> wrote:
> >
> > > Barbara:
> > >
> > > Thanks for the clarification. I'll take a look in the PointStat
> > > documentation for BCMSE.
> > >
> > > Best,
> > > -Marcel
> > >
> > > On Mon, Jun 21, 2021 at 6:03 PM Barbara Brown via RT
> > > <met_help at ucar.edu>
> > > wrote:
> > >
> > >> On Tue Jun 15 08:03:27 2021, marcel.caron at noaa.gov wrote:
> > >> > Hi,
> > >> >
> > >> > I'm wondering if the RMSE as it is currently calculated in
MET is
> > >> centered
> > >> > (as, for example, the Anomaly Correlation Coefficient is
centered)
> > >> > or if
> > >> > the centered RMSE can be derived using other MET statistics?
If
> > >> > not,
> > >> would
> > >> > it be possible to include a centered version of the RMSE
among the
> > >> > continuous statistics offered?
> > >> >
> > >> > I would find it useful to create versions of the Taylor
> > >> > Diagram–for
> > >> > example, one that displays the centered Anomaly Correlation
> > >> Coefficient–and
> > >> > these versions would require either the centered RMSE or the
> > >> > standard
> > >> > deviations of both the forecast and observed anomalies.
> > >> > Admittedly I
> > >> > haven't seen the centered RMSE used in applications other
than
> > >> > Taylor
> > >> > Diagrams.
> > >> >
> > >> > For context, I'm using MET v9.0.2 and METplus v3.0.2
> > >> >
> > >> > Best,
> > >> > -Marcel
> > >> >
> > >>
> > >> Hi Marcel,
> > >>
> > >> The standard MSE in MET is not centered. However, MET also
computes
> > >> the
> > >> "Bias-corrected MSE (BCMSE)", which is equivalent to the
centered
> > >> MSE
> > >> described by Taylor (and used to create a Taylor diagram). This
> > >> statistic
> > >> is described in the documentation for point-stat and is also
> > >> computed by
> > >> grid-stat; BCMSE is also defined in Appendix C of the MET
User's
> > >> Guide.
> > >>
> > >> I also want to note that METviewer includes an option to create
> > >> Taylor
> > >> diagrams, which is described in the METviewer documentation.
> > >>
> > >> Please let us know if you have any additional questions.
> > >>
> > >> Barbara Brown
> > >>
> > >
>
>
>
>
------------------------------------------------
Subject: Question about RMSE in MET
From: Barbara Brown
Time: Fri Jul 09 11:44:48 2021
Hi Marcel,
Glad that you were able to test that it works! I've attached here a
couple of pages that show the algebra - let me know if you have any
questions (or see any errors in logic!).
Barbara
On Wed Jul 07 13:20:49 2021, marcel.caron at noaa.gov wrote:
> Barbara:
>
> This is just what I was looking for. Thank you! I appreciate your
time
> with this.
>
> Although I was able to confirm that this works with sample data, if
it
> isn't too much trouble for you I would definitely be interested to
see the
> algebra.
>
> Best,
> -Marcel
>
> On Tue, Jul 6, 2021 at 3:01 PM Barbara Brown via RT
<met_help at ucar.edu>
> wrote:
>
> > Hi Marcel,
> >
> > While MET does not directly compute these standard deviations, it
turns
> > out that - with a bit of algebra - you can show that your
FAvariance is
> > equivalent to
> >
> > FFABAR - FABAR**2
> >
> > Note both of these terms are included in the SAL12 output.
Analogously,
> > you could compute OAvariance from OABAR and OOABAR. I can send the
bit of
> > algebra to you if you are interested in seeing it.
> >
> > Hope this helps! Let us know if you have further questions.
> >
> > Barbara
> >
> > On Wed Jun 23 09:43:58 2021, marcel.caron at noaa.gov wrote:
> > > Hi Barbara:
> > >
> > > I have one quick follow-up question about the available centered
> > > statistics. The scalar L1L2 stats include the mean forecast and
> > > reference
> > > anomalies (FABAR and OABAR), but not their standard deviations,
which
> > > would
> > > be useful for normalizing the BCMSE. Is there a way to
calculate the
> > > standard deviation of the anomaly fields using the provided
stats?
> > >
> > > Best,
> > > -Marcel
> > >
> > > On Tue, Jun 22, 2021 at 8:31 AM Marcel Caron - NOAA Affiliate <
> > > marcel.caron at noaa.gov> wrote:
> > >
> > > > Barbara:
> > > >
> > > > Thanks for the clarification. I'll take a look in the
PointStat
> > > > documentation for BCMSE.
> > > >
> > > > Best,
> > > > -Marcel
> > > >
> > > > On Mon, Jun 21, 2021 at 6:03 PM Barbara Brown via RT
> > > > <met_help at ucar.edu>
> > > > wrote:
> > > >
> > > >> On Tue Jun 15 08:03:27 2021, marcel.caron at noaa.gov wrote:
> > > >> > Hi,
> > > >> >
> > > >> > I'm wondering if the RMSE as it is currently calculated in
MET is
> > > >> centered
> > > >> > (as, for example, the Anomaly Correlation Coefficient is
centered)
> > > >> > or if
> > > >> > the centered RMSE can be derived using other MET
statistics? If
> > > >> > not,
> > > >> would
> > > >> > it be possible to include a centered version of the RMSE
among the
> > > >> > continuous statistics offered?
> > > >> >
> > > >> > I would find it useful to create versions of the Taylor
> > > >> > Diagram–for
> > > >> > example, one that displays the centered Anomaly Correlation
> > > >> Coefficient–and
> > > >> > these versions would require either the centered RMSE or
the
> > > >> > standard
> > > >> > deviations of both the forecast and observed anomalies.
> > > >> > Admittedly I
> > > >> > haven't seen the centered RMSE used in applications other
than
> > > >> > Taylor
> > > >> > Diagrams.
> > > >> >
> > > >> > For context, I'm using MET v9.0.2 and METplus v3.0.2
> > > >> >
> > > >> > Best,
> > > >> > -Marcel
> > > >> >
> > > >>
> > > >> Hi Marcel,
> > > >>
> > > >> The standard MSE in MET is not centered. However, MET also
computes
> > > >> the
> > > >> "Bias-corrected MSE (BCMSE)", which is equivalent to the
centered
> > > >> MSE
> > > >> described by Taylor (and used to create a Taylor diagram).
This
> > > >> statistic
> > > >> is described in the documentation for point-stat and is also
> > > >> computed by
> > > >> grid-stat; BCMSE is also defined in Appendix C of the MET
User's
> > > >> Guide.
> > > >>
> > > >> I also want to note that METviewer includes an option to
create
> > > >> Taylor
> > > >> diagrams, which is described in the METviewer documentation.
> > > >>
> > > >> Please let us know if you have any additional questions.
> > > >>
> > > >> Barbara Brown
> > > >>
> > > >
> >
> >
> >
> >
------------------------------------------------
More information about the Met_help
mailing list