[Met_help] [rt.rap.ucar.edu #77170] History for Fwd: ACC [SEC=UNCLASSIFIED]

Fri Sep 2 12:10:19 MDT 2016

----------------------------------------------------------------
  Initial Request
----------------------------------------------------------------

Hi John,
There's a statistic called anomaly correlation coefficient and there are a
couple ways to calculate it. One is "centered" and one is "uncentered" .
I'm intestered in the "centered" one because that is what NCEP uses.  It is
also used in Krishnamurti et al (Eqn. 7).   The Wilks textbook has them
both and so does Jolliffe and Stephenson (2012).

In MET, right now it only has the "uncentered" version.  In other words it
does not remove the average deviation from climatology as the "centered"
version does. Willks explains how they are different and so does Jolliffe
and Stephenson (2012).

Any possibility of adding the "centered" version?
By the way, I think Beth Ebert's web page has it slightly wrong. But I've
already told her. :)

Dave

---------- Forwarded message ----------
From: Beth Ebert <E.Ebert at bom.gov.au>
Date: Wed, Jul 6, 2016 at 4:57 PM
Subject: RE: ACC [SEC=UNCLASSIFIED]
To: David Ahijevych <ahijevyc at ucar.edu>

Hi Dave,

Great to hear from you, and thanks for pointing out an error. I am no
expert on anomaly correlations, and would have simply copied the formula
from somewhere (so long ago that I no longer remember where I would have
gotten it), and it's quite possible I got it wrong. I'll check the
textbooks. Our webpages are in transition and I have temporarily lost
access to them, so it may be a little while till I can fix it.

Regards,

Beth

*From:* David Ahijevych [mailto:ahijevyc at ucar.edu]
*Sent:* Thursday, 7 July 2016 1:21 AM
*To:* Beth Ebert
*Subject:* ACC

Hi Beth!

Long time no see.  I hope you're doing well. Sorry to bother you but I have
a verification question for you and I always refer to your web page. Here
is the excerpt:

*Anomaly correlation* - [image: Anomaly correlation equation]

*Addresses the question: **How well did the forecast anomalies correspond
to the observed anomalies?*

*Range:* -1 to 1.  *Perfect score:* 1.

*Characteristics:* Measures correspondence or phase difference between
forecast and observations, subtracting out the climatological mean at each
point, *C*, rather than the sample mean values. The anomaly correlation is
frequently used to verify output from numerical weather prediction (NWP)
models. *AC* is not sensitive to forecast bias, so a good anomaly
correlation does not guarantee accurate forecasts. Both forms of the
equation are in common use -- see Jolliffe and Stephenson (2012)
<http://www.cawcr.gov.au/projects/verification/#Jolliffe_and_Stephenson_2012>
 or Wilks (2011) <http://www.cawcr.gov.au/projects/verification/#Wilks> for
further discussion.

In the example above, if the climatological temperature is 14 C, then *AC* =
0.904. *AC* is more often used in spatial verification.

I would like to calculate the centered version.   I was wondering if the
centered version should also have the mean difference from climatology
mean(F-C) and mean(O-C) subtracted from the denominator terms.  In other
words,

[image: Inline image 1]

According to Wilks (2006) I think this is what he has in Eqn. 7.59.  Is it
possible there's a typo in the equation?

P.S. MET uses the uncentered version. Before I ask them to add the centered
version, I wanted to see what you think.

Dave

----------------------------------------------------------------
  Complete Ticket History
----------------------------------------------------------------

Subject: Fwd: ACC [SEC=UNCLASSIFIED]
From: John Halley Gotway
Time: Tue Jul 19 10:09:56 2016

Dave,

Great question about AC.  We are close to releasing a small
incremental
version of met, version 5.2.  And once that's out the door, we'll
start
work on version 6.0.  I'll use the info you sent to create a
development
task for version 6.0.

We've refrained from changing output formats in minor releases, but
are
fine doing so in major releases, like 6.0.

But once we start development for 6.0, I think it'd be good to meet
with
you.  We had Binbin Zhou visit us for a week earlier this year to
discuss,
in part, NCEP's computation of anomaly correlation.  The global group
uses
the climo mean and spread to organize matched pairs into "bins".  At
each
grid point, they divide the climo PDF into 11 bins and the bin is
determined by where the observation value falls in that PDF.  Once all
the
pairs are binned, they compute 11 AC values, one for each bin.  The
total
AC is the mean of those 11 AC values.  I'd like to discuss how this
logic
relates to centered vs un-centered AC computations.

Does that sound alright to you?

Thanks,
John

On Mon, Jul 18, 2016 at 11:23 AM, David Ahijevych via RT
<met_help at ucar.edu>
wrote:

>
> Mon Jul 18 11:23:27 2016: Request 77170 was acted upon.
> Transaction: Ticket created by ahijevyc
>        Queue: met_help
>      Subject: Fwd: ACC [SEC=UNCLASSIFIED]
>        Owner: Nobody
>   Requestors: ahijevyc at ucar.edu
>       Status: new
>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=77170 >
>
>
> Hi John,
> There's a statistic called anomaly correlation coefficient and there
are a
> couple ways to calculate it. One is "centered" and one is
"uncentered" .
> I'm intestered in the "centered" one because that is what NCEP uses.
It is
> also used in Krishnamurti et al (Eqn. 7).   The Wilks textbook has
them
> both and so does Jolliffe and Stephenson (2012).
>
> In MET, right now it only has the "uncentered" version.  In other
words it
> does not remove the average deviation from climatology as the
"centered"
> version does. Willks explains how they are different and so does
Jolliffe
> and Stephenson (2012).
>
> Any possibility of adding the "centered" version?
> By the way, I think Beth Ebert's web page has it slightly wrong. But
I've
> already told her. :)
>
> Dave
>
>
>
>
>
> ---------- Forwarded message ----------
> From: Beth Ebert <E.Ebert at bom.gov.au>
> Date: Wed, Jul 6, 2016 at 4:57 PM
> Subject: RE: ACC [SEC=UNCLASSIFIED]
> To: David Ahijevych <ahijevyc at ucar.edu>
>
>
> Hi Dave,
>
>
>
> Great to hear from you, and thanks for pointing out an error. I am
no
> expert on anomaly correlations, and would have simply copied the
formula
> from somewhere (so long ago that I no longer remember where I would
have
> gotten it), and it's quite possible I got it wrong. I'll check the
> textbooks. Our webpages are in transition and I have temporarily
lost
> access to them, so it may be a little while till I can fix it.
>
>
>
> Regards,
>
> Beth
>
>
>
> *From:* David Ahijevych [mailto:ahijevyc at ucar.edu]
> *Sent:* Thursday, 7 July 2016 1:21 AM
> *To:* Beth Ebert
> *Subject:* ACC
>
>
>
> Hi Beth!
>
> Long time no see.  I hope you're doing well. Sorry to bother you but
I have
> a verification question for you and I always refer to your web page.
Here
> is the excerpt:
>
> *Anomaly correlation* - [image: Anomaly correlation equation]
>
> *Addresses the question: **How well did the forecast anomalies
correspond
> to the observed anomalies?*
>
> *Range:* -1 to 1.  *Perfect score:* 1.
>
> *Characteristics:* Measures correspondence or phase difference
between
> forecast and observations, subtracting out the climatological mean
at each
> point, *C*, rather than the sample mean values. The anomaly
correlation is
> frequently used to verify output from numerical weather prediction
(NWP)
> models. *AC* is not sensitive to forecast bias, so a good anomaly
> correlation does not guarantee accurate forecasts. Both forms of the
> equation are in common use -- see Jolliffe and Stephenson (2012)
> <
>
http://www.cawcr.gov.au/projects/verification/#Jolliffe_and_Stephenson_2012
> >
>  or Wilks (2011)
<http://www.cawcr.gov.au/projects/verification/#Wilks>
> for
> further discussion.
>
> In the example above, if the climatological temperature is 14 C,
then *AC*
> =
> 0.904. *AC* is more often used in spatial verification.
>
> I would like to calculate the centered version.   I was wondering if
the
> centered version should also have the mean difference from
climatology
> mean(F-C) and mean(O-C) subtracted from the denominator terms.  In
other
> words,
>
> [image: Inline image 1]
>
> According to Wilks (2006) I think this is what he has in Eqn. 7.59.
Is it
> possible there's a typo in the equation?
>
>
>
> P.S. MET uses the uncentered version. Before I ask them to add the
centered
> version, I wanted to see what you think.
>
>
>
> Dave
>
>

------------------------------------------------
Subject: Fwd: ACC [SEC=UNCLASSIFIED]
From: David Ahijevych
Time: Tue Jul 19 14:00:28 2016

Hi John,
Binbin Zhou's method sounds way more complicated.  We can discuss the
logic
whenever you have the chance.
Thanks,
Dave

On Tue, Jul 19, 2016 at 10:09 AM, John Halley Gotway via RT <
met_help at ucar.edu> wrote:

> Dave,
>
> Great question about AC.  We are close to releasing a small
incremental
> version of met, version 5.2.  And once that's out the door, we'll
start
> work on version 6.0.  I'll use the info you sent to create a
development
> task for version 6.0.
>
> We've refrained from changing output formats in minor releases, but
are
> fine doing so in major releases, like 6.0.
>
> But once we start development for 6.0, I think it'd be good to meet
with
> you.  We had Binbin Zhou visit us for a week earlier this year to
discuss,
> in part, NCEP's computation of anomaly correlation.  The global
group uses
> the climo mean and spread to organize matched pairs into "bins".  At
each
> grid point, they divide the climo PDF into 11 bins and the bin is
> determined by where the observation value falls in that PDF.  Once
all the
> pairs are binned, they compute 11 AC values, one for each bin.  The
total
> AC is the mean of those 11 AC values.  I'd like to discuss how this
logic
> relates to centered vs un-centered AC computations.
>
> Does that sound alright to you?
>
> Thanks,
> John
>
>
>
> On Mon, Jul 18, 2016 at 11:23 AM, David Ahijevych via RT <
> met_help at ucar.edu>
> wrote:
>
> >
> > Mon Jul 18 11:23:27 2016: Request 77170 was acted upon.
> > Transaction: Ticket created by ahijevyc
> >        Queue: met_help
> >      Subject: Fwd: ACC [SEC=UNCLASSIFIED]
> >        Owner: Nobody
> >   Requestors: ahijevyc at ucar.edu
> >       Status: new
> >  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=77170 >
> >
> >
> > Hi John,
> > There's a statistic called anomaly correlation coefficient and
there are
> a
> > couple ways to calculate it. One is "centered" and one is
"uncentered" .
> > I'm intestered in the "centered" one because that is what NCEP
uses.  It
> is
> > also used in Krishnamurti et al (Eqn. 7).   The Wilks textbook has
them
> > both and so does Jolliffe and Stephenson (2012).
> >
> > In MET, right now it only has the "uncentered" version.  In other
words
> it
> > does not remove the average deviation from climatology as the
"centered"
> > version does. Willks explains how they are different and so does
Jolliffe
> > and Stephenson (2012).
> >
> > Any possibility of adding the "centered" version?
> > By the way, I think Beth Ebert's web page has it slightly wrong.
But I've
> > already told her. :)
> >
> > Dave
> >
> >
> >
> >
> >
> > ---------- Forwarded message ----------
> > From: Beth Ebert <E.Ebert at bom.gov.au>
> > Date: Wed, Jul 6, 2016 at 4:57 PM
> > Subject: RE: ACC [SEC=UNCLASSIFIED]
> > To: David Ahijevych <ahijevyc at ucar.edu>
> >
> >
> > Hi Dave,
> >
> >
> >
> > Great to hear from you, and thanks for pointing out an error. I am
no
> > expert on anomaly correlations, and would have simply copied the
formula
> > from somewhere (so long ago that I no longer remember where I
would have
> > gotten it), and it's quite possible I got it wrong. I'll check the
> > textbooks. Our webpages are in transition and I have temporarily
lost
> > access to them, so it may be a little while till I can fix it.
> >
> >
> >
> > Regards,
> >
> > Beth
> >
> >
> >
> > *From:* David Ahijevych [mailto:ahijevyc at ucar.edu]
> > *Sent:* Thursday, 7 July 2016 1:21 AM
> > *To:* Beth Ebert
> > *Subject:* ACC
> >
> >
> >
> > Hi Beth!
> >
> > Long time no see.  I hope you're doing well. Sorry to bother you
but I
> have
> > a verification question for you and I always refer to your web
page. Here
> > is the excerpt:
> >
> > *Anomaly correlation* - [image: Anomaly correlation equation]
> >
> > *Addresses the question: **How well did the forecast anomalies
correspond
> > to the observed anomalies?*
> >
> > *Range:* -1 to 1.  *Perfect score:* 1.
> >
> > *Characteristics:* Measures correspondence or phase difference
between
> > forecast and observations, subtracting out the climatological mean
at
> each
> > point, *C*, rather than the sample mean values. The anomaly
correlation
> is
> > frequently used to verify output from numerical weather prediction
(NWP)
> > models. *AC* is not sensitive to forecast bias, so a good anomaly
> > correlation does not guarantee accurate forecasts. Both forms of
the
> > equation are in common use -- see Jolliffe and Stephenson (2012)
> > <
> >
>
http://www.cawcr.gov.au/projects/verification/#Jolliffe_and_Stephenson_2012
> > >
> >  or Wilks (2011)
<http://www.cawcr.gov.au/projects/verification/#Wilks>
> > for
> > further discussion.
> >
> > In the example above, if the climatological temperature is 14 C,
then
> *AC*
> > =
> > 0.904. *AC* is more often used in spatial verification.
> >
> > I would like to calculate the centered version.   I was wondering
if the
> > centered version should also have the mean difference from
climatology
> > mean(F-C) and mean(O-C) subtracted from the denominator terms.  In
other
> > words,
> >
> > [image: Inline image 1]
> >
> > According to Wilks (2006) I think this is what he has in Eqn.
7.59.  Is
> it
> > possible there's a typo in the equation?
> >
> >
> >
> > P.S. MET uses the uncentered version. Before I ask them to add the
> centered
> > version, I wanted to see what you think.
> >
> >
> >
> > Dave
> >
> >
>
>

------------------------------------------------