[Met_help] [rt.rap.ucar.edu #78821] History for Domain average statistics

[John Halley Gotway via RT]

----------------------------------------------------------------
  Initial Request
----------------------------------------------------------------

Dear MET staff,

I hope you are doing well at your end.

I am plotting the results of the evaluation. I would like to ask for your
help on the following.

I am comparing the performance of WRF outputs for several configurations. I
used point observations and NARR reanalysis and am making taylor plots and
spatial plots.

The taylor plots are based on the results from an "aggregate_stat" job from
StatAnalysis of both pointStat and gridStat previous runs. Am using the
FULL domain and an additional mask.

The spatial plots are based on the results from SeriesAnalysis

However after comparing these plots, results are different for some
statistics, for both the FULL domain and the MASK region. For instance,
surface temperature (Temp Z2):

- statAnalysis (from gridStat) suggests the Pearson Correlation is *0.862*
(FULL)

- a domain average with NCO for the seriesAnalysis output suggests the
Pearson correlation is *0.495* (FULL)

I ran gridStat with an interpolation width of 1. (UW_MEAN, width = 1)

In general the results from statAnalysis (gridStat) are higher than those
for the domain average for the series analysis outputs. However, for Mean
Error and MBIAS they are very similar.

So, my question is if the aggregated statistic of gridStat using
StatAnalysis is/should be comparable to the domain average of
seriesAnalysis ?
Could you please comment on this ?

For many reasons, I am still using MET v4.1.

Thanks in advance for your time,

Best regards

Victor


----------------------------------------------------------------
  Complete Ticket History
----------------------------------------------------------------

Subject: Domain average statistics
From: Tressa Fowler
Time: Mon Dec 05 12:33:29 2016

Dear Victor,

I see that you have questions about summarizing results from MET.

Averages, as a means as summarizing or combining information, work
quite well in some cases and very poorly in other cases. In the case
of bias (mean error) and MAE, you should expect similar results from
both methods because these are themselves averages.

However, Pearson correlation is not an average, it is a ratio of
variances. If you wished to average these, the Fisher Z transformation
is recommended (see Wikipedia). Then you average the transformed
values and transform back. There is no reason to expect the
correlation of a set of values to be similar to the average of the
correlations of subsets of those values.

I hope this information is useful to you. If you have further
questions, please let me know.

Regards,

Tressa

On Mon Dec 05 11:34:31 2016, victor.al.vlz at gmail.com wrote:
> Dear MET staff,
>
> I hope you are doing well at your end.
>
> I am plotting the results of the evaluation. I would like to ask for
your
> help on the following.
>
> I am comparing the performance of WRF outputs for several
configurations. I
> used point observations and NARR reanalysis and am making taylor
plots and
> spatial plots.
>
> The taylor plots are based on the results from an "aggregate_stat"
job from
> StatAnalysis of both pointStat and gridStat previous runs. Am using
the
> FULL domain and an additional mask.
>
> The spatial plots are based on the results from SeriesAnalysis
>
> However after comparing these plots, results are different for some
> statistics, for both the FULL domain and the MASK region. For
instance,
> surface temperature (Temp Z2):
>
> - statAnalysis (from gridStat) suggests the Pearson Correlation is
*0.862*
> (FULL)
>
> - a domain average with NCO for the seriesAnalysis output suggests
the
> Pearson correlation is *0.495* (FULL)
>
> I ran gridStat with an interpolation width of 1. (UW_MEAN, width =
1)
>
> In general the results from statAnalysis (gridStat) are higher than
those
> for the domain average for the series analysis outputs. However, for
Mean
> Error and MBIAS they are very similar.
>
> So, my question is if the aggregated statistic of gridStat using
> StatAnalysis is/should be comparable to the domain average of
> seriesAnalysis ?
> Could you please comment on this ?
>
> For many reasons, I am still using MET v4.1.
>
> Thanks in advance for your time,
>
> Best regards
>
> Victor


------------------------------------------------
Subject: Domain average statistics
From: Victor Almanza
Time: Mon Dec 05 12:56:44 2016

Dear Tressa,

Thank you very much for your time and the explanation. It clarifies
and
helps me understand now.

Best wishes !!!

Victor

On Mon, Dec 5, 2016 at 11:33 AM, Tressa Fowler via RT
<met_help at ucar.edu>
wrote:

> Dear Victor,
>
> I see that you have questions about summarizing results from MET.
>
> Averages, as a means as summarizing or combining information, work
quite
> well in some cases and very poorly in other cases. In the case of
bias
> (mean error) and MAE, you should expect similar results from both
methods
> because these are themselves averages.
>
> However, Pearson correlation is not an average, it is a ratio of
> variances. If you wished to average these, the Fisher Z
transformation is
> recommended (see Wikipedia). Then you average the transformed values
and
> transform back. There is no reason to expect the correlation of a
set of
> values to be similar to the average of the correlations of subsets
of those
> values.
>
> I hope this information is useful to you. If you have further
questions,
> please let me know.
>
> Regards,
>
> Tressa
>
> On Mon Dec 05 11:34:31 2016, victor.al.vlz at gmail.com wrote:
> > Dear MET staff,
> >
> > I hope you are doing well at your end.
> >
> > I am plotting the results of the evaluation. I would like to ask
for your
> > help on the following.
> >
> > I am comparing the performance of WRF outputs for several
> configurations. I
> > used point observations and NARR reanalysis and am making taylor
plots
> and
> > spatial plots.
> >
> > The taylor plots are based on the results from an "aggregate_stat"
job
> from
> > StatAnalysis of both pointStat and gridStat previous runs. Am
using the
> > FULL domain and an additional mask.
> >
> > The spatial plots are based on the results from SeriesAnalysis
> >
> > However after comparing these plots, results are different for
some
> > statistics, for both the FULL domain and the MASK region. For
instance,
> > surface temperature (Temp Z2):
> >
> > - statAnalysis (from gridStat) suggests the Pearson Correlation is
> *0.862*
> > (FULL)
> >
> > - a domain average with NCO for the seriesAnalysis output suggests
the
> > Pearson correlation is *0.495* (FULL)
> >
> > I ran gridStat with an interpolation width of 1. (UW_MEAN, width =
1)
> >
> > In general the results from statAnalysis (gridStat) are higher
than those
> > for the domain average for the series analysis outputs. However,
for Mean
> > Error and MBIAS they are very similar.
> >
> > So, my question is if the aggregated statistic of gridStat using
> > StatAnalysis is/should be comparable to the domain average of
> > seriesAnalysis ?
> > Could you please comment on this ?
> >
> > For many reasons, I am still using MET v4.1.
> >
> > Thanks in advance for your time,
> >
> > Best regards
> >
> > Victor
>
>
>

------------------------------------------------