[Met_help] [rt.rap.ucar.edu #99965] History for stats in the stat files and how they are used in the statistics calculations
John Halley Gotway via RT
met_help at ucar.edu
Mon May 24 10:16:37 MDT 2021
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hello,
I searched around in the met user's guide to determine what values are
actually being reported in the stat files for categories SL1L2, FHO, and
CTC since I use them for verification. I have primarily relied on the
SL1L2 to calculate averages, RMSE, and PR. The output in the stats from
what I can gather include OBAR, FBAR, FOBAR, FFBAR, OOBAR, and MAE. These
values are used to calculate the stats when I process the results with
metviewer after setting up my plotting xml files. All of the quantities
are already averaged and yet are somehow used to calculate other stats such
as RMSE and PR. The averages are straightforward since they are already
tabulated. However, stats such as RMSE cannot use the above information
alone since all the above quantities have been processed for all sites
already. Note that, since RMSE = sqrt(1/n*sum((fi - oi)**2)), and that "i"
would be for each site which is then summed up over all sites for a region
while "n" represents the site number. Somehow I can still produce this
quantity on a plot using Metviewer even though RMSE is not being calculated
and stored in the stat file. I'm wondering how this is possible? I have
similar concerns about PR as well.
As an example a line from the stat file for SL1L2 we have:
SL1L2 1132 47.35316 45.28799 2259.42668 2367.10324 2311.21027
8.68585
which, for stat type SL1L2 corresponds to OBAR, FBAR, FOBAR, FFBAR, OOBAR,
and MAE in that order.
Thanks in advance
Edward Strobach
EMC/NCEP/NWS/ IMSG
Coding Logic (if/then/else)
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: stats in the stat files and how they are used in the statistics calculations
From: John Halley Gotway
Time: Mon May 24 09:54:23 2021
Ed,
I see that you have a question about the computation of RMSE from the
SL1L2
partial sums. This derivation dates back to the original VSDB code on
which
the MET software is based. The columns included in the SL1L2 line type
were
chosen explicitly because those are what is needed to derive
continuous
statistics like ME, MSE, and RMSE.
Here's the first line of the function that reads SL1L2 partial sums
and
derives the corresponding CNT statistics:
https://github.com/dtcenter/MET/blob/main_v10.0/met/src/libcode/vx_statistics/compute_stats.cc#L28
Note that there are many continuous stats that are NOT derivable, and
those
are just set to bad data at the top of that function.
On line 102, MSE is computed as:
cnt_info.mse.v = ffbar + oobar - 2.0*fobar;
And then RMSE is simply the square root of that.
Hope that helps clarify.
John
On Mon, May 24, 2021 at 9:16 AM Edward Strobach - NOAA Affiliate via
RT <
met_help at ucar.edu> wrote:
>
> Mon May 24 09:15:34 2021: Request 99965 was acted upon.
> Transaction: Ticket created by edward.strobach at noaa.gov
> Queue: met_help
> Subject: stats in the stat files and how they are used in the
> statistics calculations
> Owner: Nobody
> Requestors: edward.strobach at noaa.gov
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99965 >
>
>
> Hello,
>
> I searched around in the met user's guide to determine what values
are
> actually being reported in the stat files for categories SL1L2, FHO,
and
> CTC since I use them for verification. I have primarily relied on
the
> SL1L2 to calculate averages, RMSE, and PR. The output in the stats
from
> what I can gather include OBAR, FBAR, FOBAR, FFBAR, OOBAR, and MAE.
These
> values are used to calculate the stats when I process the results
with
> metviewer after setting up my plotting xml files. All of the
quantities
> are already averaged and yet are somehow used to calculate other
stats such
> as RMSE and PR. The averages are straightforward since they are
already
> tabulated. However, stats such as RMSE cannot use the above
information
> alone since all the above quantities have been processed for all
sites
> already. Note that, since RMSE = sqrt(1/n*sum((fi - oi)**2)), and
that "i"
> would be for each site which is then summed up over all sites for a
region
> while "n" represents the site number. Somehow I can still produce
this
> quantity on a plot using Metviewer even though RMSE is not being
calculated
> and stored in the stat file. I'm wondering how this is possible? I
have
> similar concerns about PR as well.
>
> As an example a line from the stat file for SL1L2 we have:
> SL1L2 1132 47.35316 45.28799 2259.42668 2367.10324
2311.21027
> 8.68585
>
> which, for stat type SL1L2 corresponds to OBAR, FBAR, FOBAR, FFBAR,
OOBAR,
> and MAE in that order.
>
>
> Thanks in advance
>
> Edward Strobach
> EMC/NCEP/NWS/ IMSG
> Coding Logic (if/then/else)
>
>
------------------------------------------------
Subject: stats in the stat files and how they are used in the statistics calculations
From: Edward Strobach - NOAA Affiliate
Time: Mon May 24 10:04:36 2021
yes it does. Thanks
Edward Strobach
EMC/NCEP/NWS/ IMSG
Coding Logic (if/then/else)
On Mon, May 24, 2021 at 11:54 AM John Halley Gotway via RT <
met_help at ucar.edu> wrote:
> Ed,
>
> I see that you have a question about the computation of RMSE from
the SL1L2
> partial sums. This derivation dates back to the original VSDB code
on which
> the MET software is based. The columns included in the SL1L2 line
type were
> chosen explicitly because those are what is needed to derive
continuous
> statistics like ME, MSE, and RMSE.
>
> Here's the first line of the function that reads SL1L2 partial sums
and
> derives the corresponding CNT statistics:
>
>
https://github.com/dtcenter/MET/blob/main_v10.0/met/src/libcode/vx_statistics/compute_stats.cc#L28
>
> Note that there are many continuous stats that are NOT derivable,
and those
> are just set to bad data at the top of that function.
>
> On line 102, MSE is computed as:
> cnt_info.mse.v = ffbar + oobar - 2.0*fobar;
>
> And then RMSE is simply the square root of that.
>
> Hope that helps clarify.
>
> John
>
> On Mon, May 24, 2021 at 9:16 AM Edward Strobach - NOAA Affiliate via
RT <
> met_help at ucar.edu> wrote:
>
> >
> > Mon May 24 09:15:34 2021: Request 99965 was acted upon.
> > Transaction: Ticket created by edward.strobach at noaa.gov
> > Queue: met_help
> > Subject: stats in the stat files and how they are used in the
> > statistics calculations
> > Owner: Nobody
> > Requestors: edward.strobach at noaa.gov
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99965 >
> >
> >
> > Hello,
> >
> > I searched around in the met user's guide to determine what values
are
> > actually being reported in the stat files for categories SL1L2,
FHO, and
> > CTC since I use them for verification. I have primarily relied on
the
> > SL1L2 to calculate averages, RMSE, and PR. The output in the
stats from
> > what I can gather include OBAR, FBAR, FOBAR, FFBAR, OOBAR, and
MAE.
> These
> > values are used to calculate the stats when I process the results
with
> > metviewer after setting up my plotting xml files. All of the
quantities
> > are already averaged and yet are somehow used to calculate other
stats
> such
> > as RMSE and PR. The averages are straightforward since they are
already
> > tabulated. However, stats such as RMSE cannot use the above
information
> > alone since all the above quantities have been processed for all
sites
> > already. Note that, since RMSE = sqrt(1/n*sum((fi - oi)**2)), and
that
> "i"
> > would be for each site which is then summed up over all sites for
a
> region
> > while "n" represents the site number. Somehow I can still produce
this
> > quantity on a plot using Metviewer even though RMSE is not being
> calculated
> > and stored in the stat file. I'm wondering how this is possible?
I have
> > similar concerns about PR as well.
> >
> > As an example a line from the stat file for SL1L2 we have:
> > SL1L2 1132 47.35316 45.28799 2259.42668 2367.10324
> 2311.21027
> > 8.68585
> >
> > which, for stat type SL1L2 corresponds to OBAR, FBAR, FOBAR,
FFBAR,
> OOBAR,
> > and MAE in that order.
> >
> >
> > Thanks in advance
> >
> > Edward Strobach
> > EMC/NCEP/NWS/ IMSG
> > Coding Logic (if/then/else)
> >
> >
>
>
------------------------------------------------
More information about the Met_help
mailing list