[Met_help] [rt.rap.ucar.edu #83891] History for Scorecard - VL1L2_RMSE doesn't work with agg_stat

[Tatiana Burek via RT]

----------------------------------------------------------------
  Initial Request
----------------------------------------------------------------

Hi,

I am in the process of putting together scorecards for the RAP and HRRR
models for the NCEP director briefing on Friday afternoon.  When I tried
creating the RAP scorecard with mv_scorecard.sh, I noticed that the
VL1L2_RMSE stats for vector wind do not work when I have agg_stat set to
true.  VL1L2_BIAS does work so I'm not sure why RMSE doesn't.  However,
when agg_stat is set to false, the VL1L2_RMSE stats for vector winds work
with no issues.

I've attached my scorecard XML and the resulting image.  I think vector
wind RMSE should have values at f00/f06/f12/f18. Note that for testing
purposes I changed boot_repl to 10 but normally I have it at 1000 (which
takes longer to run).

Thanks!
Ben


----------------------------------------------------------------
  Complete Ticket History
----------------------------------------------------------------

Subject: Scorecard - VL1L2_RMSE doesn't work with agg_stat
From: Tatiana Burek
Time: Thu Feb 01 08:54:36 2018

Hi Ben,

I edited your XML and set all <view_..> tags to true and increased
boot_repl to 500.
<home>/scorecard_mv_met_bblake_rap_wind.xml

The missing values for f03, f09,f15.f21 were braking rmse
calculations. It should be fixed moe.
Please take a look at
/metviewer/staging/bblake/scorecard_mv_met_bblake_rap_wind.png:
p-values in columns with the existing values are preset.

Tatiana
On Wed Jan 31 13:58:14 2018, benjamin.blake at noaa.gov wrote:
> Hi,
>
> I am in the process of putting together scorecards for the RAP and
HRRR
> models for the NCEP director briefing on Friday afternoon.  When I
tried
> creating the RAP scorecard with mv_scorecard.sh, I noticed that the
> VL1L2_RMSE stats for vector wind do not work when I have agg_stat
set to
> true.  VL1L2_BIAS does work so I'm not sure why RMSE doesn't.
However,
> when agg_stat is set to false, the VL1L2_RMSE stats for vector winds
work
> with no issues.
>
> I've attached my scorecard XML and the resulting image.  I think
vector
> wind RMSE should have values at f00/f06/f12/f18. Note that for
testing
> purposes I changed boot_repl to 10 but normally I have it at 1000
(which
> takes longer to run).
>
> Thanks!
> Ben



------------------------------------------------
Subject: Scorecard - VL1L2_RMSE doesn't work with agg_stat
From: Benjamin Blake - NOAA Affiliate
Time: Thu Feb 01 09:31:40 2018

Hi Tatiana,

I re-ran your script and was able to re-create the
scorecard_mv_met_bblake_rap_wind.png with values for vector wind rmse.

I also re-ran my scorecard_bblake_rap.xml script and was able to
generate
values for vector wind rmse.

Thanks very much for helping to fix this!

Ben

On Thu, Feb 1, 2018 at 10:54 AM, Tatiana Burek via RT
<met_help at ucar.edu>
wrote:

> Hi Ben,
>
> I edited your XML and set all <view_..> tags to true and increased
> boot_repl to 500.
> <home>/scorecard_mv_met_bblake_rap_wind.xml
>
> The missing values for f03, f09,f15.f21 were braking rmse
calculations. It
> should be fixed moe.
> Please take a look at /metviewer/staging/bblake/
> scorecard_mv_met_bblake_rap_wind.png:
> p-values in columns with the existing values are preset.
>
> Tatiana
> On Wed Jan 31 13:58:14 2018, benjamin.blake at noaa.gov wrote:
> > Hi,
> >
> > I am in the process of putting together scorecards for the RAP and
HRRR
> > models for the NCEP director briefing on Friday afternoon.  When I
tried
> > creating the RAP scorecard with mv_scorecard.sh, I noticed that
the
> > VL1L2_RMSE stats for vector wind do not work when I have agg_stat
set to
> > true.  VL1L2_BIAS does work so I'm not sure why RMSE doesn't.
However,
> > when agg_stat is set to false, the VL1L2_RMSE stats for vector
winds work
> > with no issues.
> >
> > I've attached my scorecard XML and the resulting image.  I think
vector
> > wind RMSE should have values at f00/f06/f12/f18. Note that for
testing
> > purposes I changed boot_repl to 10 but normally I have it at 1000
(which
> > takes longer to run).
> >
> > Thanks!
> > Ben
>
>
>
>

------------------------------------------------
Subject: Scorecard - VL1L2_RMSE doesn't work with agg_stat
From: Tatiana Burek
Time: Thu Feb 01 09:48:20 2018

Glad to hear that it worked for you.
I will close this ticket.

Tatiana
On Thu Feb 01 09:31:40 2018, benjamin.blake at noaa.gov wrote:
> Hi Tatiana,
>
> I re-ran your script and was able to re-create the
> scorecard_mv_met_bblake_rap_wind.png with values for vector wind
rmse.
>
> I also re-ran my scorecard_bblake_rap.xml script and was able to
> generate
> values for vector wind rmse.
>
> Thanks very much for helping to fix this!
>
> Ben
>
> On Thu, Feb 1, 2018 at 10:54 AM, Tatiana Burek via RT
> <met_help at ucar.edu>
> wrote:
>
> > Hi Ben,
> >
> > I edited your XML and set all <view_..> tags to true and increased
> > boot_repl to 500.
> > <home>/scorecard_mv_met_bblake_rap_wind.xml
> >
> > The missing values for f03, f09,f15.f21 were braking rmse
> > calculations. It
> > should be fixed moe.
> > Please take a look at /metviewer/staging/bblake/
> > scorecard_mv_met_bblake_rap_wind.png:
> > p-values in columns with the existing values are preset.
> >
> > Tatiana
> > On Wed Jan 31 13:58:14 2018, benjamin.blake at noaa.gov wrote:
> > > Hi,
> > >
> > > I am in the process of putting together scorecards for the RAP
and
> > > HRRR
> > > models for the NCEP director briefing on Friday afternoon.  When
I
> > > tried
> > > creating the RAP scorecard with mv_scorecard.sh, I noticed that
the
> > > VL1L2_RMSE stats for vector wind do not work when I have
agg_stat
> > > set to
> > > true.  VL1L2_BIAS does work so I'm not sure why RMSE doesn't.
> > > However,
> > > when agg_stat is set to false, the VL1L2_RMSE stats for vector
> > > winds work
> > > with no issues.
> > >
> > > I've attached my scorecard XML and the resulting image.  I think
> > > vector
> > > wind RMSE should have values at f00/f06/f12/f18. Note that for
> > > testing
> > > purposes I changed boot_repl to 10 but normally I have it at
1000
> > > (which
> > > takes longer to run).
> > >
> > > Thanks!
> > > Ben
> >
> >
> >
> >



------------------------------------------------