[Met_help] [rt.rap.ucar.edu #86521] History for MetViewer: plot scores of the same model at different forecast lengths
John Halley Gotway via RT
met_help at ucar.edu
Wed Aug 8 16:02:56 MDT 2018
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hello. I'd like to be plot scores of the same model at different
forecast lengths (say, two separate lines for GFS GSS, the first line
for the forecast lead time of 24h, the 2nd for the forecast lead time of
48h.
If I select one model and choose two different forecast lead times as
two separate entries (rather than selecting '24' and '48' on the same
line for the forecast lead times), the plot would just be a single line
of scores for aggregated stats from 24h+48h lead times.
Next I tried to select the model 'GFS' twice, on two separate lines (as
Group-y1_1 and Group-y1_2), then for fixed values choose two separate
forecast leads times, 24h and 48h, hoping that MetViewer would apply the
24h to Group_y1_1 and the 48h Group-y1_2. It didn't seem likely that
MetViewer would interpret this the way I intended, and indeed when I
tried to plot I got an "R error" (see attached Screenshot).
Then I tried to have the "GFS at 24h lead time" as Y1 axis variable and
"GFS at 48h lead time" as Y2 axis variable (that seemed unlikely to be
what the Y1/Y2 option was meant for, but I was out of ideas), see the
attached plot_*.xml and plot_*.png - it seems impossible to have
different lead times for Y1/Y2. In the resulting plot, while the red
line is indeed the daily time series of GFS' GSS score at 24h lead time,
I've no idea what the blue line is (it's not the scores at 48h), and the
R-data offered no clue.
Is there a way to do this? Thanks!
Ying
--
Ying Lin
NCEP/EMC/Verification, Post-processing and Product Generation Branch
NCWCP Cubicle No. 2015
Ying.Lin at noaa.gov
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: MetViewer: plot scores of the same model at different forecast lengths
From: John Halley Gotway
Time: Wed Aug 08 13:17:18 2018
Hi Ying,
I see that you're having trouble using METviewer to create a series
plot
where the lines on the plot represent the performance for different
forecast lead times.
Tatiana and I took a look at this together and figured it out.
First let me say, that yes, of course, METviewer should be able to
make
this type of plot.
Next, for the "Series" plot template, you define the series variables
(i.e.
lines on the plot) in the "Y1 Series Variables" section. In your
case, you
want 2 entries: MODEL = GFS and FCST_LEAD = 24,48. Actually, you
don't
technically *need* MODEL = GFS since that's all that's present in this
database. But it doesn't hurt to include it. You don't need to use
the Y2
axis for something this simple.
When I ran with your settings, I got a lengthy error like this:
failed to run plot plot_20180808_190845 - reason: R error: ==== Start
Rscript error ==== Error in cat(" boot time: ",
formatTimeSpan(dblBootTime), "\n", " boot.ci time: ", : object
'listStat'
not found Execution halted ==== End Rscript error ====
The real problem is that your query returned no data... but tracing
through
the flow of the logic, this is the first error that gets thrown.
Tatiana
is creating a development ticket to write a more useful error in the
future. Ideally, it'd say something like:
failed to run plot plot_20180808_190845 - reason: R error: ==== Start
Rscript error ==== The query returned no data. Execution halted ====
End
Rscript error ====
But **why** does the query return no data. This database only
contains
forecasts initialized at 12Z. If you're evaluating 24 and 48 hour
lead
times, they'd be **valid** at 12Z. But you selected 00Z valid times
for
the x-axis. There are no 24 or 48 hour lead times valid at 00Z. Just
switch the valid times from 00Z to 12Z and you'll see data.
See attached XML and PNG with separate lines for 24, 48, 72, and 96
hour
lead times.
Thanks,
John
On Wed, Aug 8, 2018 at 10:06 AM Ying Lin via RT <met_help at ucar.edu>
wrote:
>
> Wed Aug 08 10:06:02 2018: Request 86521 was acted upon.
> Transaction: Ticket created by ying.lin at noaa.gov
> Queue: met_help
> Subject: MetViewer: plot scores of the same model at different
> forecast lengths
> Owner: Nobody
> Requestors: ying.lin at noaa.gov
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86521 >
>
>
> Hello. I'd like to be plot scores of the same model at different
> forecast lengths (say, two separate lines for GFS GSS, the first
line
> for the forecast lead time of 24h, the 2nd for the forecast lead
time of
> 48h.
>
> If I select one model and choose two different forecast lead times
as
> two separate entries (rather than selecting '24' and '48' on the
same
> line for the forecast lead times), the plot would just be a single
line
> of scores for aggregated stats from 24h+48h lead times.
>
> Next I tried to select the model 'GFS' twice, on two separate lines
(as
> Group-y1_1 and Group-y1_2), then for fixed values choose two
separate
> forecast leads times, 24h and 48h, hoping that MetViewer would apply
the
> 24h to Group_y1_1 and the 48h Group-y1_2. It didn't seem likely
that
> MetViewer would interpret this the way I intended, and indeed when I
> tried to plot I got an "R error" (see attached Screenshot).
>
> Then I tried to have the "GFS at 24h lead time" as Y1 axis variable
and
> "GFS at 48h lead time" as Y2 axis variable (that seemed unlikely to
be
> what the Y1/Y2 option was meant for, but I was out of ideas), see
the
> attached plot_*.xml and plot_*.png - it seems impossible to have
> different lead times for Y1/Y2. In the resulting plot, while the
red
> line is indeed the daily time series of GFS' GSS score at 24h lead
time,
> I've no idea what the blue line is (it's not the scores at 48h), and
the
> R-data offered no clue.
>
> Is there a way to do this? Thanks!
>
> Ying
>
> --
> Ying Lin
> NCEP/EMC/Verification, Post-processing and Product Generation Branch
> NCWCP Cubicle No. 2015
> Ying.Lin at noaa.gov
>
>
>
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #86521] MetViewer: plot scores of the same model at different forecast lengths
From: Ying Lin
Time: Wed Aug 08 14:47:51 2018
Hi John (and Tatiana),
Thank you! It works and matches what FVS produces. I see what
the problem was now - I didn't realize that the lead times should have
been defined in the Series Variables section.
Also thanks much for your help yesterday on how to load *.stats
for
multiple models w/o putting them in separate subdirs. Since that
ticket was closed yesterday I hesitated about replying with a thank
you
in case it would inadvertently re-open the ticket.
Ying
On 08/08/2018 03:17 PM, John Halley Gotway via RT wrote:
> Hi Ying,
>
> I see that you're having trouble using METviewer to create a series
plot
> where the lines on the plot represent the performance for different
> forecast lead times.
>
> Tatiana and I took a look at this together and figured it out.
>
> First let me say, that yes, of course, METviewer should be able to
make
> this type of plot.
>
> Next, for the "Series" plot template, you define the series
variables (i.e.
> lines on the plot) in the "Y1 Series Variables" section. In your
case, you
> want 2 entries: MODEL = GFS and FCST_LEAD = 24,48. Actually, you
don't
> technically *need* MODEL = GFS since that's all that's present in
this
> database. But it doesn't hurt to include it. You don't need to use
the Y2
> axis for something this simple.
>
> When I ran with your settings, I got a lengthy error like this:
> failed to run plot plot_20180808_190845 - reason: R error: ====
Start
> Rscript error ==== Error in cat(" boot time: ",
> formatTimeSpan(dblBootTime), "\n", " boot.ci time: ", : object
'listStat'
> not found Execution halted ==== End Rscript error ====
>
> The real problem is that your query returned no data... but tracing
through
> the flow of the logic, this is the first error that gets thrown.
Tatiana
> is creating a development ticket to write a more useful error in the
> future. Ideally, it'd say something like:
> failed to run plot plot_20180808_190845 - reason: R error: ====
Start
> Rscript error ==== The query returned no data. Execution halted ====
End
> Rscript error ====
>
> But **why** does the query return no data. This database only
contains
> forecasts initialized at 12Z. If you're evaluating 24 and 48 hour
lead
> times, they'd be **valid** at 12Z. But you selected 00Z valid times
for
> the x-axis. There are no 24 or 48 hour lead times valid at 00Z.
Just
> switch the valid times from 00Z to 12Z and you'll see data.
>
> See attached XML and PNG with separate lines for 24, 48, 72, and 96
hour
> lead times.
>
> Thanks,
> John
>
> On Wed, Aug 8, 2018 at 10:06 AM Ying Lin via RT <met_help at ucar.edu>
wrote:
>
>> Wed Aug 08 10:06:02 2018: Request 86521 was acted upon.
>> Transaction: Ticket created by ying.lin at noaa.gov
>> Queue: met_help
>> Subject: MetViewer: plot scores of the same model at
different
>> forecast lengths
>> Owner: Nobody
>> Requestors: ying.lin at noaa.gov
>> Status: new
>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86521 >
>>
>>
>> Hello. I'd like to be plot scores of the same model at different
>> forecast lengths (say, two separate lines for GFS GSS, the first
line
>> for the forecast lead time of 24h, the 2nd for the forecast lead
time of
>> 48h.
>>
>> If I select one model and choose two different forecast lead times
as
>> two separate entries (rather than selecting '24' and '48' on the
same
>> line for the forecast lead times), the plot would just be a single
line
>> of scores for aggregated stats from 24h+48h lead times.
>>
>> Next I tried to select the model 'GFS' twice, on two separate lines
(as
>> Group-y1_1 and Group-y1_2), then for fixed values choose two
separate
>> forecast leads times, 24h and 48h, hoping that MetViewer would
apply the
>> 24h to Group_y1_1 and the 48h Group-y1_2. It didn't seem likely
that
>> MetViewer would interpret this the way I intended, and indeed when
I
>> tried to plot I got an "R error" (see attached Screenshot).
>>
>> Then I tried to have the "GFS at 24h lead time" as Y1 axis variable
and
>> "GFS at 48h lead time" as Y2 axis variable (that seemed unlikely to
be
>> what the Y1/Y2 option was meant for, but I was out of ideas), see
the
>> attached plot_*.xml and plot_*.png - it seems impossible to have
>> different lead times for Y1/Y2. In the resulting plot, while the
red
>> line is indeed the daily time series of GFS' GSS score at 24h lead
time,
>> I've no idea what the blue line is (it's not the scores at 48h),
and the
>> R-data offered no clue.
>>
>> Is there a way to do this? Thanks!
>>
>> Ying
>>
>> --
>> Ying Lin
>> NCEP/EMC/Verification, Post-processing and Product Generation
Branch
>> NCWCP Cubicle No. 2015
>> Ying.Lin at noaa.gov
>>
>>
>>
>>
--
Ying Lin
NCEP/EMC/Verification, Post-processing and Product Generation Branch
NCWCP Cubicle No. 2015
Ying.Lin at noaa.gov
------------------------------------------------
Subject: MetViewer: plot scores of the same model at different forecast lengths
From: John Halley Gotway
Time: Wed Aug 08 16:02:12 2018
Great, glad that did the trick!
Please continue to let us know when you run across confusing error
messages. Hopefully, we can identify common problems and come up with
useful error messages that point users in the right direction.
I'll go ahead and resolve this one now.
Thanks,
John
On Wed, Aug 8, 2018 at 2:48 PM Ying Lin via RT <met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86521 >
>
> Hi John (and Tatiana),
>
> Thank you! It works and matches what FVS produces. I see
what
> the problem was now - I didn't realize that the lead times should
have
> been defined in the Series Variables section.
>
> Also thanks much for your help yesterday on how to load *.stats
for
> multiple models w/o putting them in separate subdirs. Since that
> ticket was closed yesterday I hesitated about replying with a thank
you
> in case it would inadvertently re-open the ticket.
>
> Ying
>
> On 08/08/2018 03:17 PM, John Halley Gotway via RT wrote:
> > Hi Ying,
> >
> > I see that you're having trouble using METviewer to create a
series plot
> > where the lines on the plot represent the performance for
different
> > forecast lead times.
> >
> > Tatiana and I took a look at this together and figured it out.
> >
> > First let me say, that yes, of course, METviewer should be able to
make
> > this type of plot.
> >
> > Next, for the "Series" plot template, you define the series
variables
> (i.e.
> > lines on the plot) in the "Y1 Series Variables" section. In your
case,
> you
> > want 2 entries: MODEL = GFS and FCST_LEAD = 24,48. Actually, you
don't
> > technically *need* MODEL = GFS since that's all that's present in
this
> > database. But it doesn't hurt to include it. You don't need to
use the
> Y2
> > axis for something this simple.
> >
> > When I ran with your settings, I got a lengthy error like this:
> > failed to run plot plot_20180808_190845 - reason: R error: ====
Start
> > Rscript error ==== Error in cat(" boot time: ",
> > formatTimeSpan(dblBootTime), "\n", " boot.ci time: ", : object
> 'listStat'
> > not found Execution halted ==== End Rscript error ====
> >
> > The real problem is that your query returned no data... but
tracing
> through
> > the flow of the logic, this is the first error that gets thrown.
Tatiana
> > is creating a development ticket to write a more useful error in
the
> > future. Ideally, it'd say something like:
> > failed to run plot plot_20180808_190845 - reason: R error: ====
Start
> > Rscript error ==== The query returned no data. Execution halted
==== End
> > Rscript error ====
> >
> > But **why** does the query return no data. This database only
contains
> > forecasts initialized at 12Z. If you're evaluating 24 and 48 hour
lead
> > times, they'd be **valid** at 12Z. But you selected 00Z valid
times for
> > the x-axis. There are no 24 or 48 hour lead times valid at 00Z.
Just
> > switch the valid times from 00Z to 12Z and you'll see data.
> >
> > See attached XML and PNG with separate lines for 24, 48, 72, and
96 hour
> > lead times.
> >
> > Thanks,
> > John
> >
> > On Wed, Aug 8, 2018 at 10:06 AM Ying Lin via RT
<met_help at ucar.edu>
> wrote:
> >
> >> Wed Aug 08 10:06:02 2018: Request 86521 was acted upon.
> >> Transaction: Ticket created by ying.lin at noaa.gov
> >> Queue: met_help
> >> Subject: MetViewer: plot scores of the same model at
different
> >> forecast lengths
> >> Owner: Nobody
> >> Requestors: ying.lin at noaa.gov
> >> Status: new
> >> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=86521
> >
> >>
> >>
> >> Hello. I'd like to be plot scores of the same model at different
> >> forecast lengths (say, two separate lines for GFS GSS, the first
line
> >> for the forecast lead time of 24h, the 2nd for the forecast lead
time of
> >> 48h.
> >>
> >> If I select one model and choose two different forecast lead
times as
> >> two separate entries (rather than selecting '24' and '48' on the
same
> >> line for the forecast lead times), the plot would just be a
single line
> >> of scores for aggregated stats from 24h+48h lead times.
> >>
> >> Next I tried to select the model 'GFS' twice, on two separate
lines (as
> >> Group-y1_1 and Group-y1_2), then for fixed values choose two
separate
> >> forecast leads times, 24h and 48h, hoping that MetViewer would
apply the
> >> 24h to Group_y1_1 and the 48h Group-y1_2. It didn't seem likely
that
> >> MetViewer would interpret this the way I intended, and indeed
when I
> >> tried to plot I got an "R error" (see attached Screenshot).
> >>
> >> Then I tried to have the "GFS at 24h lead time" as Y1 axis
variable and
> >> "GFS at 48h lead time" as Y2 axis variable (that seemed unlikely
to be
> >> what the Y1/Y2 option was meant for, but I was out of ideas), see
the
> >> attached plot_*.xml and plot_*.png - it seems impossible to have
> >> different lead times for Y1/Y2. In the resulting plot, while the
red
> >> line is indeed the daily time series of GFS' GSS score at 24h
lead time,
> >> I've no idea what the blue line is (it's not the scores at 48h),
and the
> >> R-data offered no clue.
> >>
> >> Is there a way to do this? Thanks!
> >>
> >> Ying
> >>
> >> --
> >> Ying Lin
> >> NCEP/EMC/Verification, Post-processing and Product Generation
Branch
> >> NCWCP Cubicle No. 2015
> >> Ying.Lin at noaa.gov
> >>
> >>
> >>
> >>
>
> --
> Ying Lin
> NCEP/EMC/Verification, Post-processing and Product Generation Branch
> NCWCP Cubicle No. 2015
> Ying.Lin at noaa.gov
>
>
>
>
------------------------------------------------
More information about the Met_help
mailing list