[Met_help] [rt.rap.ucar.edu #91645] History for Help with interpretation of MTD output
Randy Bullock via RT
met_help at ucar.edu
Tue Aug 27 11:18:08 MDT 2019
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hi Randy,
hope you're doing well.
I've got MTD to work on some of the chlorophyll files comparing a forecast with a satellite analysis. This is the first time I've ever inspected the paired output for MTD. When I used the tool this bit hadn't existed yet! So, I now have some output which I am trying to make sense of. Having read the guide I am wondering a couple of things, and hoping you can help?
* for the time delta, what does the minus mean? Is the difference F minus O? So if the forecast object existed from day 1 and the observed object existed only from day 3, then the difference is negative, so the forecast was too early?
* Same for speed?
* Interest values for clusters are -1? I think I understand that.
Thanks,
Marion
--
Dr Marion Mittermaier Manager: Model diagnostics and novel verification
Met Office FitzRoy Road Exeter EX1 3PB United Kingdom
Tel: +44 (0)1392 884830 / +44 (0) 330 135 1604 Fax: +44 (0)1392 885681
E-mail: marion.mittermaier at metoffice.gov.uk<mailto:marion.mittermaier at metoffice.gov.uk> http://www.metoffice.gov.uk<http://www.metoffice.gov.uk/>
http://www.metoffice.gov.uk/research/people/marion-mittermaier
I am co-chair of the WMO WWRP Joint Working Group on Forecast Verification Research (JWGFVR) http://www.wmo.int/pages/prog/arep/wwrp/new/Forecast_Verification.html
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Help with interpretation of MTD output
From: Minna Win
Time: Thu Aug 22 12:30:07 2019
Hi Marion,
It looks like you have specific MTD questions for Randy. I'm
assigning
this ticket to him, please allow a few business days for his response.
Regards,
Minna
---------------
Minna Win
NCAR
Research Applications Lab
Phone: 303-497-8423
Fax: 303-497-8401
On Thu, Aug 22, 2019 at 12:08 PM marion.mittermaier at metoffice.gov.uk
via RT
<met_help at ucar.edu> wrote:
>
> Thu Aug 22 11:55:33 2019: Request 91645 was acted upon.
> Transaction: Ticket created by marion.mittermaier at metoffice.gov.uk
> Queue: met_help
> Subject: Help with interpretation of MTD output
> Owner: Nobody
> Requestors: marion.mittermaier at metoffice.gov.uk
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=91645 >
>
>
> Hi Randy,
>
> hope you're doing well.
>
> I've got MTD to work on some of the chlorophyll files comparing a
forecast
> with a satellite analysis. This is the first time I've ever
inspected the
> paired output for MTD. When I used the tool this bit hadn't existed
yet!
> So, I now have some output which I am trying to make sense of.
Having read
> the guide I am wondering a couple of things, and hoping you can
help?
>
> * for the time delta, what does the minus mean? Is the
difference F
> minus O? So if the forecast object existed from day 1 and the
observed
> object existed only from day 3, then the difference is negative, so
the
> forecast was too early?
> * Same for speed?
> * Interest values for clusters are -1? I think I understand
that.
>
> Thanks,
> Marion
> --
> Dr Marion Mittermaier Manager: Model diagnostics and novel
verification
>
> Met Office FitzRoy Road Exeter EX1 3PB United Kingdom
> Tel: +44 (0)1392 884830 / +44 (0) 330 135 1604 Fax: +44 (0)1392
885681
> E-mail: marion.mittermaier at metoffice.gov.uk<mailto:
> marion.mittermaier at metoffice.gov.uk> http://www.metoffice.gov.uk<
> http://www.metoffice.gov.uk/>
>
> http://www.metoffice.gov.uk/research/people/marion-mittermaier
>
> I am co-chair of the WMO WWRP Joint Working Group on Forecast
Verification
> Research (JWGFVR)
>
http://www.wmo.int/pages/prog/arep/wwrp/new/Forecast_Verification.html
>
>
>
------------------------------------------------
Subject: Help with interpretation of MTD output
From: John Halley Gotway
Time: Thu Aug 22 15:15:16 2019
Hi Marion,
I took a quick look in the MTD source code and found the answer on
line
1490 of 3d_att.cc:
https://github.com/NCAR/MET/blob/master_v8.1/met/src/tools/other/mode_time_domain/3d_att.cc
As you guessed, the delta is computed as fcst minus obs. So if the
fcst
started at time step 2 and the obs started at time step 4, the delta
of -2
indicates that the fcst started early relative to the obs. Values
greater
than zero indicate that it's late.
1486 //
1487 // start time delta
1488 //
1489
1490 t = fcst_att.tmin() - obs_att.tmin();
1491
1492 p.set_start_time_delta(t);
Speed delta is set on line 1446 of that same file, as fcst - obs:
1446 p.set_speed_delta(fcst_att.speed() - obs_att.speed());
As for the interest values for the cluster pairs being -1, that
indicates
that the total interest wasn't computed for them. The
matching/merging in
MTD (just like MODE) is done using the interest values for the pairs
of
single objects. And those are used to construct the clusters.
In MODE, we do an additional step of passing those clusters back
through
the scoring algorithm to compute an interest value for each matched
cluster
pair. But those interest values are really just informational. They
have
no impact on the matching/merging since that step has already been
completed. However, this step of computing interest values on the
cluster
pairs is not currently being done in MTD. Perhaps, it's already slow
enough. But presumably, that logic could be added into MTD if its
useful.
Thanks,
John
On Thu, Aug 22, 2019 at 12:08 PM marion.mittermaier at metoffice.gov.uk
via RT
<met_help at ucar.edu> wrote:
>
> Thu Aug 22 11:55:33 2019: Request 91645 was acted upon.
> Transaction: Ticket created by marion.mittermaier at metoffice.gov.uk
> Queue: met_help
> Subject: Help with interpretation of MTD output
> Owner: Nobody
> Requestors: marion.mittermaier at metoffice.gov.uk
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=91645 >
>
>
> Hi Randy,
>
> hope you're doing well.
>
> I've got MTD to work on some of the chlorophyll files comparing a
forecast
> with a satellite analysis. This is the first time I've ever
inspected the
> paired output for MTD. When I used the tool this bit hadn't existed
yet!
> So, I now have some output which I am trying to make sense of.
Having read
> the guide I am wondering a couple of things, and hoping you can
help?
>
> * for the time delta, what does the minus mean? Is the
difference F
> minus O? So if the forecast object existed from day 1 and the
observed
> object existed only from day 3, then the difference is negative, so
the
> forecast was too early?
> * Same for speed?
> * Interest values for clusters are -1? I think I understand
that.
>
> Thanks,
> Marion
> --
> Dr Marion Mittermaier Manager: Model diagnostics and novel
verification
>
> Met Office FitzRoy Road Exeter EX1 3PB United Kingdom
> Tel: +44 (0)1392 884830 / +44 (0) 330 135 1604 Fax: +44 (0)1392
885681
> E-mail: marion.mittermaier at metoffice.gov.uk<mailto:
> marion.mittermaier at metoffice.gov.uk> http://www.metoffice.gov.uk<
> http://www.metoffice.gov.uk/>
>
> http://www.metoffice.gov.uk/research/people/marion-mittermaier
>
> I am co-chair of the WMO WWRP Joint Working Group on Forecast
Verification
> Research (JWGFVR)
>
http://www.wmo.int/pages/prog/arep/wwrp/new/Forecast_Verification.html
>
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #91645] Help with interpretation of MTD output
From: marion.mittermaier at metoffice.gov.uk
Time: Tue Aug 27 06:02:07 2019
Thanks John.
I keep forgetting I can look in the code...! Though it would take a
bit of time to find the bit I'm looking for.
The good news is that I am getting the code to run! I now need to
decide what to do with the output. I presume no one has tried to write
some generic plotting/manipulation of MTD output? I doubt my R code
from years ago will work now. Back then there was no output for the
matched objects anyway. If I manage to get something together I'll
send it back to you. Maybe you want to include it in the R scripts
section at a later stage...
Marion
-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: 22 August 2019 22:15
To: Mittermaier, Marion <marion.mittermaier at metoffice.gov.uk>
Subject: Re: [rt.rap.ucar.edu #91645] Help with interpretation of MTD
output
Hi Marion,
I took a quick look in the MTD source code and found the answer on
line
1490 of 3d_att.cc:
https://github.com/NCAR/MET/blob/master_v8.1/met/src/tools/other/mode_time_domain/3d_att.cc
As you guessed, the delta is computed as fcst minus obs. So if the
fcst started at time step 2 and the obs started at time step 4, the
delta of -2 indicates that the fcst started early relative to the obs.
Values greater than zero indicate that it's late.
1486 //
1487 // start time delta
1488 //
1489
1490 t = fcst_att.tmin() - obs_att.tmin();
1491
1492 p.set_start_time_delta(t);
Speed delta is set on line 1446 of that same file, as fcst - obs:
1446 p.set_speed_delta(fcst_att.speed() - obs_att.speed());
As for the interest values for the cluster pairs being -1, that
indicates that the total interest wasn't computed for them. The
matching/merging in MTD (just like MODE) is done using the interest
values for the pairs of single objects. And those are used to
construct the clusters.
In MODE, we do an additional step of passing those clusters back
through the scoring algorithm to compute an interest value for each
matched cluster pair. But those interest values are really just
informational. They have no impact on the matching/merging since that
step has already been completed. However, this step of computing
interest values on the cluster pairs is not currently being done in
MTD. Perhaps, it's already slow enough. But presumably, that logic
could be added into MTD if its useful.
Thanks,
John
On Thu, Aug 22, 2019 at 12:08 PM marion.mittermaier at metoffice.gov.uk
via RT <met_help at ucar.edu> wrote:
>
> Thu Aug 22 11:55:33 2019: Request 91645 was acted upon.
> Transaction: Ticket created by marion.mittermaier at metoffice.gov.uk
> Queue: met_help
> Subject: Help with interpretation of MTD output
> Owner: Nobody
> Requestors: marion.mittermaier at metoffice.gov.uk
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=91645
> >
>
>
> Hi Randy,
>
> hope you're doing well.
>
> I've got MTD to work on some of the chlorophyll files comparing a
> forecast with a satellite analysis. This is the first time I've ever
> inspected the paired output for MTD. When I used the tool this bit
hadn't existed yet!
> So, I now have some output which I am trying to make sense of.
Having
> read the guide I am wondering a couple of things, and hoping you can
help?
>
> * for the time delta, what does the minus mean? Is the
difference F
> minus O? So if the forecast object existed from day 1 and the
observed
> object existed only from day 3, then the difference is negative, so
> the forecast was too early?
> * Same for speed?
> * Interest values for clusters are -1? I think I understand
that.
>
> Thanks,
> Marion
> --
> Dr Marion Mittermaier Manager: Model diagnostics and novel
verification
>
> Met Office FitzRoy Road Exeter EX1 3PB United Kingdom
> Tel: +44 (0)1392 884830 / +44 (0) 330 135 1604 Fax: +44 (0)1392
> 885681
> E-mail: marion.mittermaier at metoffice.gov.uk<mailto:
> marion.mittermaier at metoffice.gov.uk> http://www.metoffice.gov.uk<
> http://www.metoffice.gov.uk/>
>
> http://www.metoffice.gov.uk/research/people/marion-mittermaier
>
> I am co-chair of the WMO WWRP Joint Working Group on Forecast
> Verification Research (JWGFVR)
>
http://www.wmo.int/pages/prog/arep/wwrp/new/Forecast_Verification.html
>
>
>
------------------------------------------------
Subject: Help with interpretation of MTD output
From: John Halley Gotway
Time: Tue Aug 27 08:34:39 2019
Marion,
You’re right, we don’t currently have any general purpose plotting
code for
the output of MTD yet. However, we do plan on writing some for the
hydromet testbed (HMT) project for the NOAA Weather Prediction Center
(WPC). And the result of that will ultimately live in the METplotpy
repo (
github.com/NCAR/METplotpy). That repo is currently private but we’ll
make
it public once it has some semblance of order.
Sure, yes, we’d be happy to see your plots and R code. The MTD object
attributes can be plotted using METviewer, but those are line or
boxplots,
not spatial plots of the objects themselves.
Thanks
John
On Tue, Aug 27, 2019 at 6:02 AM marion.mittermaier at metoffice.gov.uk
via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=91645 >
>
> Thanks John.
>
> I keep forgetting I can look in the code...! Though it would take a
bit of
> time to find the bit I'm looking for.
>
> The good news is that I am getting the code to run! I now need to
decide
> what to do with the output. I presume no one has tried to write some
> generic plotting/manipulation of MTD output? I doubt my R code from
years
> ago will work now. Back then there was no output for the matched
objects
> anyway. If I manage to get something together I'll send it back to
you.
> Maybe you want to include it in the R scripts section at a later
stage...
>
> Marion
>
> -----Original Message-----
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: 22 August 2019 22:15
> To: Mittermaier, Marion <marion.mittermaier at metoffice.gov.uk>
> Subject: Re: [rt.rap.ucar.edu #91645] Help with interpretation of
MTD
> output
>
> Hi Marion,
>
> I took a quick look in the MTD source code and found the answer on
line
> 1490 of 3d_att.cc:
>
>
>
https://github.com/NCAR/MET/blob/master_v8.1/met/src/tools/other/mode_time_domain/3d_att.cc
>
> As you guessed, the delta is computed as fcst minus obs. So if the
fcst
> started at time step 2 and the obs started at time step 4, the delta
of -2
> indicates that the fcst started early relative to the obs. Values
greater
> than zero indicate that it's late.
>
> 1486 //
> 1487 // start time delta
> 1488 //
> 1489
> 1490 t = fcst_att.tmin() - obs_att.tmin();
> 1491
> 1492 p.set_start_time_delta(t);
>
> Speed delta is set on line 1446 of that same file, as fcst - obs:
>
> 1446 p.set_speed_delta(fcst_att.speed() - obs_att.speed());
>
> As for the interest values for the cluster pairs being -1, that
indicates
> that the total interest wasn't computed for them. The
matching/merging in
> MTD (just like MODE) is done using the interest values for the pairs
of
> single objects. And those are used to construct the clusters.
>
> In MODE, we do an additional step of passing those clusters back
through
> the scoring algorithm to compute an interest value for each matched
cluster
> pair. But those interest values are really just informational.
They have
> no impact on the matching/merging since that step has already been
> completed. However, this step of computing interest values on the
cluster
> pairs is not currently being done in MTD. Perhaps, it's already
slow
> enough. But presumably, that logic could be added into MTD if its
useful.
>
> Thanks,
> John
>
> On Thu, Aug 22, 2019 at 12:08 PM marion.mittermaier at metoffice.gov.uk
via
> RT <met_help at ucar.edu> wrote:
>
> >
> > Thu Aug 22 11:55:33 2019: Request 91645 was acted upon.
> > Transaction: Ticket created by marion.mittermaier at metoffice.gov.uk
> > Queue: met_help
> > Subject: Help with interpretation of MTD output
> > Owner: Nobody
> > Requestors: marion.mittermaier at metoffice.gov.uk
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=91645
> > >
> >
> >
> > Hi Randy,
> >
> > hope you're doing well.
> >
> > I've got MTD to work on some of the chlorophyll files comparing a
> > forecast with a satellite analysis. This is the first time I've
ever
> > inspected the paired output for MTD. When I used the tool this bit
> hadn't existed yet!
> > So, I now have some output which I am trying to make sense of.
Having
> > read the guide I am wondering a couple of things, and hoping you
can
> help?
> >
> > * for the time delta, what does the minus mean? Is the
difference F
> > minus O? So if the forecast object existed from day 1 and the
observed
> > object existed only from day 3, then the difference is negative,
so
> > the forecast was too early?
> > * Same for speed?
> > * Interest values for clusters are -1? I think I understand
that.
> >
> > Thanks,
> > Marion
> > --
> > Dr Marion Mittermaier Manager: Model diagnostics and novel
> verification
> >
> > Met Office FitzRoy Road Exeter EX1 3PB United Kingdom
> > Tel: +44 (0)1392 884830 / +44 (0) 330 135 1604 Fax: +44 (0)1392
> > 885681
> > E-mail: marion.mittermaier at metoffice.gov.uk<mailto:
> > marion.mittermaier at metoffice.gov.uk> http://www.metoffice.gov.uk<
> > http://www.metoffice.gov.uk/>
> >
> > http://www.metoffice.gov.uk/research/people/marion-mittermaier
> >
> > I am co-chair of the WMO WWRP Joint Working Group on Forecast
> > Verification Research (JWGFVR)
> >
http://www.wmo.int/pages/prog/arep/wwrp/new/Forecast_Verification.html
> >
> >
> >
>
>
>
>
------------------------------------------------
More information about the Met_help
mailing list