[Met_help] [rt.rap.ucar.edu #91536] History for Verifying point observations at each forecast grid cell

[John Halley Gotway via RT]

----------------------------------------------------------------
  Initial Request
----------------------------------------------------------------

I'm looking for a way to verify forecasts on a grid using local storm
reports as observations.

For example, I would like to see if a grid cell with wind over 25 m/s has a
local storm report in the same grid cell. If it does, I would consider that
grid cell a hit; if not, I would call that grid cell a false alarm.

These irregularly-spaced storm reports must first be interpolated to the
forecast grid somehow, but I can't figure out how to do this in MET. I
already converted local storm reports to a netCDF file that point_stat can
read using ascii2nc.

Problem is, when I run them through point_stat or ensemble_stat, the
verification matching occurs at each storm report location, not at each
forecast grid cell. I would like each grid cell to be a matched
forecast/observation pair.

I tried using the regrid at to_grid setting in the point_stat config file but
it only affects gridded input.

Is there a way to do verify point observations on a forecast grid in MET?
Or should I convert the point observations to a grid myself and then run
grid_stat?

Hope my question makes sense, and hope I didn't miss something obvious in
the User's Guide.
Thanks.
Dave


----------------------------------------------------------------
  Complete Ticket History
----------------------------------------------------------------

Subject: Verifying point observations at each forecast grid cell
From: Julie Prestopnik
Time: Thu Aug 15 10:10:55 2019

Hi Dave.

I see that you have some questions regarding the verification of point
observations.  I've assigned this ticket to John Halley Gotway.  You
should
be
hearing from him in the next couple of days.

Thanks,
Julie

On Thu, Aug 15, 2019 at 9:58 AM David Ahijevych via RT
<met_help at ucar.edu>
wrote:

>
> Thu Aug 15 09:57:50 2019: Request 91536 was acted upon.
> Transaction: Ticket created by ahijevyc
>        Queue: met_help
>      Subject: Verifying point observations at each forecast grid
cell
>        Owner: Nobody
>   Requestors: ahijevyc at ucar.edu
>       Status: new
>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=91536 >
>
>
> I'm looking for a way to verify forecasts on a grid using local
storm
> reports as observations.
>
> For example, I would like to see if a grid cell with wind over 25
m/s has a
> local storm report in the same grid cell. If it does, I would
consider that
> grid cell a hit; if not, I would call that grid cell a false alarm.
>
> These irregularly-spaced storm reports must first be interpolated to
the
> forecast grid somehow, but I can't figure out how to do this in MET.
I
> already converted local storm reports to a netCDF file that
point_stat can
> read using ascii2nc.
>
> Problem is, when I run them through point_stat or ensemble_stat, the
> verification matching occurs at each storm report location, not at
each
> forecast grid cell. I would like each grid cell to be a matched
> forecast/observation pair.
>
> I tried using the regrid at to_grid setting in the point_stat config
file but
> it only affects gridded input.
>
> Is there a way to do verify point observations on a forecast grid in
MET?
> Or should I convert the point observations to a grid myself and then
run
> grid_stat?
>
> Hope my question makes sense, and hope I didn't miss something
obvious in
> the User's Guide.
> Thanks.
> Dave
>
>

--
Julie Prestopnik
Software Engineer
National Center for Atmospheric Research
Research Applications Laboratory
Phone: 303.497.8399
Email: jpresto at ucar.edu

My working day may not be your working day.  Please do not feel
obliged to
reply to this email outside of your normal working hours.

------------------------------------------------
Subject: Verifying point observations at each forecast grid cell
From: John Halley Gotway
Time: Thu Aug 15 12:11:14 2019

Hi Dave,

That's an interesting question.  Yes, I agree, it really does sound
like
you want to do a grid-to-grid verification with Grid-Stat to implement
the
vx logic you're describing.  So the real question is, how can you put
the
local storm reports onto a grid?

And the answer is that in the current version of MET, there are no
tools to
support that.

However, Howard Soh (engineer) and Lindsay Blank (scientist) are
working on
adding that support in met-9.0.  This is called the "surrogate-severe"
product at SPC/NSSL which uses a gaussian filter to place local storm
reports on a grid.  But the MET implementation really isn't ready for
testing yet.

However, met-8.1 does include support for "python-embedding".  That
means,
the MET tools can execute a python script and read a gridded field's
data
in memory from that script.  There is a current DTC visitor named Eric
Loken (cc'ed here) who's been running surrogate-severe.

Here's what I'd recommend...
- Talk to Eric about deriving surrogate-severe.  He may have a python
script he could share with you.
- Run or adapt Eric's methodology/code to derive the gridded field
from the
local storm reports.
- Use python embedding to run Grid-Stat and verify the model output
against
the python-derived gridded field of observations.
- Collaborate with Lindsay (cc'ed here) to make sure that your use-
case is
covered by the future MET development.

Hope that helps.

Thanks,
John



On Thu, Aug 15, 2019 at 9:58 AM David Ahijevych via RT
<met_help at ucar.edu>
wrote:

>
> Thu Aug 15 09:57:50 2019: Request 91536 was acted upon.
> Transaction: Ticket created by ahijevyc
>        Queue: met_help
>      Subject: Verifying point observations at each forecast grid
cell
>        Owner: Nobody
>   Requestors: ahijevyc at ucar.edu
>       Status: new
>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=91536 >
>
>
> I'm looking for a way to verify forecasts on a grid using local
storm
> reports as observations.
>
> For example, I would like to see if a grid cell with wind over 25
m/s has a
> local storm report in the same grid cell. If it does, I would
consider that
> grid cell a hit; if not, I would call that grid cell a false alarm.
>
> These irregularly-spaced storm reports must first be interpolated to
the
> forecast grid somehow, but I can't figure out how to do this in MET.
I
> already converted local storm reports to a netCDF file that
point_stat can
> read using ascii2nc.
>
> Problem is, when I run them through point_stat or ensemble_stat, the
> verification matching occurs at each storm report location, not at
each
> forecast grid cell. I would like each grid cell to be a matched
> forecast/observation pair.
>
> I tried using the regrid at to_grid setting in the point_stat config
file but
> it only affects gridded input.
>
> Is there a way to do verify point observations on a forecast grid in
MET?
> Or should I convert the point observations to a grid myself and then
run
> grid_stat?
>
> Hope my question makes sense, and hope I didn't miss something
obvious in
> the User's Guide.
> Thanks.
> Dave
>
>

------------------------------------------------
Subject: Verifying point observations at each forecast grid cell
From: David Ahijevych
Time: Fri Aug 16 09:42:37 2019

Thanks, John.  I'll talk to Eric about surrogate-severe.  Maybe he has
some
tips for me.

On Thu, Aug 15, 2019 at 12:11 PM John Halley Gotway via RT <
met_help at ucar.edu> wrote:

> Hi Dave,
>
> That's an interesting question.  Yes, I agree, it really does sound
like
> you want to do a grid-to-grid verification with Grid-Stat to
implement the
> vx logic you're describing.  So the real question is, how can you
put the
> local storm reports onto a grid?
>
> And the answer is that in the current version of MET, there are no
tools to
> support that.
>
> However, Howard Soh (engineer) and Lindsay Blank (scientist) are
working on
> adding that support in met-9.0.  This is called the "surrogate-
severe"
> product at SPC/NSSL which uses a gaussian filter to place local
storm
> reports on a grid.  But the MET implementation really isn't ready
for
> testing yet.
>
> However, met-8.1 does include support for "python-embedding".  That
means,
> the MET tools can execute a python script and read a gridded field's
data
> in memory from that script.  There is a current DTC visitor named
Eric
> Loken (cc'ed here) who's been running surrogate-severe.
>
> Here's what I'd recommend...
> - Talk to Eric about deriving surrogate-severe.  He may have a
python
> script he could share with you.
> - Run or adapt Eric's methodology/code to derive the gridded field
from the
> local storm reports.
> - Use python embedding to run Grid-Stat and verify the model output
against
> the python-derived gridded field of observations.
> - Collaborate with Lindsay (cc'ed here) to make sure that your use-
case is
> covered by the future MET development.
>
> Hope that helps.
>
> Thanks,
> John
>
>
>
> On Thu, Aug 15, 2019 at 9:58 AM David Ahijevych via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > Thu Aug 15 09:57:50 2019: Request 91536 was acted upon.
> > Transaction: Ticket created by ahijevyc
> >        Queue: met_help
> >      Subject: Verifying point observations at each forecast grid
cell
> >        Owner: Nobody
> >   Requestors: ahijevyc at ucar.edu
> >       Status: new
> >  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=91536 >
> >
> >
> > I'm looking for a way to verify forecasts on a grid using local
storm
> > reports as observations.
> >
> > For example, I would like to see if a grid cell with wind over 25
m/s
> has a
> > local storm report in the same grid cell. If it does, I would
consider
> that
> > grid cell a hit; if not, I would call that grid cell a false
alarm.
> >
> > These irregularly-spaced storm reports must first be interpolated
to the
> > forecast grid somehow, but I can't figure out how to do this in
MET. I
> > already converted local storm reports to a netCDF file that
point_stat
> can
> > read using ascii2nc.
> >
> > Problem is, when I run them through point_stat or ensemble_stat,
the
> > verification matching occurs at each storm report location, not at
each
> > forecast grid cell. I would like each grid cell to be a matched
> > forecast/observation pair.
> >
> > I tried using the regrid at to_grid setting in the point_stat config
file
> but
> > it only affects gridded input.
> >
> > Is there a way to do verify point observations on a forecast grid
in MET?
> > Or should I convert the point observations to a grid myself and
then run
> > grid_stat?
> >
> > Hope my question makes sense, and hope I didn't miss something
obvious in
> > the User's Guide.
> > Thanks.
> > Dave
> >
> >
>
>

------------------------------------------------