[Met_help] [rt.rap.ucar.edu #96239] History for Quick question about capability of MET to verify point forecasts
Minna Win via RT
met_help at ucar.edu
Thu Aug 13 15:07:57 MDT 2020
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hi,
I have a quick, maybe naive, question. I know MET has the capability to
compare model gridded data with point-based observations (e.g., point_stat)
and model gridded data with gridded observations (e.g., grid_stat).
Does MET also have the ability to compare a *point* forecast (e.g., 10-m
wind speed, 2-m temperature forecast at a single point in Albany, NY)
against point observations? And/or over aggregated over several point
forecasts that are not spread equally on a grid (e.g., three point
forecasts: Albany, NY, Chicago, IL, Los Angeles, CA)?
Thanks!
Jeremy Berman
Ph.D. Student
University at Albany, NY
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Quick question about capability of MET to verify point forecasts
From: Minna Win
Time: Thu Aug 13 11:10:33 2020
Hi Jeremy,
It looks like you are hoping that MET has a point2point tool, but
unfortunately it does not. I will need to check with a scientist to
find
an answer to your second question-performing a grid-to-point
evaluation
using grid points that are unevenly spaced. Please allow a few
business
days for a full response.
Regards,
Minna
---------------
Minna Win
National Center for Atmospheric Research
Developmental Testbed Center
Phone: 303-497-8423
Fax: 303-497-8401
On Thu, Aug 13, 2020 at 10:09 AM Jeremy Berman via RT
<met_help at ucar.edu>
wrote:
>
> Thu Aug 13 10:09:17 2020: Request 96239 was acted upon.
> Transaction: Ticket created by jeremy.berman5 at gmail.com
> Queue: met_help
> Subject: Quick question about capability of MET to verify point
> forecasts
> Owner: Nobody
> Requestors: jeremy.berman5 at gmail.com
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96239 >
>
>
> Hi,
>
> I have a quick, maybe naive, question. I know MET has the capability
to
> compare model gridded data with point-based observations (e.g.,
point_stat)
> and model gridded data with gridded observations (e.g., grid_stat).
>
> Does MET also have the ability to compare a *point* forecast (e.g.,
10-m
> wind speed, 2-m temperature forecast at a single point in Albany,
NY)
> against point observations? And/or over aggregated over several
point
> forecasts that are not spread equally on a grid (e.g., three point
> forecasts: Albany, NY, Chicago, IL, Los Angeles, CA)?
>
> Thanks!
>
> Jeremy Berman
> Ph.D. Student
> University at Albany, NY
>
>
------------------------------------------------
Subject: Quick question about capability of MET to verify point forecasts
From: Jeremy Berman
Time: Thu Aug 13 11:25:38 2020
Hi Minna,
Yes, a "point2point tool" is what I was thinking. But that's
understandable, thanks for clarifying! Also appreciate you exploring
further about the second question.
Thanks!
Jeremy
On Thu, Aug 13, 2020 at 1:10 PM Minna Win via RT <met_help at ucar.edu>
wrote:
> Hi Jeremy,
>
> It looks like you are hoping that MET has a point2point tool, but
> unfortunately it does not. I will need to check with a scientist to
find
> an answer to your second question-performing a grid-to-point
evaluation
> using grid points that are unevenly spaced. Please allow a few
business
> days for a full response.
>
> Regards,
> Minna
> ---------------
> Minna Win
> National Center for Atmospheric Research
> Developmental Testbed Center
> Phone: 303-497-8423
> Fax: 303-497-8401
>
>
>
> On Thu, Aug 13, 2020 at 10:09 AM Jeremy Berman via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > Thu Aug 13 10:09:17 2020: Request 96239 was acted upon.
> > Transaction: Ticket created by jeremy.berman5 at gmail.com
> > Queue: met_help
> > Subject: Quick question about capability of MET to verify
point
> > forecasts
> > Owner: Nobody
> > Requestors: jeremy.berman5 at gmail.com
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96239 >
> >
> >
> > Hi,
> >
> > I have a quick, maybe naive, question. I know MET has the
capability to
> > compare model gridded data with point-based observations (e.g.,
> point_stat)
> > and model gridded data with gridded observations (e.g.,
grid_stat).
> >
> > Does MET also have the ability to compare a *point* forecast
(e.g., 10-m
> > wind speed, 2-m temperature forecast at a single point in Albany,
NY)
> > against point observations? And/or over aggregated over several
point
> > forecasts that are not spread equally on a grid (e.g., three point
> > forecasts: Albany, NY, Chicago, IL, Los Angeles, CA)?
> >
> > Thanks!
> >
> > Jeremy Berman
> > Ph.D. Student
> > University at Albany, NY
> >
> >
>
>
------------------------------------------------
Subject: Quick question about capability of MET to verify point forecasts
From: John Halley Gotway
Time: Thu Aug 13 11:50:28 2020
Jeremy,
We have used 2 approaches in the past for this type of data. When
evaluating a gridded forecast, most of the hard work is in computing
the
fcst/obs matched pairs. If you already have the data paired up in this
way,
and just want to derive statistics from them, then you can use MET's
stat-analysis tool to do so.
One of the outputs of the Point-Stat tool is the matched pair (MPR)
line
type. That simply has 1 line for each fcst/obs matched pair. If you
format
your paired fcst/obs data into the MPR line type, you can pass that to
the
stat-analysis tool and derive continuous or categorical statistics
from it.
You can either do this by writing those MPR lines to an ASCII output
file,
and then passing that file to stat-analysis. Or you can do it with
python
embedding... write a python script to do the reformatting and pass the
data
in memory to stat_analysis. Here's a python script in the MET repo
which
reads an MPR file and stores the data in memory for passing to
stat_analysis:
https://github.com/NCAR/MET/blob/master_v9.1/met/scripts/python/read_ascii_mpr.py
You'd obviously change the front of the script to read/format your
particular data instead. Here's an example of how you'd call
stat_analysis
in this way:
stat_analysis -lookin python read_ascii_mpr.py
point_stat_120000L_20050807_120000V.stat \ -job aggregate_stat
-line_type
MPR -out_line_type cnt -by FCST_VAR \
-out_stat stat_analysis_mpr_to_cnt.stat
So this reads data via that python script into stat_analysis which
computes
continuous stats (CNT) for each unique variable name found. Hope that
helps. Thanks, John Halley Gotway
On Thu, Aug 13, 2020 at 11:26 AM Jeremy Berman via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96239 >
>
> Hi Minna,
>
> Yes, a "point2point tool" is what I was thinking. But that's
> understandable, thanks for clarifying! Also appreciate you exploring
> further about the second question.
>
> Thanks!
> Jeremy
>
> On Thu, Aug 13, 2020 at 1:10 PM Minna Win via RT <met_help at ucar.edu>
> wrote:
>
> > Hi Jeremy,
> >
> > It looks like you are hoping that MET has a point2point tool, but
> > unfortunately it does not. I will need to check with a scientist
to find
> > an answer to your second question-performing a grid-to-point
evaluation
> > using grid points that are unevenly spaced. Please allow a few
business
> > days for a full response.
> >
> > Regards,
> > Minna
> > ---------------
> > Minna Win
> > National Center for Atmospheric Research
> > Developmental Testbed Center
> > Phone: 303-497-8423
> > Fax: 303-497-8401
> >
> >
> >
> > On Thu, Aug 13, 2020 at 10:09 AM Jeremy Berman via RT
<met_help at ucar.edu
> >
> > wrote:
> >
> > >
> > > Thu Aug 13 10:09:17 2020: Request 96239 was acted upon.
> > > Transaction: Ticket created by jeremy.berman5 at gmail.com
> > > Queue: met_help
> > > Subject: Quick question about capability of MET to verify
point
> > > forecasts
> > > Owner: Nobody
> > > Requestors: jeremy.berman5 at gmail.com
> > > Status: new
> > > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96239
> >
> > >
> > >
> > > Hi,
> > >
> > > I have a quick, maybe naive, question. I know MET has the
capability to
> > > compare model gridded data with point-based observations (e.g.,
> > point_stat)
> > > and model gridded data with gridded observations (e.g.,
grid_stat).
> > >
> > > Does MET also have the ability to compare a *point* forecast
(e.g.,
> 10-m
> > > wind speed, 2-m temperature forecast at a single point in
Albany, NY)
> > > against point observations? And/or over aggregated over several
point
> > > forecasts that are not spread equally on a grid (e.g., three
point
> > > forecasts: Albany, NY, Chicago, IL, Los Angeles, CA)?
> > >
> > > Thanks!
> > >
> > > Jeremy Berman
> > > Ph.D. Student
> > > University at Albany, NY
> > >
> > >
> >
> >
>
>
------------------------------------------------
Subject: Quick question about capability of MET to verify point forecasts
From: Jeremy Berman
Time: Thu Aug 13 14:47:30 2020
Hi John,
That's a clever idea of how to go about deriving verification
statistics
for point forecast data: predetermine my forecast/observation pair and
enter that as a MPR line to feed into the stat_analysis tool to derive
statistics.
Thank you for the explanation and sample command to test! I'll pursue
that,
as well as familiar myself with the python embedding script.
Thanks!!
Jeremy
On Thu, Aug 13, 2020 at 1:50 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Jeremy,
>
> We have used 2 approaches in the past for this type of data. When
> evaluating a gridded forecast, most of the hard work is in computing
the
> fcst/obs matched pairs. If you already have the data paired up in
this way,
> and just want to derive statistics from them, then you can use MET's
> stat-analysis tool to do so.
>
> One of the outputs of the Point-Stat tool is the matched pair (MPR)
line
> type. That simply has 1 line for each fcst/obs matched pair. If you
format
> your paired fcst/obs data into the MPR line type, you can pass that
to the
> stat-analysis tool and derive continuous or categorical statistics
from it.
>
> You can either do this by writing those MPR lines to an ASCII output
file,
> and then passing that file to stat-analysis. Or you can do it with
python
> embedding... write a python script to do the reformatting and pass
the data
> in memory to stat_analysis. Here's a python script in the MET repo
which
> reads an MPR file and stores the data in memory for passing to
> stat_analysis:
>
>
>
https://github.com/NCAR/MET/blob/master_v9.1/met/scripts/python/read_ascii_mpr.py
>
> You'd obviously change the front of the script to read/format your
> particular data instead. Here's an example of how you'd call
stat_analysis
> in this way:
>
> stat_analysis -lookin python read_ascii_mpr.py
> point_stat_120000L_20050807_120000V.stat \ -job aggregate_stat
-line_type
> MPR -out_line_type cnt -by FCST_VAR \
> -out_stat stat_analysis_mpr_to_cnt.stat
> So this reads data via that python script into stat_analysis which
computes
> continuous stats (CNT) for each unique variable name found. Hope
that
> helps. Thanks, John Halley Gotway
>
> On Thu, Aug 13, 2020 at 11:26 AM Jeremy Berman via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96239 >
> >
> > Hi Minna,
> >
> > Yes, a "point2point tool" is what I was thinking. But that's
> > understandable, thanks for clarifying! Also appreciate you
exploring
> > further about the second question.
> >
> > Thanks!
> > Jeremy
> >
> > On Thu, Aug 13, 2020 at 1:10 PM Minna Win via RT
<met_help at ucar.edu>
> > wrote:
> >
> > > Hi Jeremy,
> > >
> > > It looks like you are hoping that MET has a point2point tool,
but
> > > unfortunately it does not. I will need to check with a
scientist to
> find
> > > an answer to your second question-performing a grid-to-point
evaluation
> > > using grid points that are unevenly spaced. Please allow a few
> business
> > > days for a full response.
> > >
> > > Regards,
> > > Minna
> > > ---------------
> > > Minna Win
> > > National Center for Atmospheric Research
> > > Developmental Testbed Center
> > > Phone: 303-497-8423
> > > Fax: 303-497-8401
> > >
> > >
> > >
> > > On Thu, Aug 13, 2020 at 10:09 AM Jeremy Berman via RT <
> met_help at ucar.edu
> > >
> > > wrote:
> > >
> > > >
> > > > Thu Aug 13 10:09:17 2020: Request 96239 was acted upon.
> > > > Transaction: Ticket created by jeremy.berman5 at gmail.com
> > > > Queue: met_help
> > > > Subject: Quick question about capability of MET to verify
point
> > > > forecasts
> > > > Owner: Nobody
> > > > Requestors: jeremy.berman5 at gmail.com
> > > > Status: new
> > > > Ticket <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96239
> > >
> > > >
> > > >
> > > > Hi,
> > > >
> > > > I have a quick, maybe naive, question. I know MET has the
capability
> to
> > > > compare model gridded data with point-based observations
(e.g.,
> > > point_stat)
> > > > and model gridded data with gridded observations (e.g.,
grid_stat).
> > > >
> > > > Does MET also have the ability to compare a *point* forecast
(e.g.,
> > 10-m
> > > > wind speed, 2-m temperature forecast at a single point in
Albany, NY)
> > > > against point observations? And/or over aggregated over
several point
> > > > forecasts that are not spread equally on a grid (e.g., three
point
> > > > forecasts: Albany, NY, Chicago, IL, Los Angeles, CA)?
> > > >
> > > > Thanks!
> > > >
> > > > Jeremy Berman
> > > > Ph.D. Student
> > > > University at Albany, NY
> > > >
> > > >
> > >
> > >
> >
> >
>
>
------------------------------------------------
Subject: Quick question about capability of MET to verify point forecasts
From: Minna Win
Time: Thu Aug 13 15:07:56 2020
Suggested approach to performing an analysis with unevenly gridded
data points has been provided to the requestor from John Halley
Gotway.
------------------------------------------------
More information about the Met_help
mailing list