[Met_help] [rt.rap.ucar.edu #87316] History for Verifying daily min/max temps from hourly data
John Halley Gotway via RT
met_help at ucar.edu
Tue Jul 9 12:07:19 MDT 2019
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hello,
I would like to verify daily min/max temps. Both observed and forecast min/max temps would be calculated from hourly data (MADIS for obs, GRIB-2 for models). I’d also like to compute stats on outliers (e.g., temp errors > 5°C). Is this possible with MET? Thank you.
Richard Carpenter
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Verifying daily min/max temps from hourly data
From: John Halley Gotway
Time: Mon Oct 08 16:25:44 2018
Richard,
I see that you're interested in verifying min/max temperatures using
the
MET software. You would like to compare hourly GRIB2 model forecasts
to
hourly MADIS observations.
The answer is *sort of* but it could be better. Let me explain.
First, pre-process your model data to compute a gridded field of daily
min/max values. This sounds a bit awkward, but the Ensemble-Stat tool
could be run to use 24 hourly files to define an *ensemble*. And
Ensemble-Stat could be configured to dump out a field of MIN and MAX
values. That would give you a forecast for daily min/max fields.
The next logic step would be to pre-process your point observations to
compute a daily min/max value for each station. Since your
observations
are madis, you'd run the madis2nc tool. However, the madis2nc tool
currently *does not* compute time summaries. The pb2nc tool (for
PREPBUFR)
and ascii2nc tool (for ascii) do compute time summaries, but madis2nc
does
not. I created a development ticket to enhance madis2nc to do so, but
as
of met-8.0, it does not.
However, all is not loss. You could run madis2nc to pre-process each
hour
of madis observations. Then run the Point-Stat tool to compare the
min/max
fields from Ensemble-Stat to the hourly madis observations directly.
Pass
a full 24 hour's worth of madis2nc output to Point-Stat but configure
Point-Stat by setting:
obs_summary = MIN;
... when verifying the min temperature, and...
obs_summary = MAX;
... when verifying the max temperature.
So Point-Stat would compare the forecast maximum value to maximum of
the 24
hourly observations... and similarly for the minimums.
Make sense? Let me know what other issues or questions arise.
Thanks,
John Halley Gotway
On Mon, Oct 8, 2018 at 2:55 PM Richard Carpenter via RT
<met_help at ucar.edu>
wrote:
>
> Mon Oct 08 14:55:01 2018: Request 87316 was acted upon.
> Transaction: Ticket created by rcarpenter at wdtinc.com
> Queue: met_help
> Subject: Verifying daily min/max temps from hourly data
> Owner: Nobody
> Requestors: rcarpenter at wdtinc.com
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=87316 >
>
>
> Hello,
>
> I would like to verify daily min/max temps. Both observed and
forecast
> min/max temps would be calculated from hourly data (MADIS for obs,
GRIB-2
> for models). I’d also like to compute stats on outliers (e.g., temp
errors
> > 5°C). Is this possible with MET? Thank you.
>
> Richard Carpenter
>
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #87316] Verifying daily min/max temps from hourly data
From: Richard Carpenter
Time: Thu Oct 11 16:02:36 2018
I'm making progress but am getting stuck running point_stat. The
ensemble_stat file contains vars like TMP_Z2_ENS_MIN but point_stat is
looking for TMPZ2. I've tried various settings for "name" and "level"
without success. Thanks.
process_fcst_climo_files() -> no fields matching TMPZ2 found in file:
ensemble_stat_20181008_000000V_ens.nc
On 10/8/18, 5:25 PM, "John Halley Gotway via RT" <met_help at ucar.edu>
wrote:
Richard,
I see that you're interested in verifying min/max temperatures using
the
MET software. You would like to compare hourly GRIB2 model forecasts
to
hourly MADIS observations.
The answer is *sort of* but it could be better. Let me explain.
First, pre-process your model data to compute a gridded field of daily
min/max values. This sounds a bit awkward, but the Ensemble-Stat tool
could be run to use 24 hourly files to define an *ensemble*. And
Ensemble-Stat could be configured to dump out a field of MIN and MAX
values. That would give you a forecast for daily min/max fields.
The next logic step would be to pre-process your point observations to
compute a daily min/max value for each station. Since your
observations
are madis, you'd run the madis2nc tool. However, the madis2nc tool
currently *does not* compute time summaries. The pb2nc tool (for
PREPBUFR)
and ascii2nc tool (for ascii) do compute time summaries, but madis2nc
does
not. I created a development ticket to enhance madis2nc to do so, but
as
of met-8.0, it does not.
However, all is not loss. You could run madis2nc to pre-process each
hour
of madis observations. Then run the Point-Stat tool to compare the
min/max
fields from Ensemble-Stat to the hourly madis observations directly.
Pass
a full 24 hour's worth of madis2nc output to Point-Stat but configure
Point-Stat by setting:
obs_summary = MIN;
... when verifying the min temperature, and...
obs_summary = MAX;
... when verifying the max temperature.
So Point-Stat would compare the forecast maximum value to maximum of
the 24
hourly observations... and similarly for the minimums.
Make sense? Let me know what other issues or questions arise.
Thanks,
John Halley Gotway
On Mon, Oct 8, 2018 at 2:55 PM Richard Carpenter via RT
<met_help at ucar.edu>
wrote:
>
> Mon Oct 08 14:55:01 2018: Request 87316 was acted upon.
> Transaction: Ticket created by rcarpenter at wdtinc.com
> Queue: met_help
> Subject: Verifying daily min/max temps from hourly data
> Owner: Nobody
> Requestors: rcarpenter at wdtinc.com
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=87316 >
>
>
> Hello,
>
> I would like to verify daily min/max temps. Both observed and
forecast
> min/max temps would be calculated from hourly data (MADIS for obs,
GRIB-2
> for models). I’d also like to compute stats on outliers (e.g., temp
errors
> > 5°C). Is this possible with MET? Thank you.
>
> Richard Carpenter
>
>
------------------------------------------------
Subject: Verifying daily min/max temps from hourly data
From: John Halley Gotway
Time: Mon Oct 22 13:34:54 2018
Richard,
I see this MET-Help ticket still open and wanted to check back in with
you
to see if you're still stuck.
Thanks,
John
On Thu, Oct 11, 2018 at 4:03 PM Richard Carpenter via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=87316 >
>
> I'm making progress but am getting stuck running point_stat. The
> ensemble_stat file contains vars like TMP_Z2_ENS_MIN but point_stat
is
> looking for TMPZ2. I've tried various settings for "name" and
"level"
> without success. Thanks.
>
> process_fcst_climo_files() -> no fields matching TMPZ2 found in
file:
> ensemble_stat_20181008_000000V_ens.nc
>
> On 10/8/18, 5:25 PM, "John Halley Gotway via RT"
<met_help at ucar.edu>
> wrote:
>
> Richard,
>
> I see that you're interested in verifying min/max temperatures using
the
> MET software. You would like to compare hourly GRIB2 model
forecasts to
> hourly MADIS observations.
>
> The answer is *sort of* but it could be better. Let me explain.
>
> First, pre-process your model data to compute a gridded field of
daily
> min/max values. This sounds a bit awkward, but the Ensemble-Stat
tool
> could be run to use 24 hourly files to define an *ensemble*. And
> Ensemble-Stat could be configured to dump out a field of MIN and MAX
> values. That would give you a forecast for daily min/max fields.
>
> The next logic step would be to pre-process your point observations
to
> compute a daily min/max value for each station. Since your
observations
> are madis, you'd run the madis2nc tool. However, the madis2nc tool
> currently *does not* compute time summaries. The pb2nc tool (for
PREPBUFR)
> and ascii2nc tool (for ascii) do compute time summaries, but
madis2nc does
> not. I created a development ticket to enhance madis2nc to do so,
but as
> of met-8.0, it does not.
>
> However, all is not loss. You could run madis2nc to pre-process
each hour
> of madis observations. Then run the Point-Stat tool to compare the
min/max
> fields from Ensemble-Stat to the hourly madis observations directly.
Pass
> a full 24 hour's worth of madis2nc output to Point-Stat but
configure
> Point-Stat by setting:
> obs_summary = MIN;
> ... when verifying the min temperature, and...
> obs_summary = MAX;
> ... when verifying the max temperature.
>
> So Point-Stat would compare the forecast maximum value to maximum of
the 24
> hourly observations... and similarly for the minimums.
>
> Make sense? Let me know what other issues or questions arise.
>
> Thanks,
> John Halley Gotway
>
> On Mon, Oct 8, 2018 at 2:55 PM Richard Carpenter via RT
<met_help at ucar.edu
> >
> wrote:
>
> >
> > Mon Oct 08 14:55:01 2018: Request 87316 was acted upon.
> > Transaction: Ticket created by rcarpenter at wdtinc.com
> > Queue: met_help
> > Subject: Verifying daily min/max temps from hourly data
> > Owner: Nobody
> > Requestors: rcarpenter at wdtinc.com
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=87316 >
> >
> >
> > Hello,
> >
> > I would like to verify daily min/max temps. Both observed and
forecast
> > min/max temps would be calculated from hourly data (MADIS for obs,
GRIB-2
> > for models). I’d also like to compute stats on outliers (e.g.,
temp
> errors
> > > 5°C). Is this possible with MET? Thank you.
> >
> > Richard Carpenter
> >
> >
>
>
>
>
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #87316] Verifying daily min/max temps from hourly data
From: Richard Carpenter
Time: Mon Oct 22 13:47:39 2018
John,
I was able to figure that one out. Separate fcst and obs groups with
different field/name.
That does bring me to my next question, is it possible to compute
"outliers", e.g., percent of values exceeding a certain error
threshold? I know it wasn't possible in the past, but there might be a
way now.
Thanks.
Richard Carpenter
On 10/22/18, 2:34 PM, "John Halley Gotway via RT" <met_help at ucar.edu>
wrote:
Richard,
I see this MET-Help ticket still open and wanted to check back in with
you
to see if you're still stuck.
Thanks,
John
On Thu, Oct 11, 2018 at 4:03 PM Richard Carpenter via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=87316 >
>
> I'm making progress but am getting stuck running point_stat. The
> ensemble_stat file contains vars like TMP_Z2_ENS_MIN but point_stat
is
> looking for TMPZ2. I've tried various settings for "name" and
"level"
> without success. Thanks.
>
> process_fcst_climo_files() -> no fields matching TMPZ2 found in
file:
> ensemble_stat_20181008_000000V_ens.nc
>
> On 10/8/18, 5:25 PM, "John Halley Gotway via RT" <met_help at ucar.edu>
> wrote:
>
> Richard,
>
> I see that you're interested in verifying min/max temperatures using
the
> MET software. You would like to compare hourly GRIB2 model
forecasts to
> hourly MADIS observations.
>
> The answer is *sort of* but it could be better. Let me explain.
>
> First, pre-process your model data to compute a gridded field of
daily
> min/max values. This sounds a bit awkward, but the Ensemble-Stat
tool
> could be run to use 24 hourly files to define an *ensemble*. And
> Ensemble-Stat could be configured to dump out a field of MIN and MAX
> values. That would give you a forecast for daily min/max fields.
>
> The next logic step would be to pre-process your point observations
to
> compute a daily min/max value for each station. Since your
observations
> are madis, you'd run the madis2nc tool. However, the madis2nc tool
> currently *does not* compute time summaries. The pb2nc tool (for
PREPBUFR)
> and ascii2nc tool (for ascii) do compute time summaries, but
madis2nc does
> not. I created a development ticket to enhance madis2nc to do so,
but as
> of met-8.0, it does not.
>
> However, all is not loss. You could run madis2nc to pre-process
each hour
> of madis observations. Then run the Point-Stat tool to compare the
min/max
> fields from Ensemble-Stat to the hourly madis observations directly.
Pass
> a full 24 hour's worth of madis2nc output to Point-Stat but
configure
> Point-Stat by setting:
> obs_summary = MIN;
> ... when verifying the min temperature, and...
> obs_summary = MAX;
> ... when verifying the max temperature.
>
> So Point-Stat would compare the forecast maximum value to maximum of
the 24
> hourly observations... and similarly for the minimums.
>
> Make sense? Let me know what other issues or questions arise.
>
> Thanks,
> John Halley Gotway
>
> On Mon, Oct 8, 2018 at 2:55 PM Richard Carpenter via RT
<met_help at ucar.edu
> >
> wrote:
>
> >
> > Mon Oct 08 14:55:01 2018: Request 87316 was acted upon.
> > Transaction: Ticket created by rcarpenter at wdtinc.com
> > Queue: met_help
> > Subject: Verifying daily min/max temps from hourly data
> > Owner: Nobody
> > Requestors: rcarpenter at wdtinc.com
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=87316 >
> >
> >
> > Hello,
> >
> > I would like to verify daily min/max temps. Both observed and
forecast
> > min/max temps would be calculated from hourly data (MADIS for obs,
GRIB-2
> > for models). I’d also like to compute stats on outliers (e.g.,
temp
> errors
> > > 5°C). Is this possible with MET? Thank you.
> >
> > Richard Carpenter
> >
> >
>
>
>
>
>
------------------------------------------------
Subject: Verifying daily min/max temps from hourly data
From: John Halley Gotway
Time: Mon Oct 22 14:33:17 2018
Richard,
OK, so you're interested in seeing more information about the
distribution
of the errors. Here are a few thoughts:
(1) The CNT line type from Point-Stat and Grid-Stat including columns
for
the various percentiles of the error as well as the inter-quartile
ranges:
E10, E25, E50, E75, E90, EIQR
That gives you a more complete picture of the distribution of the
errors.
(2) But you'd specifically like the know the percentage of the errors
exceeding a certain threshold. We did port some logic over from the
TC-Stat tool to the STAT-Analysis tool that may be useful. Let's say
you've run Point-Stat and generated the MPR output line type, you
could run
this type of job on it:
bin/stat_analysis \
-lookin out/point_stat/point_stat_360000L_20070331_120000V_mpr.txt \
-job filter -line_type MPR -fcst_var TMP \
-dump_row large_errors.txt \
-column_thresh 'ABS(FCST-OBS)' gt5
This is a "filter" job where we're picking out individual MPR lines
for the
temperature (TMP) variable... and writing them out to the specified
output
file name. The "-column_thresh" option is the interesting one here.
We've
specified that we only want lines where the absolute value of the FCST
column minus the OBS column exceeds 5. Now that doesn't tell you the
percentage exactly, but it gives you a way of filtering the
differences and
may help you identify outliers.
Hope that helps clarify.
Thanks,
John
On Mon, Oct 22, 2018 at 1:48 PM Richard Carpenter via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=87316 >
>
> John,
>
> I was able to figure that one out. Separate fcst and obs groups with
> different field/name.
>
> That does bring me to my next question, is it possible to compute
> "outliers", e.g., percent of values exceeding a certain error
threshold? I
> know it wasn't possible in the past, but there might be a way now.
>
> Thanks.
>
> Richard Carpenter
>
> On 10/22/18, 2:34 PM, "John Halley Gotway via RT"
<met_help at ucar.edu>
> wrote:
>
> Richard,
>
> I see this MET-Help ticket still open and wanted to check back in
with you
> to see if you're still stuck.
>
> Thanks,
> John
>
> On Thu, Oct 11, 2018 at 4:03 PM Richard Carpenter via RT <
> met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=87316 >
> >
> > I'm making progress but am getting stuck running point_stat. The
> > ensemble_stat file contains vars like TMP_Z2_ENS_MIN but
point_stat is
> > looking for TMPZ2. I've tried various settings for "name" and
"level"
> > without success. Thanks.
> >
> > process_fcst_climo_files() -> no fields matching TMPZ2 found in
file:
> > ensemble_stat_20181008_000000V_ens.nc
> >
> > On 10/8/18, 5:25 PM, "John Halley Gotway via RT"
<met_help at ucar.edu>
> > wrote:
> >
> > Richard,
> >
> > I see that you're interested in verifying min/max temperatures
using the
> > MET software. You would like to compare hourly GRIB2 model
forecasts to
> > hourly MADIS observations.
> >
> > The answer is *sort of* but it could be better. Let me explain.
> >
> > First, pre-process your model data to compute a gridded field of
daily
> > min/max values. This sounds a bit awkward, but the Ensemble-Stat
tool
> > could be run to use 24 hourly files to define an *ensemble*. And
> > Ensemble-Stat could be configured to dump out a field of MIN and
MAX
> > values. That would give you a forecast for daily min/max fields.
> >
> > The next logic step would be to pre-process your point
observations to
> > compute a daily min/max value for each station. Since your
observations
> > are madis, you'd run the madis2nc tool. However, the madis2nc
tool
> > currently *does not* compute time summaries. The pb2nc tool (for
> PREPBUFR)
> > and ascii2nc tool (for ascii) do compute time summaries, but
madis2nc
> does
> > not. I created a development ticket to enhance madis2nc to do so,
but as
> > of met-8.0, it does not.
> >
> > However, all is not loss. You could run madis2nc to pre-process
each
> hour
> > of madis observations. Then run the Point-Stat tool to compare
the
> min/max
> > fields from Ensemble-Stat to the hourly madis observations
directly.
> Pass
> > a full 24 hour's worth of madis2nc output to Point-Stat but
configure
> > Point-Stat by setting:
> > obs_summary = MIN;
> > ... when verifying the min temperature, and...
> > obs_summary = MAX;
> > ... when verifying the max temperature.
> >
> > So Point-Stat would compare the forecast maximum value to maximum
of the
> 24
> > hourly observations... and similarly for the minimums.
> >
> > Make sense? Let me know what other issues or questions arise.
> >
> > Thanks,
> > John Halley Gotway
> >
> > On Mon, Oct 8, 2018 at 2:55 PM Richard Carpenter via RT <
> met_help at ucar.edu
> > >
> > wrote:
> >
> > >
> > > Mon Oct 08 14:55:01 2018: Request 87316 was acted upon.
> > > Transaction: Ticket created by rcarpenter at wdtinc.com
> > > Queue: met_help
> > > Subject: Verifying daily min/max temps from hourly data
> > > Owner: Nobody
> > > Requestors: rcarpenter at wdtinc.com
> > > Status: new
> > > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=87316
> >
> > >
> > >
> > > Hello,
> > >
> > > I would like to verify daily min/max temps. Both observed and
forecast
> > > min/max temps would be calculated from hourly data (MADIS for
obs,
> GRIB-2
> > > for models). I’d also like to compute stats on outliers (e.g.,
temp
> > errors
> > > > 5°C). Is this possible with MET? Thank you.
> > >
> > > Richard Carpenter
> > >
> > >
> >
> >
> >
> >
> >
>
>
>
>
>
------------------------------------------------
More information about the Met_help
mailing list