[Met_help] [rt.rap.ucar.edu #72307] History for Using Different Grib Tables
John Halley Gotway via RT
met_help at ucar.edu
Thu Jun 18 13:44:55 MDT 2015
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hi John, I am running point stat on some ensemble probability files. MET doesn't recognize the grib field name : PWS25 (probability of windspeed > 25) or the grib code AFWA defined for it: 200. How do I get met to use our local grib table or override the grib settings it using.
Thanks
Bob Craig
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Using Different Grib Tables
From: Julie Prestopnik
Time: Fri Jun 12 15:49:26 2015
Hi Bob.
I believe John may have left for the day. I thought I would try to
help
out.
In previous versions of MET, the GRIB tables were hard-coded, but the
code
is now reading them from a flat file at run time.
For METv4.1, that flat file can be found here:
METv4.1/data/table_files/nceptab_flat.txt
And in met-5.0, it's reading it from the installed met-5.0 directory
"share/met/table_files/nceptab_flat.txt".
If you'd like to change MET's behavior, you could edit that flat file
to
use whichever abbreviations and units you'd like. The next time you
run the
MET tools, it'll just read in the updated strings and use them.
I hope that helps.
Julie
On Fri, Jun 12, 2015 at 3:23 PM, robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> Fri Jun 12 15:23:06 2015: Request 72307 was acted upon.
> Transaction: Ticket created by robert.craig.2 at us.af.mil
> Queue: met_help
> Subject: Using Different Grib Tables
> Owner: Nobody
> Requestors: robert.craig.2 at us.af.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=72307 >
>
>
> Hi John, I am running point stat on some ensemble probability files.
MET
> doesn't recognize the grib field name : PWS25 (probability of
windspeed >
> 25) or the grib code AFWA defined for it: 200. How do I get met to
use
> our local grib table or override the grib settings it using.
>
> Thanks
>
> Bob Craig
>
>
>
--
Julie Prestopnik
National Center for Atmospheric Research
Research Applications Laboratory
Phone: 303.497.8399
Email: jpresto at ucar.edu
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #72307] Using Different Grib Tables
From: robert.craig.2 at us.af.mil
Time: Fri Jun 12 15:50:39 2015
Thanks, I will check that out.
Bob
-----Original Message-----
From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
Sent: Friday, June 12, 2015 4:49 PM
To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
Subject: Re: [rt.rap.ucar.edu #72307] Using Different Grib Tables
Hi Bob.
I believe John may have left for the day. I thought I would try to
help out.
In previous versions of MET, the GRIB tables were hard-coded, but the
code is now reading them from a flat file at run time.
For METv4.1, that flat file can be found here:
METv4.1/data/table_files/nceptab_flat.txt
And in met-5.0, it's reading it from the installed met-5.0 directory
"share/met/table_files/nceptab_flat.txt".
If you'd like to change MET's behavior, you could edit that flat file
to use whichever abbreviations and units you'd like. The next time you
run the MET tools, it'll just read in the updated strings and use
them.
I hope that helps.
Julie
On Fri, Jun 12, 2015 at 3:23 PM, robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> Fri Jun 12 15:23:06 2015: Request 72307 was acted upon.
> Transaction: Ticket created by robert.craig.2 at us.af.mil
> Queue: met_help
> Subject: Using Different Grib Tables
> Owner: Nobody
> Requestors: robert.craig.2 at us.af.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=72307
> >
>
>
> Hi John, I am running point stat on some ensemble probability files.
> MET doesn't recognize the grib field name : PWS25 (probability of
> windspeed >
> 25) or the grib code AFWA defined for it: 200. How do I get met to
> use our local grib table or override the grib settings it using.
>
> Thanks
>
> Bob Craig
>
>
>
--
Julie Prestopnik
National Center for Atmospheric Research Research Applications
Laboratory
Phone: 303.497.8399
Email: jpresto at ucar.edu
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #72307] Using Different Grib Tables
From: robert.craig.2 at us.af.mil
Time: Mon Jun 15 08:07:41 2015
Thanks Julie, that got me past the grib errors. Now I need some
assistance on properly configuring the configuration file for Point
Stat. I want to verify wind probability forecast of >25kts against
observations to calculate Brier Skill Scores. Below is an exert from
the configuration file:
//
// Forecast and observation fields to be verified
//
fcst = {
wind_thresh = [ NA ];
message_type = [ "ADPSFC" ];
field = [
{
name = "PWS25";
level = [ "L0" ];
prob = TRUE;
cat_thresh = [ >=12.9 ];
prob_thresh = ["ge0","ge0.5","ge1"]; ???
}
];
};
obs = {
wind_thresh = [ NA ];
message_type = [ "ADPSFC" ];
field = [
{
name = "WIND";
level = [ "L0" ];
cat_thresh = [>=12.9];
}
When I run this I get the following error:
DEBUG 1: Default Config File:
/home/qcteam/METv4.1/data/config/PointStatConfig_default
DEBUG 1: User Config File: data/config/PointStatConfig_geps
ERROR :
ERROR : check_prob_thresh() -> When verifying a probability field,
you must select at least 3 thresholds beginning with 0.0 and ending
with 1.0.
Then documentation shows a cat_thresh field but not a prob_thresh
field. So I am not sure where the three prob thresholds go or what
they refer to.
Thanks
Bob Craig
-----Original Message-----
From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
Sent: Friday, June 12, 2015 4:49 PM
To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
Subject: Re: [rt.rap.ucar.edu #72307] Using Different Grib Tables
Hi Bob.
I believe John may have left for the day. I thought I would try to
help out.
In previous versions of MET, the GRIB tables were hard-coded, but the
code is now reading them from a flat file at run time.
For METv4.1, that flat file can be found here:
METv4.1/data/table_files/nceptab_flat.txt
And in met-5.0, it's reading it from the installed met-5.0 directory
"share/met/table_files/nceptab_flat.txt".
If you'd like to change MET's behavior, you could edit that flat file
to use whichever abbreviations and units you'd like. The next time you
run the MET tools, it'll just read in the updated strings and use
them.
I hope that helps.
Julie
On Fri, Jun 12, 2015 at 3:23 PM, robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> Fri Jun 12 15:23:06 2015: Request 72307 was acted upon.
> Transaction: Ticket created by robert.craig.2 at us.af.mil
> Queue: met_help
> Subject: Using Different Grib Tables
> Owner: Nobody
> Requestors: robert.craig.2 at us.af.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=72307
> >
>
>
> Hi John, I am running point stat on some ensemble probability files.
> MET doesn't recognize the grib field name : PWS25 (probability of
> windspeed >
> 25) or the grib code AFWA defined for it: 200. How do I get met to
> use our local grib table or override the grib settings it using.
>
> Thanks
>
> Bob Craig
>
>
>
--
Julie Prestopnik
National Center for Atmospheric Research Research Applications
Laboratory
Phone: 303.497.8399
Email: jpresto at ucar.edu
------------------------------------------------
Subject: Using Different Grib Tables
From: John Halley Gotway
Time: Mon Jun 15 12:58:41 2015
Hello Bob,
This is John. Thought I'd jump in here on this question.
It looks like you're wondering how to specify thresholds for verifying
probability forecasts in MET. There is an example of verifying
probabilities included in the MET test scripts. Take a look at this
config
file for Grid-Stat:
met-5.0/scripts/config/GridStatConfig_POP_12
Notice that the "cat_thresh" setting in the "fcst" section begins at
0,
ends at 1, and all the thresholds are all of the same type (>=). In
this
example, they're spaced every 0.1. Verifying probabilities in MET is
done
by creating an Nx2 contingency table, where N is the number of
probability
bins you define. The "cat_thresh" setting in the "obs" section should
be a
single threshold which defines the event. For probability of winds >
25kt,
I'd set the obs cat_thresh as ">25".
Also, make sure that you've turned on the PCT, PSTD, PRC, and PJC
output
lines in the config file. Those output lines contain the counts from
the
probabilistic Nx2 table and the statistics which can be derived from
it.
Hopefully that helps get you going. Just let us know if more issues
or
questions arise.
Thanks,
John
On Mon, Jun 15, 2015 at 8:07 AM, robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=72307 >
>
> Thanks Julie, that got me past the grib errors. Now I need some
> assistance on properly configuring the configuration file for Point
Stat.
> I want to verify wind probability forecast of >25kts against
observations
> to calculate Brier Skill Scores. Below is an exert from the
configuration
> file:
>
> //
> // Forecast and observation fields to be verified
> //
> fcst = {
> wind_thresh = [ NA ];
> message_type = [ "ADPSFC" ];
>
> field = [
> {
> name = "PWS25";
> level = [ "L0" ];
> prob = TRUE;
> cat_thresh = [ >=12.9 ];
> prob_thresh = ["ge0","ge0.5","ge1"]; ???
> }
>
>
>
> ];
>
> };
>
> obs = {
> wind_thresh = [ NA ];
> message_type = [ "ADPSFC" ];
>
> field = [
> {
> name = "WIND";
> level = [ "L0" ];
> cat_thresh = [>=12.9];
> }
>
> When I run this I get the following error:
>
> DEBUG 1: Default Config File:
> /home/qcteam/METv4.1/data/config/PointStatConfig_default
> DEBUG 1: User Config File: data/config/PointStatConfig_geps
> ERROR :
> ERROR : check_prob_thresh() -> When verifying a probability field,
you
> must select at least 3 thresholds beginning with 0.0 and ending with
1.0.
>
>
> Then documentation shows a cat_thresh field but not a prob_thresh
field.
> So I am not sure where the three prob thresholds go or what they
refer to.
>
> Thanks
>
> Bob Craig
>
>
> -----Original Message-----
> From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> Sent: Friday, June 12, 2015 4:49 PM
> To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
> Subject: Re: [rt.rap.ucar.edu #72307] Using Different Grib Tables
>
> Hi Bob.
>
> I believe John may have left for the day. I thought I would try to
help
> out.
>
> In previous versions of MET, the GRIB tables were hard-coded, but
the code
> is now reading them from a flat file at run time.
>
> For METv4.1, that flat file can be found here:
>
> METv4.1/data/table_files/nceptab_flat.txt
>
> And in met-5.0, it's reading it from the installed met-5.0 directory
> "share/met/table_files/nceptab_flat.txt".
>
> If you'd like to change MET's behavior, you could edit that flat
file to
> use whichever abbreviations and units you'd like. The next time you
run the
> MET tools, it'll just read in the updated strings and use them.
>
> I hope that helps.
>
> Julie
>
> On Fri, Jun 12, 2015 at 3:23 PM, robert.craig.2 at us.af.mil via RT <
> met_help at ucar.edu> wrote:
>
> >
> > Fri Jun 12 15:23:06 2015: Request 72307 was acted upon.
> > Transaction: Ticket created by robert.craig.2 at us.af.mil
> > Queue: met_help
> > Subject: Using Different Grib Tables
> > Owner: Nobody
> > Requestors: robert.craig.2 at us.af.mil
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=72307
> > >
> >
> >
> > Hi John, I am running point stat on some ensemble probability
files.
> > MET doesn't recognize the grib field name : PWS25 (probability of
> > windspeed >
> > 25) or the grib code AFWA defined for it: 200. How do I get met
to
> > use our local grib table or override the grib settings it using.
> >
> > Thanks
> >
> > Bob Craig
> >
> >
> >
>
>
> --
> Julie Prestopnik
> National Center for Atmospheric Research Research Applications
Laboratory
> Phone: 303.497.8399
> Email: jpresto at ucar.edu
>
>
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #72307] Using Different Grib Tables
From: robert.craig.2 at us.af.mil
Time: Mon Jun 15 13:55:19 2015
John, thanks. I figured some of that out today. I noticed in your
response that for wind >25kts I should use >25. This would imply that
MET is converting the units of the observation from m/s to kts? I
want to make sure that is true.
On a more general level, it looks like MET doesn't provide the Brier
Skill Score directly (at least in version 4.1). According to Wilks,
I can calculate BSS by taking the difference of the Resolution and
Reliablity / uncertainty. Looking at my output for a one run case:
VERSION MODEL FCST_LEAD FCST_VALID_BEG FCST_VALID_END OBS_LEAD
OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_LEV OBS_VAR OBS_LEV
OBTYPE VX_MASK INTERP_MTHD INTERP_PNTS FCST_THRESH
OBS_THRESH COV_THRESH ALPHA LINE_TYPE TOTAL N_THRESH BASER
BASER_NCL BASER_NCU RELIABILITY RESOLUTION UNCERTAINTY ROC_AUC BRIER
BRIER_NCL BRIER_NCU THRESH_1 THRESH_2 THRESH_3 THRESH_4 THRESH_5
THRESH_6 THRESH_7 THRESH_8 THRESH_9 THRESH_10 THRESH_11 THRESH_12
V4.1 WRF 240000 20141115_000000 20141115_000000 000000
20141114_223000 20141115_013000 PSW15 Z10 WIND L0
ADPSFC FULL BILIN 4
>=0.000,>=0.050,>=0.150,>=0.250,>=0.350,>=0.450,>=0.550,>=0.650,>=0.750,>=0.850,>=0.950,>=1.000
>=15.000 NA 0.05000 PSTD 12660 12 0.00047 0.00022
0.00103 0.02405 0.00000 0.00047 0.92353 0.02452 nan
nan 0.00000 0.05000 0.15000 0.25000 0.35000 0.45000
0.55000 0.65000 0.75000 0.85000 0.95000 1.00000
The resolution is 0 for this case: sustained winds > 15kts. This
would mean a negative Brier Skill Score. However, the area under the
ROC curve indicates skill and I would suspect skill in this case. I
will be looking at more cases now that I have it working and see if I
can get some positive skill.
Bob Craig
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Monday, June 15, 2015 1:59 PM
To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
Subject: Re: [rt.rap.ucar.edu #72307] Using Different Grib Tables
Hello Bob,
This is John. Thought I'd jump in here on this question.
It looks like you're wondering how to specify thresholds for verifying
probability forecasts in MET. There is an example of verifying
probabilities included in the MET test scripts. Take a look at this
config file for Grid-Stat:
met-5.0/scripts/config/GridStatConfig_POP_12
Notice that the "cat_thresh" setting in the "fcst" section begins at
0, ends at 1, and all the thresholds are all of the same type (>=).
In this example, they're spaced every 0.1. Verifying probabilities in
MET is done by creating an Nx2 contingency table, where N is the
number of probability bins you define. The "cat_thresh" setting in
the "obs" section should be a single threshold which defines the
event. For probability of winds > 25kt, I'd set the obs cat_thresh as
">25".
Also, make sure that you've turned on the PCT, PSTD, PRC, and PJC
output lines in the config file. Those output lines contain the
counts from the probabilistic Nx2 table and the statistics which can
be derived from it.
Hopefully that helps get you going. Just let us know if more issues
or questions arise.
Thanks,
John
On Mon, Jun 15, 2015 at 8:07 AM, robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=72307 >
>
> Thanks Julie, that got me past the grib errors. Now I need some
> assistance on properly configuring the configuration file for Point
Stat.
> I want to verify wind probability forecast of >25kts against
> observations to calculate Brier Skill Scores. Below is an exert
from
> the configuration
> file:
>
> //
> // Forecast and observation fields to be verified // fcst = {
> wind_thresh = [ NA ];
> message_type = [ "ADPSFC" ];
>
> field = [
> {
> name = "PWS25";
> level = [ "L0" ];
> prob = TRUE;
> cat_thresh = [ >=12.9 ];
> prob_thresh = ["ge0","ge0.5","ge1"]; ???
> }
>
>
>
> ];
>
> };
>
> obs = {
> wind_thresh = [ NA ];
> message_type = [ "ADPSFC" ];
>
> field = [
> {
> name = "WIND";
> level = [ "L0" ];
> cat_thresh = [>=12.9];
> }
>
> When I run this I get the following error:
>
> DEBUG 1: Default Config File:
> /home/qcteam/METv4.1/data/config/PointStatConfig_default
> DEBUG 1: User Config File: data/config/PointStatConfig_geps ERROR :
> ERROR : check_prob_thresh() -> When verifying a probability field,
> you must select at least 3 thresholds beginning with 0.0 and ending
with 1.0.
>
>
> Then documentation shows a cat_thresh field but not a prob_thresh
field.
> So I am not sure where the three prob thresholds go or what they
refer to.
>
> Thanks
>
> Bob Craig
>
>
> -----Original Message-----
> From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> Sent: Friday, June 12, 2015 4:49 PM
> To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
> Subject: Re: [rt.rap.ucar.edu #72307] Using Different Grib Tables
>
> Hi Bob.
>
> I believe John may have left for the day. I thought I would try to
> help out.
>
> In previous versions of MET, the GRIB tables were hard-coded, but
the
> code is now reading them from a flat file at run time.
>
> For METv4.1, that flat file can be found here:
>
> METv4.1/data/table_files/nceptab_flat.txt
>
> And in met-5.0, it's reading it from the installed met-5.0 directory
> "share/met/table_files/nceptab_flat.txt".
>
> If you'd like to change MET's behavior, you could edit that flat
file
> to use whichever abbreviations and units you'd like. The next time
you
> run the MET tools, it'll just read in the updated strings and use
them.
>
> I hope that helps.
>
> Julie
>
> On Fri, Jun 12, 2015 at 3:23 PM, robert.craig.2 at us.af.mil via RT <
> met_help at ucar.edu> wrote:
>
> >
> > Fri Jun 12 15:23:06 2015: Request 72307 was acted upon.
> > Transaction: Ticket created by robert.craig.2 at us.af.mil
> > Queue: met_help
> > Subject: Using Different Grib Tables
> > Owner: Nobody
> > Requestors: robert.craig.2 at us.af.mil
> > Status: new
> > Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=72307
> > >
> >
> >
> > Hi John, I am running point stat on some ensemble probability
files.
> > MET doesn't recognize the grib field name : PWS25 (probability of
> > windspeed >
> > 25) or the grib code AFWA defined for it: 200. How do I get met
to
> > use our local grib table or override the grib settings it using.
> >
> > Thanks
> >
> > Bob Craig
> >
> >
> >
>
>
> --
> Julie Prestopnik
> National Center for Atmospheric Research Research Applications
> Laboratory
> Phone: 303.497.8399
> Email: jpresto at ucar.edu
>
>
>
>
------------------------------------------------
Subject: Using Different Grib Tables
From: John Halley Gotway
Time: Mon Jun 15 14:12:15 2015
Bob,
To clarify, no, MET is not doing any unit conversion. You need to
account
for the wind speed units yourself. If your observations are in m/s
(which
is the GRIB standard), you should define your thresholds in m/s. But
if
you have point observations that are given in kts, you should define
your
observation threshold in kts.
Regarding BSS, I think I should refer you to Tressa Fowler, our
resident
statistician. Hopefully, she can clarify.
Thanks,
John
On Mon, Jun 15, 2015 at 1:55 PM, robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=72307 >
>
> John, thanks. I figured some of that out today. I noticed in your
> response that for wind >25kts I should use >25. This would imply
that MET
> is converting the units of the observation from m/s to kts? I want
to make
> sure that is true.
>
> On a more general level, it looks like MET doesn't provide the Brier
Skill
> Score directly (at least in version 4.1). According to Wilks, I
can
> calculate BSS by taking the difference of the Resolution and
Reliablity /
> uncertainty. Looking at my output for a one run case:
>
> VERSION MODEL FCST_LEAD FCST_VALID_BEG FCST_VALID_END OBS_LEAD
> OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_LEV OBS_VAR OBS_LEV
OBTYPE
> VX_MASK INTERP_MTHD INTERP_PNTS FCST_THRESH
> OBS_THRESH
COV_THRESH
> ALPHA LINE_TYPE TOTAL N_THRESH BASER BASER_NCL BASER_NCU
RELIABILITY
> RESOLUTION UNCERTAINTY ROC_AUC BRIER BRIER_NCL BRIER_NCU THRESH_1
> THRESH_2 THRESH_3 THRESH_4 THRESH_5 THRESH_6 THRESH_7 THRESH_8
THRESH_9
> THRESH_10 THRESH_11 THRESH_12
> V4.1 WRF 240000 20141115_000000 20141115_000000 000000
> 20141114_223000 20141115_013000 PSW15 Z10 WIND L0
ADPSFC
> FULL BILIN 4
>
>=0.000,>=0.050,>=0.150,>=0.250,>=0.350,>=0.450,>=0.550,>=0.650,>=0.750,>=0.850,>=0.950,>=1.000
> >=15.000 NA 0.05000 PSTD 12660 12 0.00047
0.00022
> 0.00103 0.02405 0.00000 0.00047 0.92353 0.02452 nan
> nan 0.00000 0.05000 0.15000 0.25000 0.35000 0.45000
0.55000
> 0.65000 0.75000 0.85000 0.95000 1.00000
>
> The resolution is 0 for this case: sustained winds > 15kts. This
would
> mean a negative Brier Skill Score. However, the area under the ROC
curve
> indicates skill and I would suspect skill in this case. I will be
looking
> at more cases now that I have it working and see if I can get some
positive
> skill.
>
> Bob Craig
>
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Monday, June 15, 2015 1:59 PM
> To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
> Subject: Re: [rt.rap.ucar.edu #72307] Using Different Grib Tables
>
> Hello Bob,
>
> This is John. Thought I'd jump in here on this question.
>
> It looks like you're wondering how to specify thresholds for
verifying
> probability forecasts in MET. There is an example of verifying
> probabilities included in the MET test scripts. Take a look at this
config
> file for Grid-Stat:
> met-5.0/scripts/config/GridStatConfig_POP_12
>
> Notice that the "cat_thresh" setting in the "fcst" section begins at
0,
> ends at 1, and all the thresholds are all of the same type (>=). In
this
> example, they're spaced every 0.1. Verifying probabilities in MET
is done
> by creating an Nx2 contingency table, where N is the number of
probability
> bins you define. The "cat_thresh" setting in the "obs" section
should be a
> single threshold which defines the event. For probability of winds
> 25kt,
> I'd set the obs cat_thresh as ">25".
>
> Also, make sure that you've turned on the PCT, PSTD, PRC, and PJC
output
> lines in the config file. Those output lines contain the counts
from the
> probabilistic Nx2 table and the statistics which can be derived from
it.
>
> Hopefully that helps get you going. Just let us know if more issues
or
> questions arise.
>
> Thanks,
> John
>
>
>
> On Mon, Jun 15, 2015 at 8:07 AM, robert.craig.2 at us.af.mil via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=72307 >
> >
> > Thanks Julie, that got me past the grib errors. Now I need some
> > assistance on properly configuring the configuration file for
Point Stat.
> > I want to verify wind probability forecast of >25kts against
> > observations to calculate Brier Skill Scores. Below is an exert
from
> > the configuration
> > file:
> >
> > //
> > // Forecast and observation fields to be verified // fcst = {
> > wind_thresh = [ NA ];
> > message_type = [ "ADPSFC" ];
> >
> > field = [
> > {
> > name = "PWS25";
> > level = [ "L0" ];
> > prob = TRUE;
> > cat_thresh = [ >=12.9 ];
> > prob_thresh = ["ge0","ge0.5","ge1"]; ???
> > }
> >
> >
> >
> > ];
> >
> > };
> >
> > obs = {
> > wind_thresh = [ NA ];
> > message_type = [ "ADPSFC" ];
> >
> > field = [
> > {
> > name = "WIND";
> > level = [ "L0" ];
> > cat_thresh = [>=12.9];
> > }
> >
> > When I run this I get the following error:
> >
> > DEBUG 1: Default Config File:
> > /home/qcteam/METv4.1/data/config/PointStatConfig_default
> > DEBUG 1: User Config File: data/config/PointStatConfig_geps ERROR
:
> > ERROR : check_prob_thresh() -> When verifying a probability
field,
> > you must select at least 3 thresholds beginning with 0.0 and
ending with
> 1.0.
> >
> >
> > Then documentation shows a cat_thresh field but not a prob_thresh
field.
> > So I am not sure where the three prob thresholds go or what they
refer
> to.
> >
> > Thanks
> >
> > Bob Craig
> >
> >
> > -----Original Message-----
> > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > Sent: Friday, June 12, 2015 4:49 PM
> > To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
> > Subject: Re: [rt.rap.ucar.edu #72307] Using Different Grib Tables
> >
> > Hi Bob.
> >
> > I believe John may have left for the day. I thought I would try
to
> > help out.
> >
> > In previous versions of MET, the GRIB tables were hard-coded, but
the
> > code is now reading them from a flat file at run time.
> >
> > For METv4.1, that flat file can be found here:
> >
> > METv4.1/data/table_files/nceptab_flat.txt
> >
> > And in met-5.0, it's reading it from the installed met-5.0
directory
> > "share/met/table_files/nceptab_flat.txt".
> >
> > If you'd like to change MET's behavior, you could edit that flat
file
> > to use whichever abbreviations and units you'd like. The next time
you
> > run the MET tools, it'll just read in the updated strings and use
them.
> >
> > I hope that helps.
> >
> > Julie
> >
> > On Fri, Jun 12, 2015 at 3:23 PM, robert.craig.2 at us.af.mil via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > Fri Jun 12 15:23:06 2015: Request 72307 was acted upon.
> > > Transaction: Ticket created by robert.craig.2 at us.af.mil
> > > Queue: met_help
> > > Subject: Using Different Grib Tables
> > > Owner: Nobody
> > > Requestors: robert.craig.2 at us.af.mil
> > > Status: new
> > > Ticket <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=72307
> > > >
> > >
> > >
> > > Hi John, I am running point stat on some ensemble probability
files.
> > > MET doesn't recognize the grib field name : PWS25 (probability
of
> > > windspeed >
> > > 25) or the grib code AFWA defined for it: 200. How do I get
met to
> > > use our local grib table or override the grib settings it using.
> > >
> > > Thanks
> > >
> > > Bob Craig
> > >
> > >
> > >
> >
> >
> > --
> > Julie Prestopnik
> > National Center for Atmospheric Research Research Applications
> > Laboratory
> > Phone: 303.497.8399
> > Email: jpresto at ucar.edu
> >
> >
> >
> >
>
>
>
>
------------------------------------------------
Subject: Using Different Grib Tables
From: Tressa Fowler
Time: Wed Jun 17 09:47:29 2015
Hi Bob,
Regarding the BSS, the very small denominator makes this an unstable
calculation for this one case. Perhaps this is a relatively rare
event? It's a well known problem with BSS. When you get a larger
collection of cases, see if the results make more sense. If not, let
us know.
Thanks,
Tressa
On Fri Jun 12 15:23:06 2015, robert.craig.2 at us.af.mil wrote:
> Hi John, I am running point stat on some ensemble probability files.
> MET doesn't recognize the grib field name : PWS25 (probability of
> windspeed > 25) or the grib code AFWA defined for it: 200. How
do
> I get met to use our local grib table or override the grib
settings
> it using.
>
> Thanks
>
> Bob Craig
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #72307] Using Different Grib Tables
From: robert.craig.2 at us.af.mil
Time: Wed Jun 17 10:06:58 2015
Thanks Tresa, I agree that one case is not enough. I ran some more
cases to research this. In the Prep_buffer obs I was using, I was
retrieving "wind" and not "gust" - I have added "gust" to the
retrieval (at least I think I have but don't know how to verify that
the netCDF file contains it). So in the case of evaluating
probability forecasts >25kts, I wasn't getting the highest winds
available so that would drive down the skill. Looking in the MET
MPR.txt files produced by point_stat, I noticed that anyone
particular station could have multiple entries (specials etc) for a
particular range of time (one hour period around the ob time). This
looks like it would be a problem for verifying probs of winds > say
25kts since each report from a station in that one hour period that
didn't meet the wind criteria would be a hit on the skill. If I had a
particular station reporting winds of 26, 12,10,15 kts during that one
hour period, the 26kts would contribute to skill but the others would
reduce skill. Is there a way to tell MET to just use the highest wind
ob during the time period of interest?
Thanks
Bob Craig
According to the Book of Wilks, I should be able to divide the brier
score by the uncertainty term to get Brier Skill Score, so that is
what I will do unless you have a better method.
-----Original Message-----
From: Tressa Fowler via RT [mailto:met_help at ucar.edu]
Sent: Wednesday, June 17, 2015 10:48 AM
To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
Subject: [rt.rap.ucar.edu #72307] Using Different Grib Tables
Hi Bob,
Regarding the BSS, the very small denominator makes this an unstable
calculation for this one case. Perhaps this is a relatively rare
event? It's a well known problem with BSS. When you get a larger
collection of cases, see if the results make more sense. If not, let
us know.
Thanks,
Tressa
On Fri Jun 12 15:23:06 2015, robert.craig.2 at us.af.mil wrote:
> Hi John, I am running point stat on some ensemble probability files.
> MET doesn't recognize the grib field name : PWS25 (probability of
> windspeed > 25) or the grib code AFWA defined for it: 200. How
do
> I get met to use our local grib table or override the grib
settings
> it using.
>
> Thanks
>
> Bob Craig
>
------------------------------------------------
Subject: Using Different Grib Tables
From: John Halley Gotway
Time: Thu Jun 18 12:45:25 2015
Bob,
Tressa forwarded this question to me. Since you're using PREPBUFR
observations, I believe the answer is no.
We do have some logic in the ascii2nc tool that we added for handling
SURFRAD data. Since the SURFRAD data is so dense in time, we added
logic
to define a moving time window and compute a summary of all the data
falling in that window. The available summary metrics include the
maximum
value. So ascii2nc can compute an hourly maximum wind speed for each
station but PB2NC does not.
In the point-stat tool itself, we also have some logic that's close to
what
you need but not exactly right. It's controlled by the
"duplicate_flag"
config file setting for point-stat. Listed at the bottom is an
excerpt
from the configuration README file about it. Setting it to SINGLE
means
that when multiple observations for the same station fall within the
time
window, use the one whose valid time is closest to the forecast value
time.
If that option were to support MIN, MAX, MEAN, MEDIAN, and so on, I
think
your problem would be solved. You'd just pick MAX and the maximum
wind
speed value in that time window would be compared to your forecast.
Do you agree? Would that solve your problem?
Thanks,
John
//
// The "duplicate_flag" entry specifies how to handle duplicate point
// observations in Point-Stat and Ensemble-Stat. Three techniques are
// currently supported:
//
// - "NONE" to use all point observations (legacy behavior)
// - "UNIQUE" if two or more observations match identically except
in the
// station ID field, only use a single observation
// - "SINGLE" if two or more observations appear at a single
location
// (lat, lon, level, elv), use only the observation that has the
valid
// time closest to the forecast valid time
//
// The reporting mechanism for this feature can be activated by
specifying
// a verbosity level of three or higher. The report will show
information
// about where duplicates were detected and which observations were
used
// in those cases.
//
On Wed, Jun 17, 2015 at 10:06 AM, robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=72307 >
>
>
> Thanks Tresa, I agree that one case is not enough. I ran some more
cases
> to research this. In the Prep_buffer obs I was using, I was
retrieving
> "wind" and not "gust" - I have added "gust" to the retrieval (at
least I
> think I have but don't know how to verify that the netCDF file
contains
> it). So in the case of evaluating probability forecasts >25kts, I
wasn't
> getting the highest winds available so that would drive down the
skill.
> Looking in the MET MPR.txt files produced by point_stat, I noticed
that
> anyone particular station could have multiple entries (specials
etc) for a
> particular range of time (one hour period around the ob time). This
looks
> like it would be a problem for verifying probs of winds > say 25kts
since
> each report from a station in that one hour period that didn't meet
the
> wind criteria would be a hit on the skill. If I had a particular
station
> reporting winds of 26, 12,10,15 kts during that one hour period, the
26kts
> would contribute to skill bu!
> t the others would reduce skill. Is there a way to tell MET to
just use
> the highest wind ob during the time period of interest?
>
> Thanks
>
> Bob Craig
>
>
> According to the Book of Wilks, I should be able to divide the brier
score
> by the uncertainty term to get Brier Skill Score, so that is what I
will do
> unless you have a better method.
> -----Original Message-----
> From: Tressa Fowler via RT [mailto:met_help at ucar.edu]
> Sent: Wednesday, June 17, 2015 10:48 AM
> To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
> Subject: [rt.rap.ucar.edu #72307] Using Different Grib Tables
>
> Hi Bob,
>
> Regarding the BSS, the very small denominator makes this an unstable
> calculation for this one case. Perhaps this is a relatively rare
event?
> It's a well known problem with BSS. When you get a larger collection
of
> cases, see if the results make more sense. If not, let us know.
>
> Thanks,
>
> Tressa
>
>
> On Fri Jun 12 15:23:06 2015, robert.craig.2 at us.af.mil wrote:
> > Hi John, I am running point stat on some ensemble probability
files.
> > MET doesn't recognize the grib field name : PWS25 (probability
of
> > windspeed > 25) or the grib code AFWA defined for it: 200.
How do
> > I get met to use our local grib table or override the grib
settings
> > it using.
> >
> > Thanks
> >
> > Bob Craig
> >
>
>
>
>
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #72307] Using Different Grib Tables
From: robert.craig.2 at us.af.mil
Time: Thu Jun 18 12:57:02 2015
Yes that would help. I wish I had some money to send you. We have
been using our home grown verification code for ensemble forecasts (it
predates MET), written by Matt Sittel, up until now. Matt, due to
contract changes, is no longer able to enhance or maintain his code,
so we thought it might be a good time to switch to MET for this
verification. That why all the questions.
Thanks
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Thursday, June 18, 2015 1:45 PM
To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
Cc: tressa at ucar.edu
Subject: Re: [rt.rap.ucar.edu #72307] Using Different Grib Tables
Bob,
Tressa forwarded this question to me. Since you're using PREPBUFR
observations, I believe the answer is no.
We do have some logic in the ascii2nc tool that we added for handling
SURFRAD data. Since the SURFRAD data is so dense in time, we added
logic to define a moving time window and compute a summary of all the
data falling in that window. The available summary metrics include
the maximum value. So ascii2nc can compute an hourly maximum wind
speed for each station but PB2NC does not.
In the point-stat tool itself, we also have some logic that's close to
what you need but not exactly right. It's controlled by the
"duplicate_flag"
config file setting for point-stat. Listed at the bottom is an
excerpt from the configuration README file about it. Setting it to
SINGLE means that when multiple observations for the same station fall
within the time window, use the one whose valid time is closest to the
forecast value time.
If that option were to support MIN, MAX, MEAN, MEDIAN, and so on, I
think your problem would be solved. You'd just pick MAX and the
maximum wind speed value in that time window would be compared to your
forecast.
Do you agree? Would that solve your problem?
Thanks,
John
//
// The "duplicate_flag" entry specifies how to handle duplicate point
// observations in Point-Stat and Ensemble-Stat. Three techniques are
// currently supported:
//
// - "NONE" to use all point observations (legacy behavior)
// - "UNIQUE" if two or more observations match identically except
in the
// station ID field, only use a single observation
// - "SINGLE" if two or more observations appear at a single
location
// (lat, lon, level, elv), use only the observation that has the
valid
// time closest to the forecast valid time
//
// The reporting mechanism for this feature can be activated by
specifying // a verbosity level of three or higher. The report will
show information // about where duplicates were detected and which
observations were used // in those cases.
//
On Wed, Jun 17, 2015 at 10:06 AM, robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=72307 >
>
>
> Thanks Tresa, I agree that one case is not enough. I ran some more
> cases to research this. In the Prep_buffer obs I was using, I was
> retrieving "wind" and not "gust" - I have added "gust" to the
> retrieval (at least I think I have but don't know how to verify that
the netCDF file contains
> it). So in the case of evaluating probability forecasts >25kts, I
wasn't
> getting the highest winds available so that would drive down the
skill.
> Looking in the MET MPR.txt files produced by point_stat, I noticed
> that anyone particular station could have multiple entries
(specials
> etc) for a particular range of time (one hour period around the ob
> time). This looks like it would be a problem for verifying probs of
> winds > say 25kts since each report from a station in that one hour
> period that didn't meet the wind criteria would be a hit on the
skill.
> If I had a particular station reporting winds of 26, 12,10,15 kts
> during that one hour period, the 26kts would contribute to skill bu!
> t the others would reduce skill. Is there a way to tell MET to
just
> use the highest wind ob during the time period of interest?
>
> Thanks
>
> Bob Craig
>
>
> According to the Book of Wilks, I should be able to divide the brier
> score by the uncertainty term to get Brier Skill Score, so that is
> what I will do unless you have a better method.
> -----Original Message-----
> From: Tressa Fowler via RT [mailto:met_help at ucar.edu]
> Sent: Wednesday, June 17, 2015 10:48 AM
> To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
> Subject: [rt.rap.ucar.edu #72307] Using Different Grib Tables
>
> Hi Bob,
>
> Regarding the BSS, the very small denominator makes this an unstable
> calculation for this one case. Perhaps this is a relatively rare
event?
> It's a well known problem with BSS. When you get a larger collection
> of cases, see if the results make more sense. If not, let us know.
>
> Thanks,
>
> Tressa
>
>
> On Fri Jun 12 15:23:06 2015, robert.craig.2 at us.af.mil wrote:
> > Hi John, I am running point stat on some ensemble probability
files.
> > MET doesn't recognize the grib field name : PWS25 (probability
of
> > windspeed > 25) or the grib code AFWA defined for it: 200.
How do
> > I get met to use our local grib table or override the grib
settings
> > it using.
> >
> > Thanks
> >
> > Bob Craig
> >
>
>
>
>
>
>
------------------------------------------------
More information about the Met_help
mailing list