[Met_help] [rt.rap.ucar.edu #95790] History for MET V9.0 Point-Stat error (UNCLASSIFIED)
John Halley Gotway via RT
met_help at ucar.edu
Wed Jul 8 21:02:52 MDT 2020
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
CLASSIFICATION: UNCLASSIFIED
I keep getting the same error when trying to run Point-Stat with the HiRA
verification logic enabled. The error involves the probability field
thresholds:
ERROR:check_prob_thresh() -> When verifying a probability field, you must
select at least 3 thresholds beginning with 0.0 and ending with 1.0.
During troubleshooting, I've changed the settings for prob_cat_thresh in the
attached config file several times thinking that the formatting of my
settings was the issue. Maybe I'm on the wrong track. Could you take a look
at the config file to check the format or other possible issue which might
be causing this issue?
Thanks.
R/
John
Mr. John W. Raby
U.S. Army Research Laboratory
White Sands Missile Range, NM 88002
Office:(575) 678-2004 DSN 258-2004
Teleworking: Contact Admin Specialist at (410) 278-6974
FAX (575) 678-1230 DSN 258-1230
Email: john.w.raby2.civ at mail.mil
CLASSIFICATION: UNCLASSIFIED
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: MET V9.0 Point-Stat error (UNCLASSIFIED)
From: John Halley Gotway
Time: Tue Jun 30 17:11:05 2020
John,
Please try using this shortcut:
prob_cat_thresh = [ ==0.25 ];
Does that do it?
Also, your "desc" setting can't contain embedded whitespace. And
Point-Stat
should error out and tell you that.
John
On Tue, Jun 30, 2020 at 4:57 PM Raby, John W USA CIV via RT <
met_help at ucar.edu> wrote:
>
> Tue Jun 30 16:56:45 2020: Request 95790 was acted upon.
> Transaction: Ticket created by john.w.raby2.civ at mail.mil
> Queue: met_help
> Subject: MET V9.0 Point-Stat error (UNCLASSIFIED)
> Owner: Nobody
> Requestors: john.w.raby2.civ at mail.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
>
>
> CLASSIFICATION: UNCLASSIFIED
>
> I keep getting the same error when trying to run Point-Stat with the
HiRA
> verification logic enabled. The error involves the probability field
> thresholds:
>
> ERROR:check_prob_thresh() -> When verifying a probability field, you
must
> select at least 3 thresholds beginning with 0.0 and ending with 1.0.
>
> During troubleshooting, I've changed the settings for
prob_cat_thresh in
> the
> attached config file several times thinking that the formatting of
my
> settings was the issue. Maybe I'm on the wrong track. Could you take
a look
> at the config file to check the format or other possible issue which
might
> be causing this issue?
>
> Thanks.
>
> R/
> John
>
> Mr. John W. Raby
> U.S. Army Research Laboratory
> White Sands Missile Range, NM 88002
> Office:(575) 678-2004 DSN 258-2004
> Teleworking: Contact Admin Specialist at (410) 278-6974
> FAX (575) 678-1230 DSN 258-1230
> Email: john.w.raby2.civ at mail.mil
>
> CLASSIFICATION: UNCLASSIFIED
>
>
------------------------------------------------
Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0 Point-Stat error (UNCLASSIFIED)
From: Raby, John W USA CIV
Time: Tue Jun 30 18:11:01 2020
Thanks for taking a look, John. I'll give that a try. Appreciate the
heads-up on a future error in the "desc".
R/
John
________________________________________
From: John Halley Gotway via RT [met_help at ucar.edu]
Sent: Tuesday, June 30, 2020 5:11 PM
To: Raby, John W CIV USARMY CCDC ARL (USA)
Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0 Point-
Stat error (UNCLASSIFIED)
All active links contained in this email were disabled. Please verify
the identity of the sender, and confirm the authenticity of all links
contained within the message prior to copying and pasting the address
to a Web browser.
----
John,
Please try using this shortcut:
prob_cat_thresh = [ ==0.25 ];
Does that do it?
Also, your "desc" setting can't contain embedded whitespace. And
Point-Stat
should error out and tell you that.
John
On Tue, Jun 30, 2020 at 4:57 PM Raby, John W USA CIV via RT <
met_help at ucar.edu> wrote:
>
> Tue Jun 30 16:56:45 2020: Request 95790 was acted upon.
> Transaction: Ticket created by john.w.raby2.civ at mail.mil
> Queue: met_help
> Subject: MET V9.0 Point-Stat error (UNCLASSIFIED)
> Owner: Nobody
> Requestors: john.w.raby2.civ at mail.mil
> Status: new
> Ticket <Caution-url: Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
>
>
> CLASSIFICATION: UNCLASSIFIED
>
> I keep getting the same error when trying to run Point-Stat with the
HiRA
> verification logic enabled. The error involves the probability field
> thresholds:
>
> ERROR:check_prob_thresh() -> When verifying a probability field, you
must
> select at least 3 thresholds beginning with 0.0 and ending with 1.0.
>
> During troubleshooting, I've changed the settings for
prob_cat_thresh in
> the
> attached config file several times thinking that the formatting of
my
> settings was the issue. Maybe I'm on the wrong track. Could you take
a look
> at the config file to check the format or other possible issue which
might
> be causing this issue?
>
> Thanks.
>
> R/
> John
>
> Mr. John W. Raby
> U.S. Army Research Laboratory
> White Sands Missile Range, NM 88002
> Office:(575) 678-2004 DSN 258-2004
> Teleworking: Contact Admin Specialist at (410) 278-6974
> FAX (575) 678-1230 DSN 258-1230
> Email: john.w.raby2.civ at mail.mil
>
> CLASSIFICATION: UNCLASSIFIED
>
>
------------------------------------------------
Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0 Point-Stat error (UNCLASSIFIED)
From: Raby, John W USA CIV
Time: Tue Jun 30 19:00:30 2020
I edited the config file to use the shortcut and corrected the
whitespace issue, but I get the same error.
R/
John
________________________________________
From: Raby, John W CIV USARMY CCDC ARL (USA)
Sent: Tuesday, June 30, 2020 6:10 PM
To: met_help at ucar.edu
Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
Point-Stat error (UNCLASSIFIED)
Thanks for taking a look, John. I'll give that a try. Appreciate the
heads-up on a future error in the "desc".
R/
John
________________________________________
From: John Halley Gotway via RT [met_help at ucar.edu]
Sent: Tuesday, June 30, 2020 5:11 PM
To: Raby, John W CIV USARMY CCDC ARL (USA)
Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0 Point-
Stat error (UNCLASSIFIED)
All active links contained in this email were disabled. Please verify
the identity of the sender, and confirm the authenticity of all links
contained within the message prior to copying and pasting the address
to a Web browser.
----
John,
Please try using this shortcut:
prob_cat_thresh = [ ==0.25 ];
Does that do it?
Also, your "desc" setting can't contain embedded whitespace. And
Point-Stat
should error out and tell you that.
John
On Tue, Jun 30, 2020 at 4:57 PM Raby, John W USA CIV via RT <
met_help at ucar.edu> wrote:
>
> Tue Jun 30 16:56:45 2020: Request 95790 was acted upon.
> Transaction: Ticket created by john.w.raby2.civ at mail.mil
> Queue: met_help
> Subject: MET V9.0 Point-Stat error (UNCLASSIFIED)
> Owner: Nobody
> Requestors: john.w.raby2.civ at mail.mil
> Status: new
> Ticket <Caution-url: Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
>
>
> CLASSIFICATION: UNCLASSIFIED
>
> I keep getting the same error when trying to run Point-Stat with the
HiRA
> verification logic enabled. The error involves the probability field
> thresholds:
>
> ERROR:check_prob_thresh() -> When verifying a probability field, you
must
> select at least 3 thresholds beginning with 0.0 and ending with 1.0.
>
> During troubleshooting, I've changed the settings for
prob_cat_thresh in
> the
> attached config file several times thinking that the formatting of
my
> settings was the issue. Maybe I'm on the wrong track. Could you take
a look
> at the config file to check the format or other possible issue which
might
> be causing this issue?
>
> Thanks.
>
> R/
> John
>
> Mr. John W. Raby
> U.S. Army Research Laboratory
> White Sands Missile Range, NM 88002
> Office:(575) 678-2004 DSN 258-2004
> Teleworking: Contact Admin Specialist at (410) 278-6974
> FAX (575) 678-1230 DSN 258-1230
> Email: john.w.raby2.civ at mail.mil
>
> CLASSIFICATION: UNCLASSIFIED
>
>
------------------------------------------------
Subject: MET V9.0 Point-Stat error (UNCLASSIFIED)
From: Raby, John W USA CIV
Time: Tue Jun 30 19:12:00 2020
I forgot to attach the log file.
________________________________________
From: Raby, John W CIV USARMY CCDC ARL (USA)
Sent: Tuesday, June 30, 2020 7:00 PM
To: met_help at ucar.edu
Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
Point-Stat error (UNCLASSIFIED)
I edited the config file to use the shortcut and corrected the
whitespace issue, but I get the same error.
R/
John
________________________________________
From: Raby, John W CIV USARMY CCDC ARL (USA)
Sent: Tuesday, June 30, 2020 6:10 PM
To: met_help at ucar.edu
Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
Point-Stat error (UNCLASSIFIED)
Thanks for taking a look, John. I'll give that a try. Appreciate the
heads-up on a future error in the "desc".
R/
John
________________________________________
From: John Halley Gotway via RT [met_help at ucar.edu]
Sent: Tuesday, June 30, 2020 5:11 PM
To: Raby, John W CIV USARMY CCDC ARL (USA)
Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0 Point-
Stat error (UNCLASSIFIED)
All active links contained in this email were disabled. Please verify
the identity of the sender, and confirm the authenticity of all links
contained within the message prior to copying and pasting the address
to a Web browser.
----
John,
Please try using this shortcut:
prob_cat_thresh = [ ==0.25 ];
Does that do it?
Also, your "desc" setting can't contain embedded whitespace. And
Point-Stat
should error out and tell you that.
John
On Tue, Jun 30, 2020 at 4:57 PM Raby, John W USA CIV via RT <
met_help at ucar.edu> wrote:
>
> Tue Jun 30 16:56:45 2020: Request 95790 was acted upon.
> Transaction: Ticket created by john.w.raby2.civ at mail.mil
> Queue: met_help
> Subject: MET V9.0 Point-Stat error (UNCLASSIFIED)
> Owner: Nobody
> Requestors: john.w.raby2.civ at mail.mil
> Status: new
> Ticket <Caution-url: Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
>
>
> CLASSIFICATION: UNCLASSIFIED
>
> I keep getting the same error when trying to run Point-Stat with the
HiRA
> verification logic enabled. The error involves the probability field
> thresholds:
>
> ERROR:check_prob_thresh() -> When verifying a probability field, you
must
> select at least 3 thresholds beginning with 0.0 and ending with 1.0.
>
> During troubleshooting, I've changed the settings for
prob_cat_thresh in
> the
> attached config file several times thinking that the formatting of
my
> settings was the issue. Maybe I'm on the wrong track. Could you take
a look
> at the config file to check the format or other possible issue which
might
> be causing this issue?
>
> Thanks.
>
> R/
> John
>
> Mr. John W. Raby
> U.S. Army Research Laboratory
> White Sands Missile Range, NM 88002
> Office:(575) 678-2004 DSN 258-2004
> Teleworking: Contact Admin Specialist at (410) 278-6974
> FAX (575) 678-1230 DSN 258-1230
> Email: john.w.raby2.civ at mail.mil
>
> CLASSIFICATION: UNCLASSIFIED
>
>
------------------------------------------------
Subject: MET V9.0 Point-Stat error (UNCLASSIFIED)
From: John Halley Gotway
Time: Wed Jul 01 11:17:15 2020
John,
Sorry for the confusion. I led you astray. The HiRA configuration
option
names are not entirely intuitive.
Please take a look at the README file starting on line 1418:
https://github.com/NCAR/MET/blob/0cee9fbd0b3c691fd3cf4dd1d665f5d2b0db9a37/met/data/config/README#L1418
I modified your hira entry like this:
hira = {
flag = TRUE; *// set to TRUE/false to turn on/off
this
logic*
width = [ 3, 5, 7, 9, 11 ]; *// defines the
neighborhood
size and must be > 1*
vld_thresh = 1.0;
cov_thresh = [ ==0.25 ]; *// this defines the probability bins
to be
used. You only get to pick one setting here (==0.25 is short for
>=0,>=0.25,>=0.5,>=0.75,>=1.0)*
shape = SQUARE;
prob_cat_thresh = [ >290, >300 ];* // this defines the threshold(s)
for
which HiRA logic should be applied*
}
So you just had things a bit mis-configured. The key thing to note
here is
that the hira.prob_cat_thresh setting defines the thresholds for which
HiRA
probabilities should be computed. This is different from the
cat_thresh
setting in the fcst and obs dictionaries. Those are used to compute
categorical counts and statistics and are currently separate.
It's nice that you don't have to use the same set of thresholds for
categorical stats and the HiRA logic... it's more configurable that
way.
However, I do see that it makes things a bit inconvenient. Since
you're
applying HiRA to many variables and each of those variables has
different
meaningful thresholds, you'd need to configure the HiRA dictionary
separately for each and every variable.
That's what is meant by this comment... that the hira dictionary can
be
specified separately for each variable:
*// HiRA verification method// May be set separately in each
"obs.field"
entry*
So you'd need to copy/paste the HiRA dictionary into every single
obs.field
dictionary entry and modify the prob_cat_thresh thresholds
accordingly.
First, please make sure that you can in fact produce HiRA output.
Second, I
can think of a tweak to the logic that might make using HiRA easier.
How
about, if the hira.prob_cat_thresh array is an empty list, we could
have
the code check to see if cat_thresh in the obs dictionary is non-
empty. And
if that's non-empty then we'd simply set prob_cat_thresh = cat_thresh.
I imagine that's what you were expecting in the first place... that
Point-Stat would just be using the thresholds from cat_thresh to
define the
HiRA probability thresholds. It still is nice to have prob_cat_thresh
be
configurable separately, but if left unset, defaulting to cat_thresh
seems
pretty reasonable to me. Do you think that would be a worthwhile
change?
Thanks,
John
On Tue, Jun 30, 2020 at 7:12 PM Raby, John W USA CIV via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
>
> I forgot to attach the log file.
>
> ________________________________________
> From: Raby, John W CIV USARMY CCDC ARL (USA)
> Sent: Tuesday, June 30, 2020 7:00 PM
> To: met_help at ucar.edu
> Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
> Point-Stat error (UNCLASSIFIED)
>
> I edited the config file to use the shortcut and corrected the
whitespace
> issue, but I get the same error.
> R/
> John
>
>
>
> ________________________________________
> From: Raby, John W CIV USARMY CCDC ARL (USA)
> Sent: Tuesday, June 30, 2020 6:10 PM
> To: met_help at ucar.edu
> Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
> Point-Stat error (UNCLASSIFIED)
>
> Thanks for taking a look, John. I'll give that a try. Appreciate the
> heads-up on a future error in the "desc".
>
> R/
> John
>
> ________________________________________
> From: John Halley Gotway via RT [met_help at ucar.edu]
> Sent: Tuesday, June 30, 2020 5:11 PM
> To: Raby, John W CIV USARMY CCDC ARL (USA)
> Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
> Point-Stat error (UNCLASSIFIED)
>
> All active links contained in this email were disabled. Please
verify the
> identity of the sender, and confirm the authenticity of all links
contained
> within the message prior to copying and pasting the address to a Web
> browser.
>
>
>
>
> ----
>
> John,
>
> Please try using this shortcut:
> prob_cat_thresh = [ ==0.25 ];
>
> Does that do it?
>
> Also, your "desc" setting can't contain embedded whitespace. And
Point-Stat
> should error out and tell you that.
>
> John
>
> On Tue, Jun 30, 2020 at 4:57 PM Raby, John W USA CIV via RT <
> met_help at ucar.edu> wrote:
>
> >
> > Tue Jun 30 16:56:45 2020: Request 95790 was acted upon.
> > Transaction: Ticket created by john.w.raby2.civ at mail.mil
> > Queue: met_help
> > Subject: MET V9.0 Point-Stat error (UNCLASSIFIED)
> > Owner: Nobody
> > Requestors: john.w.raby2.civ at mail.mil
> > Status: new
> > Ticket <Caution-url: Caution-
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> >
> >
> > CLASSIFICATION: UNCLASSIFIED
> >
> > I keep getting the same error when trying to run Point-Stat with
the HiRA
> > verification logic enabled. The error involves the probability
field
> > thresholds:
> >
> > ERROR:check_prob_thresh() -> When verifying a probability field,
you must
> > select at least 3 thresholds beginning with 0.0 and ending with
1.0.
> >
> > During troubleshooting, I've changed the settings for
prob_cat_thresh in
> > the
> > attached config file several times thinking that the formatting of
my
> > settings was the issue. Maybe I'm on the wrong track. Could you
take a
> look
> > at the config file to check the format or other possible issue
which
> might
> > be causing this issue?
> >
> > Thanks.
> >
> > R/
> > John
> >
> > Mr. John W. Raby
> > U.S. Army Research Laboratory
> > White Sands Missile Range, NM 88002
> > Office:(575) 678-2004 DSN 258-2004
> > Teleworking: Contact Admin Specialist at (410) 278-6974
> > FAX (575) 678-1230 DSN 258-1230
> > Email: john.w.raby2.civ at mail.mil
> >
> > CLASSIFICATION: UNCLASSIFIED
> >
> >
>
>
>
------------------------------------------------
Subject: MET V9.0 Point-Stat error (UNCLASSIFIED)
From: Raby, John W USA CIV
Time: Wed Jul 01 16:00:07 2020
CLASSIFICATION: UNCLASSIFIED
John -
Thanks for taking another look at it and for the more detailed
approach to
explaining how HiRA works which I've never used before. After reading
the
User's Guide, I got the impression that the categorical thresholds
(cat_thresh
array) were being used to compute the fractional coverage values
which are
evaluated as probability forecasts. (Ref the first para of the Section
on HiRA
Framework). So it was a little confusing how the prob_cat_thresh
settings
would fit in. I didn't see any example of what those threshold values
looked
like and came across something somewhere else in the User's Guide
which I
thought might be appropriate.
I need to understand what is meant by "meaningful" thresholds for each
variable. My thought was to just get it to run with one set of
thresholds fits
all approach and then hopefully gain an understanding of the impact of
using
the wrong or inappropriate threshold by looking at the output.
In your statement below: "we could have the code check to see if
cat_thresh in
the obs dictionary is non-empty", if obs = fcst why wouldn't you check
the
fcst dictionary? Just curious about why the obs dictionary would be
different/better for some reason.
>From your recommended HiRA settings, it seems to me that the
cov_thresh is the
critical setting which provides the probability bins which are applied
to the
probability forecasts. If the HiRA dictionary is best linked to the
specific
variable, then your fix to make things easier looks like a nice
improvement.
So, in reality, the cat_thresh array values are identical to the
prob_cat_thresh array values, correct?
If I'm correct in the previous sentence, I think I now understand why
the
prob_cat_thresh threshold values have to be tailored to each variable,
just as
the cat_thresh values are tailored to each variable (i.e >=270 K
doesn't make
sense for the RH variable. Is this what you mean by "meaningful"?
R/
John
-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Wednesday, July 1, 2020 11:17 AM
To: Raby, John W CIV USARMY CCDC ARL (USA) <john.w.raby2.civ at mail.mil>
Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
Point-Stat
error (UNCLASSIFIED)
All active links contained in this email were disabled. Please verify
the
identity of the sender, and confirm the authenticity of all links
contained
within the message prior to copying and pasting the address to a Web
browser.
----
John,
Sorry for the confusion. I led you astray. The HiRA configuration
option
names are not entirely intuitive.
Please take a look at the README file starting on line 1418:
Caution-
https://github.com/NCAR/MET/blob/0cee9fbd0b3c691fd3cf4dd1d665f5d2b0db9a37/met/data/config/README#L1418
I modified your hira entry like this:
hira = {
flag = TRUE; *// set to TRUE/false to turn on/off
this
logic*
width = [ 3, 5, 7, 9, 11 ]; *// defines the
neighborhood
size and must be > 1*
vld_thresh = 1.0;
cov_thresh = [ ==0.25 ]; *// this defines the probability bins
to be
used. You only get to pick one setting here (==0.25 is short for
>=0,>=0.25,>=0.5,>=0.75,>=1.0)*
shape = SQUARE;
prob_cat_thresh = [ >290, >300 ];* // this defines the threshold(s)
for
which HiRA logic should be applied* }
So you just had things a bit mis-configured. The key thing to note
here is
that the hira.prob_cat_thresh setting defines the thresholds for which
HiRA
probabilities should be computed. This is different from the
cat_thresh
setting in the fcst and obs dictionaries. Those are used to compute
categorical counts and statistics and are currently separate.
It's nice that you don't have to use the same set of thresholds for
categorical stats and the HiRA logic... it's more configurable that
way.
However, I do see that it makes things a bit inconvenient. Since
you're
applying HiRA to many variables and each of those variables has
different
meaningful thresholds, you'd need to configure the HiRA dictionary
separately
for each and every variable.
That's what is meant by this comment... that the hira dictionary can
be
specified separately for each variable:
*// HiRA verification method// May be set separately in each
"obs.field"
entry*
So you'd need to copy/paste the HiRA dictionary into every single
obs.field
dictionary entry and modify the prob_cat_thresh thresholds
accordingly.
First, please make sure that you can in fact produce HiRA output.
Second, I
can think of a tweak to the logic that might make using HiRA easier.
How
about, if the hira.prob_cat_thresh array is an empty list, we could
have the
code check to see if cat_thresh in the obs dictionary is non-empty.
And if
that's non-empty then we'd simply set prob_cat_thresh = cat_thresh.
I imagine that's what you were expecting in the first place... that
Point-Stat
would just be using the thresholds from cat_thresh to define the HiRA
probability thresholds. It still is nice to have prob_cat_thresh be
configurable separately, but if left unset, defaulting to cat_thresh
seems
pretty reasonable to me. Do you think that would be a worthwhile
change?
Thanks,
John
On Tue, Jun 30, 2020 at 7:12 PM Raby, John W USA CIV via RT <
met_help at ucar.edu> wrote:
>
> <Caution-url:
> Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
>
> I forgot to attach the log file.
>
> ________________________________________
> From: Raby, John W CIV USARMY CCDC ARL (USA)
> Sent: Tuesday, June 30, 2020 7:00 PM
> To: met_help at ucar.edu
> Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
> Point-Stat error (UNCLASSIFIED)
>
> I edited the config file to use the shortcut and corrected the
> whitespace issue, but I get the same error.
> R/
> John
>
>
>
> ________________________________________
> From: Raby, John W CIV USARMY CCDC ARL (USA)
> Sent: Tuesday, June 30, 2020 6:10 PM
> To: met_help at ucar.edu
> Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
> Point-Stat error (UNCLASSIFIED)
>
> Thanks for taking a look, John. I'll give that a try. Appreciate the
> heads-up on a future error in the "desc".
>
> R/
> John
>
> ________________________________________
> From: John Halley Gotway via RT [met_help at ucar.edu]
> Sent: Tuesday, June 30, 2020 5:11 PM
> To: Raby, John W CIV USARMY CCDC ARL (USA)
> Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
> Point-Stat error (UNCLASSIFIED)
>
> All active links contained in this email were disabled. Please
verify
> the identity of the sender, and confirm the authenticity of all
links
> contained within the message prior to copying and pasting the
address
> to a Web browser.
>
>
>
>
> ----
>
> John,
>
> Please try using this shortcut:
> prob_cat_thresh = [ ==0.25 ];
>
> Does that do it?
>
> Also, your "desc" setting can't contain embedded whitespace. And
> Point-Stat should error out and tell you that.
>
> John
>
> On Tue, Jun 30, 2020 at 4:57 PM Raby, John W USA CIV via RT <
> met_help at ucar.edu> wrote:
>
> >
> > Tue Jun 30 16:56:45 2020: Request 95790 was acted upon.
> > Transaction: Ticket created by john.w.raby2.civ at mail.mil
> > Queue: met_help
> > Subject: MET V9.0 Point-Stat error (UNCLASSIFIED)
> > Owner: Nobody
> > Requestors: john.w.raby2.civ at mail.mil
> > Status: new
> > Ticket <Caution-Caution-url: Caution-
> Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> >
> >
> > CLASSIFICATION: UNCLASSIFIED
> >
> > I keep getting the same error when trying to run Point-Stat with
the
> > HiRA verification logic enabled. The error involves the
probability
> > field
> > thresholds:
> >
> > ERROR:check_prob_thresh() -> When verifying a probability field,
you
> > must select at least 3 thresholds beginning with 0.0 and ending
with 1.0.
> >
> > During troubleshooting, I've changed the settings for
> > prob_cat_thresh in the attached config file several times thinking
> > that the formatting of my settings was the issue. Maybe I'm on the
> > wrong track. Could you take a
> look
> > at the config file to check the format or other possible issue
which
> might
> > be causing this issue?
> >
> > Thanks.
> >
> > R/
> > John
> >
> > Mr. John W. Raby
> > U.S. Army Research Laboratory
> > White Sands Missile Range, NM 88002
> > Office:(575) 678-2004 DSN 258-2004
> > Teleworking: Contact Admin Specialist at (410) 278-6974 FAX (575)
> > 678-1230 DSN 258-1230
> > Email: john.w.raby2.civ at mail.mil
> >
> > CLASSIFICATION: UNCLASSIFIED
> >
> >
>
>
>
CLASSIFICATION: UNCLASSIFIED
------------------------------------------------
Subject: MET V9.0 Point-Stat error (UNCLASSIFIED)
From: John Halley Gotway
Time: Wed Jul 01 16:47:21 2020
John,
Yes, that's exactly what I mean. ">290" makes sense for TMP in Kelvin
but
not RH or wind or precip. By having prob_cat_thresh = cat_thresh
(unless
its otherwise specified) you'd only need to set the cat_thresh once.
I'll need to write up this proposed change as a GitHub issue.
John
On Wed, Jul 1, 2020 at 4:00 PM Raby, John W USA CIV via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
>
> CLASSIFICATION: UNCLASSIFIED
>
> John -
>
> Thanks for taking another look at it and for the more detailed
approach to
> explaining how HiRA works which I've never used before. After
reading the
> User's Guide, I got the impression that the categorical thresholds
> (cat_thresh
> array) were being used to compute the fractional coverage values
which
> are
> evaluated as probability forecasts. (Ref the first para of the
Section on
> HiRA
> Framework). So it was a little confusing how the prob_cat_thresh
settings
> would fit in. I didn't see any example of what those threshold
values
> looked
> like and came across something somewhere else in the User's Guide
which I
> thought might be appropriate.
>
> I need to understand what is meant by "meaningful" thresholds for
each
> variable. My thought was to just get it to run with one set of
thresholds
> fits
> all approach and then hopefully gain an understanding of the impact
of
> using
> the wrong or inappropriate threshold by looking at the output.
>
> In your statement below: "we could have the code check to see if
> cat_thresh in
> the obs dictionary is non-empty", if obs = fcst why wouldn't you
check the
> fcst dictionary? Just curious about why the obs dictionary would be
> different/better for some reason.
>
> From your recommended HiRA settings, it seems to me that the
cov_thresh is
> the
> critical setting which provides the probability bins which are
applied to
> the
> probability forecasts. If the HiRA dictionary is best linked to the
> specific
> variable, then your fix to make things easier looks like a nice
> improvement.
> So, in reality, the cat_thresh array values are identical to the
> prob_cat_thresh array values, correct?
>
> If I'm correct in the previous sentence, I think I now understand
why the
> prob_cat_thresh threshold values have to be tailored to each
variable,
> just as
> the cat_thresh values are tailored to each variable (i.e >=270 K
doesn't
> make
> sense for the RH variable. Is this what you mean by "meaningful"?
>
> R/
> John
>
> -----Original Message-----
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Wednesday, July 1, 2020 11:17 AM
> To: Raby, John W CIV USARMY CCDC ARL (USA)
<john.w.raby2.civ at mail.mil>
> Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
> Point-Stat
> error (UNCLASSIFIED)
>
> All active links contained in this email were disabled. Please
verify the
> identity of the sender, and confirm the authenticity of all links
> contained
> within the message prior to copying and pasting the address to a Web
> browser.
>
>
>
>
> ----
>
> John,
>
> Sorry for the confusion. I led you astray. The HiRA configuration
option
> names are not entirely intuitive.
>
> Please take a look at the README file starting on line 1418:
> Caution-
>
https://github.com/NCAR/MET/blob/0cee9fbd0b3c691fd3cf4dd1d665f5d2b0db9a37/met/data/config/README#L1418
>
> I modified your hira entry like this:
>
> hira = {
> flag = TRUE; *// set to TRUE/false to turn on/off
this
> logic*
> width = [ 3, 5, 7, 9, 11 ]; *// defines the
neighborhood
> size and must be > 1*
> vld_thresh = 1.0;
> cov_thresh = [ ==0.25 ]; *// this defines the probability
bins to be
> used. You only get to pick one setting here (==0.25 is short for
> >=0,>=0.25,>=0.5,>=0.75,>=1.0)*
> shape = SQUARE;
> prob_cat_thresh = [ >290, >300 ];* // this defines the
threshold(s) for
> which HiRA logic should be applied* }
>
> So you just had things a bit mis-configured. The key thing to note
here is
> that the hira.prob_cat_thresh setting defines the thresholds for
which
> HiRA
> probabilities should be computed. This is different from the
cat_thresh
> setting in the fcst and obs dictionaries. Those are used to compute
> categorical counts and statistics and are currently separate.
>
> It's nice that you don't have to use the same set of thresholds for
> categorical stats and the HiRA logic... it's more configurable that
way.
> However, I do see that it makes things a bit inconvenient. Since
you're
> applying HiRA to many variables and each of those variables has
different
> meaningful thresholds, you'd need to configure the HiRA dictionary
> separately
> for each and every variable.
>
> That's what is meant by this comment... that the hira dictionary can
be
> specified separately for each variable:
>
> *// HiRA verification method// May be set separately in each
"obs.field"
> entry*
>
> So you'd need to copy/paste the HiRA dictionary into every single
> obs.field
> dictionary entry and modify the prob_cat_thresh thresholds
accordingly.
>
> First, please make sure that you can in fact produce HiRA output.
Second,
> I
> can think of a tweak to the logic that might make using HiRA easier.
How
> about, if the hira.prob_cat_thresh array is an empty list, we could
have
> the
> code check to see if cat_thresh in the obs dictionary is non-empty.
And if
> that's non-empty then we'd simply set prob_cat_thresh = cat_thresh.
>
> I imagine that's what you were expecting in the first place... that
> Point-Stat
> would just be using the thresholds from cat_thresh to define the
HiRA
> probability thresholds. It still is nice to have prob_cat_thresh be
> configurable separately, but if left unset, defaulting to cat_thresh
seems
> pretty reasonable to me. Do you think that would be a worthwhile
change?
>
> Thanks,
> John
>
> On Tue, Jun 30, 2020 at 7:12 PM Raby, John W USA CIV via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <Caution-url:
> > Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> >
> > I forgot to attach the log file.
> >
> > ________________________________________
> > From: Raby, John W CIV USARMY CCDC ARL (USA)
> > Sent: Tuesday, June 30, 2020 7:00 PM
> > To: met_help at ucar.edu
> > Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > Point-Stat error (UNCLASSIFIED)
> >
> > I edited the config file to use the shortcut and corrected the
> > whitespace issue, but I get the same error.
> > R/
> > John
> >
> >
> >
> > ________________________________________
> > From: Raby, John W CIV USARMY CCDC ARL (USA)
> > Sent: Tuesday, June 30, 2020 6:10 PM
> > To: met_help at ucar.edu
> > Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > Point-Stat error (UNCLASSIFIED)
> >
> > Thanks for taking a look, John. I'll give that a try. Appreciate
the
> > heads-up on a future error in the "desc".
> >
> > R/
> > John
> >
> > ________________________________________
> > From: John Halley Gotway via RT [met_help at ucar.edu]
> > Sent: Tuesday, June 30, 2020 5:11 PM
> > To: Raby, John W CIV USARMY CCDC ARL (USA)
> > Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
> > Point-Stat error (UNCLASSIFIED)
> >
> > All active links contained in this email were disabled. Please
verify
> > the identity of the sender, and confirm the authenticity of all
links
> > contained within the message prior to copying and pasting the
address
> > to a Web browser.
> >
> >
> >
> >
> > ----
> >
> > John,
> >
> > Please try using this shortcut:
> > prob_cat_thresh = [ ==0.25 ];
> >
> > Does that do it?
> >
> > Also, your "desc" setting can't contain embedded whitespace. And
> > Point-Stat should error out and tell you that.
> >
> > John
> >
> > On Tue, Jun 30, 2020 at 4:57 PM Raby, John W USA CIV via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > Tue Jun 30 16:56:45 2020: Request 95790 was acted upon.
> > > Transaction: Ticket created by john.w.raby2.civ at mail.mil
> > > Queue: met_help
> > > Subject: MET V9.0 Point-Stat error (UNCLASSIFIED)
> > > Owner: Nobody
> > > Requestors: john.w.raby2.civ at mail.mil
> > > Status: new
> > > Ticket <Caution-Caution-url: Caution-
> > Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> > >
> > >
> > > CLASSIFICATION: UNCLASSIFIED
> > >
> > > I keep getting the same error when trying to run Point-Stat with
the
> > > HiRA verification logic enabled. The error involves the
probability
> > > field
> > > thresholds:
> > >
> > > ERROR:check_prob_thresh() -> When verifying a probability field,
you
> > > must select at least 3 thresholds beginning with 0.0 and ending
with
> 1.0.
> > >
> > > During troubleshooting, I've changed the settings for
> > > prob_cat_thresh in the attached config file several times
thinking
> > > that the formatting of my settings was the issue. Maybe I'm on
the
> > > wrong track. Could you take a
> > look
> > > at the config file to check the format or other possible issue
which
> > might
> > > be causing this issue?
> > >
> > > Thanks.
> > >
> > > R/
> > > John
> > >
> > > Mr. John W. Raby
> > > U.S. Army Research Laboratory
> > > White Sands Missile Range, NM 88002
> > > Office:(575) 678-2004 DSN 258-2004
> > > Teleworking: Contact Admin Specialist at (410) 278-6974 FAX
(575)
> > > 678-1230 DSN 258-1230
> > > Email: john.w.raby2.civ at mail.mil
> > >
> > > CLASSIFICATION: UNCLASSIFIED
> > >
> > >
> >
> >
> >
>
> CLASSIFICATION: UNCLASSIFIED
>
>
------------------------------------------------
Subject: MET V9.0 Point-Stat error (UNCLASSIFIED)
From: Raby, John W USA CIV
Time: Wed Jul 01 18:09:53 2020
CLASSIFICATION: UNCLASSIFIED
Thanks for confirming this for me. The recommended sw change looks
good to me.
I'll test the chg to the HiRA dictionary and the fcst/ob dictionaries
which
you provided tomorrow and let you know.
R/
John
-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Wednesday, July 1, 2020 4:47 PM
To: Raby, John W CIV USARMY CCDC ARL (USA) <john.w.raby2.civ at mail.mil>
Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
Point-Stat
error (UNCLASSIFIED)
All active links contained in this email were disabled. Please verify
the
identity of the sender, and confirm the authenticity of all links
contained
within the message prior to copying and pasting the address to a Web
browser.
----
John,
Yes, that's exactly what I mean. ">290" makes sense for TMP in Kelvin
but not
RH or wind or precip. By having prob_cat_thresh = cat_thresh (unless
its
otherwise specified) you'd only need to set the cat_thresh once.
I'll need to write up this proposed change as a GitHub issue.
John
On Wed, Jul 1, 2020 at 4:00 PM Raby, John W USA CIV via RT <
met_help at ucar.edu> wrote:
>
> <Caution-url:
> Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
>
> CLASSIFICATION: UNCLASSIFIED
>
> John -
>
> Thanks for taking another look at it and for the more detailed
> approach to explaining how HiRA works which I've never used before.
> After reading the User's Guide, I got the impression that the
> categorical thresholds (cat_thresh
> array) were being used to compute the fractional coverage values
> which are evaluated as probability forecasts. (Ref the first para of
> the Section on HiRA Framework). So it was a little confusing how the
> prob_cat_thresh settings would fit in. I didn't see any example of
> what those threshold values looked like and came across something
> somewhere else in the User's Guide which I thought might be
> appropriate.
>
> I need to understand what is meant by "meaningful" thresholds for
each
> variable. My thought was to just get it to run with one set of
> thresholds fits all approach and then hopefully gain an
understanding
> of the impact of using the wrong or inappropriate threshold by
looking
> at the output.
>
> In your statement below: "we could have the code check to see if
> cat_thresh in the obs dictionary is non-empty", if obs = fcst why
> wouldn't you check the fcst dictionary? Just curious about why the
obs
> dictionary would be different/better for some reason.
>
> From your recommended HiRA settings, it seems to me that the
> cov_thresh is the critical setting which provides the probability
bins
> which are applied to the probability forecasts. If the HiRA
dictionary
> is best linked to the specific variable, then your fix to make
things
> easier looks like a nice improvement.
> So, in reality, the cat_thresh array values are identical to the
> prob_cat_thresh array values, correct?
>
> If I'm correct in the previous sentence, I think I now understand
why
> the prob_cat_thresh threshold values have to be tailored to each
> variable, just as the cat_thresh values are tailored to each
variable
> (i.e >=270 K doesn't make sense for the RH variable. Is this what
you
> mean by "meaningful"?
>
> R/
> John
>
> -----Original Message-----
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Wednesday, July 1, 2020 11:17 AM
> To: Raby, John W CIV USARMY CCDC ARL (USA)
<john.w.raby2.civ at mail.mil>
> Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
> Point-Stat error (UNCLASSIFIED)
>
> All active links contained in this email were disabled. Please
verify
> the identity of the sender, and confirm the authenticity of all
links
> contained within the message prior to copying and pasting the
address
> to a Web browser.
>
>
>
>
> ----
>
> John,
>
> Sorry for the confusion. I led you astray. The HiRA configuration
> option names are not entirely intuitive.
>
> Please take a look at the README file starting on line 1418:
> Caution-
> Caution-
https://github.com/NCAR/MET/blob/0cee9fbd0b3c691fd3cf4dd1d665f
> 5d2b0db9a37/met/data/config/README#L1418
>
> I modified your hira entry like this:
>
> hira = {
> flag = TRUE; *// set to TRUE/false to turn on/off
this
> logic*
> width = [ 3, 5, 7, 9, 11 ]; *// defines the
neighborhood
> size and must be > 1*
> vld_thresh = 1.0;
> cov_thresh = [ ==0.25 ]; *// this defines the probability
bins to be
> used. You only get to pick one setting here (==0.25 is short for
> >=0,>=0.25,>=0.5,>=0.75,>=1.0)*
> shape = SQUARE;
> prob_cat_thresh = [ >290, >300 ];* // this defines the
threshold(s)
> for which HiRA logic should be applied* }
>
> So you just had things a bit mis-configured. The key thing to note
> here is that the hira.prob_cat_thresh setting defines the thresholds
> for which HiRA probabilities should be computed. This is different
> from the cat_thresh setting in the fcst and obs dictionaries. Those
> are used to compute categorical counts and statistics and are
> currently separate.
>
> It's nice that you don't have to use the same set of thresholds for
> categorical stats and the HiRA logic... it's more configurable that
way.
> However, I do see that it makes things a bit inconvenient. Since
> you're applying HiRA to many variables and each of those variables
has
> different meaningful thresholds, you'd need to configure the HiRA
> dictionary separately for each and every variable.
>
> That's what is meant by this comment... that the hira dictionary can
> be specified separately for each variable:
>
> *// HiRA verification method// May be set separately in each
"obs.field"
> entry*
>
> So you'd need to copy/paste the HiRA dictionary into every single
> obs.field dictionary entry and modify the prob_cat_thresh thresholds
> accordingly.
>
> First, please make sure that you can in fact produce HiRA output.
> Second, I can think of a tweak to the logic that might make using
HiRA
> easier. How about, if the hira.prob_cat_thresh array is an empty
list,
> we could have the code check to see if cat_thresh in the obs
> dictionary is non-empty. And if that's non-empty then we'd simply
set
> prob_cat_thresh = cat_thresh.
>
> I imagine that's what you were expecting in the first place... that
> Point-Stat would just be using the thresholds from cat_thresh to
> define the HiRA probability thresholds. It still is nice to have
> prob_cat_thresh be configurable separately, but if left unset,
> defaulting to cat_thresh seems pretty reasonable to me. Do you think
> that would be a worthwhile change?
>
> Thanks,
> John
>
> On Tue, Jun 30, 2020 at 7:12 PM Raby, John W USA CIV via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <Caution-Caution-url:
> > Caution-Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95
> > 790 >
> >
> > I forgot to attach the log file.
> >
> > ________________________________________
> > From: Raby, John W CIV USARMY CCDC ARL (USA)
> > Sent: Tuesday, June 30, 2020 7:00 PM
> > To: met_help at ucar.edu
> > Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > Point-Stat error (UNCLASSIFIED)
> >
> > I edited the config file to use the shortcut and corrected the
> > whitespace issue, but I get the same error.
> > R/
> > John
> >
> >
> >
> > ________________________________________
> > From: Raby, John W CIV USARMY CCDC ARL (USA)
> > Sent: Tuesday, June 30, 2020 6:10 PM
> > To: met_help at ucar.edu
> > Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > Point-Stat error (UNCLASSIFIED)
> >
> > Thanks for taking a look, John. I'll give that a try. Appreciate
the
> > heads-up on a future error in the "desc".
> >
> > R/
> > John
> >
> > ________________________________________
> > From: John Halley Gotway via RT [met_help at ucar.edu]
> > Sent: Tuesday, June 30, 2020 5:11 PM
> > To: Raby, John W CIV USARMY CCDC ARL (USA)
> > Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
> > Point-Stat error (UNCLASSIFIED)
> >
> > All active links contained in this email were disabled. Please
> > verify the identity of the sender, and confirm the authenticity of
> > all links contained within the message prior to copying and
pasting
> > the address to a Web browser.
> >
> >
> >
> >
> > ----
> >
> > John,
> >
> > Please try using this shortcut:
> > prob_cat_thresh = [ ==0.25 ];
> >
> > Does that do it?
> >
> > Also, your "desc" setting can't contain embedded whitespace. And
> > Point-Stat should error out and tell you that.
> >
> > John
> >
> > On Tue, Jun 30, 2020 at 4:57 PM Raby, John W USA CIV via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > Tue Jun 30 16:56:45 2020: Request 95790 was acted upon.
> > > Transaction: Ticket created by john.w.raby2.civ at mail.mil
> > > Queue: met_help
> > > Subject: MET V9.0 Point-Stat error (UNCLASSIFIED)
> > > Owner: Nobody
> > > Requestors: john.w.raby2.civ at mail.mil
> > > Status: new
> > > Ticket <Caution-Caution-Caution-url: Caution-
> > Caution-Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95
> > 790 >
> > >
> > >
> > > CLASSIFICATION: UNCLASSIFIED
> > >
> > > I keep getting the same error when trying to run Point-Stat with
> > > the HiRA verification logic enabled. The error involves the
> > > probability field
> > > thresholds:
> > >
> > > ERROR:check_prob_thresh() -> When verifying a probability field,
> > > you must select at least 3 thresholds beginning with 0.0 and
> > > ending with
> 1.0.
> > >
> > > During troubleshooting, I've changed the settings for
> > > prob_cat_thresh in the attached config file several times
thinking
> > > that the formatting of my settings was the issue. Maybe I'm on
the
> > > wrong track. Could you take a
> > look
> > > at the config file to check the format or other possible issue
> > > which
> > might
> > > be causing this issue?
> > >
> > > Thanks.
> > >
> > > R/
> > > John
> > >
> > > Mr. John W. Raby
> > > U.S. Army Research Laboratory
> > > White Sands Missile Range, NM 88002
> > > Office:(575) 678-2004 DSN 258-2004
> > > Teleworking: Contact Admin Specialist at (410) 278-6974 FAX
(575)
> > > 678-1230 DSN 258-1230
> > > Email: john.w.raby2.civ at mail.mil
> > >
> > > CLASSIFICATION: UNCLASSIFIED
> > >
> > >
> >
> >
> >
>
> CLASSIFICATION: UNCLASSIFIED
>
>
CLASSIFICATION: UNCLASSIFIED
------------------------------------------------
Subject: MET V9.0 Point-Stat error (UNCLASSIFIED)
From: John Halley Gotway
Time: Wed Jul 01 18:22:26 2020
Sounds good. Thanks.
FYI, here's the GitHub issue I wrote up for this change:
https://github.com/NCAR/MET/issues/1400
I tentatively assigned it to the next release, but can't guarantee
we'll
get it in there.
John
On Wed, Jul 1, 2020 at 6:10 PM Raby, John W USA CIV via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
>
> CLASSIFICATION: UNCLASSIFIED
>
> Thanks for confirming this for me. The recommended sw change looks
good to
> me.
> I'll test the chg to the HiRA dictionary and the fcst/ob
dictionaries
> which
> you provided tomorrow and let you know.
>
> R/
> John
>
> -----Original Message-----
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Wednesday, July 1, 2020 4:47 PM
> To: Raby, John W CIV USARMY CCDC ARL (USA)
<john.w.raby2.civ at mail.mil>
> Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
> Point-Stat
> error (UNCLASSIFIED)
>
> All active links contained in this email were disabled. Please
verify the
> identity of the sender, and confirm the authenticity of all links
> contained
> within the message prior to copying and pasting the address to a Web
> browser.
>
>
>
>
> ----
>
> John,
>
> Yes, that's exactly what I mean. ">290" makes sense for TMP in
Kelvin but
> not
> RH or wind or precip. By having prob_cat_thresh = cat_thresh (unless
its
> otherwise specified) you'd only need to set the cat_thresh once.
>
> I'll need to write up this proposed change as a GitHub issue.
>
> John
>
> On Wed, Jul 1, 2020 at 4:00 PM Raby, John W USA CIV via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <Caution-url:
> > Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> >
> > CLASSIFICATION: UNCLASSIFIED
> >
> > John -
> >
> > Thanks for taking another look at it and for the more detailed
> > approach to explaining how HiRA works which I've never used
before.
> > After reading the User's Guide, I got the impression that the
> > categorical thresholds (cat_thresh
> > array) were being used to compute the fractional coverage values
> > which are evaluated as probability forecasts. (Ref the first para
of
> > the Section on HiRA Framework). So it was a little confusing how
the
> > prob_cat_thresh settings would fit in. I didn't see any example of
> > what those threshold values looked like and came across something
> > somewhere else in the User's Guide which I thought might be
> > appropriate.
> >
> > I need to understand what is meant by "meaningful" thresholds for
each
> > variable. My thought was to just get it to run with one set of
> > thresholds fits all approach and then hopefully gain an
understanding
> > of the impact of using the wrong or inappropriate threshold by
looking
> > at the output.
> >
> > In your statement below: "we could have the code check to see if
> > cat_thresh in the obs dictionary is non-empty", if obs = fcst why
> > wouldn't you check the fcst dictionary? Just curious about why the
obs
> > dictionary would be different/better for some reason.
> >
> > From your recommended HiRA settings, it seems to me that the
> > cov_thresh is the critical setting which provides the probability
bins
> > which are applied to the probability forecasts. If the HiRA
dictionary
> > is best linked to the specific variable, then your fix to make
things
> > easier looks like a nice improvement.
> > So, in reality, the cat_thresh array values are identical to the
> > prob_cat_thresh array values, correct?
> >
> > If I'm correct in the previous sentence, I think I now understand
why
> > the prob_cat_thresh threshold values have to be tailored to each
> > variable, just as the cat_thresh values are tailored to each
variable
> > (i.e >=270 K doesn't make sense for the RH variable. Is this what
you
> > mean by "meaningful"?
> >
> > R/
> > John
> >
> > -----Original Message-----
> > From: John Halley Gotway via RT <met_help at ucar.edu>
> > Sent: Wednesday, July 1, 2020 11:17 AM
> > To: Raby, John W CIV USARMY CCDC ARL (USA)
<john.w.raby2.civ at mail.mil>
> > Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > Point-Stat error (UNCLASSIFIED)
> >
> > All active links contained in this email were disabled. Please
verify
> > the identity of the sender, and confirm the authenticity of all
links
> > contained within the message prior to copying and pasting the
address
> > to a Web browser.
> >
> >
> >
> >
> > ----
> >
> > John,
> >
> > Sorry for the confusion. I led you astray. The HiRA configuration
> > option names are not entirely intuitive.
> >
> > Please take a look at the README file starting on line 1418:
> > Caution-
> > Caution-
https://github.com/NCAR/MET/blob/0cee9fbd0b3c691fd3cf4dd1d665f
> > 5d2b0db9a37/met/data/config/README#L1418
> >
> > I modified your hira entry like this:
> >
> > hira = {
> > flag = TRUE; *// set to TRUE/false to turn
on/off this
> > logic*
> > width = [ 3, 5, 7, 9, 11 ]; *// defines the
neighborhood
> > size and must be > 1*
> > vld_thresh = 1.0;
> > cov_thresh = [ ==0.25 ]; *// this defines the probability
bins to
> be
> > used. You only get to pick one setting here (==0.25 is short for
> > >=0,>=0.25,>=0.5,>=0.75,>=1.0)*
> > shape = SQUARE;
> > prob_cat_thresh = [ >290, >300 ];* // this defines the
threshold(s)
> > for which HiRA logic should be applied* }
> >
> > So you just had things a bit mis-configured. The key thing to note
> > here is that the hira.prob_cat_thresh setting defines the
thresholds
> > for which HiRA probabilities should be computed. This is different
> > from the cat_thresh setting in the fcst and obs dictionaries.
Those
> > are used to compute categorical counts and statistics and are
> > currently separate.
> >
> > It's nice that you don't have to use the same set of thresholds
for
> > categorical stats and the HiRA logic... it's more configurable
that way.
> > However, I do see that it makes things a bit inconvenient. Since
> > you're applying HiRA to many variables and each of those variables
has
> > different meaningful thresholds, you'd need to configure the HiRA
> > dictionary separately for each and every variable.
> >
> > That's what is meant by this comment... that the hira dictionary
can
> > be specified separately for each variable:
> >
> > *// HiRA verification method// May be set separately in each
"obs.field"
> > entry*
> >
> > So you'd need to copy/paste the HiRA dictionary into every single
> > obs.field dictionary entry and modify the prob_cat_thresh
thresholds
> > accordingly.
> >
> > First, please make sure that you can in fact produce HiRA output.
> > Second, I can think of a tweak to the logic that might make using
HiRA
> > easier. How about, if the hira.prob_cat_thresh array is an empty
list,
> > we could have the code check to see if cat_thresh in the obs
> > dictionary is non-empty. And if that's non-empty then we'd simply
set
> > prob_cat_thresh = cat_thresh.
> >
> > I imagine that's what you were expecting in the first place...
that
> > Point-Stat would just be using the thresholds from cat_thresh to
> > define the HiRA probability thresholds. It still is nice to have
> > prob_cat_thresh be configurable separately, but if left unset,
> > defaulting to cat_thresh seems pretty reasonable to me. Do you
think
> > that would be a worthwhile change?
> >
> > Thanks,
> > John
> >
> > On Tue, Jun 30, 2020 at 7:12 PM Raby, John W USA CIV via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <Caution-Caution-url:
> > > Caution-Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95
> > > 790 >
> > >
> > > I forgot to attach the log file.
> > >
> > > ________________________________________
> > > From: Raby, John W CIV USARMY CCDC ARL (USA)
> > > Sent: Tuesday, June 30, 2020 7:00 PM
> > > To: met_help at ucar.edu
> > > Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > > Point-Stat error (UNCLASSIFIED)
> > >
> > > I edited the config file to use the shortcut and corrected the
> > > whitespace issue, but I get the same error.
> > > R/
> > > John
> > >
> > >
> > >
> > > ________________________________________
> > > From: Raby, John W CIV USARMY CCDC ARL (USA)
> > > Sent: Tuesday, June 30, 2020 6:10 PM
> > > To: met_help at ucar.edu
> > > Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > > Point-Stat error (UNCLASSIFIED)
> > >
> > > Thanks for taking a look, John. I'll give that a try. Appreciate
the
> > > heads-up on a future error in the "desc".
> > >
> > > R/
> > > John
> > >
> > > ________________________________________
> > > From: John Halley Gotway via RT [met_help at ucar.edu]
> > > Sent: Tuesday, June 30, 2020 5:11 PM
> > > To: Raby, John W CIV USARMY CCDC ARL (USA)
> > > Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
> > > Point-Stat error (UNCLASSIFIED)
> > >
> > > All active links contained in this email were disabled. Please
> > > verify the identity of the sender, and confirm the authenticity
of
> > > all links contained within the message prior to copying and
pasting
> > > the address to a Web browser.
> > >
> > >
> > >
> > >
> > > ----
> > >
> > > John,
> > >
> > > Please try using this shortcut:
> > > prob_cat_thresh = [ ==0.25 ];
> > >
> > > Does that do it?
> > >
> > > Also, your "desc" setting can't contain embedded whitespace. And
> > > Point-Stat should error out and tell you that.
> > >
> > > John
> > >
> > > On Tue, Jun 30, 2020 at 4:57 PM Raby, John W USA CIV via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > Tue Jun 30 16:56:45 2020: Request 95790 was acted upon.
> > > > Transaction: Ticket created by john.w.raby2.civ at mail.mil
> > > > Queue: met_help
> > > > Subject: MET V9.0 Point-Stat error (UNCLASSIFIED)
> > > > Owner: Nobody
> > > > Requestors: john.w.raby2.civ at mail.mil
> > > > Status: new
> > > > Ticket <Caution-Caution-Caution-url: Caution-
> > > Caution-Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95
> > > 790 >
> > > >
> > > >
> > > > CLASSIFICATION: UNCLASSIFIED
> > > >
> > > > I keep getting the same error when trying to run Point-Stat
with
> > > > the HiRA verification logic enabled. The error involves the
> > > > probability field
> > > > thresholds:
> > > >
> > > > ERROR:check_prob_thresh() -> When verifying a probability
field,
> > > > you must select at least 3 thresholds beginning with 0.0 and
> > > > ending with
> > 1.0.
> > > >
> > > > During troubleshooting, I've changed the settings for
> > > > prob_cat_thresh in the attached config file several times
thinking
> > > > that the formatting of my settings was the issue. Maybe I'm on
the
> > > > wrong track. Could you take a
> > > look
> > > > at the config file to check the format or other possible issue
> > > > which
> > > might
> > > > be causing this issue?
> > > >
> > > > Thanks.
> > > >
> > > > R/
> > > > John
> > > >
> > > > Mr. John W. Raby
> > > > U.S. Army Research Laboratory
> > > > White Sands Missile Range, NM 88002
> > > > Office:(575) 678-2004 DSN 258-2004
> > > > Teleworking: Contact Admin Specialist at (410) 278-6974 FAX
(575)
> > > > 678-1230 DSN 258-1230
> > > > Email: john.w.raby2.civ at mail.mil
> > > >
> > > > CLASSIFICATION: UNCLASSIFIED
> > > >
> > > >
> > >
> > >
> > >
> >
> > CLASSIFICATION: UNCLASSIFIED
> >
> >
>
> CLASSIFICATION: UNCLASSIFIED
>
>
------------------------------------------------
Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0 Point-Stat error (UNCLASSIFIED)
From: Raby, John W USA CIV
Time: Wed Jul 01 18:26:25 2020
Thanks for creating the sw chg request and for the link.
R/
John
________________________________________
From: John Halley Gotway via RT [met_help at ucar.edu]
Sent: Wednesday, July 01, 2020 6:22 PM
To: Raby, John W CIV USARMY CCDC ARL (USA)
Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
Point-Stat error (UNCLASSIFIED)
All active links contained in this email were disabled. Please verify
the identity of the sender, and confirm the authenticity of all links
contained within the message prior to copying and pasting the address
to a Web browser.
----
Sounds good. Thanks.
FYI, here's the GitHub issue I wrote up for this change:
Caution-https://github.com/NCAR/MET/issues/1400
I tentatively assigned it to the next release, but can't guarantee
we'll
get it in there.
John
On Wed, Jul 1, 2020 at 6:10 PM Raby, John W USA CIV via RT <
met_help at ucar.edu> wrote:
>
> <Caution-url: Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
>
> CLASSIFICATION: UNCLASSIFIED
>
> Thanks for confirming this for me. The recommended sw change looks
good to
> me.
> I'll test the chg to the HiRA dictionary and the fcst/ob
dictionaries
> which
> you provided tomorrow and let you know.
>
> R/
> John
>
> -----Original Message-----
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Wednesday, July 1, 2020 4:47 PM
> To: Raby, John W CIV USARMY CCDC ARL (USA)
<john.w.raby2.civ at mail.mil>
> Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
> Point-Stat
> error (UNCLASSIFIED)
>
> All active links contained in this email were disabled. Please
verify the
> identity of the sender, and confirm the authenticity of all links
> contained
> within the message prior to copying and pasting the address to a Web
> browser.
>
>
>
>
> ----
>
> John,
>
> Yes, that's exactly what I mean. ">290" makes sense for TMP in
Kelvin but
> not
> RH or wind or precip. By having prob_cat_thresh = cat_thresh (unless
its
> otherwise specified) you'd only need to set the cat_thresh once.
>
> I'll need to write up this proposed change as a GitHub issue.
>
> John
>
> On Wed, Jul 1, 2020 at 4:00 PM Raby, John W USA CIV via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <Caution-Caution-url:
> > Caution-Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> >
> > CLASSIFICATION: UNCLASSIFIED
> >
> > John -
> >
> > Thanks for taking another look at it and for the more detailed
> > approach to explaining how HiRA works which I've never used
before.
> > After reading the User's Guide, I got the impression that the
> > categorical thresholds (cat_thresh
> > array) were being used to compute the fractional coverage values
> > which are evaluated as probability forecasts. (Ref the first para
of
> > the Section on HiRA Framework). So it was a little confusing how
the
> > prob_cat_thresh settings would fit in. I didn't see any example of
> > what those threshold values looked like and came across something
> > somewhere else in the User's Guide which I thought might be
> > appropriate.
> >
> > I need to understand what is meant by "meaningful" thresholds for
each
> > variable. My thought was to just get it to run with one set of
> > thresholds fits all approach and then hopefully gain an
understanding
> > of the impact of using the wrong or inappropriate threshold by
looking
> > at the output.
> >
> > In your statement below: "we could have the code check to see if
> > cat_thresh in the obs dictionary is non-empty", if obs = fcst why
> > wouldn't you check the fcst dictionary? Just curious about why the
obs
> > dictionary would be different/better for some reason.
> >
> > From your recommended HiRA settings, it seems to me that the
> > cov_thresh is the critical setting which provides the probability
bins
> > which are applied to the probability forecasts. If the HiRA
dictionary
> > is best linked to the specific variable, then your fix to make
things
> > easier looks like a nice improvement.
> > So, in reality, the cat_thresh array values are identical to the
> > prob_cat_thresh array values, correct?
> >
> > If I'm correct in the previous sentence, I think I now understand
why
> > the prob_cat_thresh threshold values have to be tailored to each
> > variable, just as the cat_thresh values are tailored to each
variable
> > (i.e >=270 K doesn't make sense for the RH variable. Is this what
you
> > mean by "meaningful"?
> >
> > R/
> > John
> >
> > -----Original Message-----
> > From: John Halley Gotway via RT <met_help at ucar.edu>
> > Sent: Wednesday, July 1, 2020 11:17 AM
> > To: Raby, John W CIV USARMY CCDC ARL (USA)
<john.w.raby2.civ at mail.mil>
> > Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > Point-Stat error (UNCLASSIFIED)
> >
> > All active links contained in this email were disabled. Please
verify
> > the identity of the sender, and confirm the authenticity of all
links
> > contained within the message prior to copying and pasting the
address
> > to a Web browser.
> >
> >
> >
> >
> > ----
> >
> > John,
> >
> > Sorry for the confusion. I led you astray. The HiRA configuration
> > option names are not entirely intuitive.
> >
> > Please take a look at the README file starting on line 1418:
> > Caution-
> > Caution-Caution-
https://github.com/NCAR/MET/blob/0cee9fbd0b3c691fd3cf4dd1d665f
> > 5d2b0db9a37/met/data/config/README#L1418
> >
> > I modified your hira entry like this:
> >
> > hira = {
> > flag = TRUE; *// set to TRUE/false to turn
on/off this
> > logic*
> > width = [ 3, 5, 7, 9, 11 ]; *// defines the
neighborhood
> > size and must be > 1*
> > vld_thresh = 1.0;
> > cov_thresh = [ ==0.25 ]; *// this defines the probability
bins to
> be
> > used. You only get to pick one setting here (==0.25 is short for
> > >=0,>=0.25,>=0.5,>=0.75,>=1.0)*
> > shape = SQUARE;
> > prob_cat_thresh = [ >290, >300 ];* // this defines the
threshold(s)
> > for which HiRA logic should be applied* }
> >
> > So you just had things a bit mis-configured. The key thing to note
> > here is that the hira.prob_cat_thresh setting defines the
thresholds
> > for which HiRA probabilities should be computed. This is different
> > from the cat_thresh setting in the fcst and obs dictionaries.
Those
> > are used to compute categorical counts and statistics and are
> > currently separate.
> >
> > It's nice that you don't have to use the same set of thresholds
for
> > categorical stats and the HiRA logic... it's more configurable
that way.
> > However, I do see that it makes things a bit inconvenient. Since
> > you're applying HiRA to many variables and each of those variables
has
> > different meaningful thresholds, you'd need to configure the HiRA
> > dictionary separately for each and every variable.
> >
> > That's what is meant by this comment... that the hira dictionary
can
> > be specified separately for each variable:
> >
> > *// HiRA verification method// May be set separately in each
"obs.field"
> > entry*
> >
> > So you'd need to copy/paste the HiRA dictionary into every single
> > obs.field dictionary entry and modify the prob_cat_thresh
thresholds
> > accordingly.
> >
> > First, please make sure that you can in fact produce HiRA output.
> > Second, I can think of a tweak to the logic that might make using
HiRA
> > easier. How about, if the hira.prob_cat_thresh array is an empty
list,
> > we could have the code check to see if cat_thresh in the obs
> > dictionary is non-empty. And if that's non-empty then we'd simply
set
> > prob_cat_thresh = cat_thresh.
> >
> > I imagine that's what you were expecting in the first place...
that
> > Point-Stat would just be using the thresholds from cat_thresh to
> > define the HiRA probability thresholds. It still is nice to have
> > prob_cat_thresh be configurable separately, but if left unset,
> > defaulting to cat_thresh seems pretty reasonable to me. Do you
think
> > that would be a worthwhile change?
> >
> > Thanks,
> > John
> >
> > On Tue, Jun 30, 2020 at 7:12 PM Raby, John W USA CIV via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <Caution-Caution-Caution-url:
> > > Caution-Caution-Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95
> > > 790 >
> > >
> > > I forgot to attach the log file.
> > >
> > > ________________________________________
> > > From: Raby, John W CIV USARMY CCDC ARL (USA)
> > > Sent: Tuesday, June 30, 2020 7:00 PM
> > > To: met_help at ucar.edu
> > > Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > > Point-Stat error (UNCLASSIFIED)
> > >
> > > I edited the config file to use the shortcut and corrected the
> > > whitespace issue, but I get the same error.
> > > R/
> > > John
> > >
> > >
> > >
> > > ________________________________________
> > > From: Raby, John W CIV USARMY CCDC ARL (USA)
> > > Sent: Tuesday, June 30, 2020 6:10 PM
> > > To: met_help at ucar.edu
> > > Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > > Point-Stat error (UNCLASSIFIED)
> > >
> > > Thanks for taking a look, John. I'll give that a try. Appreciate
the
> > > heads-up on a future error in the "desc".
> > >
> > > R/
> > > John
> > >
> > > ________________________________________
> > > From: John Halley Gotway via RT [met_help at ucar.edu]
> > > Sent: Tuesday, June 30, 2020 5:11 PM
> > > To: Raby, John W CIV USARMY CCDC ARL (USA)
> > > Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
> > > Point-Stat error (UNCLASSIFIED)
> > >
> > > All active links contained in this email were disabled. Please
> > > verify the identity of the sender, and confirm the authenticity
of
> > > all links contained within the message prior to copying and
pasting
> > > the address to a Web browser.
> > >
> > >
> > >
> > >
> > > ----
> > >
> > > John,
> > >
> > > Please try using this shortcut:
> > > prob_cat_thresh = [ ==0.25 ];
> > >
> > > Does that do it?
> > >
> > > Also, your "desc" setting can't contain embedded whitespace. And
> > > Point-Stat should error out and tell you that.
> > >
> > > John
> > >
> > > On Tue, Jun 30, 2020 at 4:57 PM Raby, John W USA CIV via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > Tue Jun 30 16:56:45 2020: Request 95790 was acted upon.
> > > > Transaction: Ticket created by john.w.raby2.civ at mail.mil
> > > > Queue: met_help
> > > > Subject: MET V9.0 Point-Stat error (UNCLASSIFIED)
> > > > Owner: Nobody
> > > > Requestors: john.w.raby2.civ at mail.mil
> > > > Status: new
> > > > Ticket <Caution-Caution-Caution-Caution-url: Caution-
> > > Caution-Caution-Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95
> > > 790 >
> > > >
> > > >
> > > > CLASSIFICATION: UNCLASSIFIED
> > > >
> > > > I keep getting the same error when trying to run Point-Stat
with
> > > > the HiRA verification logic enabled. The error involves the
> > > > probability field
> > > > thresholds:
> > > >
> > > > ERROR:check_prob_thresh() -> When verifying a probability
field,
> > > > you must select at least 3 thresholds beginning with 0.0 and
> > > > ending with
> > 1.0.
> > > >
> > > > During troubleshooting, I've changed the settings for
> > > > prob_cat_thresh in the attached config file several times
thinking
> > > > that the formatting of my settings was the issue. Maybe I'm on
the
> > > > wrong track. Could you take a
> > > look
> > > > at the config file to check the format or other possible issue
> > > > which
> > > might
> > > > be causing this issue?
> > > >
> > > > Thanks.
> > > >
> > > > R/
> > > > John
> > > >
> > > > Mr. John W. Raby
> > > > U.S. Army Research Laboratory
> > > > White Sands Missile Range, NM 88002
> > > > Office:(575) 678-2004 DSN 258-2004
> > > > Teleworking: Contact Admin Specialist at (410) 278-6974 FAX
(575)
> > > > 678-1230 DSN 258-1230
> > > > Email: john.w.raby2.civ at mail.mil
> > > >
> > > > CLASSIFICATION: UNCLASSIFIED
> > > >
> > > >
> > >
> > >
> > >
> >
> > CLASSIFICATION: UNCLASSIFIED
> >
> >
>
> CLASSIFICATION: UNCLASSIFIED
>
>
------------------------------------------------
Subject: MET V9.0 Point-Stat error (UNCLASSIFIED)
From: Raby, John W USA CIV
Time: Thu Jul 02 15:42:08 2020
CLASSIFICATION: UNCLASSIFIED
John -
I edited the attached config file to add the HiRA dictionary to each
obs.field
dictionary entry as you described, but it seems like it's not
expecting the
hira dictionary to be where I placed it. Do you see where I went wrong
with
this? The error dialog is also attached.
Thanks and have a great 4th of July!
R/
John
-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Wednesday, July 1, 2020 11:17 AM
To: Raby, John W CIV USARMY CCDC ARL (USA) <john.w.raby2.civ at mail.mil>
Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
Point-Stat
error (UNCLASSIFIED)
All active links contained in this email were disabled. Please verify
the
identity of the sender, and confirm the authenticity of all links
contained
within the message prior to copying and pasting the address to a Web
browser.
----
John,
Sorry for the confusion. I led you astray. The HiRA configuration
option
names are not entirely intuitive.
Please take a look at the README file starting on line 1418:
Caution-
https://github.com/NCAR/MET/blob/0cee9fbd0b3c691fd3cf4dd1d665f5d2b0db9a37/met/data/config/README#L1418
I modified your hira entry like this:
hira = {
flag = TRUE; *// set to TRUE/false to turn on/off
this
logic*
width = [ 3, 5, 7, 9, 11 ]; *// defines the
neighborhood
size and must be > 1*
vld_thresh = 1.0;
cov_thresh = [ ==0.25 ]; *// this defines the probability bins
to be
used. You only get to pick one setting here (==0.25 is short for
>=0,>=0.25,>=0.5,>=0.75,>=1.0)*
shape = SQUARE;
prob_cat_thresh = [ >290, >300 ];* // this defines the threshold(s)
for
which HiRA logic should be applied* }
So you just had things a bit mis-configured. The key thing to note
here is
that the hira.prob_cat_thresh setting defines the thresholds for which
HiRA
probabilities should be computed. This is different from the
cat_thresh
setting in the fcst and obs dictionaries. Those are used to compute
categorical counts and statistics and are currently separate.
It's nice that you don't have to use the same set of thresholds for
categorical stats and the HiRA logic... it's more configurable that
way.
However, I do see that it makes things a bit inconvenient. Since
you're
applying HiRA to many variables and each of those variables has
different
meaningful thresholds, you'd need to configure the HiRA dictionary
separately
for each and every variable.
That's what is meant by this comment... that the hira dictionary can
be
specified separately for each variable:
*// HiRA verification method// May be set separately in each
"obs.field"
entry*
So you'd need to copy/paste the HiRA dictionary into every single
obs.field
dictionary entry and modify the prob_cat_thresh thresholds
accordingly.
First, please make sure that you can in fact produce HiRA output.
Second, I
can think of a tweak to the logic that might make using HiRA easier.
How
about, if the hira.prob_cat_thresh array is an empty list, we could
have the
code check to see if cat_thresh in the obs dictionary is non-empty.
And if
that's non-empty then we'd simply set prob_cat_thresh = cat_thresh.
I imagine that's what you were expecting in the first place... that
Point-Stat
would just be using the thresholds from cat_thresh to define the HiRA
probability thresholds. It still is nice to have prob_cat_thresh be
configurable separately, but if left unset, defaulting to cat_thresh
seems
pretty reasonable to me. Do you think that would be a worthwhile
change?
Thanks,
John
On Tue, Jun 30, 2020 at 7:12 PM Raby, John W USA CIV via RT <
met_help at ucar.edu> wrote:
>
> <Caution-url:
> Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
>
> I forgot to attach the log file.
>
> ________________________________________
> From: Raby, John W CIV USARMY CCDC ARL (USA)
> Sent: Tuesday, June 30, 2020 7:00 PM
> To: met_help at ucar.edu
> Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
> Point-Stat error (UNCLASSIFIED)
>
> I edited the config file to use the shortcut and corrected the
> whitespace issue, but I get the same error.
> R/
> John
>
>
>
> ________________________________________
> From: Raby, John W CIV USARMY CCDC ARL (USA)
> Sent: Tuesday, June 30, 2020 6:10 PM
> To: met_help at ucar.edu
> Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
> Point-Stat error (UNCLASSIFIED)
>
> Thanks for taking a look, John. I'll give that a try. Appreciate the
> heads-up on a future error in the "desc".
>
> R/
> John
>
> ________________________________________
> From: John Halley Gotway via RT [met_help at ucar.edu]
> Sent: Tuesday, June 30, 2020 5:11 PM
> To: Raby, John W CIV USARMY CCDC ARL (USA)
> Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
> Point-Stat error (UNCLASSIFIED)
>
> All active links contained in this email were disabled. Please
verify
> the identity of the sender, and confirm the authenticity of all
links
> contained within the message prior to copying and pasting the
address
> to a Web browser.
>
>
>
>
> ----
>
> John,
>
> Please try using this shortcut:
> prob_cat_thresh = [ ==0.25 ];
>
> Does that do it?
>
> Also, your "desc" setting can't contain embedded whitespace. And
> Point-Stat should error out and tell you that.
>
> John
>
> On Tue, Jun 30, 2020 at 4:57 PM Raby, John W USA CIV via RT <
> met_help at ucar.edu> wrote:
>
> >
> > Tue Jun 30 16:56:45 2020: Request 95790 was acted upon.
> > Transaction: Ticket created by john.w.raby2.civ at mail.mil
> > Queue: met_help
> > Subject: MET V9.0 Point-Stat error (UNCLASSIFIED)
> > Owner: Nobody
> > Requestors: john.w.raby2.civ at mail.mil
> > Status: new
> > Ticket <Caution-Caution-url: Caution-
> Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> >
> >
> > CLASSIFICATION: UNCLASSIFIED
> >
> > I keep getting the same error when trying to run Point-Stat with
the
> > HiRA verification logic enabled. The error involves the
probability
> > field
> > thresholds:
> >
> > ERROR:check_prob_thresh() -> When verifying a probability field,
you
> > must select at least 3 thresholds beginning with 0.0 and ending
with 1.0.
> >
> > During troubleshooting, I've changed the settings for
> > prob_cat_thresh in the attached config file several times thinking
> > that the formatting of my settings was the issue. Maybe I'm on the
> > wrong track. Could you take a
> look
> > at the config file to check the format or other possible issue
which
> might
> > be causing this issue?
> >
> > Thanks.
> >
> > R/
> > John
> >
> > Mr. John W. Raby
> > U.S. Army Research Laboratory
> > White Sands Missile Range, NM 88002
> > Office:(575) 678-2004 DSN 258-2004
> > Teleworking: Contact Admin Specialist at (410) 278-6974 FAX (575)
> > 678-1230 DSN 258-1230
> > Email: john.w.raby2.civ at mail.mil
> >
> > CLASSIFICATION: UNCLASSIFIED
> >
> >
>
>
>
CLASSIFICATION: UNCLASSIFIED
------------------------------------------------
Subject: MET V9.0 Point-Stat error (UNCLASSIFIED)
From: John Halley Gotway
Time: Thu Jul 02 16:14:44 2020
John,
Sorry for not answering your previous question. Yes, since you're
using
obs=fcst, you can just define it in the fcst dictionary and copy it
over to
the obs.
Yes, I do see a problem in your config file. The hira dictionary needs
to
be INSIDE of the field array dictionary entry, as shown below.
Also, for initial testing, I'd recommend using a much smaller set of
thresholds and widths. Running with all of these will take a long
time.
John
obs = {
field = [
{
name = "TMP";
level = [ "Z2" ];
cat_thresh = [ >=250, >=260, >=270, >=280, >=290, >=300,
<=273,
>=273 ];
hira = {
flag = TRUE;
width = [ 3, 5, 7, 9, 11, 13, 15, 17, 19, 21, 23,
25,
27, 29, 31, 33, 35, 37, 39, 41, 43, 45, 47, 49 ];
vld_thresh = 1.0;
cov_thresh = [ ==0.25 ];
shape = SQUARE;
prob_cat_thresh = [ >=250, >=260, >=270, >=280, >=290,
>=300,
<=273, >=273 ];
}
},
...
On Thu, Jul 2, 2020 at 3:42 PM Raby, John W USA CIV via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
>
> CLASSIFICATION: UNCLASSIFIED
>
> John -
>
> I edited the attached config file to add the HiRA dictionary to each
> obs.field
> dictionary entry as you described, but it seems like it's not
expecting
> the
> hira dictionary to be where I placed it. Do you see where I went
wrong
> with
> this? The error dialog is also attached.
>
> Thanks and have a great 4th of July!
>
> R/
> John
>
>
>
> -----Original Message-----
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Wednesday, July 1, 2020 11:17 AM
> To: Raby, John W CIV USARMY CCDC ARL (USA)
<john.w.raby2.civ at mail.mil>
> Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
> Point-Stat
> error (UNCLASSIFIED)
>
> All active links contained in this email were disabled. Please
verify the
> identity of the sender, and confirm the authenticity of all links
> contained
> within the message prior to copying and pasting the address to a Web
> browser.
>
>
>
>
> ----
>
> John,
>
> Sorry for the confusion. I led you astray. The HiRA configuration
option
> names are not entirely intuitive.
>
> Please take a look at the README file starting on line 1418:
> Caution-
>
https://github.com/NCAR/MET/blob/0cee9fbd0b3c691fd3cf4dd1d665f5d2b0db9a37/met/data/config/README#L1418
>
> I modified your hira entry like this:
>
> hira = {
> flag = TRUE; *// set to TRUE/false to turn on/off
this
> logic*
> width = [ 3, 5, 7, 9, 11 ]; *// defines the
neighborhood
> size and must be > 1*
> vld_thresh = 1.0;
> cov_thresh = [ ==0.25 ]; *// this defines the probability
bins to be
> used. You only get to pick one setting here (==0.25 is short for
> >=0,>=0.25,>=0.5,>=0.75,>=1.0)*
> shape = SQUARE;
> prob_cat_thresh = [ >290, >300 ];* // this defines the
threshold(s) for
> which HiRA logic should be applied* }
>
> So you just had things a bit mis-configured. The key thing to note
here is
> that the hira.prob_cat_thresh setting defines the thresholds for
which
> HiRA
> probabilities should be computed. This is different from the
cat_thresh
> setting in the fcst and obs dictionaries. Those are used to compute
> categorical counts and statistics and are currently separate.
>
> It's nice that you don't have to use the same set of thresholds for
> categorical stats and the HiRA logic... it's more configurable that
way.
> However, I do see that it makes things a bit inconvenient. Since
you're
> applying HiRA to many variables and each of those variables has
different
> meaningful thresholds, you'd need to configure the HiRA dictionary
> separately
> for each and every variable.
>
> That's what is meant by this comment... that the hira dictionary can
be
> specified separately for each variable:
>
> *// HiRA verification method// May be set separately in each
"obs.field"
> entry*
>
> So you'd need to copy/paste the HiRA dictionary into every single
> obs.field
> dictionary entry and modify the prob_cat_thresh thresholds
accordingly.
>
> First, please make sure that you can in fact produce HiRA output.
Second,
> I
> can think of a tweak to the logic that might make using HiRA easier.
How
> about, if the hira.prob_cat_thresh array is an empty list, we could
have
> the
> code check to see if cat_thresh in the obs dictionary is non-empty.
And if
> that's non-empty then we'd simply set prob_cat_thresh = cat_thresh.
>
> I imagine that's what you were expecting in the first place... that
> Point-Stat
> would just be using the thresholds from cat_thresh to define the
HiRA
> probability thresholds. It still is nice to have prob_cat_thresh be
> configurable separately, but if left unset, defaulting to cat_thresh
seems
> pretty reasonable to me. Do you think that would be a worthwhile
change?
>
> Thanks,
> John
>
> On Tue, Jun 30, 2020 at 7:12 PM Raby, John W USA CIV via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <Caution-url:
> > Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> >
> > I forgot to attach the log file.
> >
> > ________________________________________
> > From: Raby, John W CIV USARMY CCDC ARL (USA)
> > Sent: Tuesday, June 30, 2020 7:00 PM
> > To: met_help at ucar.edu
> > Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > Point-Stat error (UNCLASSIFIED)
> >
> > I edited the config file to use the shortcut and corrected the
> > whitespace issue, but I get the same error.
> > R/
> > John
> >
> >
> >
> > ________________________________________
> > From: Raby, John W CIV USARMY CCDC ARL (USA)
> > Sent: Tuesday, June 30, 2020 6:10 PM
> > To: met_help at ucar.edu
> > Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > Point-Stat error (UNCLASSIFIED)
> >
> > Thanks for taking a look, John. I'll give that a try. Appreciate
the
> > heads-up on a future error in the "desc".
> >
> > R/
> > John
> >
> > ________________________________________
> > From: John Halley Gotway via RT [met_help at ucar.edu]
> > Sent: Tuesday, June 30, 2020 5:11 PM
> > To: Raby, John W CIV USARMY CCDC ARL (USA)
> > Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
> > Point-Stat error (UNCLASSIFIED)
> >
> > All active links contained in this email were disabled. Please
verify
> > the identity of the sender, and confirm the authenticity of all
links
> > contained within the message prior to copying and pasting the
address
> > to a Web browser.
> >
> >
> >
> >
> > ----
> >
> > John,
> >
> > Please try using this shortcut:
> > prob_cat_thresh = [ ==0.25 ];
> >
> > Does that do it?
> >
> > Also, your "desc" setting can't contain embedded whitespace. And
> > Point-Stat should error out and tell you that.
> >
> > John
> >
> > On Tue, Jun 30, 2020 at 4:57 PM Raby, John W USA CIV via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > Tue Jun 30 16:56:45 2020: Request 95790 was acted upon.
> > > Transaction: Ticket created by john.w.raby2.civ at mail.mil
> > > Queue: met_help
> > > Subject: MET V9.0 Point-Stat error (UNCLASSIFIED)
> > > Owner: Nobody
> > > Requestors: john.w.raby2.civ at mail.mil
> > > Status: new
> > > Ticket <Caution-Caution-url: Caution-
> > Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> > >
> > >
> > > CLASSIFICATION: UNCLASSIFIED
> > >
> > > I keep getting the same error when trying to run Point-Stat with
the
> > > HiRA verification logic enabled. The error involves the
probability
> > > field
> > > thresholds:
> > >
> > > ERROR:check_prob_thresh() -> When verifying a probability field,
you
> > > must select at least 3 thresholds beginning with 0.0 and ending
with
> 1.0.
> > >
> > > During troubleshooting, I've changed the settings for
> > > prob_cat_thresh in the attached config file several times
thinking
> > > that the formatting of my settings was the issue. Maybe I'm on
the
> > > wrong track. Could you take a
> > look
> > > at the config file to check the format or other possible issue
which
> > might
> > > be causing this issue?
> > >
> > > Thanks.
> > >
> > > R/
> > > John
> > >
> > > Mr. John W. Raby
> > > U.S. Army Research Laboratory
> > > White Sands Missile Range, NM 88002
> > > Office:(575) 678-2004 DSN 258-2004
> > > Teleworking: Contact Admin Specialist at (410) 278-6974 FAX
(575)
> > > 678-1230 DSN 258-1230
> > > Email: john.w.raby2.civ at mail.mil
> > >
> > > CLASSIFICATION: UNCLASSIFIED
> > >
> > >
> >
> >
> >
>
> CLASSIFICATION: UNCLASSIFIED
>
>
------------------------------------------------
Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0 Point-Stat error (UNCLASSIFIED)
From: Raby, John W USA CIV
Time: Thu Jul 02 16:30:49 2020
John -
Thanks for taking a look and your quick reply. I see now about
including the hira dictionary inside the field array dictionary.
Thanks for showing that below.
So if I have the fcst dictionary only (obs=fcst) in my confiig file
then I all I have to do is put the hira dictionary with settings for
the fcst variables, correct? I guess I'm "not understanding "copying
it to the obs" when I don't explicitly show the obs dictionary.
Good point about limiting the number of threshold and widths for
testing.
R/
John
________________________________________
From: John Halley Gotway via RT [met_help at ucar.edu]
Sent: Thursday, July 02, 2020 4:14 PM
To: Raby, John W CIV USARMY CCDC ARL (USA)
Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
Point-Stat error (UNCLASSIFIED)
All active links contained in this email were disabled. Please verify
the identity of the sender, and confirm the authenticity of all links
contained within the message prior to copying and pasting the address
to a Web browser.
----
John,
Sorry for not answering your previous question. Yes, since you're
using
obs=fcst, you can just define it in the fcst dictionary and copy it
over to
the obs.
Yes, I do see a problem in your config file. The hira dictionary needs
to
be INSIDE of the field array dictionary entry, as shown below.
Also, for initial testing, I'd recommend using a much smaller set of
thresholds and widths. Running with all of these will take a long
time.
John
obs = {
field = [
{
name = "TMP";
level = [ "Z2" ];
cat_thresh = [ >=250, >=260, >=270, >=280, >=290, >=300,
<=273,
>=273 ];
hira = {
flag = TRUE;
width = [ 3, 5, 7, 9, 11, 13, 15, 17, 19, 21, 23,
25,
27, 29, 31, 33, 35, 37, 39, 41, 43, 45, 47, 49 ];
vld_thresh = 1.0;
cov_thresh = [ ==0.25 ];
shape = SQUARE;
prob_cat_thresh = [ >=250, >=260, >=270, >=280, >=290,
>=300,
<=273, >=273 ];
}
},
...
On Thu, Jul 2, 2020 at 3:42 PM Raby, John W USA CIV via RT <
met_help at ucar.edu> wrote:
>
> <Caution-url: Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
>
> CLASSIFICATION: UNCLASSIFIED
>
> John -
>
> I edited the attached config file to add the HiRA dictionary to each
> obs.field
> dictionary entry as you described, but it seems like it's not
expecting
> the
> hira dictionary to be where I placed it. Do you see where I went
wrong
> with
> this? The error dialog is also attached.
>
> Thanks and have a great 4th of July!
>
> R/
> John
>
>
>
> -----Original Message-----
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Wednesday, July 1, 2020 11:17 AM
> To: Raby, John W CIV USARMY CCDC ARL (USA)
<john.w.raby2.civ at mail.mil>
> Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
> Point-Stat
> error (UNCLASSIFIED)
>
> All active links contained in this email were disabled. Please
verify the
> identity of the sender, and confirm the authenticity of all links
> contained
> within the message prior to copying and pasting the address to a Web
> browser.
>
>
>
>
> ----
>
> John,
>
> Sorry for the confusion. I led you astray. The HiRA configuration
option
> names are not entirely intuitive.
>
> Please take a look at the README file starting on line 1418:
> Caution-
> Caution-
https://github.com/NCAR/MET/blob/0cee9fbd0b3c691fd3cf4dd1d665f5d2b0db9a37/met/data/config/README#L1418
>
> I modified your hira entry like this:
>
> hira = {
> flag = TRUE; *// set to TRUE/false to turn on/off
this
> logic*
> width = [ 3, 5, 7, 9, 11 ]; *// defines the
neighborhood
> size and must be > 1*
> vld_thresh = 1.0;
> cov_thresh = [ ==0.25 ]; *// this defines the probability
bins to be
> used. You only get to pick one setting here (==0.25 is short for
> >=0,>=0.25,>=0.5,>=0.75,>=1.0)*
> shape = SQUARE;
> prob_cat_thresh = [ >290, >300 ];* // this defines the
threshold(s) for
> which HiRA logic should be applied* }
>
> So you just had things a bit mis-configured. The key thing to note
here is
> that the hira.prob_cat_thresh setting defines the thresholds for
which
> HiRA
> probabilities should be computed. This is different from the
cat_thresh
> setting in the fcst and obs dictionaries. Those are used to compute
> categorical counts and statistics and are currently separate.
>
> It's nice that you don't have to use the same set of thresholds for
> categorical stats and the HiRA logic... it's more configurable that
way.
> However, I do see that it makes things a bit inconvenient. Since
you're
> applying HiRA to many variables and each of those variables has
different
> meaningful thresholds, you'd need to configure the HiRA dictionary
> separately
> for each and every variable.
>
> That's what is meant by this comment... that the hira dictionary can
be
> specified separately for each variable:
>
> *// HiRA verification method// May be set separately in each
"obs.field"
> entry*
>
> So you'd need to copy/paste the HiRA dictionary into every single
> obs.field
> dictionary entry and modify the prob_cat_thresh thresholds
accordingly.
>
> First, please make sure that you can in fact produce HiRA output.
Second,
> I
> can think of a tweak to the logic that might make using HiRA easier.
How
> about, if the hira.prob_cat_thresh array is an empty list, we could
have
> the
> code check to see if cat_thresh in the obs dictionary is non-empty.
And if
> that's non-empty then we'd simply set prob_cat_thresh = cat_thresh.
>
> I imagine that's what you were expecting in the first place... that
> Point-Stat
> would just be using the thresholds from cat_thresh to define the
HiRA
> probability thresholds. It still is nice to have prob_cat_thresh be
> configurable separately, but if left unset, defaulting to cat_thresh
seems
> pretty reasonable to me. Do you think that would be a worthwhile
change?
>
> Thanks,
> John
>
> On Tue, Jun 30, 2020 at 7:12 PM Raby, John W USA CIV via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <Caution-Caution-url:
> > Caution-Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> >
> > I forgot to attach the log file.
> >
> > ________________________________________
> > From: Raby, John W CIV USARMY CCDC ARL (USA)
> > Sent: Tuesday, June 30, 2020 7:00 PM
> > To: met_help at ucar.edu
> > Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > Point-Stat error (UNCLASSIFIED)
> >
> > I edited the config file to use the shortcut and corrected the
> > whitespace issue, but I get the same error.
> > R/
> > John
> >
> >
> >
> > ________________________________________
> > From: Raby, John W CIV USARMY CCDC ARL (USA)
> > Sent: Tuesday, June 30, 2020 6:10 PM
> > To: met_help at ucar.edu
> > Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > Point-Stat error (UNCLASSIFIED)
> >
> > Thanks for taking a look, John. I'll give that a try. Appreciate
the
> > heads-up on a future error in the "desc".
> >
> > R/
> > John
> >
> > ________________________________________
> > From: John Halley Gotway via RT [met_help at ucar.edu]
> > Sent: Tuesday, June 30, 2020 5:11 PM
> > To: Raby, John W CIV USARMY CCDC ARL (USA)
> > Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
> > Point-Stat error (UNCLASSIFIED)
> >
> > All active links contained in this email were disabled. Please
verify
> > the identity of the sender, and confirm the authenticity of all
links
> > contained within the message prior to copying and pasting the
address
> > to a Web browser.
> >
> >
> >
> >
> > ----
> >
> > John,
> >
> > Please try using this shortcut:
> > prob_cat_thresh = [ ==0.25 ];
> >
> > Does that do it?
> >
> > Also, your "desc" setting can't contain embedded whitespace. And
> > Point-Stat should error out and tell you that.
> >
> > John
> >
> > On Tue, Jun 30, 2020 at 4:57 PM Raby, John W USA CIV via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > Tue Jun 30 16:56:45 2020: Request 95790 was acted upon.
> > > Transaction: Ticket created by john.w.raby2.civ at mail.mil
> > > Queue: met_help
> > > Subject: MET V9.0 Point-Stat error (UNCLASSIFIED)
> > > Owner: Nobody
> > > Requestors: john.w.raby2.civ at mail.mil
> > > Status: new
> > > Ticket <Caution-Caution-Caution-url: Caution-
> > Caution-Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> > >
> > >
> > > CLASSIFICATION: UNCLASSIFIED
> > >
> > > I keep getting the same error when trying to run Point-Stat with
the
> > > HiRA verification logic enabled. The error involves the
probability
> > > field
> > > thresholds:
> > >
> > > ERROR:check_prob_thresh() -> When verifying a probability field,
you
> > > must select at least 3 thresholds beginning with 0.0 and ending
with
> 1.0.
> > >
> > > During troubleshooting, I've changed the settings for
> > > prob_cat_thresh in the attached config file several times
thinking
> > > that the formatting of my settings was the issue. Maybe I'm on
the
> > > wrong track. Could you take a
> > look
> > > at the config file to check the format or other possible issue
which
> > might
> > > be causing this issue?
> > >
> > > Thanks.
> > >
> > > R/
> > > John
> > >
> > > Mr. John W. Raby
> > > U.S. Army Research Laboratory
> > > White Sands Missile Range, NM 88002
> > > Office:(575) 678-2004 DSN 258-2004
> > > Teleworking: Contact Admin Specialist at (410) 278-6974 FAX
(575)
> > > 678-1230 DSN 258-1230
> > > Email: john.w.raby2.civ at mail.mil
> > >
> > > CLASSIFICATION: UNCLASSIFIED
> > >
> > >
> >
> >
> >
>
> CLASSIFICATION: UNCLASSIFIED
>
>
------------------------------------------------
Subject: MET V9.0 Point-Stat error (UNCLASSIFIED)
From: John Halley Gotway
Time: Thu Jul 02 16:45:42 2020
Yes, that's right, obs=fcst will copy everything over. And it really
doesn't hurt having hira in both. Point-Stat just parses it from the
obs
dictionary.
John
On Thu, Jul 2, 2020 at 4:31 PM Raby, John W USA CIV via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
>
> John -
>
> Thanks for taking a look and your quick reply. I see now about
including
> the hira dictionary inside the field array dictionary. Thanks for
showing
> that below.
>
> So if I have the fcst dictionary only (obs=fcst) in my confiig file
then I
> all I have to do is put the hira dictionary with settings for the
fcst
> variables, correct? I guess I'm "not understanding "copying it to
the obs"
> when I don't explicitly show the obs dictionary.
>
> Good point about limiting the number of threshold and widths for
testing.
>
> R/
> John
>
>
> ________________________________________
> From: John Halley Gotway via RT [met_help at ucar.edu]
> Sent: Thursday, July 02, 2020 4:14 PM
> To: Raby, John W CIV USARMY CCDC ARL (USA)
> Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
> Point-Stat error (UNCLASSIFIED)
>
> All active links contained in this email were disabled. Please
verify the
> identity of the sender, and confirm the authenticity of all links
contained
> within the message prior to copying and pasting the address to a Web
> browser.
>
>
>
>
> ----
>
> John,
>
> Sorry for not answering your previous question. Yes, since you're
using
> obs=fcst, you can just define it in the fcst dictionary and copy it
over to
> the obs.
>
> Yes, I do see a problem in your config file. The hira dictionary
needs to
> be INSIDE of the field array dictionary entry, as shown below.
>
> Also, for initial testing, I'd recommend using a much smaller set of
> thresholds and widths. Running with all of these will take a long
time.
>
> John
>
> obs = {
> field = [
> {
> name = "TMP";
> level = [ "Z2" ];
> cat_thresh = [ >=250, >=260, >=270, >=280, >=290, >=300,
<=273,
> >=273 ];
> hira = {
> flag = TRUE;
> width = [ 3, 5, 7, 9, 11, 13, 15, 17, 19, 21,
23, 25,
> 27, 29, 31, 33, 35, 37, 39, 41, 43, 45, 47, 49 ];
> vld_thresh = 1.0;
> cov_thresh = [ ==0.25 ];
> shape = SQUARE;
> prob_cat_thresh = [ >=250, >=260, >=270, >=280, >=290,
>=300,
> <=273, >=273 ];
> }
> },
> ...
>
>
> On Thu, Jul 2, 2020 at 3:42 PM Raby, John W USA CIV via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <Caution-url: Caution-
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> >
> > CLASSIFICATION: UNCLASSIFIED
> >
> > John -
> >
> > I edited the attached config file to add the HiRA dictionary to
each
> > obs.field
> > dictionary entry as you described, but it seems like it's not
expecting
> > the
> > hira dictionary to be where I placed it. Do you see where I went
wrong
> > with
> > this? The error dialog is also attached.
> >
> > Thanks and have a great 4th of July!
> >
> > R/
> > John
> >
> >
> >
> > -----Original Message-----
> > From: John Halley Gotway via RT <met_help at ucar.edu>
> > Sent: Wednesday, July 1, 2020 11:17 AM
> > To: Raby, John W CIV USARMY CCDC ARL (USA)
<john.w.raby2.civ at mail.mil>
> > Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > Point-Stat
> > error (UNCLASSIFIED)
> >
> > All active links contained in this email were disabled. Please
verify
> the
> > identity of the sender, and confirm the authenticity of all links
> > contained
> > within the message prior to copying and pasting the address to a
Web
> > browser.
> >
> >
> >
> >
> > ----
> >
> > John,
> >
> > Sorry for the confusion. I led you astray. The HiRA configuration
option
> > names are not entirely intuitive.
> >
> > Please take a look at the README file starting on line 1418:
> > Caution-
> > Caution-
>
https://github.com/NCAR/MET/blob/0cee9fbd0b3c691fd3cf4dd1d665f5d2b0db9a37/met/data/config/README#L1418
> >
> > I modified your hira entry like this:
> >
> > hira = {
> > flag = TRUE; *// set to TRUE/false to turn
on/off this
> > logic*
> > width = [ 3, 5, 7, 9, 11 ]; *// defines the
neighborhood
> > size and must be > 1*
> > vld_thresh = 1.0;
> > cov_thresh = [ ==0.25 ]; *// this defines the probability
bins to
> be
> > used. You only get to pick one setting here (==0.25 is short for
> > >=0,>=0.25,>=0.5,>=0.75,>=1.0)*
> > shape = SQUARE;
> > prob_cat_thresh = [ >290, >300 ];* // this defines the
threshold(s)
> for
> > which HiRA logic should be applied* }
> >
> > So you just had things a bit mis-configured. The key thing to note
here
> is
> > that the hira.prob_cat_thresh setting defines the thresholds for
which
> > HiRA
> > probabilities should be computed. This is different from the
cat_thresh
> > setting in the fcst and obs dictionaries. Those are used to
compute
> > categorical counts and statistics and are currently separate.
> >
> > It's nice that you don't have to use the same set of thresholds
for
> > categorical stats and the HiRA logic... it's more configurable
that way.
> > However, I do see that it makes things a bit inconvenient. Since
you're
> > applying HiRA to many variables and each of those variables has
different
> > meaningful thresholds, you'd need to configure the HiRA dictionary
> > separately
> > for each and every variable.
> >
> > That's what is meant by this comment... that the hira dictionary
can be
> > specified separately for each variable:
> >
> > *// HiRA verification method// May be set separately in each
"obs.field"
> > entry*
> >
> > So you'd need to copy/paste the HiRA dictionary into every single
> > obs.field
> > dictionary entry and modify the prob_cat_thresh thresholds
accordingly.
> >
> > First, please make sure that you can in fact produce HiRA output.
Second,
> > I
> > can think of a tweak to the logic that might make using HiRA
easier. How
> > about, if the hira.prob_cat_thresh array is an empty list, we
could have
> > the
> > code check to see if cat_thresh in the obs dictionary is non-
empty. And
> if
> > that's non-empty then we'd simply set prob_cat_thresh =
cat_thresh.
> >
> > I imagine that's what you were expecting in the first place...
that
> > Point-Stat
> > would just be using the thresholds from cat_thresh to define the
HiRA
> > probability thresholds. It still is nice to have prob_cat_thresh
be
> > configurable separately, but if left unset, defaulting to
cat_thresh
> seems
> > pretty reasonable to me. Do you think that would be a worthwhile
change?
> >
> > Thanks,
> > John
> >
> > On Tue, Jun 30, 2020 at 7:12 PM Raby, John W USA CIV via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <Caution-Caution-url:
> > > Caution-Caution-
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> > >
> > > I forgot to attach the log file.
> > >
> > > ________________________________________
> > > From: Raby, John W CIV USARMY CCDC ARL (USA)
> > > Sent: Tuesday, June 30, 2020 7:00 PM
> > > To: met_help at ucar.edu
> > > Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > > Point-Stat error (UNCLASSIFIED)
> > >
> > > I edited the config file to use the shortcut and corrected the
> > > whitespace issue, but I get the same error.
> > > R/
> > > John
> > >
> > >
> > >
> > > ________________________________________
> > > From: Raby, John W CIV USARMY CCDC ARL (USA)
> > > Sent: Tuesday, June 30, 2020 6:10 PM
> > > To: met_help at ucar.edu
> > > Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > > Point-Stat error (UNCLASSIFIED)
> > >
> > > Thanks for taking a look, John. I'll give that a try. Appreciate
the
> > > heads-up on a future error in the "desc".
> > >
> > > R/
> > > John
> > >
> > > ________________________________________
> > > From: John Halley Gotway via RT [met_help at ucar.edu]
> > > Sent: Tuesday, June 30, 2020 5:11 PM
> > > To: Raby, John W CIV USARMY CCDC ARL (USA)
> > > Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
> > > Point-Stat error (UNCLASSIFIED)
> > >
> > > All active links contained in this email were disabled. Please
verify
> > > the identity of the sender, and confirm the authenticity of all
links
> > > contained within the message prior to copying and pasting the
address
> > > to a Web browser.
> > >
> > >
> > >
> > >
> > > ----
> > >
> > > John,
> > >
> > > Please try using this shortcut:
> > > prob_cat_thresh = [ ==0.25 ];
> > >
> > > Does that do it?
> > >
> > > Also, your "desc" setting can't contain embedded whitespace. And
> > > Point-Stat should error out and tell you that.
> > >
> > > John
> > >
> > > On Tue, Jun 30, 2020 at 4:57 PM Raby, John W USA CIV via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > Tue Jun 30 16:56:45 2020: Request 95790 was acted upon.
> > > > Transaction: Ticket created by john.w.raby2.civ at mail.mil
> > > > Queue: met_help
> > > > Subject: MET V9.0 Point-Stat error (UNCLASSIFIED)
> > > > Owner: Nobody
> > > > Requestors: john.w.raby2.civ at mail.mil
> > > > Status: new
> > > > Ticket <Caution-Caution-Caution-url: Caution-
> > > Caution-Caution-
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> > > >
> > > >
> > > > CLASSIFICATION: UNCLASSIFIED
> > > >
> > > > I keep getting the same error when trying to run Point-Stat
with the
> > > > HiRA verification logic enabled. The error involves the
probability
> > > > field
> > > > thresholds:
> > > >
> > > > ERROR:check_prob_thresh() -> When verifying a probability
field, you
> > > > must select at least 3 thresholds beginning with 0.0 and
ending with
> > 1.0.
> > > >
> > > > During troubleshooting, I've changed the settings for
> > > > prob_cat_thresh in the attached config file several times
thinking
> > > > that the formatting of my settings was the issue. Maybe I'm on
the
> > > > wrong track. Could you take a
> > > look
> > > > at the config file to check the format or other possible issue
which
> > > might
> > > > be causing this issue?
> > > >
> > > > Thanks.
> > > >
> > > > R/
> > > > John
> > > >
> > > > Mr. John W. Raby
> > > > U.S. Army Research Laboratory
> > > > White Sands Missile Range, NM 88002
> > > > Office:(575) 678-2004 DSN 258-2004
> > > > Teleworking: Contact Admin Specialist at (410) 278-6974 FAX
(575)
> > > > 678-1230 DSN 258-1230
> > > > Email: john.w.raby2.civ at mail.mil
> > > >
> > > > CLASSIFICATION: UNCLASSIFIED
> > > >
> > > >
> > >
> > >
> > >
> >
> > CLASSIFICATION: UNCLASSIFIED
> >
> >
>
>
>
>
------------------------------------------------
Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0 Point-Stat error (UNCLASSIFIED)
From: Raby, John W USA CIV
Time: Thu Jul 02 18:17:44 2020
John -
I think this particular point has become clearer. I'll continue
testing with a reduced number of widths and thresholds.
Thanks.
R/
John
________________________________________
From: John Halley Gotway via RT [met_help at ucar.edu]
Sent: Thursday, July 02, 2020 4:45 PM
To: Raby, John W CIV USARMY CCDC ARL (USA)
Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
Point-Stat error (UNCLASSIFIED)
All active links contained in this email were disabled. Please verify
the identity of the sender, and confirm the authenticity of all links
contained within the message prior to copying and pasting the address
to a Web browser.
----
Yes, that's right, obs=fcst will copy everything over. And it really
doesn't hurt having hira in both. Point-Stat just parses it from the
obs
dictionary.
John
On Thu, Jul 2, 2020 at 4:31 PM Raby, John W USA CIV via RT <
met_help at ucar.edu> wrote:
>
> <Caution-url: Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
>
> John -
>
> Thanks for taking a look and your quick reply. I see now about
including
> the hira dictionary inside the field array dictionary. Thanks for
showing
> that below.
>
> So if I have the fcst dictionary only (obs=fcst) in my confiig file
then I
> all I have to do is put the hira dictionary with settings for the
fcst
> variables, correct? I guess I'm "not understanding "copying it to
the obs"
> when I don't explicitly show the obs dictionary.
>
> Good point about limiting the number of threshold and widths for
testing.
>
> R/
> John
>
>
> ________________________________________
> From: John Halley Gotway via RT [met_help at ucar.edu]
> Sent: Thursday, July 02, 2020 4:14 PM
> To: Raby, John W CIV USARMY CCDC ARL (USA)
> Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
> Point-Stat error (UNCLASSIFIED)
>
> All active links contained in this email were disabled. Please
verify the
> identity of the sender, and confirm the authenticity of all links
contained
> within the message prior to copying and pasting the address to a Web
> browser.
>
>
>
>
> ----
>
> John,
>
> Sorry for not answering your previous question. Yes, since you're
using
> obs=fcst, you can just define it in the fcst dictionary and copy it
over to
> the obs.
>
> Yes, I do see a problem in your config file. The hira dictionary
needs to
> be INSIDE of the field array dictionary entry, as shown below.
>
> Also, for initial testing, I'd recommend using a much smaller set of
> thresholds and widths. Running with all of these will take a long
time.
>
> John
>
> obs = {
> field = [
> {
> name = "TMP";
> level = [ "Z2" ];
> cat_thresh = [ >=250, >=260, >=270, >=280, >=290, >=300,
<=273,
> >=273 ];
> hira = {
> flag = TRUE;
> width = [ 3, 5, 7, 9, 11, 13, 15, 17, 19, 21,
23, 25,
> 27, 29, 31, 33, 35, 37, 39, 41, 43, 45, 47, 49 ];
> vld_thresh = 1.0;
> cov_thresh = [ ==0.25 ];
> shape = SQUARE;
> prob_cat_thresh = [ >=250, >=260, >=270, >=280, >=290,
>=300,
> <=273, >=273 ];
> }
> },
> ...
>
>
> On Thu, Jul 2, 2020 at 3:42 PM Raby, John W USA CIV via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <Caution-Caution-url: Caution-
> Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> >
> > CLASSIFICATION: UNCLASSIFIED
> >
> > John -
> >
> > I edited the attached config file to add the HiRA dictionary to
each
> > obs.field
> > dictionary entry as you described, but it seems like it's not
expecting
> > the
> > hira dictionary to be where I placed it. Do you see where I went
wrong
> > with
> > this? The error dialog is also attached.
> >
> > Thanks and have a great 4th of July!
> >
> > R/
> > John
> >
> >
> >
> > -----Original Message-----
> > From: John Halley Gotway via RT <met_help at ucar.edu>
> > Sent: Wednesday, July 1, 2020 11:17 AM
> > To: Raby, John W CIV USARMY CCDC ARL (USA)
<john.w.raby2.civ at mail.mil>
> > Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > Point-Stat
> > error (UNCLASSIFIED)
> >
> > All active links contained in this email were disabled. Please
verify
> the
> > identity of the sender, and confirm the authenticity of all links
> > contained
> > within the message prior to copying and pasting the address to a
Web
> > browser.
> >
> >
> >
> >
> > ----
> >
> > John,
> >
> > Sorry for the confusion. I led you astray. The HiRA configuration
option
> > names are not entirely intuitive.
> >
> > Please take a look at the README file starting on line 1418:
> > Caution-
> > Caution-
> Caution-
https://github.com/NCAR/MET/blob/0cee9fbd0b3c691fd3cf4dd1d665f5d2b0db9a37/met/data/config/README#L1418
> >
> > I modified your hira entry like this:
> >
> > hira = {
> > flag = TRUE; *// set to TRUE/false to turn
on/off this
> > logic*
> > width = [ 3, 5, 7, 9, 11 ]; *// defines the
neighborhood
> > size and must be > 1*
> > vld_thresh = 1.0;
> > cov_thresh = [ ==0.25 ]; *// this defines the probability
bins to
> be
> > used. You only get to pick one setting here (==0.25 is short for
> > >=0,>=0.25,>=0.5,>=0.75,>=1.0)*
> > shape = SQUARE;
> > prob_cat_thresh = [ >290, >300 ];* // this defines the
threshold(s)
> for
> > which HiRA logic should be applied* }
> >
> > So you just had things a bit mis-configured. The key thing to note
here
> is
> > that the hira.prob_cat_thresh setting defines the thresholds for
which
> > HiRA
> > probabilities should be computed. This is different from the
cat_thresh
> > setting in the fcst and obs dictionaries. Those are used to
compute
> > categorical counts and statistics and are currently separate.
> >
> > It's nice that you don't have to use the same set of thresholds
for
> > categorical stats and the HiRA logic... it's more configurable
that way.
> > However, I do see that it makes things a bit inconvenient. Since
you're
> > applying HiRA to many variables and each of those variables has
different
> > meaningful thresholds, you'd need to configure the HiRA dictionary
> > separately
> > for each and every variable.
> >
> > That's what is meant by this comment... that the hira dictionary
can be
> > specified separately for each variable:
> >
> > *// HiRA verification method// May be set separately in each
"obs.field"
> > entry*
> >
> > So you'd need to copy/paste the HiRA dictionary into every single
> > obs.field
> > dictionary entry and modify the prob_cat_thresh thresholds
accordingly.
> >
> > First, please make sure that you can in fact produce HiRA output.
Second,
> > I
> > can think of a tweak to the logic that might make using HiRA
easier. How
> > about, if the hira.prob_cat_thresh array is an empty list, we
could have
> > the
> > code check to see if cat_thresh in the obs dictionary is non-
empty. And
> if
> > that's non-empty then we'd simply set prob_cat_thresh =
cat_thresh.
> >
> > I imagine that's what you were expecting in the first place...
that
> > Point-Stat
> > would just be using the thresholds from cat_thresh to define the
HiRA
> > probability thresholds. It still is nice to have prob_cat_thresh
be
> > configurable separately, but if left unset, defaulting to
cat_thresh
> seems
> > pretty reasonable to me. Do you think that would be a worthwhile
change?
> >
> > Thanks,
> > John
> >
> > On Tue, Jun 30, 2020 at 7:12 PM Raby, John W USA CIV via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <Caution-Caution-Caution-url:
> > > Caution-Caution-
> Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> > >
> > > I forgot to attach the log file.
> > >
> > > ________________________________________
> > > From: Raby, John W CIV USARMY CCDC ARL (USA)
> > > Sent: Tuesday, June 30, 2020 7:00 PM
> > > To: met_help at ucar.edu
> > > Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > > Point-Stat error (UNCLASSIFIED)
> > >
> > > I edited the config file to use the shortcut and corrected the
> > > whitespace issue, but I get the same error.
> > > R/
> > > John
> > >
> > >
> > >
> > > ________________________________________
> > > From: Raby, John W CIV USARMY CCDC ARL (USA)
> > > Sent: Tuesday, June 30, 2020 6:10 PM
> > > To: met_help at ucar.edu
> > > Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > > Point-Stat error (UNCLASSIFIED)
> > >
> > > Thanks for taking a look, John. I'll give that a try. Appreciate
the
> > > heads-up on a future error in the "desc".
> > >
> > > R/
> > > John
> > >
> > > ________________________________________
> > > From: John Halley Gotway via RT [met_help at ucar.edu]
> > > Sent: Tuesday, June 30, 2020 5:11 PM
> > > To: Raby, John W CIV USARMY CCDC ARL (USA)
> > > Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
> > > Point-Stat error (UNCLASSIFIED)
> > >
> > > All active links contained in this email were disabled. Please
verify
> > > the identity of the sender, and confirm the authenticity of all
links
> > > contained within the message prior to copying and pasting the
address
> > > to a Web browser.
> > >
> > >
> > >
> > >
> > > ----
> > >
> > > John,
> > >
> > > Please try using this shortcut:
> > > prob_cat_thresh = [ ==0.25 ];
> > >
> > > Does that do it?
> > >
> > > Also, your "desc" setting can't contain embedded whitespace. And
> > > Point-Stat should error out and tell you that.
> > >
> > > John
> > >
> > > On Tue, Jun 30, 2020 at 4:57 PM Raby, John W USA CIV via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > Tue Jun 30 16:56:45 2020: Request 95790 was acted upon.
> > > > Transaction: Ticket created by john.w.raby2.civ at mail.mil
> > > > Queue: met_help
> > > > Subject: MET V9.0 Point-Stat error (UNCLASSIFIED)
> > > > Owner: Nobody
> > > > Requestors: john.w.raby2.civ at mail.mil
> > > > Status: new
> > > > Ticket <Caution-Caution-Caution-Caution-url: Caution-
> > > Caution-Caution-
> Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> > > >
> > > >
> > > > CLASSIFICATION: UNCLASSIFIED
> > > >
> > > > I keep getting the same error when trying to run Point-Stat
with the
> > > > HiRA verification logic enabled. The error involves the
probability
> > > > field
> > > > thresholds:
> > > >
> > > > ERROR:check_prob_thresh() -> When verifying a probability
field, you
> > > > must select at least 3 thresholds beginning with 0.0 and
ending with
> > 1.0.
> > > >
> > > > During troubleshooting, I've changed the settings for
> > > > prob_cat_thresh in the attached config file several times
thinking
> > > > that the formatting of my settings was the issue. Maybe I'm on
the
> > > > wrong track. Could you take a
> > > look
> > > > at the config file to check the format or other possible issue
which
> > > might
> > > > be causing this issue?
> > > >
> > > > Thanks.
> > > >
> > > > R/
> > > > John
> > > >
> > > > Mr. John W. Raby
> > > > U.S. Army Research Laboratory
> > > > White Sands Missile Range, NM 88002
> > > > Office:(575) 678-2004 DSN 258-2004
> > > > Teleworking: Contact Admin Specialist at (410) 278-6974 FAX
(575)
> > > > 678-1230 DSN 258-1230
> > > > Email: john.w.raby2.civ at mail.mil
> > > >
> > > > CLASSIFICATION: UNCLASSIFIED
> > > >
> > > >
> > >
> > >
> > >
> >
> > CLASSIFICATION: UNCLASSIFIED
> >
> >
>
>
>
>
------------------------------------------------
Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0 Point-Stat error (UNCLASSIFIED)
From: Raby, John W USA CIV
Time: Thu Jul 02 19:49:05 2020
John -
How do I handle the HiRA dictionary which is located at the bottom of
the config file just before "// Statistical output types"? I set that
dictionary "ineffective" by the first statement "flag =
FALSE;". Will this work? Seems redundant to have the HiRA dictionary
in two different places.
R/
John
________________________________________
From: John Halley Gotway via RT [met_help at ucar.edu]
Sent: Thursday, July 02, 2020 4:45 PM
To: Raby, John W CIV USARMY CCDC ARL (USA)
Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
Point-Stat error (UNCLASSIFIED)
All active links contained in this email were disabled. Please verify
the identity of the sender, and confirm the authenticity of all links
contained within the message prior to copying and pasting the address
to a Web browser.
----
Yes, that's right, obs=fcst will copy everything over. And it really
doesn't hurt having hira in both. Point-Stat just parses it from the
obs
dictionary.
John
On Thu, Jul 2, 2020 at 4:31 PM Raby, John W USA CIV via RT <
met_help at ucar.edu> wrote:
>
> <Caution-url: Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
>
> John -
>
> Thanks for taking a look and your quick reply. I see now about
including
> the hira dictionary inside the field array dictionary. Thanks for
showing
> that below.
>
> So if I have the fcst dictionary only (obs=fcst) in my confiig file
then I
> all I have to do is put the hira dictionary with settings for the
fcst
> variables, correct? I guess I'm "not understanding "copying it to
the obs"
> when I don't explicitly show the obs dictionary.
>
> Good point about limiting the number of threshold and widths for
testing.
>
> R/
> John
>
>
> ________________________________________
> From: John Halley Gotway via RT [met_help at ucar.edu]
> Sent: Thursday, July 02, 2020 4:14 PM
> To: Raby, John W CIV USARMY CCDC ARL (USA)
> Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
> Point-Stat error (UNCLASSIFIED)
>
> All active links contained in this email were disabled. Please
verify the
> identity of the sender, and confirm the authenticity of all links
contained
> within the message prior to copying and pasting the address to a Web
> browser.
>
>
>
>
> ----
>
> John,
>
> Sorry for not answering your previous question. Yes, since you're
using
> obs=fcst, you can just define it in the fcst dictionary and copy it
over to
> the obs.
>
> Yes, I do see a problem in your config file. The hira dictionary
needs to
> be INSIDE of the field array dictionary entry, as shown below.
>
> Also, for initial testing, I'd recommend using a much smaller set of
> thresholds and widths. Running with all of these will take a long
time.
>
> John
>
> obs = {
> field = [
> {
> name = "TMP";
> level = [ "Z2" ];
> cat_thresh = [ >=250, >=260, >=270, >=280, >=290, >=300,
<=273,
> >=273 ];
> hira = {
> flag = TRUE;
> width = [ 3, 5, 7, 9, 11, 13, 15, 17, 19, 21,
23, 25,
> 27, 29, 31, 33, 35, 37, 39, 41, 43, 45, 47, 49 ];
> vld_thresh = 1.0;
> cov_thresh = [ ==0.25 ];
> shape = SQUARE;
> prob_cat_thresh = [ >=250, >=260, >=270, >=280, >=290,
>=300,
> <=273, >=273 ];
> }
> },
> ...
>
>
> On Thu, Jul 2, 2020 at 3:42 PM Raby, John W USA CIV via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <Caution-Caution-url: Caution-
> Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> >
> > CLASSIFICATION: UNCLASSIFIED
> >
> > John -
> >
> > I edited the attached config file to add the HiRA dictionary to
each
> > obs.field
> > dictionary entry as you described, but it seems like it's not
expecting
> > the
> > hira dictionary to be where I placed it. Do you see where I went
wrong
> > with
> > this? The error dialog is also attached.
> >
> > Thanks and have a great 4th of July!
> >
> > R/
> > John
> >
> >
> >
> > -----Original Message-----
> > From: John Halley Gotway via RT <met_help at ucar.edu>
> > Sent: Wednesday, July 1, 2020 11:17 AM
> > To: Raby, John W CIV USARMY CCDC ARL (USA)
<john.w.raby2.civ at mail.mil>
> > Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > Point-Stat
> > error (UNCLASSIFIED)
> >
> > All active links contained in this email were disabled. Please
verify
> the
> > identity of the sender, and confirm the authenticity of all links
> > contained
> > within the message prior to copying and pasting the address to a
Web
> > browser.
> >
> >
> >
> >
> > ----
> >
> > John,
> >
> > Sorry for the confusion. I led you astray. The HiRA configuration
option
> > names are not entirely intuitive.
> >
> > Please take a look at the README file starting on line 1418:
> > Caution-
> > Caution-
> Caution-
https://github.com/NCAR/MET/blob/0cee9fbd0b3c691fd3cf4dd1d665f5d2b0db9a37/met/data/config/README#L1418
> >
> > I modified your hira entry like this:
> >
> > hira = {
> > flag = TRUE; *// set to TRUE/false to turn
on/off this
> > logic*
> > width = [ 3, 5, 7, 9, 11 ]; *// defines the
neighborhood
> > size and must be > 1*
> > vld_thresh = 1.0;
> > cov_thresh = [ ==0.25 ]; *// this defines the probability
bins to
> be
> > used. You only get to pick one setting here (==0.25 is short for
> > >=0,>=0.25,>=0.5,>=0.75,>=1.0)*
> > shape = SQUARE;
> > prob_cat_thresh = [ >290, >300 ];* // this defines the
threshold(s)
> for
> > which HiRA logic should be applied* }
> >
> > So you just had things a bit mis-configured. The key thing to note
here
> is
> > that the hira.prob_cat_thresh setting defines the thresholds for
which
> > HiRA
> > probabilities should be computed. This is different from the
cat_thresh
> > setting in the fcst and obs dictionaries. Those are used to
compute
> > categorical counts and statistics and are currently separate.
> >
> > It's nice that you don't have to use the same set of thresholds
for
> > categorical stats and the HiRA logic... it's more configurable
that way.
> > However, I do see that it makes things a bit inconvenient. Since
you're
> > applying HiRA to many variables and each of those variables has
different
> > meaningful thresholds, you'd need to configure the HiRA dictionary
> > separately
> > for each and every variable.
> >
> > That's what is meant by this comment... that the hira dictionary
can be
> > specified separately for each variable:
> >
> > *// HiRA verification method// May be set separately in each
"obs.field"
> > entry*
> >
> > So you'd need to copy/paste the HiRA dictionary into every single
> > obs.field
> > dictionary entry and modify the prob_cat_thresh thresholds
accordingly.
> >
> > First, please make sure that you can in fact produce HiRA output.
Second,
> > I
> > can think of a tweak to the logic that might make using HiRA
easier. How
> > about, if the hira.prob_cat_thresh array is an empty list, we
could have
> > the
> > code check to see if cat_thresh in the obs dictionary is non-
empty. And
> if
> > that's non-empty then we'd simply set prob_cat_thresh =
cat_thresh.
> >
> > I imagine that's what you were expecting in the first place...
that
> > Point-Stat
> > would just be using the thresholds from cat_thresh to define the
HiRA
> > probability thresholds. It still is nice to have prob_cat_thresh
be
> > configurable separately, but if left unset, defaulting to
cat_thresh
> seems
> > pretty reasonable to me. Do you think that would be a worthwhile
change?
> >
> > Thanks,
> > John
> >
> > On Tue, Jun 30, 2020 at 7:12 PM Raby, John W USA CIV via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <Caution-Caution-Caution-url:
> > > Caution-Caution-
> Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> > >
> > > I forgot to attach the log file.
> > >
> > > ________________________________________
> > > From: Raby, John W CIV USARMY CCDC ARL (USA)
> > > Sent: Tuesday, June 30, 2020 7:00 PM
> > > To: met_help at ucar.edu
> > > Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > > Point-Stat error (UNCLASSIFIED)
> > >
> > > I edited the config file to use the shortcut and corrected the
> > > whitespace issue, but I get the same error.
> > > R/
> > > John
> > >
> > >
> > >
> > > ________________________________________
> > > From: Raby, John W CIV USARMY CCDC ARL (USA)
> > > Sent: Tuesday, June 30, 2020 6:10 PM
> > > To: met_help at ucar.edu
> > > Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > > Point-Stat error (UNCLASSIFIED)
> > >
> > > Thanks for taking a look, John. I'll give that a try. Appreciate
the
> > > heads-up on a future error in the "desc".
> > >
> > > R/
> > > John
> > >
> > > ________________________________________
> > > From: John Halley Gotway via RT [met_help at ucar.edu]
> > > Sent: Tuesday, June 30, 2020 5:11 PM
> > > To: Raby, John W CIV USARMY CCDC ARL (USA)
> > > Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
> > > Point-Stat error (UNCLASSIFIED)
> > >
> > > All active links contained in this email were disabled. Please
verify
> > > the identity of the sender, and confirm the authenticity of all
links
> > > contained within the message prior to copying and pasting the
address
> > > to a Web browser.
> > >
> > >
> > >
> > >
> > > ----
> > >
> > > John,
> > >
> > > Please try using this shortcut:
> > > prob_cat_thresh = [ ==0.25 ];
> > >
> > > Does that do it?
> > >
> > > Also, your "desc" setting can't contain embedded whitespace. And
> > > Point-Stat should error out and tell you that.
> > >
> > > John
> > >
> > > On Tue, Jun 30, 2020 at 4:57 PM Raby, John W USA CIV via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > Tue Jun 30 16:56:45 2020: Request 95790 was acted upon.
> > > > Transaction: Ticket created by john.w.raby2.civ at mail.mil
> > > > Queue: met_help
> > > > Subject: MET V9.0 Point-Stat error (UNCLASSIFIED)
> > > > Owner: Nobody
> > > > Requestors: john.w.raby2.civ at mail.mil
> > > > Status: new
> > > > Ticket <Caution-Caution-Caution-Caution-url: Caution-
> > > Caution-Caution-
> Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> > > >
> > > >
> > > > CLASSIFICATION: UNCLASSIFIED
> > > >
> > > > I keep getting the same error when trying to run Point-Stat
with the
> > > > HiRA verification logic enabled. The error involves the
probability
> > > > field
> > > > thresholds:
> > > >
> > > > ERROR:check_prob_thresh() -> When verifying a probability
field, you
> > > > must select at least 3 thresholds beginning with 0.0 and
ending with
> > 1.0.
> > > >
> > > > During troubleshooting, I've changed the settings for
> > > > prob_cat_thresh in the attached config file several times
thinking
> > > > that the formatting of my settings was the issue. Maybe I'm on
the
> > > > wrong track. Could you take a
> > > look
> > > > at the config file to check the format or other possible issue
which
> > > might
> > > > be causing this issue?
> > > >
> > > > Thanks.
> > > >
> > > > R/
> > > > John
> > > >
> > > > Mr. John W. Raby
> > > > U.S. Army Research Laboratory
> > > > White Sands Missile Range, NM 88002
> > > > Office:(575) 678-2004 DSN 258-2004
> > > > Teleworking: Contact Admin Specialist at (410) 278-6974 FAX
(575)
> > > > 678-1230 DSN 258-1230
> > > > Email: john.w.raby2.civ at mail.mil
> > > >
> > > > CLASSIFICATION: UNCLASSIFIED
> > > >
> > > >
> > >
> > >
> > >
> >
> > CLASSIFICATION: UNCLASSIFIED
> >
> >
>
>
>
>
------------------------------------------------
Subject: MET V9.0 Point-Stat error (UNCLASSIFIED)
From: Raby, John W USA CIV
Time: Thu Jul 02 20:49:30 2020
John -
Using your suggestion below, I was able to get Point-Stat to run. See
attached log file and config file. I haven't looked at the output yet,
but it was generated and there are no 0-sized files.
Thanks for your help in getting it running!
R/
John
________________________________________
From: John Halley Gotway via RT [met_help at ucar.edu]
Sent: Thursday, July 02, 2020 4:45 PM
To: Raby, John W CIV USARMY CCDC ARL (USA)
Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
Point-Stat error (UNCLASSIFIED)
All active links contained in this email were disabled. Please verify
the identity of the sender, and confirm the authenticity of all links
contained within the message prior to copying and pasting the address
to a Web browser.
----
Yes, that's right, obs=fcst will copy everything over. And it really
doesn't hurt having hira in both. Point-Stat just parses it from the
obs
dictionary.
John
On Thu, Jul 2, 2020 at 4:31 PM Raby, John W USA CIV via RT <
met_help at ucar.edu> wrote:
>
> <Caution-url: Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
>
> John -
>
> Thanks for taking a look and your quick reply. I see now about
including
> the hira dictionary inside the field array dictionary. Thanks for
showing
> that below.
>
> So if I have the fcst dictionary only (obs=fcst) in my confiig file
then I
> all I have to do is put the hira dictionary with settings for the
fcst
> variables, correct? I guess I'm "not understanding "copying it to
the obs"
> when I don't explicitly show the obs dictionary.
>
> Good point about limiting the number of threshold and widths for
testing.
>
> R/
> John
>
>
> ________________________________________
> From: John Halley Gotway via RT [met_help at ucar.edu]
> Sent: Thursday, July 02, 2020 4:14 PM
> To: Raby, John W CIV USARMY CCDC ARL (USA)
> Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
> Point-Stat error (UNCLASSIFIED)
>
> All active links contained in this email were disabled. Please
verify the
> identity of the sender, and confirm the authenticity of all links
contained
> within the message prior to copying and pasting the address to a Web
> browser.
>
>
>
>
> ----
>
> John,
>
> Sorry for not answering your previous question. Yes, since you're
using
> obs=fcst, you can just define it in the fcst dictionary and copy it
over to
> the obs.
>
> Yes, I do see a problem in your config file. The hira dictionary
needs to
> be INSIDE of the field array dictionary entry, as shown below.
>
> Also, for initial testing, I'd recommend using a much smaller set of
> thresholds and widths. Running with all of these will take a long
time.
>
> John
>
> obs = {
> field = [
> {
> name = "TMP";
> level = [ "Z2" ];
> cat_thresh = [ >=250, >=260, >=270, >=280, >=290, >=300,
<=273,
> >=273 ];
> hira = {
> flag = TRUE;
> width = [ 3, 5, 7, 9, 11, 13, 15, 17, 19, 21,
23, 25,
> 27, 29, 31, 33, 35, 37, 39, 41, 43, 45, 47, 49 ];
> vld_thresh = 1.0;
> cov_thresh = [ ==0.25 ];
> shape = SQUARE;
> prob_cat_thresh = [ >=250, >=260, >=270, >=280, >=290,
>=300,
> <=273, >=273 ];
> }
> },
> ...
>
>
> On Thu, Jul 2, 2020 at 3:42 PM Raby, John W USA CIV via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <Caution-Caution-url: Caution-
> Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> >
> > CLASSIFICATION: UNCLASSIFIED
> >
> > John -
> >
> > I edited the attached config file to add the HiRA dictionary to
each
> > obs.field
> > dictionary entry as you described, but it seems like it's not
expecting
> > the
> > hira dictionary to be where I placed it. Do you see where I went
wrong
> > with
> > this? The error dialog is also attached.
> >
> > Thanks and have a great 4th of July!
> >
> > R/
> > John
> >
> >
> >
> > -----Original Message-----
> > From: John Halley Gotway via RT <met_help at ucar.edu>
> > Sent: Wednesday, July 1, 2020 11:17 AM
> > To: Raby, John W CIV USARMY CCDC ARL (USA)
<john.w.raby2.civ at mail.mil>
> > Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > Point-Stat
> > error (UNCLASSIFIED)
> >
> > All active links contained in this email were disabled. Please
verify
> the
> > identity of the sender, and confirm the authenticity of all links
> > contained
> > within the message prior to copying and pasting the address to a
Web
> > browser.
> >
> >
> >
> >
> > ----
> >
> > John,
> >
> > Sorry for the confusion. I led you astray. The HiRA configuration
option
> > names are not entirely intuitive.
> >
> > Please take a look at the README file starting on line 1418:
> > Caution-
> > Caution-
> Caution-
https://github.com/NCAR/MET/blob/0cee9fbd0b3c691fd3cf4dd1d665f5d2b0db9a37/met/data/config/README#L1418
> >
> > I modified your hira entry like this:
> >
> > hira = {
> > flag = TRUE; *// set to TRUE/false to turn
on/off this
> > logic*
> > width = [ 3, 5, 7, 9, 11 ]; *// defines the
neighborhood
> > size and must be > 1*
> > vld_thresh = 1.0;
> > cov_thresh = [ ==0.25 ]; *// this defines the probability
bins to
> be
> > used. You only get to pick one setting here (==0.25 is short for
> > >=0,>=0.25,>=0.5,>=0.75,>=1.0)*
> > shape = SQUARE;
> > prob_cat_thresh = [ >290, >300 ];* // this defines the
threshold(s)
> for
> > which HiRA logic should be applied* }
> >
> > So you just had things a bit mis-configured. The key thing to note
here
> is
> > that the hira.prob_cat_thresh setting defines the thresholds for
which
> > HiRA
> > probabilities should be computed. This is different from the
cat_thresh
> > setting in the fcst and obs dictionaries. Those are used to
compute
> > categorical counts and statistics and are currently separate.
> >
> > It's nice that you don't have to use the same set of thresholds
for
> > categorical stats and the HiRA logic... it's more configurable
that way.
> > However, I do see that it makes things a bit inconvenient. Since
you're
> > applying HiRA to many variables and each of those variables has
different
> > meaningful thresholds, you'd need to configure the HiRA dictionary
> > separately
> > for each and every variable.
> >
> > That's what is meant by this comment... that the hira dictionary
can be
> > specified separately for each variable:
> >
> > *// HiRA verification method// May be set separately in each
"obs.field"
> > entry*
> >
> > So you'd need to copy/paste the HiRA dictionary into every single
> > obs.field
> > dictionary entry and modify the prob_cat_thresh thresholds
accordingly.
> >
> > First, please make sure that you can in fact produce HiRA output.
Second,
> > I
> > can think of a tweak to the logic that might make using HiRA
easier. How
> > about, if the hira.prob_cat_thresh array is an empty list, we
could have
> > the
> > code check to see if cat_thresh in the obs dictionary is non-
empty. And
> if
> > that's non-empty then we'd simply set prob_cat_thresh =
cat_thresh.
> >
> > I imagine that's what you were expecting in the first place...
that
> > Point-Stat
> > would just be using the thresholds from cat_thresh to define the
HiRA
> > probability thresholds. It still is nice to have prob_cat_thresh
be
> > configurable separately, but if left unset, defaulting to
cat_thresh
> seems
> > pretty reasonable to me. Do you think that would be a worthwhile
change?
> >
> > Thanks,
> > John
> >
> > On Tue, Jun 30, 2020 at 7:12 PM Raby, John W USA CIV via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <Caution-Caution-Caution-url:
> > > Caution-Caution-
> Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> > >
> > > I forgot to attach the log file.
> > >
> > > ________________________________________
> > > From: Raby, John W CIV USARMY CCDC ARL (USA)
> > > Sent: Tuesday, June 30, 2020 7:00 PM
> > > To: met_help at ucar.edu
> > > Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > > Point-Stat error (UNCLASSIFIED)
> > >
> > > I edited the config file to use the shortcut and corrected the
> > > whitespace issue, but I get the same error.
> > > R/
> > > John
> > >
> > >
> > >
> > > ________________________________________
> > > From: Raby, John W CIV USARMY CCDC ARL (USA)
> > > Sent: Tuesday, June 30, 2020 6:10 PM
> > > To: met_help at ucar.edu
> > > Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > > Point-Stat error (UNCLASSIFIED)
> > >
> > > Thanks for taking a look, John. I'll give that a try. Appreciate
the
> > > heads-up on a future error in the "desc".
> > >
> > > R/
> > > John
> > >
> > > ________________________________________
> > > From: John Halley Gotway via RT [met_help at ucar.edu]
> > > Sent: Tuesday, June 30, 2020 5:11 PM
> > > To: Raby, John W CIV USARMY CCDC ARL (USA)
> > > Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
> > > Point-Stat error (UNCLASSIFIED)
> > >
> > > All active links contained in this email were disabled. Please
verify
> > > the identity of the sender, and confirm the authenticity of all
links
> > > contained within the message prior to copying and pasting the
address
> > > to a Web browser.
> > >
> > >
> > >
> > >
> > > ----
> > >
> > > John,
> > >
> > > Please try using this shortcut:
> > > prob_cat_thresh = [ ==0.25 ];
> > >
> > > Does that do it?
> > >
> > > Also, your "desc" setting can't contain embedded whitespace. And
> > > Point-Stat should error out and tell you that.
> > >
> > > John
> > >
> > > On Tue, Jun 30, 2020 at 4:57 PM Raby, John W USA CIV via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > Tue Jun 30 16:56:45 2020: Request 95790 was acted upon.
> > > > Transaction: Ticket created by john.w.raby2.civ at mail.mil
> > > > Queue: met_help
> > > > Subject: MET V9.0 Point-Stat error (UNCLASSIFIED)
> > > > Owner: Nobody
> > > > Requestors: john.w.raby2.civ at mail.mil
> > > > Status: new
> > > > Ticket <Caution-Caution-Caution-Caution-url: Caution-
> > > Caution-Caution-
> Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> > > >
> > > >
> > > > CLASSIFICATION: UNCLASSIFIED
> > > >
> > > > I keep getting the same error when trying to run Point-Stat
with the
> > > > HiRA verification logic enabled. The error involves the
probability
> > > > field
> > > > thresholds:
> > > >
> > > > ERROR:check_prob_thresh() -> When verifying a probability
field, you
> > > > must select at least 3 thresholds beginning with 0.0 and
ending with
> > 1.0.
> > > >
> > > > During troubleshooting, I've changed the settings for
> > > > prob_cat_thresh in the attached config file several times
thinking
> > > > that the formatting of my settings was the issue. Maybe I'm on
the
> > > > wrong track. Could you take a
> > > look
> > > > at the config file to check the format or other possible issue
which
> > > might
> > > > be causing this issue?
> > > >
> > > > Thanks.
> > > >
> > > > R/
> > > > John
> > > >
> > > > Mr. John W. Raby
> > > > U.S. Army Research Laboratory
> > > > White Sands Missile Range, NM 88002
> > > > Office:(575) 678-2004 DSN 258-2004
> > > > Teleworking: Contact Admin Specialist at (410) 278-6974 FAX
(575)
> > > > 678-1230 DSN 258-1230
> > > > Email: john.w.raby2.civ at mail.mil
> > > >
> > > > CLASSIFICATION: UNCLASSIFIED
> > > >
> > > >
> > >
> > >
> > >
> >
> > CLASSIFICATION: UNCLASSIFIED
> >
> >
>
>
>
>
------------------------------------------------
Subject: MET V9.0 Point-Stat error (UNCLASSIFIED)
From: John Halley Gotway
Time: Fri Jul 03 18:06:12 2020
John,
I was looking at the code today and suspect that I've misled you a
little
bit. I was confused about it myself. I updated this GitHub issue to
clarify:
https://github.com/NCAR/MET/issues/1400
Upon review, you're correct that Point-Stat DOES use the cat_thresh
settings from the fcst and obs dictionaries while processing HiRA. It
uses
those thresholds to derive probability fields which are evaluated in
the
output PCT, PTSD, PJC, and PRC line types. It also processes the
neighborhood of points as a "spatial ensemble"... and from those
ensemble
points, it computes the ECNT and RPS line types. And it's in the
computation of the RPS line type that the prob_cat_thresh setting is
used.
So prob_cat_thresh really only impacts the RPS line type.
I am still making the change we discussed, but I wanted to clarify how
that
change is actually being used (since I was confused about that
myself).
You can see the tweak to the logic on line 1841 of:
https://github.com/NCAR/MET/blob/05f2e7d56bae3af092a69a5a9db4867f202eaae0/met/src/tools/core/point_stat/point_stat.cc#L1841
Thanks,
John
On Thu, Jul 2, 2020 at 8:50 PM Raby, John W USA CIV via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
>
> John -
>
> Using your suggestion below, I was able to get Point-Stat to run.
See
> attached log file and config file. I haven't looked at the output
yet, but
> it was generated and there are no 0-sized files.
>
> Thanks for your help in getting it running!
>
> R/
> John
>
> ________________________________________
> From: John Halley Gotway via RT [met_help at ucar.edu]
> Sent: Thursday, July 02, 2020 4:45 PM
> To: Raby, John W CIV USARMY CCDC ARL (USA)
> Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
> Point-Stat error (UNCLASSIFIED)
>
> All active links contained in this email were disabled. Please
verify the
> identity of the sender, and confirm the authenticity of all links
contained
> within the message prior to copying and pasting the address to a Web
> browser.
>
>
>
>
> ----
>
> Yes, that's right, obs=fcst will copy everything over. And it really
> doesn't hurt having hira in both. Point-Stat just parses it from the
obs
> dictionary.
>
> John
>
> On Thu, Jul 2, 2020 at 4:31 PM Raby, John W USA CIV via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <Caution-url: Caution-
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> >
> > John -
> >
> > Thanks for taking a look and your quick reply. I see now about
including
> > the hira dictionary inside the field array dictionary. Thanks for
showing
> > that below.
> >
> > So if I have the fcst dictionary only (obs=fcst) in my confiig
file then
> I
> > all I have to do is put the hira dictionary with settings for the
fcst
> > variables, correct? I guess I'm "not understanding "copying it to
the
> obs"
> > when I don't explicitly show the obs dictionary.
> >
> > Good point about limiting the number of threshold and widths for
testing.
> >
> > R/
> > John
> >
> >
> > ________________________________________
> > From: John Halley Gotway via RT [met_help at ucar.edu]
> > Sent: Thursday, July 02, 2020 4:14 PM
> > To: Raby, John W CIV USARMY CCDC ARL (USA)
> > Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > Point-Stat error (UNCLASSIFIED)
> >
> > All active links contained in this email were disabled. Please
verify
> the
> > identity of the sender, and confirm the authenticity of all links
> contained
> > within the message prior to copying and pasting the address to a
Web
> > browser.
> >
> >
> >
> >
> > ----
> >
> > John,
> >
> > Sorry for not answering your previous question. Yes, since you're
using
> > obs=fcst, you can just define it in the fcst dictionary and copy
it over
> to
> > the obs.
> >
> > Yes, I do see a problem in your config file. The hira dictionary
needs to
> > be INSIDE of the field array dictionary entry, as shown below.
> >
> > Also, for initial testing, I'd recommend using a much smaller set
of
> > thresholds and widths. Running with all of these will take a long
time.
> >
> > John
> >
> > obs = {
> > field = [
> > {
> > name = "TMP";
> > level = [ "Z2" ];
> > cat_thresh = [ >=250, >=260, >=270, >=280, >=290, >=300,
<=273,
> > >=273 ];
> > hira = {
> > flag = TRUE;
> > width = [ 3, 5, 7, 9, 11, 13, 15, 17, 19, 21,
23, 25,
> > 27, 29, 31, 33, 35, 37, 39, 41, 43, 45, 47, 49 ];
> > vld_thresh = 1.0;
> > cov_thresh = [ ==0.25 ];
> > shape = SQUARE;
> > prob_cat_thresh = [ >=250, >=260, >=270, >=280, >=290,
>=300,
> > <=273, >=273 ];
> > }
> > },
> > ...
> >
> >
> > On Thu, Jul 2, 2020 at 3:42 PM Raby, John W USA CIV via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <Caution-Caution-url: Caution-
> > Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> > >
> > > CLASSIFICATION: UNCLASSIFIED
> > >
> > > John -
> > >
> > > I edited the attached config file to add the HiRA dictionary to
each
> > > obs.field
> > > dictionary entry as you described, but it seems like it's not
expecting
> > > the
> > > hira dictionary to be where I placed it. Do you see where I went
wrong
> > > with
> > > this? The error dialog is also attached.
> > >
> > > Thanks and have a great 4th of July!
> > >
> > > R/
> > > John
> > >
> > >
> > >
> > > -----Original Message-----
> > > From: John Halley Gotway via RT <met_help at ucar.edu>
> > > Sent: Wednesday, July 1, 2020 11:17 AM
> > > To: Raby, John W CIV USARMY CCDC ARL (USA)
<john.w.raby2.civ at mail.mil>
> > > Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > > Point-Stat
> > > error (UNCLASSIFIED)
> > >
> > > All active links contained in this email were disabled. Please
verify
> > the
> > > identity of the sender, and confirm the authenticity of all
links
> > > contained
> > > within the message prior to copying and pasting the address to a
Web
> > > browser.
> > >
> > >
> > >
> > >
> > > ----
> > >
> > > John,
> > >
> > > Sorry for the confusion. I led you astray. The HiRA
configuration
> option
> > > names are not entirely intuitive.
> > >
> > > Please take a look at the README file starting on line 1418:
> > > Caution-
> > > Caution-
> > Caution-
>
https://github.com/NCAR/MET/blob/0cee9fbd0b3c691fd3cf4dd1d665f5d2b0db9a37/met/data/config/README#L1418
> > >
> > > I modified your hira entry like this:
> > >
> > > hira = {
> > > flag = TRUE; *// set to TRUE/false to turn
on/off
> this
> > > logic*
> > > width = [ 3, 5, 7, 9, 11 ]; *// defines the
> neighborhood
> > > size and must be > 1*
> > > vld_thresh = 1.0;
> > > cov_thresh = [ ==0.25 ]; *// this defines the probability
bins
> to
> > be
> > > used. You only get to pick one setting here (==0.25 is short for
> > > >=0,>=0.25,>=0.5,>=0.75,>=1.0)*
> > > shape = SQUARE;
> > > prob_cat_thresh = [ >290, >300 ];* // this defines the
threshold(s)
> > for
> > > which HiRA logic should be applied* }
> > >
> > > So you just had things a bit mis-configured. The key thing to
note here
> > is
> > > that the hira.prob_cat_thresh setting defines the thresholds for
which
> > > HiRA
> > > probabilities should be computed. This is different from the
cat_thresh
> > > setting in the fcst and obs dictionaries. Those are used to
compute
> > > categorical counts and statistics and are currently separate.
> > >
> > > It's nice that you don't have to use the same set of thresholds
for
> > > categorical stats and the HiRA logic... it's more configurable
that
> way.
> > > However, I do see that it makes things a bit inconvenient. Since
you're
> > > applying HiRA to many variables and each of those variables has
> different
> > > meaningful thresholds, you'd need to configure the HiRA
dictionary
> > > separately
> > > for each and every variable.
> > >
> > > That's what is meant by this comment... that the hira dictionary
can be
> > > specified separately for each variable:
> > >
> > > *// HiRA verification method// May be set separately in each
> "obs.field"
> > > entry*
> > >
> > > So you'd need to copy/paste the HiRA dictionary into every
single
> > > obs.field
> > > dictionary entry and modify the prob_cat_thresh thresholds
accordingly.
> > >
> > > First, please make sure that you can in fact produce HiRA
output.
> Second,
> > > I
> > > can think of a tweak to the logic that might make using HiRA
easier.
> How
> > > about, if the hira.prob_cat_thresh array is an empty list, we
could
> have
> > > the
> > > code check to see if cat_thresh in the obs dictionary is non-
empty. And
> > if
> > > that's non-empty then we'd simply set prob_cat_thresh =
cat_thresh.
> > >
> > > I imagine that's what you were expecting in the first place...
that
> > > Point-Stat
> > > would just be using the thresholds from cat_thresh to define the
HiRA
> > > probability thresholds. It still is nice to have prob_cat_thresh
be
> > > configurable separately, but if left unset, defaulting to
cat_thresh
> > seems
> > > pretty reasonable to me. Do you think that would be a worthwhile
> change?
> > >
> > > Thanks,
> > > John
> > >
> > > On Tue, Jun 30, 2020 at 7:12 PM Raby, John W USA CIV via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <Caution-Caution-Caution-url:
> > > > Caution-Caution-
> > Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> > > >
> > > > I forgot to attach the log file.
> > > >
> > > > ________________________________________
> > > > From: Raby, John W CIV USARMY CCDC ARL (USA)
> > > > Sent: Tuesday, June 30, 2020 7:00 PM
> > > > To: met_help at ucar.edu
> > > > Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > > > Point-Stat error (UNCLASSIFIED)
> > > >
> > > > I edited the config file to use the shortcut and corrected the
> > > > whitespace issue, but I get the same error.
> > > > R/
> > > > John
> > > >
> > > >
> > > >
> > > > ________________________________________
> > > > From: Raby, John W CIV USARMY CCDC ARL (USA)
> > > > Sent: Tuesday, June 30, 2020 6:10 PM
> > > > To: met_help at ucar.edu
> > > > Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > > > Point-Stat error (UNCLASSIFIED)
> > > >
> > > > Thanks for taking a look, John. I'll give that a try.
Appreciate the
> > > > heads-up on a future error in the "desc".
> > > >
> > > > R/
> > > > John
> > > >
> > > > ________________________________________
> > > > From: John Halley Gotway via RT [met_help at ucar.edu]
> > > > Sent: Tuesday, June 30, 2020 5:11 PM
> > > > To: Raby, John W CIV USARMY CCDC ARL (USA)
> > > > Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > > > Point-Stat error (UNCLASSIFIED)
> > > >
> > > > All active links contained in this email were disabled.
Please
> verify
> > > > the identity of the sender, and confirm the authenticity of
all links
> > > > contained within the message prior to copying and pasting the
address
> > > > to a Web browser.
> > > >
> > > >
> > > >
> > > >
> > > > ----
> > > >
> > > > John,
> > > >
> > > > Please try using this shortcut:
> > > > prob_cat_thresh = [ ==0.25 ];
> > > >
> > > > Does that do it?
> > > >
> > > > Also, your "desc" setting can't contain embedded whitespace.
And
> > > > Point-Stat should error out and tell you that.
> > > >
> > > > John
> > > >
> > > > On Tue, Jun 30, 2020 at 4:57 PM Raby, John W USA CIV via RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > Tue Jun 30 16:56:45 2020: Request 95790 was acted upon.
> > > > > Transaction: Ticket created by john.w.raby2.civ at mail.mil
> > > > > Queue: met_help
> > > > > Subject: MET V9.0 Point-Stat error (UNCLASSIFIED)
> > > > > Owner: Nobody
> > > > > Requestors: john.w.raby2.civ at mail.mil
> > > > > Status: new
> > > > > Ticket <Caution-Caution-Caution-Caution-url: Caution-
> > > > Caution-Caution-
> > Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> > > > >
> > > > >
> > > > > CLASSIFICATION: UNCLASSIFIED
> > > > >
> > > > > I keep getting the same error when trying to run Point-Stat
with
> the
> > > > > HiRA verification logic enabled. The error involves the
probability
> > > > > field
> > > > > thresholds:
> > > > >
> > > > > ERROR:check_prob_thresh() -> When verifying a probability
field,
> you
> > > > > must select at least 3 thresholds beginning with 0.0 and
ending
> with
> > > 1.0.
> > > > >
> > > > > During troubleshooting, I've changed the settings for
> > > > > prob_cat_thresh in the attached config file several times
thinking
> > > > > that the formatting of my settings was the issue. Maybe I'm
on the
> > > > > wrong track. Could you take a
> > > > look
> > > > > at the config file to check the format or other possible
issue
> which
> > > > might
> > > > > be causing this issue?
> > > > >
> > > > > Thanks.
> > > > >
> > > > > R/
> > > > > John
> > > > >
> > > > > Mr. John W. Raby
> > > > > U.S. Army Research Laboratory
> > > > > White Sands Missile Range, NM 88002
> > > > > Office:(575) 678-2004 DSN 258-2004
> > > > > Teleworking: Contact Admin Specialist at (410) 278-6974 FAX
(575)
> > > > > 678-1230 DSN 258-1230
> > > > > Email: john.w.raby2.civ at mail.mil
> > > > >
> > > > > CLASSIFICATION: UNCLASSIFIED
> > > > >
> > > > >
> > > >
> > > >
> > > >
> > >
> > > CLASSIFICATION: UNCLASSIFIED
> > >
> > >
> >
> >
> >
> >
>
>
>
------------------------------------------------
Subject: MET V9.0 Point-Stat error (UNCLASSIFIED)
From: Raby, John W USA CIV
Time: Fri Jul 03 18:44:03 2020
John -
Your GitHub issue and the logic really help to better understand how
the HiRA methodology works. I was also pretty confused about that.
Maybe a re-look at the User's Guide description would be good to
consider too. I've attached the RPS and ECNT output file in case you
wanted to see them. Thank you for working through this with me.
Have a great 4th of July!
R/
John
________________________________________
From: John Halley Gotway via RT [met_help at ucar.edu]
Sent: Friday, July 03, 2020 6:06 PM
To: Raby, John W CIV USARMY CCDC ARL (USA)
Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
Point-Stat error (UNCLASSIFIED)
All active links contained in this email were disabled. Please verify
the identity of the sender, and confirm the authenticity of all links
contained within the message prior to copying and pasting the address
to a Web browser.
----
John,
I was looking at the code today and suspect that I've misled you a
little
bit. I was confused about it myself. I updated this GitHub issue to
clarify:
Caution-https://github.com/NCAR/MET/issues/1400
Upon review, you're correct that Point-Stat DOES use the cat_thresh
settings from the fcst and obs dictionaries while processing HiRA. It
uses
those thresholds to derive probability fields which are evaluated in
the
output PCT, PTSD, PJC, and PRC line types. It also processes the
neighborhood of points as a "spatial ensemble"... and from those
ensemble
points, it computes the ECNT and RPS line types. And it's in the
computation of the RPS line type that the prob_cat_thresh setting is
used.
So prob_cat_thresh really only impacts the RPS line type.
I am still making the change we discussed, but I wanted to clarify how
that
change is actually being used (since I was confused about that
myself).
You can see the tweak to the logic on line 1841 of:
Caution-
https://github.com/NCAR/MET/blob/05f2e7d56bae3af092a69a5a9db4867f202eaae0/met/src/tools/core/point_stat/point_stat.cc#L1841
Thanks,
John
On Thu, Jul 2, 2020 at 8:50 PM Raby, John W USA CIV via RT <
met_help at ucar.edu> wrote:
>
> <Caution-url: Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
>
> John -
>
> Using your suggestion below, I was able to get Point-Stat to run.
See
> attached log file and config file. I haven't looked at the output
yet, but
> it was generated and there are no 0-sized files.
>
> Thanks for your help in getting it running!
>
> R/
> John
>
> ________________________________________
> From: John Halley Gotway via RT [met_help at ucar.edu]
> Sent: Thursday, July 02, 2020 4:45 PM
> To: Raby, John W CIV USARMY CCDC ARL (USA)
> Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
> Point-Stat error (UNCLASSIFIED)
>
> All active links contained in this email were disabled. Please
verify the
> identity of the sender, and confirm the authenticity of all links
contained
> within the message prior to copying and pasting the address to a Web
> browser.
>
>
>
>
> ----
>
> Yes, that's right, obs=fcst will copy everything over. And it really
> doesn't hurt having hira in both. Point-Stat just parses it from the
obs
> dictionary.
>
> John
>
> On Thu, Jul 2, 2020 at 4:31 PM Raby, John W USA CIV via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <Caution-Caution-url: Caution-
> Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> >
> > John -
> >
> > Thanks for taking a look and your quick reply. I see now about
including
> > the hira dictionary inside the field array dictionary. Thanks for
showing
> > that below.
> >
> > So if I have the fcst dictionary only (obs=fcst) in my confiig
file then
> I
> > all I have to do is put the hira dictionary with settings for the
fcst
> > variables, correct? I guess I'm "not understanding "copying it to
the
> obs"
> > when I don't explicitly show the obs dictionary.
> >
> > Good point about limiting the number of threshold and widths for
testing.
> >
> > R/
> > John
> >
> >
> > ________________________________________
> > From: John Halley Gotway via RT [met_help at ucar.edu]
> > Sent: Thursday, July 02, 2020 4:14 PM
> > To: Raby, John W CIV USARMY CCDC ARL (USA)
> > Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > Point-Stat error (UNCLASSIFIED)
> >
> > All active links contained in this email were disabled. Please
verify
> the
> > identity of the sender, and confirm the authenticity of all links
> contained
> > within the message prior to copying and pasting the address to a
Web
> > browser.
> >
> >
> >
> >
> > ----
> >
> > John,
> >
> > Sorry for not answering your previous question. Yes, since you're
using
> > obs=fcst, you can just define it in the fcst dictionary and copy
it over
> to
> > the obs.
> >
> > Yes, I do see a problem in your config file. The hira dictionary
needs to
> > be INSIDE of the field array dictionary entry, as shown below.
> >
> > Also, for initial testing, I'd recommend using a much smaller set
of
> > thresholds and widths. Running with all of these will take a long
time.
> >
> > John
> >
> > obs = {
> > field = [
> > {
> > name = "TMP";
> > level = [ "Z2" ];
> > cat_thresh = [ >=250, >=260, >=270, >=280, >=290, >=300,
<=273,
> > >=273 ];
> > hira = {
> > flag = TRUE;
> > width = [ 3, 5, 7, 9, 11, 13, 15, 17, 19, 21,
23, 25,
> > 27, 29, 31, 33, 35, 37, 39, 41, 43, 45, 47, 49 ];
> > vld_thresh = 1.0;
> > cov_thresh = [ ==0.25 ];
> > shape = SQUARE;
> > prob_cat_thresh = [ >=250, >=260, >=270, >=280, >=290,
>=300,
> > <=273, >=273 ];
> > }
> > },
> > ...
> >
> >
> > On Thu, Jul 2, 2020 at 3:42 PM Raby, John W USA CIV via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <Caution-Caution-Caution-url: Caution-
> > Caution-Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> > >
> > > CLASSIFICATION: UNCLASSIFIED
> > >
> > > John -
> > >
> > > I edited the attached config file to add the HiRA dictionary to
each
> > > obs.field
> > > dictionary entry as you described, but it seems like it's not
expecting
> > > the
> > > hira dictionary to be where I placed it. Do you see where I went
wrong
> > > with
> > > this? The error dialog is also attached.
> > >
> > > Thanks and have a great 4th of July!
> > >
> > > R/
> > > John
> > >
> > >
> > >
> > > -----Original Message-----
> > > From: John Halley Gotway via RT <met_help at ucar.edu>
> > > Sent: Wednesday, July 1, 2020 11:17 AM
> > > To: Raby, John W CIV USARMY CCDC ARL (USA)
<john.w.raby2.civ at mail.mil>
> > > Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > > Point-Stat
> > > error (UNCLASSIFIED)
> > >
> > > All active links contained in this email were disabled. Please
verify
> > the
> > > identity of the sender, and confirm the authenticity of all
links
> > > contained
> > > within the message prior to copying and pasting the address to a
Web
> > > browser.
> > >
> > >
> > >
> > >
> > > ----
> > >
> > > John,
> > >
> > > Sorry for the confusion. I led you astray. The HiRA
configuration
> option
> > > names are not entirely intuitive.
> > >
> > > Please take a look at the README file starting on line 1418:
> > > Caution-
> > > Caution-
> > Caution-
> Caution-
https://github.com/NCAR/MET/blob/0cee9fbd0b3c691fd3cf4dd1d665f5d2b0db9a37/met/data/config/README#L1418
> > >
> > > I modified your hira entry like this:
> > >
> > > hira = {
> > > flag = TRUE; *// set to TRUE/false to turn
on/off
> this
> > > logic*
> > > width = [ 3, 5, 7, 9, 11 ]; *// defines the
> neighborhood
> > > size and must be > 1*
> > > vld_thresh = 1.0;
> > > cov_thresh = [ ==0.25 ]; *// this defines the probability
bins
> to
> > be
> > > used. You only get to pick one setting here (==0.25 is short for
> > > >=0,>=0.25,>=0.5,>=0.75,>=1.0)*
> > > shape = SQUARE;
> > > prob_cat_thresh = [ >290, >300 ];* // this defines the
threshold(s)
> > for
> > > which HiRA logic should be applied* }
> > >
> > > So you just had things a bit mis-configured. The key thing to
note here
> > is
> > > that the hira.prob_cat_thresh setting defines the thresholds for
which
> > > HiRA
> > > probabilities should be computed. This is different from the
cat_thresh
> > > setting in the fcst and obs dictionaries. Those are used to
compute
> > > categorical counts and statistics and are currently separate.
> > >
> > > It's nice that you don't have to use the same set of thresholds
for
> > > categorical stats and the HiRA logic... it's more configurable
that
> way.
> > > However, I do see that it makes things a bit inconvenient. Since
you're
> > > applying HiRA to many variables and each of those variables has
> different
> > > meaningful thresholds, you'd need to configure the HiRA
dictionary
> > > separately
> > > for each and every variable.
> > >
> > > That's what is meant by this comment... that the hira dictionary
can be
> > > specified separately for each variable:
> > >
> > > *// HiRA verification method// May be set separately in each
> "obs.field"
> > > entry*
> > >
> > > So you'd need to copy/paste the HiRA dictionary into every
single
> > > obs.field
> > > dictionary entry and modify the prob_cat_thresh thresholds
accordingly.
> > >
> > > First, please make sure that you can in fact produce HiRA
output.
> Second,
> > > I
> > > can think of a tweak to the logic that might make using HiRA
easier.
> How
> > > about, if the hira.prob_cat_thresh array is an empty list, we
could
> have
> > > the
> > > code check to see if cat_thresh in the obs dictionary is non-
empty. And
> > if
> > > that's non-empty then we'd simply set prob_cat_thresh =
cat_thresh.
> > >
> > > I imagine that's what you were expecting in the first place...
that
> > > Point-Stat
> > > would just be using the thresholds from cat_thresh to define the
HiRA
> > > probability thresholds. It still is nice to have prob_cat_thresh
be
> > > configurable separately, but if left unset, defaulting to
cat_thresh
> > seems
> > > pretty reasonable to me. Do you think that would be a worthwhile
> change?
> > >
> > > Thanks,
> > > John
> > >
> > > On Tue, Jun 30, 2020 at 7:12 PM Raby, John W USA CIV via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <Caution-Caution-Caution-Caution-url:
> > > > Caution-Caution-
> > Caution-Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> > > >
> > > > I forgot to attach the log file.
> > > >
> > > > ________________________________________
> > > > From: Raby, John W CIV USARMY CCDC ARL (USA)
> > > > Sent: Tuesday, June 30, 2020 7:00 PM
> > > > To: met_help at ucar.edu
> > > > Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > > > Point-Stat error (UNCLASSIFIED)
> > > >
> > > > I edited the config file to use the shortcut and corrected the
> > > > whitespace issue, but I get the same error.
> > > > R/
> > > > John
> > > >
> > > >
> > > >
> > > > ________________________________________
> > > > From: Raby, John W CIV USARMY CCDC ARL (USA)
> > > > Sent: Tuesday, June 30, 2020 6:10 PM
> > > > To: met_help at ucar.edu
> > > > Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > > > Point-Stat error (UNCLASSIFIED)
> > > >
> > > > Thanks for taking a look, John. I'll give that a try.
Appreciate the
> > > > heads-up on a future error in the "desc".
> > > >
> > > > R/
> > > > John
> > > >
> > > > ________________________________________
> > > > From: John Halley Gotway via RT [met_help at ucar.edu]
> > > > Sent: Tuesday, June 30, 2020 5:11 PM
> > > > To: Raby, John W CIV USARMY CCDC ARL (USA)
> > > > Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > > > Point-Stat error (UNCLASSIFIED)
> > > >
> > > > All active links contained in this email were disabled.
Please
> verify
> > > > the identity of the sender, and confirm the authenticity of
all links
> > > > contained within the message prior to copying and pasting the
address
> > > > to a Web browser.
> > > >
> > > >
> > > >
> > > >
> > > > ----
> > > >
> > > > John,
> > > >
> > > > Please try using this shortcut:
> > > > prob_cat_thresh = [ ==0.25 ];
> > > >
> > > > Does that do it?
> > > >
> > > > Also, your "desc" setting can't contain embedded whitespace.
And
> > > > Point-Stat should error out and tell you that.
> > > >
> > > > John
> > > >
> > > > On Tue, Jun 30, 2020 at 4:57 PM Raby, John W USA CIV via RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > Tue Jun 30 16:56:45 2020: Request 95790 was acted upon.
> > > > > Transaction: Ticket created by john.w.raby2.civ at mail.mil
> > > > > Queue: met_help
> > > > > Subject: MET V9.0 Point-Stat error (UNCLASSIFIED)
> > > > > Owner: Nobody
> > > > > Requestors: john.w.raby2.civ at mail.mil
> > > > > Status: new
> > > > > Ticket <Caution-Caution-Caution-Caution-Caution-url:
Caution-
> > > > Caution-Caution-
> > Caution-Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> > > > >
> > > > >
> > > > > CLASSIFICATION: UNCLASSIFIED
> > > > >
> > > > > I keep getting the same error when trying to run Point-Stat
with
> the
> > > > > HiRA verification logic enabled. The error involves the
probability
> > > > > field
> > > > > thresholds:
> > > > >
> > > > > ERROR:check_prob_thresh() -> When verifying a probability
field,
> you
> > > > > must select at least 3 thresholds beginning with 0.0 and
ending
> with
> > > 1.0.
> > > > >
> > > > > During troubleshooting, I've changed the settings for
> > > > > prob_cat_thresh in the attached config file several times
thinking
> > > > > that the formatting of my settings was the issue. Maybe I'm
on the
> > > > > wrong track. Could you take a
> > > > look
> > > > > at the config file to check the format or other possible
issue
> which
> > > > might
> > > > > be causing this issue?
> > > > >
> > > > > Thanks.
> > > > >
> > > > > R/
> > > > > John
> > > > >
> > > > > Mr. John W. Raby
> > > > > U.S. Army Research Laboratory
> > > > > White Sands Missile Range, NM 88002
> > > > > Office:(575) 678-2004 DSN 258-2004
> > > > > Teleworking: Contact Admin Specialist at (410) 278-6974 FAX
(575)
> > > > > 678-1230 DSN 258-1230
> > > > > Email: john.w.raby2.civ at mail.mil
> > > > >
> > > > > CLASSIFICATION: UNCLASSIFIED
> > > > >
> > > > >
> > > >
> > > >
> > > >
> > >
> > > CLASSIFICATION: UNCLASSIFIED
> > >
> > >
> >
> >
> >
> >
>
>
>
------------------------------------------------
Subject: MET V9.0 Point-Stat error (UNCLASSIFIED)
From: Raby, John W USA CIV
Time: Fri Jul 03 18:44:03 2020
VERSION MODEL DESC FCST_LEAD FCST_VALID_BEG
FCST_VALID_END OBS_LEAD OBS_VALID_BEG OBS_VALID_END FCST_VAR
FCST_UNITS FCST_LEV OBS_VAR OBS_UNITS OBS_LEV OBTYPE VX_MASK
INTERP_MTHD INTERP_PNTS FCST_THRESH OBS_THRESH COV_THRESH ALPHA
LINE_TYPE TOTAL N_ENS CRPS CRPSS IGN ME RMSE SPREAD
ME_OERR RMSE_OERR SPREAD_OERR SPREAD_PLUS_OERR
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 TMP K
Z2 TMP NA Z2 ADPSFC FULL NBRHD_SQUARE 9
NA NA NA NA ECNT 90 9 0.95563
NA 39.95526 0.56981 1.4039 0.4494 NA NA NA
NA
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 TMP K
Z2 TMP NA Z2 ADPSFC FULL NBRHD_SQUARE 25
NA NA NA NA ECNT 89 25 0.96228
NA 11.78029 0.55743 1.45671 0.57945 NA NA NA
NA
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 TMP K
Z2 TMP NA Z2 ADPSFC FULL NBRHD_SQUARE 49
NA NA NA NA ECNT 89 49 0.98542
NA 6.07649 0.56571 1.51684 0.68287 NA NA NA
NA
------------------------------------------------
Subject: MET V9.0 Point-Stat error (UNCLASSIFIED)
From: Raby, John W USA CIV
Time: Fri Jul 03 18:44:03 2020
VERSION MODEL DESC FCST_LEAD FCST_VALID_BEG
FCST_VALID_END OBS_LEAD OBS_VALID_BEG OBS_VALID_END FCST_VAR
FCST_UNITS FCST_LEV OBS_VAR OBS_UNITS OBS_LEV OBTYPE VX_MASK
INTERP_MTHD INTERP_PNTS FCST_THRESH OBS_THRESH COV_THRESH ALPHA
LINE_TYPE TOTAL N_PROB RPS_REL RPS_RES RPS_UNC RPS RPSS
RPSS_SMPL
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 TMP K
Z2 TMP NA Z2 ADPSFC FULL NBRHD_SQUARE 9
>=250,>=260,>=270 NA NA NA RPS 90 9
0.0040924 0.0036626 0.0036626 0.0040924 NA -0.11735
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 TMP K
Z2 TMP NA Z2 ADPSFC FULL NBRHD_SQUARE 25
>=250,>=260,>=270 NA NA NA RPS 89 25
0.002797 0.0037032 0.0037032 0.002797 NA 0.24471
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 TMP K
Z2 TMP NA Z2 ADPSFC FULL NBRHD_SQUARE 49
>=250,>=260,>=270 NA NA NA RPS 89 49
0.0012296 0.0018306 0.0037032 0.0031022 NA 0.16229
------------------------------------------------
Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0 Point-Stat error (UNCLASSIFIED)
From: Raby, John W USA CIV
Time: Sun Jul 05 10:34:03 2020
John -
I tried running with the again with the HiRa method but with the
following thresholds:
>=250, >=260, >=270, >=280, >=290, >=300, <=273, >=273
It gave me an error because the thresholds are not monotonically
increasing and with the same inequality. I thought that this error
only occurs when you have selected the MCTC and MCTS output which are
currently not selected. I recall running Grid-Stat with similar
thresholds without MCTC and MCTS output and it ran fine. What is it
that ohnis causing this error?
Thanks.
R/
John
________________________________________
From: Raby, John W CIV USARMY CCDC ARL (USA)
Sent: Friday, July 03, 2020 6:43 PM
To: met_help at ucar.edu
Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
Point-Stat error (UNCLASSIFIED)
John -
Your GitHub issue and the logic really help to better understand how
the HiRA methodology works. I was also pretty confused about that.
Maybe a re-look at the User's Guide description would be good to
consider too. I've attached the RPS and ECNT output file in case you
wanted to see them. Thank you for working through this with me.
Have a great 4th of July!
R/
John
________________________________________
From: John Halley Gotway via RT [met_help at ucar.edu]
Sent: Friday, July 03, 2020 6:06 PM
To: Raby, John W CIV USARMY CCDC ARL (USA)
Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
Point-Stat error (UNCLASSIFIED)
All active links contained in this email were disabled. Please verify
the identity of the sender, and confirm the authenticity of all links
contained within the message prior to copying and pasting the address
to a Web browser.
----
John,
I was looking at the code today and suspect that I've misled you a
little
bit. I was confused about it myself. I updated this GitHub issue to
clarify:
Caution-https://github.com/NCAR/MET/issues/1400
Upon review, you're correct that Point-Stat DOES use the cat_thresh
settings from the fcst and obs dictionaries while processing HiRA. It
uses
those thresholds to derive probability fields which are evaluated in
the
output PCT, PTSD, PJC, and PRC line types. It also processes the
neighborhood of points as a "spatial ensemble"... and from those
ensemble
points, it computes the ECNT and RPS line types. And it's in the
computation of the RPS line type that the prob_cat_thresh setting is
used.
So prob_cat_thresh really only impacts the RPS line type.
I am still making the change we discussed, but I wanted to clarify how
that
change is actually being used (since I was confused about that
myself).
You can see the tweak to the logic on line 1841 of:
Caution-
https://github.com/NCAR/MET/blob/05f2e7d56bae3af092a69a5a9db4867f202eaae0/met/src/tools/core/point_stat/point_stat.cc#L1841
Thanks,
John
On Thu, Jul 2, 2020 at 8:50 PM Raby, John W USA CIV via RT <
met_help at ucar.edu> wrote:
>
> <Caution-url: Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
>
> John -
>
> Using your suggestion below, I was able to get Point-Stat to run.
See
> attached log file and config file. I haven't looked at the output
yet, but
> it was generated and there are no 0-sized files.
>
> Thanks for your help in getting it running!
>
> R/
> John
>
> ________________________________________
> From: John Halley Gotway via RT [met_help at ucar.edu]
> Sent: Thursday, July 02, 2020 4:45 PM
> To: Raby, John W CIV USARMY CCDC ARL (USA)
> Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
> Point-Stat error (UNCLASSIFIED)
>
> All active links contained in this email were disabled. Please
verify the
> identity of the sender, and confirm the authenticity of all links
contained
> within the message prior to copying and pasting the address to a Web
> browser.
>
>
>
>
> ----
>
> Yes, that's right, obs=fcst will copy everything over. And it really
> doesn't hurt having hira in both. Point-Stat just parses it from the
obs
> dictionary.
>
> John
>
> On Thu, Jul 2, 2020 at 4:31 PM Raby, John W USA CIV via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <Caution-Caution-url: Caution-
> Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> >
> > John -
> >
> > Thanks for taking a look and your quick reply. I see now about
including
> > the hira dictionary inside the field array dictionary. Thanks for
showing
> > that below.
> >
> > So if I have the fcst dictionary only (obs=fcst) in my confiig
file then
> I
> > all I have to do is put the hira dictionary with settings for the
fcst
> > variables, correct? I guess I'm "not understanding "copying it to
the
> obs"
> > when I don't explicitly show the obs dictionary.
> >
> > Good point about limiting the number of threshold and widths for
testing.
> >
> > R/
> > John
> >
> >
> > ________________________________________
> > From: John Halley Gotway via RT [met_help at ucar.edu]
> > Sent: Thursday, July 02, 2020 4:14 PM
> > To: Raby, John W CIV USARMY CCDC ARL (USA)
> > Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > Point-Stat error (UNCLASSIFIED)
> >
> > All active links contained in this email were disabled. Please
verify
> the
> > identity of the sender, and confirm the authenticity of all links
> contained
> > within the message prior to copying and pasting the address to a
Web
> > browser.
> >
> >
> >
> >
> > ----
> >
> > John,
> >
> > Sorry for not answering your previous question. Yes, since you're
using
> > obs=fcst, you can just define it in the fcst dictionary and copy
it over
> to
> > the obs.
> >
> > Yes, I do see a problem in your config file. The hira dictionary
needs to
> > be INSIDE of the field array dictionary entry, as shown below.
> >
> > Also, for initial testing, I'd recommend using a much smaller set
of
> > thresholds and widths. Running with all of these will take a long
time.
> >
> > John
> >
> > obs = {
> > field = [
> > {
> > name = "TMP";
> > level = [ "Z2" ];
> > cat_thresh = [ >=250, >=260, >=270, >=280, >=290, >=300,
<=273,
> > >=273 ];
> > hira = {
> > flag = TRUE;
> > width = [ 3, 5, 7, 9, 11, 13, 15, 17, 19, 21,
23, 25,
> > 27, 29, 31, 33, 35, 37, 39, 41, 43, 45, 47, 49 ];
> > vld_thresh = 1.0;
> > cov_thresh = [ ==0.25 ];
> > shape = SQUARE;
> > prob_cat_thresh = [ >=250, >=260, >=270, >=280, >=290,
>=300,
> > <=273, >=273 ];
> > }
> > },
> > ...
> >
> >
> > On Thu, Jul 2, 2020 at 3:42 PM Raby, John W USA CIV via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <Caution-Caution-Caution-url: Caution-
> > Caution-Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> > >
> > > CLASSIFICATION: UNCLASSIFIED
> > >
> > > John -
> > >
> > > I edited the attached config file to add the HiRA dictionary to
each
> > > obs.field
> > > dictionary entry as you described, but it seems like it's not
expecting
> > > the
> > > hira dictionary to be where I placed it. Do you see where I went
wrong
> > > with
> > > this? The error dialog is also attached.
> > >
> > > Thanks and have a great 4th of July!
> > >
> > > R/
> > > John
> > >
> > >
> > >
> > > -----Original Message-----
> > > From: John Halley Gotway via RT <met_help at ucar.edu>
> > > Sent: Wednesday, July 1, 2020 11:17 AM
> > > To: Raby, John W CIV USARMY CCDC ARL (USA)
<john.w.raby2.civ at mail.mil>
> > > Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > > Point-Stat
> > > error (UNCLASSIFIED)
> > >
> > > All active links contained in this email were disabled. Please
verify
> > the
> > > identity of the sender, and confirm the authenticity of all
links
> > > contained
> > > within the message prior to copying and pasting the address to a
Web
> > > browser.
> > >
> > >
> > >
> > >
> > > ----
> > >
> > > John,
> > >
> > > Sorry for the confusion. I led you astray. The HiRA
configuration
> option
> > > names are not entirely intuitive.
> > >
> > > Please take a look at the README file starting on line 1418:
> > > Caution-
> > > Caution-
> > Caution-
> Caution-
https://github.com/NCAR/MET/blob/0cee9fbd0b3c691fd3cf4dd1d665f5d2b0db9a37/met/data/config/README#L1418
> > >
> > > I modified your hira entry like this:
> > >
> > > hira = {
> > > flag = TRUE; *// set to TRUE/false to turn
on/off
> this
> > > logic*
> > > width = [ 3, 5, 7, 9, 11 ]; *// defines the
> neighborhood
> > > size and must be > 1*
> > > vld_thresh = 1.0;
> > > cov_thresh = [ ==0.25 ]; *// this defines the probability
bins
> to
> > be
> > > used. You only get to pick one setting here (==0.25 is short for
> > > >=0,>=0.25,>=0.5,>=0.75,>=1.0)*
> > > shape = SQUARE;
> > > prob_cat_thresh = [ >290, >300 ];* // this defines the
threshold(s)
> > for
> > > which HiRA logic should be applied* }
> > >
> > > So you just had things a bit mis-configured. The key thing to
note here
> > is
> > > that the hira.prob_cat_thresh setting defines the thresholds for
which
> > > HiRA
> > > probabilities should be computed. This is different from the
cat_thresh
> > > setting in the fcst and obs dictionaries. Those are used to
compute
> > > categorical counts and statistics and are currently separate.
> > >
> > > It's nice that you don't have to use the same set of thresholds
for
> > > categorical stats and the HiRA logic... it's more configurable
that
> way.
> > > However, I do see that it makes things a bit inconvenient. Since
you're
> > > applying HiRA to many variables and each of those variables has
> different
> > > meaningful thresholds, you'd need to configure the HiRA
dictionary
> > > separately
> > > for each and every variable.
> > >
> > > That's what is meant by this comment... that the hira dictionary
can be
> > > specified separately for each variable:
> > >
> > > *// HiRA verification method// May be set separately in each
> "obs.field"
> > > entry*
> > >
> > > So you'd need to copy/paste the HiRA dictionary into every
single
> > > obs.field
> > > dictionary entry and modify the prob_cat_thresh thresholds
accordingly.
> > >
> > > First, please make sure that you can in fact produce HiRA
output.
> Second,
> > > I
> > > can think of a tweak to the logic that might make using HiRA
easier.
> How
> > > about, if the hira.prob_cat_thresh array is an empty list, we
could
> have
> > > the
> > > code check to see if cat_thresh in the obs dictionary is non-
empty. And
> > if
> > > that's non-empty then we'd simply set prob_cat_thresh =
cat_thresh.
> > >
> > > I imagine that's what you were expecting in the first place...
that
> > > Point-Stat
> > > would just be using the thresholds from cat_thresh to define the
HiRA
> > > probability thresholds. It still is nice to have prob_cat_thresh
be
> > > configurable separately, but if left unset, defaulting to
cat_thresh
> > seems
> > > pretty reasonable to me. Do you think that would be a worthwhile
> change?
> > >
> > > Thanks,
> > > John
> > >
> > > On Tue, Jun 30, 2020 at 7:12 PM Raby, John W USA CIV via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <Caution-Caution-Caution-Caution-url:
> > > > Caution-Caution-
> > Caution-Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> > > >
> > > > I forgot to attach the log file.
> > > >
> > > > ________________________________________
> > > > From: Raby, John W CIV USARMY CCDC ARL (USA)
> > > > Sent: Tuesday, June 30, 2020 7:00 PM
> > > > To: met_help at ucar.edu
> > > > Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > > > Point-Stat error (UNCLASSIFIED)
> > > >
> > > > I edited the config file to use the shortcut and corrected the
> > > > whitespace issue, but I get the same error.
> > > > R/
> > > > John
> > > >
> > > >
> > > >
> > > > ________________________________________
> > > > From: Raby, John W CIV USARMY CCDC ARL (USA)
> > > > Sent: Tuesday, June 30, 2020 6:10 PM
> > > > To: met_help at ucar.edu
> > > > Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > > > Point-Stat error (UNCLASSIFIED)
> > > >
> > > > Thanks for taking a look, John. I'll give that a try.
Appreciate the
> > > > heads-up on a future error in the "desc".
> > > >
> > > > R/
> > > > John
> > > >
> > > > ________________________________________
> > > > From: John Halley Gotway via RT [met_help at ucar.edu]
> > > > Sent: Tuesday, June 30, 2020 5:11 PM
> > > > To: Raby, John W CIV USARMY CCDC ARL (USA)
> > > > Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > > > Point-Stat error (UNCLASSIFIED)
> > > >
> > > > All active links contained in this email were disabled.
Please
> verify
> > > > the identity of the sender, and confirm the authenticity of
all links
> > > > contained within the message prior to copying and pasting the
address
> > > > to a Web browser.
> > > >
> > > >
> > > >
> > > >
> > > > ----
> > > >
> > > > John,
> > > >
> > > > Please try using this shortcut:
> > > > prob_cat_thresh = [ ==0.25 ];
> > > >
> > > > Does that do it?
> > > >
> > > > Also, your "desc" setting can't contain embedded whitespace.
And
> > > > Point-Stat should error out and tell you that.
> > > >
> > > > John
> > > >
> > > > On Tue, Jun 30, 2020 at 4:57 PM Raby, John W USA CIV via RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > Tue Jun 30 16:56:45 2020: Request 95790 was acted upon.
> > > > > Transaction: Ticket created by john.w.raby2.civ at mail.mil
> > > > > Queue: met_help
> > > > > Subject: MET V9.0 Point-Stat error (UNCLASSIFIED)
> > > > > Owner: Nobody
> > > > > Requestors: john.w.raby2.civ at mail.mil
> > > > > Status: new
> > > > > Ticket <Caution-Caution-Caution-Caution-Caution-url:
Caution-
> > > > Caution-Caution-
> > Caution-Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> > > > >
> > > > >
> > > > > CLASSIFICATION: UNCLASSIFIED
> > > > >
> > > > > I keep getting the same error when trying to run Point-Stat
with
> the
> > > > > HiRA verification logic enabled. The error involves the
probability
> > > > > field
> > > > > thresholds:
> > > > >
> > > > > ERROR:check_prob_thresh() -> When verifying a probability
field,
> you
> > > > > must select at least 3 thresholds beginning with 0.0 and
ending
> with
> > > 1.0.
> > > > >
> > > > > During troubleshooting, I've changed the settings for
> > > > > prob_cat_thresh in the attached config file several times
thinking
> > > > > that the formatting of my settings was the issue. Maybe I'm
on the
> > > > > wrong track. Could you take a
> > > > look
> > > > > at the config file to check the format or other possible
issue
> which
> > > > might
> > > > > be causing this issue?
> > > > >
> > > > > Thanks.
> > > > >
> > > > > R/
> > > > > John
> > > > >
> > > > > Mr. John W. Raby
> > > > > U.S. Army Research Laboratory
> > > > > White Sands Missile Range, NM 88002
> > > > > Office:(575) 678-2004 DSN 258-2004
> > > > > Teleworking: Contact Admin Specialist at (410) 278-6974 FAX
(575)
> > > > > 678-1230 DSN 258-1230
> > > > > Email: john.w.raby2.civ at mail.mil
> > > > >
> > > > > CLASSIFICATION: UNCLASSIFIED
> > > > >
> > > > >
> > > >
> > > >
> > > >
> > >
> > > CLASSIFICATION: UNCLASSIFIED
> > >
> > >
> >
> >
> >
> >
>
>
>
------------------------------------------------
Subject: MET V9.0 Point-Stat error (UNCLASSIFIED)
From: John Halley Gotway
Time: Sun Jul 05 10:46:36 2020
John,
Just disable the RPS output line type (rps = NONE;). I believe that’s
where those threshold requirements originate. Or, if you would like
RPS
output, read up on that statistic and you’ll see why those
requirements are
in place.
On Sun, Jul 5, 2020 at 10:34 AM Raby, John W USA CIV via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
>
> John -
>
> I tried running with the again with the HiRa method but with the
following
> thresholds:
> >=250, >=260, >=270, >=280, >=290, >=300, <=273, >=273
> It gave me an error because the thresholds are not monotonically
> increasing and with the same inequality. I thought that this error
only
> occurs when you have selected the MCTC and MCTS output which are
currently
> not selected. I recall running Grid-Stat with similar thresholds
without
> MCTC and MCTS output and it ran fine. What is it that ohnis causing
this
> error?
>
> Thanks.
>
> R/
> John
> ________________________________________
> From: Raby, John W CIV USARMY CCDC ARL (USA)
> Sent: Friday, July 03, 2020 6:43 PM
> To: met_help at ucar.edu
> Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
> Point-Stat error (UNCLASSIFIED)
>
> John -
>
> Your GitHub issue and the logic really help to better understand how
the
> HiRA methodology works. I was also pretty confused about that. Maybe
a
> re-look at the User's Guide description would be good to consider
too. I've
> attached the RPS and ECNT output file in case you wanted to see
them. Thank
> you for working through this with me.
>
> Have a great 4th of July!
>
> R/
> John
> ________________________________________
> From: John Halley Gotway via RT [met_help at ucar.edu]
> Sent: Friday, July 03, 2020 6:06 PM
> To: Raby, John W CIV USARMY CCDC ARL (USA)
> Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
> Point-Stat error (UNCLASSIFIED)
>
> All active links contained in this email were disabled. Please
verify the
> identity of the sender, and confirm the authenticity of all links
contained
> within the message prior to copying and pasting the address to a Web
> browser.
>
>
>
>
> ----
>
> John,
>
> I was looking at the code today and suspect that I've misled you a
little
> bit. I was confused about it myself. I updated this GitHub issue to
> clarify:
> Caution-https://github.com/NCAR/MET/issues/1400
>
> Upon review, you're correct that Point-Stat DOES use the cat_thresh
> settings from the fcst and obs dictionaries while processing HiRA.
It uses
> those thresholds to derive probability fields which are evaluated in
the
> output PCT, PTSD, PJC, and PRC line types. It also processes the
> neighborhood of points as a "spatial ensemble"... and from those
ensemble
> points, it computes the ECNT and RPS line types. And it's in the
> computation of the RPS line type that the prob_cat_thresh setting is
used.
>
> So prob_cat_thresh really only impacts the RPS line type.
>
> I am still making the change we discussed, but I wanted to clarify
how that
> change is actually being used (since I was confused about that
myself).
>
> You can see the tweak to the logic on line 1841 of:
> Caution-
>
https://github.com/NCAR/MET/blob/05f2e7d56bae3af092a69a5a9db4867f202eaae0/met/src/tools/core/point_stat/point_stat.cc#L1841
>
> Thanks,
> John
>
>
>
> On Thu, Jul 2, 2020 at 8:50 PM Raby, John W USA CIV via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <Caution-url: Caution-
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> >
> > John -
> >
> > Using your suggestion below, I was able to get Point-Stat to run.
See
> > attached log file and config file. I haven't looked at the output
yet,
> but
> > it was generated and there are no 0-sized files.
> >
> > Thanks for your help in getting it running!
> >
> > R/
> > John
> >
> > ________________________________________
> > From: John Halley Gotway via RT [met_help at ucar.edu]
> > Sent: Thursday, July 02, 2020 4:45 PM
> > To: Raby, John W CIV USARMY CCDC ARL (USA)
> > Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > Point-Stat error (UNCLASSIFIED)
> >
> > All active links contained in this email were disabled. Please
verify
> the
> > identity of the sender, and confirm the authenticity of all links
> contained
> > within the message prior to copying and pasting the address to a
Web
> > browser.
> >
> >
> >
> >
> > ----
> >
> > Yes, that's right, obs=fcst will copy everything over. And it
really
> > doesn't hurt having hira in both. Point-Stat just parses it from
the obs
> > dictionary.
> >
> > John
> >
> > On Thu, Jul 2, 2020 at 4:31 PM Raby, John W USA CIV via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <Caution-Caution-url: Caution-
> > Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> > >
> > > John -
> > >
> > > Thanks for taking a look and your quick reply. I see now about
> including
> > > the hira dictionary inside the field array dictionary. Thanks
for
> showing
> > > that below.
> > >
> > > So if I have the fcst dictionary only (obs=fcst) in my confiig
file
> then
> > I
> > > all I have to do is put the hira dictionary with settings for
the fcst
> > > variables, correct? I guess I'm "not understanding "copying it
to the
> > obs"
> > > when I don't explicitly show the obs dictionary.
> > >
> > > Good point about limiting the number of threshold and widths for
> testing.
> > >
> > > R/
> > > John
> > >
> > >
> > > ________________________________________
> > > From: John Halley Gotway via RT [met_help at ucar.edu]
> > > Sent: Thursday, July 02, 2020 4:14 PM
> > > To: Raby, John W CIV USARMY CCDC ARL (USA)
> > > Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > > Point-Stat error (UNCLASSIFIED)
> > >
> > > All active links contained in this email were disabled. Please
verify
> > the
> > > identity of the sender, and confirm the authenticity of all
links
> > contained
> > > within the message prior to copying and pasting the address to a
Web
> > > browser.
> > >
> > >
> > >
> > >
> > > ----
> > >
> > > John,
> > >
> > > Sorry for not answering your previous question. Yes, since
you're using
> > > obs=fcst, you can just define it in the fcst dictionary and copy
it
> over
> > to
> > > the obs.
> > >
> > > Yes, I do see a problem in your config file. The hira dictionary
needs
> to
> > > be INSIDE of the field array dictionary entry, as shown below.
> > >
> > > Also, for initial testing, I'd recommend using a much smaller
set of
> > > thresholds and widths. Running with all of these will take a
long time.
> > >
> > > John
> > >
> > > obs = {
> > > field = [
> > > {
> > > name = "TMP";
> > > level = [ "Z2" ];
> > > cat_thresh = [ >=250, >=260, >=270, >=280, >=290, >=300,
<=273,
> > > >=273 ];
> > > hira = {
> > > flag = TRUE;
> > > width = [ 3, 5, 7, 9, 11, 13, 15, 17, 19,
21, 23,
> 25,
> > > 27, 29, 31, 33, 35, 37, 39, 41, 43, 45, 47, 49 ];
> > > vld_thresh = 1.0;
> > > cov_thresh = [ ==0.25 ];
> > > shape = SQUARE;
> > > prob_cat_thresh = [ >=250, >=260, >=270, >=280, >=290,
>=300,
> > > <=273, >=273 ];
> > > }
> > > },
> > > ...
> > >
> > >
> > > On Thu, Jul 2, 2020 at 3:42 PM Raby, John W USA CIV via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <Caution-Caution-Caution-url: Caution-
> > > Caution-Caution-
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> > > >
> > > > CLASSIFICATION: UNCLASSIFIED
> > > >
> > > > John -
> > > >
> > > > I edited the attached config file to add the HiRA dictionary
to each
> > > > obs.field
> > > > dictionary entry as you described, but it seems like it's not
> expecting
> > > > the
> > > > hira dictionary to be where I placed it. Do you see where I
went
> wrong
> > > > with
> > > > this? The error dialog is also attached.
> > > >
> > > > Thanks and have a great 4th of July!
> > > >
> > > > R/
> > > > John
> > > >
> > > >
> > > >
> > > > -----Original Message-----
> > > > From: John Halley Gotway via RT <met_help at ucar.edu>
> > > > Sent: Wednesday, July 1, 2020 11:17 AM
> > > > To: Raby, John W CIV USARMY CCDC ARL (USA) <
> john.w.raby2.civ at mail.mil>
> > > > Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > > > Point-Stat
> > > > error (UNCLASSIFIED)
> > > >
> > > > All active links contained in this email were disabled.
Please
> verify
> > > the
> > > > identity of the sender, and confirm the authenticity of all
links
> > > > contained
> > > > within the message prior to copying and pasting the address to
a Web
> > > > browser.
> > > >
> > > >
> > > >
> > > >
> > > > ----
> > > >
> > > > John,
> > > >
> > > > Sorry for the confusion. I led you astray. The HiRA
configuration
> > option
> > > > names are not entirely intuitive.
> > > >
> > > > Please take a look at the README file starting on line 1418:
> > > > Caution-
> > > > Caution-
> > > Caution-
> > Caution-
>
https://github.com/NCAR/MET/blob/0cee9fbd0b3c691fd3cf4dd1d665f5d2b0db9a37/met/data/config/README#L1418
> > > >
> > > > I modified your hira entry like this:
> > > >
> > > > hira = {
> > > > flag = TRUE; *// set to TRUE/false to turn
on/off
> > this
> > > > logic*
> > > > width = [ 3, 5, 7, 9, 11 ]; *// defines the
> > neighborhood
> > > > size and must be > 1*
> > > > vld_thresh = 1.0;
> > > > cov_thresh = [ ==0.25 ]; *// this defines the
probability bins
> > to
> > > be
> > > > used. You only get to pick one setting here (==0.25 is short
for
> > > > >=0,>=0.25,>=0.5,>=0.75,>=1.0)*
> > > > shape = SQUARE;
> > > > prob_cat_thresh = [ >290, >300 ];* // this defines the
> threshold(s)
> > > for
> > > > which HiRA logic should be applied* }
> > > >
> > > > So you just had things a bit mis-configured. The key thing to
note
> here
> > > is
> > > > that the hira.prob_cat_thresh setting defines the thresholds
for
> which
> > > > HiRA
> > > > probabilities should be computed. This is different from the
> cat_thresh
> > > > setting in the fcst and obs dictionaries. Those are used to
compute
> > > > categorical counts and statistics and are currently separate.
> > > >
> > > > It's nice that you don't have to use the same set of
thresholds for
> > > > categorical stats and the HiRA logic... it's more configurable
that
> > way.
> > > > However, I do see that it makes things a bit inconvenient.
Since
> you're
> > > > applying HiRA to many variables and each of those variables
has
> > different
> > > > meaningful thresholds, you'd need to configure the HiRA
dictionary
> > > > separately
> > > > for each and every variable.
> > > >
> > > > That's what is meant by this comment... that the hira
dictionary can
> be
> > > > specified separately for each variable:
> > > >
> > > > *// HiRA verification method// May be set separately in each
> > "obs.field"
> > > > entry*
> > > >
> > > > So you'd need to copy/paste the HiRA dictionary into every
single
> > > > obs.field
> > > > dictionary entry and modify the prob_cat_thresh thresholds
> accordingly.
> > > >
> > > > First, please make sure that you can in fact produce HiRA
output.
> > Second,
> > > > I
> > > > can think of a tweak to the logic that might make using HiRA
easier.
> > How
> > > > about, if the hira.prob_cat_thresh array is an empty list, we
could
> > have
> > > > the
> > > > code check to see if cat_thresh in the obs dictionary is non-
empty.
> And
> > > if
> > > > that's non-empty then we'd simply set prob_cat_thresh =
cat_thresh.
> > > >
> > > > I imagine that's what you were expecting in the first place...
that
> > > > Point-Stat
> > > > would just be using the thresholds from cat_thresh to define
the HiRA
> > > > probability thresholds. It still is nice to have
prob_cat_thresh be
> > > > configurable separately, but if left unset, defaulting to
cat_thresh
> > > seems
> > > > pretty reasonable to me. Do you think that would be a
worthwhile
> > change?
> > > >
> > > > Thanks,
> > > > John
> > > >
> > > > On Tue, Jun 30, 2020 at 7:12 PM Raby, John W USA CIV via RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > <Caution-Caution-Caution-Caution-url:
> > > > > Caution-Caution-
> > > Caution-Caution-
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> > > > >
> > > > > I forgot to attach the log file.
> > > > >
> > > > > ________________________________________
> > > > > From: Raby, John W CIV USARMY CCDC ARL (USA)
> > > > > Sent: Tuesday, June 30, 2020 7:00 PM
> > > > > To: met_help at ucar.edu
> > > > > Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790]
MET
> V9.0
> > > > > Point-Stat error (UNCLASSIFIED)
> > > > >
> > > > > I edited the config file to use the shortcut and corrected
the
> > > > > whitespace issue, but I get the same error.
> > > > > R/
> > > > > John
> > > > >
> > > > >
> > > > >
> > > > > ________________________________________
> > > > > From: Raby, John W CIV USARMY CCDC ARL (USA)
> > > > > Sent: Tuesday, June 30, 2020 6:10 PM
> > > > > To: met_help at ucar.edu
> > > > > Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790]
MET
> V9.0
> > > > > Point-Stat error (UNCLASSIFIED)
> > > > >
> > > > > Thanks for taking a look, John. I'll give that a try.
Appreciate
> the
> > > > > heads-up on a future error in the "desc".
> > > > >
> > > > > R/
> > > > > John
> > > > >
> > > > > ________________________________________
> > > > > From: John Halley Gotway via RT [met_help at ucar.edu]
> > > > > Sent: Tuesday, June 30, 2020 5:11 PM
> > > > > To: Raby, John W CIV USARMY CCDC ARL (USA)
> > > > > Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > > > > Point-Stat error (UNCLASSIFIED)
> > > > >
> > > > > All active links contained in this email were disabled.
Please
> > verify
> > > > > the identity of the sender, and confirm the authenticity of
all
> links
> > > > > contained within the message prior to copying and pasting
the
> address
> > > > > to a Web browser.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > ----
> > > > >
> > > > > John,
> > > > >
> > > > > Please try using this shortcut:
> > > > > prob_cat_thresh = [ ==0.25 ];
> > > > >
> > > > > Does that do it?
> > > > >
> > > > > Also, your "desc" setting can't contain embedded whitespace.
And
> > > > > Point-Stat should error out and tell you that.
> > > > >
> > > > > John
> > > > >
> > > > > On Tue, Jun 30, 2020 at 4:57 PM Raby, John W USA CIV via RT
<
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > > >
> > > > > > Tue Jun 30 16:56:45 2020: Request 95790 was acted upon.
> > > > > > Transaction: Ticket created by john.w.raby2.civ at mail.mil
> > > > > > Queue: met_help
> > > > > > Subject: MET V9.0 Point-Stat error (UNCLASSIFIED)
> > > > > > Owner: Nobody
> > > > > > Requestors: john.w.raby2.civ at mail.mil
> > > > > > Status: new
> > > > > > Ticket <Caution-Caution-Caution-Caution-Caution-url:
Caution-
> > > > > Caution-Caution-
> > > Caution-Caution-
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> > > > > >
> > > > > >
> > > > > > CLASSIFICATION: UNCLASSIFIED
> > > > > >
> > > > > > I keep getting the same error when trying to run Point-
Stat with
> > the
> > > > > > HiRA verification logic enabled. The error involves the
> probability
> > > > > > field
> > > > > > thresholds:
> > > > > >
> > > > > > ERROR:check_prob_thresh() -> When verifying a probability
field,
> > you
> > > > > > must select at least 3 thresholds beginning with 0.0 and
ending
> > with
> > > > 1.0.
> > > > > >
> > > > > > During troubleshooting, I've changed the settings for
> > > > > > prob_cat_thresh in the attached config file several times
> thinking
> > > > > > that the formatting of my settings was the issue. Maybe
I'm on
> the
> > > > > > wrong track. Could you take a
> > > > > look
> > > > > > at the config file to check the format or other possible
issue
> > which
> > > > > might
> > > > > > be causing this issue?
> > > > > >
> > > > > > Thanks.
> > > > > >
> > > > > > R/
> > > > > > John
> > > > > >
> > > > > > Mr. John W. Raby
> > > > > > U.S. Army Research Laboratory
> > > > > > White Sands Missile Range, NM 88002
> > > > > > Office:(575) 678-2004 DSN 258-2004
> > > > > > Teleworking: Contact Admin Specialist at (410) 278-6974
FAX (575)
> > > > > > 678-1230 DSN 258-1230
> > > > > > Email: john.w.raby2.civ at mail.mil
> > > > > >
> > > > > > CLASSIFICATION: UNCLASSIFIED
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > > CLASSIFICATION: UNCLASSIFIED
> > > >
> > > >
> > >
> > >
> > >
> > >
> >
> >
> >
>
>
>
>
------------------------------------------------
Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0 Point-Stat error (UNCLASSIFIED)
From: Raby, John W USA CIV
Time: Sun Jul 05 16:17:03 2020
Thanks, John. This helps. I'll read up on RPS to understand why those
requirements are required.
R/
John
________________________________________
From: John Halley Gotway via RT [met_help at ucar.edu]
Sent: Sunday, July 05, 2020 10:46 AM
To: Raby, John W CIV USARMY CCDC ARL (USA)
Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
Point-Stat error (UNCLASSIFIED)
All active links contained in this email were disabled. Please verify
the identity of the sender, and confirm the authenticity of all links
contained within the message prior to copying and pasting the address
to a Web browser.
----
John,
Just disable the RPS output line type (rps = NONE;). I believe that’s
where those threshold requirements originate. Or, if you would like
RPS
output, read up on that statistic and you’ll see why those
requirements are
in place.
On Sun, Jul 5, 2020 at 10:34 AM Raby, John W USA CIV via RT <
met_help at ucar.edu> wrote:
>
> <Caution-url: Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
>
> John -
>
> I tried running with the again with the HiRa method but with the
following
> thresholds:
> >=250, >=260, >=270, >=280, >=290, >=300, <=273, >=273
> It gave me an error because the thresholds are not monotonically
> increasing and with the same inequality. I thought that this error
only
> occurs when you have selected the MCTC and MCTS output which are
currently
> not selected. I recall running Grid-Stat with similar thresholds
without
> MCTC and MCTS output and it ran fine. What is it that ohnis causing
this
> error?
>
> Thanks.
>
> R/
> John
> ________________________________________
> From: Raby, John W CIV USARMY CCDC ARL (USA)
> Sent: Friday, July 03, 2020 6:43 PM
> To: met_help at ucar.edu
> Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
> Point-Stat error (UNCLASSIFIED)
>
> John -
>
> Your GitHub issue and the logic really help to better understand how
the
> HiRA methodology works. I was also pretty confused about that. Maybe
a
> re-look at the User's Guide description would be good to consider
too. I've
> attached the RPS and ECNT output file in case you wanted to see
them. Thank
> you for working through this with me.
>
> Have a great 4th of July!
>
> R/
> John
> ________________________________________
> From: John Halley Gotway via RT [met_help at ucar.edu]
> Sent: Friday, July 03, 2020 6:06 PM
> To: Raby, John W CIV USARMY CCDC ARL (USA)
> Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
> Point-Stat error (UNCLASSIFIED)
>
> All active links contained in this email were disabled. Please
verify the
> identity of the sender, and confirm the authenticity of all links
contained
> within the message prior to copying and pasting the address to a Web
> browser.
>
>
>
>
> ----
>
> John,
>
> I was looking at the code today and suspect that I've misled you a
little
> bit. I was confused about it myself. I updated this GitHub issue to
> clarify:
> Caution-Caution-https://github.com/NCAR/MET/issues/1400
>
> Upon review, you're correct that Point-Stat DOES use the cat_thresh
> settings from the fcst and obs dictionaries while processing HiRA.
It uses
> those thresholds to derive probability fields which are evaluated in
the
> output PCT, PTSD, PJC, and PRC line types. It also processes the
> neighborhood of points as a "spatial ensemble"... and from those
ensemble
> points, it computes the ECNT and RPS line types. And it's in the
> computation of the RPS line type that the prob_cat_thresh setting is
used.
>
> So prob_cat_thresh really only impacts the RPS line type.
>
> I am still making the change we discussed, but I wanted to clarify
how that
> change is actually being used (since I was confused about that
myself).
>
> You can see the tweak to the logic on line 1841 of:
> Caution-
> Caution-
https://github.com/NCAR/MET/blob/05f2e7d56bae3af092a69a5a9db4867f202eaae0/met/src/tools/core/point_stat/point_stat.cc#L1841
>
> Thanks,
> John
>
>
>
> On Thu, Jul 2, 2020 at 8:50 PM Raby, John W USA CIV via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <Caution-Caution-url: Caution-
> Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> >
> > John -
> >
> > Using your suggestion below, I was able to get Point-Stat to run.
See
> > attached log file and config file. I haven't looked at the output
yet,
> but
> > it was generated and there are no 0-sized files.
> >
> > Thanks for your help in getting it running!
> >
> > R/
> > John
> >
> > ________________________________________
> > From: John Halley Gotway via RT [met_help at ucar.edu]
> > Sent: Thursday, July 02, 2020 4:45 PM
> > To: Raby, John W CIV USARMY CCDC ARL (USA)
> > Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > Point-Stat error (UNCLASSIFIED)
> >
> > All active links contained in this email were disabled. Please
verify
> the
> > identity of the sender, and confirm the authenticity of all links
> contained
> > within the message prior to copying and pasting the address to a
Web
> > browser.
> >
> >
> >
> >
> > ----
> >
> > Yes, that's right, obs=fcst will copy everything over. And it
really
> > doesn't hurt having hira in both. Point-Stat just parses it from
the obs
> > dictionary.
> >
> > John
> >
> > On Thu, Jul 2, 2020 at 4:31 PM Raby, John W USA CIV via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <Caution-Caution-Caution-url: Caution-
> > Caution-Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> > >
> > > John -
> > >
> > > Thanks for taking a look and your quick reply. I see now about
> including
> > > the hira dictionary inside the field array dictionary. Thanks
for
> showing
> > > that below.
> > >
> > > So if I have the fcst dictionary only (obs=fcst) in my confiig
file
> then
> > I
> > > all I have to do is put the hira dictionary with settings for
the fcst
> > > variables, correct? I guess I'm "not understanding "copying it
to the
> > obs"
> > > when I don't explicitly show the obs dictionary.
> > >
> > > Good point about limiting the number of threshold and widths for
> testing.
> > >
> > > R/
> > > John
> > >
> > >
> > > ________________________________________
> > > From: John Halley Gotway via RT [met_help at ucar.edu]
> > > Sent: Thursday, July 02, 2020 4:14 PM
> > > To: Raby, John W CIV USARMY CCDC ARL (USA)
> > > Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > > Point-Stat error (UNCLASSIFIED)
> > >
> > > All active links contained in this email were disabled. Please
verify
> > the
> > > identity of the sender, and confirm the authenticity of all
links
> > contained
> > > within the message prior to copying and pasting the address to a
Web
> > > browser.
> > >
> > >
> > >
> > >
> > > ----
> > >
> > > John,
> > >
> > > Sorry for not answering your previous question. Yes, since
you're using
> > > obs=fcst, you can just define it in the fcst dictionary and copy
it
> over
> > to
> > > the obs.
> > >
> > > Yes, I do see a problem in your config file. The hira dictionary
needs
> to
> > > be INSIDE of the field array dictionary entry, as shown below.
> > >
> > > Also, for initial testing, I'd recommend using a much smaller
set of
> > > thresholds and widths. Running with all of these will take a
long time.
> > >
> > > John
> > >
> > > obs = {
> > > field = [
> > > {
> > > name = "TMP";
> > > level = [ "Z2" ];
> > > cat_thresh = [ >=250, >=260, >=270, >=280, >=290, >=300,
<=273,
> > > >=273 ];
> > > hira = {
> > > flag = TRUE;
> > > width = [ 3, 5, 7, 9, 11, 13, 15, 17, 19,
21, 23,
> 25,
> > > 27, 29, 31, 33, 35, 37, 39, 41, 43, 45, 47, 49 ];
> > > vld_thresh = 1.0;
> > > cov_thresh = [ ==0.25 ];
> > > shape = SQUARE;
> > > prob_cat_thresh = [ >=250, >=260, >=270, >=280, >=290,
>=300,
> > > <=273, >=273 ];
> > > }
> > > },
> > > ...
> > >
> > >
> > > On Thu, Jul 2, 2020 at 3:42 PM Raby, John W USA CIV via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <Caution-Caution-Caution-Caution-url: Caution-
> > > Caution-Caution-
> Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> > > >
> > > > CLASSIFICATION: UNCLASSIFIED
> > > >
> > > > John -
> > > >
> > > > I edited the attached config file to add the HiRA dictionary
to each
> > > > obs.field
> > > > dictionary entry as you described, but it seems like it's not
> expecting
> > > > the
> > > > hira dictionary to be where I placed it. Do you see where I
went
> wrong
> > > > with
> > > > this? The error dialog is also attached.
> > > >
> > > > Thanks and have a great 4th of July!
> > > >
> > > > R/
> > > > John
> > > >
> > > >
> > > >
> > > > -----Original Message-----
> > > > From: John Halley Gotway via RT <met_help at ucar.edu>
> > > > Sent: Wednesday, July 1, 2020 11:17 AM
> > > > To: Raby, John W CIV USARMY CCDC ARL (USA) <
> john.w.raby2.civ at mail.mil>
> > > > Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > > > Point-Stat
> > > > error (UNCLASSIFIED)
> > > >
> > > > All active links contained in this email were disabled.
Please
> verify
> > > the
> > > > identity of the sender, and confirm the authenticity of all
links
> > > > contained
> > > > within the message prior to copying and pasting the address to
a Web
> > > > browser.
> > > >
> > > >
> > > >
> > > >
> > > > ----
> > > >
> > > > John,
> > > >
> > > > Sorry for the confusion. I led you astray. The HiRA
configuration
> > option
> > > > names are not entirely intuitive.
> > > >
> > > > Please take a look at the README file starting on line 1418:
> > > > Caution-
> > > > Caution-
> > > Caution-
> > Caution-
> Caution-
https://github.com/NCAR/MET/blob/0cee9fbd0b3c691fd3cf4dd1d665f5d2b0db9a37/met/data/config/README#L1418
> > > >
> > > > I modified your hira entry like this:
> > > >
> > > > hira = {
> > > > flag = TRUE; *// set to TRUE/false to turn
on/off
> > this
> > > > logic*
> > > > width = [ 3, 5, 7, 9, 11 ]; *// defines the
> > neighborhood
> > > > size and must be > 1*
> > > > vld_thresh = 1.0;
> > > > cov_thresh = [ ==0.25 ]; *// this defines the
probability bins
> > to
> > > be
> > > > used. You only get to pick one setting here (==0.25 is short
for
> > > > >=0,>=0.25,>=0.5,>=0.75,>=1.0)*
> > > > shape = SQUARE;
> > > > prob_cat_thresh = [ >290, >300 ];* // this defines the
> threshold(s)
> > > for
> > > > which HiRA logic should be applied* }
> > > >
> > > > So you just had things a bit mis-configured. The key thing to
note
> here
> > > is
> > > > that the hira.prob_cat_thresh setting defines the thresholds
for
> which
> > > > HiRA
> > > > probabilities should be computed. This is different from the
> cat_thresh
> > > > setting in the fcst and obs dictionaries. Those are used to
compute
> > > > categorical counts and statistics and are currently separate.
> > > >
> > > > It's nice that you don't have to use the same set of
thresholds for
> > > > categorical stats and the HiRA logic... it's more configurable
that
> > way.
> > > > However, I do see that it makes things a bit inconvenient.
Since
> you're
> > > > applying HiRA to many variables and each of those variables
has
> > different
> > > > meaningful thresholds, you'd need to configure the HiRA
dictionary
> > > > separately
> > > > for each and every variable.
> > > >
> > > > That's what is meant by this comment... that the hira
dictionary can
> be
> > > > specified separately for each variable:
> > > >
> > > > *// HiRA verification method// May be set separately in each
> > "obs.field"
> > > > entry*
> > > >
> > > > So you'd need to copy/paste the HiRA dictionary into every
single
> > > > obs.field
> > > > dictionary entry and modify the prob_cat_thresh thresholds
> accordingly.
> > > >
> > > > First, please make sure that you can in fact produce HiRA
output.
> > Second,
> > > > I
> > > > can think of a tweak to the logic that might make using HiRA
easier.
> > How
> > > > about, if the hira.prob_cat_thresh array is an empty list, we
could
> > have
> > > > the
> > > > code check to see if cat_thresh in the obs dictionary is non-
empty.
> And
> > > if
> > > > that's non-empty then we'd simply set prob_cat_thresh =
cat_thresh.
> > > >
> > > > I imagine that's what you were expecting in the first place...
that
> > > > Point-Stat
> > > > would just be using the thresholds from cat_thresh to define
the HiRA
> > > > probability thresholds. It still is nice to have
prob_cat_thresh be
> > > > configurable separately, but if left unset, defaulting to
cat_thresh
> > > seems
> > > > pretty reasonable to me. Do you think that would be a
worthwhile
> > change?
> > > >
> > > > Thanks,
> > > > John
> > > >
> > > > On Tue, Jun 30, 2020 at 7:12 PM Raby, John W USA CIV via RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > <Caution-Caution-Caution-Caution-Caution-url:
> > > > > Caution-Caution-
> > > Caution-Caution-
> Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> > > > >
> > > > > I forgot to attach the log file.
> > > > >
> > > > > ________________________________________
> > > > > From: Raby, John W CIV USARMY CCDC ARL (USA)
> > > > > Sent: Tuesday, June 30, 2020 7:00 PM
> > > > > To: met_help at ucar.edu
> > > > > Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790]
MET
> V9.0
> > > > > Point-Stat error (UNCLASSIFIED)
> > > > >
> > > > > I edited the config file to use the shortcut and corrected
the
> > > > > whitespace issue, but I get the same error.
> > > > > R/
> > > > > John
> > > > >
> > > > >
> > > > >
> > > > > ________________________________________
> > > > > From: Raby, John W CIV USARMY CCDC ARL (USA)
> > > > > Sent: Tuesday, June 30, 2020 6:10 PM
> > > > > To: met_help at ucar.edu
> > > > > Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790]
MET
> V9.0
> > > > > Point-Stat error (UNCLASSIFIED)
> > > > >
> > > > > Thanks for taking a look, John. I'll give that a try.
Appreciate
> the
> > > > > heads-up on a future error in the "desc".
> > > > >
> > > > > R/
> > > > > John
> > > > >
> > > > > ________________________________________
> > > > > From: John Halley Gotway via RT [met_help at ucar.edu]
> > > > > Sent: Tuesday, June 30, 2020 5:11 PM
> > > > > To: Raby, John W CIV USARMY CCDC ARL (USA)
> > > > > Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > > > > Point-Stat error (UNCLASSIFIED)
> > > > >
> > > > > All active links contained in this email were disabled.
Please
> > verify
> > > > > the identity of the sender, and confirm the authenticity of
all
> links
> > > > > contained within the message prior to copying and pasting
the
> address
> > > > > to a Web browser.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > ----
> > > > >
> > > > > John,
> > > > >
> > > > > Please try using this shortcut:
> > > > > prob_cat_thresh = [ ==0.25 ];
> > > > >
> > > > > Does that do it?
> > > > >
> > > > > Also, your "desc" setting can't contain embedded whitespace.
And
> > > > > Point-Stat should error out and tell you that.
> > > > >
> > > > > John
> > > > >
> > > > > On Tue, Jun 30, 2020 at 4:57 PM Raby, John W USA CIV via RT
<
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > > >
> > > > > > Tue Jun 30 16:56:45 2020: Request 95790 was acted upon.
> > > > > > Transaction: Ticket created by john.w.raby2.civ at mail.mil
> > > > > > Queue: met_help
> > > > > > Subject: MET V9.0 Point-Stat error (UNCLASSIFIED)
> > > > > > Owner: Nobody
> > > > > > Requestors: john.w.raby2.civ at mail.mil
> > > > > > Status: new
> > > > > > Ticket <Caution-Caution-Caution-Caution-Caution-Caution-
url: Caution-
> > > > > Caution-Caution-
> > > Caution-Caution-
> Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> > > > > >
> > > > > >
> > > > > > CLASSIFICATION: UNCLASSIFIED
> > > > > >
> > > > > > I keep getting the same error when trying to run Point-
Stat with
> > the
> > > > > > HiRA verification logic enabled. The error involves the
> probability
> > > > > > field
> > > > > > thresholds:
> > > > > >
> > > > > > ERROR:check_prob_thresh() -> When verifying a probability
field,
> > you
> > > > > > must select at least 3 thresholds beginning with 0.0 and
ending
> > with
> > > > 1.0.
> > > > > >
> > > > > > During troubleshooting, I've changed the settings for
> > > > > > prob_cat_thresh in the attached config file several times
> thinking
> > > > > > that the formatting of my settings was the issue. Maybe
I'm on
> the
> > > > > > wrong track. Could you take a
> > > > > look
> > > > > > at the config file to check the format or other possible
issue
> > which
> > > > > might
> > > > > > be causing this issue?
> > > > > >
> > > > > > Thanks.
> > > > > >
> > > > > > R/
> > > > > > John
> > > > > >
> > > > > > Mr. John W. Raby
> > > > > > U.S. Army Research Laboratory
> > > > > > White Sands Missile Range, NM 88002
> > > > > > Office:(575) 678-2004 DSN 258-2004
> > > > > > Teleworking: Contact Admin Specialist at (410) 278-6974
FAX (575)
> > > > > > 678-1230 DSN 258-1230
> > > > > > Email: john.w.raby2.civ at mail.mil
> > > > > >
> > > > > > CLASSIFICATION: UNCLASSIFIED
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > > CLASSIFICATION: UNCLASSIFIED
> > > >
> > > >
> > >
> > >
> > >
> > >
> >
> >
> >
>
>
>
>
------------------------------------------------
Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0 Point-Stat error (UNCLASSIFIED)
From: Raby, John W USA CIV
Time: Sun Jul 05 17:10:40 2020
John -
In researching RPS used the link you gave me a while back for
accessing the issues on GitHub. Particularly like the ref numbers in
the User's Guide which link to the issue and discussion with
references, etc. Wealth of unfo!
Thanks.
R/
John
________________________________________
From: Raby, John W CIV USARMY CCDC ARL (USA)
Sent: Sunday, July 05, 2020 4:16 PM
To: met_help at ucar.edu
Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
Point-Stat error (UNCLASSIFIED)
Thanks, John. This helps. I'll read up on RPS to understand why those
requirements are required.
R/
John
________________________________________
From: John Halley Gotway via RT [met_help at ucar.edu]
Sent: Sunday, July 05, 2020 10:46 AM
To: Raby, John W CIV USARMY CCDC ARL (USA)
Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
Point-Stat error (UNCLASSIFIED)
All active links contained in this email were disabled. Please verify
the identity of the sender, and confirm the authenticity of all links
contained within the message prior to copying and pasting the address
to a Web browser.
----
John,
Just disable the RPS output line type (rps = NONE;). I believe that’s
where those threshold requirements originate. Or, if you would like
RPS
output, read up on that statistic and you’ll see why those
requirements are
in place.
On Sun, Jul 5, 2020 at 10:34 AM Raby, John W USA CIV via RT <
met_help at ucar.edu> wrote:
>
> <Caution-url: Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
>
> John -
>
> I tried running with the again with the HiRa method but with the
following
> thresholds:
> >=250, >=260, >=270, >=280, >=290, >=300, <=273, >=273
> It gave me an error because the thresholds are not monotonically
> increasing and with the same inequality. I thought that this error
only
> occurs when you have selected the MCTC and MCTS output which are
currently
> not selected. I recall running Grid-Stat with similar thresholds
without
> MCTC and MCTS output and it ran fine. What is it that ohnis causing
this
> error?
>
> Thanks.
>
> R/
> John
> ________________________________________
> From: Raby, John W CIV USARMY CCDC ARL (USA)
> Sent: Friday, July 03, 2020 6:43 PM
> To: met_help at ucar.edu
> Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
> Point-Stat error (UNCLASSIFIED)
>
> John -
>
> Your GitHub issue and the logic really help to better understand how
the
> HiRA methodology works. I was also pretty confused about that. Maybe
a
> re-look at the User's Guide description would be good to consider
too. I've
> attached the RPS and ECNT output file in case you wanted to see
them. Thank
> you for working through this with me.
>
> Have a great 4th of July!
>
> R/
> John
> ________________________________________
> From: John Halley Gotway via RT [met_help at ucar.edu]
> Sent: Friday, July 03, 2020 6:06 PM
> To: Raby, John W CIV USARMY CCDC ARL (USA)
> Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
> Point-Stat error (UNCLASSIFIED)
>
> All active links contained in this email were disabled. Please
verify the
> identity of the sender, and confirm the authenticity of all links
contained
> within the message prior to copying and pasting the address to a Web
> browser.
>
>
>
>
> ----
>
> John,
>
> I was looking at the code today and suspect that I've misled you a
little
> bit. I was confused about it myself. I updated this GitHub issue to
> clarify:
> Caution-Caution-https://github.com/NCAR/MET/issues/1400
>
> Upon review, you're correct that Point-Stat DOES use the cat_thresh
> settings from the fcst and obs dictionaries while processing HiRA.
It uses
> those thresholds to derive probability fields which are evaluated in
the
> output PCT, PTSD, PJC, and PRC line types. It also processes the
> neighborhood of points as a "spatial ensemble"... and from those
ensemble
> points, it computes the ECNT and RPS line types. And it's in the
> computation of the RPS line type that the prob_cat_thresh setting is
used.
>
> So prob_cat_thresh really only impacts the RPS line type.
>
> I am still making the change we discussed, but I wanted to clarify
how that
> change is actually being used (since I was confused about that
myself).
>
> You can see the tweak to the logic on line 1841 of:
> Caution-
> Caution-
https://github.com/NCAR/MET/blob/05f2e7d56bae3af092a69a5a9db4867f202eaae0/met/src/tools/core/point_stat/point_stat.cc#L1841
>
> Thanks,
> John
>
>
>
> On Thu, Jul 2, 2020 at 8:50 PM Raby, John W USA CIV via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <Caution-Caution-url: Caution-
> Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> >
> > John -
> >
> > Using your suggestion below, I was able to get Point-Stat to run.
See
> > attached log file and config file. I haven't looked at the output
yet,
> but
> > it was generated and there are no 0-sized files.
> >
> > Thanks for your help in getting it running!
> >
> > R/
> > John
> >
> > ________________________________________
> > From: John Halley Gotway via RT [met_help at ucar.edu]
> > Sent: Thursday, July 02, 2020 4:45 PM
> > To: Raby, John W CIV USARMY CCDC ARL (USA)
> > Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > Point-Stat error (UNCLASSIFIED)
> >
> > All active links contained in this email were disabled. Please
verify
> the
> > identity of the sender, and confirm the authenticity of all links
> contained
> > within the message prior to copying and pasting the address to a
Web
> > browser.
> >
> >
> >
> >
> > ----
> >
> > Yes, that's right, obs=fcst will copy everything over. And it
really
> > doesn't hurt having hira in both. Point-Stat just parses it from
the obs
> > dictionary.
> >
> > John
> >
> > On Thu, Jul 2, 2020 at 4:31 PM Raby, John W USA CIV via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <Caution-Caution-Caution-url: Caution-
> > Caution-Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> > >
> > > John -
> > >
> > > Thanks for taking a look and your quick reply. I see now about
> including
> > > the hira dictionary inside the field array dictionary. Thanks
for
> showing
> > > that below.
> > >
> > > So if I have the fcst dictionary only (obs=fcst) in my confiig
file
> then
> > I
> > > all I have to do is put the hira dictionary with settings for
the fcst
> > > variables, correct? I guess I'm "not understanding "copying it
to the
> > obs"
> > > when I don't explicitly show the obs dictionary.
> > >
> > > Good point about limiting the number of threshold and widths for
> testing.
> > >
> > > R/
> > > John
> > >
> > >
> > > ________________________________________
> > > From: John Halley Gotway via RT [met_help at ucar.edu]
> > > Sent: Thursday, July 02, 2020 4:14 PM
> > > To: Raby, John W CIV USARMY CCDC ARL (USA)
> > > Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > > Point-Stat error (UNCLASSIFIED)
> > >
> > > All active links contained in this email were disabled. Please
verify
> > the
> > > identity of the sender, and confirm the authenticity of all
links
> > contained
> > > within the message prior to copying and pasting the address to a
Web
> > > browser.
> > >
> > >
> > >
> > >
> > > ----
> > >
> > > John,
> > >
> > > Sorry for not answering your previous question. Yes, since
you're using
> > > obs=fcst, you can just define it in the fcst dictionary and copy
it
> over
> > to
> > > the obs.
> > >
> > > Yes, I do see a problem in your config file. The hira dictionary
needs
> to
> > > be INSIDE of the field array dictionary entry, as shown below.
> > >
> > > Also, for initial testing, I'd recommend using a much smaller
set of
> > > thresholds and widths. Running with all of these will take a
long time.
> > >
> > > John
> > >
> > > obs = {
> > > field = [
> > > {
> > > name = "TMP";
> > > level = [ "Z2" ];
> > > cat_thresh = [ >=250, >=260, >=270, >=280, >=290, >=300,
<=273,
> > > >=273 ];
> > > hira = {
> > > flag = TRUE;
> > > width = [ 3, 5, 7, 9, 11, 13, 15, 17, 19,
21, 23,
> 25,
> > > 27, 29, 31, 33, 35, 37, 39, 41, 43, 45, 47, 49 ];
> > > vld_thresh = 1.0;
> > > cov_thresh = [ ==0.25 ];
> > > shape = SQUARE;
> > > prob_cat_thresh = [ >=250, >=260, >=270, >=280, >=290,
>=300,
> > > <=273, >=273 ];
> > > }
> > > },
> > > ...
> > >
> > >
> > > On Thu, Jul 2, 2020 at 3:42 PM Raby, John W USA CIV via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <Caution-Caution-Caution-Caution-url: Caution-
> > > Caution-Caution-
> Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> > > >
> > > > CLASSIFICATION: UNCLASSIFIED
> > > >
> > > > John -
> > > >
> > > > I edited the attached config file to add the HiRA dictionary
to each
> > > > obs.field
> > > > dictionary entry as you described, but it seems like it's not
> expecting
> > > > the
> > > > hira dictionary to be where I placed it. Do you see where I
went
> wrong
> > > > with
> > > > this? The error dialog is also attached.
> > > >
> > > > Thanks and have a great 4th of July!
> > > >
> > > > R/
> > > > John
> > > >
> > > >
> > > >
> > > > -----Original Message-----
> > > > From: John Halley Gotway via RT <met_help at ucar.edu>
> > > > Sent: Wednesday, July 1, 2020 11:17 AM
> > > > To: Raby, John W CIV USARMY CCDC ARL (USA) <
> john.w.raby2.civ at mail.mil>
> > > > Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > > > Point-Stat
> > > > error (UNCLASSIFIED)
> > > >
> > > > All active links contained in this email were disabled.
Please
> verify
> > > the
> > > > identity of the sender, and confirm the authenticity of all
links
> > > > contained
> > > > within the message prior to copying and pasting the address to
a Web
> > > > browser.
> > > >
> > > >
> > > >
> > > >
> > > > ----
> > > >
> > > > John,
> > > >
> > > > Sorry for the confusion. I led you astray. The HiRA
configuration
> > option
> > > > names are not entirely intuitive.
> > > >
> > > > Please take a look at the README file starting on line 1418:
> > > > Caution-
> > > > Caution-
> > > Caution-
> > Caution-
> Caution-
https://github.com/NCAR/MET/blob/0cee9fbd0b3c691fd3cf4dd1d665f5d2b0db9a37/met/data/config/README#L1418
> > > >
> > > > I modified your hira entry like this:
> > > >
> > > > hira = {
> > > > flag = TRUE; *// set to TRUE/false to turn
on/off
> > this
> > > > logic*
> > > > width = [ 3, 5, 7, 9, 11 ]; *// defines the
> > neighborhood
> > > > size and must be > 1*
> > > > vld_thresh = 1.0;
> > > > cov_thresh = [ ==0.25 ]; *// this defines the
probability bins
> > to
> > > be
> > > > used. You only get to pick one setting here (==0.25 is short
for
> > > > >=0,>=0.25,>=0.5,>=0.75,>=1.0)*
> > > > shape = SQUARE;
> > > > prob_cat_thresh = [ >290, >300 ];* // this defines the
> threshold(s)
> > > for
> > > > which HiRA logic should be applied* }
> > > >
> > > > So you just had things a bit mis-configured. The key thing to
note
> here
> > > is
> > > > that the hira.prob_cat_thresh setting defines the thresholds
for
> which
> > > > HiRA
> > > > probabilities should be computed. This is different from the
> cat_thresh
> > > > setting in the fcst and obs dictionaries. Those are used to
compute
> > > > categorical counts and statistics and are currently separate.
> > > >
> > > > It's nice that you don't have to use the same set of
thresholds for
> > > > categorical stats and the HiRA logic... it's more configurable
that
> > way.
> > > > However, I do see that it makes things a bit inconvenient.
Since
> you're
> > > > applying HiRA to many variables and each of those variables
has
> > different
> > > > meaningful thresholds, you'd need to configure the HiRA
dictionary
> > > > separately
> > > > for each and every variable.
> > > >
> > > > That's what is meant by this comment... that the hira
dictionary can
> be
> > > > specified separately for each variable:
> > > >
> > > > *// HiRA verification method// May be set separately in each
> > "obs.field"
> > > > entry*
> > > >
> > > > So you'd need to copy/paste the HiRA dictionary into every
single
> > > > obs.field
> > > > dictionary entry and modify the prob_cat_thresh thresholds
> accordingly.
> > > >
> > > > First, please make sure that you can in fact produce HiRA
output.
> > Second,
> > > > I
> > > > can think of a tweak to the logic that might make using HiRA
easier.
> > How
> > > > about, if the hira.prob_cat_thresh array is an empty list, we
could
> > have
> > > > the
> > > > code check to see if cat_thresh in the obs dictionary is non-
empty.
> And
> > > if
> > > > that's non-empty then we'd simply set prob_cat_thresh =
cat_thresh.
> > > >
> > > > I imagine that's what you were expecting in the first place...
that
> > > > Point-Stat
> > > > would just be using the thresholds from cat_thresh to define
the HiRA
> > > > probability thresholds. It still is nice to have
prob_cat_thresh be
> > > > configurable separately, but if left unset, defaulting to
cat_thresh
> > > seems
> > > > pretty reasonable to me. Do you think that would be a
worthwhile
> > change?
> > > >
> > > > Thanks,
> > > > John
> > > >
> > > > On Tue, Jun 30, 2020 at 7:12 PM Raby, John W USA CIV via RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > <Caution-Caution-Caution-Caution-Caution-url:
> > > > > Caution-Caution-
> > > Caution-Caution-
> Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> > > > >
> > > > > I forgot to attach the log file.
> > > > >
> > > > > ________________________________________
> > > > > From: Raby, John W CIV USARMY CCDC ARL (USA)
> > > > > Sent: Tuesday, June 30, 2020 7:00 PM
> > > > > To: met_help at ucar.edu
> > > > > Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790]
MET
> V9.0
> > > > > Point-Stat error (UNCLASSIFIED)
> > > > >
> > > > > I edited the config file to use the shortcut and corrected
the
> > > > > whitespace issue, but I get the same error.
> > > > > R/
> > > > > John
> > > > >
> > > > >
> > > > >
> > > > > ________________________________________
> > > > > From: Raby, John W CIV USARMY CCDC ARL (USA)
> > > > > Sent: Tuesday, June 30, 2020 6:10 PM
> > > > > To: met_help at ucar.edu
> > > > > Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790]
MET
> V9.0
> > > > > Point-Stat error (UNCLASSIFIED)
> > > > >
> > > > > Thanks for taking a look, John. I'll give that a try.
Appreciate
> the
> > > > > heads-up on a future error in the "desc".
> > > > >
> > > > > R/
> > > > > John
> > > > >
> > > > > ________________________________________
> > > > > From: John Halley Gotway via RT [met_help at ucar.edu]
> > > > > Sent: Tuesday, June 30, 2020 5:11 PM
> > > > > To: Raby, John W CIV USARMY CCDC ARL (USA)
> > > > > Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > > > > Point-Stat error (UNCLASSIFIED)
> > > > >
> > > > > All active links contained in this email were disabled.
Please
> > verify
> > > > > the identity of the sender, and confirm the authenticity of
all
> links
> > > > > contained within the message prior to copying and pasting
the
> address
> > > > > to a Web browser.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > ----
> > > > >
> > > > > John,
> > > > >
> > > > > Please try using this shortcut:
> > > > > prob_cat_thresh = [ ==0.25 ];
> > > > >
> > > > > Does that do it?
> > > > >
> > > > > Also, your "desc" setting can't contain embedded whitespace.
And
> > > > > Point-Stat should error out and tell you that.
> > > > >
> > > > > John
> > > > >
> > > > > On Tue, Jun 30, 2020 at 4:57 PM Raby, John W USA CIV via RT
<
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > > >
> > > > > > Tue Jun 30 16:56:45 2020: Request 95790 was acted upon.
> > > > > > Transaction: Ticket created by john.w.raby2.civ at mail.mil
> > > > > > Queue: met_help
> > > > > > Subject: MET V9.0 Point-Stat error (UNCLASSIFIED)
> > > > > > Owner: Nobody
> > > > > > Requestors: john.w.raby2.civ at mail.mil
> > > > > > Status: new
> > > > > > Ticket <Caution-Caution-Caution-Caution-Caution-Caution-
url: Caution-
> > > > > Caution-Caution-
> > > Caution-Caution-
> Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> > > > > >
> > > > > >
> > > > > > CLASSIFICATION: UNCLASSIFIED
> > > > > >
> > > > > > I keep getting the same error when trying to run Point-
Stat with
> > the
> > > > > > HiRA verification logic enabled. The error involves the
> probability
> > > > > > field
> > > > > > thresholds:
> > > > > >
> > > > > > ERROR:check_prob_thresh() -> When verifying a probability
field,
> > you
> > > > > > must select at least 3 thresholds beginning with 0.0 and
ending
> > with
> > > > 1.0.
> > > > > >
> > > > > > During troubleshooting, I've changed the settings for
> > > > > > prob_cat_thresh in the attached config file several times
> thinking
> > > > > > that the formatting of my settings was the issue. Maybe
I'm on
> the
> > > > > > wrong track. Could you take a
> > > > > look
> > > > > > at the config file to check the format or other possible
issue
> > which
> > > > > might
> > > > > > be causing this issue?
> > > > > >
> > > > > > Thanks.
> > > > > >
> > > > > > R/
> > > > > > John
> > > > > >
> > > > > > Mr. John W. Raby
> > > > > > U.S. Army Research Laboratory
> > > > > > White Sands Missile Range, NM 88002
> > > > > > Office:(575) 678-2004 DSN 258-2004
> > > > > > Teleworking: Contact Admin Specialist at (410) 278-6974
FAX (575)
> > > > > > 678-1230 DSN 258-1230
> > > > > > Email: john.w.raby2.civ at mail.mil
> > > > > >
> > > > > > CLASSIFICATION: UNCLASSIFIED
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > > CLASSIFICATION: UNCLASSIFIED
> > > >
> > > >
> > >
> > >
> > >
> > >
> >
> >
> >
>
>
>
>
------------------------------------------------
Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0 Point-Stat error (UNCLASSIFIED)
From: Raby, John W USA CIV
Time: Sun Jul 05 20:51:46 2020
John -
After researching the RPS score, found that it's not in Appendix C of
the User's Guide. Consider adding it in.
R/
John
________________________________________
From: Raby, John W CIV USARMY CCDC ARL (USA)
Sent: Sunday, July 05, 2020 5:10 PM
To: met_help at ucar.edu
Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
Point-Stat error (UNCLASSIFIED)
John -
In researching RPS used the link you gave me a while back for
accessing the issues on GitHub. Particularly like the ref numbers in
the User's Guide which link to the issue and discussion with
references, etc. Wealth of unfo!
Thanks.
R/
John
________________________________________
From: Raby, John W CIV USARMY CCDC ARL (USA)
Sent: Sunday, July 05, 2020 4:16 PM
To: met_help at ucar.edu
Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
Point-Stat error (UNCLASSIFIED)
Thanks, John. This helps. I'll read up on RPS to understand why those
requirements are required.
R/
John
________________________________________
From: John Halley Gotway via RT [met_help at ucar.edu]
Sent: Sunday, July 05, 2020 10:46 AM
To: Raby, John W CIV USARMY CCDC ARL (USA)
Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
Point-Stat error (UNCLASSIFIED)
All active links contained in this email were disabled. Please verify
the identity of the sender, and confirm the authenticity of all links
contained within the message prior to copying and pasting the address
to a Web browser.
----
John,
Just disable the RPS output line type (rps = NONE;). I believe that’s
where those threshold requirements originate. Or, if you would like
RPS
output, read up on that statistic and you’ll see why those
requirements are
in place.
On Sun, Jul 5, 2020 at 10:34 AM Raby, John W USA CIV via RT <
met_help at ucar.edu> wrote:
>
> <Caution-url: Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
>
> John -
>
> I tried running with the again with the HiRa method but with the
following
> thresholds:
> >=250, >=260, >=270, >=280, >=290, >=300, <=273, >=273
> It gave me an error because the thresholds are not monotonically
> increasing and with the same inequality. I thought that this error
only
> occurs when you have selected the MCTC and MCTS output which are
currently
> not selected. I recall running Grid-Stat with similar thresholds
without
> MCTC and MCTS output and it ran fine. What is it that ohnis causing
this
> error?
>
> Thanks.
>
> R/
> John
> ________________________________________
> From: Raby, John W CIV USARMY CCDC ARL (USA)
> Sent: Friday, July 03, 2020 6:43 PM
> To: met_help at ucar.edu
> Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
> Point-Stat error (UNCLASSIFIED)
>
> John -
>
> Your GitHub issue and the logic really help to better understand how
the
> HiRA methodology works. I was also pretty confused about that. Maybe
a
> re-look at the User's Guide description would be good to consider
too. I've
> attached the RPS and ECNT output file in case you wanted to see
them. Thank
> you for working through this with me.
>
> Have a great 4th of July!
>
> R/
> John
> ________________________________________
> From: John Halley Gotway via RT [met_help at ucar.edu]
> Sent: Friday, July 03, 2020 6:06 PM
> To: Raby, John W CIV USARMY CCDC ARL (USA)
> Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
> Point-Stat error (UNCLASSIFIED)
>
> All active links contained in this email were disabled. Please
verify the
> identity of the sender, and confirm the authenticity of all links
contained
> within the message prior to copying and pasting the address to a Web
> browser.
>
>
>
>
> ----
>
> John,
>
> I was looking at the code today and suspect that I've misled you a
little
> bit. I was confused about it myself. I updated this GitHub issue to
> clarify:
> Caution-Caution-https://github.com/NCAR/MET/issues/1400
>
> Upon review, you're correct that Point-Stat DOES use the cat_thresh
> settings from the fcst and obs dictionaries while processing HiRA.
It uses
> those thresholds to derive probability fields which are evaluated in
the
> output PCT, PTSD, PJC, and PRC line types. It also processes the
> neighborhood of points as a "spatial ensemble"... and from those
ensemble
> points, it computes the ECNT and RPS line types. And it's in the
> computation of the RPS line type that the prob_cat_thresh setting is
used.
>
> So prob_cat_thresh really only impacts the RPS line type.
>
> I am still making the change we discussed, but I wanted to clarify
how that
> change is actually being used (since I was confused about that
myself).
>
> You can see the tweak to the logic on line 1841 of:
> Caution-
> Caution-
https://github.com/NCAR/MET/blob/05f2e7d56bae3af092a69a5a9db4867f202eaae0/met/src/tools/core/point_stat/point_stat.cc#L1841
>
> Thanks,
> John
>
>
>
> On Thu, Jul 2, 2020 at 8:50 PM Raby, John W USA CIV via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <Caution-Caution-url: Caution-
> Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> >
> > John -
> >
> > Using your suggestion below, I was able to get Point-Stat to run.
See
> > attached log file and config file. I haven't looked at the output
yet,
> but
> > it was generated and there are no 0-sized files.
> >
> > Thanks for your help in getting it running!
> >
> > R/
> > John
> >
> > ________________________________________
> > From: John Halley Gotway via RT [met_help at ucar.edu]
> > Sent: Thursday, July 02, 2020 4:45 PM
> > To: Raby, John W CIV USARMY CCDC ARL (USA)
> > Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > Point-Stat error (UNCLASSIFIED)
> >
> > All active links contained in this email were disabled. Please
verify
> the
> > identity of the sender, and confirm the authenticity of all links
> contained
> > within the message prior to copying and pasting the address to a
Web
> > browser.
> >
> >
> >
> >
> > ----
> >
> > Yes, that's right, obs=fcst will copy everything over. And it
really
> > doesn't hurt having hira in both. Point-Stat just parses it from
the obs
> > dictionary.
> >
> > John
> >
> > On Thu, Jul 2, 2020 at 4:31 PM Raby, John W USA CIV via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <Caution-Caution-Caution-url: Caution-
> > Caution-Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> > >
> > > John -
> > >
> > > Thanks for taking a look and your quick reply. I see now about
> including
> > > the hira dictionary inside the field array dictionary. Thanks
for
> showing
> > > that below.
> > >
> > > So if I have the fcst dictionary only (obs=fcst) in my confiig
file
> then
> > I
> > > all I have to do is put the hira dictionary with settings for
the fcst
> > > variables, correct? I guess I'm "not understanding "copying it
to the
> > obs"
> > > when I don't explicitly show the obs dictionary.
> > >
> > > Good point about limiting the number of threshold and widths for
> testing.
> > >
> > > R/
> > > John
> > >
> > >
> > > ________________________________________
> > > From: John Halley Gotway via RT [met_help at ucar.edu]
> > > Sent: Thursday, July 02, 2020 4:14 PM
> > > To: Raby, John W CIV USARMY CCDC ARL (USA)
> > > Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > > Point-Stat error (UNCLASSIFIED)
> > >
> > > All active links contained in this email were disabled. Please
verify
> > the
> > > identity of the sender, and confirm the authenticity of all
links
> > contained
> > > within the message prior to copying and pasting the address to a
Web
> > > browser.
> > >
> > >
> > >
> > >
> > > ----
> > >
> > > John,
> > >
> > > Sorry for not answering your previous question. Yes, since
you're using
> > > obs=fcst, you can just define it in the fcst dictionary and copy
it
> over
> > to
> > > the obs.
> > >
> > > Yes, I do see a problem in your config file. The hira dictionary
needs
> to
> > > be INSIDE of the field array dictionary entry, as shown below.
> > >
> > > Also, for initial testing, I'd recommend using a much smaller
set of
> > > thresholds and widths. Running with all of these will take a
long time.
> > >
> > > John
> > >
> > > obs = {
> > > field = [
> > > {
> > > name = "TMP";
> > > level = [ "Z2" ];
> > > cat_thresh = [ >=250, >=260, >=270, >=280, >=290, >=300,
<=273,
> > > >=273 ];
> > > hira = {
> > > flag = TRUE;
> > > width = [ 3, 5, 7, 9, 11, 13, 15, 17, 19,
21, 23,
> 25,
> > > 27, 29, 31, 33, 35, 37, 39, 41, 43, 45, 47, 49 ];
> > > vld_thresh = 1.0;
> > > cov_thresh = [ ==0.25 ];
> > > shape = SQUARE;
> > > prob_cat_thresh = [ >=250, >=260, >=270, >=280, >=290,
>=300,
> > > <=273, >=273 ];
> > > }
> > > },
> > > ...
> > >
> > >
> > > On Thu, Jul 2, 2020 at 3:42 PM Raby, John W USA CIV via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <Caution-Caution-Caution-Caution-url: Caution-
> > > Caution-Caution-
> Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> > > >
> > > > CLASSIFICATION: UNCLASSIFIED
> > > >
> > > > John -
> > > >
> > > > I edited the attached config file to add the HiRA dictionary
to each
> > > > obs.field
> > > > dictionary entry as you described, but it seems like it's not
> expecting
> > > > the
> > > > hira dictionary to be where I placed it. Do you see where I
went
> wrong
> > > > with
> > > > this? The error dialog is also attached.
> > > >
> > > > Thanks and have a great 4th of July!
> > > >
> > > > R/
> > > > John
> > > >
> > > >
> > > >
> > > > -----Original Message-----
> > > > From: John Halley Gotway via RT <met_help at ucar.edu>
> > > > Sent: Wednesday, July 1, 2020 11:17 AM
> > > > To: Raby, John W CIV USARMY CCDC ARL (USA) <
> john.w.raby2.civ at mail.mil>
> > > > Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > > > Point-Stat
> > > > error (UNCLASSIFIED)
> > > >
> > > > All active links contained in this email were disabled.
Please
> verify
> > > the
> > > > identity of the sender, and confirm the authenticity of all
links
> > > > contained
> > > > within the message prior to copying and pasting the address to
a Web
> > > > browser.
> > > >
> > > >
> > > >
> > > >
> > > > ----
> > > >
> > > > John,
> > > >
> > > > Sorry for the confusion. I led you astray. The HiRA
configuration
> > option
> > > > names are not entirely intuitive.
> > > >
> > > > Please take a look at the README file starting on line 1418:
> > > > Caution-
> > > > Caution-
> > > Caution-
> > Caution-
> Caution-
https://github.com/NCAR/MET/blob/0cee9fbd0b3c691fd3cf4dd1d665f5d2b0db9a37/met/data/config/README#L1418
> > > >
> > > > I modified your hira entry like this:
> > > >
> > > > hira = {
> > > > flag = TRUE; *// set to TRUE/false to turn
on/off
> > this
> > > > logic*
> > > > width = [ 3, 5, 7, 9, 11 ]; *// defines the
> > neighborhood
> > > > size and must be > 1*
> > > > vld_thresh = 1.0;
> > > > cov_thresh = [ ==0.25 ]; *// this defines the
probability bins
> > to
> > > be
> > > > used. You only get to pick one setting here (==0.25 is short
for
> > > > >=0,>=0.25,>=0.5,>=0.75,>=1.0)*
> > > > shape = SQUARE;
> > > > prob_cat_thresh = [ >290, >300 ];* // this defines the
> threshold(s)
> > > for
> > > > which HiRA logic should be applied* }
> > > >
> > > > So you just had things a bit mis-configured. The key thing to
note
> here
> > > is
> > > > that the hira.prob_cat_thresh setting defines the thresholds
for
> which
> > > > HiRA
> > > > probabilities should be computed. This is different from the
> cat_thresh
> > > > setting in the fcst and obs dictionaries. Those are used to
compute
> > > > categorical counts and statistics and are currently separate.
> > > >
> > > > It's nice that you don't have to use the same set of
thresholds for
> > > > categorical stats and the HiRA logic... it's more configurable
that
> > way.
> > > > However, I do see that it makes things a bit inconvenient.
Since
> you're
> > > > applying HiRA to many variables and each of those variables
has
> > different
> > > > meaningful thresholds, you'd need to configure the HiRA
dictionary
> > > > separately
> > > > for each and every variable.
> > > >
> > > > That's what is meant by this comment... that the hira
dictionary can
> be
> > > > specified separately for each variable:
> > > >
> > > > *// HiRA verification method// May be set separately in each
> > "obs.field"
> > > > entry*
> > > >
> > > > So you'd need to copy/paste the HiRA dictionary into every
single
> > > > obs.field
> > > > dictionary entry and modify the prob_cat_thresh thresholds
> accordingly.
> > > >
> > > > First, please make sure that you can in fact produce HiRA
output.
> > Second,
> > > > I
> > > > can think of a tweak to the logic that might make using HiRA
easier.
> > How
> > > > about, if the hira.prob_cat_thresh array is an empty list, we
could
> > have
> > > > the
> > > > code check to see if cat_thresh in the obs dictionary is non-
empty.
> And
> > > if
> > > > that's non-empty then we'd simply set prob_cat_thresh =
cat_thresh.
> > > >
> > > > I imagine that's what you were expecting in the first place...
that
> > > > Point-Stat
> > > > would just be using the thresholds from cat_thresh to define
the HiRA
> > > > probability thresholds. It still is nice to have
prob_cat_thresh be
> > > > configurable separately, but if left unset, defaulting to
cat_thresh
> > > seems
> > > > pretty reasonable to me. Do you think that would be a
worthwhile
> > change?
> > > >
> > > > Thanks,
> > > > John
> > > >
> > > > On Tue, Jun 30, 2020 at 7:12 PM Raby, John W USA CIV via RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > <Caution-Caution-Caution-Caution-Caution-url:
> > > > > Caution-Caution-
> > > Caution-Caution-
> Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> > > > >
> > > > > I forgot to attach the log file.
> > > > >
> > > > > ________________________________________
> > > > > From: Raby, John W CIV USARMY CCDC ARL (USA)
> > > > > Sent: Tuesday, June 30, 2020 7:00 PM
> > > > > To: met_help at ucar.edu
> > > > > Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790]
MET
> V9.0
> > > > > Point-Stat error (UNCLASSIFIED)
> > > > >
> > > > > I edited the config file to use the shortcut and corrected
the
> > > > > whitespace issue, but I get the same error.
> > > > > R/
> > > > > John
> > > > >
> > > > >
> > > > >
> > > > > ________________________________________
> > > > > From: Raby, John W CIV USARMY CCDC ARL (USA)
> > > > > Sent: Tuesday, June 30, 2020 6:10 PM
> > > > > To: met_help at ucar.edu
> > > > > Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790]
MET
> V9.0
> > > > > Point-Stat error (UNCLASSIFIED)
> > > > >
> > > > > Thanks for taking a look, John. I'll give that a try.
Appreciate
> the
> > > > > heads-up on a future error in the "desc".
> > > > >
> > > > > R/
> > > > > John
> > > > >
> > > > > ________________________________________
> > > > > From: John Halley Gotway via RT [met_help at ucar.edu]
> > > > > Sent: Tuesday, June 30, 2020 5:11 PM
> > > > > To: Raby, John W CIV USARMY CCDC ARL (USA)
> > > > > Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > > > > Point-Stat error (UNCLASSIFIED)
> > > > >
> > > > > All active links contained in this email were disabled.
Please
> > verify
> > > > > the identity of the sender, and confirm the authenticity of
all
> links
> > > > > contained within the message prior to copying and pasting
the
> address
> > > > > to a Web browser.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > ----
> > > > >
> > > > > John,
> > > > >
> > > > > Please try using this shortcut:
> > > > > prob_cat_thresh = [ ==0.25 ];
> > > > >
> > > > > Does that do it?
> > > > >
> > > > > Also, your "desc" setting can't contain embedded whitespace.
And
> > > > > Point-Stat should error out and tell you that.
> > > > >
> > > > > John
> > > > >
> > > > > On Tue, Jun 30, 2020 at 4:57 PM Raby, John W USA CIV via RT
<
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > > >
> > > > > > Tue Jun 30 16:56:45 2020: Request 95790 was acted upon.
> > > > > > Transaction: Ticket created by john.w.raby2.civ at mail.mil
> > > > > > Queue: met_help
> > > > > > Subject: MET V9.0 Point-Stat error (UNCLASSIFIED)
> > > > > > Owner: Nobody
> > > > > > Requestors: john.w.raby2.civ at mail.mil
> > > > > > Status: new
> > > > > > Ticket <Caution-Caution-Caution-Caution-Caution-Caution-
url: Caution-
> > > > > Caution-Caution-
> > > Caution-Caution-
> Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> > > > > >
> > > > > >
> > > > > > CLASSIFICATION: UNCLASSIFIED
> > > > > >
> > > > > > I keep getting the same error when trying to run Point-
Stat with
> > the
> > > > > > HiRA verification logic enabled. The error involves the
> probability
> > > > > > field
> > > > > > thresholds:
> > > > > >
> > > > > > ERROR:check_prob_thresh() -> When verifying a probability
field,
> > you
> > > > > > must select at least 3 thresholds beginning with 0.0 and
ending
> > with
> > > > 1.0.
> > > > > >
> > > > > > During troubleshooting, I've changed the settings for
> > > > > > prob_cat_thresh in the attached config file several times
> thinking
> > > > > > that the formatting of my settings was the issue. Maybe
I'm on
> the
> > > > > > wrong track. Could you take a
> > > > > look
> > > > > > at the config file to check the format or other possible
issue
> > which
> > > > > might
> > > > > > be causing this issue?
> > > > > >
> > > > > > Thanks.
> > > > > >
> > > > > > R/
> > > > > > John
> > > > > >
> > > > > > Mr. John W. Raby
> > > > > > U.S. Army Research Laboratory
> > > > > > White Sands Missile Range, NM 88002
> > > > > > Office:(575) 678-2004 DSN 258-2004
> > > > > > Teleworking: Contact Admin Specialist at (410) 278-6974
FAX (575)
> > > > > > 678-1230 DSN 258-1230
> > > > > > Email: john.w.raby2.civ at mail.mil
> > > > > >
> > > > > > CLASSIFICATION: UNCLASSIFIED
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > > CLASSIFICATION: UNCLASSIFIED
> > > >
> > > >
> > >
> > >
> > >
> > >
> >
> >
> >
>
>
>
>
------------------------------------------------
Subject: MET V9.0 Point-Stat error (UNCLASSIFIED)
From: Raby, John W USA CIV
Time: Tue Jul 07 13:53:29 2020
John -
Table 9.3 of the MET User's Guide says that the RPS scores should be
for each threshold. See the following excerpt:
RPS_REL RPS Reliability, mean of the reliabilities for each RPS
threshold
28 RPS_RES RPS Resolution, mean of the resolutions for each RPS
threshold
29 RPS_UNC RPS Uncertainty, mean of the uncertainties for each RPS
threshold
30 RPS Ranked Probability Score, mean of the Brier Scores for each
RPS threshold
The attached output file contains the RPS scores for a run I did where
I expanded the number of thresholds to accomplish what I needed. It
looks like there is only one value for all thresholds, but there is
one score per neighborhood size. If the prob_cat_thresh entry defines
the thresholds to be used in computing the ranked probability score in
the RPS output line type, then there must be some dependency of RPS on
threshold value. Could you check into this to see what I should expect
in the output?
Thanks.
R/
John
________________________________________
From: Raby, John W CIV USARMY CCDC ARL (USA)
Sent: Sunday, July 05, 2020 8:51 PM
To: met_help at ucar.edu
Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
Point-Stat error (UNCLASSIFIED)
John -
After researching the RPS score, found that it's not in Appendix C of
the User's Guide. Consider adding it in.
R/
John
________________________________________
From: Raby, John W CIV USARMY CCDC ARL (USA)
Sent: Sunday, July 05, 2020 5:10 PM
To: met_help at ucar.edu
Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
Point-Stat error (UNCLASSIFIED)
John -
In researching RPS used the link you gave me a while back for
accessing the issues on GitHub. Particularly like the ref numbers in
the User's Guide which link to the issue and discussion with
references, etc. Wealth of unfo!
Thanks.
R/
John
________________________________________
From: Raby, John W CIV USARMY CCDC ARL (USA)
Sent: Sunday, July 05, 2020 4:16 PM
To: met_help at ucar.edu
Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
Point-Stat error (UNCLASSIFIED)
Thanks, John. This helps. I'll read up on RPS to understand why those
requirements are required.
R/
John
________________________________________
From: John Halley Gotway via RT [met_help at ucar.edu]
Sent: Sunday, July 05, 2020 10:46 AM
To: Raby, John W CIV USARMY CCDC ARL (USA)
Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
Point-Stat error (UNCLASSIFIED)
All active links contained in this email were disabled. Please verify
the identity of the sender, and confirm the authenticity of all links
contained within the message prior to copying and pasting the address
to a Web browser.
----
John,
Just disable the RPS output line type (rps = NONE;). I believe that’s
where those threshold requirements originate. Or, if you would like
RPS
output, read up on that statistic and you’ll see why those
requirements are
in place.
On Sun, Jul 5, 2020 at 10:34 AM Raby, John W USA CIV via RT <
met_help at ucar.edu> wrote:
>
> <Caution-url: Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
>
> John -
>
> I tried running with the again with the HiRa method but with the
following
> thresholds:
> >=250, >=260, >=270, >=280, >=290, >=300, <=273, >=273
> It gave me an error because the thresholds are not monotonically
> increasing and with the same inequality. I thought that this error
only
> occurs when you have selected the MCTC and MCTS output which are
currently
> not selected. I recall running Grid-Stat with similar thresholds
without
> MCTC and MCTS output and it ran fine. What is it that ohnis causing
this
> error?
>
> Thanks.
>
> R/
> John
> ________________________________________
> From: Raby, John W CIV USARMY CCDC ARL (USA)
> Sent: Friday, July 03, 2020 6:43 PM
> To: met_help at ucar.edu
> Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
> Point-Stat error (UNCLASSIFIED)
>
> John -
>
> Your GitHub issue and the logic really help to better understand how
the
> HiRA methodology works. I was also pretty confused about that. Maybe
a
> re-look at the User's Guide description would be good to consider
too. I've
> attached the RPS and ECNT output file in case you wanted to see
them. Thank
> you for working through this with me.
>
> Have a great 4th of July!
>
> R/
> John
> ________________________________________
> From: John Halley Gotway via RT [met_help at ucar.edu]
> Sent: Friday, July 03, 2020 6:06 PM
> To: Raby, John W CIV USARMY CCDC ARL (USA)
> Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
> Point-Stat error (UNCLASSIFIED)
>
> All active links contained in this email were disabled. Please
verify the
> identity of the sender, and confirm the authenticity of all links
contained
> within the message prior to copying and pasting the address to a Web
> browser.
>
>
>
>
> ----
>
> John,
>
> I was looking at the code today and suspect that I've misled you a
little
> bit. I was confused about it myself. I updated this GitHub issue to
> clarify:
> Caution-Caution-https://github.com/NCAR/MET/issues/1400
>
> Upon review, you're correct that Point-Stat DOES use the cat_thresh
> settings from the fcst and obs dictionaries while processing HiRA.
It uses
> those thresholds to derive probability fields which are evaluated in
the
> output PCT, PTSD, PJC, and PRC line types. It also processes the
> neighborhood of points as a "spatial ensemble"... and from those
ensemble
> points, it computes the ECNT and RPS line types. And it's in the
> computation of the RPS line type that the prob_cat_thresh setting is
used.
>
> So prob_cat_thresh really only impacts the RPS line type.
>
> I am still making the change we discussed, but I wanted to clarify
how that
> change is actually being used (since I was confused about that
myself).
>
> You can see the tweak to the logic on line 1841 of:
> Caution-
> Caution-
https://github.com/NCAR/MET/blob/05f2e7d56bae3af092a69a5a9db4867f202eaae0/met/src/tools/core/point_stat/point_stat.cc#L1841
>
> Thanks,
> John
>
>
>
> On Thu, Jul 2, 2020 at 8:50 PM Raby, John W USA CIV via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <Caution-Caution-url: Caution-
> Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> >
> > John -
> >
> > Using your suggestion below, I was able to get Point-Stat to run.
See
> > attached log file and config file. I haven't looked at the output
yet,
> but
> > it was generated and there are no 0-sized files.
> >
> > Thanks for your help in getting it running!
> >
> > R/
> > John
> >
> > ________________________________________
> > From: John Halley Gotway via RT [met_help at ucar.edu]
> > Sent: Thursday, July 02, 2020 4:45 PM
> > To: Raby, John W CIV USARMY CCDC ARL (USA)
> > Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > Point-Stat error (UNCLASSIFIED)
> >
> > All active links contained in this email were disabled. Please
verify
> the
> > identity of the sender, and confirm the authenticity of all links
> contained
> > within the message prior to copying and pasting the address to a
Web
> > browser.
> >
> >
> >
> >
> > ----
> >
> > Yes, that's right, obs=fcst will copy everything over. And it
really
> > doesn't hurt having hira in both. Point-Stat just parses it from
the obs
> > dictionary.
> >
> > John
> >
> > On Thu, Jul 2, 2020 at 4:31 PM Raby, John W USA CIV via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <Caution-Caution-Caution-url: Caution-
> > Caution-Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> > >
> > > John -
> > >
> > > Thanks for taking a look and your quick reply. I see now about
> including
> > > the hira dictionary inside the field array dictionary. Thanks
for
> showing
> > > that below.
> > >
> > > So if I have the fcst dictionary only (obs=fcst) in my confiig
file
> then
> > I
> > > all I have to do is put the hira dictionary with settings for
the fcst
> > > variables, correct? I guess I'm "not understanding "copying it
to the
> > obs"
> > > when I don't explicitly show the obs dictionary.
> > >
> > > Good point about limiting the number of threshold and widths for
> testing.
> > >
> > > R/
> > > John
> > >
> > >
> > > ________________________________________
> > > From: John Halley Gotway via RT [met_help at ucar.edu]
> > > Sent: Thursday, July 02, 2020 4:14 PM
> > > To: Raby, John W CIV USARMY CCDC ARL (USA)
> > > Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > > Point-Stat error (UNCLASSIFIED)
> > >
> > > All active links contained in this email were disabled. Please
verify
> > the
> > > identity of the sender, and confirm the authenticity of all
links
> > contained
> > > within the message prior to copying and pasting the address to a
Web
> > > browser.
> > >
> > >
> > >
> > >
> > > ----
> > >
> > > John,
> > >
> > > Sorry for not answering your previous question. Yes, since
you're using
> > > obs=fcst, you can just define it in the fcst dictionary and copy
it
> over
> > to
> > > the obs.
> > >
> > > Yes, I do see a problem in your config file. The hira dictionary
needs
> to
> > > be INSIDE of the field array dictionary entry, as shown below.
> > >
> > > Also, for initial testing, I'd recommend using a much smaller
set of
> > > thresholds and widths. Running with all of these will take a
long time.
> > >
> > > John
> > >
> > > obs = {
> > > field = [
> > > {
> > > name = "TMP";
> > > level = [ "Z2" ];
> > > cat_thresh = [ >=250, >=260, >=270, >=280, >=290, >=300,
<=273,
> > > >=273 ];
> > > hira = {
> > > flag = TRUE;
> > > width = [ 3, 5, 7, 9, 11, 13, 15, 17, 19,
21, 23,
> 25,
> > > 27, 29, 31, 33, 35, 37, 39, 41, 43, 45, 47, 49 ];
> > > vld_thresh = 1.0;
> > > cov_thresh = [ ==0.25 ];
> > > shape = SQUARE;
> > > prob_cat_thresh = [ >=250, >=260, >=270, >=280, >=290,
>=300,
> > > <=273, >=273 ];
> > > }
> > > },
> > > ...
> > >
> > >
> > > On Thu, Jul 2, 2020 at 3:42 PM Raby, John W USA CIV via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <Caution-Caution-Caution-Caution-url: Caution-
> > > Caution-Caution-
> Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> > > >
> > > > CLASSIFICATION: UNCLASSIFIED
> > > >
> > > > John -
> > > >
> > > > I edited the attached config file to add the HiRA dictionary
to each
> > > > obs.field
> > > > dictionary entry as you described, but it seems like it's not
> expecting
> > > > the
> > > > hira dictionary to be where I placed it. Do you see where I
went
> wrong
> > > > with
> > > > this? The error dialog is also attached.
> > > >
> > > > Thanks and have a great 4th of July!
> > > >
> > > > R/
> > > > John
> > > >
> > > >
> > > >
> > > > -----Original Message-----
> > > > From: John Halley Gotway via RT <met_help at ucar.edu>
> > > > Sent: Wednesday, July 1, 2020 11:17 AM
> > > > To: Raby, John W CIV USARMY CCDC ARL (USA) <
> john.w.raby2.civ at mail.mil>
> > > > Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > > > Point-Stat
> > > > error (UNCLASSIFIED)
> > > >
> > > > All active links contained in this email were disabled.
Please
> verify
> > > the
> > > > identity of the sender, and confirm the authenticity of all
links
> > > > contained
> > > > within the message prior to copying and pasting the address to
a Web
> > > > browser.
> > > >
> > > >
> > > >
> > > >
> > > > ----
> > > >
> > > > John,
> > > >
> > > > Sorry for the confusion. I led you astray. The HiRA
configuration
> > option
> > > > names are not entirely intuitive.
> > > >
> > > > Please take a look at the README file starting on line 1418:
> > > > Caution-
> > > > Caution-
> > > Caution-
> > Caution-
> Caution-
https://github.com/NCAR/MET/blob/0cee9fbd0b3c691fd3cf4dd1d665f5d2b0db9a37/met/data/config/README#L1418
> > > >
> > > > I modified your hira entry like this:
> > > >
> > > > hira = {
> > > > flag = TRUE; *// set to TRUE/false to turn
on/off
> > this
> > > > logic*
> > > > width = [ 3, 5, 7, 9, 11 ]; *// defines the
> > neighborhood
> > > > size and must be > 1*
> > > > vld_thresh = 1.0;
> > > > cov_thresh = [ ==0.25 ]; *// this defines the
probability bins
> > to
> > > be
> > > > used. You only get to pick one setting here (==0.25 is short
for
> > > > >=0,>=0.25,>=0.5,>=0.75,>=1.0)*
> > > > shape = SQUARE;
> > > > prob_cat_thresh = [ >290, >300 ];* // this defines the
> threshold(s)
> > > for
> > > > which HiRA logic should be applied* }
> > > >
> > > > So you just had things a bit mis-configured. The key thing to
note
> here
> > > is
> > > > that the hira.prob_cat_thresh setting defines the thresholds
for
> which
> > > > HiRA
> > > > probabilities should be computed. This is different from the
> cat_thresh
> > > > setting in the fcst and obs dictionaries. Those are used to
compute
> > > > categorical counts and statistics and are currently separate.
> > > >
> > > > It's nice that you don't have to use the same set of
thresholds for
> > > > categorical stats and the HiRA logic... it's more configurable
that
> > way.
> > > > However, I do see that it makes things a bit inconvenient.
Since
> you're
> > > > applying HiRA to many variables and each of those variables
has
> > different
> > > > meaningful thresholds, you'd need to configure the HiRA
dictionary
> > > > separately
> > > > for each and every variable.
> > > >
> > > > That's what is meant by this comment... that the hira
dictionary can
> be
> > > > specified separately for each variable:
> > > >
> > > > *// HiRA verification method// May be set separately in each
> > "obs.field"
> > > > entry*
> > > >
> > > > So you'd need to copy/paste the HiRA dictionary into every
single
> > > > obs.field
> > > > dictionary entry and modify the prob_cat_thresh thresholds
> accordingly.
> > > >
> > > > First, please make sure that you can in fact produce HiRA
output.
> > Second,
> > > > I
> > > > can think of a tweak to the logic that might make using HiRA
easier.
> > How
> > > > about, if the hira.prob_cat_thresh array is an empty list, we
could
> > have
> > > > the
> > > > code check to see if cat_thresh in the obs dictionary is non-
empty.
> And
> > > if
> > > > that's non-empty then we'd simply set prob_cat_thresh =
cat_thresh.
> > > >
> > > > I imagine that's what you were expecting in the first place...
that
> > > > Point-Stat
> > > > would just be using the thresholds from cat_thresh to define
the HiRA
> > > > probability thresholds. It still is nice to have
prob_cat_thresh be
> > > > configurable separately, but if left unset, defaulting to
cat_thresh
> > > seems
> > > > pretty reasonable to me. Do you think that would be a
worthwhile
> > change?
> > > >
> > > > Thanks,
> > > > John
> > > >
> > > > On Tue, Jun 30, 2020 at 7:12 PM Raby, John W USA CIV via RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > <Caution-Caution-Caution-Caution-Caution-url:
> > > > > Caution-Caution-
> > > Caution-Caution-
> Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> > > > >
> > > > > I forgot to attach the log file.
> > > > >
> > > > > ________________________________________
> > > > > From: Raby, John W CIV USARMY CCDC ARL (USA)
> > > > > Sent: Tuesday, June 30, 2020 7:00 PM
> > > > > To: met_help at ucar.edu
> > > > > Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790]
MET
> V9.0
> > > > > Point-Stat error (UNCLASSIFIED)
> > > > >
> > > > > I edited the config file to use the shortcut and corrected
the
> > > > > whitespace issue, but I get the same error.
> > > > > R/
> > > > > John
> > > > >
> > > > >
> > > > >
> > > > > ________________________________________
> > > > > From: Raby, John W CIV USARMY CCDC ARL (USA)
> > > > > Sent: Tuesday, June 30, 2020 6:10 PM
> > > > > To: met_help at ucar.edu
> > > > > Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790]
MET
> V9.0
> > > > > Point-Stat error (UNCLASSIFIED)
> > > > >
> > > > > Thanks for taking a look, John. I'll give that a try.
Appreciate
> the
> > > > > heads-up on a future error in the "desc".
> > > > >
> > > > > R/
> > > > > John
> > > > >
> > > > > ________________________________________
> > > > > From: John Halley Gotway via RT [met_help at ucar.edu]
> > > > > Sent: Tuesday, June 30, 2020 5:11 PM
> > > > > To: Raby, John W CIV USARMY CCDC ARL (USA)
> > > > > Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > > > > Point-Stat error (UNCLASSIFIED)
> > > > >
> > > > > All active links contained in this email were disabled.
Please
> > verify
> > > > > the identity of the sender, and confirm the authenticity of
all
> links
> > > > > contained within the message prior to copying and pasting
the
> address
> > > > > to a Web browser.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > ----
> > > > >
> > > > > John,
> > > > >
> > > > > Please try using this shortcut:
> > > > > prob_cat_thresh = [ ==0.25 ];
> > > > >
> > > > > Does that do it?
> > > > >
> > > > > Also, your "desc" setting can't contain embedded whitespace.
And
> > > > > Point-Stat should error out and tell you that.
> > > > >
> > > > > John
> > > > >
> > > > > On Tue, Jun 30, 2020 at 4:57 PM Raby, John W USA CIV via RT
<
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > > >
> > > > > > Tue Jun 30 16:56:45 2020: Request 95790 was acted upon.
> > > > > > Transaction: Ticket created by john.w.raby2.civ at mail.mil
> > > > > > Queue: met_help
> > > > > > Subject: MET V9.0 Point-Stat error (UNCLASSIFIED)
> > > > > > Owner: Nobody
> > > > > > Requestors: john.w.raby2.civ at mail.mil
> > > > > > Status: new
> > > > > > Ticket <Caution-Caution-Caution-Caution-Caution-Caution-
url: Caution-
> > > > > Caution-Caution-
> > > Caution-Caution-
> Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> > > > > >
> > > > > >
> > > > > > CLASSIFICATION: UNCLASSIFIED
> > > > > >
> > > > > > I keep getting the same error when trying to run Point-
Stat with
> > the
> > > > > > HiRA verification logic enabled. The error involves the
> probability
> > > > > > field
> > > > > > thresholds:
> > > > > >
> > > > > > ERROR:check_prob_thresh() -> When verifying a probability
field,
> > you
> > > > > > must select at least 3 thresholds beginning with 0.0 and
ending
> > with
> > > > 1.0.
> > > > > >
> > > > > > During troubleshooting, I've changed the settings for
> > > > > > prob_cat_thresh in the attached config file several times
> thinking
> > > > > > that the formatting of my settings was the issue. Maybe
I'm on
> the
> > > > > > wrong track. Could you take a
> > > > > look
> > > > > > at the config file to check the format or other possible
issue
> > which
> > > > > might
> > > > > > be causing this issue?
> > > > > >
> > > > > > Thanks.
> > > > > >
> > > > > > R/
> > > > > > John
> > > > > >
> > > > > > Mr. John W. Raby
> > > > > > U.S. Army Research Laboratory
> > > > > > White Sands Missile Range, NM 88002
> > > > > > Office:(575) 678-2004 DSN 258-2004
> > > > > > Teleworking: Contact Admin Specialist at (410) 278-6974
FAX (575)
> > > > > > 678-1230 DSN 258-1230
> > > > > > Email: john.w.raby2.civ at mail.mil
> > > > > >
> > > > > > CLASSIFICATION: UNCLASSIFIED
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > > CLASSIFICATION: UNCLASSIFIED
> > > >
> > > >
> > >
> > >
> > >
> > >
> >
> >
> >
>
>
>
>
------------------------------------------------
Subject: MET V9.0 Point-Stat error (UNCLASSIFIED)
From: Raby, John W USA CIV
Time: Tue Jul 07 13:53:29 2020
VERSION MODEL DESC FCST_LEAD FCST_VALID_BEG
FCST_VALID_END OBS_LEAD OBS_VALID_BEG OBS_VALID_END FCST_VAR
FCST_UNITS FCST_LEV OBS_VAR OBS_UNITS OBS_LEV OBTYPE VX_MASK
INTERP_MTHD INTERP_PNTS FCST_THRESH
OBS_THRESH COV_THRESH ALPHA LINE_TYPE TOTAL N_PROB RPS_REL RPS_RES
RPS_UNC RPS RPSS RPSS_SMPL
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 TMP K
Z2 TMP NA Z2 ADPSFC MSA_d03 NBRHD_SQUARE 9
>=250,>=260,>=270,>=273,>=280,>=290,>=300
NA NA NA RPS 90 9 0.010877 0.0283
0.046102 0.028679 NA 0.37792
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 TMP K
Z2 TMP NA Z2 ADPSFC MSA_d03 NBRHD_SQUARE 25
>=250,>=260,>=270,>=273,>=280,>=290,>=300
NA NA NA RPS 89 25 0.0099064 0.030415
0.046026 0.025517 NA 0.44559
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 TMP K
Z2 TMP NA Z2 ADPSFC MSA_d03 NBRHD_SQUARE 49
>=250,>=260,>=270,>=273,>=280,>=290,>=300
NA NA NA RPS 89 49 0.012276 0.033264
0.046026 0.025038 NA 0.45601
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 TMP K
Z2 TMP NA Z2 ADPSFC MSA_d03 NBRHD_SQUARE 81
>=250,>=260,>=270,>=273,>=280,>=290,>=300
NA NA NA RPS 89 81 0.01551 0.03518
0.046026 0.026356 NA 0.42737
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 TMP K
Z2 TMP NA Z2 ADPSFC MSA_d03 NBRHD_SQUARE 121
>=250,>=260,>=270,>=273,>=280,>=290,>=300
NA NA NA RPS 89 121 0.018709 0.037078
0.046026 0.027656 NA 0.39912
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 TMP K
Z2 TMP NA Z2 ADPSFC MSA_d03 NBRHD_SQUARE 169
>=250,>=260,>=270,>=273,>=280,>=290,>=300
NA NA NA RPS 88 169 0.019028 0.036006
0.046285 0.029306 NA 0.36682
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 TMP K
Z2 TMP NA Z2 ADPSFC MSA_d03 NBRHD_SQUARE 225
>=250,>=260,>=270,>=273,>=280,>=290,>=300
NA NA NA RPS 88 225 0.021595 0.037449
0.046285 0.030431 NA 0.34252
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 TMP K
Z2 TMP NA Z2 ADPSFC MSA_d03 NBRHD_SQUARE 289
>=250,>=260,>=270,>=273,>=280,>=290,>=300
NA NA NA RPS 85 289 0.025313 0.040126
0.045635 0.030822 NA 0.32461
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 TMP K
Z2 TMP NA Z2 ADPSFC MSA_d03 NBRHD_SQUARE 361
>=250,>=260,>=270,>=273,>=280,>=290,>=300
NA NA NA RPS 83 361 0.026731 0.039415
0.044626 0.031942 NA 0.28422
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 TMP K
Z2 TMP NA Z2 ADPSFC MSA_d03 NBRHD_SQUARE 441
>=250,>=260,>=270,>=273,>=280,>=290,>=300
NA NA NA RPS 83 441 0.02828 0.040782
0.044626 0.032124 NA 0.28016
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 TMP K
Z2 TMP NA Z2 ADPSFC MSA_d03 NBRHD_SQUARE 529
>=250,>=260,>=270,>=273,>=280,>=290,>=300
NA NA NA RPS 83 529 0.027201 0.039865
0.044626 0.031962 NA 0.28379
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 TMP K
Z2 TMP NA Z2 ADPSFC MSA_d03 NBRHD_SQUARE 625
>=250,>=260,>=270,>=273,>=280,>=290,>=300
NA NA NA RPS 82 625 0.029371 0.041919
0.044574 0.032026 NA 0.28151
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 TMP K
Z2 TMP NA Z2 ADPSFC MSA_d03 NBRHD_SQUARE 729
>=250,>=260,>=270,>=273,>=280,>=290,>=300
NA NA NA RPS 80 729 0.028449 0.041133
0.045045 0.03236 NA 0.28159
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 TMP K
Z2 TMP NA Z2 ADPSFC MSA_d03 NBRHD_SQUARE 841
>=250,>=260,>=270,>=273,>=280,>=290,>=300
NA NA NA RPS 79 841 0.028276 0.041376
0.045277 0.032177 NA 0.28933
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 TMP K
Z2 TMP NA Z2 ADPSFC MSA_d03 NBRHD_SQUARE 961
>=250,>=260,>=270,>=273,>=280,>=290,>=300
NA NA NA RPS 79 961 0.027622 0.041165
0.045277 0.031734 NA 0.29912
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 TMP K
Z2 TMP NA Z2 ADPSFC MSA_d03 NBRHD_SQUARE 1089
>=250,>=260,>=270,>=273,>=280,>=290,>=300
NA NA NA RPS 78 1089 0.026959 0.040026
0.043886 0.030818 NA 0.29776
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 TMP K
Z2 TMP NA Z2 ADPSFC MSA_d03 NBRHD_SQUARE 1225
>=250,>=260,>=270,>=273,>=280,>=290,>=300
NA NA NA RPS 78 1225 0.027454 0.040789
0.043886 0.03055 NA 0.30388
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 TMP K
Z2 TMP NA Z2 ADPSFC MSA_d03 NBRHD_SQUARE 1369
>=250,>=260,>=270,>=273,>=280,>=290,>=300
NA NA NA RPS 78 1369 0.026372 0.039902
0.043886 0.030355 NA 0.30831
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 TMP K
Z2 TMP NA Z2 ADPSFC MSA_d03 NBRHD_SQUARE 1521
>=250,>=260,>=270,>=273,>=280,>=290,>=300
NA NA NA RPS 77 1521 0.02743 0.04096
0.044093 0.030563 NA 0.30685
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 TMP K
Z2 TMP NA Z2 ADPSFC MSA_d03 NBRHD_SQUARE 1681
>=250,>=260,>=270,>=273,>=280,>=290,>=300
NA NA NA RPS 76 1681 0.027586 0.041059
0.044297 0.030823 NA 0.30417
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 TMP K
Z2 TMP NA Z2 ADPSFC MSA_d03 NBRHD_SQUARE 1849
>=250,>=260,>=270,>=273,>=280,>=290,>=300
NA NA NA RPS 76 1849 0.027412 0.040977
0.044297 0.030732 NA 0.30623
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 TMP K
Z2 TMP NA Z2 ADPSFC MSA_d03 NBRHD_SQUARE 2025
>=250,>=260,>=270,>=273,>=280,>=290,>=300
NA NA NA RPS 76 2025 0.02739 0.041007
0.044297 0.03068 NA 0.30741
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 TMP K
Z2 TMP NA Z2 ADPSFC MSA_d03 NBRHD_SQUARE 2209
>=250,>=260,>=270,>=273,>=280,>=290,>=300
NA NA NA RPS 76 2209 0.027267 0.040962
0.044297 0.030601 NA 0.30917
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 TMP K
Z2 TMP NA Z2 ADPSFC MSA_d03 NBRHD_SQUARE 2401
>=250,>=260,>=270,>=273,>=280,>=290,>=300
NA NA NA RPS 76 2401 0.027788 0.041634
0.044297 0.030451 NA 0.31256
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 UGRD m/s
Z10 UGRD NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 9
>=-16.0,>=-8.0,>=0.0,>=8.0,>=16.0
NA NA NA RPS 12 9 0.021502
0.0069444 0.015278 0.029835 NA -0.95286
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 UGRD m/s
Z10 UGRD NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 25
>=-16.0,>=-8.0,>=0.0,>=8.0,>=16.0
NA NA NA RPS 12 25 0.03496 0.015278
0.015278 0.03496 NA -1.28829
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 UGRD m/s
Z10 UGRD NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 49
>=-16.0,>=-8.0,>=0.0,>=8.0,>=16.0
NA NA NA RPS 12 49 0.040004 0.015278
0.015278 0.040004 NA -1.61845
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 UGRD m/s
Z10 UGRD NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 81
>=-16.0,>=-8.0,>=0.0,>=8.0,>=16.0
NA NA NA RPS 12 81 0.04398 0.015278
0.015278 0.04398 NA -1.87866
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 UGRD m/s
Z10 UGRD NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 121
>=-16.0,>=-8.0,>=0.0,>=8.0,>=16.0
NA NA NA RPS 12 121 0.046023 0.015278
0.015278 0.046023 NA -2.01239
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 UGRD m/s
Z10 UGRD NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 169
>=-16.0,>=-8.0,>=0.0,>=8.0,>=16.0
NA NA NA RPS 12 169 0.048804 0.015278
0.015278 0.048804 NA -2.19443
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 UGRD m/s
Z10 UGRD NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 225
>=-16.0,>=-8.0,>=0.0,>=8.0,>=16.0
NA NA NA RPS 12 225 0.051761 0.015278
0.015278 0.051761 NA -2.38797
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 UGRD m/s
Z10 UGRD NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 289
>=-16.0,>=-8.0,>=0.0,>=8.0,>=16.0
NA NA NA RPS 12 289 0.046069
0.0069444 0.015278 0.054402 NA -2.56086
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 UGRD m/s
Z10 UGRD NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 361
>=-16.0,>=-8.0,>=0.0,>=8.0,>=16.0
NA NA NA RPS 12 361 0.057475 0.015278
0.015278 0.057475 NA -2.76198
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 UGRD m/s
Z10 UGRD NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 441
>=-16.0,>=-8.0,>=0.0,>=8.0,>=16.0
NA NA NA RPS 12 441 0.059778 0.015278
0.015278 0.059778 NA -2.91275
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 UGRD m/s
Z10 UGRD NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 529
>=-16.0,>=-8.0,>=0.0,>=8.0,>=16.0
NA NA NA RPS 12 529 0.060863 0.015278
0.015278 0.060863 NA -2.98376
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 UGRD m/s
Z10 UGRD NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 625
>=-16.0,>=-8.0,>=0.0,>=8.0,>=16.0
NA NA NA RPS 12 625 0.06149 0.015278
0.015278 0.06149 NA -3.0248
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 UGRD m/s
Z10 UGRD NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 729
>=-16.0,>=-8.0,>=0.0,>=8.0,>=16.0
NA NA NA RPS 12 729 0.061676 0.015278
0.015278 0.061676 NA -3.03698
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 UGRD m/s
Z10 UGRD NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 841
>=-16.0,>=-8.0,>=0.0,>=8.0,>=16.0
NA NA NA RPS 12 841 0.060972 0.015278
0.015278 0.060972 NA -2.9909
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 UGRD m/s
Z10 UGRD NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 961
>=-16.0,>=-8.0,>=0.0,>=8.0,>=16.0
NA NA NA RPS 12 961 0.060242 0.015278
0.015278 0.060242 NA -2.94313
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 UGRD m/s
Z10 UGRD NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 1089
>=-16.0,>=-8.0,>=0.0,>=8.0,>=16.0
NA NA NA RPS 12 1089 0.060041 0.015278
0.015278 0.060041 NA -2.92996
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 UGRD m/s
Z10 UGRD NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 1225
>=-16.0,>=-8.0,>=0.0,>=8.0,>=16.0
NA NA NA RPS 12 1225 0.059991 0.015278
0.015278 0.059991 NA -2.92668
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 UGRD m/s
Z10 UGRD NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 1369
>=-16.0,>=-8.0,>=0.0,>=8.0,>=16.0
NA NA NA RPS 12 1369 0.05991 0.015278
0.015278 0.05991 NA -2.92141
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 UGRD m/s
Z10 UGRD NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 1521
>=-16.0,>=-8.0,>=0.0,>=8.0,>=16.0
NA NA NA RPS 12 1521 0.059845 0.015278
0.015278 0.059845 NA -2.91711
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 UGRD m/s
Z10 UGRD NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 1681
>=-16.0,>=-8.0,>=0.0,>=8.0,>=16.0
NA NA NA RPS 11 1681 0.063243 0.016529
0.016529 0.063243 NA -2.82621
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 UGRD m/s
Z10 UGRD NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 1849
>=-16.0,>=-8.0,>=0.0,>=8.0,>=16.0
NA NA NA RPS 11 1849 0.06301 0.016529
0.016529 0.06301 NA -2.8121
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 UGRD m/s
Z10 UGRD NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 2025
>=-16.0,>=-8.0,>=0.0,>=8.0,>=16.0
NA NA NA RPS 11 2025 0.062508 0.016529
0.016529 0.062508 NA -2.78173
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 UGRD m/s
Z10 UGRD NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 2209
>=-16.0,>=-8.0,>=0.0,>=8.0,>=16.0
NA NA NA RPS 11 2209 0.062101 0.016529
0.016529 0.062101 NA -2.75708
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 UGRD m/s
Z10 UGRD NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 2401
>=-16.0,>=-8.0,>=0.0,>=8.0,>=16.0
NA NA NA RPS 11 2401 0.061692 0.016529
0.016529 0.061692 NA -2.73239
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 VGRD m/s
Z10 VGRD NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 9
>=-16.0,>=-8.0,>=0.0,>=8.0,>=16.0
NA NA NA RPS 12 9 0.0027497 0.012626
0.027778 0.017901 NA 0.35556
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 VGRD m/s
Z10 VGRD NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 25
>=-16.0,>=-8.0,>=0.0,>=8.0,>=16.0
NA NA NA RPS 12 25 0.013947 0.027778
0.027778 0.013947 NA 0.49792
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 VGRD m/s
Z10 VGRD NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 49
>=-16.0,>=-8.0,>=0.0,>=8.0,>=16.0
NA NA NA RPS 12 49 0.011544 0.027778
0.027778 0.011544 NA 0.58442
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 VGRD m/s
Z10 VGRD NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 81
>=-16.0,>=-8.0,>=0.0,>=8.0,>=16.0
NA NA NA RPS 12 81 0.011403 0.027778
0.027778 0.011403 NA 0.58948
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 VGRD m/s
Z10 VGRD NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 121
>=-16.0,>=-8.0,>=0.0,>=8.0,>=16.0
NA NA NA RPS 12 121 0.012544 0.027778
0.027778 0.012544 NA 0.54843
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 VGRD m/s
Z10 VGRD NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 169
>=-16.0,>=-8.0,>=0.0,>=8.0,>=16.0
NA NA NA RPS 12 169 0.013925 0.027778
0.027778 0.013925 NA 0.49869
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 VGRD m/s
Z10 VGRD NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 225
>=-16.0,>=-8.0,>=0.0,>=8.0,>=16.0
NA NA NA RPS 12 225 0.015055 0.027778
0.027778 0.015055 NA 0.458
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 VGRD m/s
Z10 VGRD NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 289
>=-16.0,>=-8.0,>=0.0,>=8.0,>=16.0
NA NA NA RPS 12 289 0.015822 0.027778
0.027778 0.015822 NA 0.4304
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 VGRD m/s
Z10 VGRD NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 361
>=-16.0,>=-8.0,>=0.0,>=8.0,>=16.0
NA NA NA RPS 12 361 0.016524 0.027778
0.027778 0.016524 NA 0.40514
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 VGRD m/s
Z10 VGRD NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 441
>=-16.0,>=-8.0,>=0.0,>=8.0,>=16.0
NA NA NA RPS 12 441 0.017664 0.027778
0.027778 0.017664 NA 0.3641
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 VGRD m/s
Z10 VGRD NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 529
>=-16.0,>=-8.0,>=0.0,>=8.0,>=16.0
NA NA NA RPS 12 529 0.010224 0.019444
0.027778 0.018558 NA 0.33193
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 VGRD m/s
Z10 VGRD NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 625
>=-16.0,>=-8.0,>=0.0,>=8.0,>=16.0
NA NA NA RPS 12 625 0.018678 0.027778
0.027778 0.018678 NA 0.32761
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 VGRD m/s
Z10 VGRD NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 729
>=-16.0,>=-8.0,>=0.0,>=8.0,>=16.0
NA NA NA RPS 12 729 0.019277 0.027778
0.027778 0.019277 NA 0.30604
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 VGRD m/s
Z10 VGRD NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 841
>=-16.0,>=-8.0,>=0.0,>=8.0,>=16.0
NA NA NA RPS 12 841 0.019904 0.027778
0.027778 0.019904 NA 0.28346
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 VGRD m/s
Z10 VGRD NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 961
>=-16.0,>=-8.0,>=0.0,>=8.0,>=16.0
NA NA NA RPS 12 961 0.020608 0.027778
0.027778 0.020608 NA 0.2581
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 VGRD m/s
Z10 VGRD NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 1089
>=-16.0,>=-8.0,>=0.0,>=8.0,>=16.0
NA NA NA RPS 12 1089 0.021203 0.027778
0.027778 0.021203 NA 0.2367
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 VGRD m/s
Z10 VGRD NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 1225
>=-16.0,>=-8.0,>=0.0,>=8.0,>=16.0
NA NA NA RPS 12 1225 0.021588 0.027778
0.027778 0.021588 NA 0.22282
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 VGRD m/s
Z10 VGRD NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 1369
>=-16.0,>=-8.0,>=0.0,>=8.0,>=16.0
NA NA NA RPS 12 1369 0.02202 0.027778
0.027778 0.02202 NA 0.20729
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 VGRD m/s
Z10 VGRD NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 1521
>=-16.0,>=-8.0,>=0.0,>=8.0,>=16.0
NA NA NA RPS 12 1521 0.02256 0.027778
0.027778 0.02256 NA 0.18785
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 VGRD m/s
Z10 VGRD NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 1681
>=-16.0,>=-8.0,>=0.0,>=8.0,>=16.0
NA NA NA RPS 11 1681 0.025259 0.029752
0.029752 0.025259 NA 0.151
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 VGRD m/s
Z10 VGRD NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 1849
>=-16.0,>=-8.0,>=0.0,>=8.0,>=16.0
NA NA NA RPS 11 1849 0.025789 0.029752
0.029752 0.025789 NA 0.1332
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 VGRD m/s
Z10 VGRD NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 2025
>=-16.0,>=-8.0,>=0.0,>=8.0,>=16.0
NA NA NA RPS 11 2025 0.026221 0.029752
0.029752 0.026221 NA 0.11869
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 VGRD m/s
Z10 VGRD NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 2209
>=-16.0,>=-8.0,>=0.0,>=8.0,>=16.0
NA NA NA RPS 11 2209 0.026523 0.029752
0.029752 0.026523 NA 0.10852
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 VGRD m/s
Z10 VGRD NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 2401
>=-16.0,>=-8.0,>=0.0,>=8.0,>=16.0
NA NA NA RPS 11 2401 0.026879 0.029752
0.029752 0.026879 NA 0.096568
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 DPT K
Z2 DPT NA Z2 ADPSFC MSA_d03 NBRHD_SQUARE 9
>=205,>=220,>=235,>=250,>=265,>=280
NA NA NA RPS 76 9 0.0030864 0
0 0.0030864 NA NA
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 DPT K
Z2 DPT NA Z2 ADPSFC MSA_d03 NBRHD_SQUARE 25
>=205,>=220,>=235,>=250,>=265,>=280
NA NA NA RPS 76 25 0.0004 0
0 0.0004 NA NA
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 DPT K
Z2 DPT NA Z2 ADPSFC MSA_d03 NBRHD_SQUARE 49
>=205,>=220,>=235,>=250,>=265,>=280
NA NA NA RPS 76 49 0.00010412 0
0 0.00010412 NA NA
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 DPT K
Z2 DPT NA Z2 ADPSFC MSA_d03 NBRHD_SQUARE 81
>=205,>=220,>=235,>=250,>=265,>=280
NA NA NA RPS 76 81 3.8104e-05 0
0 3.8104e-05 NA NA
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 DPT K
Z2 DPT NA Z2 ADPSFC MSA_d03 NBRHD_SQUARE 121
>=205,>=220,>=235,>=250,>=265,>=280
NA NA NA RPS 76 121 1.7075e-05 0
0 1.7075e-05 NA NA
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 DPT K
Z2 DPT NA Z2 ADPSFC MSA_d03 NBRHD_SQUARE 169
>=205,>=220,>=235,>=250,>=265,>=280
NA NA NA RPS 75 169 8.7532e-06 0
0 8.7532e-06 NA NA
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 DPT K
Z2 DPT NA Z2 ADPSFC MSA_d03 NBRHD_SQUARE 225
>=205,>=220,>=235,>=250,>=265,>=280
NA NA NA RPS 75 225 4.9383e-06 0
0 4.9383e-06 NA NA
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 DPT K
Z2 DPT NA Z2 ADPSFC MSA_d03 NBRHD_SQUARE 289
>=205,>=220,>=235,>=250,>=265,>=280
NA NA NA RPS 72 289 2.9933e-06 0
0 2.9933e-06 NA NA
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 DPT K
Z2 DPT NA Z2 ADPSFC MSA_d03 NBRHD_SQUARE 361
>=205,>=220,>=235,>=250,>=265,>=280
NA NA NA RPS 70 361 1.9183e-06 0
0 1.9183e-06 NA NA
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 DPT K
Z2 DPT NA Z2 ADPSFC MSA_d03 NBRHD_SQUARE 441
>=205,>=220,>=235,>=250,>=265,>=280
NA NA NA RPS 70 441 1.2855e-06 0
0 1.2855e-06 NA NA
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 DPT K
Z2 DPT NA Z2 ADPSFC MSA_d03 NBRHD_SQUARE 529
>=205,>=220,>=235,>=250,>=265,>=280
NA NA NA RPS 70 529 8.9336e-07 0
0 8.9336e-07 NA NA
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 DPT K
Z2 DPT NA Z2 ADPSFC MSA_d03 NBRHD_SQUARE 625
>=205,>=220,>=235,>=250,>=265,>=280
NA NA NA RPS 70 625 6.4e-07 0
0 6.4e-07 NA NA
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 DPT K
Z2 DPT NA Z2 ADPSFC MSA_d03 NBRHD_SQUARE 729
>=205,>=220,>=235,>=250,>=265,>=280
NA NA NA RPS 68 729 4.7042e-07 0
0 4.7042e-07 NA NA
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 DPT K
Z2 DPT NA Z2 ADPSFC MSA_d03 NBRHD_SQUARE 841
>=205,>=220,>=235,>=250,>=265,>=280
NA NA NA RPS 67 841 3.5347e-07 0
0 3.5347e-07 NA NA
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 DPT K
Z2 DPT NA Z2 ADPSFC MSA_d03 NBRHD_SQUARE 961
>=205,>=220,>=235,>=250,>=265,>=280
NA NA NA RPS 67 961 2.707e-07 0
0 2.707e-07 NA NA
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 DPT K
Z2 DPT NA Z2 ADPSFC MSA_d03 NBRHD_SQUARE 1089
>=205,>=220,>=235,>=250,>=265,>=280
NA NA NA RPS 66 1089 2.1081e-07 0
0 2.1081e-07 NA NA
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 DPT K
Z2 DPT NA Z2 ADPSFC MSA_d03 NBRHD_SQUARE 1225
>=205,>=220,>=235,>=250,>=265,>=280
NA NA NA RPS 66 1225 1.666e-07 0
0 1.666e-07 NA NA
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 DPT K
Z2 DPT NA Z2 ADPSFC MSA_d03 NBRHD_SQUARE 1369
>=205,>=220,>=235,>=250,>=265,>=280
NA NA NA RPS 66 1369 1.3339e-07 0
0 1.3339e-07 NA NA
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 DPT K
Z2 DPT NA Z2 ADPSFC MSA_d03 NBRHD_SQUARE 1521
>=205,>=220,>=235,>=250,>=265,>=280
NA NA NA RPS 65 1521 1.0806e-07 0
0 1.0806e-07 NA NA
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 DPT K
Z2 DPT NA Z2 ADPSFC MSA_d03 NBRHD_SQUARE 1681
>=205,>=220,>=235,>=250,>=265,>=280
NA NA NA RPS 64 1681 8.8472e-08 0
0 8.8472e-08 NA NA
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 DPT K
Z2 DPT NA Z2 ADPSFC MSA_d03 NBRHD_SQUARE 1849
>=205,>=220,>=235,>=250,>=265,>=280
NA NA NA RPS 64 1849 7.3125e-08 0
0 7.3125e-08 NA NA
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 DPT K
Z2 DPT NA Z2 ADPSFC MSA_d03 NBRHD_SQUARE 2025
>=205,>=220,>=235,>=250,>=265,>=280
NA NA NA RPS 64 2025 6.0966e-08 0
0 6.0966e-08 NA NA
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 DPT K
Z2 DPT NA Z2 ADPSFC MSA_d03 NBRHD_SQUARE 2209
>=205,>=220,>=235,>=250,>=265,>=280
NA NA NA RPS 64 2209 5.1233e-08 0
0 5.1233e-08 NA NA
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 DPT K
Z2 DPT NA Z2 ADPSFC MSA_d03 NBRHD_SQUARE 2401
>=205,>=220,>=235,>=250,>=265,>=280
NA NA NA RPS 64 2401 4.3367e-08 0
0 4.3367e-08 NA NA
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 WIND m/s
Z10 WIND NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 9
>=3.0,>=8.0,>=13.0,>=14.0,>=18.0,>=23.0
NA NA NA RPS 12 9 0.006773 0.024306
0.03125 0.013717 NA 0.56104
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 WIND m/s
Z10 WIND NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 25
>=3.0,>=8.0,>=13.0,>=14.0,>=18.0,>=23.0
NA NA NA RPS 12 25 0.013067 0.03125
0.03125 0.013067 NA 0.58187
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 WIND m/s
Z10 WIND NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 49
>=3.0,>=8.0,>=13.0,>=14.0,>=18.0,>=23.0
NA NA NA RPS 12 49 0.01283 0.03125
0.03125 0.01283 NA 0.58943
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 WIND m/s
Z10 WIND NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 81
>=3.0,>=8.0,>=13.0,>=14.0,>=18.0,>=23.0
NA NA NA RPS 12 81 0.012477 0.03125
0.03125 0.012477 NA 0.60074
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 WIND m/s
Z10 WIND NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 121
>=3.0,>=8.0,>=13.0,>=14.0,>=18.0,>=23.0
NA NA NA RPS 12 121 0.013216 0.03125
0.03125 0.013216 NA 0.57708
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 WIND m/s
Z10 WIND NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 169
>=3.0,>=8.0,>=13.0,>=14.0,>=18.0,>=23.0
NA NA NA RPS 12 169 0.013508 0.03125
0.03125 0.013508 NA 0.56774
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 WIND m/s
Z10 WIND NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 225
>=3.0,>=8.0,>=13.0,>=14.0,>=18.0,>=23.0
NA NA NA RPS 12 225 0.014404 0.03125
0.03125 0.014404 NA 0.53908
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 WIND m/s
Z10 WIND NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 289
>=3.0,>=8.0,>=13.0,>=14.0,>=18.0,>=23.0
NA NA NA RPS 12 289 0.016022 0.03125
0.03125 0.016022 NA 0.4873
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 WIND m/s
Z10 WIND NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 361
>=3.0,>=8.0,>=13.0,>=14.0,>=18.0,>=23.0
NA NA NA RPS 12 361 0.018193 0.03125
0.03125 0.018193 NA 0.41782
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 WIND m/s
Z10 WIND NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 441
>=3.0,>=8.0,>=13.0,>=14.0,>=18.0,>=23.0
NA NA NA RPS 12 441 0.020506 0.03125
0.03125 0.020506 NA 0.34381
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 WIND m/s
Z10 WIND NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 529
>=3.0,>=8.0,>=13.0,>=14.0,>=18.0,>=23.0
NA NA NA RPS 12 529 0.022911 0.03125
0.03125 0.022911 NA 0.26685
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 WIND m/s
Z10 WIND NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 625
>=3.0,>=8.0,>=13.0,>=14.0,>=18.0,>=23.0
NA NA NA RPS 12 625 0.025214 0.03125
0.03125 0.025214 NA 0.19315
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 WIND m/s
Z10 WIND NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 729
>=3.0,>=8.0,>=13.0,>=14.0,>=18.0,>=23.0
NA NA NA RPS 12 729 0.027138 0.03125
0.03125 0.027138 NA 0.1316
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 WIND m/s
Z10 WIND NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 841
>=3.0,>=8.0,>=13.0,>=14.0,>=18.0,>=23.0
NA NA NA RPS 12 841 0.029032 0.03125
0.03125 0.029032 NA 0.070988
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 WIND m/s
Z10 WIND NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 961
>=3.0,>=8.0,>=13.0,>=14.0,>=18.0,>=23.0
NA NA NA RPS 12 961 0.031032 0.03125
0.03125 0.031032 NA 0.0069782
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 WIND m/s
Z10 WIND NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 1089
>=3.0,>=8.0,>=13.0,>=14.0,>=18.0,>=23.0
NA NA NA RPS 12 1089 0.032933 0.03125
0.03125 0.032933 NA -0.053863
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 WIND m/s
Z10 WIND NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 1225
>=3.0,>=8.0,>=13.0,>=14.0,>=18.0,>=23.0
NA NA NA RPS 12 1225 0.034518 0.03125
0.03125 0.034518 NA -0.10456
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 WIND m/s
Z10 WIND NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 1369
>=3.0,>=8.0,>=13.0,>=14.0,>=18.0,>=23.0
NA NA NA RPS 12 1369 0.035856 0.03125
0.03125 0.035856 NA -0.1474
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 WIND m/s
Z10 WIND NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 1521
>=3.0,>=8.0,>=13.0,>=14.0,>=18.0,>=23.0
NA NA NA RPS 12 1521 0.037043 0.03125
0.03125 0.037043 NA -0.18538
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 WIND m/s
Z10 WIND NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 1681
>=3.0,>=8.0,>=13.0,>=14.0,>=18.0,>=23.0
NA NA NA RPS 11 1681 0.03368 0.033058
0.033058 0.03368 NA -0.018811
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 WIND m/s
Z10 WIND NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 1849
>=3.0,>=8.0,>=13.0,>=14.0,>=18.0,>=23.0
NA NA NA RPS 11 1849 0.03492 0.033058
0.033058 0.03492 NA -0.056334
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 WIND m/s
Z10 WIND NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 2025
>=3.0,>=8.0,>=13.0,>=14.0,>=18.0,>=23.0
NA NA NA RPS 11 2025 0.035885 0.033058
0.033058 0.035885 NA -0.085523
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 WIND m/s
Z10 WIND NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 2209
>=3.0,>=8.0,>=13.0,>=14.0,>=18.0,>=23.0
NA NA NA RPS 11 2209 0.036774 0.033058
0.033058 0.036774 NA -0.11241
V9.0 1km_WRE-N_MSA 1km_WRE-N_HiRA_test 030000 20161111_120000
20161111_120000 000000 20161111_114000 20161111_122000 WIND m/s
Z10 WIND NA Z10 ADPSFC MSA_d03 NBRHD_SQUARE 2401
>=3.0,>=8.0,>=13.0,>=14.0,>=18.0,>=23.0
NA NA NA RPS 11 2401 0.037467 0.033058
0.033058 0.037467 NA -0.13338
------------------------------------------------
Subject: MET V9.0 Point-Stat error (UNCLASSIFIED)
From: John Halley Gotway
Time: Wed Jul 08 06:35:30 2020
John,
I am out of the office all of this week and next week. Yes, the RPS is
computed using multiple thresholds. For each threshold, we get
probabilistic scores and the values reported in the RPS line types are
the
averages of those scores computed across those thresholds. Yes, you’d
get a
different RPS line type for each neighborhood size since each
neighborhood
constitutes a different “ensemble” forecast.
John
On Tue, Jul 7, 2020 at 3:54 PM Raby, John W USA CIV via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
>
> John -
>
> Table 9.3 of the MET User's Guide says that the RPS scores should be
for
> each threshold. See the following excerpt:
> RPS_REL RPS Reliability, mean of the reliabilities for each RPS
> threshold
> 28 RPS_RES RPS Resolution, mean of the resolutions for each RPS
> threshold
> 29 RPS_UNC RPS Uncertainty, mean of the uncertainties for each RPS
> threshold
> 30 RPS Ranked Probability Score, mean of the Brier Scores for each
> RPS threshold
>
> The attached output file contains the RPS scores for a run I did
where I
> expanded the number of thresholds to accomplish what I needed. It
looks
> like there is only one value for all thresholds, but there is one
score per
> neighborhood size. If the prob_cat_thresh entry defines the
thresholds to
> be used in computing the ranked probability score in the RPS output
line
> type, then there must be some dependency of RPS on threshold value.
Could
> you check into this to see what I should expect in the output?
>
> Thanks.
>
> R/
> John
>
> ________________________________________
> From: Raby, John W CIV USARMY CCDC ARL (USA)
> Sent: Sunday, July 05, 2020 8:51 PM
> To: met_help at ucar.edu
> Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
> Point-Stat error (UNCLASSIFIED)
>
> John -
>
> After researching the RPS score, found that it's not in Appendix C
of the
> User's Guide. Consider adding it in.
>
> R/
> John
>
>
> ________________________________________
> From: Raby, John W CIV USARMY CCDC ARL (USA)
> Sent: Sunday, July 05, 2020 5:10 PM
> To: met_help at ucar.edu
> Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
> Point-Stat error (UNCLASSIFIED)
>
> John -
>
> In researching RPS used the link you gave me a while back for
accessing
> the issues on GitHub. Particularly like the ref numbers in the
User's Guide
> which link to the issue and discussion with references, etc. Wealth
of unfo!
>
> Thanks.
>
> R/
> John
> ________________________________________
> From: Raby, John W CIV USARMY CCDC ARL (USA)
> Sent: Sunday, July 05, 2020 4:16 PM
> To: met_help at ucar.edu
> Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
> Point-Stat error (UNCLASSIFIED)
>
> Thanks, John. This helps. I'll read up on RPS to understand why
those
> requirements are required.
>
> R/
> John
> ________________________________________
> From: John Halley Gotway via RT [met_help at ucar.edu]
> Sent: Sunday, July 05, 2020 10:46 AM
> To: Raby, John W CIV USARMY CCDC ARL (USA)
> Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
> Point-Stat error (UNCLASSIFIED)
>
> All active links contained in this email were disabled. Please
verify the
> identity of the sender, and confirm the authenticity of all links
contained
> within the message prior to copying and pasting the address to a Web
> browser.
>
>
>
>
> ----
>
> John,
>
> Just disable the RPS output line type (rps = NONE;). I believe
that’s
> where those threshold requirements originate. Or, if you would like
RPS
> output, read up on that statistic and you’ll see why those
requirements are
> in place.
>
> On Sun, Jul 5, 2020 at 10:34 AM Raby, John W USA CIV via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <Caution-url: Caution-
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> >
> > John -
> >
> > I tried running with the again with the HiRa method but with the
> following
> > thresholds:
> > >=250, >=260, >=270, >=280, >=290, >=300, <=273, >=273
> > It gave me an error because the thresholds are not monotonically
> > increasing and with the same inequality. I thought that this error
only
> > occurs when you have selected the MCTC and MCTS output which are
> currently
> > not selected. I recall running Grid-Stat with similar thresholds
without
> > MCTC and MCTS output and it ran fine. What is it that ohnis
causing this
> > error?
> >
> > Thanks.
> >
> > R/
> > John
> > ________________________________________
> > From: Raby, John W CIV USARMY CCDC ARL (USA)
> > Sent: Friday, July 03, 2020 6:43 PM
> > To: met_help at ucar.edu
> > Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > Point-Stat error (UNCLASSIFIED)
> >
> > John -
> >
> > Your GitHub issue and the logic really help to better understand
how the
> > HiRA methodology works. I was also pretty confused about that.
Maybe a
> > re-look at the User's Guide description would be good to consider
too.
> I've
> > attached the RPS and ECNT output file in case you wanted to see
them.
> Thank
> > you for working through this with me.
> >
> > Have a great 4th of July!
> >
> > R/
> > John
> > ________________________________________
> > From: John Halley Gotway via RT [met_help at ucar.edu]
> > Sent: Friday, July 03, 2020 6:06 PM
> > To: Raby, John W CIV USARMY CCDC ARL (USA)
> > Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > Point-Stat error (UNCLASSIFIED)
> >
> > All active links contained in this email were disabled. Please
verify
> the
> > identity of the sender, and confirm the authenticity of all links
> contained
> > within the message prior to copying and pasting the address to a
Web
> > browser.
> >
> >
> >
> >
> > ----
> >
> > John,
> >
> > I was looking at the code today and suspect that I've misled you a
little
> > bit. I was confused about it myself. I updated this GitHub issue
to
> > clarify:
> > Caution-Caution-https://github.com/NCAR/MET/issues/1400
> >
> > Upon review, you're correct that Point-Stat DOES use the
cat_thresh
> > settings from the fcst and obs dictionaries while processing HiRA.
It
> uses
> > those thresholds to derive probability fields which are evaluated
in the
> > output PCT, PTSD, PJC, and PRC line types. It also processes the
> > neighborhood of points as a "spatial ensemble"... and from those
ensemble
> > points, it computes the ECNT and RPS line types. And it's in the
> > computation of the RPS line type that the prob_cat_thresh setting
is
> used.
> >
> > So prob_cat_thresh really only impacts the RPS line type.
> >
> > I am still making the change we discussed, but I wanted to clarify
how
> that
> > change is actually being used (since I was confused about that
myself).
> >
> > You can see the tweak to the logic on line 1841 of:
> > Caution-
> > Caution-
>
https://github.com/NCAR/MET/blob/05f2e7d56bae3af092a69a5a9db4867f202eaae0/met/src/tools/core/point_stat/point_stat.cc#L1841
> >
> > Thanks,
> > John
> >
> >
> >
> > On Thu, Jul 2, 2020 at 8:50 PM Raby, John W USA CIV via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <Caution-Caution-url: Caution-
> > Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> > >
> > > John -
> > >
> > > Using your suggestion below, I was able to get Point-Stat to
run. See
> > > attached log file and config file. I haven't looked at the
output yet,
> > but
> > > it was generated and there are no 0-sized files.
> > >
> > > Thanks for your help in getting it running!
> > >
> > > R/
> > > John
> > >
> > > ________________________________________
> > > From: John Halley Gotway via RT [met_help at ucar.edu]
> > > Sent: Thursday, July 02, 2020 4:45 PM
> > > To: Raby, John W CIV USARMY CCDC ARL (USA)
> > > Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > > Point-Stat error (UNCLASSIFIED)
> > >
> > > All active links contained in this email were disabled. Please
verify
> > the
> > > identity of the sender, and confirm the authenticity of all
links
> > contained
> > > within the message prior to copying and pasting the address to a
Web
> > > browser.
> > >
> > >
> > >
> > >
> > > ----
> > >
> > > Yes, that's right, obs=fcst will copy everything over. And it
really
> > > doesn't hurt having hira in both. Point-Stat just parses it from
the
> obs
> > > dictionary.
> > >
> > > John
> > >
> > > On Thu, Jul 2, 2020 at 4:31 PM Raby, John W USA CIV via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <Caution-Caution-Caution-url: Caution-
> > > Caution-Caution-
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> > > >
> > > > John -
> > > >
> > > > Thanks for taking a look and your quick reply. I see now about
> > including
> > > > the hira dictionary inside the field array dictionary. Thanks
for
> > showing
> > > > that below.
> > > >
> > > > So if I have the fcst dictionary only (obs=fcst) in my confiig
file
> > then
> > > I
> > > > all I have to do is put the hira dictionary with settings for
the
> fcst
> > > > variables, correct? I guess I'm "not understanding "copying it
to the
> > > obs"
> > > > when I don't explicitly show the obs dictionary.
> > > >
> > > > Good point about limiting the number of threshold and widths
for
> > testing.
> > > >
> > > > R/
> > > > John
> > > >
> > > >
> > > > ________________________________________
> > > > From: John Halley Gotway via RT [met_help at ucar.edu]
> > > > Sent: Thursday, July 02, 2020 4:14 PM
> > > > To: Raby, John W CIV USARMY CCDC ARL (USA)
> > > > Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > > > Point-Stat error (UNCLASSIFIED)
> > > >
> > > > All active links contained in this email were disabled.
Please
> verify
> > > the
> > > > identity of the sender, and confirm the authenticity of all
links
> > > contained
> > > > within the message prior to copying and pasting the address to
a Web
> > > > browser.
> > > >
> > > >
> > > >
> > > >
> > > > ----
> > > >
> > > > John,
> > > >
> > > > Sorry for not answering your previous question. Yes, since
you're
> using
> > > > obs=fcst, you can just define it in the fcst dictionary and
copy it
> > over
> > > to
> > > > the obs.
> > > >
> > > > Yes, I do see a problem in your config file. The hira
dictionary
> needs
> > to
> > > > be INSIDE of the field array dictionary entry, as shown below.
> > > >
> > > > Also, for initial testing, I'd recommend using a much smaller
set of
> > > > thresholds and widths. Running with all of these will take a
long
> time.
> > > >
> > > > John
> > > >
> > > > obs = {
> > > > field = [
> > > > {
> > > > name = "TMP";
> > > > level = [ "Z2" ];
> > > > cat_thresh = [ >=250, >=260, >=270, >=280, >=290,
>=300,
> <=273,
> > > > >=273 ];
> > > > hira = {
> > > > flag = TRUE;
> > > > width = [ 3, 5, 7, 9, 11, 13, 15, 17, 19,
21, 23,
> > 25,
> > > > 27, 29, 31, 33, 35, 37, 39, 41, 43, 45, 47, 49 ];
> > > > vld_thresh = 1.0;
> > > > cov_thresh = [ ==0.25 ];
> > > > shape = SQUARE;
> > > > prob_cat_thresh = [ >=250, >=260, >=270, >=280,
>=290,
> >=300,
> > > > <=273, >=273 ];
> > > > }
> > > > },
> > > > ...
> > > >
> > > >
> > > > On Thu, Jul 2, 2020 at 3:42 PM Raby, John W USA CIV via RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > <Caution-Caution-Caution-Caution-url: Caution-
> > > > Caution-Caution-
> > Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> > > > >
> > > > > CLASSIFICATION: UNCLASSIFIED
> > > > >
> > > > > John -
> > > > >
> > > > > I edited the attached config file to add the HiRA dictionary
to
> each
> > > > > obs.field
> > > > > dictionary entry as you described, but it seems like it's
not
> > expecting
> > > > > the
> > > > > hira dictionary to be where I placed it. Do you see where I
went
> > wrong
> > > > > with
> > > > > this? The error dialog is also attached.
> > > > >
> > > > > Thanks and have a great 4th of July!
> > > > >
> > > > > R/
> > > > > John
> > > > >
> > > > >
> > > > >
> > > > > -----Original Message-----
> > > > > From: John Halley Gotway via RT <met_help at ucar.edu>
> > > > > Sent: Wednesday, July 1, 2020 11:17 AM
> > > > > To: Raby, John W CIV USARMY CCDC ARL (USA) <
> > john.w.raby2.civ at mail.mil>
> > > > > Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790]
MET
> V9.0
> > > > > Point-Stat
> > > > > error (UNCLASSIFIED)
> > > > >
> > > > > All active links contained in this email were disabled.
Please
> > verify
> > > > the
> > > > > identity of the sender, and confirm the authenticity of all
links
> > > > > contained
> > > > > within the message prior to copying and pasting the address
to a
> Web
> > > > > browser.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > ----
> > > > >
> > > > > John,
> > > > >
> > > > > Sorry for the confusion. I led you astray. The HiRA
configuration
> > > option
> > > > > names are not entirely intuitive.
> > > > >
> > > > > Please take a look at the README file starting on line 1418:
> > > > > Caution-
> > > > > Caution-
> > > > Caution-
> > > Caution-
> > Caution-
>
https://github.com/NCAR/MET/blob/0cee9fbd0b3c691fd3cf4dd1d665f5d2b0db9a37/met/data/config/README#L1418
> > > > >
> > > > > I modified your hira entry like this:
> > > > >
> > > > > hira = {
> > > > > flag = TRUE; *// set to TRUE/false to
turn
> on/off
> > > this
> > > > > logic*
> > > > > width = [ 3, 5, 7, 9, 11 ]; *// defines the
> > > neighborhood
> > > > > size and must be > 1*
> > > > > vld_thresh = 1.0;
> > > > > cov_thresh = [ ==0.25 ]; *// this defines the
probability
> bins
> > > to
> > > > be
> > > > > used. You only get to pick one setting here (==0.25 is short
for
> > > > > >=0,>=0.25,>=0.5,>=0.75,>=1.0)*
> > > > > shape = SQUARE;
> > > > > prob_cat_thresh = [ >290, >300 ];* // this defines the
> > threshold(s)
> > > > for
> > > > > which HiRA logic should be applied* }
> > > > >
> > > > > So you just had things a bit mis-configured. The key thing
to note
> > here
> > > > is
> > > > > that the hira.prob_cat_thresh setting defines the thresholds
for
> > which
> > > > > HiRA
> > > > > probabilities should be computed. This is different from the
> > cat_thresh
> > > > > setting in the fcst and obs dictionaries. Those are used to
compute
> > > > > categorical counts and statistics and are currently
separate.
> > > > >
> > > > > It's nice that you don't have to use the same set of
thresholds for
> > > > > categorical stats and the HiRA logic... it's more
configurable that
> > > way.
> > > > > However, I do see that it makes things a bit inconvenient.
Since
> > you're
> > > > > applying HiRA to many variables and each of those variables
has
> > > different
> > > > > meaningful thresholds, you'd need to configure the HiRA
dictionary
> > > > > separately
> > > > > for each and every variable.
> > > > >
> > > > > That's what is meant by this comment... that the hira
dictionary
> can
> > be
> > > > > specified separately for each variable:
> > > > >
> > > > > *// HiRA verification method// May be set separately in each
> > > "obs.field"
> > > > > entry*
> > > > >
> > > > > So you'd need to copy/paste the HiRA dictionary into every
single
> > > > > obs.field
> > > > > dictionary entry and modify the prob_cat_thresh thresholds
> > accordingly.
> > > > >
> > > > > First, please make sure that you can in fact produce HiRA
output.
> > > Second,
> > > > > I
> > > > > can think of a tweak to the logic that might make using HiRA
> easier.
> > > How
> > > > > about, if the hira.prob_cat_thresh array is an empty list,
we could
> > > have
> > > > > the
> > > > > code check to see if cat_thresh in the obs dictionary is
non-empty.
> > And
> > > > if
> > > > > that's non-empty then we'd simply set prob_cat_thresh =
cat_thresh.
> > > > >
> > > > > I imagine that's what you were expecting in the first
place... that
> > > > > Point-Stat
> > > > > would just be using the thresholds from cat_thresh to define
the
> HiRA
> > > > > probability thresholds. It still is nice to have
prob_cat_thresh be
> > > > > configurable separately, but if left unset, defaulting to
> cat_thresh
> > > > seems
> > > > > pretty reasonable to me. Do you think that would be a
worthwhile
> > > change?
> > > > >
> > > > > Thanks,
> > > > > John
> > > > >
> > > > > On Tue, Jun 30, 2020 at 7:12 PM Raby, John W USA CIV via RT
<
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > > >
> > > > > > <Caution-Caution-Caution-Caution-Caution-url:
> > > > > > Caution-Caution-
> > > > Caution-Caution-
> > Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> > > > > >
> > > > > > I forgot to attach the log file.
> > > > > >
> > > > > > ________________________________________
> > > > > > From: Raby, John W CIV USARMY CCDC ARL (USA)
> > > > > > Sent: Tuesday, June 30, 2020 7:00 PM
> > > > > > To: met_help at ucar.edu
> > > > > > Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790]
MET
> > V9.0
> > > > > > Point-Stat error (UNCLASSIFIED)
> > > > > >
> > > > > > I edited the config file to use the shortcut and corrected
the
> > > > > > whitespace issue, but I get the same error.
> > > > > > R/
> > > > > > John
> > > > > >
> > > > > >
> > > > > >
> > > > > > ________________________________________
> > > > > > From: Raby, John W CIV USARMY CCDC ARL (USA)
> > > > > > Sent: Tuesday, June 30, 2020 6:10 PM
> > > > > > To: met_help at ucar.edu
> > > > > > Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790]
MET
> > V9.0
> > > > > > Point-Stat error (UNCLASSIFIED)
> > > > > >
> > > > > > Thanks for taking a look, John. I'll give that a try.
Appreciate
> > the
> > > > > > heads-up on a future error in the "desc".
> > > > > >
> > > > > > R/
> > > > > > John
> > > > > >
> > > > > > ________________________________________
> > > > > > From: John Halley Gotway via RT [met_help at ucar.edu]
> > > > > > Sent: Tuesday, June 30, 2020 5:11 PM
> > > > > > To: Raby, John W CIV USARMY CCDC ARL (USA)
> > > > > > Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > > > > > Point-Stat error (UNCLASSIFIED)
> > > > > >
> > > > > > All active links contained in this email were disabled.
Please
> > > verify
> > > > > > the identity of the sender, and confirm the authenticity
of all
> > links
> > > > > > contained within the message prior to copying and pasting
the
> > address
> > > > > > to a Web browser.
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > ----
> > > > > >
> > > > > > John,
> > > > > >
> > > > > > Please try using this shortcut:
> > > > > > prob_cat_thresh = [ ==0.25 ];
> > > > > >
> > > > > > Does that do it?
> > > > > >
> > > > > > Also, your "desc" setting can't contain embedded
whitespace. And
> > > > > > Point-Stat should error out and tell you that.
> > > > > >
> > > > > > John
> > > > > >
> > > > > > On Tue, Jun 30, 2020 at 4:57 PM Raby, John W USA CIV via
RT <
> > > > > > met_help at ucar.edu> wrote:
> > > > > >
> > > > > > >
> > > > > > > Tue Jun 30 16:56:45 2020: Request 95790 was acted upon.
> > > > > > > Transaction: Ticket created by john.w.raby2.civ at mail.mil
> > > > > > > Queue: met_help
> > > > > > > Subject: MET V9.0 Point-Stat error (UNCLASSIFIED)
> > > > > > > Owner: Nobody
> > > > > > > Requestors: john.w.raby2.civ at mail.mil
> > > > > > > Status: new
> > > > > > > Ticket <Caution-Caution-Caution-Caution-Caution-
Caution-url:
> Caution-
> > > > > > Caution-Caution-
> > > > Caution-Caution-
> > Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> > > > > > >
> > > > > > >
> > > > > > > CLASSIFICATION: UNCLASSIFIED
> > > > > > >
> > > > > > > I keep getting the same error when trying to run Point-
Stat
> with
> > > the
> > > > > > > HiRA verification logic enabled. The error involves the
> > probability
> > > > > > > field
> > > > > > > thresholds:
> > > > > > >
> > > > > > > ERROR:check_prob_thresh() -> When verifying a
probability
> field,
> > > you
> > > > > > > must select at least 3 thresholds beginning with 0.0 and
ending
> > > with
> > > > > 1.0.
> > > > > > >
> > > > > > > During troubleshooting, I've changed the settings for
> > > > > > > prob_cat_thresh in the attached config file several
times
> > thinking
> > > > > > > that the formatting of my settings was the issue. Maybe
I'm on
> > the
> > > > > > > wrong track. Could you take a
> > > > > > look
> > > > > > > at the config file to check the format or other possible
issue
> > > which
> > > > > > might
> > > > > > > be causing this issue?
> > > > > > >
> > > > > > > Thanks.
> > > > > > >
> > > > > > > R/
> > > > > > > John
> > > > > > >
> > > > > > > Mr. John W. Raby
> > > > > > > U.S. Army Research Laboratory
> > > > > > > White Sands Missile Range, NM 88002
> > > > > > > Office:(575) 678-2004 DSN 258-2004
> > > > > > > Teleworking: Contact Admin Specialist at (410) 278-6974
FAX
> (575)
> > > > > > > 678-1230 DSN 258-1230
> > > > > > > Email: john.w.raby2.civ at mail.mil
> > > > > > >
> > > > > > > CLASSIFICATION: UNCLASSIFIED
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > > CLASSIFICATION: UNCLASSIFIED
> > > > >
> > > > >
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> > >
> >
> >
> >
> >
>
>
>
------------------------------------------------
Subject: MET V9.0 Point-Stat error (UNCLASSIFIED)
From: Raby, John W USA CIV
Time: Wed Jul 08 07:20:25 2020
CLASSIFICATION: UNCLASSIFIED
John -
That answers my question. Enjoy your time off and stay safe!
R/
John
-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Wednesday, July 8, 2020 6:36 AM
To: Raby, John W CIV USARMY CCDC ARL (USA) <john.w.raby2.civ at mail.mil>
Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
Point-Stat error (UNCLASSIFIED)
All active links contained in this email were disabled. Please verify
the identity of the sender, and confirm the authenticity of all links
contained within the message prior to copying and pasting the address
to a Web browser.
----
John,
I am out of the office all of this week and next week. Yes, the RPS is
computed using multiple thresholds. For each threshold, we get
probabilistic scores and the values reported in the RPS line types are
the averages of those scores computed across those thresholds. Yes,
you’d get a different RPS line type for each neighborhood size since
each neighborhood constitutes a different “ensemble” forecast.
John
On Tue, Jul 7, 2020 at 3:54 PM Raby, John W USA CIV via RT <
met_help at ucar.edu> wrote:
>
> <Caution-url:
> Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
>
> John -
>
> Table 9.3 of the MET User's Guide says that the RPS scores should be
> for each threshold. See the following excerpt:
> RPS_REL RPS Reliability, mean of the reliabilities for each RPS
> threshold
> 28 RPS_RES RPS Resolution, mean of the resolutions for each RPS
> threshold
> 29 RPS_UNC RPS Uncertainty, mean of the uncertainties for each RPS
> threshold
> 30 RPS Ranked Probability Score, mean of the Brier Scores for each
RPS
> threshold
>
> The attached output file contains the RPS scores for a run I did
where
> I expanded the number of thresholds to accomplish what I needed. It
> looks like there is only one value for all thresholds, but there is
> one score per neighborhood size. If the prob_cat_thresh entry
defines
> the thresholds to be used in computing the ranked probability score
in
> the RPS output line type, then there must be some dependency of RPS
on
> threshold value. Could you check into this to see what I should
expect in the output?
>
> Thanks.
>
> R/
> John
>
> ________________________________________
> From: Raby, John W CIV USARMY CCDC ARL (USA)
> Sent: Sunday, July 05, 2020 8:51 PM
> To: met_help at ucar.edu
> Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
> Point-Stat error (UNCLASSIFIED)
>
> John -
>
> After researching the RPS score, found that it's not in Appendix C
of
> the User's Guide. Consider adding it in.
>
> R/
> John
>
>
> ________________________________________
> From: Raby, John W CIV USARMY CCDC ARL (USA)
> Sent: Sunday, July 05, 2020 5:10 PM
> To: met_help at ucar.edu
> Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
> Point-Stat error (UNCLASSIFIED)
>
> John -
>
> In researching RPS used the link you gave me a while back for
> accessing the issues on GitHub. Particularly like the ref numbers in
> the User's Guide which link to the issue and discussion with
references, etc. Wealth of unfo!
>
> Thanks.
>
> R/
> John
> ________________________________________
> From: Raby, John W CIV USARMY CCDC ARL (USA)
> Sent: Sunday, July 05, 2020 4:16 PM
> To: met_help at ucar.edu
> Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
> Point-Stat error (UNCLASSIFIED)
>
> Thanks, John. This helps. I'll read up on RPS to understand why
those
> requirements are required.
>
> R/
> John
> ________________________________________
> From: John Halley Gotway via RT [met_help at ucar.edu]
> Sent: Sunday, July 05, 2020 10:46 AM
> To: Raby, John W CIV USARMY CCDC ARL (USA)
> Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET V9.0
> Point-Stat error (UNCLASSIFIED)
>
> All active links contained in this email were disabled. Please
verify
> the identity of the sender, and confirm the authenticity of all
links
> contained within the message prior to copying and pasting the
address
> to a Web browser.
>
>
>
>
> ----
>
> John,
>
> Just disable the RPS output line type (rps = NONE;). I believe
that’s
> where those threshold requirements originate. Or, if you would like
> RPS output, read up on that statistic and you’ll see why those
> requirements are in place.
>
> On Sun, Jul 5, 2020 at 10:34 AM Raby, John W USA CIV via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <Caution-Caution-url: Caution-
> Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> >
> > John -
> >
> > I tried running with the again with the HiRa method but with the
> following
> > thresholds:
> > >=250, >=260, >=270, >=280, >=290, >=300, <=273, >=273
> > It gave me an error because the thresholds are not monotonically
> > increasing and with the same inequality. I thought that this error
> > only occurs when you have selected the MCTC and MCTS output which
> > are
> currently
> > not selected. I recall running Grid-Stat with similar thresholds
> > without MCTC and MCTS output and it ran fine. What is it that
ohnis
> > causing this error?
> >
> > Thanks.
> >
> > R/
> > John
> > ________________________________________
> > From: Raby, John W CIV USARMY CCDC ARL (USA)
> > Sent: Friday, July 03, 2020 6:43 PM
> > To: met_help at ucar.edu
> > Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > Point-Stat error (UNCLASSIFIED)
> >
> > John -
> >
> > Your GitHub issue and the logic really help to better understand
how
> > the HiRA methodology works. I was also pretty confused about that.
> > Maybe a re-look at the User's Guide description would be good to
consider too.
> I've
> > attached the RPS and ECNT output file in case you wanted to see
them.
> Thank
> > you for working through this with me.
> >
> > Have a great 4th of July!
> >
> > R/
> > John
> > ________________________________________
> > From: John Halley Gotway via RT [met_help at ucar.edu]
> > Sent: Friday, July 03, 2020 6:06 PM
> > To: Raby, John W CIV USARMY CCDC ARL (USA)
> > Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
V9.0
> > Point-Stat error (UNCLASSIFIED)
> >
> > All active links contained in this email were disabled. Please
> > verify
> the
> > identity of the sender, and confirm the authenticity of all links
> contained
> > within the message prior to copying and pasting the address to a
Web
> > browser.
> >
> >
> >
> >
> > ----
> >
> > John,
> >
> > I was looking at the code today and suspect that I've misled you a
> > little bit. I was confused about it myself. I updated this GitHub
> > issue to
> > clarify:
> > Caution-Caution-Caution-https://github.com/NCAR/MET/issues/1400
> >
> > Upon review, you're correct that Point-Stat DOES use the
cat_thresh
> > settings from the fcst and obs dictionaries while processing HiRA.
> > It
> uses
> > those thresholds to derive probability fields which are evaluated
in
> > the output PCT, PTSD, PJC, and PRC line types. It also processes
the
> > neighborhood of points as a "spatial ensemble"... and from those
> > ensemble points, it computes the ECNT and RPS line types. And it's
> > in the computation of the RPS line type that the prob_cat_thresh
> > setting is
> used.
> >
> > So prob_cat_thresh really only impacts the RPS line type.
> >
> > I am still making the change we discussed, but I wanted to clarify
> > how
> that
> > change is actually being used (since I was confused about that
myself).
> >
> > You can see the tweak to the logic on line 1841 of:
> > Caution-
> > Caution-
> Caution-
https://github.com/NCAR/MET/blob/05f2e7d56bae3af092a69a5a9db48
> 67f202eaae0/met/src/tools/core/point_stat/point_stat.cc#L1841
> >
> > Thanks,
> > John
> >
> >
> >
> > On Thu, Jul 2, 2020 at 8:50 PM Raby, John W USA CIV via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <Caution-Caution-Caution-url: Caution-
> > Caution-Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95
> > 790 >
> > >
> > > John -
> > >
> > > Using your suggestion below, I was able to get Point-Stat to
run.
> > > See attached log file and config file. I haven't looked at the
> > > output yet,
> > but
> > > it was generated and there are no 0-sized files.
> > >
> > > Thanks for your help in getting it running!
> > >
> > > R/
> > > John
> > >
> > > ________________________________________
> > > From: John Halley Gotway via RT [met_help at ucar.edu]
> > > Sent: Thursday, July 02, 2020 4:45 PM
> > > To: Raby, John W CIV USARMY CCDC ARL (USA)
> > > Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
> > > V9.0 Point-Stat error (UNCLASSIFIED)
> > >
> > > All active links contained in this email were disabled. Please
> > > verify
> > the
> > > identity of the sender, and confirm the authenticity of all
links
> > contained
> > > within the message prior to copying and pasting the address to a
> > > Web browser.
> > >
> > >
> > >
> > >
> > > ----
> > >
> > > Yes, that's right, obs=fcst will copy everything over. And it
> > > really doesn't hurt having hira in both. Point-Stat just parses
it
> > > from the
> obs
> > > dictionary.
> > >
> > > John
> > >
> > > On Thu, Jul 2, 2020 at 4:31 PM Raby, John W USA CIV via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <Caution-Caution-Caution-Caution-url: Caution-
> > > Caution-Caution-
> Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95790 >
> > > >
> > > > John -
> > > >
> > > > Thanks for taking a look and your quick reply. I see now about
> > including
> > > > the hira dictionary inside the field array dictionary. Thanks
> > > > for
> > showing
> > > > that below.
> > > >
> > > > So if I have the fcst dictionary only (obs=fcst) in my confiig
> > > > file
> > then
> > > I
> > > > all I have to do is put the hira dictionary with settings for
> > > > the
> fcst
> > > > variables, correct? I guess I'm "not understanding "copying it
> > > > to the
> > > obs"
> > > > when I don't explicitly show the obs dictionary.
> > > >
> > > > Good point about limiting the number of threshold and widths
for
> > testing.
> > > >
> > > > R/
> > > > John
> > > >
> > > >
> > > > ________________________________________
> > > > From: John Halley Gotway via RT [met_help at ucar.edu]
> > > > Sent: Thursday, July 02, 2020 4:14 PM
> > > > To: Raby, John W CIV USARMY CCDC ARL (USA)
> > > > Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
> > > > V9.0 Point-Stat error (UNCLASSIFIED)
> > > >
> > > > All active links contained in this email were disabled.
Please
> verify
> > > the
> > > > identity of the sender, and confirm the authenticity of all
> > > > links
> > > contained
> > > > within the message prior to copying and pasting the address to
a
> > > > Web browser.
> > > >
> > > >
> > > >
> > > >
> > > > ----
> > > >
> > > > John,
> > > >
> > > > Sorry for not answering your previous question. Yes, since
> > > > you're
> using
> > > > obs=fcst, you can just define it in the fcst dictionary and
copy
> > > > it
> > over
> > > to
> > > > the obs.
> > > >
> > > > Yes, I do see a problem in your config file. The hira
dictionary
> needs
> > to
> > > > be INSIDE of the field array dictionary entry, as shown below.
> > > >
> > > > Also, for initial testing, I'd recommend using a much smaller
> > > > set of thresholds and widths. Running with all of these will
> > > > take a long
> time.
> > > >
> > > > John
> > > >
> > > > obs = {
> > > > field = [
> > > > {
> > > > name = "TMP";
> > > > level = [ "Z2" ];
> > > > cat_thresh = [ >=250, >=260, >=270, >=280, >=290,
>=300,
> <=273,
> > > > >=273 ];
> > > > hira = {
> > > > flag = TRUE;
> > > > width = [ 3, 5, 7, 9, 11, 13, 15, 17, 19,
21, 23,
> > 25,
> > > > 27, 29, 31, 33, 35, 37, 39, 41, 43, 45, 47, 49 ];
> > > > vld_thresh = 1.0;
> > > > cov_thresh = [ ==0.25 ];
> > > > shape = SQUARE;
> > > > prob_cat_thresh = [ >=250, >=260, >=270, >=280,
>=290,
> >=300,
> > > > <=273, >=273 ];
> > > > }
> > > > },
> > > > ...
> > > >
> > > >
> > > > On Thu, Jul 2, 2020 at 3:42 PM Raby, John W USA CIV via RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > <Caution-Caution-Caution-Caution-Caution-url: Caution-
> > > > Caution-Caution-
> > Caution-Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95
> > 790 >
> > > > >
> > > > > CLASSIFICATION: UNCLASSIFIED
> > > > >
> > > > > John -
> > > > >
> > > > > I edited the attached config file to add the HiRA dictionary
> > > > > to
> each
> > > > > obs.field
> > > > > dictionary entry as you described, but it seems like it's
not
> > expecting
> > > > > the
> > > > > hira dictionary to be where I placed it. Do you see where I
> > > > > went
> > wrong
> > > > > with
> > > > > this? The error dialog is also attached.
> > > > >
> > > > > Thanks and have a great 4th of July!
> > > > >
> > > > > R/
> > > > > John
> > > > >
> > > > >
> > > > >
> > > > > -----Original Message-----
> > > > > From: John Halley Gotway via RT <met_help at ucar.edu>
> > > > > Sent: Wednesday, July 1, 2020 11:17 AM
> > > > > To: Raby, John W CIV USARMY CCDC ARL (USA) <
> > john.w.raby2.civ at mail.mil>
> > > > > Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790]
MET
> V9.0
> > > > > Point-Stat
> > > > > error (UNCLASSIFIED)
> > > > >
> > > > > All active links contained in this email were disabled.
> > > > > Please
> > verify
> > > > the
> > > > > identity of the sender, and confirm the authenticity of all
> > > > > links contained within the message prior to copying and
> > > > > pasting the address to a
> Web
> > > > > browser.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > ----
> > > > >
> > > > > John,
> > > > >
> > > > > Sorry for the confusion. I led you astray. The HiRA
> > > > > configuration
> > > option
> > > > > names are not entirely intuitive.
> > > > >
> > > > > Please take a look at the README file starting on line 1418:
> > > > > Caution-
> > > > > Caution-
> > > > Caution-
> > > Caution-
> > Caution-
> Caution-
https://github.com/NCAR/MET/blob/0cee9fbd0b3c691fd3cf4dd1d665f
> 5d2b0db9a37/met/data/config/README#L1418
> > > > >
> > > > > I modified your hira entry like this:
> > > > >
> > > > > hira = {
> > > > > flag = TRUE; *// set to TRUE/false to
turn
> on/off
> > > this
> > > > > logic*
> > > > > width = [ 3, 5, 7, 9, 11 ]; *// defines the
> > > neighborhood
> > > > > size and must be > 1*
> > > > > vld_thresh = 1.0;
> > > > > cov_thresh = [ ==0.25 ]; *// this defines the
probability
> bins
> > > to
> > > > be
> > > > > used. You only get to pick one setting here (==0.25 is short
> > > > > for
> > > > > >=0,>=0.25,>=0.5,>=0.75,>=1.0)*
> > > > > shape = SQUARE;
> > > > > prob_cat_thresh = [ >290, >300 ];* // this defines the
> > threshold(s)
> > > > for
> > > > > which HiRA logic should be applied* }
> > > > >
> > > > > So you just had things a bit mis-configured. The key thing
to
> > > > > note
> > here
> > > > is
> > > > > that the hira.prob_cat_thresh setting defines the thresholds
> > > > > for
> > which
> > > > > HiRA
> > > > > probabilities should be computed. This is different from the
> > cat_thresh
> > > > > setting in the fcst and obs dictionaries. Those are used to
> > > > > compute categorical counts and statistics and are currently
separate.
> > > > >
> > > > > It's nice that you don't have to use the same set of
> > > > > thresholds for categorical stats and the HiRA logic... it's
> > > > > more configurable that
> > > way.
> > > > > However, I do see that it makes things a bit inconvenient.
> > > > > Since
> > you're
> > > > > applying HiRA to many variables and each of those variables
> > > > > has
> > > different
> > > > > meaningful thresholds, you'd need to configure the HiRA
> > > > > dictionary separately for each and every variable.
> > > > >
> > > > > That's what is meant by this comment... that the hira
> > > > > dictionary
> can
> > be
> > > > > specified separately for each variable:
> > > > >
> > > > > *// HiRA verification method// May be set separately in each
> > > "obs.field"
> > > > > entry*
> > > > >
> > > > > So you'd need to copy/paste the HiRA dictionary into every
> > > > > single obs.field dictionary entry and modify the
> > > > > prob_cat_thresh thresholds
> > accordingly.
> > > > >
> > > > > First, please make sure that you can in fact produce HiRA
output.
> > > Second,
> > > > > I
> > > > > can think of a tweak to the logic that might make using HiRA
> easier.
> > > How
> > > > > about, if the hira.prob_cat_thresh array is an empty list,
we
> > > > > could
> > > have
> > > > > the
> > > > > code check to see if cat_thresh in the obs dictionary is
non-empty.
> > And
> > > > if
> > > > > that's non-empty then we'd simply set prob_cat_thresh =
cat_thresh.
> > > > >
> > > > > I imagine that's what you were expecting in the first
place...
> > > > > that Point-Stat would just be using the thresholds from
> > > > > cat_thresh to define the
> HiRA
> > > > > probability thresholds. It still is nice to have
> > > > > prob_cat_thresh be configurable separately, but if left
unset,
> > > > > defaulting to
> cat_thresh
> > > > seems
> > > > > pretty reasonable to me. Do you think that would be a
> > > > > worthwhile
> > > change?
> > > > >
> > > > > Thanks,
> > > > > John
> > > > >
> > > > > On Tue, Jun 30, 2020 at 7:12 PM Raby, John W USA CIV via RT
<
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > > >
> > > > > > <Caution-Caution-Caution-Caution-Caution-Caution-url:
> > > > > > Caution-Caution-
> > > > Caution-Caution-
> > Caution-Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95
> > 790 >
> > > > > >
> > > > > > I forgot to attach the log file.
> > > > > >
> > > > > > ________________________________________
> > > > > > From: Raby, John W CIV USARMY CCDC ARL (USA)
> > > > > > Sent: Tuesday, June 30, 2020 7:00 PM
> > > > > > To: met_help at ucar.edu
> > > > > > Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790]
> > > > > > MET
> > V9.0
> > > > > > Point-Stat error (UNCLASSIFIED)
> > > > > >
> > > > > > I edited the config file to use the shortcut and corrected
> > > > > > the whitespace issue, but I get the same error.
> > > > > > R/
> > > > > > John
> > > > > >
> > > > > >
> > > > > >
> > > > > > ________________________________________
> > > > > > From: Raby, John W CIV USARMY CCDC ARL (USA)
> > > > > > Sent: Tuesday, June 30, 2020 6:10 PM
> > > > > > To: met_help at ucar.edu
> > > > > > Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790]
> > > > > > MET
> > V9.0
> > > > > > Point-Stat error (UNCLASSIFIED)
> > > > > >
> > > > > > Thanks for taking a look, John. I'll give that a try.
> > > > > > Appreciate
> > the
> > > > > > heads-up on a future error in the "desc".
> > > > > >
> > > > > > R/
> > > > > > John
> > > > > >
> > > > > > ________________________________________
> > > > > > From: John Halley Gotway via RT [met_help at ucar.edu]
> > > > > > Sent: Tuesday, June 30, 2020 5:11 PM
> > > > > > To: Raby, John W CIV USARMY CCDC ARL (USA)
> > > > > > Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #95790] MET
> > > > > > V9.0 Point-Stat error (UNCLASSIFIED)
> > > > > >
> > > > > > All active links contained in this email were disabled.
> > > > > > Please
> > > verify
> > > > > > the identity of the sender, and confirm the authenticity
of
> > > > > > all
> > links
> > > > > > contained within the message prior to copying and pasting
> > > > > > the
> > address
> > > > > > to a Web browser.
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > ----
> > > > > >
> > > > > > John,
> > > > > >
> > > > > > Please try using this shortcut:
> > > > > > prob_cat_thresh = [ ==0.25 ];
> > > > > >
> > > > > > Does that do it?
> > > > > >
> > > > > > Also, your "desc" setting can't contain embedded
whitespace.
> > > > > > And Point-Stat should error out and tell you that.
> > > > > >
> > > > > > John
> > > > > >
> > > > > > On Tue, Jun 30, 2020 at 4:57 PM Raby, John W USA CIV via
RT
> > > > > > < met_help at ucar.edu> wrote:
> > > > > >
> > > > > > >
> > > > > > > Tue Jun 30 16:56:45 2020: Request 95790 was acted upon.
> > > > > > > Transaction: Ticket created by john.w.raby2.civ at mail.mil
> > > > > > > Queue: met_help
> > > > > > > Subject: MET V9.0 Point-Stat error (UNCLASSIFIED)
> > > > > > > Owner: Nobody
> > > > > > > Requestors: john.w.raby2.civ at mail.mil
> > > > > > > Status: new
> > > > > > > Ticket <Caution-Caution-Caution-Caution-Caution-
Caution-Caution-url:
> Caution-
> > > > > > Caution-Caution-
> > > > Caution-Caution-
> > Caution-Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95
> > 790 >
> > > > > > >
> > > > > > >
> > > > > > > CLASSIFICATION: UNCLASSIFIED
> > > > > > >
> > > > > > > I keep getting the same error when trying to run
> > > > > > > Point-Stat
> with
> > > the
> > > > > > > HiRA verification logic enabled. The error involves the
> > probability
> > > > > > > field
> > > > > > > thresholds:
> > > > > > >
> > > > > > > ERROR:check_prob_thresh() -> When verifying a
probability
> field,
> > > you
> > > > > > > must select at least 3 thresholds beginning with 0.0 and
> > > > > > > ending
> > > with
> > > > > 1.0.
> > > > > > >
> > > > > > > During troubleshooting, I've changed the settings for
> > > > > > > prob_cat_thresh in the attached config file several
times
> > thinking
> > > > > > > that the formatting of my settings was the issue. Maybe
> > > > > > > I'm on
> > the
> > > > > > > wrong track. Could you take a
> > > > > > look
> > > > > > > at the config file to check the format or other possible
> > > > > > > issue
> > > which
> > > > > > might
> > > > > > > be causing this issue?
> > > > > > >
> > > > > > > Thanks.
> > > > > > >
> > > > > > > R/
> > > > > > > John
> > > > > > >
> > > > > > > Mr. John W. Raby
> > > > > > > U.S. Army Research Laboratory White Sands Missile Range,
> > > > > > > NM 88002
> > > > > > > Office:(575) 678-2004 DSN 258-2004
> > > > > > > Teleworking: Contact Admin Specialist at (410) 278-6974
> > > > > > > FAX
> (575)
> > > > > > > 678-1230 DSN 258-1230
> > > > > > > Email: john.w.raby2.civ at mail.mil
> > > > > > >
> > > > > > > CLASSIFICATION: UNCLASSIFIED
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > > CLASSIFICATION: UNCLASSIFIED
> > > > >
> > > > >
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> > >
> >
> >
> >
> >
>
>
>
CLASSIFICATION: UNCLASSIFIED
------------------------------------------------
More information about the Met_help
mailing list