[Met_help] [rt.rap.ucar.edu #98171] History for MODE - verifying a probabilistic field with a deterministic field
John Halley Gotway via RT
met_help at ucar.edu
Wed Feb 10 09:06:48 MST 2021
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hello,
I want to verify a forecast probabilistic field that is comprised of
separate fields with a single deterministic field.
I have probabilities of ceiling height being below a series of thresholds
T1, T2, T3. The arrays are in GRIB2 files with this structure (from wgrib2):
9:10956329:d=2020121812:HGT:cloud ceiling:1 hour fcst:prob <152:prob fcst
0/26:probability forecast
10:14766812:d=2020121812:HGT:cloud ceiling:1 hour fcst:prob <305:prob fcst
0/26:probability forecast
11:18577295:d=2020121812:HGT:cloud ceiling:1 hour fcst:prob <915:prob fcst
0/26:probability forecast
These are not the only fields in this file with the abbreviation "HGT", so
that is already one complication since the other HGT fields are also valid
on only a certain custom level, so using "L0" for level in the config file
confuses me. Regardless, here is my entry for fcst in the config file
fcst = {
field = [
{ name = "PROB"; level = "L000"; prob = { name = "HGT"; thresh_hi =
152; } },
{ name = "PROB"; level = "L000"; prob = { name = "HGT"; thresh_hi =
305; thresh_lo = 152; } },
{ name = "PROB"; level = "L000"; prob = { name = "HGT"; thresh_hi =
915; thresh_lo = 305; } } ];
censor_thresh = [];
censor_val = [];
conv_radius = 60.0/grid_res; // in grid squares
conv_thresh =
[>0.0,>=10.0,>=20.0,>=30.0,>=40.0,>=50.0,>=60.0,>=70.0,>=80.0,>=90.0,>99.0];
vld_thresh = 0.5;
filter_attr_name = [];
filter_attr_thresh = [];
merge_thresh = >=1.25;
merge_flag = NONE;
}
I'm not even sure it is okay to use both "thresh_hi" and "thresh_lo" in a
field entry, but I do want to verify the probabilistic field at multiple
ceiling height thresholds, and this seems logical as to how to do it.
But when I run MODE I get these error messages:
WARNING:
WARNING: Dictionary::lookup_dictionary() -> dictionary lookup failed for
name "field"
WARNING:
ERROR :
ERROR : Dictionary::lookup_dictionary() -> lookup failed for name "field"
ERROR :
Config file is attached. I am using version 9.1.
Jeff Duda
--
Research Scientist I, University of Colorado Boulder
Cooperative Institute for Research in Environmental Sciences
NOAA/OAR/ESRL/Global Systems Laboratory/Assimilation and Verification
Innovation Division
Boulder, CO
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: MODE - verifying a probabilistic field with a deterministic field
From: John Halley Gotway
Time: Fri Jan 08 15:48:01 2021
Hi Jeff,
I see that the config file parsing code is erroring out on your config
file. I think you just need to terminate the "prob" setting with a
semicolon, as shown below. In addition, I think you should only be
using
the thresh_hi setting. That'll match the structure of the GRIB records
you
listed.
field = [
{ name = "PROB"; level = "L000"; prob = { name = "HGT"; thresh_hi
=
152; }; },
{ name = "PROB"; level = "L000"; prob = { name = "HGT"; thresh_hi
=
305; }; },
{ name = "PROB"; level = "L000"; prob = { name = "HGT"; thresh_hi
=
915; }; }
];
Please let me know if that gets you any further and what other issues
pop
up.
Thanks,
John Halley Gotway
On Fri, Jan 8, 2021 at 2:43 PM Jeffrey Duda - NOAA Affiliate via RT <
met_help at ucar.edu> wrote:
>
> Fri Jan 08 14:43:21 2021: Request 98171 was acted upon.
> Transaction: Ticket created by jeffrey.duda at noaa.gov
> Queue: met_help
> Subject: MODE - verifying a probabilistic field with a
deterministic
> field
> Owner: Nobody
> Requestors: jeffrey.duda at noaa.gov
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98171 >
>
>
> Hello,
> I want to verify a forecast probabilistic field that is comprised of
> separate fields with a single deterministic field.
>
> I have probabilities of ceiling height being below a series of
thresholds
> T1, T2, T3. The arrays are in GRIB2 files with this structure (from
> wgrib2):
>
> 9:10956329:d=2020121812:HGT:cloud ceiling:1 hour fcst:prob <152:prob
fcst
> 0/26:probability forecast
> 10:14766812:d=2020121812:HGT:cloud ceiling:1 hour fcst:prob
<305:prob fcst
> 0/26:probability forecast
> 11:18577295:d=2020121812:HGT:cloud ceiling:1 hour fcst:prob
<915:prob fcst
> 0/26:probability forecast
>
> These are not the only fields in this file with the abbreviation
"HGT", so
> that is already one complication since the other HGT fields are also
valid
> on only a certain custom level, so using "L0" for level in the
config file
> confuses me. Regardless, here is my entry for fcst in the config
file
>
> fcst = {
> field = [
> { name = "PROB"; level = "L000"; prob = { name = "HGT";
thresh_hi =
> 152; } },
> { name = "PROB"; level = "L000"; prob = { name = "HGT";
thresh_hi =
> 305; thresh_lo = 152; } },
> { name = "PROB"; level = "L000"; prob = { name = "HGT";
thresh_hi =
> 915; thresh_lo = 305; } } ];
>
> censor_thresh = [];
> censor_val = [];
> conv_radius = 60.0/grid_res; // in grid squares
> conv_thresh =
>
>
[>0.0,>=10.0,>=20.0,>=30.0,>=40.0,>=50.0,>=60.0,>=70.0,>=80.0,>=90.0,>99.0];
> vld_thresh = 0.5;
> filter_attr_name = [];
> filter_attr_thresh = [];
> merge_thresh = >=1.25;
> merge_flag = NONE;
> }
>
> I'm not even sure it is okay to use both "thresh_hi" and "thresh_lo"
in a
> field entry, but I do want to verify the probabilistic field at
multiple
> ceiling height thresholds, and this seems logical as to how to do
it.
>
> But when I run MODE I get these error messages:
> WARNING:
> WARNING: Dictionary::lookup_dictionary() -> dictionary lookup failed
for
> name "field"
> WARNING:
> ERROR :
> ERROR : Dictionary::lookup_dictionary() -> lookup failed for name
"field"
> ERROR :
>
> Config file is attached. I am using version 9.1.
>
> Jeff Duda
> --
> Research Scientist I, University of Colorado Boulder
> Cooperative Institute for Research in Environmental Sciences
> NOAA/OAR/ESRL/Global Systems Laboratory/Assimilation and
Verification
> Innovation Division
> Boulder, CO
>
>
------------------------------------------------
Subject: MODE - verifying a probabilistic field with a deterministic field
From: Jeffrey Duda - NOAA Affiliate
Time: Mon Jan 11 11:31:36 2021
Hey John,
Unfortunately, that suggestion did not work, as I got the same error
message.
I have tried this:
field = [
{ name = "PROB"; level = "L0"; prob = { name = "HGT"; thresh_hi =
152;
}; },
{ name = "PROB"; level = "L0"; prob = { name = "HGT"; thresh_hi =
305;
}; },
{ name = "PROB"; level = "L0"; prob = { name = "HGT"; thresh_hi =
915;
}; } ];
and this:
field = [
{ name = "PROB"; level = "L0"; prob = { name = "HGT"; thresh_hi =
152;
}; };,
{ name = "PROB"; level = "L0"; prob = { name = "HGT"; thresh_hi =
305;
}; };,
{ name = "PROB"; level = "L0"; prob = { name = "HGT"; thresh_hi =
915;
}; }; ];
and am getting the same result. Perhaps the problem lies somewhere
else?
FWIW, I am referencing this part of section 3.5.1 of the MET User's
Guide:
The "prob" entry in the forecast dictionary defines probability
// information. It may either be set as a boolean (i.e. TRUE or
FALSE)
// or as a dictionary defining probabilistic field information.
//
// When set as a boolean to TRUE, it indicates that the
"fcst.field" data
// should be treated as probabilities. For example, when
verifying the
// probabilistic NetCDF output of Ensemble-Stat, one could
configure the
// Grid-Stat or Point-Stat tools as follows:
//
// fcst = {
// field = [ { name = "APCP_24_A24_ENS_FREQ_gt0.0";
// level = "(*,*)";
// prob = TRUE; } ];
// }
//
// Setting "prob = TRUE" indicates that the
"APCP_24_A24_ENS_FREQ_gt0.0"
// data should be processed as probabilities.
//
// When set as a dictionary, it defines the probabilistic field
to be
// used. For example, when verifying GRIB files containing
probabilistic
// data, one could configure the Grid-Stat or Point-Stat tools
as
// follows:
//*// fcst = {
// field = [ { name = "PROB"; level = "A24";
// prob = { name = "APCP"; thresh_lo = 2.54;
} },
// { name = "PROB"; level = "P850";
// prob = { name = "TMP"; thresh_hi = 273; }
} ];
// }*
//
// The example above selects two probabilistic fields. In both,
"name"
// is set to "PROB", the GRIB abbreviation for probabilities.
The "level"
// entry defines the level information (i.e. "A24" for a 24-
hour
// accumulation and "P850" for 850mb). The "prob" dictionary
defines the
// event for which the probability is defined. The "thresh_lo"
// (i.e. APCP > 2.54) and/or "thresh_hi" (i.e. TMP < 273)
entries are
// used to define the event threshold(s).
//
// Probability fields should contain values in the range
// [0, 1] or [0, 100]. However, when MET encounters a
probability field
// with a range [0, 100], it will automatically rescale it to
be [0, 1]
// before applying the probabilistic verification methods.
//
I'm not sure if there are any syntax errors in that reference code, or
if
maybe I need to appeal to the *prob_as_scalar* flag as well. Since I
have
never formally run MODE on a probabilistic field (I have run MODE on
probabilistic fields by just assuming it was a scalar field), this is
new
to me.
Jeff
On Fri, Jan 8, 2021 at 3:48 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Hi Jeff,
>
> I see that the config file parsing code is erroring out on your
config
> file. I think you just need to terminate the "prob" setting with a
> semicolon, as shown below. In addition, I think you should only be
using
> the thresh_hi setting. That'll match the structure of the GRIB
records you
> listed.
>
> field = [
> { name = "PROB"; level = "L000"; prob = { name = "HGT";
thresh_hi =
> 152; }; },
> { name = "PROB"; level = "L000"; prob = { name = "HGT";
thresh_hi =
> 305; }; },
> { name = "PROB"; level = "L000"; prob = { name = "HGT";
thresh_hi =
> 915; }; }
> ];
>
> Please let me know if that gets you any further and what other
issues pop
> up.
>
> Thanks,
> John Halley Gotway
>
> On Fri, Jan 8, 2021 at 2:43 PM Jeffrey Duda - NOAA Affiliate via RT
<
> met_help at ucar.edu> wrote:
>
> >
> > Fri Jan 08 14:43:21 2021: Request 98171 was acted upon.
> > Transaction: Ticket created by jeffrey.duda at noaa.gov
> > Queue: met_help
> > Subject: MODE - verifying a probabilistic field with a
deterministic
> > field
> > Owner: Nobody
> > Requestors: jeffrey.duda at noaa.gov
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98171 >
> >
> >
> > Hello,
> > I want to verify a forecast probabilistic field that is comprised
of
> > separate fields with a single deterministic field.
> >
> > I have probabilities of ceiling height being below a series of
thresholds
> > T1, T2, T3. The arrays are in GRIB2 files with this structure
(from
> > wgrib2):
> >
> > 9:10956329:d=2020121812:HGT:cloud ceiling:1 hour fcst:prob
<152:prob fcst
> > 0/26:probability forecast
> > 10:14766812:d=2020121812:HGT:cloud ceiling:1 hour fcst:prob
<305:prob
> fcst
> > 0/26:probability forecast
> > 11:18577295:d=2020121812:HGT:cloud ceiling:1 hour fcst:prob
<915:prob
> fcst
> > 0/26:probability forecast
> >
> > These are not the only fields in this file with the abbreviation
"HGT",
> so
> > that is already one complication since the other HGT fields are
also
> valid
> > on only a certain custom level, so using "L0" for level in the
config
> file
> > confuses me. Regardless, here is my entry for fcst in the config
file
> >
> > fcst = {
> > field = [
> > { name = "PROB"; level = "L000"; prob = { name = "HGT";
thresh_hi =
> > 152; } },
> > { name = "PROB"; level = "L000"; prob = { name = "HGT";
thresh_hi =
> > 305; thresh_lo = 152; } },
> > { name = "PROB"; level = "L000"; prob = { name = "HGT";
thresh_hi =
> > 915; thresh_lo = 305; } } ];
> >
> > censor_thresh = [];
> > censor_val = [];
> > conv_radius = 60.0/grid_res; // in grid squares
> > conv_thresh =
> >
> >
>
[>0.0,>=10.0,>=20.0,>=30.0,>=40.0,>=50.0,>=60.0,>=70.0,>=80.0,>=90.0,>99.0];
> > vld_thresh = 0.5;
> > filter_attr_name = [];
> > filter_attr_thresh = [];
> > merge_thresh = >=1.25;
> > merge_flag = NONE;
> > }
> >
> > I'm not even sure it is okay to use both "thresh_hi" and
"thresh_lo" in a
> > field entry, but I do want to verify the probabilistic field at
multiple
> > ceiling height thresholds, and this seems logical as to how to do
it.
> >
> > But when I run MODE I get these error messages:
> > WARNING:
> > WARNING: Dictionary::lookup_dictionary() -> dictionary lookup
failed for
> > name "field"
> > WARNING:
> > ERROR :
> > ERROR : Dictionary::lookup_dictionary() -> lookup failed for name
> "field"
> > ERROR :
> >
> > Config file is attached. I am using version 9.1.
> >
> > Jeff Duda
> > --
> > Research Scientist I, University of Colorado Boulder
> > Cooperative Institute for Research in Environmental Sciences
> > NOAA/OAR/ESRL/Global Systems Laboratory/Assimilation and
Verification
> > Innovation Division
> > Boulder, CO
> >
> >
>
>
--
Research Scientist I, University of Colorado Boulder
Cooperative Institute for Research in Environmental Sciences
NOAA/OAR/ESRL/Global Systems Laboratory/Assimilation and Verification
Innovation Division
Boulder, CO
------------------------------------------------
Subject: MODE - verifying a probabilistic field with a deterministic field
From: John Halley Gotway
Time: Mon Jan 11 11:56:19 2021
Jeff,
I'd better go take a closer look at one of these files and figure out
what
works.
Can you please point me to a sample file...
- on the web
- somewhere on WCOSS or hera
- or post a sample to our anonymous ftp site (
http://www.dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk#ftp
)
Thanks,
John
On Mon, Jan 11, 2021 at 11:32 AM Jeffrey Duda - NOAA Affiliate via RT
<
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98171 >
>
> Hey John,
> Unfortunately, that suggestion did not work, as I got the same error
> message.
>
> I have tried this:
>
> field = [
> { name = "PROB"; level = "L0"; prob = { name = "HGT"; thresh_hi
= 152;
> }; },
> { name = "PROB"; level = "L0"; prob = { name = "HGT"; thresh_hi
= 305;
> }; },
> { name = "PROB"; level = "L0"; prob = { name = "HGT"; thresh_hi
= 915;
> }; } ];
>
> and this:
>
> field = [
> { name = "PROB"; level = "L0"; prob = { name = "HGT"; thresh_hi
= 152;
> }; };,
> { name = "PROB"; level = "L0"; prob = { name = "HGT"; thresh_hi
= 305;
> }; };,
> { name = "PROB"; level = "L0"; prob = { name = "HGT"; thresh_hi
= 915;
> }; }; ];
>
> and am getting the same result. Perhaps the problem lies somewhere
else?
>
> FWIW, I am referencing this part of section 3.5.1 of the MET User's
Guide:
>
> The "prob" entry in the forecast dictionary defines probability
> // information. It may either be set as a boolean (i.e. TRUE
or
> FALSE)
> // or as a dictionary defining probabilistic field
information.
> //
> // When set as a boolean to TRUE, it indicates that the
> "fcst.field" data
> // should be treated as probabilities. For example, when
verifying
> the
> // probabilistic NetCDF output of Ensemble-Stat, one could
> configure the
> // Grid-Stat or Point-Stat tools as follows:
> //
> // fcst = {
> // field = [ { name = "APCP_24_A24_ENS_FREQ_gt0.0";
> // level = "(*,*)";
> // prob = TRUE; } ];
> // }
> //
> // Setting "prob = TRUE" indicates that the
> "APCP_24_A24_ENS_FREQ_gt0.0"
> // data should be processed as probabilities.
> //
> // When set as a dictionary, it defines the probabilistic
field to
> be
> // used. For example, when verifying GRIB files containing
> probabilistic
> // data, one could configure the Grid-Stat or Point-Stat
tools as
> // follows:
> //*// fcst = {
> // field = [ { name = "PROB"; level = "A24";
> // prob = { name = "APCP"; thresh_lo =
2.54; } },
> // { name = "PROB"; level = "P850";
> // prob = { name = "TMP"; thresh_hi = 273;
} } ];
> // }*
> //
> // The example above selects two probabilistic fields. In
both,
> "name"
> // is set to "PROB", the GRIB abbreviation for probabilities.
The
> "level"
> // entry defines the level information (i.e. "A24" for a 24-
hour
> // accumulation and "P850" for 850mb). The "prob" dictionary
> defines the
> // event for which the probability is defined. The
"thresh_lo"
> // (i.e. APCP > 2.54) and/or "thresh_hi" (i.e. TMP < 273)
entries
> are
> // used to define the event threshold(s).
> //
> // Probability fields should contain values in the range
> // [0, 1] or [0, 100]. However, when MET encounters a
probability
> field
> // with a range [0, 100], it will automatically rescale it to
be
> [0, 1]
> // before applying the probabilistic verification methods.
> //
>
> I'm not sure if there are any syntax errors in that reference code,
or if
> maybe I need to appeal to the *prob_as_scalar* flag as well. Since I
have
> never formally run MODE on a probabilistic field (I have run MODE on
> probabilistic fields by just assuming it was a scalar field), this
is new
> to me.
>
> Jeff
>
> On Fri, Jan 8, 2021 at 3:48 PM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > Hi Jeff,
> >
> > I see that the config file parsing code is erroring out on your
config
> > file. I think you just need to terminate the "prob" setting with a
> > semicolon, as shown below. In addition, I think you should only be
using
> > the thresh_hi setting. That'll match the structure of the GRIB
records
> you
> > listed.
> >
> > field = [
> > { name = "PROB"; level = "L000"; prob = { name = "HGT";
thresh_hi =
> > 152; }; },
> > { name = "PROB"; level = "L000"; prob = { name = "HGT";
thresh_hi =
> > 305; }; },
> > { name = "PROB"; level = "L000"; prob = { name = "HGT";
thresh_hi =
> > 915; }; }
> > ];
> >
> > Please let me know if that gets you any further and what other
issues pop
> > up.
> >
> > Thanks,
> > John Halley Gotway
> >
> > On Fri, Jan 8, 2021 at 2:43 PM Jeffrey Duda - NOAA Affiliate via
RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > Fri Jan 08 14:43:21 2021: Request 98171 was acted upon.
> > > Transaction: Ticket created by jeffrey.duda at noaa.gov
> > > Queue: met_help
> > > Subject: MODE - verifying a probabilistic field with a
> deterministic
> > > field
> > > Owner: Nobody
> > > Requestors: jeffrey.duda at noaa.gov
> > > Status: new
> > > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98171
> >
> > >
> > >
> > > Hello,
> > > I want to verify a forecast probabilistic field that is
comprised of
> > > separate fields with a single deterministic field.
> > >
> > > I have probabilities of ceiling height being below a series of
> thresholds
> > > T1, T2, T3. The arrays are in GRIB2 files with this structure
(from
> > > wgrib2):
> > >
> > > 9:10956329:d=2020121812:HGT:cloud ceiling:1 hour fcst:prob
<152:prob
> fcst
> > > 0/26:probability forecast
> > > 10:14766812:d=2020121812:HGT:cloud ceiling:1 hour fcst:prob
<305:prob
> > fcst
> > > 0/26:probability forecast
> > > 11:18577295:d=2020121812:HGT:cloud ceiling:1 hour fcst:prob
<915:prob
> > fcst
> > > 0/26:probability forecast
> > >
> > > These are not the only fields in this file with the abbreviation
"HGT",
> > so
> > > that is already one complication since the other HGT fields are
also
> > valid
> > > on only a certain custom level, so using "L0" for level in the
config
> > file
> > > confuses me. Regardless, here is my entry for fcst in the config
file
> > >
> > > fcst = {
> > > field = [
> > > { name = "PROB"; level = "L000"; prob = { name = "HGT";
thresh_hi
> =
> > > 152; } },
> > > { name = "PROB"; level = "L000"; prob = { name = "HGT";
thresh_hi
> =
> > > 305; thresh_lo = 152; } },
> > > { name = "PROB"; level = "L000"; prob = { name = "HGT";
thresh_hi
> =
> > > 915; thresh_lo = 305; } } ];
> > >
> > > censor_thresh = [];
> > > censor_val = [];
> > > conv_radius = 60.0/grid_res; // in grid squares
> > > conv_thresh =
> > >
> > >
> >
>
[>0.0,>=10.0,>=20.0,>=30.0,>=40.0,>=50.0,>=60.0,>=70.0,>=80.0,>=90.0,>99.0];
> > > vld_thresh = 0.5;
> > > filter_attr_name = [];
> > > filter_attr_thresh = [];
> > > merge_thresh = >=1.25;
> > > merge_flag = NONE;
> > > }
> > >
> > > I'm not even sure it is okay to use both "thresh_hi" and
"thresh_lo"
> in a
> > > field entry, but I do want to verify the probabilistic field at
> multiple
> > > ceiling height thresholds, and this seems logical as to how to
do it.
> > >
> > > But when I run MODE I get these error messages:
> > > WARNING:
> > > WARNING: Dictionary::lookup_dictionary() -> dictionary lookup
failed
> for
> > > name "field"
> > > WARNING:
> > > ERROR :
> > > ERROR : Dictionary::lookup_dictionary() -> lookup failed for
name
> > "field"
> > > ERROR :
> > >
> > > Config file is attached. I am using version 9.1.
> > >
> > > Jeff Duda
> > > --
> > > Research Scientist I, University of Colorado Boulder
> > > Cooperative Institute for Research in Environmental Sciences
> > > NOAA/OAR/ESRL/Global Systems Laboratory/Assimilation and
Verification
> > > Innovation Division
> > > Boulder, CO
> > >
> > >
> >
> >
>
> --
> Research Scientist I, University of Colorado Boulder
> Cooperative Institute for Research in Environmental Sciences
> NOAA/OAR/ESRL/Global Systems Laboratory/Assimilation and
Verification
> Innovation Division
> Boulder, CO
>
>
------------------------------------------------
Subject: MODE - verifying a probabilistic field with a deterministic field
From: Jeffrey Duda - NOAA Affiliate
Time: Mon Jan 11 12:08:44 2021
The files I'm working on are generally only available on HPSS, but I
have
some sitting on Hera right now for your use:
Forecast file (HRRRE probabilistic output):
/scratch2/BMC/wrfruc/jdduda/ceiling_vis_verif/HRRRE_fcst/postprd_ensprod/wrftwo_conus_ensprd_??.grib2
Obs file (RTMA):
/scratch2/BMC/wrfruc/jdduda/ceiling_vis_verif/RTMA/2035315000000
Config file is also right there in
/scratch2/BMC/wrfruc/jdduda/ceiling_vis_verif/MODE
Jeff
On Mon, Jan 11, 2021 at 11:56 AM John Halley Gotway via RT <
met_help at ucar.edu> wrote:
> Jeff,
>
> I'd better go take a closer look at one of these files and figure
out what
> works.
>
> Can you please point me to a sample file...
> - on the web
> - somewhere on WCOSS or hera
> - or post a sample to our anonymous ftp site (
>
> http://www.dtcenter.org/community-code/model-evaluation-tools-
met/met-help-desk#ftp
> )
>
> Thanks,
> John
>
> On Mon, Jan 11, 2021 at 11:32 AM Jeffrey Duda - NOAA Affiliate via
RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98171 >
> >
> > Hey John,
> > Unfortunately, that suggestion did not work, as I got the same
error
> > message.
> >
> > I have tried this:
> >
> > field = [
> > { name = "PROB"; level = "L0"; prob = { name = "HGT";
thresh_hi =
> 152;
> > }; },
> > { name = "PROB"; level = "L0"; prob = { name = "HGT";
thresh_hi =
> 305;
> > }; },
> > { name = "PROB"; level = "L0"; prob = { name = "HGT";
thresh_hi =
> 915;
> > }; } ];
> >
> > and this:
> >
> > field = [
> > { name = "PROB"; level = "L0"; prob = { name = "HGT";
thresh_hi =
> 152;
> > }; };,
> > { name = "PROB"; level = "L0"; prob = { name = "HGT";
thresh_hi =
> 305;
> > }; };,
> > { name = "PROB"; level = "L0"; prob = { name = "HGT";
thresh_hi =
> 915;
> > }; }; ];
> >
> > and am getting the same result. Perhaps the problem lies somewhere
else?
> >
> > FWIW, I am referencing this part of section 3.5.1 of the MET
User's
> Guide:
> >
> > The "prob" entry in the forecast dictionary defines probability
> > // information. It may either be set as a boolean (i.e.
TRUE or
> > FALSE)
> > // or as a dictionary defining probabilistic field
information.
> > //
> > // When set as a boolean to TRUE, it indicates that the
> > "fcst.field" data
> > // should be treated as probabilities. For example, when
verifying
> > the
> > // probabilistic NetCDF output of Ensemble-Stat, one could
> > configure the
> > // Grid-Stat or Point-Stat tools as follows:
> > //
> > // fcst = {
> > // field = [ { name = "APCP_24_A24_ENS_FREQ_gt0.0";
> > // level = "(*,*)";
> > // prob = TRUE; } ];
> > // }
> > //
> > // Setting "prob = TRUE" indicates that the
> > "APCP_24_A24_ENS_FREQ_gt0.0"
> > // data should be processed as probabilities.
> > //
> > // When set as a dictionary, it defines the probabilistic
field to
> > be
> > // used. For example, when verifying GRIB files containing
> > probabilistic
> > // data, one could configure the Grid-Stat or Point-Stat
tools as
> > // follows:
> > //*// fcst = {
> > // field = [ { name = "PROB"; level = "A24";
> > // prob = { name = "APCP"; thresh_lo =
2.54; }
> },
> > // { name = "PROB"; level = "P850";
> > // prob = { name = "TMP"; thresh_hi =
273; } }
> ];
> > // }*
> > //
> > // The example above selects two probabilistic fields. In
both,
> > "name"
> > // is set to "PROB", the GRIB abbreviation for
probabilities. The
> > "level"
> > // entry defines the level information (i.e. "A24" for a
24-hour
> > // accumulation and "P850" for 850mb). The "prob"
dictionary
> > defines the
> > // event for which the probability is defined. The
"thresh_lo"
> > // (i.e. APCP > 2.54) and/or "thresh_hi" (i.e. TMP < 273)
entries
> > are
> > // used to define the event threshold(s).
> > //
> > // Probability fields should contain values in the range
> > // [0, 1] or [0, 100]. However, when MET encounters a
probability
> > field
> > // with a range [0, 100], it will automatically rescale it
to be
> > [0, 1]
> > // before applying the probabilistic verification methods.
> > //
> >
> > I'm not sure if there are any syntax errors in that reference
code, or if
> > maybe I need to appeal to the *prob_as_scalar* flag as well. Since
I have
> > never formally run MODE on a probabilistic field (I have run MODE
on
> > probabilistic fields by just assuming it was a scalar field), this
is new
> > to me.
> >
> > Jeff
> >
> > On Fri, Jan 8, 2021 at 3:48 PM John Halley Gotway via RT <
> > met_help at ucar.edu>
> > wrote:
> >
> > > Hi Jeff,
> > >
> > > I see that the config file parsing code is erroring out on your
config
> > > file. I think you just need to terminate the "prob" setting with
a
> > > semicolon, as shown below. In addition, I think you should only
be
> using
> > > the thresh_hi setting. That'll match the structure of the GRIB
records
> > you
> > > listed.
> > >
> > > field = [
> > > { name = "PROB"; level = "L000"; prob = { name = "HGT";
thresh_hi
> =
> > > 152; }; },
> > > { name = "PROB"; level = "L000"; prob = { name = "HGT";
thresh_hi
> =
> > > 305; }; },
> > > { name = "PROB"; level = "L000"; prob = { name = "HGT";
thresh_hi
> =
> > > 915; }; }
> > > ];
> > >
> > > Please let me know if that gets you any further and what other
issues
> pop
> > > up.
> > >
> > > Thanks,
> > > John Halley Gotway
> > >
> > > On Fri, Jan 8, 2021 at 2:43 PM Jeffrey Duda - NOAA Affiliate via
RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > Fri Jan 08 14:43:21 2021: Request 98171 was acted upon.
> > > > Transaction: Ticket created by jeffrey.duda at noaa.gov
> > > > Queue: met_help
> > > > Subject: MODE - verifying a probabilistic field with a
> > deterministic
> > > > field
> > > > Owner: Nobody
> > > > Requestors: jeffrey.duda at noaa.gov
> > > > Status: new
> > > > Ticket <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98171
> > >
> > > >
> > > >
> > > > Hello,
> > > > I want to verify a forecast probabilistic field that is
comprised of
> > > > separate fields with a single deterministic field.
> > > >
> > > > I have probabilities of ceiling height being below a series of
> > thresholds
> > > > T1, T2, T3. The arrays are in GRIB2 files with this structure
(from
> > > > wgrib2):
> > > >
> > > > 9:10956329:d=2020121812:HGT:cloud ceiling:1 hour fcst:prob
<152:prob
> > fcst
> > > > 0/26:probability forecast
> > > > 10:14766812:d=2020121812:HGT:cloud ceiling:1 hour fcst:prob
<305:prob
> > > fcst
> > > > 0/26:probability forecast
> > > > 11:18577295:d=2020121812:HGT:cloud ceiling:1 hour fcst:prob
<915:prob
> > > fcst
> > > > 0/26:probability forecast
> > > >
> > > > These are not the only fields in this file with the
abbreviation
> "HGT",
> > > so
> > > > that is already one complication since the other HGT fields
are also
> > > valid
> > > > on only a certain custom level, so using "L0" for level in the
config
> > > file
> > > > confuses me. Regardless, here is my entry for fcst in the
config file
> > > >
> > > > fcst = {
> > > > field = [
> > > > { name = "PROB"; level = "L000"; prob = { name = "HGT";
> thresh_hi
> > =
> > > > 152; } },
> > > > { name = "PROB"; level = "L000"; prob = { name = "HGT";
> thresh_hi
> > =
> > > > 305; thresh_lo = 152; } },
> > > > { name = "PROB"; level = "L000"; prob = { name = "HGT";
> thresh_hi
> > =
> > > > 915; thresh_lo = 305; } } ];
> > > >
> > > > censor_thresh = [];
> > > > censor_val = [];
> > > > conv_radius = 60.0/grid_res; // in grid squares
> > > > conv_thresh =
> > > >
> > > >
> > >
> >
>
[>0.0,>=10.0,>=20.0,>=30.0,>=40.0,>=50.0,>=60.0,>=70.0,>=80.0,>=90.0,>99.0];
> > > > vld_thresh = 0.5;
> > > > filter_attr_name = [];
> > > > filter_attr_thresh = [];
> > > > merge_thresh = >=1.25;
> > > > merge_flag = NONE;
> > > > }
> > > >
> > > > I'm not even sure it is okay to use both "thresh_hi" and
"thresh_lo"
> > in a
> > > > field entry, but I do want to verify the probabilistic field
at
> > multiple
> > > > ceiling height thresholds, and this seems logical as to how to
do it.
> > > >
> > > > But when I run MODE I get these error messages:
> > > > WARNING:
> > > > WARNING: Dictionary::lookup_dictionary() -> dictionary lookup
failed
> > for
> > > > name "field"
> > > > WARNING:
> > > > ERROR :
> > > > ERROR : Dictionary::lookup_dictionary() -> lookup failed for
name
> > > "field"
> > > > ERROR :
> > > >
> > > > Config file is attached. I am using version 9.1.
> > > >
> > > > Jeff Duda
> > > > --
> > > > Research Scientist I, University of Colorado Boulder
> > > > Cooperative Institute for Research in Environmental Sciences
> > > > NOAA/OAR/ESRL/Global Systems Laboratory/Assimilation and
Verification
> > > > Innovation Division
> > > > Boulder, CO
> > > >
> > > >
> > >
> > >
> >
> > --
> > Research Scientist I, University of Colorado Boulder
> > Cooperative Institute for Research in Environmental Sciences
> > NOAA/OAR/ESRL/Global Systems Laboratory/Assimilation and
Verification
> > Innovation Division
> > Boulder, CO
> >
> >
>
>
--
Research Scientist I, University of Colorado Boulder
Cooperative Institute for Research in Environmental Sciences
NOAA/OAR/ESRL/Global Systems Laboratory/Assimilation and Verification
Innovation Division
Boulder, CO
------------------------------------------------
Subject: MODE - verifying a probabilistic field with a deterministic field
From: John Halley Gotway
Time: Mon Jan 11 13:23:52 2021
Jeff,
I understand what's going on here. First, I used plot_data_plane to
confirm
that the problem is not in how you setup the config options for
reading
probability data:
cd /home/John.H.Gotway/MET_Help/duda_data_20210111
*plot_data_plane wrftwo_conus_ensprod_03.grib2 prob_HGT_lt152.ps
'name="PROB"; level="L0"; prob={name="HGT";thresh_hi=152;}' -v 3*
DEBUG 3: MetGrib2DataFile::data_plane() -> Found exact match for
'PROB(HGT<152.000)/L0/PROB' in GRIB2 record 14 field 1 of GRIB2 file
'wrftwo_conus_ensprod_03.grib2'
*plot_data_plane wrftwo_conus_ensprod_03.grib2 prob_HGT_lt305.ps
'name="PROB"; level="L0"; prob={name="HGT";thresh_hi=305;}' -v 3*
DEBUG 3: MetGrib2DataFile::data_plane() -> Found exact match for
'PROB(HGT<305.000)/L0/PROB' in GRIB2 record 15 field 1 of GRIB2 file
'wrftwo_conus_ensprod_03.grib2'
*plot_data_plane wrftwo_conus_ensprod_03.grib2 prob_HGT_lt915.ps
'name="PROB"; level="L0"; prob={name="HGT";thresh_hi=915;}' -v 3*
DEBUG 3: MetGrib2DataFile::data_plane() -> Found exact match for
'PROB(HGT<915.000)/L0/PROB' in GRIB2 record 16 field 1 of GRIB2 file
'wrftwo_conus_ensprod_03.grib2'
The log DEBUG 3 log messages confirm that plot_data_plane matches
records
14, 15, and 16.
The problem is in what you're trying to do. The MODE tool in MET
version
9.1 can only be run on a single input field, not 3. The config file
parsing
errors out because you've set "field" to an ARRAY (with [ ]) instead
of a
single dictionary (with { }).
Perhaps you saw Linsday Blank's work on the development of multi-
variate
MODE and assumed it was included in met-9.1? It is not, but is slated
for
inclusion in met-10.0.0. Here's the GitHub issue that describes that
work:
https://github.com/dtcenter/MET/issues/1184
But you can run MODE on ONE of those fields at a time. I've attached a
version of the MODE config file which runs on this data. My goal was
to
just get a reasonable set of objects.
I compared forecast PROB(HGT<152) >0 to observations of Cloud Base
Height
<152.
This makes...
DEBUG 2: Identified: 14 forecast objects and 37 observation objects.
Applying the CONUS.poly mask to a dense grid is slow. Instead, I ran
gen_vx_mask to generate a NetCDF file for this mask:
*gen_vx_mask wrftwo_conus_ensprod_03.grib2
/contrib/met/9.1/share/met/poly/CONUS.poly wrftwo_CONUS_mask.nc -type
poly
-name CONUS*
And I updated the MODE config file to point to that output file:
* poly = "./wrftwo_CONUS_mask.nc";*
I've attached a PDF of the resulting PostScript output. I notice a
problem
here. In the observations, you have fields of valid cloud base values
surrounded by BAD DATA values. Presumably, there are no clouds there
so the
cloud base is "infinity". The probability values are surrounded by
values
of 0's which affect the convolution smoothing step. The obs values are
surrounded by bad data, which has no effect on the smoothing. To make
it a
more fair comparison in this way, I tried censoring the data,
replacing bad
data with a large value, like 16000:
censor_thresh = [ ==-9999 ];
censor_val = [ 16000 ];
That config file and a pdf of the result is also attached.
Ultimately it's up to you to decide what comparison to make, what
smoothing
and thresholds to apply, and whether or not to "censor" the data to
make
the smoothing step more fair. Since the forecast probabilities are
already
very smooth, you could probably set the convolution radius to 0 to
skip the
smoothing step.
Hope this helps get you going.
Thanks,
John
On Mon, Jan 11, 2021 at 12:09 PM Jeffrey Duda - NOAA Affiliate via RT
<
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98171 >
>
> The files I'm working on are generally only available on HPSS, but I
have
> some sitting on Hera right now for your use:
>
> Forecast file (HRRRE probabilistic output):
>
>
/scratch2/BMC/wrfruc/jdduda/ceiling_vis_verif/HRRRE_fcst/postprd_ensprod/wrftwo_conus_ensprd_??.grib2
> Obs file (RTMA):
> /scratch2/BMC/wrfruc/jdduda/ceiling_vis_verif/RTMA/2035315000000
>
> Config file is also right there in
> /scratch2/BMC/wrfruc/jdduda/ceiling_vis_verif/MODE
>
> Jeff
>
> On Mon, Jan 11, 2021 at 11:56 AM John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
> > Jeff,
> >
> > I'd better go take a closer look at one of these files and figure
out
> what
> > works.
> >
> > Can you please point me to a sample file...
> > - on the web
> > - somewhere on WCOSS or hera
> > - or post a sample to our anonymous ftp site (
> >
> >
> http://www.dtcenter.org/community-code/model-evaluation-tools-
met/met-help-desk#ftp
> > )
> >
> > Thanks,
> > John
> >
> > On Mon, Jan 11, 2021 at 11:32 AM Jeffrey Duda - NOAA Affiliate via
RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98171 >
> > >
> > > Hey John,
> > > Unfortunately, that suggestion did not work, as I got the same
error
> > > message.
> > >
> > > I have tried this:
> > >
> > > field = [
> > > { name = "PROB"; level = "L0"; prob = { name = "HGT";
thresh_hi =
> > 152;
> > > }; },
> > > { name = "PROB"; level = "L0"; prob = { name = "HGT";
thresh_hi =
> > 305;
> > > }; },
> > > { name = "PROB"; level = "L0"; prob = { name = "HGT";
thresh_hi =
> > 915;
> > > }; } ];
> > >
> > > and this:
> > >
> > > field = [
> > > { name = "PROB"; level = "L0"; prob = { name = "HGT";
thresh_hi =
> > 152;
> > > }; };,
> > > { name = "PROB"; level = "L0"; prob = { name = "HGT";
thresh_hi =
> > 305;
> > > }; };,
> > > { name = "PROB"; level = "L0"; prob = { name = "HGT";
thresh_hi =
> > 915;
> > > }; }; ];
> > >
> > > and am getting the same result. Perhaps the problem lies
somewhere
> else?
> > >
> > > FWIW, I am referencing this part of section 3.5.1 of the MET
User's
> > Guide:
> > >
> > > The "prob" entry in the forecast dictionary defines probability
> > > // information. It may either be set as a boolean (i.e.
TRUE or
> > > FALSE)
> > > // or as a dictionary defining probabilistic field
information.
> > > //
> > > // When set as a boolean to TRUE, it indicates that the
> > > "fcst.field" data
> > > // should be treated as probabilities. For example, when
> verifying
> > > the
> > > // probabilistic NetCDF output of Ensemble-Stat, one
could
> > > configure the
> > > // Grid-Stat or Point-Stat tools as follows:
> > > //
> > > // fcst = {
> > > // field = [ { name =
"APCP_24_A24_ENS_FREQ_gt0.0";
> > > // level = "(*,*)";
> > > // prob = TRUE; } ];
> > > // }
> > > //
> > > // Setting "prob = TRUE" indicates that the
> > > "APCP_24_A24_ENS_FREQ_gt0.0"
> > > // data should be processed as probabilities.
> > > //
> > > // When set as a dictionary, it defines the probabilistic
field
> to
> > > be
> > > // used. For example, when verifying GRIB files
containing
> > > probabilistic
> > > // data, one could configure the Grid-Stat or Point-Stat
tools
> as
> > > // follows:
> > > //*// fcst = {
> > > // field = [ { name = "PROB"; level = "A24";
> > > // prob = { name = "APCP"; thresh_lo =
2.54; }
> > },
> > > // { name = "PROB"; level = "P850";
> > > // prob = { name = "TMP"; thresh_hi =
273; } }
> > ];
> > > // }*
> > > //
> > > // The example above selects two probabilistic fields. In
both,
> > > "name"
> > > // is set to "PROB", the GRIB abbreviation for
probabilities.
> The
> > > "level"
> > > // entry defines the level information (i.e. "A24" for a
24-hour
> > > // accumulation and "P850" for 850mb). The "prob"
dictionary
> > > defines the
> > > // event for which the probability is defined. The
"thresh_lo"
> > > // (i.e. APCP > 2.54) and/or "thresh_hi" (i.e. TMP < 273)
> entries
> > > are
> > > // used to define the event threshold(s).
> > > //
> > > // Probability fields should contain values in the range
> > > // [0, 1] or [0, 100]. However, when MET encounters a
> probability
> > > field
> > > // with a range [0, 100], it will automatically rescale
it to be
> > > [0, 1]
> > > // before applying the probabilistic verification
methods.
> > > //
> > >
> > > I'm not sure if there are any syntax errors in that reference
code, or
> if
> > > maybe I need to appeal to the *prob_as_scalar* flag as well.
Since I
> have
> > > never formally run MODE on a probabilistic field (I have run
MODE on
> > > probabilistic fields by just assuming it was a scalar field),
this is
> new
> > > to me.
> > >
> > > Jeff
> > >
> > > On Fri, Jan 8, 2021 at 3:48 PM John Halley Gotway via RT <
> > > met_help at ucar.edu>
> > > wrote:
> > >
> > > > Hi Jeff,
> > > >
> > > > I see that the config file parsing code is erroring out on
your
> config
> > > > file. I think you just need to terminate the "prob" setting
with a
> > > > semicolon, as shown below. In addition, I think you should
only be
> > using
> > > > the thresh_hi setting. That'll match the structure of the GRIB
> records
> > > you
> > > > listed.
> > > >
> > > > field = [
> > > > { name = "PROB"; level = "L000"; prob = { name = "HGT";
> thresh_hi
> > =
> > > > 152; }; },
> > > > { name = "PROB"; level = "L000"; prob = { name = "HGT";
> thresh_hi
> > =
> > > > 305; }; },
> > > > { name = "PROB"; level = "L000"; prob = { name = "HGT";
> thresh_hi
> > =
> > > > 915; }; }
> > > > ];
> > > >
> > > > Please let me know if that gets you any further and what other
issues
> > pop
> > > > up.
> > > >
> > > > Thanks,
> > > > John Halley Gotway
> > > >
> > > > On Fri, Jan 8, 2021 at 2:43 PM Jeffrey Duda - NOAA Affiliate
via RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > Fri Jan 08 14:43:21 2021: Request 98171 was acted upon.
> > > > > Transaction: Ticket created by jeffrey.duda at noaa.gov
> > > > > Queue: met_help
> > > > > Subject: MODE - verifying a probabilistic field with a
> > > deterministic
> > > > > field
> > > > > Owner: Nobody
> > > > > Requestors: jeffrey.duda at noaa.gov
> > > > > Status: new
> > > > > Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98171
> > > >
> > > > >
> > > > >
> > > > > Hello,
> > > > > I want to verify a forecast probabilistic field that is
comprised
> of
> > > > > separate fields with a single deterministic field.
> > > > >
> > > > > I have probabilities of ceiling height being below a series
of
> > > thresholds
> > > > > T1, T2, T3. The arrays are in GRIB2 files with this
structure (from
> > > > > wgrib2):
> > > > >
> > > > > 9:10956329:d=2020121812:HGT:cloud ceiling:1 hour fcst:prob
> <152:prob
> > > fcst
> > > > > 0/26:probability forecast
> > > > > 10:14766812:d=2020121812:HGT:cloud ceiling:1 hour fcst:prob
> <305:prob
> > > > fcst
> > > > > 0/26:probability forecast
> > > > > 11:18577295:d=2020121812:HGT:cloud ceiling:1 hour fcst:prob
> <915:prob
> > > > fcst
> > > > > 0/26:probability forecast
> > > > >
> > > > > These are not the only fields in this file with the
abbreviation
> > "HGT",
> > > > so
> > > > > that is already one complication since the other HGT fields
are
> also
> > > > valid
> > > > > on only a certain custom level, so using "L0" for level in
the
> config
> > > > file
> > > > > confuses me. Regardless, here is my entry for fcst in the
config
> file
> > > > >
> > > > > fcst = {
> > > > > field = [
> > > > > { name = "PROB"; level = "L000"; prob = { name = "HGT";
> > thresh_hi
> > > =
> > > > > 152; } },
> > > > > { name = "PROB"; level = "L000"; prob = { name = "HGT";
> > thresh_hi
> > > =
> > > > > 305; thresh_lo = 152; } },
> > > > > { name = "PROB"; level = "L000"; prob = { name = "HGT";
> > thresh_hi
> > > =
> > > > > 915; thresh_lo = 305; } } ];
> > > > >
> > > > > censor_thresh = [];
> > > > > censor_val = [];
> > > > > conv_radius = 60.0/grid_res; // in grid squares
> > > > > conv_thresh =
> > > > >
> > > > >
> > > >
> > >
> >
>
[>0.0,>=10.0,>=20.0,>=30.0,>=40.0,>=50.0,>=60.0,>=70.0,>=80.0,>=90.0,>99.0];
> > > > > vld_thresh = 0.5;
> > > > > filter_attr_name = [];
> > > > > filter_attr_thresh = [];
> > > > > merge_thresh = >=1.25;
> > > > > merge_flag = NONE;
> > > > > }
> > > > >
> > > > > I'm not even sure it is okay to use both "thresh_hi" and
> "thresh_lo"
> > > in a
> > > > > field entry, but I do want to verify the probabilistic field
at
> > > multiple
> > > > > ceiling height thresholds, and this seems logical as to how
to do
> it.
> > > > >
> > > > > But when I run MODE I get these error messages:
> > > > > WARNING:
> > > > > WARNING: Dictionary::lookup_dictionary() -> dictionary
lookup
> failed
> > > for
> > > > > name "field"
> > > > > WARNING:
> > > > > ERROR :
> > > > > ERROR : Dictionary::lookup_dictionary() -> lookup failed
for name
> > > > "field"
> > > > > ERROR :
> > > > >
> > > > > Config file is attached. I am using version 9.1.
> > > > >
> > > > > Jeff Duda
> > > > > --
> > > > > Research Scientist I, University of Colorado Boulder
> > > > > Cooperative Institute for Research in Environmental Sciences
> > > > > NOAA/OAR/ESRL/Global Systems Laboratory/Assimilation and
> Verification
> > > > > Innovation Division
> > > > > Boulder, CO
> > > > >
> > > > >
> > > >
> > > >
> > >
> > > --
> > > Research Scientist I, University of Colorado Boulder
> > > Cooperative Institute for Research in Environmental Sciences
> > > NOAA/OAR/ESRL/Global Systems Laboratory/Assimilation and
Verification
> > > Innovation Division
> > > Boulder, CO
> > >
> > >
> >
> >
>
> --
> Research Scientist I, University of Colorado Boulder
> Cooperative Institute for Research in Environmental Sciences
> NOAA/OAR/ESRL/Global Systems Laboratory/Assimilation and
Verification
> Innovation Division
> Boulder, CO
>
>
------------------------------------------------
Subject: MODE - verifying a probabilistic field with a deterministic field
From: Jeffrey Duda - NOAA Affiliate
Time: Mon Jan 11 13:42:33 2021
Thanks John.
I guess I got confused by the example in the User's Guide. Can you
explain
to me what that example is doing differently and why that is not in
violation of MODE's single-field capabilities?
Jeff
On Mon, Jan 11, 2021 at 1:24 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Jeff,
>
> I understand what's going on here. First, I used plot_data_plane to
confirm
> that the problem is not in how you setup the config options for
reading
> probability data:
>
> cd /home/John.H.Gotway/MET_Help/duda_data_20210111
> *plot_data_plane wrftwo_conus_ensprod_03.grib2 prob_HGT_lt152.ps
> 'name="PROB"; level="L0"; prob={name="HGT";thresh_hi=152;}' -v 3*
> DEBUG 3: MetGrib2DataFile::data_plane() -> Found exact match for
> 'PROB(HGT<152.000)/L0/PROB' in GRIB2 record 14 field 1 of GRIB2 file
> 'wrftwo_conus_ensprod_03.grib2'
> *plot_data_plane wrftwo_conus_ensprod_03.grib2 prob_HGT_lt305.ps
> 'name="PROB"; level="L0"; prob={name="HGT";thresh_hi=305;}' -v 3*
> DEBUG 3: MetGrib2DataFile::data_plane() -> Found exact match for
> 'PROB(HGT<305.000)/L0/PROB' in GRIB2 record 15 field 1 of GRIB2 file
> 'wrftwo_conus_ensprod_03.grib2'
> *plot_data_plane wrftwo_conus_ensprod_03.grib2 prob_HGT_lt915.ps
> 'name="PROB"; level="L0"; prob={name="HGT";thresh_hi=915;}' -v 3*
> DEBUG 3: MetGrib2DataFile::data_plane() -> Found exact match for
> 'PROB(HGT<915.000)/L0/PROB' in GRIB2 record 16 field 1 of GRIB2 file
> 'wrftwo_conus_ensprod_03.grib2'
>
> The log DEBUG 3 log messages confirm that plot_data_plane matches
records
> 14, 15, and 16.
>
> The problem is in what you're trying to do. The MODE tool in MET
version
> 9.1 can only be run on a single input field, not 3. The config file
parsing
> errors out because you've set "field" to an ARRAY (with [ ]) instead
of a
> single dictionary (with { }).
>
> Perhaps you saw Linsday Blank's work on the development of multi-
variate
> MODE and assumed it was included in met-9.1? It is not, but is
slated for
> inclusion in met-10.0.0. Here's the GitHub issue that describes that
work:
> https://github.com/dtcenter/MET/issues/1184
>
> But you can run MODE on ONE of those fields at a time. I've attached
a
> version of the MODE config file which runs on this data. My goal was
to
> just get a reasonable set of objects.
>
> I compared forecast PROB(HGT<152) >0 to observations of Cloud Base
Height
> <152.
>
> This makes...
> DEBUG 2: Identified: 14 forecast objects and 37 observation objects.
>
> Applying the CONUS.poly mask to a dense grid is slow. Instead, I ran
> gen_vx_mask to generate a NetCDF file for this mask:
> *gen_vx_mask wrftwo_conus_ensprod_03.grib2
> /contrib/met/9.1/share/met/poly/CONUS.poly wrftwo_CONUS_mask.nc
-type poly
> -name CONUS*
>
> And I updated the MODE config file to point to that output file:
> * poly = "./wrftwo_CONUS_mask.nc";*
>
> I've attached a PDF of the resulting PostScript output. I notice a
problem
> here. In the observations, you have fields of valid cloud base
values
> surrounded by BAD DATA values. Presumably, there are no clouds there
so the
> cloud base is "infinity". The probability values are surrounded by
values
> of 0's which affect the convolution smoothing step. The obs values
are
> surrounded by bad data, which has no effect on the smoothing. To
make it a
> more fair comparison in this way, I tried censoring the data,
replacing bad
> data with a large value, like 16000:
>
> censor_thresh = [ ==-9999 ];
> censor_val = [ 16000 ];
>
> That config file and a pdf of the result is also attached.
>
> Ultimately it's up to you to decide what comparison to make, what
smoothing
> and thresholds to apply, and whether or not to "censor" the data to
make
> the smoothing step more fair. Since the forecast probabilities are
already
> very smooth, you could probably set the convolution radius to 0 to
skip the
> smoothing step.
>
> Hope this helps get you going.
>
> Thanks,
> John
>
> On Mon, Jan 11, 2021 at 12:09 PM Jeffrey Duda - NOAA Affiliate via
RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98171 >
> >
> > The files I'm working on are generally only available on HPSS, but
I have
> > some sitting on Hera right now for your use:
> >
> > Forecast file (HRRRE probabilistic output):
> >
> >
>
/scratch2/BMC/wrfruc/jdduda/ceiling_vis_verif/HRRRE_fcst/postprd_ensprod/wrftwo_conus_ensprd_??.grib2
> > Obs file (RTMA):
> > /scratch2/BMC/wrfruc/jdduda/ceiling_vis_verif/RTMA/2035315000000
> >
> > Config file is also right there in
> > /scratch2/BMC/wrfruc/jdduda/ceiling_vis_verif/MODE
> >
> > Jeff
> >
> > On Mon, Jan 11, 2021 at 11:56 AM John Halley Gotway via RT <
> > met_help at ucar.edu> wrote:
> >
> > > Jeff,
> > >
> > > I'd better go take a closer look at one of these files and
figure out
> > what
> > > works.
> > >
> > > Can you please point me to a sample file...
> > > - on the web
> > > - somewhere on WCOSS or hera
> > > - or post a sample to our anonymous ftp site (
> > >
> > >
> >
> http://www.dtcenter.org/community-code/model-evaluation-tools-
met/met-help-desk#ftp
> > > )
> > >
> > > Thanks,
> > > John
> > >
> > > On Mon, Jan 11, 2021 at 11:32 AM Jeffrey Duda - NOAA Affiliate
via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98171
>
> > > >
> > > > Hey John,
> > > > Unfortunately, that suggestion did not work, as I got the same
error
> > > > message.
> > > >
> > > > I have tried this:
> > > >
> > > > field = [
> > > > { name = "PROB"; level = "L0"; prob = { name = "HGT";
thresh_hi
> =
> > > 152;
> > > > }; },
> > > > { name = "PROB"; level = "L0"; prob = { name = "HGT";
thresh_hi
> =
> > > 305;
> > > > }; },
> > > > { name = "PROB"; level = "L0"; prob = { name = "HGT";
thresh_hi
> =
> > > 915;
> > > > }; } ];
> > > >
> > > > and this:
> > > >
> > > > field = [
> > > > { name = "PROB"; level = "L0"; prob = { name = "HGT";
thresh_hi
> =
> > > 152;
> > > > }; };,
> > > > { name = "PROB"; level = "L0"; prob = { name = "HGT";
thresh_hi
> =
> > > 305;
> > > > }; };,
> > > > { name = "PROB"; level = "L0"; prob = { name = "HGT";
thresh_hi
> =
> > > 915;
> > > > }; }; ];
> > > >
> > > > and am getting the same result. Perhaps the problem lies
somewhere
> > else?
> > > >
> > > > FWIW, I am referencing this part of section 3.5.1 of the MET
User's
> > > Guide:
> > > >
> > > > The "prob" entry in the forecast dictionary defines
probability
> > > > // information. It may either be set as a boolean (i.e.
TRUE
> or
> > > > FALSE)
> > > > // or as a dictionary defining probabilistic field
> information.
> > > > //
> > > > // When set as a boolean to TRUE, it indicates that the
> > > > "fcst.field" data
> > > > // should be treated as probabilities. For example,
when
> > verifying
> > > > the
> > > > // probabilistic NetCDF output of Ensemble-Stat, one
could
> > > > configure the
> > > > // Grid-Stat or Point-Stat tools as follows:
> > > > //
> > > > // fcst = {
> > > > // field = [ { name =
"APCP_24_A24_ENS_FREQ_gt0.0";
> > > > // level = "(*,*)";
> > > > // prob = TRUE; } ];
> > > > // }
> > > > //
> > > > // Setting "prob = TRUE" indicates that the
> > > > "APCP_24_A24_ENS_FREQ_gt0.0"
> > > > // data should be processed as probabilities.
> > > > //
> > > > // When set as a dictionary, it defines the
probabilistic
> field
> > to
> > > > be
> > > > // used. For example, when verifying GRIB files
containing
> > > > probabilistic
> > > > // data, one could configure the Grid-Stat or Point-
Stat
> tools
> > as
> > > > // follows:
> > > > //*// fcst = {
> > > > // field = [ { name = "PROB"; level = "A24";
> > > > // prob = { name = "APCP"; thresh_lo
=
> 2.54; }
> > > },
> > > > // { name = "PROB"; level = "P850";
> > > > // prob = { name = "TMP"; thresh_hi =
273;
> } }
> > > ];
> > > > // }*
> > > > //
> > > > // The example above selects two probabilistic fields.
In
> both,
> > > > "name"
> > > > // is set to "PROB", the GRIB abbreviation for
probabilities.
> > The
> > > > "level"
> > > > // entry defines the level information (i.e. "A24" for
a
> 24-hour
> > > > // accumulation and "P850" for 850mb). The "prob"
dictionary
> > > > defines the
> > > > // event for which the probability is defined. The
"thresh_lo"
> > > > // (i.e. APCP > 2.54) and/or "thresh_hi" (i.e. TMP <
273)
> > entries
> > > > are
> > > > // used to define the event threshold(s).
> > > > //
> > > > // Probability fields should contain values in the
range
> > > > // [0, 1] or [0, 100]. However, when MET encounters a
> > probability
> > > > field
> > > > // with a range [0, 100], it will automatically rescale
it to
> be
> > > > [0, 1]
> > > > // before applying the probabilistic verification
methods.
> > > > //
> > > >
> > > > I'm not sure if there are any syntax errors in that reference
code,
> or
> > if
> > > > maybe I need to appeal to the *prob_as_scalar* flag as well.
Since I
> > have
> > > > never formally run MODE on a probabilistic field (I have run
MODE on
> > > > probabilistic fields by just assuming it was a scalar field),
this is
> > new
> > > > to me.
> > > >
> > > > Jeff
> > > >
> > > > On Fri, Jan 8, 2021 at 3:48 PM John Halley Gotway via RT <
> > > > met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > > Hi Jeff,
> > > > >
> > > > > I see that the config file parsing code is erroring out on
your
> > config
> > > > > file. I think you just need to terminate the "prob" setting
with a
> > > > > semicolon, as shown below. In addition, I think you should
only be
> > > using
> > > > > the thresh_hi setting. That'll match the structure of the
GRIB
> > records
> > > > you
> > > > > listed.
> > > > >
> > > > > field = [
> > > > > { name = "PROB"; level = "L000"; prob = { name = "HGT";
> > thresh_hi
> > > =
> > > > > 152; }; },
> > > > > { name = "PROB"; level = "L000"; prob = { name = "HGT";
> > thresh_hi
> > > =
> > > > > 305; }; },
> > > > > { name = "PROB"; level = "L000"; prob = { name = "HGT";
> > thresh_hi
> > > =
> > > > > 915; }; }
> > > > > ];
> > > > >
> > > > > Please let me know if that gets you any further and what
other
> issues
> > > pop
> > > > > up.
> > > > >
> > > > > Thanks,
> > > > > John Halley Gotway
> > > > >
> > > > > On Fri, Jan 8, 2021 at 2:43 PM Jeffrey Duda - NOAA Affiliate
via
> RT <
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > > >
> > > > > > Fri Jan 08 14:43:21 2021: Request 98171 was acted upon.
> > > > > > Transaction: Ticket created by jeffrey.duda at noaa.gov
> > > > > > Queue: met_help
> > > > > > Subject: MODE - verifying a probabilistic field with
a
> > > > deterministic
> > > > > > field
> > > > > > Owner: Nobody
> > > > > > Requestors: jeffrey.duda at noaa.gov
> > > > > > Status: new
> > > > > > Ticket <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98171
> > > > >
> > > > > >
> > > > > >
> > > > > > Hello,
> > > > > > I want to verify a forecast probabilistic field that is
comprised
> > of
> > > > > > separate fields with a single deterministic field.
> > > > > >
> > > > > > I have probabilities of ceiling height being below a
series of
> > > > thresholds
> > > > > > T1, T2, T3. The arrays are in GRIB2 files with this
structure
> (from
> > > > > > wgrib2):
> > > > > >
> > > > > > 9:10956329:d=2020121812:HGT:cloud ceiling:1 hour fcst:prob
> > <152:prob
> > > > fcst
> > > > > > 0/26:probability forecast
> > > > > > 10:14766812:d=2020121812:HGT:cloud ceiling:1 hour
fcst:prob
> > <305:prob
> > > > > fcst
> > > > > > 0/26:probability forecast
> > > > > > 11:18577295:d=2020121812:HGT:cloud ceiling:1 hour
fcst:prob
> > <915:prob
> > > > > fcst
> > > > > > 0/26:probability forecast
> > > > > >
> > > > > > These are not the only fields in this file with the
abbreviation
> > > "HGT",
> > > > > so
> > > > > > that is already one complication since the other HGT
fields are
> > also
> > > > > valid
> > > > > > on only a certain custom level, so using "L0" for level in
the
> > config
> > > > > file
> > > > > > confuses me. Regardless, here is my entry for fcst in the
config
> > file
> > > > > >
> > > > > > fcst = {
> > > > > > field = [
> > > > > > { name = "PROB"; level = "L000"; prob = { name =
"HGT";
> > > thresh_hi
> > > > =
> > > > > > 152; } },
> > > > > > { name = "PROB"; level = "L000"; prob = { name =
"HGT";
> > > thresh_hi
> > > > =
> > > > > > 305; thresh_lo = 152; } },
> > > > > > { name = "PROB"; level = "L000"; prob = { name =
"HGT";
> > > thresh_hi
> > > > =
> > > > > > 915; thresh_lo = 305; } } ];
> > > > > >
> > > > > > censor_thresh = [];
> > > > > > censor_val = [];
> > > > > > conv_radius = 60.0/grid_res; // in grid squares
> > > > > > conv_thresh =
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
[>0.0,>=10.0,>=20.0,>=30.0,>=40.0,>=50.0,>=60.0,>=70.0,>=80.0,>=90.0,>99.0];
> > > > > > vld_thresh = 0.5;
> > > > > > filter_attr_name = [];
> > > > > > filter_attr_thresh = [];
> > > > > > merge_thresh = >=1.25;
> > > > > > merge_flag = NONE;
> > > > > > }
> > > > > >
> > > > > > I'm not even sure it is okay to use both "thresh_hi" and
> > "thresh_lo"
> > > > in a
> > > > > > field entry, but I do want to verify the probabilistic
field at
> > > > multiple
> > > > > > ceiling height thresholds, and this seems logical as to
how to do
> > it.
> > > > > >
> > > > > > But when I run MODE I get these error messages:
> > > > > > WARNING:
> > > > > > WARNING: Dictionary::lookup_dictionary() -> dictionary
lookup
> > failed
> > > > for
> > > > > > name "field"
> > > > > > WARNING:
> > > > > > ERROR :
> > > > > > ERROR : Dictionary::lookup_dictionary() -> lookup failed
for
> name
> > > > > "field"
> > > > > > ERROR :
> > > > > >
> > > > > > Config file is attached. I am using version 9.1.
> > > > > >
> > > > > > Jeff Duda
> > > > > > --
> > > > > > Research Scientist I, University of Colorado Boulder
> > > > > > Cooperative Institute for Research in Environmental
Sciences
> > > > > > NOAA/OAR/ESRL/Global Systems Laboratory/Assimilation and
> > Verification
> > > > > > Innovation Division
> > > > > > Boulder, CO
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > > --
> > > > Research Scientist I, University of Colorado Boulder
> > > > Cooperative Institute for Research in Environmental Sciences
> > > > NOAA/OAR/ESRL/Global Systems Laboratory/Assimilation and
Verification
> > > > Innovation Division
> > > > Boulder, CO
> > > >
> > > >
> > >
> > >
> >
> > --
> > Research Scientist I, University of Colorado Boulder
> > Cooperative Institute for Research in Environmental Sciences
> > NOAA/OAR/ESRL/Global Systems Laboratory/Assimilation and
Verification
> > Innovation Division
> > Boulder, CO
> >
> >
>
>
--
Research Scientist I, University of Colorado Boulder
Cooperative Institute for Research in Environmental Sciences
NOAA/OAR/ESRL/Global Systems Laboratory/Assimilation and Verification
Innovation Division
Boulder, CO
------------------------------------------------
Subject: MODE - verifying a probabilistic field with a deterministic field
From: John Halley Gotway
Time: Mon Jan 11 14:00:09 2021
Jeff,
Looking in the MET User's Guide at the chapter about MODE:
https://dtcenter.github.io/MET/latest/Users_Guide/mode.html
I see this:
[image: Screen Shot 2021-01-11 at 1.54.45 PM.png]
And all we're doing there is referring to the common config file
options
section about setting the "field" entry. I realize that we should
modify
that to point out that in MODE the "field" is defined as a single
dictionary entry... and not as an array of fields.
However, one complication is that in MET version 10.0.0, "field" can
either be set to a single entry or an array of entries. If it's an
array of
entries, then those multiple fields are combined into a single set of
MODE
objects rather than looping over the list and running MODE once for
each.
I'm going to copy/paste this email as a comment on the GitHub issue
for
updating the multi-variate MODE documentation:
https://github.com/dtcenter/MET/issues/1592
Thanks for pointing out the confusion.
John
On Mon, Jan 11, 2021 at 1:43 PM Jeffrey Duda - NOAA Affiliate via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98171 >
>
> Thanks John.
>
> I guess I got confused by the example in the User's Guide. Can you
explain
> to me what that example is doing differently and why that is not in
> violation of MODE's single-field capabilities?
>
> Jeff
>
> On Mon, Jan 11, 2021 at 1:24 PM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > Jeff,
> >
> > I understand what's going on here. First, I used plot_data_plane
to
> confirm
> > that the problem is not in how you setup the config options for
reading
> > probability data:
> >
> > cd /home/John.H.Gotway/MET_Help/duda_data_20210111
> > *plot_data_plane wrftwo_conus_ensprod_03.grib2 prob_HGT_lt152.ps
> > 'name="PROB"; level="L0"; prob={name="HGT";thresh_hi=152;}' -v 3*
> > DEBUG 3: MetGrib2DataFile::data_plane() -> Found exact match for
> > 'PROB(HGT<152.000)/L0/PROB' in GRIB2 record 14 field 1 of GRIB2
file
> > 'wrftwo_conus_ensprod_03.grib2'
> > *plot_data_plane wrftwo_conus_ensprod_03.grib2 prob_HGT_lt305.ps
> > 'name="PROB"; level="L0"; prob={name="HGT";thresh_hi=305;}' -v 3*
> > DEBUG 3: MetGrib2DataFile::data_plane() -> Found exact match for
> > 'PROB(HGT<305.000)/L0/PROB' in GRIB2 record 15 field 1 of GRIB2
file
> > 'wrftwo_conus_ensprod_03.grib2'
> > *plot_data_plane wrftwo_conus_ensprod_03.grib2 prob_HGT_lt915.ps
> > 'name="PROB"; level="L0"; prob={name="HGT";thresh_hi=915;}' -v 3*
> > DEBUG 3: MetGrib2DataFile::data_plane() -> Found exact match for
> > 'PROB(HGT<915.000)/L0/PROB' in GRIB2 record 16 field 1 of GRIB2
file
> > 'wrftwo_conus_ensprod_03.grib2'
> >
> > The log DEBUG 3 log messages confirm that plot_data_plane matches
records
> > 14, 15, and 16.
> >
> > The problem is in what you're trying to do. The MODE tool in MET
version
> > 9.1 can only be run on a single input field, not 3. The config
file
> parsing
> > errors out because you've set "field" to an ARRAY (with [ ])
instead of a
> > single dictionary (with { }).
> >
> > Perhaps you saw Linsday Blank's work on the development of multi-
variate
> > MODE and assumed it was included in met-9.1? It is not, but is
slated for
> > inclusion in met-10.0.0. Here's the GitHub issue that describes
that
> work:
> > https://github.com/dtcenter/MET/issues/1184
> >
> > But you can run MODE on ONE of those fields at a time. I've
attached a
> > version of the MODE config file which runs on this data. My goal
was to
> > just get a reasonable set of objects.
> >
> > I compared forecast PROB(HGT<152) >0 to observations of Cloud Base
Height
> > <152.
> >
> > This makes...
> > DEBUG 2: Identified: 14 forecast objects and 37 observation
objects.
> >
> > Applying the CONUS.poly mask to a dense grid is slow. Instead, I
ran
> > gen_vx_mask to generate a NetCDF file for this mask:
> > *gen_vx_mask wrftwo_conus_ensprod_03.grib2
> > /contrib/met/9.1/share/met/poly/CONUS.poly wrftwo_CONUS_mask.nc
-type
> poly
> > -name CONUS*
> >
> > And I updated the MODE config file to point to that output file:
> > * poly = "./wrftwo_CONUS_mask.nc";*
> >
> > I've attached a PDF of the resulting PostScript output. I notice a
> problem
> > here. In the observations, you have fields of valid cloud base
values
> > surrounded by BAD DATA values. Presumably, there are no clouds
there so
> the
> > cloud base is "infinity". The probability values are surrounded by
values
> > of 0's which affect the convolution smoothing step. The obs values
are
> > surrounded by bad data, which has no effect on the smoothing. To
make it
> a
> > more fair comparison in this way, I tried censoring the data,
replacing
> bad
> > data with a large value, like 16000:
> >
> > censor_thresh = [ ==-9999 ];
> > censor_val = [ 16000 ];
> >
> > That config file and a pdf of the result is also attached.
> >
> > Ultimately it's up to you to decide what comparison to make, what
> smoothing
> > and thresholds to apply, and whether or not to "censor" the data
to make
> > the smoothing step more fair. Since the forecast probabilities are
> already
> > very smooth, you could probably set the convolution radius to 0 to
skip
> the
> > smoothing step.
> >
> > Hope this helps get you going.
> >
> > Thanks,
> > John
> >
> > On Mon, Jan 11, 2021 at 12:09 PM Jeffrey Duda - NOAA Affiliate via
RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98171 >
> > >
> > > The files I'm working on are generally only available on HPSS,
but I
> have
> > > some sitting on Hera right now for your use:
> > >
> > > Forecast file (HRRRE probabilistic output):
> > >
> > >
> >
>
/scratch2/BMC/wrfruc/jdduda/ceiling_vis_verif/HRRRE_fcst/postprd_ensprod/wrftwo_conus_ensprd_??.grib2
> > > Obs file (RTMA):
> > > /scratch2/BMC/wrfruc/jdduda/ceiling_vis_verif/RTMA/2035315000000
> > >
> > > Config file is also right there in
> > > /scratch2/BMC/wrfruc/jdduda/ceiling_vis_verif/MODE
> > >
> > > Jeff
> > >
> > > On Mon, Jan 11, 2021 at 11:56 AM John Halley Gotway via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > > Jeff,
> > > >
> > > > I'd better go take a closer look at one of these files and
figure out
> > > what
> > > > works.
> > > >
> > > > Can you please point me to a sample file...
> > > > - on the web
> > > > - somewhere on WCOSS or hera
> > > > - or post a sample to our anonymous ftp site (
> > > >
> > > >
> > >
> >
> http://www.dtcenter.org/community-code/model-evaluation-tools-
met/met-help-desk#ftp
> > > > )
> > > >
> > > > Thanks,
> > > > John
> > > >
> > > > On Mon, Jan 11, 2021 at 11:32 AM Jeffrey Duda - NOAA Affiliate
via
> RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98171 >
> > > > >
> > > > > Hey John,
> > > > > Unfortunately, that suggestion did not work, as I got the
same
> error
> > > > > message.
> > > > >
> > > > > I have tried this:
> > > > >
> > > > > field = [
> > > > > { name = "PROB"; level = "L0"; prob = { name = "HGT";
> thresh_hi
> > =
> > > > 152;
> > > > > }; },
> > > > > { name = "PROB"; level = "L0"; prob = { name = "HGT";
> thresh_hi
> > =
> > > > 305;
> > > > > }; },
> > > > > { name = "PROB"; level = "L0"; prob = { name = "HGT";
> thresh_hi
> > =
> > > > 915;
> > > > > }; } ];
> > > > >
> > > > > and this:
> > > > >
> > > > > field = [
> > > > > { name = "PROB"; level = "L0"; prob = { name = "HGT";
> thresh_hi
> > =
> > > > 152;
> > > > > }; };,
> > > > > { name = "PROB"; level = "L0"; prob = { name = "HGT";
> thresh_hi
> > =
> > > > 305;
> > > > > }; };,
> > > > > { name = "PROB"; level = "L0"; prob = { name = "HGT";
> thresh_hi
> > =
> > > > 915;
> > > > > }; }; ];
> > > > >
> > > > > and am getting the same result. Perhaps the problem lies
somewhere
> > > else?
> > > > >
> > > > > FWIW, I am referencing this part of section 3.5.1 of the MET
User's
> > > > Guide:
> > > > >
> > > > > The "prob" entry in the forecast dictionary defines
probability
> > > > > // information. It may either be set as a boolean
(i.e. TRUE
> > or
> > > > > FALSE)
> > > > > // or as a dictionary defining probabilistic field
> > information.
> > > > > //
> > > > > // When set as a boolean to TRUE, it indicates that
the
> > > > > "fcst.field" data
> > > > > // should be treated as probabilities. For example,
when
> > > verifying
> > > > > the
> > > > > // probabilistic NetCDF output of Ensemble-Stat, one
could
> > > > > configure the
> > > > > // Grid-Stat or Point-Stat tools as follows:
> > > > > //
> > > > > // fcst = {
> > > > > // field = [ { name =
"APCP_24_A24_ENS_FREQ_gt0.0";
> > > > > // level = "(*,*)";
> > > > > // prob = TRUE; } ];
> > > > > // }
> > > > > //
> > > > > // Setting "prob = TRUE" indicates that the
> > > > > "APCP_24_A24_ENS_FREQ_gt0.0"
> > > > > // data should be processed as probabilities.
> > > > > //
> > > > > // When set as a dictionary, it defines the
probabilistic
> > field
> > > to
> > > > > be
> > > > > // used. For example, when verifying GRIB files
containing
> > > > > probabilistic
> > > > > // data, one could configure the Grid-Stat or Point-
Stat
> > tools
> > > as
> > > > > // follows:
> > > > > //*// fcst = {
> > > > > // field = [ { name = "PROB"; level = "A24";
> > > > > // prob = { name = "APCP";
thresh_lo =
> > 2.54; }
> > > > },
> > > > > // { name = "PROB"; level = "P850";
> > > > > // prob = { name = "TMP"; thresh_hi
= 273;
> > } }
> > > > ];
> > > > > // }*
> > > > > //
> > > > > // The example above selects two probabilistic
fields. In
> > both,
> > > > > "name"
> > > > > // is set to "PROB", the GRIB abbreviation for
> probabilities.
> > > The
> > > > > "level"
> > > > > // entry defines the level information (i.e. "A24"
for a
> > 24-hour
> > > > > // accumulation and "P850" for 850mb). The "prob"
dictionary
> > > > > defines the
> > > > > // event for which the probability is defined. The
> "thresh_lo"
> > > > > // (i.e. APCP > 2.54) and/or "thresh_hi" (i.e. TMP <
273)
> > > entries
> > > > > are
> > > > > // used to define the event threshold(s).
> > > > > //
> > > > > // Probability fields should contain values in the
range
> > > > > // [0, 1] or [0, 100]. However, when MET encounters a
> > > probability
> > > > > field
> > > > > // with a range [0, 100], it will automatically
rescale it
> to
> > be
> > > > > [0, 1]
> > > > > // before applying the probabilistic verification
methods.
> > > > > //
> > > > >
> > > > > I'm not sure if there are any syntax errors in that
reference code,
> > or
> > > if
> > > > > maybe I need to appeal to the *prob_as_scalar* flag as well.
Since
> I
> > > have
> > > > > never formally run MODE on a probabilistic field (I have run
MODE
> on
> > > > > probabilistic fields by just assuming it was a scalar
field), this
> is
> > > new
> > > > > to me.
> > > > >
> > > > > Jeff
> > > > >
> > > > > On Fri, Jan 8, 2021 at 3:48 PM John Halley Gotway via RT <
> > > > > met_help at ucar.edu>
> > > > > wrote:
> > > > >
> > > > > > Hi Jeff,
> > > > > >
> > > > > > I see that the config file parsing code is erroring out on
your
> > > config
> > > > > > file. I think you just need to terminate the "prob"
setting with
> a
> > > > > > semicolon, as shown below. In addition, I think you should
only
> be
> > > > using
> > > > > > the thresh_hi setting. That'll match the structure of the
GRIB
> > > records
> > > > > you
> > > > > > listed.
> > > > > >
> > > > > > field = [
> > > > > > { name = "PROB"; level = "L000"; prob = { name =
"HGT";
> > > thresh_hi
> > > > =
> > > > > > 152; }; },
> > > > > > { name = "PROB"; level = "L000"; prob = { name =
"HGT";
> > > thresh_hi
> > > > =
> > > > > > 305; }; },
> > > > > > { name = "PROB"; level = "L000"; prob = { name =
"HGT";
> > > thresh_hi
> > > > =
> > > > > > 915; }; }
> > > > > > ];
> > > > > >
> > > > > > Please let me know if that gets you any further and what
other
> > issues
> > > > pop
> > > > > > up.
> > > > > >
> > > > > > Thanks,
> > > > > > John Halley Gotway
> > > > > >
> > > > > > On Fri, Jan 8, 2021 at 2:43 PM Jeffrey Duda - NOAA
Affiliate via
> > RT <
> > > > > > met_help at ucar.edu> wrote:
> > > > > >
> > > > > > >
> > > > > > > Fri Jan 08 14:43:21 2021: Request 98171 was acted upon.
> > > > > > > Transaction: Ticket created by jeffrey.duda at noaa.gov
> > > > > > > Queue: met_help
> > > > > > > Subject: MODE - verifying a probabilistic field
with a
> > > > > deterministic
> > > > > > > field
> > > > > > > Owner: Nobody
> > > > > > > Requestors: jeffrey.duda at noaa.gov
> > > > > > > Status: new
> > > > > > > Ticket <URL:
> > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98171
> > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Hello,
> > > > > > > I want to verify a forecast probabilistic field that is
> comprised
> > > of
> > > > > > > separate fields with a single deterministic field.
> > > > > > >
> > > > > > > I have probabilities of ceiling height being below a
series of
> > > > > thresholds
> > > > > > > T1, T2, T3. The arrays are in GRIB2 files with this
structure
> > (from
> > > > > > > wgrib2):
> > > > > > >
> > > > > > > 9:10956329:d=2020121812:HGT:cloud ceiling:1 hour
fcst:prob
> > > <152:prob
> > > > > fcst
> > > > > > > 0/26:probability forecast
> > > > > > > 10:14766812:d=2020121812:HGT:cloud ceiling:1 hour
fcst:prob
> > > <305:prob
> > > > > > fcst
> > > > > > > 0/26:probability forecast
> > > > > > > 11:18577295:d=2020121812:HGT:cloud ceiling:1 hour
fcst:prob
> > > <915:prob
> > > > > > fcst
> > > > > > > 0/26:probability forecast
> > > > > > >
> > > > > > > These are not the only fields in this file with the
> abbreviation
> > > > "HGT",
> > > > > > so
> > > > > > > that is already one complication since the other HGT
fields are
> > > also
> > > > > > valid
> > > > > > > on only a certain custom level, so using "L0" for level
in the
> > > config
> > > > > > file
> > > > > > > confuses me. Regardless, here is my entry for fcst in
the
> config
> > > file
> > > > > > >
> > > > > > > fcst = {
> > > > > > > field = [
> > > > > > > { name = "PROB"; level = "L000"; prob = { name =
"HGT";
> > > > thresh_hi
> > > > > =
> > > > > > > 152; } },
> > > > > > > { name = "PROB"; level = "L000"; prob = { name =
"HGT";
> > > > thresh_hi
> > > > > =
> > > > > > > 305; thresh_lo = 152; } },
> > > > > > > { name = "PROB"; level = "L000"; prob = { name =
"HGT";
> > > > thresh_hi
> > > > > =
> > > > > > > 915; thresh_lo = 305; } } ];
> > > > > > >
> > > > > > > censor_thresh = [];
> > > > > > > censor_val = [];
> > > > > > > conv_radius = 60.0/grid_res; // in grid
squares
> > > > > > > conv_thresh =
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
[>0.0,>=10.0,>=20.0,>=30.0,>=40.0,>=50.0,>=60.0,>=70.0,>=80.0,>=90.0,>99.0];
> > > > > > > vld_thresh = 0.5;
> > > > > > > filter_attr_name = [];
> > > > > > > filter_attr_thresh = [];
> > > > > > > merge_thresh = >=1.25;
> > > > > > > merge_flag = NONE;
> > > > > > > }
> > > > > > >
> > > > > > > I'm not even sure it is okay to use both "thresh_hi" and
> > > "thresh_lo"
> > > > > in a
> > > > > > > field entry, but I do want to verify the probabilistic
field at
> > > > > multiple
> > > > > > > ceiling height thresholds, and this seems logical as to
how to
> do
> > > it.
> > > > > > >
> > > > > > > But when I run MODE I get these error messages:
> > > > > > > WARNING:
> > > > > > > WARNING: Dictionary::lookup_dictionary() -> dictionary
lookup
> > > failed
> > > > > for
> > > > > > > name "field"
> > > > > > > WARNING:
> > > > > > > ERROR :
> > > > > > > ERROR : Dictionary::lookup_dictionary() -> lookup
failed for
> > name
> > > > > > "field"
> > > > > > > ERROR :
> > > > > > >
> > > > > > > Config file is attached. I am using version 9.1.
> > > > > > >
> > > > > > > Jeff Duda
> > > > > > > --
> > > > > > > Research Scientist I, University of Colorado Boulder
> > > > > > > Cooperative Institute for Research in Environmental
Sciences
> > > > > > > NOAA/OAR/ESRL/Global Systems Laboratory/Assimilation and
> > > Verification
> > > > > > > Innovation Division
> > > > > > > Boulder, CO
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > > --
> > > > > Research Scientist I, University of Colorado Boulder
> > > > > Cooperative Institute for Research in Environmental Sciences
> > > > > NOAA/OAR/ESRL/Global Systems Laboratory/Assimilation and
> Verification
> > > > > Innovation Division
> > > > > Boulder, CO
> > > > >
> > > > >
> > > >
> > > >
> > >
> > > --
> > > Research Scientist I, University of Colorado Boulder
> > > Cooperative Institute for Research in Environmental Sciences
> > > NOAA/OAR/ESRL/Global Systems Laboratory/Assimilation and
Verification
> > > Innovation Division
> > > Boulder, CO
> > >
> > >
> >
> >
>
> --
> Research Scientist I, University of Colorado Boulder
> Cooperative Institute for Research in Environmental Sciences
> NOAA/OAR/ESRL/Global Systems Laboratory/Assimilation and
Verification
> Innovation Division
> Boulder, CO
>
>
------------------------------------------------
More information about the Met_help
mailing list