[Met_help] [rt.rap.ucar.edu #94848] History for PoP12 grid_stat_config

Tue Apr 7 09:44:01 MDT 2020

----------------------------------------------------------------
  Initial Request
----------------------------------------------------------------

Good afternoon MET team
Do you happen to have an example grid_stat_config file for PoP?  We're
struggling at the moment with PoP12 verification.  We can get scores in the
cnt, cts, and ctc files, but the pstd files only contains the header line.
We're trying to get the BSS.
For the forecasts, I have NDFD PoP12 forecasts.  The data look to be
correct (probabilities from 0 to 100).  For the analysis, I used
pcp_combine to add two 6-hr URMA QPF06 forecasts.  The resulting QPF12 file
appears to have valid data in mm.
When running with -v 0, I don't get any errors.  When running with -v 4, I
don't get any messages that suggest that there were any issues with the
data.
If it helps, the test script that I'm working with on venus is
/gpfs/dell3/mdl/mdlverif/noscrub/usr/John.L.Wagner/wolf/qpfvs/processing/test_po.sh.
Thanks
John

-- 
John Wagner
Verification Task Lead
COR Task Manager
NOAA/National Weather Service
Meteorological Development Laboratory
Digital Forecast Services Branch
SSMC2 Room 10106
Silver Spring, MD 20910
(301) 427-9471 (office)
(908) 902-4155 (cell/text)

----------------------------------------------------------------
  Complete Ticket History
----------------------------------------------------------------

Subject: PoP12 grid_stat_config
From: John Halley Gotway
Time: Mon Apr 06 13:17:00 2020

Hi John,

I see you have a question about verifying probabilities in Grid-Stat.
Some
output line types are appropriate for scalar forecasts (e.g. CNT, CTC,
and
CTS) while other line types are appropriate for probabilistic
forecasts
(e.g. PCT, PTSD, PRC, and PJC).  As you guessed, the trick is
configuring
Grid-Stat to indicate whether or not you're passing probabilities as
the
input.

The key is the "prob" setting in the Grid-Stat config file.  For
example,
here's a Grid-Stat config file that's used when you run "make test":
https://github.com/NCAR/MET/blob/master_v9.0/met/scripts/config/GridStatConfig_POP_12

fcst = {
field = [
{
name = "POP";
level = [ "L0" ];
prob = TRUE;
cat_thresh = [ ==0.1 ];
}
];
}
obs = {
field = [
{
name = "APCP_12";
level = [ "(*,*)" ];
cat_thresh = [ >=0.001 ];
}
];
}

Note that we've set "prob = TRUE;" to tell MET that this data should
be
processed as probabilities.  The "prob" setting can either be set to a
boolean or a dictionary, as described in this README file (
https://github.com/NCAR/MET/blob/master_v9.0/met/data/config/README).
Search for "prob" in there.

I haven't logged onto Venus to take a look at your specific case yet.
But
I can if you have more questions about it.

Thanks,
John

On Mon, Apr 6, 2020 at 12:17 PM John L Wagner - NOAA Federal via RT <
met_help at ucar.edu> wrote:

>
> Mon Apr 06 12:17:04 2020: Request 94848 was acted upon.
> Transaction: Ticket created by john.l.wagner at noaa.gov
>        Queue: met_help
>      Subject: PoP12 grid_stat_config
>        Owner: Nobody
>   Requestors: john.l.wagner at noaa.gov
>       Status: new
>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94848 >
>
>
> Good afternoon MET team
> Do you happen to have an example grid_stat_config file for PoP?
We're
> struggling at the moment with PoP12 verification.  We can get scores
in the
> cnt, cts, and ctc files, but the pstd files only contains the header
line.
> We're trying to get the BSS.
> For the forecasts, I have NDFD PoP12 forecasts.  The data look to be
> correct (probabilities from 0 to 100).  For the analysis, I used
> pcp_combine to add two 6-hr URMA QPF06 forecasts.  The resulting
QPF12 file
> appears to have valid data in mm.
> When running with -v 0, I don't get any errors.  When running with
-v 4, I
> don't get any messages that suggest that there were any issues with
the
> data.
> If it helps, the test script that I'm working with on venus is
>
>
/gpfs/dell3/mdl/mdlverif/noscrub/usr/John.L.Wagner/wolf/qpfvs/processing/test_po.sh.
> Thanks
> John
>
>
>
> --
> John Wagner
> Verification Task Lead
> COR Task Manager
> NOAA/National Weather Service
> Meteorological Development Laboratory
> Digital Forecast Services Branch
> SSMC2 Room 10106
> Silver Spring, MD 20910
> (301) 427-9471 (office)
> (908) 902-4155 (cell/text)
>
>

------------------------------------------------
Subject: PoP12 grid_stat_config
From: John L Wagner - NOAA Federal
Time: Mon Apr 06 13:50:53 2020

Thanks John!  That's what I was missing.
I'm not convinced that the categories are working correctly on my end,
but
I need to do some more testing.
Thanks
John

On Mon, Apr 6, 2020 at 3:17 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:

> Hi John,
>
> I see you have a question about verifying probabilities in Grid-
Stat.  Some
> output line types are appropriate for scalar forecasts (e.g. CNT,
CTC, and
> CTS) while other line types are appropriate for probabilistic
forecasts
> (e.g. PCT, PTSD, PRC, and PJC).  As you guessed, the trick is
configuring
> Grid-Stat to indicate whether or not you're passing probabilities as
the
> input.
>
> The key is the "prob" setting in the Grid-Stat config file.  For
example,
> here's a Grid-Stat config file that's used when you run "make test":
>
>
https://github.com/NCAR/MET/blob/master_v9.0/met/scripts/config/GridStatConfig_POP_12
>
>
> fcst = {
> field = [
> {
> name = "POP";
> level = [ "L0" ];
> prob = TRUE;
> cat_thresh = [ ==0.1 ];
> }
> ];
> }
> obs = {
> field = [
> {
> name = "APCP_12";
> level = [ "(*,*)" ];
> cat_thresh = [ >=0.001 ];
> }
> ];
> }
>
> Note that we've set "prob = TRUE;" to tell MET that this data should
be
> processed as probabilities.  The "prob" setting can either be set to
a
> boolean or a dictionary, as described in this README file (
>
https://github.com/NCAR/MET/blob/master_v9.0/met/data/config/README).
> Search for "prob" in there.
>
> I haven't logged onto Venus to take a look at your specific case
yet.  But
> I can if you have more questions about it.
>
> Thanks,
> John
>
> On Mon, Apr 6, 2020 at 12:17 PM John L Wagner - NOAA Federal via RT
<
> met_help at ucar.edu> wrote:
>
> >
> > Mon Apr 06 12:17:04 2020: Request 94848 was acted upon.
> > Transaction: Ticket created by john.l.wagner at noaa.gov
> >        Queue: met_help
> >      Subject: PoP12 grid_stat_config
> >        Owner: Nobody
> >   Requestors: john.l.wagner at noaa.gov
> >       Status: new
> >  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94848 >
> >
> >
> > Good afternoon MET team
> > Do you happen to have an example grid_stat_config file for PoP?
We're
> > struggling at the moment with PoP12 verification.  We can get
scores in
> the
> > cnt, cts, and ctc files, but the pstd files only contains the
header
> line.
> > We're trying to get the BSS.
> > For the forecasts, I have NDFD PoP12 forecasts.  The data look to
be
> > correct (probabilities from 0 to 100).  For the analysis, I used
> > pcp_combine to add two 6-hr URMA QPF06 forecasts.  The resulting
QPF12
> file
> > appears to have valid data in mm.
> > When running with -v 0, I don't get any errors.  When running with
-v 4,
> I
> > don't get any messages that suggest that there were any issues
with the
> > data.
> > If it helps, the test script that I'm working with on venus is
> >
> >
>
/gpfs/dell3/mdl/mdlverif/noscrub/usr/John.L.Wagner/wolf/qpfvs/processing/test_po.sh.
> > Thanks
> > John
> >
> >
> >
> > --
> > John Wagner
> > Verification Task Lead
> > COR Task Manager
> > NOAA/National Weather Service
> > Meteorological Development Laboratory
> > Digital Forecast Services Branch
> > SSMC2 Room 10106
> > Silver Spring, MD 20910
> > (301) 427-9471 (office)
> > (908) 902-4155 (cell/text)
> >
> >
>
>

--
John Wagner
Verification Task Lead
COR Task Manager
NOAA/National Weather Service
Meteorological Development Laboratory
Digital Forecast Services Branch
SSMC2 Room 10106
Silver Spring, MD 20910
(301) 427-9471 (office)
(908) 902-4155 (cell/text)

------------------------------------------------