[Met_help] [rt.rap.ucar.edu #78506] History for plot_data_plane error - Needs more arguments for variable
John Halley Gotway via RT
met_help at ucar.edu
Wed Nov 2 11:12:43 MDT 2016
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hi,
I am trying to use the plot_data_plane tool to visualize the observation
field of 6-hour accumulated precipitation (CCPA) and my corresponding
forecast field of probabilities of 6-hour precipitation > 12.7 mm. Both
fields are on the same grid. I am using a pairs.nc file that I generated
from MET's gridstat tool. I am able to plot the observation field when I
use the command:
plot_data_plane grid_stat_PROB_12.7_f12_120000L_20161006_120000V_pairs.nc
obs.ps 'name = "OBS_APCP_06_A06_FULL" ; level = "A06" ;' -plot_range 0 12.7
-v 4
However, when I try a similar command on the forecast field (probabilities
ranging from 0 to 1):
plot_data_plane grid_stat_PROB_12.7_f12_120000L_20161006_120000V_pairs.nc
fcst.ps 'name = "FCST_PROB\(APCP_06\>12.700\)_A06_FULL" ; level = "A06" ;'
-v 4
I get the following error:
ERROR : MetNcFile::data(NcVar *, const LongArray &, DataPlane &) -> needed
2 arguments for variable FCST_PROB(APCP_06>12.700)_A06_FULL, got 1
I think the error has to do with how I define the forecast field in the
config file I am using. In addition to name and level, I have a third line
for prob which contains multiple arguments. I tried to modify the
plot_data_plane command but I kept getting the same error. I've attached
the config file that I used to run gridstat. The pairs.nc file that I used
which was generated from gridstat was too large to attach to this email but
it is located on Theia in the following directory:
/scratch4/NCEPDEV/meso/noscrub/Benjamin.Blake
Any help would be greatly appreciated!
Thanks,
Ben Blake
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: plot_data_plane error - Needs more arguments for variable
From: John Halley Gotway
Time: Thu Oct 27 15:01:08 2016
Ben,
Thanks for pointing out this issue. I ran the code in debug mode and
see
the source of the problem.
It's having that extra set of parenthesis that's confusing the parsing
logic. I'll need to work up a bugfix and post a patch file.
For a short-term workaround, you could do the following:
module load nco
# Rename the variables to get rid of parenthesis
ncrename -v
FCST_PROB\(APCP_06\>12.700\)_A06_FULL,FCST_PROB_APCP_06\>12.700_A06_FULL
grid_stat_PROB_12.7_f12_120000L_20161006_120000V_pairs.nc -o
grid_stat_PROB_12.7_f12_120000L_20161006_120000V_pairs_rename.nc
# Plot the variable without parenthesis
/scratch4/BMC/dtc/MET/met-5.2/bin/plot_data_plane
grid_stat_PROB_12.7_f12_120000L_20161006_120000V_pairs_rename.nc
fcst.ps
'name="FCST_PROB_APCP_06>12.700_A06_FULL"; level="(*,*)";'
I'll let you know when I have a patch to fix this directly.
Thanks,
John Halley Gotway
On Thu, Oct 27, 2016 at 1:32 PM, Benjamin Blake - NOAA Affiliate via
RT <
met_help at ucar.edu> wrote:
>
> Thu Oct 27 13:32:31 2016: Request 78506 was acted upon.
> Transaction: Ticket created by benjamin.blake at noaa.gov
> Queue: met_help
> Subject: plot_data_plane error - Needs more arguments for
variable
> Owner: Nobody
> Requestors: benjamin.blake at noaa.gov
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=78506 >
>
>
> Hi,
>
> I am trying to use the plot_data_plane tool to visualize the
observation
> field of 6-hour accumulated precipitation (CCPA) and my
corresponding
> forecast field of probabilities of 6-hour precipitation > 12.7 mm.
Both
> fields are on the same grid. I am using a pairs.nc file that I
generated
> from MET's gridstat tool. I am able to plot the observation field
when I
> use the command:
>
> plot_data_plane
grid_stat_PROB_12.7_f12_120000L_20161006_120000V_pairs.nc
> obs.ps 'name = "OBS_APCP_06_A06_FULL" ; level = "A06" ;' -plot_range
0
> 12.7
> -v 4
>
> However, when I try a similar command on the forecast field
(probabilities
> ranging from 0 to 1):
>
> plot_data_plane
grid_stat_PROB_12.7_f12_120000L_20161006_120000V_pairs.nc
> fcst.ps 'name = "FCST_PROB\(APCP_06\>12.700\)_A06_FULL" ; level =
"A06" ;'
> -v 4
>
> I get the following error:
> ERROR : MetNcFile::data(NcVar *, const LongArray &, DataPlane &) ->
needed
> 2 arguments for variable FCST_PROB(APCP_06>12.700)_A06_FULL, got 1
>
> I think the error has to do with how I define the forecast field in
the
> config file I am using. In addition to name and level, I have a
third line
> for prob which contains multiple arguments. I tried to modify the
> plot_data_plane command but I kept getting the same error. I've
attached
> the config file that I used to run gridstat. The pairs.nc file that
I
> used
> which was generated from gridstat was too large to attach to this
email but
> it is located on Theia in the following directory:
> /scratch4/NCEPDEV/meso/noscrub/Benjamin.Blake
>
> Any help would be greatly appreciated!
>
> Thanks,
> Ben Blake
>
>
------------------------------------------------
Subject: plot_data_plane error - Needs more arguments for variable
From: Benjamin Blake - NOAA Affiliate
Time: Fri Oct 28 06:28:08 2016
Hi John,
Yep, that was indeed the issue. I was able to plot the probabilities.
Thanks for looking into it!
I am just starting to look into why the values on my reliability
diagram
that I sent you several weeks ago are so low, and you suggested I use
the
plot_data_plane tool to visualize the input that is being used to
generate
the statistics. I've attached the forecast and observed fields as
well as
the pjc.txt file which contains the values for base rate. I know that
this
may not necessarily be the best probability forecast, as a large area
of >
12.7 mm of precip fell over the central US that corresponds to low
probabilities. However, if you look at the pjc.txt file, the values
for
base rate are all really low, indicating a huge amount of
overforecasting.
Do you know if the areas where there is no observation data (like over
the
oceans) are being taken into account with gridstat? That would
explain why
the base rate values are so low.
Thanks again!
Ben
On Thu, Oct 27, 2016 at 5:01 PM, John Halley Gotway via RT <
met_help at ucar.edu> wrote:
> Ben,
>
> Thanks for pointing out this issue. I ran the code in debug mode
and see
> the source of the problem.
>
> It's having that extra set of parenthesis that's confusing the
parsing
> logic. I'll need to work up a bugfix and post a patch file.
>
> For a short-term workaround, you could do the following:
>
> module load nco
>
> # Rename the variables to get rid of parenthesis
>
> ncrename -v
>
FCST_PROB\(APCP_06\>12.700\)_A06_FULL,FCST_PROB_APCP_06\>12.700_A06_FULL
> grid_stat_PROB_12.7_f12_120000L_20161006_120000V_pairs.nc -o
> grid_stat_PROB_12.7_f12_120000L_20161006_120000V_pairs_rename.nc
>
> # Plot the variable without parenthesis
>
> /scratch4/BMC/dtc/MET/met-5.2/bin/plot_data_plane
> grid_stat_PROB_12.7_f12_120000L_20161006_120000V_pairs_rename.nc
fcst.ps
> 'name="FCST_PROB_APCP_06>12.700_A06_FULL"; level="(*,*)";'
>
> I'll let you know when I have a patch to fix this directly.
>
> Thanks,
> John Halley Gotway
>
> On Thu, Oct 27, 2016 at 1:32 PM, Benjamin Blake - NOAA Affiliate via
RT <
> met_help at ucar.edu> wrote:
>
> >
> > Thu Oct 27 13:32:31 2016: Request 78506 was acted upon.
> > Transaction: Ticket created by benjamin.blake at noaa.gov
> > Queue: met_help
> > Subject: plot_data_plane error - Needs more arguments for
variable
> > Owner: Nobody
> > Requestors: benjamin.blake at noaa.gov
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=78506 >
> >
> >
> > Hi,
> >
> > I am trying to use the plot_data_plane tool to visualize the
observation
> > field of 6-hour accumulated precipitation (CCPA) and my
corresponding
> > forecast field of probabilities of 6-hour precipitation > 12.7 mm.
Both
> > fields are on the same grid. I am using a pairs.nc file that I
> generated
> > from MET's gridstat tool. I am able to plot the observation field
when I
> > use the command:
> >
> > plot_data_plane grid_stat_PROB_12.7_f12_120000L_20161006_120000V_
> pairs.nc
> > obs.ps 'name = "OBS_APCP_06_A06_FULL" ; level = "A06" ;'
-plot_range 0
> > 12.7
> > -v 4
> >
> > However, when I try a similar command on the forecast field
> (probabilities
> > ranging from 0 to 1):
> >
> > plot_data_plane grid_stat_PROB_12.7_f12_120000L_20161006_120000V_
> pairs.nc
> > fcst.ps 'name = "FCST_PROB\(APCP_06\>12.700\)_A06_FULL" ; level =
"A06"
> ;'
> > -v 4
> >
> > I get the following error:
> > ERROR : MetNcFile::data(NcVar *, const LongArray &, DataPlane &)
->
> needed
> > 2 arguments for variable FCST_PROB(APCP_06>12.700)_A06_FULL, got 1
> >
> > I think the error has to do with how I define the forecast field
in the
> > config file I am using. In addition to name and level, I have a
third
> line
> > for prob which contains multiple arguments. I tried to modify the
> > plot_data_plane command but I kept getting the same error. I've
attached
> > the config file that I used to run gridstat. The pairs.nc file
that I
> > used
> > which was generated from gridstat was too large to attach to this
email
> but
> > it is located on Theia in the following directory:
> > /scratch4/NCEPDEV/meso/noscrub/Benjamin.Blake
> >
> > Any help would be greatly appreciated!
> >
> > Thanks,
> > Ben Blake
> >
> >
>
>
------------------------------------------------
Subject: plot_data_plane error - Needs more arguments for variable
From: Benjamin Blake - NOAA Affiliate
Time: Fri Oct 28 06:28:08 2016
VERSION MODEL FCST_LEAD FCST_VALID_BEG FCST_VALID_END OBS_LEAD
OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_LEV OBS_VAR
OBS_LEV OBTYPE VX_MASK INTERP_MTHD INTERP_PNTS FCST_THRESH
OBS_THRESH COV_THRESH ALPHA LINE_TYPE TOTAL N_THRESH
THRESH_1 OY_TP_1 ON_TP_1 CALIBRATION_1 REFINEMENT_1 LIKELIHOOD_1
BASER_1
THRESH_2 OY_TP_2 ON_TP_2 CALIBRATION_2 REFINEMENT_2 LIKELIHOOD_2
BASER_2
THRESH_3 OY_TP_3 ON_TP_3 CALIBRATION_3 REFINEMENT_3 LIKELIHOOD_3
BASER_3
THRESH_4 OY_TP_4 ON_TP_4 CALIBRATION_4 REFINEMENT_4 LIKELIHOOD_4
BASER_4
THRESH_5 OY_TP_5 ON_TP_5 CALIBRATION_5 REFINEMENT_5 LIKELIHOOD_5
BASER_5
THRESH_6 OY_TP_6 ON_TP_6 CALIBRATION_6 REFINEMENT_6 LIKELIHOOD_6
BASER_6
THRESH_7 OY_TP_7 ON_TP_7 CALIBRATION_7 REFINEMENT_7 LIKELIHOOD_7
BASER_7
THRESH_8 OY_TP_8 ON_TP_8 CALIBRATION_8 REFINEMENT_8 LIKELIHOOD_8
BASER_8
THRESH_9 OY_TP_9 ON_TP_9 CALIBRATION_9 REFINEMENT_9 LIKELIHOOD_9
BASER_9
THRESH_10 OY_TP_10 ON_TP_10 CALIBRATION_10 REFINEMENT_10
LIKELIHOOD_10 BASER_10
THRESH_11
V5.1 HREF-GSD 180000 20161006_180000 20161006_180000 060000
20161006_180000 20161006_180000 PROB(APCP_06>12.700) A06 APCP_06
A06 ANALYS FULL NEAREST 1
>=0.0,>=0.1,>=0.2,>=0.3,>=0.4,>=0.5,>=0.6,>=0.7,>=0.8,>=0.9,>=1.0
>=12.7 NA NA PJC 409008 11
0 4.8899e-06 0.82925 5.8967e-06 0.82925 0.00052687 5.8967e-
06
0.1 0.0010856 0.045559 0.023273 0.046645 0.11697
0.023273
0.2 0.0025819 0.028175 0.083943 0.030757 0.27819
0.083943
0.3 0.0019877 0.022755 0.080336 0.024743 0.21417
0.080336
0.4 0.0011662 0.024334 0.045733 0.025501 0.12566
0.045733
0.5 0.00066258 0.01663 0.038315 0.017293 0.071391
0.038315
0.6 0.00030317 0.012247 0.024157 0.01255 0.032666
0.024157
0.7 0.00086795 0.010215 0.078315 0.011083 0.093519
0.078315
0.8 0.00059168 0.0015574 0.27531 0.0021491 0.063751
0.27531
0.9 2.9339e-05 0 1 2.9339e-05 0.0031612
1
1
------------------------------------------------
Subject: plot_data_plane error - Needs more arguments for variable
From: John Halley Gotway
Time: Fri Oct 28 11:04:00 2016
Ben,
I clearly see the cause of your original issue with plot_data_plane in
the
code... but the best solution is less obvious. Fixing this the
"right" way
will be somewhat of a large change and will affect many files. While
I'm
fine doing that in the development version for the 6.0 release, I
hesitate
to make that large of a change as a bugfix. Do you have any
objections to
me fixing this in 6.0 but not posting a 5.2 bugfix?
On to your question... no, Grid-Stat does not use points where either
the
forecast or observation contains bad data values. When doing
verification
of continuous (i.e. non-probabilistic) fields, the NetCDF matched
pairs
file contains output variables for the forecast (FCST), observation
(OBS),
and their difference (DIFF). Looking at the difference field, it
becomes
obvious that the bad data values over the ocean are skipped. But for
probabilistic verification, the raw values aren't subtracted directly,
so
we don't write out that DIFF field.
Another thing to note is that the dimension of your grid is 1025 x
1473 =
1,509,825 grid points but the TOTAL column in the MET output lists
only
409,008 matched pairs. So only 27% of the grid points contain valid
forecast/observation data and are used in the verification.
Rather than looking at the full field of observation values, here's a
trick
for how you could threshold and then plot them:
/scratch4/BMC/dtc/MET/met-5.2/bin/gen_vx_mask \
grid_stat_PROB_12.7_f12_120000L_20161006_120000V_pairs.nc \
grid_stat_PROB_12.7_f12_120000L_20161006_120000V_pairs.nc \
obs_ge12.7.nc \
-type data \
-mask_field 'name="OBS_APCP_06_A06_FULL"; level="(*,*)";' \
-thresh ge12.7 \
-name obs_ge12.7
/scratch4/BMC/dtc/MET/met-5.2/bin/plot_data_plane \
obs_ge12.7.nc obs_ge12.7.ps 'name="obs_ge12.7"; level="(*,*)";'
The resulting image is attached. The areas of pink show values
>=12.7. I
opened that NetCDF file in R and found that there are 4446 grid points
that
are "pink". 4446 / 409008 is about 0.01. So the event occurs only
about
1% of the time in this example.
Looking at the OY_TP_i and ON_TP_i values in the data file you sent,
roughly speaking those numbers look pretty consistent with a 1% base
rate
over the entire field.
One option you could try would be lowering the observation threshold
(try
>=6.35 or >=2.54) just to see how the scores change. I realize that
the
probability is defined for >=12.7 but you could play around with the
observation threshold to better understand the performance. That
should be
as simple as adding more entries to the observation categorical
threshold
setting in the config file:
cat_thresh = [ >=12.7, >=6.35, >=2.54, >=0.254 ];
Hope that helps.
Thanks,
John
On Fri, Oct 28, 2016 at 6:28 AM, Benjamin Blake - NOAA Affiliate via
RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=78506 >
>
> Hi John,
>
> Yep, that was indeed the issue. I was able to plot the
probabilities.
> Thanks for looking into it!
>
> I am just starting to look into why the values on my reliability
diagram
> that I sent you several weeks ago are so low, and you suggested I
use the
> plot_data_plane tool to visualize the input that is being used to
generate
> the statistics. I've attached the forecast and observed fields as
well as
> the pjc.txt file which contains the values for base rate. I know
that this
> may not necessarily be the best probability forecast, as a large
area of >
> 12.7 mm of precip fell over the central US that corresponds to low
> probabilities. However, if you look at the pjc.txt file, the values
for
> base rate are all really low, indicating a huge amount of
overforecasting.
>
> Do you know if the areas where there is no observation data (like
over the
> oceans) are being taken into account with gridstat? That would
explain why
> the base rate values are so low.
>
> Thanks again!
> Ben
>
> On Thu, Oct 27, 2016 at 5:01 PM, John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
> > Ben,
> >
> > Thanks for pointing out this issue. I ran the code in debug mode
and see
> > the source of the problem.
> >
> > It's having that extra set of parenthesis that's confusing the
parsing
> > logic. I'll need to work up a bugfix and post a patch file.
> >
> > For a short-term workaround, you could do the following:
> >
> > module load nco
> >
> > # Rename the variables to get rid of parenthesis
> >
> > ncrename -v
> >
FCST_PROB\(APCP_06\>12.700\)_A06_FULL,FCST_PROB_APCP_06\>12.700_A06_FULL
> > grid_stat_PROB_12.7_f12_120000L_20161006_120000V_pairs.nc -o
> > grid_stat_PROB_12.7_f12_120000L_20161006_120000V_pairs_rename.nc
> >
> > # Plot the variable without parenthesis
> >
> > /scratch4/BMC/dtc/MET/met-5.2/bin/plot_data_plane
> > grid_stat_PROB_12.7_f12_120000L_20161006_120000V_pairs_rename.nc
fcst.ps
> > 'name="FCST_PROB_APCP_06>12.700_A06_FULL"; level="(*,*)";'
> >
> > I'll let you know when I have a patch to fix this directly.
> >
> > Thanks,
> > John Halley Gotway
> >
> > On Thu, Oct 27, 2016 at 1:32 PM, Benjamin Blake - NOAA Affiliate
via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > Thu Oct 27 13:32:31 2016: Request 78506 was acted upon.
> > > Transaction: Ticket created by benjamin.blake at noaa.gov
> > > Queue: met_help
> > > Subject: plot_data_plane error - Needs more arguments for
variable
> > > Owner: Nobody
> > > Requestors: benjamin.blake at noaa.gov
> > > Status: new
> > > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=78506
> >
> > >
> > >
> > > Hi,
> > >
> > > I am trying to use the plot_data_plane tool to visualize the
> observation
> > > field of 6-hour accumulated precipitation (CCPA) and my
corresponding
> > > forecast field of probabilities of 6-hour precipitation > 12.7
mm.
> Both
> > > fields are on the same grid. I am using a pairs.nc file that I
> > generated
> > > from MET's gridstat tool. I am able to plot the observation
field
> when I
> > > use the command:
> > >
> > > plot_data_plane
grid_stat_PROB_12.7_f12_120000L_20161006_120000V_
> > pairs.nc
> > > obs.ps 'name = "OBS_APCP_06_A06_FULL" ; level = "A06" ;'
-plot_range 0
> > > 12.7
> > > -v 4
> > >
> > > However, when I try a similar command on the forecast field
> > (probabilities
> > > ranging from 0 to 1):
> > >
> > > plot_data_plane
grid_stat_PROB_12.7_f12_120000L_20161006_120000V_
> > pairs.nc
> > > fcst.ps 'name = "FCST_PROB\(APCP_06\>12.700\)_A06_FULL" ; level
=
> "A06"
> > ;'
> > > -v 4
> > >
> > > I get the following error:
> > > ERROR : MetNcFile::data(NcVar *, const LongArray &, DataPlane
&) ->
> > needed
> > > 2 arguments for variable FCST_PROB(APCP_06>12.700)_A06_FULL, got
1
> > >
> > > I think the error has to do with how I define the forecast field
in the
> > > config file I am using. In addition to name and level, I have a
third
> > line
> > > for prob which contains multiple arguments. I tried to modify
the
> > > plot_data_plane command but I kept getting the same error. I've
> attached
> > > the config file that I used to run gridstat. The pairs.nc file
that I
> > > used
> > > which was generated from gridstat was too large to attach to
this email
> > but
> > > it is located on Theia in the following directory:
> > > /scratch4/NCEPDEV/meso/noscrub/Benjamin.Blake
> > >
> > > Any help would be greatly appreciated!
> > >
> > > Thanks,
> > > Ben Blake
> > >
> > >
> >
> >
>
>
------------------------------------------------
Subject: plot_data_plane error - Needs more arguments for variable
From: John Halley Gotway
Time: Fri Oct 28 11:36:44 2016
Ben,
When you run gen_vx_mask, try increasing the verbosity level to 3:
/scratch4/BMC/dtc/MET/met-5.2/bin/gen_vx_mask \
grid_stat_PROB_12.7_f12_120000L_20161006_120000V_pairs.nc \
grid_stat_PROB_12.7_f12_120000L_20161006_120000V_pairs.nc \
obs_ge12.7.nc \
-type data \
-mask_field 'name="OBS_APCP_06_A06_FULL"; level="(*,*)";' \
-thresh ge12.7 \
-name obs_ge12.7 \
-v 3
At verbosity level 3, it tells you the number of points included in
the
mask (i.e. the number that met the threshold):
DEBUG 3: Data Masking: 4446 of 1509825 points inside
Thanks,
John
On Fri, Oct 28, 2016 at 11:03 AM, John Halley Gotway <johnhg at ucar.edu>
wrote:
> Ben,
>
> I clearly see the cause of your original issue with plot_data_plane
in the
> code... but the best solution is less obvious. Fixing this the
"right" way
> will be somewhat of a large change and will affect many files.
While I'm
> fine doing that in the development version for the 6.0 release, I
hesitate
> to make that large of a change as a bugfix. Do you have any
objections to
> me fixing this in 6.0 but not posting a 5.2 bugfix?
>
> On to your question... no, Grid-Stat does not use points where
either the
> forecast or observation contains bad data values. When doing
verification
> of continuous (i.e. non-probabilistic) fields, the NetCDF matched
pairs
> file contains output variables for the forecast (FCST), observation
(OBS),
> and their difference (DIFF). Looking at the difference field, it
becomes
> obvious that the bad data values over the ocean are skipped. But
for
> probabilistic verification, the raw values aren't subtracted
directly, so
> we don't write out that DIFF field.
>
> Another thing to note is that the dimension of your grid is 1025 x
1473 =
> 1,509,825 grid points but the TOTAL column in the MET output lists
only
> 409,008 matched pairs. So only 27% of the grid points contain valid
> forecast/observation data and are used in the verification.
>
> Rather than looking at the full field of observation values, here's
a
> trick for how you could threshold and then plot them:
>
> /scratch4/BMC/dtc/MET/met-5.2/bin/gen_vx_mask \
> grid_stat_PROB_12.7_f12_120000L_20161006_120000V_pairs.nc \
> grid_stat_PROB_12.7_f12_120000L_20161006_120000V_pairs.nc \
> obs_ge12.7.nc \
> -type data \
> -mask_field 'name="OBS_APCP_06_A06_FULL"; level="(*,*)";' \
> -thresh ge12.7 \
> -name obs_ge12.7
>
> /scratch4/BMC/dtc/MET/met-5.2/bin/plot_data_plane \
> obs_ge12.7.nc obs_ge12.7.ps 'name="obs_ge12.7"; level="(*,*)";'
>
> The resulting image is attached. The areas of pink show values
>=12.7. I
> opened that NetCDF file in R and found that there are 4446 grid
points that
> are "pink". 4446 / 409008 is about 0.01. So the event occurs only
about
> 1% of the time in this example.
>
> Looking at the OY_TP_i and ON_TP_i values in the data file you sent,
> roughly speaking those numbers look pretty consistent with a 1% base
rate
> over the entire field.
>
> One option you could try would be lowering the observation threshold
(try
> >=6.35 or >=2.54) just to see how the scores change. I realize that
the
> probability is defined for >=12.7 but you could play around with the
> observation threshold to better understand the performance. That
should be
> as simple as adding more entries to the observation categorical
threshold
> setting in the config file:
> cat_thresh = [ >=12.7, >=6.35, >=2.54, >=0.254 ];
>
> Hope that helps.
>
> Thanks,
> John
>
>
>
>
>
>
> On Fri, Oct 28, 2016 at 6:28 AM, Benjamin Blake - NOAA Affiliate via
RT <
> met_help at ucar.edu> wrote:
>
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=78506 >
>>
>> Hi John,
>>
>> Yep, that was indeed the issue. I was able to plot the
probabilities.
>> Thanks for looking into it!
>>
>> I am just starting to look into why the values on my reliability
diagram
>> that I sent you several weeks ago are so low, and you suggested I
use the
>> plot_data_plane tool to visualize the input that is being used to
generate
>> the statistics. I've attached the forecast and observed fields as
well as
>> the pjc.txt file which contains the values for base rate. I know
that
>> this
>> may not necessarily be the best probability forecast, as a large
area of >
>> 12.7 mm of precip fell over the central US that corresponds to low
>> probabilities. However, if you look at the pjc.txt file, the
values for
>> base rate are all really low, indicating a huge amount of
overforecasting.
>>
>> Do you know if the areas where there is no observation data (like
over the
>> oceans) are being taken into account with gridstat? That would
explain
>> why
>> the base rate values are so low.
>>
>> Thanks again!
>> Ben
>>
>> On Thu, Oct 27, 2016 at 5:01 PM, John Halley Gotway via RT <
>> met_help at ucar.edu> wrote:
>>
>> > Ben,
>> >
>> > Thanks for pointing out this issue. I ran the code in debug mode
and
>> see
>> > the source of the problem.
>> >
>> > It's having that extra set of parenthesis that's confusing the
parsing
>> > logic. I'll need to work up a bugfix and post a patch file.
>> >
>> > For a short-term workaround, you could do the following:
>> >
>> > module load nco
>> >
>> > # Rename the variables to get rid of parenthesis
>> >
>> > ncrename -v
>> > FCST_PROB\(APCP_06\>12.700\)_A06_FULL,FCST_PROB_APCP_06\>12.
>> 700_A06_FULL
>> > grid_stat_PROB_12.7_f12_120000L_20161006_120000V_pairs.nc -o
>> > grid_stat_PROB_12.7_f12_120000L_20161006_120000V_pairs_rename.nc
>> >
>> > # Plot the variable without parenthesis
>> >
>> > /scratch4/BMC/dtc/MET/met-5.2/bin/plot_data_plane
>> > grid_stat_PROB_12.7_f12_120000L_20161006_120000V_pairs_rename.nc
>> fcst.ps
>> > 'name="FCST_PROB_APCP_06>12.700_A06_FULL"; level="(*,*)";'
>> >
>> > I'll let you know when I have a patch to fix this directly.
>> >
>> > Thanks,
>> > John Halley Gotway
>> >
>> > On Thu, Oct 27, 2016 at 1:32 PM, Benjamin Blake - NOAA Affiliate
via RT
>> <
>> > met_help at ucar.edu> wrote:
>> >
>> > >
>> > > Thu Oct 27 13:32:31 2016: Request 78506 was acted upon.
>> > > Transaction: Ticket created by benjamin.blake at noaa.gov
>> > > Queue: met_help
>> > > Subject: plot_data_plane error - Needs more arguments for
>> variable
>> > > Owner: Nobody
>> > > Requestors: benjamin.blake at noaa.gov
>> > > Status: new
>> > > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=78506
>> >
>> > >
>> > >
>> > > Hi,
>> > >
>> > > I am trying to use the plot_data_plane tool to visualize the
>> observation
>> > > field of 6-hour accumulated precipitation (CCPA) and my
corresponding
>> > > forecast field of probabilities of 6-hour precipitation > 12.7
mm.
>> Both
>> > > fields are on the same grid. I am using a pairs.nc file that I
>> > generated
>> > > from MET's gridstat tool. I am able to plot the observation
field
>> when I
>> > > use the command:
>> > >
>> > > plot_data_plane
grid_stat_PROB_12.7_f12_120000L_20161006_120000V_
>> > pairs.nc
>> > > obs.ps 'name = "OBS_APCP_06_A06_FULL" ; level = "A06" ;'
-plot_range
>> 0
>> > > 12.7
>> > > -v 4
>> > >
>> > > However, when I try a similar command on the forecast field
>> > (probabilities
>> > > ranging from 0 to 1):
>> > >
>> > > plot_data_plane
grid_stat_PROB_12.7_f12_120000L_20161006_120000V_
>> > pairs.nc
>> > > fcst.ps 'name = "FCST_PROB\(APCP_06\>12.700\)_A06_FULL" ; level
=
>> "A06"
>> > ;'
>> > > -v 4
>> > >
>> > > I get the following error:
>> > > ERROR : MetNcFile::data(NcVar *, const LongArray &, DataPlane
&) ->
>> > needed
>> > > 2 arguments for variable FCST_PROB(APCP_06>12.700)_A06_FULL,
got 1
>> > >
>> > > I think the error has to do with how I define the forecast
field in
>> the
>> > > config file I am using. In addition to name and level, I have
a third
>> > line
>> > > for prob which contains multiple arguments. I tried to modify
the
>> > > plot_data_plane command but I kept getting the same error.
I've
>> attached
>> > > the config file that I used to run gridstat. The pairs.nc file
that
>> I
>> > > used
>> > > which was generated from gridstat was too large to attach to
this
>> email
>> > but
>> > > it is located on Theia in the following directory:
>> > > /scratch4/NCEPDEV/meso/noscrub/Benjamin.Blake
>> > >
>> > > Any help would be greatly appreciated!
>> > >
>> > > Thanks,
>> > > Ben Blake
>> > >
>> > >
>> >
>> >
>>
>>
>
------------------------------------------------
Subject: plot_data_plane error - Needs more arguments for variable
From: Benjamin Blake - NOAA Affiliate
Time: Fri Oct 28 12:10:03 2016
Hi John,
I do not have any objections with fixing that in version 6.0.
It's definitely good to know that grid stat is not using points where
the
observation field has no data - thanks for confirming.
I will try experimenting with lowering the observation threshold to
see how
the scores change, since that may help me better understand the
performance.
Thanks,
Ben
On Fri, Oct 28, 2016 at 1:36 PM, John Halley Gotway via RT <
met_help at ucar.edu> wrote:
> Ben,
>
> When you run gen_vx_mask, try increasing the verbosity level to 3:
>
> /scratch4/BMC/dtc/MET/met-5.2/bin/gen_vx_mask \
> grid_stat_PROB_12.7_f12_120000L_20161006_120000V_pairs.nc \
> grid_stat_PROB_12.7_f12_120000L_20161006_120000V_pairs.nc \
> obs_ge12.7.nc \
> -type data \
> -mask_field 'name="OBS_APCP_06_A06_FULL"; level="(*,*)";' \
> -thresh ge12.7 \
> -name obs_ge12.7 \
> -v 3
>
> At verbosity level 3, it tells you the number of points included in
the
> mask (i.e. the number that met the threshold):
>
> DEBUG 3: Data Masking: 4446 of 1509825 points inside
>
> Thanks,
> John
>
> On Fri, Oct 28, 2016 at 11:03 AM, John Halley Gotway
<johnhg at ucar.edu>
> wrote:
>
> > Ben,
> >
> > I clearly see the cause of your original issue with
plot_data_plane in
> the
> > code... but the best solution is less obvious. Fixing this the
"right"
> way
> > will be somewhat of a large change and will affect many files.
While I'm
> > fine doing that in the development version for the 6.0 release, I
> hesitate
> > to make that large of a change as a bugfix. Do you have any
objections
> to
> > me fixing this in 6.0 but not posting a 5.2 bugfix?
> >
> > On to your question... no, Grid-Stat does not use points where
either the
> > forecast or observation contains bad data values. When doing
> verification
> > of continuous (i.e. non-probabilistic) fields, the NetCDF matched
pairs
> > file contains output variables for the forecast (FCST),
observation
> (OBS),
> > and their difference (DIFF). Looking at the difference field, it
becomes
> > obvious that the bad data values over the ocean are skipped. But
for
> > probabilistic verification, the raw values aren't subtracted
directly, so
> > we don't write out that DIFF field.
> >
> > Another thing to note is that the dimension of your grid is 1025 x
1473 =
> > 1,509,825 grid points but the TOTAL column in the MET output lists
only
> > 409,008 matched pairs. So only 27% of the grid points contain
valid
> > forecast/observation data and are used in the verification.
> >
> > Rather than looking at the full field of observation values,
here's a
> > trick for how you could threshold and then plot them:
> >
> > /scratch4/BMC/dtc/MET/met-5.2/bin/gen_vx_mask \
> > grid_stat_PROB_12.7_f12_120000L_20161006_120000V_pairs.nc \
> > grid_stat_PROB_12.7_f12_120000L_20161006_120000V_pairs.nc \
> > obs_ge12.7.nc \
> > -type data \
> > -mask_field 'name="OBS_APCP_06_A06_FULL"; level="(*,*)";' \
> > -thresh ge12.7 \
> > -name obs_ge12.7
> >
> > /scratch4/BMC/dtc/MET/met-5.2/bin/plot_data_plane \
> > obs_ge12.7.nc obs_ge12.7.ps 'name="obs_ge12.7"; level="(*,*)";'
> >
> > The resulting image is attached. The areas of pink show values
>=12.7.
> I
> > opened that NetCDF file in R and found that there are 4446 grid
points
> that
> > are "pink". 4446 / 409008 is about 0.01. So the event occurs
only about
> > 1% of the time in this example.
> >
> > Looking at the OY_TP_i and ON_TP_i values in the data file you
sent,
> > roughly speaking those numbers look pretty consistent with a 1%
base rate
> > over the entire field.
> >
> > One option you could try would be lowering the observation
threshold (try
> > >=6.35 or >=2.54) just to see how the scores change. I realize
that the
> > probability is defined for >=12.7 but you could play around with
the
> > observation threshold to better understand the performance. That
should
> be
> > as simple as adding more entries to the observation categorical
threshold
> > setting in the config file:
> > cat_thresh = [ >=12.7, >=6.35, >=2.54, >=0.254 ];
> >
> > Hope that helps.
> >
> > Thanks,
> > John
> >
> >
> >
> >
> >
> >
> > On Fri, Oct 28, 2016 at 6:28 AM, Benjamin Blake - NOAA Affiliate
via RT <
> > met_help at ucar.edu> wrote:
> >
> >>
> >> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=78506 >
> >>
> >> Hi John,
> >>
> >> Yep, that was indeed the issue. I was able to plot the
probabilities.
> >> Thanks for looking into it!
> >>
> >> I am just starting to look into why the values on my reliability
diagram
> >> that I sent you several weeks ago are so low, and you suggested I
use
> the
> >> plot_data_plane tool to visualize the input that is being used to
> generate
> >> the statistics. I've attached the forecast and observed fields
as well
> as
> >> the pjc.txt file which contains the values for base rate. I know
that
> >> this
> >> may not necessarily be the best probability forecast, as a large
area
> of >
> >> 12.7 mm of precip fell over the central US that corresponds to
low
> >> probabilities. However, if you look at the pjc.txt file, the
values for
> >> base rate are all really low, indicating a huge amount of
> overforecasting.
> >>
> >> Do you know if the areas where there is no observation data (like
over
> the
> >> oceans) are being taken into account with gridstat? That would
explain
> >> why
> >> the base rate values are so low.
> >>
> >> Thanks again!
> >> Ben
> >>
> >> On Thu, Oct 27, 2016 at 5:01 PM, John Halley Gotway via RT <
> >> met_help at ucar.edu> wrote:
> >>
> >> > Ben,
> >> >
> >> > Thanks for pointing out this issue. I ran the code in debug
mode and
> >> see
> >> > the source of the problem.
> >> >
> >> > It's having that extra set of parenthesis that's confusing the
parsing
> >> > logic. I'll need to work up a bugfix and post a patch file.
> >> >
> >> > For a short-term workaround, you could do the following:
> >> >
> >> > module load nco
> >> >
> >> > # Rename the variables to get rid of parenthesis
> >> >
> >> > ncrename -v
> >> > FCST_PROB\(APCP_06\>12.700\)_A06_FULL,FCST_PROB_APCP_06\>12.
> >> 700_A06_FULL
> >> > grid_stat_PROB_12.7_f12_120000L_20161006_120000V_pairs.nc -o
> >> >
grid_stat_PROB_12.7_f12_120000L_20161006_120000V_pairs_rename.nc
> >> >
> >> > # Plot the variable without parenthesis
> >> >
> >> > /scratch4/BMC/dtc/MET/met-5.2/bin/plot_data_plane
> >> >
grid_stat_PROB_12.7_f12_120000L_20161006_120000V_pairs_rename.nc
> >> fcst.ps
> >> > 'name="FCST_PROB_APCP_06>12.700_A06_FULL"; level="(*,*)";'
> >> >
> >> > I'll let you know when I have a patch to fix this directly.
> >> >
> >> > Thanks,
> >> > John Halley Gotway
> >> >
> >> > On Thu, Oct 27, 2016 at 1:32 PM, Benjamin Blake - NOAA
Affiliate via
> RT
> >> <
> >> > met_help at ucar.edu> wrote:
> >> >
> >> > >
> >> > > Thu Oct 27 13:32:31 2016: Request 78506 was acted upon.
> >> > > Transaction: Ticket created by benjamin.blake at noaa.gov
> >> > > Queue: met_help
> >> > > Subject: plot_data_plane error - Needs more arguments
for
> >> variable
> >> > > Owner: Nobody
> >> > > Requestors: benjamin.blake at noaa.gov
> >> > > Status: new
> >> > > Ticket <URL: https://rt.rap.ucar.edu/rt/
> Ticket/Display.html?id=78506
> >> >
> >> > >
> >> > >
> >> > > Hi,
> >> > >
> >> > > I am trying to use the plot_data_plane tool to visualize the
> >> observation
> >> > > field of 6-hour accumulated precipitation (CCPA) and my
> corresponding
> >> > > forecast field of probabilities of 6-hour precipitation >
12.7 mm.
> >> Both
> >> > > fields are on the same grid. I am using a pairs.nc file that
I
> >> > generated
> >> > > from MET's gridstat tool. I am able to plot the observation
field
> >> when I
> >> > > use the command:
> >> > >
> >> > > plot_data_plane
grid_stat_PROB_12.7_f12_120000L_20161006_120000V_
> >> > pairs.nc
> >> > > obs.ps 'name = "OBS_APCP_06_A06_FULL" ; level = "A06" ;'
> -plot_range
> >> 0
> >> > > 12.7
> >> > > -v 4
> >> > >
> >> > > However, when I try a similar command on the forecast field
> >> > (probabilities
> >> > > ranging from 0 to 1):
> >> > >
> >> > > plot_data_plane
grid_stat_PROB_12.7_f12_120000L_20161006_120000V_
> >> > pairs.nc
> >> > > fcst.ps 'name = "FCST_PROB\(APCP_06\>12.700\)_A06_FULL" ;
level =
> >> "A06"
> >> > ;'
> >> > > -v 4
> >> > >
> >> > > I get the following error:
> >> > > ERROR : MetNcFile::data(NcVar *, const LongArray &,
DataPlane &) ->
> >> > needed
> >> > > 2 arguments for variable FCST_PROB(APCP_06>12.700)_A06_FULL,
got 1
> >> > >
> >> > > I think the error has to do with how I define the forecast
field in
> >> the
> >> > > config file I am using. In addition to name and level, I
have a
> third
> >> > line
> >> > > for prob which contains multiple arguments. I tried to
modify the
> >> > > plot_data_plane command but I kept getting the same error.
I've
> >> attached
> >> > > the config file that I used to run gridstat. The pairs.nc
file
> that
> >> I
> >> > > used
> >> > > which was generated from gridstat was too large to attach to
this
> >> email
> >> > but
> >> > > it is located on Theia in the following directory:
> >> > > /scratch4/NCEPDEV/meso/noscrub/Benjamin.Blake
> >> > >
> >> > > Any help would be greatly appreciated!
> >> > >
> >> > > Thanks,
> >> > > Ben Blake
> >> > >
> >> > >
> >> >
> >> >
> >>
> >>
> >
>
>
------------------------------------------------
Subject: plot_data_plane error - Needs more arguments for variable
From: Benjamin Blake - NOAA Affiliate
Time: Fri Oct 28 13:05:08 2016
Hi John,
Based on what you said, Jacob and I think we understand what is
happening.
My probabilities are defined as the chance of exceeding the threshold
within 40-km of a point. I was comparing that directly to the
observation
field (point observations). Instead, I need to process the
observation
field into a field of 1's and 0's - where a 1 signifies that the
threshold
was exceeded somewhere within 40-km - and compare that grid to the
forecast
probabilities. We think that should fix the issue that Matt Pyle and
I
were seeing with our reliability diagrams.
Thanks again for your help!
Ben
On Fri, Oct 28, 2016 at 2:09 PM, Benjamin Blake - NOAA Affiliate <
benjamin.blake at noaa.gov> wrote:
> Hi John,
>
> I do not have any objections with fixing that in version 6.0.
>
> It's definitely good to know that grid stat is not using points
where the
> observation field has no data - thanks for confirming.
>
> I will try experimenting with lowering the observation threshold to
see
> how the scores change, since that may help me better understand the
> performance.
>
> Thanks,
> Ben
>
> On Fri, Oct 28, 2016 at 1:36 PM, John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
>> Ben,
>>
>> When you run gen_vx_mask, try increasing the verbosity level to 3:
>>
>> /scratch4/BMC/dtc/MET/met-5.2/bin/gen_vx_mask \
>> grid_stat_PROB_12.7_f12_120000L_20161006_120000V_pairs.nc \
>> grid_stat_PROB_12.7_f12_120000L_20161006_120000V_pairs.nc \
>> obs_ge12.7.nc \
>> -type data \
>> -mask_field 'name="OBS_APCP_06_A06_FULL"; level="(*,*)";' \
>> -thresh ge12.7 \
>> -name obs_ge12.7 \
>> -v 3
>>
>> At verbosity level 3, it tells you the number of points included in
the
>> mask (i.e. the number that met the threshold):
>>
>> DEBUG 3: Data Masking: 4446 of 1509825 points inside
>>
>> Thanks,
>> John
>>
>> On Fri, Oct 28, 2016 at 11:03 AM, John Halley Gotway
<johnhg at ucar.edu>
>> wrote:
>>
>> > Ben,
>> >
>> > I clearly see the cause of your original issue with
plot_data_plane in
>> the
>> > code... but the best solution is less obvious. Fixing this the
"right"
>> way
>> > will be somewhat of a large change and will affect many files.
While
>> I'm
>> > fine doing that in the development version for the 6.0 release, I
>> hesitate
>> > to make that large of a change as a bugfix. Do you have any
objections
>> to
>> > me fixing this in 6.0 but not posting a 5.2 bugfix?
>> >
>> > On to your question... no, Grid-Stat does not use points where
either
>> the
>> > forecast or observation contains bad data values. When doing
>> verification
>> > of continuous (i.e. non-probabilistic) fields, the NetCDF matched
pairs
>> > file contains output variables for the forecast (FCST),
observation
>> (OBS),
>> > and their difference (DIFF). Looking at the difference field, it
>> becomes
>> > obvious that the bad data values over the ocean are skipped. But
for
>> > probabilistic verification, the raw values aren't subtracted
directly,
>> so
>> > we don't write out that DIFF field.
>> >
>> > Another thing to note is that the dimension of your grid is 1025
x 1473
>> =
>> > 1,509,825 grid points but the TOTAL column in the MET output
lists only
>> > 409,008 matched pairs. So only 27% of the grid points contain
valid
>> > forecast/observation data and are used in the verification.
>> >
>> > Rather than looking at the full field of observation values,
here's a
>> > trick for how you could threshold and then plot them:
>> >
>> > /scratch4/BMC/dtc/MET/met-5.2/bin/gen_vx_mask \
>> > grid_stat_PROB_12.7_f12_120000L_20161006_120000V_pairs.nc \
>> > grid_stat_PROB_12.7_f12_120000L_20161006_120000V_pairs.nc \
>> > obs_ge12.7.nc \
>> > -type data \
>> > -mask_field 'name="OBS_APCP_06_A06_FULL"; level="(*,*)";' \
>> > -thresh ge12.7 \
>> > -name obs_ge12.7
>> >
>> > /scratch4/BMC/dtc/MET/met-5.2/bin/plot_data_plane \
>> > obs_ge12.7.nc obs_ge12.7.ps 'name="obs_ge12.7"; level="(*,*)";'
>> >
>> > The resulting image is attached. The areas of pink show values
>> >=12.7. I
>> > opened that NetCDF file in R and found that there are 4446 grid
points
>> that
>> > are "pink". 4446 / 409008 is about 0.01. So the event occurs
only
>> about
>> > 1% of the time in this example.
>> >
>> > Looking at the OY_TP_i and ON_TP_i values in the data file you
sent,
>> > roughly speaking those numbers look pretty consistent with a 1%
base
>> rate
>> > over the entire field.
>> >
>> > One option you could try would be lowering the observation
threshold
>> (try
>> > >=6.35 or >=2.54) just to see how the scores change. I realize
that the
>> > probability is defined for >=12.7 but you could play around with
the
>> > observation threshold to better understand the performance. That
>> should be
>> > as simple as adding more entries to the observation categorical
>> threshold
>> > setting in the config file:
>> > cat_thresh = [ >=12.7, >=6.35, >=2.54, >=0.254 ];
>> >
>> > Hope that helps.
>> >
>> > Thanks,
>> > John
>> >
>> >
>> >
>> >
>> >
>> >
>> > On Fri, Oct 28, 2016 at 6:28 AM, Benjamin Blake - NOAA Affiliate
via RT
>> <
>> > met_help at ucar.edu> wrote:
>> >
>> >>
>> >> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=78506 >
>> >>
>> >> Hi John,
>> >>
>> >> Yep, that was indeed the issue. I was able to plot the
probabilities.
>> >> Thanks for looking into it!
>> >>
>> >> I am just starting to look into why the values on my reliability
>> diagram
>> >> that I sent you several weeks ago are so low, and you suggested
I use
>> the
>> >> plot_data_plane tool to visualize the input that is being used
to
>> generate
>> >> the statistics. I've attached the forecast and observed fields
as
>> well as
>> >> the pjc.txt file which contains the values for base rate. I
know that
>> >> this
>> >> may not necessarily be the best probability forecast, as a large
area
>> of >
>> >> 12.7 mm of precip fell over the central US that corresponds to
low
>> >> probabilities. However, if you look at the pjc.txt file, the
values
>> for
>> >> base rate are all really low, indicating a huge amount of
>> overforecasting.
>> >>
>> >> Do you know if the areas where there is no observation data
(like over
>> the
>> >> oceans) are being taken into account with gridstat? That would
explain
>> >> why
>> >> the base rate values are so low.
>> >>
>> >> Thanks again!
>> >> Ben
>> >>
>> >> On Thu, Oct 27, 2016 at 5:01 PM, John Halley Gotway via RT <
>> >> met_help at ucar.edu> wrote:
>> >>
>> >> > Ben,
>> >> >
>> >> > Thanks for pointing out this issue. I ran the code in debug
mode and
>> >> see
>> >> > the source of the problem.
>> >> >
>> >> > It's having that extra set of parenthesis that's confusing the
>> parsing
>> >> > logic. I'll need to work up a bugfix and post a patch file.
>> >> >
>> >> > For a short-term workaround, you could do the following:
>> >> >
>> >> > module load nco
>> >> >
>> >> > # Rename the variables to get rid of parenthesis
>> >> >
>> >> > ncrename -v
>> >> > FCST_PROB\(APCP_06\>12.700\)_A06_FULL,FCST_PROB_APCP_06\>12.
>> >> 700_A06_FULL
>> >> > grid_stat_PROB_12.7_f12_120000L_20161006_120000V_pairs.nc -o
>> >> >
grid_stat_PROB_12.7_f12_120000L_20161006_120000V_pairs_rename.nc
>> >> >
>> >> > # Plot the variable without parenthesis
>> >> >
>> >> > /scratch4/BMC/dtc/MET/met-5.2/bin/plot_data_plane
>> >> >
grid_stat_PROB_12.7_f12_120000L_20161006_120000V_pairs_rename.nc
>> >> fcst.ps
>> >> > 'name="FCST_PROB_APCP_06>12.700_A06_FULL"; level="(*,*)";'
>> >> >
>> >> > I'll let you know when I have a patch to fix this directly.
>> >> >
>> >> > Thanks,
>> >> > John Halley Gotway
>> >> >
>> >> > On Thu, Oct 27, 2016 at 1:32 PM, Benjamin Blake - NOAA
Affiliate via
>> RT
>> >> <
>> >> > met_help at ucar.edu> wrote:
>> >> >
>> >> > >
>> >> > > Thu Oct 27 13:32:31 2016: Request 78506 was acted upon.
>> >> > > Transaction: Ticket created by benjamin.blake at noaa.gov
>> >> > > Queue: met_help
>> >> > > Subject: plot_data_plane error - Needs more arguments
for
>> >> variable
>> >> > > Owner: Nobody
>> >> > > Requestors: benjamin.blake at noaa.gov
>> >> > > Status: new
>> >> > > Ticket <URL: https://rt.rap.ucar.edu/rt/Tic
>> ket/Display.html?id=78506
>> >> >
>> >> > >
>> >> > >
>> >> > > Hi,
>> >> > >
>> >> > > I am trying to use the plot_data_plane tool to visualize the
>> >> observation
>> >> > > field of 6-hour accumulated precipitation (CCPA) and my
>> corresponding
>> >> > > forecast field of probabilities of 6-hour precipitation >
12.7 mm.
>> >> Both
>> >> > > fields are on the same grid. I am using a pairs.nc file
that I
>> >> > generated
>> >> > > from MET's gridstat tool. I am able to plot the observation
field
>> >> when I
>> >> > > use the command:
>> >> > >
>> >> > > plot_data_plane
grid_stat_PROB_12.7_f12_120000L_20161006_120000V_
>> >> > pairs.nc
>> >> > > obs.ps 'name = "OBS_APCP_06_A06_FULL" ; level = "A06" ;'
>> -plot_range
>> >> 0
>> >> > > 12.7
>> >> > > -v 4
>> >> > >
>> >> > > However, when I try a similar command on the forecast field
>> >> > (probabilities
>> >> > > ranging from 0 to 1):
>> >> > >
>> >> > > plot_data_plane
grid_stat_PROB_12.7_f12_120000L_20161006_120000V_
>> >> > pairs.nc
>> >> > > fcst.ps 'name = "FCST_PROB\(APCP_06\>12.700\)_A06_FULL" ;
level =
>> >> "A06"
>> >> > ;'
>> >> > > -v 4
>> >> > >
>> >> > > I get the following error:
>> >> > > ERROR : MetNcFile::data(NcVar *, const LongArray &,
DataPlane &)
>> ->
>> >> > needed
>> >> > > 2 arguments for variable FCST_PROB(APCP_06>12.700)_A06_FULL,
got 1
>> >> > >
>> >> > > I think the error has to do with how I define the forecast
field in
>> >> the
>> >> > > config file I am using. In addition to name and level, I
have a
>> third
>> >> > line
>> >> > > for prob which contains multiple arguments. I tried to
modify the
>> >> > > plot_data_plane command but I kept getting the same error.
I've
>> >> attached
>> >> > > the config file that I used to run gridstat. The pairs.nc
file
>> that
>> >> I
>> >> > > used
>> >> > > which was generated from gridstat was too large to attach to
this
>> >> email
>> >> > but
>> >> > > it is located on Theia in the following directory:
>> >> > > /scratch4/NCEPDEV/meso/noscrub/Benjamin.Blake
>> >> > >
>> >> > > Any help would be greatly appreciated!
>> >> > >
>> >> > > Thanks,
>> >> > > Ben Blake
>> >> > >
>> >> > >
>> >> >
>> >> >
>> >>
>> >>
>> >
>>
>>
>
------------------------------------------------
Subject: plot_data_plane error - Needs more arguments for variable
From: John Halley Gotway
Time: Fri Oct 28 13:26:04 2016
Hi Ben,
Here's an idea. When you run Grid-Stat you could specify that an
"interpolation" method be applied to the observation field prior to
verifying. In this context of grid-to-grid verification,
interpolation
really means preprocessing or smoothing.
You could try using this:
interp = {
field = OBS;
vld_thresh = 1.0;
type = [
{ method = NEAREST; width = 1; },
{ method = MAX; width = 3; },
{ method = MAX; width = 5; },
{ method = MAX; width = 7; },
{ method = MAX; width = 9; }
];
}
Rather than having a single output, you'd get output for 5 different
comparisons. The first using "NEAREST" is what you've already done.
The
second (MAX of 3) says the following... process each observation grid
point
and look at the 9 surrounding grid points (i.e. 3x3 square). Replace
the
value at the current grid point with the maximum of those 9 points.
The
3rd, 4th, and 5th ones do the same as the 2nd but using increasingly
larger
box sizes.
Presumably, one of these box sizes is somewhat close to 40km.
Note that the widths must be odd (I think) to keep the smoothing box
centered on the current grid point. MET supports several options for
"method" and we often use "UW_MEAN" for the unweighted mean, or
average.
But in this context, I think that the maximum might make sense.
Thanks,
John
And then use the resulting field in the verification.
On Fri, Oct 28, 2016 at 1:05 PM, Benjamin Blake - NOAA Affiliate via
RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=78506 >
>
> Hi John,
>
> Based on what you said, Jacob and I think we understand what is
happening.
> My probabilities are defined as the chance of exceeding the
threshold
> within 40-km of a point. I was comparing that directly to the
observation
> field (point observations). Instead, I need to process the
observation
> field into a field of 1's and 0's - where a 1 signifies that the
threshold
> was exceeded somewhere within 40-km - and compare that grid to the
forecast
> probabilities. We think that should fix the issue that Matt Pyle
and I
> were seeing with our reliability diagrams.
>
> Thanks again for your help!
> Ben
>
> On Fri, Oct 28, 2016 at 2:09 PM, Benjamin Blake - NOAA Affiliate <
> benjamin.blake at noaa.gov> wrote:
>
> > Hi John,
> >
> > I do not have any objections with fixing that in version 6.0.
> >
> > It's definitely good to know that grid stat is not using points
where the
> > observation field has no data - thanks for confirming.
> >
> > I will try experimenting with lowering the observation threshold
to see
> > how the scores change, since that may help me better understand
the
> > performance.
> >
> > Thanks,
> > Ben
> >
> > On Fri, Oct 28, 2016 at 1:36 PM, John Halley Gotway via RT <
> > met_help at ucar.edu> wrote:
> >
> >> Ben,
> >>
> >> When you run gen_vx_mask, try increasing the verbosity level to
3:
> >>
> >> /scratch4/BMC/dtc/MET/met-5.2/bin/gen_vx_mask \
> >> grid_stat_PROB_12.7_f12_120000L_20161006_120000V_pairs.nc \
> >> grid_stat_PROB_12.7_f12_120000L_20161006_120000V_pairs.nc \
> >> obs_ge12.7.nc \
> >> -type data \
> >> -mask_field 'name="OBS_APCP_06_A06_FULL"; level="(*,*)";' \
> >> -thresh ge12.7 \
> >> -name obs_ge12.7 \
> >> -v 3
> >>
> >> At verbosity level 3, it tells you the number of points included
in the
> >> mask (i.e. the number that met the threshold):
> >>
> >> DEBUG 3: Data Masking: 4446 of 1509825 points inside
> >>
> >> Thanks,
> >> John
> >>
> >> On Fri, Oct 28, 2016 at 11:03 AM, John Halley Gotway
<johnhg at ucar.edu>
> >> wrote:
> >>
> >> > Ben,
> >> >
> >> > I clearly see the cause of your original issue with
plot_data_plane in
> >> the
> >> > code... but the best solution is less obvious. Fixing this the
> "right"
> >> way
> >> > will be somewhat of a large change and will affect many files.
While
> >> I'm
> >> > fine doing that in the development version for the 6.0 release,
I
> >> hesitate
> >> > to make that large of a change as a bugfix. Do you have any
> objections
> >> to
> >> > me fixing this in 6.0 but not posting a 5.2 bugfix?
> >> >
> >> > On to your question... no, Grid-Stat does not use points where
either
> >> the
> >> > forecast or observation contains bad data values. When doing
> >> verification
> >> > of continuous (i.e. non-probabilistic) fields, the NetCDF
matched
> pairs
> >> > file contains output variables for the forecast (FCST),
observation
> >> (OBS),
> >> > and their difference (DIFF). Looking at the difference field,
it
> >> becomes
> >> > obvious that the bad data values over the ocean are skipped.
But for
> >> > probabilistic verification, the raw values aren't subtracted
directly,
> >> so
> >> > we don't write out that DIFF field.
> >> >
> >> > Another thing to note is that the dimension of your grid is
1025 x
> 1473
> >> =
> >> > 1,509,825 grid points but the TOTAL column in the MET output
lists
> only
> >> > 409,008 matched pairs. So only 27% of the grid points contain
valid
> >> > forecast/observation data and are used in the verification.
> >> >
> >> > Rather than looking at the full field of observation values,
here's a
> >> > trick for how you could threshold and then plot them:
> >> >
> >> > /scratch4/BMC/dtc/MET/met-5.2/bin/gen_vx_mask \
> >> > grid_stat_PROB_12.7_f12_120000L_20161006_120000V_pairs.nc \
> >> > grid_stat_PROB_12.7_f12_120000L_20161006_120000V_pairs.nc \
> >> > obs_ge12.7.nc \
> >> > -type data \
> >> > -mask_field 'name="OBS_APCP_06_A06_FULL"; level="(*,*)";' \
> >> > -thresh ge12.7 \
> >> > -name obs_ge12.7
> >> >
> >> > /scratch4/BMC/dtc/MET/met-5.2/bin/plot_data_plane \
> >> > obs_ge12.7.nc obs_ge12.7.ps 'name="obs_ge12.7"; level="(*,*)";'
> >> >
> >> > The resulting image is attached. The areas of pink show values
> >> >=12.7. I
> >> > opened that NetCDF file in R and found that there are 4446 grid
points
> >> that
> >> > are "pink". 4446 / 409008 is about 0.01. So the event occurs
only
> >> about
> >> > 1% of the time in this example.
> >> >
> >> > Looking at the OY_TP_i and ON_TP_i values in the data file you
sent,
> >> > roughly speaking those numbers look pretty consistent with a 1%
base
> >> rate
> >> > over the entire field.
> >> >
> >> > One option you could try would be lowering the observation
threshold
> >> (try
> >> > >=6.35 or >=2.54) just to see how the scores change. I realize
that
> the
> >> > probability is defined for >=12.7 but you could play around
with the
> >> > observation threshold to better understand the performance.
That
> >> should be
> >> > as simple as adding more entries to the observation categorical
> >> threshold
> >> > setting in the config file:
> >> > cat_thresh = [ >=12.7, >=6.35, >=2.54, >=0.254 ];
> >> >
> >> > Hope that helps.
> >> >
> >> > Thanks,
> >> > John
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > On Fri, Oct 28, 2016 at 6:28 AM, Benjamin Blake - NOAA
Affiliate via
> RT
> >> <
> >> > met_help at ucar.edu> wrote:
> >> >
> >> >>
> >> >> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=78506
>
> >> >>
> >> >> Hi John,
> >> >>
> >> >> Yep, that was indeed the issue. I was able to plot the
> probabilities.
> >> >> Thanks for looking into it!
> >> >>
> >> >> I am just starting to look into why the values on my
reliability
> >> diagram
> >> >> that I sent you several weeks ago are so low, and you
suggested I use
> >> the
> >> >> plot_data_plane tool to visualize the input that is being used
to
> >> generate
> >> >> the statistics. I've attached the forecast and observed
fields as
> >> well as
> >> >> the pjc.txt file which contains the values for base rate. I
know
> that
> >> >> this
> >> >> may not necessarily be the best probability forecast, as a
large area
> >> of >
> >> >> 12.7 mm of precip fell over the central US that corresponds to
low
> >> >> probabilities. However, if you look at the pjc.txt file, the
values
> >> for
> >> >> base rate are all really low, indicating a huge amount of
> >> overforecasting.
> >> >>
> >> >> Do you know if the areas where there is no observation data
(like
> over
> >> the
> >> >> oceans) are being taken into account with gridstat? That
would
> explain
> >> >> why
> >> >> the base rate values are so low.
> >> >>
> >> >> Thanks again!
> >> >> Ben
> >> >>
> >> >> On Thu, Oct 27, 2016 at 5:01 PM, John Halley Gotway via RT <
> >> >> met_help at ucar.edu> wrote:
> >> >>
> >> >> > Ben,
> >> >> >
> >> >> > Thanks for pointing out this issue. I ran the code in debug
mode
> and
> >> >> see
> >> >> > the source of the problem.
> >> >> >
> >> >> > It's having that extra set of parenthesis that's confusing
the
> >> parsing
> >> >> > logic. I'll need to work up a bugfix and post a patch file.
> >> >> >
> >> >> > For a short-term workaround, you could do the following:
> >> >> >
> >> >> > module load nco
> >> >> >
> >> >> > # Rename the variables to get rid of parenthesis
> >> >> >
> >> >> > ncrename -v
> >> >> > FCST_PROB\(APCP_06\>12.700\)_A06_FULL,FCST_PROB_APCP_06\>12.
> >> >> 700_A06_FULL
> >> >> > grid_stat_PROB_12.7_f12_120000L_20161006_120000V_pairs.nc -o
> >> >> >
grid_stat_PROB_12.7_f12_120000L_20161006_120000V_pairs_rename.nc
> >> >> >
> >> >> > # Plot the variable without parenthesis
> >> >> >
> >> >> > /scratch4/BMC/dtc/MET/met-5.2/bin/plot_data_plane
> >> >> >
grid_stat_PROB_12.7_f12_120000L_20161006_120000V_pairs_rename.nc
> >> >> fcst.ps
> >> >> > 'name="FCST_PROB_APCP_06>12.700_A06_FULL"; level="(*,*)";'
> >> >> >
> >> >> > I'll let you know when I have a patch to fix this directly.
> >> >> >
> >> >> > Thanks,
> >> >> > John Halley Gotway
> >> >> >
> >> >> > On Thu, Oct 27, 2016 at 1:32 PM, Benjamin Blake - NOAA
Affiliate
> via
> >> RT
> >> >> <
> >> >> > met_help at ucar.edu> wrote:
> >> >> >
> >> >> > >
> >> >> > > Thu Oct 27 13:32:31 2016: Request 78506 was acted upon.
> >> >> > > Transaction: Ticket created by benjamin.blake at noaa.gov
> >> >> > > Queue: met_help
> >> >> > > Subject: plot_data_plane error - Needs more arguments
for
> >> >> variable
> >> >> > > Owner: Nobody
> >> >> > > Requestors: benjamin.blake at noaa.gov
> >> >> > > Status: new
> >> >> > > Ticket <URL: https://rt.rap.ucar.edu/rt/Tic
> >> ket/Display.html?id=78506
> >> >> >
> >> >> > >
> >> >> > >
> >> >> > > Hi,
> >> >> > >
> >> >> > > I am trying to use the plot_data_plane tool to visualize
the
> >> >> observation
> >> >> > > field of 6-hour accumulated precipitation (CCPA) and my
> >> corresponding
> >> >> > > forecast field of probabilities of 6-hour precipitation >
12.7
> mm.
> >> >> Both
> >> >> > > fields are on the same grid. I am using a pairs.nc file
that I
> >> >> > generated
> >> >> > > from MET's gridstat tool. I am able to plot the
observation
> field
> >> >> when I
> >> >> > > use the command:
> >> >> > >
> >> >> > > plot_data_plane grid_stat_PROB_12.7_f12_
> 120000L_20161006_120000V_
> >> >> > pairs.nc
> >> >> > > obs.ps 'name = "OBS_APCP_06_A06_FULL" ; level = "A06" ;'
> >> -plot_range
> >> >> 0
> >> >> > > 12.7
> >> >> > > -v 4
> >> >> > >
> >> >> > > However, when I try a similar command on the forecast
field
> >> >> > (probabilities
> >> >> > > ranging from 0 to 1):
> >> >> > >
> >> >> > > plot_data_plane grid_stat_PROB_12.7_f12_
> 120000L_20161006_120000V_
> >> >> > pairs.nc
> >> >> > > fcst.ps 'name = "FCST_PROB\(APCP_06\>12.700\)_A06_FULL" ;
level
> =
> >> >> "A06"
> >> >> > ;'
> >> >> > > -v 4
> >> >> > >
> >> >> > > I get the following error:
> >> >> > > ERROR : MetNcFile::data(NcVar *, const LongArray &,
DataPlane &)
> >> ->
> >> >> > needed
> >> >> > > 2 arguments for variable
FCST_PROB(APCP_06>12.700)_A06_FULL,
> got 1
> >> >> > >
> >> >> > > I think the error has to do with how I define the forecast
field
> in
> >> >> the
> >> >> > > config file I am using. In addition to name and level, I
have a
> >> third
> >> >> > line
> >> >> > > for prob which contains multiple arguments. I tried to
modify
> the
> >> >> > > plot_data_plane command but I kept getting the same error.
I've
> >> >> attached
> >> >> > > the config file that I used to run gridstat. The pairs.nc
file
> >> that
> >> >> I
> >> >> > > used
> >> >> > > which was generated from gridstat was too large to attach
to this
> >> >> email
> >> >> > but
> >> >> > > it is located on Theia in the following directory:
> >> >> > > /scratch4/NCEPDEV/meso/noscrub/Benjamin.Blake
> >> >> > >
> >> >> > > Any help would be greatly appreciated!
> >> >> > >
> >> >> > > Thanks,
> >> >> > > Ben Blake
> >> >> > >
> >> >> > >
> >> >> >
> >> >> >
> >> >>
> >> >>
> >> >
> >>
> >>
> >
>
>
------------------------------------------------
Subject: plot_data_plane error - Needs more arguments for variable
From: Benjamin Blake - NOAA Affiliate
Time: Mon Oct 31 05:36:13 2016
Hi John,
Great, thanks! I thought there might be an option like that in the
configure file and I was planning on looking into that today but that
saves
me some time. My grid spacing is 5-km so a width of 9 should suffice,
and
yes I believe the maximum is what I'd be interested in.
Thanks,
Ben
On Fri, Oct 28, 2016 at 3:26 PM, John Halley Gotway via RT <
met_help at ucar.edu> wrote:
> Hi Ben,
>
> Here's an idea. When you run Grid-Stat you could specify that an
> "interpolation" method be applied to the observation field prior to
> verifying. In this context of grid-to-grid verification,
interpolation
> really means preprocessing or smoothing.
>
> You could try using this:
>
> interp = {
> field = OBS;
> vld_thresh = 1.0;
>
> type = [
> { method = NEAREST; width = 1; },
> { method = MAX; width = 3; },
> { method = MAX; width = 5; },
> { method = MAX; width = 7; },
> { method = MAX; width = 9; }
> ];
> }
>
> Rather than having a single output, you'd get output for 5 different
> comparisons. The first using "NEAREST" is what you've already done.
The
> second (MAX of 3) says the following... process each observation
grid point
> and look at the 9 surrounding grid points (i.e. 3x3 square).
Replace the
> value at the current grid point with the maximum of those 9 points.
The
> 3rd, 4th, and 5th ones do the same as the 2nd but using increasingly
larger
> box sizes.
>
> Presumably, one of these box sizes is somewhat close to 40km.
>
> Note that the widths must be odd (I think) to keep the smoothing box
> centered on the current grid point. MET supports several options
for
> "method" and we often use "UW_MEAN" for the unweighted mean, or
average.
> But in this context, I think that the maximum might make sense.
>
> Thanks,
> John
>
>
>
> And then use the resulting field in the verification.
>
>
> On Fri, Oct 28, 2016 at 1:05 PM, Benjamin Blake - NOAA Affiliate via
RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=78506 >
> >
> > Hi John,
> >
> > Based on what you said, Jacob and I think we understand what is
> happening.
> > My probabilities are defined as the chance of exceeding the
threshold
> > within 40-km of a point. I was comparing that directly to the
> observation
> > field (point observations). Instead, I need to process the
observation
> > field into a field of 1's and 0's - where a 1 signifies that the
> threshold
> > was exceeded somewhere within 40-km - and compare that grid to the
> forecast
> > probabilities. We think that should fix the issue that Matt Pyle
and I
> > were seeing with our reliability diagrams.
> >
> > Thanks again for your help!
> > Ben
> >
> > On Fri, Oct 28, 2016 at 2:09 PM, Benjamin Blake - NOAA Affiliate <
> > benjamin.blake at noaa.gov> wrote:
> >
> > > Hi John,
> > >
> > > I do not have any objections with fixing that in version 6.0.
> > >
> > > It's definitely good to know that grid stat is not using points
where
> the
> > > observation field has no data - thanks for confirming.
> > >
> > > I will try experimenting with lowering the observation threshold
to see
> > > how the scores change, since that may help me better understand
the
> > > performance.
> > >
> > > Thanks,
> > > Ben
> > >
> > > On Fri, Oct 28, 2016 at 1:36 PM, John Halley Gotway via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > >> Ben,
> > >>
> > >> When you run gen_vx_mask, try increasing the verbosity level to
3:
> > >>
> > >> /scratch4/BMC/dtc/MET/met-5.2/bin/gen_vx_mask \
> > >> grid_stat_PROB_12.7_f12_120000L_20161006_120000V_pairs.nc \
> > >> grid_stat_PROB_12.7_f12_120000L_20161006_120000V_pairs.nc \
> > >> obs_ge12.7.nc \
> > >> -type data \
> > >> -mask_field 'name="OBS_APCP_06_A06_FULL"; level="(*,*)";' \
> > >> -thresh ge12.7 \
> > >> -name obs_ge12.7 \
> > >> -v 3
> > >>
> > >> At verbosity level 3, it tells you the number of points
included in
> the
> > >> mask (i.e. the number that met the threshold):
> > >>
> > >> DEBUG 3: Data Masking: 4446 of 1509825 points inside
> > >>
> > >> Thanks,
> > >> John
> > >>
> > >> On Fri, Oct 28, 2016 at 11:03 AM, John Halley Gotway
<johnhg at ucar.edu
> >
> > >> wrote:
> > >>
> > >> > Ben,
> > >> >
> > >> > I clearly see the cause of your original issue with
plot_data_plane
> in
> > >> the
> > >> > code... but the best solution is less obvious. Fixing this
the
> > "right"
> > >> way
> > >> > will be somewhat of a large change and will affect many
files.
> While
> > >> I'm
> > >> > fine doing that in the development version for the 6.0
release, I
> > >> hesitate
> > >> > to make that large of a change as a bugfix. Do you have any
> > objections
> > >> to
> > >> > me fixing this in 6.0 but not posting a 5.2 bugfix?
> > >> >
> > >> > On to your question... no, Grid-Stat does not use points
where
> either
> > >> the
> > >> > forecast or observation contains bad data values. When doing
> > >> verification
> > >> > of continuous (i.e. non-probabilistic) fields, the NetCDF
matched
> > pairs
> > >> > file contains output variables for the forecast (FCST),
observation
> > >> (OBS),
> > >> > and their difference (DIFF). Looking at the difference
field, it
> > >> becomes
> > >> > obvious that the bad data values over the ocean are skipped.
But
> for
> > >> > probabilistic verification, the raw values aren't subtracted
> directly,
> > >> so
> > >> > we don't write out that DIFF field.
> > >> >
> > >> > Another thing to note is that the dimension of your grid is
1025 x
> > 1473
> > >> =
> > >> > 1,509,825 grid points but the TOTAL column in the MET output
lists
> > only
> > >> > 409,008 matched pairs. So only 27% of the grid points
contain valid
> > >> > forecast/observation data and are used in the verification.
> > >> >
> > >> > Rather than looking at the full field of observation values,
here's
> a
> > >> > trick for how you could threshold and then plot them:
> > >> >
> > >> > /scratch4/BMC/dtc/MET/met-5.2/bin/gen_vx_mask \
> > >> > grid_stat_PROB_12.7_f12_120000L_20161006_120000V_pairs.nc \
> > >> > grid_stat_PROB_12.7_f12_120000L_20161006_120000V_pairs.nc \
> > >> > obs_ge12.7.nc \
> > >> > -type data \
> > >> > -mask_field 'name="OBS_APCP_06_A06_FULL"; level="(*,*)";' \
> > >> > -thresh ge12.7 \
> > >> > -name obs_ge12.7
> > >> >
> > >> > /scratch4/BMC/dtc/MET/met-5.2/bin/plot_data_plane \
> > >> > obs_ge12.7.nc obs_ge12.7.ps 'name="obs_ge12.7";
level="(*,*)";'
> > >> >
> > >> > The resulting image is attached. The areas of pink show
values
> > >> >=12.7. I
> > >> > opened that NetCDF file in R and found that there are 4446
grid
> points
> > >> that
> > >> > are "pink". 4446 / 409008 is about 0.01. So the event
occurs only
> > >> about
> > >> > 1% of the time in this example.
> > >> >
> > >> > Looking at the OY_TP_i and ON_TP_i values in the data file
you sent,
> > >> > roughly speaking those numbers look pretty consistent with a
1% base
> > >> rate
> > >> > over the entire field.
> > >> >
> > >> > One option you could try would be lowering the observation
threshold
> > >> (try
> > >> > >=6.35 or >=2.54) just to see how the scores change. I
realize that
> > the
> > >> > probability is defined for >=12.7 but you could play around
with the
> > >> > observation threshold to better understand the performance.
That
> > >> should be
> > >> > as simple as adding more entries to the observation
categorical
> > >> threshold
> > >> > setting in the config file:
> > >> > cat_thresh = [ >=12.7, >=6.35, >=2.54, >=0.254 ];
> > >> >
> > >> > Hope that helps.
> > >> >
> > >> > Thanks,
> > >> > John
> > >> >
> > >> >
> > >> >
> > >> >
> > >> >
> > >> >
> > >> > On Fri, Oct 28, 2016 at 6:28 AM, Benjamin Blake - NOAA
Affiliate via
> > RT
> > >> <
> > >> > met_help at ucar.edu> wrote:
> > >> >
> > >> >>
> > >> >> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=78506 >
> > >> >>
> > >> >> Hi John,
> > >> >>
> > >> >> Yep, that was indeed the issue. I was able to plot the
> > probabilities.
> > >> >> Thanks for looking into it!
> > >> >>
> > >> >> I am just starting to look into why the values on my
reliability
> > >> diagram
> > >> >> that I sent you several weeks ago are so low, and you
suggested I
> use
> > >> the
> > >> >> plot_data_plane tool to visualize the input that is being
used to
> > >> generate
> > >> >> the statistics. I've attached the forecast and observed
fields as
> > >> well as
> > >> >> the pjc.txt file which contains the values for base rate. I
know
> > that
> > >> >> this
> > >> >> may not necessarily be the best probability forecast, as a
large
> area
> > >> of >
> > >> >> 12.7 mm of precip fell over the central US that corresponds
to low
> > >> >> probabilities. However, if you look at the pjc.txt file,
the
> values
> > >> for
> > >> >> base rate are all really low, indicating a huge amount of
> > >> overforecasting.
> > >> >>
> > >> >> Do you know if the areas where there is no observation data
(like
> > over
> > >> the
> > >> >> oceans) are being taken into account with gridstat? That
would
> > explain
> > >> >> why
> > >> >> the base rate values are so low.
> > >> >>
> > >> >> Thanks again!
> > >> >> Ben
> > >> >>
> > >> >> On Thu, Oct 27, 2016 at 5:01 PM, John Halley Gotway via RT <
> > >> >> met_help at ucar.edu> wrote:
> > >> >>
> > >> >> > Ben,
> > >> >> >
> > >> >> > Thanks for pointing out this issue. I ran the code in
debug mode
> > and
> > >> >> see
> > >> >> > the source of the problem.
> > >> >> >
> > >> >> > It's having that extra set of parenthesis that's confusing
the
> > >> parsing
> > >> >> > logic. I'll need to work up a bugfix and post a patch
file.
> > >> >> >
> > >> >> > For a short-term workaround, you could do the following:
> > >> >> >
> > >> >> > module load nco
> > >> >> >
> > >> >> > # Rename the variables to get rid of parenthesis
> > >> >> >
> > >> >> > ncrename -v
> > >> >> >
FCST_PROB\(APCP_06\>12.700\)_A06_FULL,FCST_PROB_APCP_06\>12.
> > >> >> 700_A06_FULL
> > >> >> > grid_stat_PROB_12.7_f12_120000L_20161006_120000V_pairs.nc
-o
> > >> >> >
grid_stat_PROB_12.7_f12_120000L_20161006_120000V_pairs_rename.nc
> > >> >> >
> > >> >> > # Plot the variable without parenthesis
> > >> >> >
> > >> >> > /scratch4/BMC/dtc/MET/met-5.2/bin/plot_data_plane
> > >> >> >
grid_stat_PROB_12.7_f12_120000L_20161006_120000V_pairs_rename.nc
> > >> >> fcst.ps
> > >> >> > 'name="FCST_PROB_APCP_06>12.700_A06_FULL"; level="(*,*)";'
> > >> >> >
> > >> >> > I'll let you know when I have a patch to fix this
directly.
> > >> >> >
> > >> >> > Thanks,
> > >> >> > John Halley Gotway
> > >> >> >
> > >> >> > On Thu, Oct 27, 2016 at 1:32 PM, Benjamin Blake - NOAA
Affiliate
> > via
> > >> RT
> > >> >> <
> > >> >> > met_help at ucar.edu> wrote:
> > >> >> >
> > >> >> > >
> > >> >> > > Thu Oct 27 13:32:31 2016: Request 78506 was acted upon.
> > >> >> > > Transaction: Ticket created by benjamin.blake at noaa.gov
> > >> >> > > Queue: met_help
> > >> >> > > Subject: plot_data_plane error - Needs more
arguments for
> > >> >> variable
> > >> >> > > Owner: Nobody
> > >> >> > > Requestors: benjamin.blake at noaa.gov
> > >> >> > > Status: new
> > >> >> > > Ticket <URL: https://rt.rap.ucar.edu/rt/Tic
> > >> ket/Display.html?id=78506
> > >> >> >
> > >> >> > >
> > >> >> > >
> > >> >> > > Hi,
> > >> >> > >
> > >> >> > > I am trying to use the plot_data_plane tool to visualize
the
> > >> >> observation
> > >> >> > > field of 6-hour accumulated precipitation (CCPA) and my
> > >> corresponding
> > >> >> > > forecast field of probabilities of 6-hour precipitation
> 12.7
> > mm.
> > >> >> Both
> > >> >> > > fields are on the same grid. I am using a pairs.nc file
that
> I
> > >> >> > generated
> > >> >> > > from MET's gridstat tool. I am able to plot the
observation
> > field
> > >> >> when I
> > >> >> > > use the command:
> > >> >> > >
> > >> >> > > plot_data_plane grid_stat_PROB_12.7_f12_
> > 120000L_20161006_120000V_
> > >> >> > pairs.nc
> > >> >> > > obs.ps 'name = "OBS_APCP_06_A06_FULL" ; level = "A06" ;'
> > >> -plot_range
> > >> >> 0
> > >> >> > > 12.7
> > >> >> > > -v 4
> > >> >> > >
> > >> >> > > However, when I try a similar command on the forecast
field
> > >> >> > (probabilities
> > >> >> > > ranging from 0 to 1):
> > >> >> > >
> > >> >> > > plot_data_plane grid_stat_PROB_12.7_f12_
> > 120000L_20161006_120000V_
> > >> >> > pairs.nc
> > >> >> > > fcst.ps 'name = "FCST_PROB\(APCP_06\>12.700\)_A06_FULL"
;
> level
> > =
> > >> >> "A06"
> > >> >> > ;'
> > >> >> > > -v 4
> > >> >> > >
> > >> >> > > I get the following error:
> > >> >> > > ERROR : MetNcFile::data(NcVar *, const LongArray &,
DataPlane
> &)
> > >> ->
> > >> >> > needed
> > >> >> > > 2 arguments for variable
FCST_PROB(APCP_06>12.700)_A06_FULL,
> > got 1
> > >> >> > >
> > >> >> > > I think the error has to do with how I define the
forecast
> field
> > in
> > >> >> the
> > >> >> > > config file I am using. In addition to name and level,
I have
> a
> > >> third
> > >> >> > line
> > >> >> > > for prob which contains multiple arguments. I tried to
modify
> > the
> > >> >> > > plot_data_plane command but I kept getting the same
error.
> I've
> > >> >> attached
> > >> >> > > the config file that I used to run gridstat. The
pairs.nc
> file
> > >> that
> > >> >> I
> > >> >> > > used
> > >> >> > > which was generated from gridstat was too large to
attach to
> this
> > >> >> email
> > >> >> > but
> > >> >> > > it is located on Theia in the following directory:
> > >> >> > > /scratch4/NCEPDEV/meso/noscrub/Benjamin.Blake
> > >> >> > >
> > >> >> > > Any help would be greatly appreciated!
> > >> >> > >
> > >> >> > > Thanks,
> > >> >> > > Ben Blake
> > >> >> > >
> > >> >> > >
> > >> >> >
> > >> >> >
> > >> >>
> > >> >>
> > >> >
> > >>
> > >>
> > >
> >
> >
>
>
------------------------------------------------
More information about the Met_help
mailing list