[Met_help] [rt.rap.ucar.edu #91448] History for Changing Units in STATS files
John Halley Gotway via RT
met_help at ucar.edu
Fri Aug 9 11:29:51 MDT 2019
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
John, I am verifying precipitation accumulation probabilities. I am using a
netcdf file with 6 hour accumulations. When I look through the stat file
produced, the units field keeps changing values (ie. w/m^2, gpm, kg/m^2,
0..9,m,.etc) and there will be two unit types for each precip threshold.
The listing indicates MET is using the correct grib record number for the
variable. Is this the expected behavior or is something not right. The
list of the run is below:
DEBUG 1: Reading user-defined grib1 MET_GRIB_TABLES file:
/h/data/global/WXQC/data/met/config/grib2/grib1_af_131_57.txt
DEBUG 1: Reading user-defined grib2 MET_GRIB_TABLES file:
/h/data/global/WXQC/data/met/config/grib2/grib2_geps.txt
DEBUG 1: Default Config File:
/h/MET/share/met/config/PointStatConfig_default
DEBUG 1: User Config File:
/h/data/global/WXQC/data/met/config/met_config/PointStatConfig_meps_tp_conus
_updated
GSL_RNG_TYPE=mt19937
GSL_RNG_SEED=18446744071888718941
DEBUG 1: Forecast File:
/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.2019080400.0048
DEBUG 1: Observation File:
/h/data/global/WXQC/data/met/nc_obs/precip/2019080600.nc
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match for
VarInfo "/A006" in GRIB record 4 of GRIB file
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.2019080400.004
8".
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB records
matching VarInfo "/A006" in GRIB file
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.2019080400.004
8".
DEBUG 2:
DEBUG 2:
----------------------------------------------------------------------------
----
DEBUG 2:
DEBUG 2: Reading data for /A006.
DEBUG 3: Use the forecast grid.
DEBUG 3: Grid Definition: Projection: Lambert Conformal Nx: 1170 Ny: 720
Lat_LL: 23.159 Lon_LL: 119.272 Lon_orient: 97.000 Alpha: 3171.947 Cone:
0.616 Bx: 582.1130 By: 2385.7494
DEBUG 2: Processing masking regions.
DEBUG 3: Processing grid mask: FULL
DEBUG 2: Processing geography data.
DEBUG 3: Rescaling probabilistic field from [0,100] to [0,1].
DEBUG 3: Applying conversion function.
DEBUG 1: Regridding climatology CFSR_APCP_T01_00(*,*) to the verification
grid.
DEBUG 3: For forecast valid at 20190806_000000, found 1 climatology field(s)
valid at 20190806_000000.
DEBUG 3: Found 1 climatology fields.
DEBUG 2: For /A006 found 1 forecast levels, 1 climatology mean levels, and 0
climatology standard deviation levels.
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match for
VarInfo "/A006" in GRIB record 7 of GRIB file
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.2019080400.004
8".
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB records
matching VarInfo "/A006" in GRIB file
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.2019080400.004
8".
DEBUG 2:
DEBUG 2:
----------------------------------------------------------------------------
----
DEBUG 2:
DEBUG 2: Reading data for /A006.
DEBUG 3: Rescaling probabilistic field from [0,100] to [0,1].
DEBUG 3: Applying conversion function.
DEBUG 1: Regridding climatology CFSR_APCP_T10_00(*,*) to the verification
grid.
DEBUG 3: For forecast valid at 20190806_000000, found 1 climatology field(s)
valid at 20190806_000000.
DEBUG 3: Found 1 climatology fields.
DEBUG 2: For /A006 found 1 forecast levels, 1 climatology mean levels, and 0
climatology standard deviation levels.
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match for
VarInfo "/A006" in GRIB record 8 of GRIB file
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.2019080400.004
8".
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB records
matching VarInfo "/A006" in GRIB file
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.2019080400.004
8".
DEBUG 2:
DEBUG 2:
----------------------------------------------------------------------------
----
DEBUG 2:
DEBUG 2: Reading data for /A006.
DEBUG 3: Rescaling probabilistic field from [0,100] to [0,1].
DEBUG 3: Applying conversion function.
DEBUG 1: Regridding climatology CFSR_APCP_T25_00(*,*) to the verification
grid.
DEBUG 3: For forecast valid at 20190806_000000, found 1 climatology field(s)
valid at 20190806_000000.
DEBUG 3: Found 1 climatology fields.
DEBUG 2: For /A006 found 1 forecast levels, 1 climatology mean levels, and 0
climatology standard deviation levels.
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match for
VarInfo "/A006" in GRIB record 9 of GRIB file
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.2019080400.004
8".
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB records
matching VarInfo "/A006" in GRIB file
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.2019080400.004
8".
DEBUG 2:
DEBUG 2:
----------------------------------------------------------------------------
----
DEBUG 2:
DEBUG 2: Reading data for /A006.
DEBUG 3: Rescaling probabilistic field from [0,100] to [0,1].
DEBUG 3: Applying conversion function.
DEBUG 1: Regridding climatology CFSR_APCP_T50_00(*,*) to the verification
grid.
DEBUG 3: For forecast valid at 20190806_000000, found 1 climatology field(s)
valid at 20190806_000000.
DEBUG 3: Found 1 climatology fields.
DEBUG 2: For /A006 found 1 forecast levels, 1 climatology mean levels, and 0
climatology standard deviation levels.
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match for
VarInfo "/A006" in GRIB record 11 of GRIB file
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.2019080400.004
8".
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB records
matching VarInfo "/A006" in GRIB file
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.2019080400.004
8".
DEBUG 2:
DEBUG 2:
----------------------------------------------------------------------------
----
DEBUG 2:
DEBUG 2: Reading data for /A006.
DEBUG 3: Rescaling probabilistic field from [0,100] to [0,1].
DEBUG 3: Applying conversion function.
DEBUG 1: Regridding climatology CFSR_APCP_T100_00(*,*) to the verification
grid.
DEBUG 3: For forecast valid at 20190806_000000, found 1 climatology field(s)
valid at 20190806_000000.
DEBUG 3: Found 1 climatology fields.
DEBUG 2: For /A006 found 1 forecast levels, 1 climatology mean levels, and 0
climatology standard deviation levels.
DEBUG 2:
DEBUG 2:
----------------------------------------------------------------------------
----
DEBUG 2:
DEBUG 2: Searching 1739 observations from 1739 messages.
DEBUG 2:
DEBUG 2:
----------------------------------------------------------------------------
----
DEBUG 2:
WARNING:
WARNING: check_hdr_str() -> null string!
WARNING:
DEBUG 2: Processing /A006 versus APCP_06/L0, for observation type ADPSFC,
over region FULL, for interpolation method NEAREST(1), using 611 pairs.
DEBUG 3: Number of matched pairs = 611
DEBUG 3: Observations processed = 1739
DEBUG 3: Rejected: SID exclusion = 0
DEBUG 3: Rejected: obs type = 0
DEBUG 3: Rejected: valid time = 0
DEBUG 3: Rejected: bad obs value = 0
DEBUG 3: Rejected: off the grid = 1128
DEBUG 3: Rejected: topography = 0
DEBUG 3: Rejected: level mismatch = 0
DEBUG 3: Rejected: quality marker = 0
DEBUG 3: Rejected: message type = 0
DEBUG 3: Rejected: masking region = 0
DEBUG 3: Rejected: bad fcst value = 0
DEBUG 3: Rejected: duplicates = 0
DEBUG 2: Computing Probabilistic Statistics.
DEBUG 2:
DEBUG 2:
----------------------------------------------------------------------------
----
DEBUG 2:
WARNING:
WARNING: check_hdr_str() -> null string!
WARNING:
DEBUG 2: Processing /A006 versus APCP_06/L0, for observation type ADPSFC,
over region FULL, for interpolation method NEAREST(1), using 611 pairs.
DEBUG 3: Number of matched pairs = 611
DEBUG 3: Observations processed = 1739
DEBUG 3: Rejected: SID exclusion = 0
DEBUG 3: Rejected: obs type = 0
DEBUG 3: Rejected: valid time = 0
DEBUG 3: Rejected: bad obs value = 0
DEBUG 3: Rejected: off the grid = 1128
DEBUG 3: Rejected: topography = 0
DEBUG 3: Rejected: level mismatch = 0
DEBUG 3: Rejected: quality marker = 0
DEBUG 3: Rejected: message type = 0
DEBUG 3: Rejected: masking region = 0
DEBUG 3: Rejected: bad fcst value = 0
DEBUG 3: Rejected: duplicates = 0
DEBUG 2: Computing Probabilistic Statistics.
DEBUG 2:
DEBUG 2:
----------------------------------------------------------------------------
----
DEBUG 2:
WARNING:
WARNING: check_hdr_str() -> null string!
WARNING:
DEBUG 2: Processing /A006 versus APCP_06/L0, for observation type ADPSFC,
over region FULL, for interpolation method NEAREST(1), using 611 pairs.
DEBUG 3: Number of matched pairs = 611
DEBUG 3: Observations processed = 1739
DEBUG 3: Rejected: SID exclusion = 0
DEBUG 3: Rejected: obs type = 0
DEBUG 3: Rejected: valid time = 0
DEBUG 3: Rejected: bad obs value = 0
DEBUG 3: Rejected: off the grid = 1128
DEBUG 3: Rejected: topography = 0
DEBUG 3: Rejected: level mismatch = 0
DEBUG 3: Rejected: quality marker = 0
DEBUG 3: Rejected: message type = 0
DEBUG 3: Rejected: masking region = 0
DEBUG 3: Rejected: bad fcst value = 0
DEBUG 3: Rejected: duplicates = 0
DEBUG 2: Computing Probabilistic Statistics.
DEBUG 2:
DEBUG 2:
----------------------------------------------------------------------------
----
DEBUG 2:
WARNING:
WARNING: check_hdr_str() -> null string!
WARNING:
DEBUG 2: Processing /A006 versus APCP_06/L0, for observation type ADPSFC,
over region FULL, for interpolation method NEAREST(1), using 611 pairs.
DEBUG 3: Number of matched pairs = 611
DEBUG 3: Observations processed = 1739
DEBUG 3: Rejected: SID exclusion = 0
DEBUG 3: Rejected: obs type = 0
DEBUG 3: Rejected: valid time = 0
DEBUG 3: Rejected: bad obs value = 0
DEBUG 3: Rejected: off the grid = 1128
DEBUG 3: Rejected: topography = 0
DEBUG 3: Rejected: level mismatch = 0
DEBUG 3: Rejected: quality marker = 0
DEBUG 3: Rejected: message type = 0
DEBUG 3: Rejected: masking region = 0
DEBUG 3: Rejected: bad fcst value = 0
DEBUG 3: Rejected: duplicates = 0
DEBUG 2: Computing Probabilistic Statistics.
DEBUG 2:
DEBUG 2:
----------------------------------------------------------------------------
----
DEBUG 2:
WARNING:
WARNING: check_hdr_str() -> null string!
WARNING:
DEBUG 2: Processing /A006 versus APCP_06/L0, for observation type ADPSFC,
over region FULL, for interpolation method NEAREST(1), using 611 pairs.
DEBUG 3: Number of matched pairs = 611
DEBUG 3: Observations processed = 1739
DEBUG 3: Rejected: SID exclusion = 0
DEBUG 3: Rejected: obs type = 0
DEBUG 3: Rejected: valid time = 0
DEBUG 3: Rejected: bad obs value = 0
DEBUG 3: Rejected: off the grid = 1128
DEBUG 3: Rejected: topography = 0
DEBUG 3: Rejected: level mismatch = 0
DEBUG 3: Rejected: quality marker = 0
DEBUG 3: Rejected: message type = 0
DEBUG 3: Rejected: masking region = 0
DEBUG 3: Rejected: bad fcst value = 0
DEBUG 3: Rejected: duplicates = 0
DEBUG 2: Computing Probabilistic Statistics.
DEBUG 2:
DEBUG 2:
----------------------------------------------------------------------------
----
DEBUG 2:
DEBUG 1: Output file:
/h/data/global/WXQC/data/met/ptstat/mdlob_pairs/meps/TP/point_stat_meps_conu
s_480000L_20190806_000000V.stat
DEBUG 1: Output file:
/h/data/global/WXQC/data/met/ptstat/mdlob_pairs/meps/TP/point_stat_meps_conu
s_480000L_20190806_000000V_pct.txt
DEBUG 1: Output file:
/h/data/global/WXQC/data/met/ptstat/mdlob_pairs/meps/TP/point_stat_meps_conu
s_480000L_20190806_000000V_pstd.txt
Processing 54hour
Thanks
Bob
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Changing Units in STATS files
From: John Halley Gotway
Time: Thu Aug 08 17:06:04 2019
Bob,
Thanks for sending the sample data. Looking at the .stat output file
and
the log messages you sent, I noticed something pretty interesting.
The
forecast variables have an empty "name". You can see this in the log
messages:
* DEBUG 2: Processing /A006 versus APCP_06/L0, ...*
And you can see this in the FCST_VAR output column in the .stat file,
where
FCST_VAR=NA. That means that Point-Stat doesn't have a string to
assign to
your forecast data which makes it pretty difficult to distinguish in
the
output!
Looking in your Point-Stat config file, I see why...
fcst = {
field = [
{ GRIB1_code=206; level = "A006"; prob = TRUE;},
{ GRIB1_code=222; level = "A006"; prob = TRUE;},
{ GRIB1_code=223; level = "A006"; prob = TRUE;},
{ GRIB1_code=224; level = "A006"; prob = TRUE;},
{ GRIB1_code=226; level = "A006"; prob = TRUE;}
];
cat_thresh=[==0.1];
}
You're specifying the GRIB1_code instead of a "name" string. Since I
don't
have your exact input forecast data, I can't exactly replicate this
behavior. I did try to mimic with the data we have here, but wasn't
able
to get the exact behavior. I used TMP and RH, but consistently got
units
of K and %.
Here's what's going on... you request GRIB1_code = 206...
Point-Stat does a table lookup to find a matching line in the GRIB
tables.
Since you didn't specify the parameter table version number, it uses
the
default value of 2. It finds a GRIB table record for ptv = 2 and code
=
206 and uses the units specified on that line.
You'd need to look in your GRIB tables, but perhaps for lines ptv = 2
and
code = 206, 222, 223, 224, and 226, you don't have the same units
values
listed? If you're using a different parameter table version number,
please
specify that... for example...
{ GRIB1_code=206; *GRIB1_ptv=10;* level = "A006"; prob = TRUE;},
So while using these GRIB1 codes to access the data is possible, it
isn't
sufficient to accurately grab the units string from the GRIB1 tables.
A
better alternative would be to just get the typical option of
specifying
the "name" to work well.
Thanks,
John
On Thu, Aug 8, 2019 at 1:52 PM robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> Thu Aug 08 13:51:49 2019: Request 91448 was acted upon.
> Transaction: Ticket created by robert.craig.2 at us.af.mil
> Queue: met_help
> Subject: Changing Units in STATS files
> Owner: Nobody
> Requestors: robert.craig.2 at us.af.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=91448 >
>
>
> John, I am verifying precipitation accumulation probabilities. I am
using
> a
> netcdf file with 6 hour accumulations. When I look through the stat
file
> produced, the units field keeps changing values (ie. w/m^2, gpm,
kg/m^2,
> 0..9,m,.etc) and there will be two unit types for each precip
threshold.
> The listing indicates MET is using the correct grib record number
for the
> variable. Is this the expected behavior or is something not right.
The
> list of the run is below:
>
>
>
> DEBUG 1: Reading user-defined grib1 MET_GRIB_TABLES file:
> /h/data/global/WXQC/data/met/config/grib2/grib1_af_131_57.txt
>
> DEBUG 1: Reading user-defined grib2 MET_GRIB_TABLES file:
> /h/data/global/WXQC/data/met/config/grib2/grib2_geps.txt
>
> DEBUG 1: Default Config File:
> /h/MET/share/met/config/PointStatConfig_default
>
> DEBUG 1: User Config File:
>
>
/h/data/global/WXQC/data/met/config/met_config/PointStatConfig_meps_tp_conus
> _updated
>
> GSL_RNG_TYPE=mt19937
>
> GSL_RNG_SEED=18446744071888718941
>
> DEBUG 1: Forecast File:
>
>
/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.2019080400.0048
>
> DEBUG 1: Observation File:
> /h/data/global/WXQC/data/met/nc_obs/precip/2019080600.nc
>
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match
for
> VarInfo "/A006" in GRIB record 4 of GRIB file
>
>
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.2019080400.004
> 8".
>
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "/A006" in GRIB file
>
>
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.2019080400.004
> 8".
>
> DEBUG 2:
>
> DEBUG 2:
>
>
----------------------------------------------------------------------------
> ----
>
> DEBUG 2:
>
> DEBUG 2: Reading data for /A006.
>
> DEBUG 3: Use the forecast grid.
>
> DEBUG 3: Grid Definition: Projection: Lambert Conformal Nx: 1170 Ny:
720
> Lat_LL: 23.159 Lon_LL: 119.272 Lon_orient: 97.000 Alpha: 3171.947
Cone:
> 0.616 Bx: 582.1130 By: 2385.7494
>
> DEBUG 2: Processing masking regions.
>
> DEBUG 3: Processing grid mask: FULL
>
> DEBUG 2: Processing geography data.
>
> DEBUG 3: Rescaling probabilistic field from [0,100] to [0,1].
>
> DEBUG 3: Applying conversion function.
>
> DEBUG 1: Regridding climatology CFSR_APCP_T01_00(*,*) to the
verification
> grid.
>
> DEBUG 3: For forecast valid at 20190806_000000, found 1 climatology
> field(s)
> valid at 20190806_000000.
>
> DEBUG 3: Found 1 climatology fields.
>
> DEBUG 2: For /A006 found 1 forecast levels, 1 climatology mean
levels, and
> 0
> climatology standard deviation levels.
>
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match
for
> VarInfo "/A006" in GRIB record 7 of GRIB file
>
>
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.2019080400.004
> 8".
>
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "/A006" in GRIB file
>
>
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.2019080400.004
> 8".
>
> DEBUG 2:
>
> DEBUG 2:
>
>
----------------------------------------------------------------------------
> ----
>
> DEBUG 2:
>
> DEBUG 2: Reading data for /A006.
>
> DEBUG 3: Rescaling probabilistic field from [0,100] to [0,1].
>
> DEBUG 3: Applying conversion function.
>
> DEBUG 1: Regridding climatology CFSR_APCP_T10_00(*,*) to the
verification
> grid.
>
> DEBUG 3: For forecast valid at 20190806_000000, found 1 climatology
> field(s)
> valid at 20190806_000000.
>
> DEBUG 3: Found 1 climatology fields.
>
> DEBUG 2: For /A006 found 1 forecast levels, 1 climatology mean
levels, and
> 0
> climatology standard deviation levels.
>
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match
for
> VarInfo "/A006" in GRIB record 8 of GRIB file
>
>
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.2019080400.004
> 8".
>
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "/A006" in GRIB file
>
>
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.2019080400.004
> 8".
>
> DEBUG 2:
>
> DEBUG 2:
>
>
----------------------------------------------------------------------------
> ----
>
> DEBUG 2:
>
> DEBUG 2: Reading data for /A006.
>
> DEBUG 3: Rescaling probabilistic field from [0,100] to [0,1].
>
> DEBUG 3: Applying conversion function.
>
> DEBUG 1: Regridding climatology CFSR_APCP_T25_00(*,*) to the
verification
> grid.
>
> DEBUG 3: For forecast valid at 20190806_000000, found 1 climatology
> field(s)
> valid at 20190806_000000.
>
> DEBUG 3: Found 1 climatology fields.
>
> DEBUG 2: For /A006 found 1 forecast levels, 1 climatology mean
levels, and
> 0
> climatology standard deviation levels.
>
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match
for
> VarInfo "/A006" in GRIB record 9 of GRIB file
>
>
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.2019080400.004
> 8".
>
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "/A006" in GRIB file
>
>
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.2019080400.004
> 8".
>
> DEBUG 2:
>
> DEBUG 2:
>
>
----------------------------------------------------------------------------
> ----
>
> DEBUG 2:
>
> DEBUG 2: Reading data for /A006.
>
> DEBUG 3: Rescaling probabilistic field from [0,100] to [0,1].
>
> DEBUG 3: Applying conversion function.
>
> DEBUG 1: Regridding climatology CFSR_APCP_T50_00(*,*) to the
verification
> grid.
>
> DEBUG 3: For forecast valid at 20190806_000000, found 1 climatology
> field(s)
> valid at 20190806_000000.
>
> DEBUG 3: Found 1 climatology fields.
>
> DEBUG 2: For /A006 found 1 forecast levels, 1 climatology mean
levels, and
> 0
> climatology standard deviation levels.
>
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match
for
> VarInfo "/A006" in GRIB record 11 of GRIB file
>
>
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.2019080400.004
> 8".
>
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "/A006" in GRIB file
>
>
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.2019080400.004
> 8".
>
> DEBUG 2:
>
> DEBUG 2:
>
>
----------------------------------------------------------------------------
> ----
>
> DEBUG 2:
>
> DEBUG 2: Reading data for /A006.
>
> DEBUG 3: Rescaling probabilistic field from [0,100] to [0,1].
>
> DEBUG 3: Applying conversion function.
>
> DEBUG 1: Regridding climatology CFSR_APCP_T100_00(*,*) to the
verification
> grid.
>
> DEBUG 3: For forecast valid at 20190806_000000, found 1 climatology
> field(s)
> valid at 20190806_000000.
>
> DEBUG 3: Found 1 climatology fields.
>
> DEBUG 2: For /A006 found 1 forecast levels, 1 climatology mean
levels, and
> 0
> climatology standard deviation levels.
>
> DEBUG 2:
>
> DEBUG 2:
>
>
----------------------------------------------------------------------------
> ----
>
> DEBUG 2:
>
> DEBUG 2: Searching 1739 observations from 1739 messages.
>
> DEBUG 2:
>
> DEBUG 2:
>
>
----------------------------------------------------------------------------
> ----
>
> DEBUG 2:
>
> WARNING:
>
> WARNING: check_hdr_str() -> null string!
>
> WARNING:
>
> DEBUG 2: Processing /A006 versus APCP_06/L0, for observation type
ADPSFC,
> over region FULL, for interpolation method NEAREST(1), using 611
pairs.
>
> DEBUG 3: Number of matched pairs = 611
>
> DEBUG 3: Observations processed = 1739
>
> DEBUG 3: Rejected: SID exclusion = 0
>
> DEBUG 3: Rejected: obs type = 0
>
> DEBUG 3: Rejected: valid time = 0
>
> DEBUG 3: Rejected: bad obs value = 0
>
> DEBUG 3: Rejected: off the grid = 1128
>
> DEBUG 3: Rejected: topography = 0
>
> DEBUG 3: Rejected: level mismatch = 0
>
> DEBUG 3: Rejected: quality marker = 0
>
> DEBUG 3: Rejected: message type = 0
>
> DEBUG 3: Rejected: masking region = 0
>
> DEBUG 3: Rejected: bad fcst value = 0
>
> DEBUG 3: Rejected: duplicates = 0
>
> DEBUG 2: Computing Probabilistic Statistics.
>
> DEBUG 2:
>
> DEBUG 2:
>
>
----------------------------------------------------------------------------
> ----
>
> DEBUG 2:
>
> WARNING:
>
> WARNING: check_hdr_str() -> null string!
>
> WARNING:
>
> DEBUG 2: Processing /A006 versus APCP_06/L0, for observation type
ADPSFC,
> over region FULL, for interpolation method NEAREST(1), using 611
pairs.
>
> DEBUG 3: Number of matched pairs = 611
>
> DEBUG 3: Observations processed = 1739
>
> DEBUG 3: Rejected: SID exclusion = 0
>
> DEBUG 3: Rejected: obs type = 0
>
> DEBUG 3: Rejected: valid time = 0
>
> DEBUG 3: Rejected: bad obs value = 0
>
> DEBUG 3: Rejected: off the grid = 1128
>
> DEBUG 3: Rejected: topography = 0
>
> DEBUG 3: Rejected: level mismatch = 0
>
> DEBUG 3: Rejected: quality marker = 0
>
> DEBUG 3: Rejected: message type = 0
>
> DEBUG 3: Rejected: masking region = 0
>
> DEBUG 3: Rejected: bad fcst value = 0
>
> DEBUG 3: Rejected: duplicates = 0
>
> DEBUG 2: Computing Probabilistic Statistics.
>
> DEBUG 2:
>
> DEBUG 2:
>
>
----------------------------------------------------------------------------
> ----
>
> DEBUG 2:
>
> WARNING:
>
> WARNING: check_hdr_str() -> null string!
>
> WARNING:
>
> DEBUG 2: Processing /A006 versus APCP_06/L0, for observation type
ADPSFC,
> over region FULL, for interpolation method NEAREST(1), using 611
pairs.
>
> DEBUG 3: Number of matched pairs = 611
>
> DEBUG 3: Observations processed = 1739
>
> DEBUG 3: Rejected: SID exclusion = 0
>
> DEBUG 3: Rejected: obs type = 0
>
> DEBUG 3: Rejected: valid time = 0
>
> DEBUG 3: Rejected: bad obs value = 0
>
> DEBUG 3: Rejected: off the grid = 1128
>
> DEBUG 3: Rejected: topography = 0
>
> DEBUG 3: Rejected: level mismatch = 0
>
> DEBUG 3: Rejected: quality marker = 0
>
> DEBUG 3: Rejected: message type = 0
>
> DEBUG 3: Rejected: masking region = 0
>
> DEBUG 3: Rejected: bad fcst value = 0
>
> DEBUG 3: Rejected: duplicates = 0
>
> DEBUG 2: Computing Probabilistic Statistics.
>
> DEBUG 2:
>
> DEBUG 2:
>
>
----------------------------------------------------------------------------
> ----
>
> DEBUG 2:
>
> WARNING:
>
> WARNING: check_hdr_str() -> null string!
>
> WARNING:
>
> DEBUG 2: Processing /A006 versus APCP_06/L0, for observation type
ADPSFC,
> over region FULL, for interpolation method NEAREST(1), using 611
pairs.
>
> DEBUG 3: Number of matched pairs = 611
>
> DEBUG 3: Observations processed = 1739
>
> DEBUG 3: Rejected: SID exclusion = 0
>
> DEBUG 3: Rejected: obs type = 0
>
> DEBUG 3: Rejected: valid time = 0
>
> DEBUG 3: Rejected: bad obs value = 0
>
> DEBUG 3: Rejected: off the grid = 1128
>
> DEBUG 3: Rejected: topography = 0
>
> DEBUG 3: Rejected: level mismatch = 0
>
> DEBUG 3: Rejected: quality marker = 0
>
> DEBUG 3: Rejected: message type = 0
>
> DEBUG 3: Rejected: masking region = 0
>
> DEBUG 3: Rejected: bad fcst value = 0
>
> DEBUG 3: Rejected: duplicates = 0
>
> DEBUG 2: Computing Probabilistic Statistics.
>
> DEBUG 2:
>
> DEBUG 2:
>
>
----------------------------------------------------------------------------
> ----
>
> DEBUG 2:
>
> WARNING:
>
> WARNING: check_hdr_str() -> null string!
>
> WARNING:
>
> DEBUG 2: Processing /A006 versus APCP_06/L0, for observation type
ADPSFC,
> over region FULL, for interpolation method NEAREST(1), using 611
pairs.
>
> DEBUG 3: Number of matched pairs = 611
>
> DEBUG 3: Observations processed = 1739
>
> DEBUG 3: Rejected: SID exclusion = 0
>
> DEBUG 3: Rejected: obs type = 0
>
> DEBUG 3: Rejected: valid time = 0
>
> DEBUG 3: Rejected: bad obs value = 0
>
> DEBUG 3: Rejected: off the grid = 1128
>
> DEBUG 3: Rejected: topography = 0
>
> DEBUG 3: Rejected: level mismatch = 0
>
> DEBUG 3: Rejected: quality marker = 0
>
> DEBUG 3: Rejected: message type = 0
>
> DEBUG 3: Rejected: masking region = 0
>
> DEBUG 3: Rejected: bad fcst value = 0
>
> DEBUG 3: Rejected: duplicates = 0
>
> DEBUG 2: Computing Probabilistic Statistics.
>
> DEBUG 2:
>
> DEBUG 2:
>
>
----------------------------------------------------------------------------
> ----
>
> DEBUG 2:
>
> DEBUG 1: Output file:
>
>
/h/data/global/WXQC/data/met/ptstat/mdlob_pairs/meps/TP/point_stat_meps_conu
> s_480000L_20190806_000000V.stat
>
> DEBUG 1: Output file:
>
>
/h/data/global/WXQC/data/met/ptstat/mdlob_pairs/meps/TP/point_stat_meps_conu
> s_480000L_20190806_000000V_pct.txt
>
> DEBUG 1: Output file:
>
>
/h/data/global/WXQC/data/met/ptstat/mdlob_pairs/meps/TP/point_stat_meps_conu
> s_480000L_20190806_000000V_pstd.txt
>
> Processing 54hour
>
>
>
> Thanks
>
> Bob
>
>
>
------------------------------------------------
Subject: Changing Units in STATS files
From: robert.craig.2 at us.af.mil
Time: Fri Aug 09 07:35:43 2019
John, I had tried the name option (name="PQP1";) before but MET gives
me the
following:
Processing 6hour
Using forecast file ->
/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080500/grib.2019080500.0006
Using observation file ->
/h/data/global/WXQC/data/met/nc_obs/precip/2019080506.nc
Using climo file ->
/h/data/global/WXQC/data/climo/cfsr/cfsr_freq_p6_world_mo_8.nc
Using config file ->
/h/data/global/WXQC/data/met/config/met_config/PointStatConfig_meps_tp_conus
Running MET command string -> ['/h/MET/bin/point_stat',
'/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080500/grib.2019080500.0006',
'/h/data/global/WXQC/data/met/nc_obs/precip/2019080506.nc',
'/h/data/global/WXQC/data/met/config/met_config/PointStatConfig_meps_tp_conus_updated',
'-outdir', '/h/data/global/WXQC/data/met/ptstat/mdlob_pairs/meps/TP',
'-v',
'3']
DEBUG 1: Reading user-defined grib1 MET_GRIB_TABLES file:
/h/data/global/WXQC/data/met/config/grib2/grib1_af_131_57.txt
DEBUG 1: Reading user-defined grib2 MET_GRIB_TABLES file:
/h/data/global/WXQC/data/met/config/grib2/grib2_geps.txt
DEBUG 1: Default Config File:
/h/MET/share/met/config/PointStatConfig_default
DEBUG 1: User Config File:
/h/data/global/WXQC/data/met/config/met_config/PointStatConfig_meps_tp_conus_updated
GSL_RNG_TYPE=mt19937
GSL_RNG_SEED=18446744071988989277
DEBUG 1: Forecast File:
/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080500/grib.2019080500.0006
DEBUG 1: Observation File:
/h/data/global/WXQC/data/met/nc_obs/precip/2019080506.nc
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 0 GRIB records
matching
VarInfo "PQP1/A006" in GRIB file
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080500/grib.2019080500.0006".
I attached the grib1 table I am using. I added PQP1 to it. I also
added the
GRIB1_ptv number though I am not sure if it is 57 or 131 - I tried
both to no
avail. The grib1 data file is in AMERDEC for you to look at and the
ob file
was attached to yesterday's email. Normally, this ensemble
probability data
would be in GRIB2 but for the short term it's going to be grib1 so I
have to
make this work.
What am I doing wrong?
Thanks
Bob
-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Thursday, August 8, 2019 6:06 PM
To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
<robert.craig.2 at us.af.mil>
Cc: SITTEL, MATTHEW C CTR USAF AFMC AFLCMC/HBAW-OL
<matthew.sittel.ctr at us.af.mil>
Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #91448] Changing Units
in STATS
files
Bob,
Thanks for sending the sample data. Looking at the .stat output file
and the
log messages you sent, I noticed something pretty interesting. The
forecast
variables have an empty "name". You can see this in the log
messages:
* DEBUG 2: Processing /A006 versus APCP_06/L0, ...*
And you can see this in the FCST_VAR output column in the .stat file,
where
FCST_VAR=NA. That means that Point-Stat doesn't have a string to
assign to
your forecast data which makes it pretty difficult to distinguish in
the
output!
Looking in your Point-Stat config file, I see why...
fcst = {
field = [
{ GRIB1_code=206; level = "A006"; prob = TRUE;},
{ GRIB1_code=222; level = "A006"; prob = TRUE;},
{ GRIB1_code=223; level = "A006"; prob = TRUE;},
{ GRIB1_code=224; level = "A006"; prob = TRUE;},
{ GRIB1_code=226; level = "A006"; prob = TRUE;}
];
cat_thresh=[==0.1];
}
You're specifying the GRIB1_code instead of a "name" string. Since I
don't
have your exact input forecast data, I can't exactly replicate this
behavior.
I did try to mimic with the data we have here, but wasn't able to get
the
exact behavior. I used TMP and RH, but consistently got units of K
and %.
Here's what's going on... you request GRIB1_code = 206...
Point-Stat does a table lookup to find a matching line in the GRIB
tables.
Since you didn't specify the parameter table version number, it uses
the
default value of 2. It finds a GRIB table record for ptv = 2 and code
=
206 and uses the units specified on that line.
You'd need to look in your GRIB tables, but perhaps for lines ptv = 2
and code
= 206, 222, 223, 224, and 226, you don't have the same units values
listed?
If you're using a different parameter table version number, please
specify
that... for example...
{ GRIB1_code=206; *GRIB1_ptv=10;* level = "A006"; prob = TRUE;},
So while using these GRIB1 codes to access the data is possible, it
isn't
sufficient to accurately grab the units string from the GRIB1 tables.
A
better alternative would be to just get the typical option of
specifying the
"name" to work well.
Thanks,
John
On Thu, Aug 8, 2019 at 1:52 PM robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> Thu Aug 08 13:51:49 2019: Request 91448 was acted upon.
> Transaction: Ticket created by robert.craig.2 at us.af.mil
> Queue: met_help
> Subject: Changing Units in STATS files
> Owner: Nobody
> Requestors: robert.craig.2 at us.af.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=91448
> >
>
>
> John, I am verifying precipitation accumulation probabilities. I am
> using a netcdf file with 6 hour accumulations. When I look through
> the stat file produced, the units field keeps changing values (ie.
> w/m^2, gpm, kg/m^2,
> 0..9,m,.etc) and there will be two unit types for each precip
threshold.
> The listing indicates MET is using the correct grib record number
for the
> variable. Is this the expected behavior or is something not right.
The
> list of the run is below:
>
>
>
> DEBUG 1: Reading user-defined grib1 MET_GRIB_TABLES file:
> /h/data/global/WXQC/data/met/config/grib2/grib1_af_131_57.txt
>
> DEBUG 1: Reading user-defined grib2 MET_GRIB_TABLES file:
> /h/data/global/WXQC/data/met/config/grib2/grib2_geps.txt
>
> DEBUG 1: Default Config File:
> /h/MET/share/met/config/PointStatConfig_default
>
> DEBUG 1: User Config File:
>
>
/h/data/global/WXQC/data/met/config/met_config/PointStatConfig_meps_tp
> _conus
> _updated
>
> GSL_RNG_TYPE=mt19937
>
> GSL_RNG_SEED=18446744071888718941
>
> DEBUG 1: Forecast File:
>
>
/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.201908040
> 0.0048
>
> DEBUG 1: Observation File:
> /h/data/global/WXQC/data/met/nc_obs/precip/2019080600.nc
>
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match
for
> VarInfo "/A006" in GRIB record 4 of GRIB file
>
>
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.20190804
> 00.004
> 8".
>
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "/A006" in GRIB file
>
>
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.20190804
> 00.004
> 8".
>
> DEBUG 2:
>
> DEBUG 2:
>
>
----------------------------------------------------------------------
> ------
> ----
>
> DEBUG 2:
>
> DEBUG 2: Reading data for /A006.
>
> DEBUG 3: Use the forecast grid.
>
> DEBUG 3: Grid Definition: Projection: Lambert Conformal Nx: 1170 Ny:
> 720
> Lat_LL: 23.159 Lon_LL: 119.272 Lon_orient: 97.000 Alpha: 3171.947
Cone:
> 0.616 Bx: 582.1130 By: 2385.7494
>
> DEBUG 2: Processing masking regions.
>
> DEBUG 3: Processing grid mask: FULL
>
> DEBUG 2: Processing geography data.
>
> DEBUG 3: Rescaling probabilistic field from [0,100] to [0,1].
>
> DEBUG 3: Applying conversion function.
>
> DEBUG 1: Regridding climatology CFSR_APCP_T01_00(*,*) to the
> verification grid.
>
> DEBUG 3: For forecast valid at 20190806_000000, found 1 climatology
> field(s)
> valid at 20190806_000000.
>
> DEBUG 3: Found 1 climatology fields.
>
> DEBUG 2: For /A006 found 1 forecast levels, 1 climatology mean
levels,
> and
> 0
> climatology standard deviation levels.
>
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match
for
> VarInfo "/A006" in GRIB record 7 of GRIB file
>
>
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.20190804
> 00.004
> 8".
>
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "/A006" in GRIB file
>
>
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.20190804
> 00.004
> 8".
>
> DEBUG 2:
>
> DEBUG 2:
>
>
----------------------------------------------------------------------
> ------
> ----
>
> DEBUG 2:
>
> DEBUG 2: Reading data for /A006.
>
> DEBUG 3: Rescaling probabilistic field from [0,100] to [0,1].
>
> DEBUG 3: Applying conversion function.
>
> DEBUG 1: Regridding climatology CFSR_APCP_T10_00(*,*) to the
> verification grid.
>
> DEBUG 3: For forecast valid at 20190806_000000, found 1 climatology
> field(s)
> valid at 20190806_000000.
>
> DEBUG 3: Found 1 climatology fields.
>
> DEBUG 2: For /A006 found 1 forecast levels, 1 climatology mean
levels,
> and
> 0
> climatology standard deviation levels.
>
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match
for
> VarInfo "/A006" in GRIB record 8 of GRIB file
>
>
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.20190804
> 00.004
> 8".
>
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "/A006" in GRIB file
>
>
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.20190804
> 00.004
> 8".
>
> DEBUG 2:
>
> DEBUG 2:
>
>
----------------------------------------------------------------------
> ------
> ----
>
> DEBUG 2:
>
> DEBUG 2: Reading data for /A006.
>
> DEBUG 3: Rescaling probabilistic field from [0,100] to [0,1].
>
> DEBUG 3: Applying conversion function.
>
> DEBUG 1: Regridding climatology CFSR_APCP_T25_00(*,*) to the
> verification grid.
>
> DEBUG 3: For forecast valid at 20190806_000000, found 1 climatology
> field(s)
> valid at 20190806_000000.
>
> DEBUG 3: Found 1 climatology fields.
>
> DEBUG 2: For /A006 found 1 forecast levels, 1 climatology mean
levels,
> and
> 0
> climatology standard deviation levels.
>
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match
for
> VarInfo "/A006" in GRIB record 9 of GRIB file
>
>
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.20190804
> 00.004
> 8".
>
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "/A006" in GRIB file
>
>
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.20190804
> 00.004
> 8".
>
> DEBUG 2:
>
> DEBUG 2:
>
>
----------------------------------------------------------------------
> ------
> ----
>
> DEBUG 2:
>
> DEBUG 2: Reading data for /A006.
>
> DEBUG 3: Rescaling probabilistic field from [0,100] to [0,1].
>
> DEBUG 3: Applying conversion function.
>
> DEBUG 1: Regridding climatology CFSR_APCP_T50_00(*,*) to the
> verification grid.
>
> DEBUG 3: For forecast valid at 20190806_000000, found 1 climatology
> field(s)
> valid at 20190806_000000.
>
> DEBUG 3: Found 1 climatology fields.
>
> DEBUG 2: For /A006 found 1 forecast levels, 1 climatology mean
levels,
> and
> 0
> climatology standard deviation levels.
>
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match
for
> VarInfo "/A006" in GRIB record 11 of GRIB file
>
>
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.20190804
> 00.004
> 8".
>
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "/A006" in GRIB file
>
>
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.20190804
> 00.004
> 8".
>
> DEBUG 2:
>
> DEBUG 2:
>
>
----------------------------------------------------------------------
> ------
> ----
>
> DEBUG 2:
>
> DEBUG 2: Reading data for /A006.
>
> DEBUG 3: Rescaling probabilistic field from [0,100] to [0,1].
>
> DEBUG 3: Applying conversion function.
>
> DEBUG 1: Regridding climatology CFSR_APCP_T100_00(*,*) to the
> verification grid.
>
> DEBUG 3: For forecast valid at 20190806_000000, found 1 climatology
> field(s)
> valid at 20190806_000000.
>
> DEBUG 3: Found 1 climatology fields.
>
> DEBUG 2: For /A006 found 1 forecast levels, 1 climatology mean
levels,
> and
> 0
> climatology standard deviation levels.
>
> DEBUG 2:
>
> DEBUG 2:
>
>
----------------------------------------------------------------------
> ------
> ----
>
> DEBUG 2:
>
> DEBUG 2: Searching 1739 observations from 1739 messages.
>
> DEBUG 2:
>
> DEBUG 2:
>
>
----------------------------------------------------------------------
> ------
> ----
>
> DEBUG 2:
>
> WARNING:
>
> WARNING: check_hdr_str() -> null string!
>
> WARNING:
>
> DEBUG 2: Processing /A006 versus APCP_06/L0, for observation type
> ADPSFC, over region FULL, for interpolation method NEAREST(1), using
611
> pairs.
>
> DEBUG 3: Number of matched pairs = 611
>
> DEBUG 3: Observations processed = 1739
>
> DEBUG 3: Rejected: SID exclusion = 0
>
> DEBUG 3: Rejected: obs type = 0
>
> DEBUG 3: Rejected: valid time = 0
>
> DEBUG 3: Rejected: bad obs value = 0
>
> DEBUG 3: Rejected: off the grid = 1128
>
> DEBUG 3: Rejected: topography = 0
>
> DEBUG 3: Rejected: level mismatch = 0
>
> DEBUG 3: Rejected: quality marker = 0
>
> DEBUG 3: Rejected: message type = 0
>
> DEBUG 3: Rejected: masking region = 0
>
> DEBUG 3: Rejected: bad fcst value = 0
>
> DEBUG 3: Rejected: duplicates = 0
>
> DEBUG 2: Computing Probabilistic Statistics.
>
> DEBUG 2:
>
> DEBUG 2:
>
>
----------------------------------------------------------------------
> ------
> ----
>
> DEBUG 2:
>
> WARNING:
>
> WARNING: check_hdr_str() -> null string!
>
> WARNING:
>
> DEBUG 2: Processing /A006 versus APCP_06/L0, for observation type
> ADPSFC, over region FULL, for interpolation method NEAREST(1), using
611
> pairs.
>
> DEBUG 3: Number of matched pairs = 611
>
> DEBUG 3: Observations processed = 1739
>
> DEBUG 3: Rejected: SID exclusion = 0
>
> DEBUG 3: Rejected: obs type = 0
>
> DEBUG 3: Rejected: valid time = 0
>
> DEBUG 3: Rejected: bad obs value = 0
>
> DEBUG 3: Rejected: off the grid = 1128
>
> DEBUG 3: Rejected: topography = 0
>
> DEBUG 3: Rejected: level mismatch = 0
>
> DEBUG 3: Rejected: quality marker = 0
>
> DEBUG 3: Rejected: message type = 0
>
> DEBUG 3: Rejected: masking region = 0
>
> DEBUG 3: Rejected: bad fcst value = 0
>
> DEBUG 3: Rejected: duplicates = 0
>
> DEBUG 2: Computing Probabilistic Statistics.
>
> DEBUG 2:
>
> DEBUG 2:
>
>
----------------------------------------------------------------------
> ------
> ----
>
> DEBUG 2:
>
> WARNING:
>
> WARNING: check_hdr_str() -> null string!
>
> WARNING:
>
> DEBUG 2: Processing /A006 versus APCP_06/L0, for observation type
> ADPSFC, over region FULL, for interpolation method NEAREST(1), using
611
> pairs.
>
> DEBUG 3: Number of matched pairs = 611
>
> DEBUG 3: Observations processed = 1739
>
> DEBUG 3: Rejected: SID exclusion = 0
>
> DEBUG 3: Rejected: obs type = 0
>
> DEBUG 3: Rejected: valid time = 0
>
> DEBUG 3: Rejected: bad obs value = 0
>
> DEBUG 3: Rejected: off the grid = 1128
>
> DEBUG 3: Rejected: topography = 0
>
> DEBUG 3: Rejected: level mismatch = 0
>
> DEBUG 3: Rejected: quality marker = 0
>
> DEBUG 3: Rejected: message type = 0
>
> DEBUG 3: Rejected: masking region = 0
>
> DEBUG 3: Rejected: bad fcst value = 0
>
> DEBUG 3: Rejected: duplicates = 0
>
> DEBUG 2: Computing Probabilistic Statistics.
>
> DEBUG 2:
>
> DEBUG 2:
>
>
----------------------------------------------------------------------
> ------
> ----
>
> DEBUG 2:
>
> WARNING:
>
> WARNING: check_hdr_str() -> null string!
>
> WARNING:
>
> DEBUG 2: Processing /A006 versus APCP_06/L0, for observation type
> ADPSFC, over region FULL, for interpolation method NEAREST(1), using
611
> pairs.
>
> DEBUG 3: Number of matched pairs = 611
>
> DEBUG 3: Observations processed = 1739
>
> DEBUG 3: Rejected: SID exclusion = 0
>
> DEBUG 3: Rejected: obs type = 0
>
> DEBUG 3: Rejected: valid time = 0
>
> DEBUG 3: Rejected: bad obs value = 0
>
> DEBUG 3: Rejected: off the grid = 1128
>
> DEBUG 3: Rejected: topography = 0
>
> DEBUG 3: Rejected: level mismatch = 0
>
> DEBUG 3: Rejected: quality marker = 0
>
> DEBUG 3: Rejected: message type = 0
>
> DEBUG 3: Rejected: masking region = 0
>
> DEBUG 3: Rejected: bad fcst value = 0
>
> DEBUG 3: Rejected: duplicates = 0
>
> DEBUG 2: Computing Probabilistic Statistics.
>
> DEBUG 2:
>
> DEBUG 2:
>
>
----------------------------------------------------------------------
> ------
> ----
>
> DEBUG 2:
>
> WARNING:
>
> WARNING: check_hdr_str() -> null string!
>
> WARNING:
>
> DEBUG 2: Processing /A006 versus APCP_06/L0, for observation type
> ADPSFC, over region FULL, for interpolation method NEAREST(1), using
611
> pairs.
>
> DEBUG 3: Number of matched pairs = 611
>
> DEBUG 3: Observations processed = 1739
>
> DEBUG 3: Rejected: SID exclusion = 0
>
> DEBUG 3: Rejected: obs type = 0
>
> DEBUG 3: Rejected: valid time = 0
>
> DEBUG 3: Rejected: bad obs value = 0
>
> DEBUG 3: Rejected: off the grid = 1128
>
> DEBUG 3: Rejected: topography = 0
>
> DEBUG 3: Rejected: level mismatch = 0
>
> DEBUG 3: Rejected: quality marker = 0
>
> DEBUG 3: Rejected: message type = 0
>
> DEBUG 3: Rejected: masking region = 0
>
> DEBUG 3: Rejected: bad fcst value = 0
>
> DEBUG 3: Rejected: duplicates = 0
>
> DEBUG 2: Computing Probabilistic Statistics.
>
> DEBUG 2:
>
> DEBUG 2:
>
>
----------------------------------------------------------------------
> ------
> ----
>
> DEBUG 2:
>
> DEBUG 1: Output file:
>
>
/h/data/global/WXQC/data/met/ptstat/mdlob_pairs/meps/TP/point_stat_mep
> s_conu
> s_480000L_20190806_000000V.stat
>
> DEBUG 1: Output file:
>
>
/h/data/global/WXQC/data/met/ptstat/mdlob_pairs/meps/TP/point_stat_mep
> s_conu
> s_480000L_20190806_000000V_pct.txt
>
> DEBUG 1: Output file:
>
>
/h/data/global/WXQC/data/met/ptstat/mdlob_pairs/meps/TP/point_stat_mep
> s_conu
> s_480000L_20190806_000000V_pstd.txt
>
> Processing 54hour
>
>
>
> Thanks
>
> Bob
>
>
>
------------------------------------------------
Subject: Changing Units in STATS files
From: robert.craig.2 at us.af.mil
Time: Fri Aug 09 07:35:43 2019
GRIB1
0 131 57 2 "var0" "undefined" ""
1 131 57 2 "PRES" "Pressure" "Pa"
2 131 57 2 "PRMSL" "Pressure reduced to MSL" "Pa"
3 131 57 2 "PTEND" "Pressure tendency" "Pa/s"
4 131 57 2 "var4" "undefined" ""
5 131 57 2 "var5" "undefined" ""
6 131 57 2 "GP" "Geopotential" "m^2/s^2"
7 131 57 2 "HGT" "Geopotential height" "gpm"
8 131 57 2 "DIST" "Geometric height" "m"
9 131 57 2 "HSTDV" "Std dev of height" "m"
10 131 57 2 "HVAR" "Variance of height" "m^2"
11 131 57 2 "TMP" "Temp." "K"
12 131 57 2 "VTMP" "Virtual temp." "K"
13 131 57 2 "POT" "Potential temp." "K"
14 131 57 2 "EPOT" "Pseudo-adiabatic pot. temp." "K"
15 131 57 2 "TMAX" "Max. temp." "K"
16 131 57 2 "TMIN" "Min. temp." "K"
17 131 57 2 "DPT" "Dew point temp." "K"
18 131 57 2 "DEPR" "Dew point depression" "K"
19 131 57 2 "LAPR" "Lapse rate" "K/m"
20 131 57 2 "VISIB" "Visibility" "m"
21 131 57 2 "RDSP1" "Radar spectra (1)" "non-dim"
22 131 57 2 "RDSP2" "Radar spectra (2)" "non-dim"
23 131 57 2 "RDSP3" "Radar spectra (3)" "non-dim"
24 131 57 2 "var24" "undefined" ""
25 131 57 2 "TMPA" "Temp. anomaly" "K"
26 131 57 2 "PRESA" "Pressure anomaly" "Pa"
27 131 57 2 "GPA" "Geopotential height anomaly" "gpm"
28 131 57 2 "WVSP1" "Wave spectra (1)" "non-dim"
29 131 57 2 "WVSP2" "Wave spectra (2)" "non-dim"
30 131 57 2 "WVSP3" "Wave spectra (3)" "non-dim"
31 131 57 2 "WDIR" "Wind direction" "deg"
32 131 57 2 "WIND" "Wind speed" "m/s"
33 131 57 2 "UGRD" "u wind" "m/s"
34 131 57 2 "VGRD" "v wind" "m/s"
35 131 57 2 "STRM" "Stream function" "m^2/s"
36 131 57 2 "VPOT" "Velocity potential" "m^2/s"
37 131 57 2 "MNTSF" "Montgomery stream function" "m^2/s^2"
38 131 57 2 "SGCVV" "Sigma coord. vertical velocity" "/s"
39 131 57 2 "VVEL" "Pressure vertical velocity" "Pa/s"
40 131 57 2 "DZDT" "Geometric vertical velocity" "m/s"
41 131 57 2 "ABSV" "Absolute vorticity" "/s"
42 131 57 2 "ABSD" "Absolute divergence" "/s"
43 131 57 2 "RELV" "Relative vorticity" "/s"
44 131 57 2 "RELD" "Relative divergence" "/s"
45 131 57 2 "VUCSH" "Vertical u shear" "/s"
46 131 57 2 "VVCSH" "Vertical v shear" "/s"
47 131 57 2 "DIRC" "Direction of current" "deg"
48 131 57 2 "SPC" "Speed of current" "m/s"
49 131 57 2 "UOGRD" "u of current" "m/s"
50 131 57 2 "VOGRD" "v of current" "m/s"
51 131 57 2 "SPFH" "Specific humidity" "kg/kg"
52 131 57 2 "RH" "Relative humidity" "%"
53 131 57 2 "MIXR" "Humidity mixing ratio" "kg/kg"
54 131 57 2 "PWAT" "Precipitable water" "kg/m^2"
55 131 57 2 "VAPP" "Vapor pressure" "Pa"
56 131 57 2 "SATD" "Saturation deficit" "Pa"
57 131 57 2 "EVP" "Evaporation" "kg/m^2"
58 131 57 2 "CICE" "Cloud Ice" "kg/kg"
59 131 57 2 "PRATE" "Precipitation rate" "kg/m^2/s"
60 131 57 2 "TSTM" "Thunderstorm probability" "%"
61 131 57 2 "APCP" "Total precipitation" "kg/m^2"
62 131 57 2 "NCPCP" "Large scale precipitation" "kg/m^2"
63 131 57 2 "ACPCP" "Convective precipitation" "kg/m^2"
64 131 57 2 "SRWEQ" "Snowfall rate water equiv." "kg/m^2/s"
65 131 57 2 "WEASD" "Accum. snow" "kg/m^2"
66 131 57 2 "SNOD" "Snow depth" "m"
67 131 57 2 "MIXHT" "Mixed layer depth" "m"
68 131 57 2 "TTHDP" "Transient thermocline depth" "m"
69 131 57 2 "MTHD" "Main thermocline depth" "m"
70 131 57 2 "MTHA" "Main thermocline anomaly" "m"
71 131 57 2 "TCDC" "Total cloud cover" "%"
72 131 57 2 "CDCON" "Convective cloud cover" "%"
73 131 57 2 "LCDC" "Low level cloud cover" "%"
74 131 57 2 "MCDC" "Mid level cloud cover" "%"
75 131 57 2 "HCDC" "High level cloud cover" "%"
76 131 57 2 "CWAT" "Cloud water" "kg/m^2"
77 131 57 2 "var77" "undefined" ""
78 131 57 2 "SNOC" "Convective snow" "kg/m^2"
79 131 57 2 "SNOL" "Large scale snow" "kg/m^2"
80 131 57 2 "WTMP" "Water temp." "K"
81 131 57 2 "LAND" "Land-sea mask" "1=land; 0=sea"
82 131 57 2 "DSLM" "Deviation of sea level from mean" "m"
83 131 57 2 "SFCR" "Surface roughness" "m"
84 131 57 2 "ALBDO" "Albedo" "%"
85 131 57 2 "TSOIL" "Soil temp." "K"
86 131 57 2 "SOILM" "Soil moisture content" "kg/m^2"
87 131 57 2 "VEG" "Vegetation" "%"
88 131 57 2 "SALTY" "Salinity" "kg/kg"
89 131 57 2 "DEN" "Density" "kg/m^3"
90 131 57 2 "RUNOF" "Runoff" "kg/m^2"
91 131 57 2 "ICEC" "Ice concentration" "ice=1;no ice=0"
92 131 57 2 "ICETK" "Ice thickness" "m"
93 131 57 2 "DICED" "Direction of ice drift" "deg"
94 131 57 2 "SICED" "Speed of ice drift" "m/s"
95 131 57 2 "UICE" "u of ice drift" "m/s"
96 131 57 2 "VICE" "v of ice drift" "m/s"
97 131 57 2 "ICEG" "Ice growth rate" "m/s"
98 131 57 2 "ICED" "Ice divergence" "/s"
99 131 57 2 "SNOM" "Snow melt" "kg/m^2"
100 131 57 2 "HTSGW" "Sig height of wind waves and swell" "m"
101 131 57 2 "WVDIR" "Direction of wind waves" "deg"
102 131 57 2 "WVHGT" "Sig height of wind waves" "m"
103 131 57 2 "WVPER" "Mean period of wind waves" "s"
104 131 57 2 "SWDIR" "Direction of swell waves" "deg"
105 131 57 2 "SWELL" "Sig height of swell waves" "m"
106 131 57 2 "SWPER" "Mean period of swell waves" "s"
107 131 57 2 "DIRPW" "Primary wave direction" "deg"
108 131 57 2 "PERPW" "Primary wave mean period" "s"
109 131 57 2 "DIRSW" "Secondary wave direction" "deg"
110 131 57 2 "PERSW" "Secondary wave mean period" "s"
111 131 57 2 "NSWRS" "Net short wave (surface)" "W/m^2"
112 131 57 2 "NLWRS" "Net long wave (surface)" "W/m^2"
113 131 57 2 "NSWRT" "Net short wave (top)" "W/m^2"
114 131 57 2 "NLWRT" "Net long wave (top)" "W/m^2"
115 131 57 2 "LWAVR" "Long wave" "W/m^2"
116 131 57 2 "SWAVR" "Short wave" "W/m^2"
117 131 57 2 "GRAD" "Global radiation" "W/m^2"
118 131 57 2 "var118" "undefined" ""
119 131 57 2 "var119" "undefined" ""
120 131 57 2 "var120" "undefined" ""
121 131 57 2 "LHTFL" "Latent heat flux" "W/m^2"
122 131 57 2 "SHTFL" "Sensible heat flux" "W/m^2"
123 131 57 2 "BLYDP" "Boundary layer dissipation" "W/m^2"a
124 131 57 2 "UFLX" "Zonal momentum flux" "N/m^2"
125 131 57 2 "VFLX" "Meridional momentum flux" "N/m^2"
126 131 57 2 "WMIXE" "Wind mixing energy" "J"
127 131 57 2 "IMGD" "Image data" "integer"
128 131 57 2 "iceCoverage" "Ice Concentration" "%"
129 131 57 2 "iceAge" "Ice Age" "days"
130 131 57 2 "CIG3000" "Prob Ceiling < 3000" "%"
131 131 57 2 "CIG1000" "PRob Ceiling < 1000" "%"
132 131 57 2 "CIG500" "Prob Ceiling < 500" "%"
133 131 57 2 "PQP1" "Prob precip >.01" "%"
134 131 57 2 "var134" "undefined" ""
135 131 57 2 "dustVisibility" "Dust Visibility" "m"
136 131 57 2 "dustConcentration" "Dust Concentration" "?g m^-3"
137 131 57 2 "var137" "undefined" ""
138 131 57 2 "var138" "undefined" ""
139 131 57 2 "var139" "undefined" ""
140 131 57 2 "cloudHeightCeiling" "Ceiling" "m"
141 131 57 2 "var141" "undefined" ""
142 131 57 2 "var142" "undefined" ""
143 131 57 2 "var143" "undefined" ""
144 131 57 2 "fluxLongWaveRadiationDownward" "Downward longwave
radiation flux" "W m^2"
145 131 57 2 "fluxShortWaveRadiationDownward" "Downward shortwave
radiation flux " "W m^2"
146 131 57 2 "var146" "undefined" ""
147 131 57 2 "var147" "undefined" ""
148 131 57 2 "var148" "undefined" ""
149 131 57 2 "var149" "undefined" ""
150 131 57 2 "var150" "undefined" ""
151 131 57 2 "var151" "undefined" ""
152 131 57 2 "var152" "undefined" ""
153 131 57 2 "var153" "undefined" ""
154 131 57 2 "var154" "undefined" ""
155 131 57 2 "fluxHeatSurface" "Ground Heat Flux" "W m^2"
156 131 57 2 "var156" "undefined" ""
157 131 57 2 "var157" "undefined" ""
158 131 57 2 "var158" "undefined" ""
159 131 57 2 "var159" "undefined" ""
160 131 57 2 "var160" "undefined" ""
161 131 57 2 "terrainSurfaceHeight" "Terrain height" "m"
162 131 57 2 "var162" "undefined" ""
163 131 57 2 "cloudAmountLayer" "Cloud Amount (layer)" "%"
164 131 57 2 "cloudTypeLayer" "Cloud Type (layer)" "code"
165 131 57 2 "cloudBrightnessInfrared" "Cloud Brightness IR" "non-dim"
166 131 57 2 "var166" "undefined" ""
167 131 57 2 "var167" "undefined" ""
168 131 57 2 "var168" "undefined" ""
169 131 57 2 "var169" "undefined" ""
170 131 57 2 "var170" "undefined" ""
171 131 57 2 "var171" "undefined" ""
172 131 57 2 "var172" "undefined" ""
173 131 57 2 "var173" "undefined" ""
174 131 57 2 "precipitationSnowDepthInches" "Snow Depth" "in"
175 131 57 2 "precipitationSnowAge" "Snow Age" "days"
176 131 57 2 "var176" "undefined" ""
177 131 57 2 "var177" "undefined" ""
178 131 57 2 "var178" "undefined" ""
179 131 57 2 "cloudAmountLayer" "Layer Cloud Amount" "%"
180 131 57 2 "cloudHeightBaseLayer" "Layer Cloud Base" "code"
181 131 57 2 "cloudHeightTopLayer" "Layer Cloud Top" "code"
182 131 57 2 "cloudTypeLayer" "Layer Cloud Type" "code"s
183 131 57 2 "timeLastUpdate" "Time of last update from base time"
"Minutes"
184 131 57 2 "var184" "undefined" ""
185 131 57 2 "cloudLayerOpticalDepth" "Layer Cloud Optical Depth"
"number"
186 131 57 2 "cloudLayerIcePercentage" "Layer Cloud Ice Percentage"
"%"
187 131 57 2 "cloudLayerWaterPath" "Layer Cloud Water Path" "g m^-2"
188 131 57 2 "cloudLayerParticleSize" "Layer Cloud Particle Size" "?m"
189 131 57 2 "var189" "undefined" ""
190 131 57 2 "cloudLayerNumber" "Cloud Layer Number" "number"
191 131 57 2 "cloudAnalysisQualityIndex" "Cloud Analysis Quality
Index" "%"
192 131 57 2 "var192" "undefined" ""
193 131 57 2 "precipitationQuality" "Precipitation Quality Factor
(Goodness)" "non-dim"
194 131 57 2 "satelliteIdentifier" "Satellite Identifier" "code"
195 131 57 2 "satelliteMerged" "Satellite Merged - CDFS II" "code"
196 131 57 2 "var196" "undefined" ""
197 131 57 2 "var197" "undefined" ""
198 131 57 2 "precipitationDerivedGeostationarySatelliteBased"
"Estimated precipitation - geostationary satellite based" "mm"
199 131 57 2 "precipitationRankGeostationarySatellite" "Geostationary
Satellite precipitation rank" "category"
200 131 57 2 "soilMoistureRelative" "Relative soil moisture" "numeric"
201 131 57 2 "soilMoistureVolumetricTotal" "Volumetric Soil Moisture
(liquid + frozen)" "volumetric; m^3 m^-3"
202 131 57 2 "precipitationAccumulated" "Precipitation - real amounts"
"mm"
203 131 57 2 "precipitationAccumulatedMerged" "Precipitation - merged
analysis" "mm/24 hrs"
204 131 57 2 "evapotranspirationActual" "Evapotranspiration - actual"
"mm/24 hrs"
205 131 57 2 "soilType" "Soil Type" "code"
206 131 57 2 "var206" "undefined" ""
207 131 57 2 "vegetationPlantCanopyMoistureContent" "Plant Canopy
moisture content" "mm"
208 131 57 2 "evapotranspirationPotential" "Evapotranspiration -
potential" "mm hr^-1"
209 131 57 2 "windRun" "Wind run" "km/24 hrs"
210 131 57 2 "humidityRelativeAtMinimumTemperature" "Relative Humidity
at minimum temperature" "%"
211 131 57 2 "soilMoistureVolumetricLiquid" "Volumetric Soil Moisture
(liquid only)" "volumetric; m^3 ^m-3"
212 131 57 2 "vegetationType" "Vegetation Type Category" "category"
213 131 57 2 "vegetationGreennessCoverage" "Vegetation Greenness" "%"
214 131 57 2 "var214" "undefined" ""
215 131 57 2 "var215" "undefined" ""
216 131 57 2 "var216" "undefined" ""
217 131 57 2 "var217" "undefined" ""
218 131 57 2 "var218" "undefined" ""
219 131 57 2 "var219" "undefined" ""
220 131 57 2 "var220" "undefined" ""
221 131 57 2 "var221" "undefined" ""
222 131 57 2 "var222" "undefined" ""
223 131 57 2 "var223" "undefined" ""
224 131 57 2 "var224" "undefined" ""
225 131 57 2 "var225" "undefined" ""
226 131 57 2 "var226" "undefined" ""
227 131 57 2 "CDCB" "Cloud base" "m"
228 131 57 2 "cloudHeightTop" "Cloud top" "m"
229 131 57 2 "var229" "undefined" ""
230 131 57 2 "latitude" "Latitude" "deg"
231 131 57 2 "longitude" "Longitude" "deg"
232 131 57 2 "var232" "undefined" ""
233 131 57 2 "terrainSurfaceHeight" "Model terrain height" "m"
234 131 57 2 "runoffGroundwaterBaseflow" "Baseflow - groundwater
runoff" "mm"
235 131 57 2 "runoffStorm" "Storm surface runoff" "mm"
236 131 57 2 "var236" "undefined" ""
237 131 57 2 "var237" "undefined" ""
238 131 57 2 "var238" "undefined" ""
239 131 57 2 "var239" "undefined" ""
240 131 57 2 "var240" "undefined" ""
241 131 57 2 "landUseCategory" "Land-use" "category"
242 131 57 2 "var242" "undefined" ""
243 131 57 2 "var243" "undefined" ""
244 131 57 2 "var244" "undefined" ""
245 131 57 2 "var245" "undefined" ""
246 131 57 2 "var246" "undefined" ""
247 131 57 2 "var247" "undefined" ""
248 131 57 2 "var248" "undefined" ""
248 131 57 2 "var249" "undefined" ""
250 131 57 2 "N/A" "Quality Control Display " "code"
251 131 57 2 "var251" "undefined" ""
252 131 57 2 "var252" "undefined" ""
253 131 57 2 "var253" "undefined" ""
254 131 57 2 "var254" "undefined" ""
255 131 57 2 "var255" "undefined" ""
256 133 57 1 "CWGST" "Convective gust windspeed" "m s^-1"
257 133 57 1 "WBZHGT" "Wet Bulb Zero Height" "m"
258 133 57 1 "HAIL" "Hail size" "cm"
259 133 57 1 "CT1TOP" "Contrail Engine Type 1 Top" "m"
260 133 57 1 "CT1BASE" "Contrail Engine Type 1 Base" "m"
261 133 57 1 "CT2TOP" "Contrail Engine Type 2 Top" "m"
262 133 57 1 "CT2BASE" "Contrail Engine Type 2 Base" "m"
263 133 57 1 "CT3TOP" "Contrail Engine Type 3 Top" "m"
264 133 57 1 "CT3BASE" "Contrail Engine Type 3 Base" "m"
265 133 57 1 "PVORT" "Potential Vorticity" "km^2 kg s^-1"
266 133 57 1 "STTOP" "SLAT Turbulence Top" "NA"
267 133 57 1 "STBASE" "SLAT Turbulence Base" "NA"
268 133 57 1 "STI" "SLAT Turbulence Index" "NA"
269 133 57 1 "PANIND" "Panofsky Turbulence Index" "-200-500"
270 133 57 1 "ALSTG" "Altimeter setting " "in Hg"
271 133 57 1 "CAPE" "Convective Available Potential Energy" "J kg^-1"
272 133 57 1 "CIN" "Convective Inhibition" "J kg^-1"
273 133 57 1 "COND" "Condensate" "kg kg^-1"
274 133 57 1 "QCLOUD" "Cloud water Mixing Ratio" "kg kg^-1"
275 133 57 1 "QICE" "Ice Water Mixing Ratio" "kg kg^-1"
276 133 57 1 "QRAIN" "Rain Water Mixing Ratio" "kg kg^-1"
277 133 57 1 "MCONV" "Horizontal Moisture Convergence" "NA"
278 133 57 1 "TINT" "Turbulence Intensity" "NA"
279 133 57 1 "PCPTYPE" "Precipitation Type [0=none, 1=Rain, 2=TRW,
3=ZR, 4=mixed, 5=SN, 6=SVRTRW]" "NA"
280 133 57 1 "ICINT" "Icing Intensity" "NA"
------------------------------------------------
Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #91448] Changing Units in STATS files
From: robert.craig.2 at us.af.mil
Time: Fri Aug 09 10:17:39 2019
Update, since I sent you the last email, I add PCP1 to code 206 in our
default
grib1 table ( it wasn't in the one I attached on the last email).. I
then
tried to run with name=PQP1; GRIB1_ptv=2; level = "A006"; prob=TRUE;
but MET
still wasn't finding the grib record.
Bob
-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Thursday, August 8, 2019 6:06 PM
To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
<robert.craig.2 at us.af.mil>
Cc: SITTEL, MATTHEW C CTR USAF AFMC AFLCMC/HBAW-OL
<matthew.sittel.ctr at us.af.mil>
Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #91448] Changing Units
in STATS files
Bob,
Thanks for sending the sample data. Looking at the .stat output file
and the log messages you sent, I noticed something pretty interesting.
The forecast variables have an empty "name". You can see this in the
log
messages:
* DEBUG 2: Processing /A006 versus APCP_06/L0, ...*
And you can see this in the FCST_VAR output column in the .stat file,
where FCST_VAR=NA. That means that Point-Stat doesn't have a string
to assign to your forecast data which makes it pretty difficult to
distinguish in the output!
Looking in your Point-Stat config file, I see why...
fcst = {
field = [
{ GRIB1_code=206; level = "A006"; prob = TRUE;},
{ GRIB1_code=222; level = "A006"; prob = TRUE;},
{ GRIB1_code=223; level = "A006"; prob = TRUE;},
{ GRIB1_code=224; level = "A006"; prob = TRUE;},
{ GRIB1_code=226; level = "A006"; prob = TRUE;}
];
cat_thresh=[==0.1];
}
You're specifying the GRIB1_code instead of a "name" string. Since I
don't have your exact input forecast data, I can't exactly replicate
this behavior. I did try to mimic with the data we have here, but
wasn't able to get the exact behavior. I used TMP and RH, but
consistently got units of K and %.
Here's what's going on... you request GRIB1_code = 206...
Point-Stat does a table lookup to find a matching line in the GRIB
tables.
Since you didn't specify the parameter table version number, it uses
the default value of 2. It finds a GRIB table record for ptv = 2 and
code =
206 and uses the units specified on that line.
You'd need to look in your GRIB tables, but perhaps for lines ptv = 2
and code = 206, 222, 223, 224, and 226, you don't have the same units
values listed? If you're using a different parameter table version
number, please specify that... for example...
{ GRIB1_code=206; *GRIB1_ptv=10;* level = "A006"; prob = TRUE;},
So while using these GRIB1 codes to access the data is possible, it
isn't sufficient to accurately grab the units string from the GRIB1
tables. A better alternative would be to just get the typical option
of specifying the "name" to work well.
Thanks,
John
On Thu, Aug 8, 2019 at 1:52 PM robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> Thu Aug 08 13:51:49 2019: Request 91448 was acted upon.
> Transaction: Ticket created by robert.craig.2 at us.af.mil
> Queue: met_help
> Subject: Changing Units in STATS files
> Owner: Nobody
> Requestors: robert.craig.2 at us.af.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=91448
> >
>
>
> John, I am verifying precipitation accumulation probabilities. I am
> using a netcdf file with 6 hour accumulations. When I look through
> the stat file produced, the units field keeps changing values (ie.
> w/m^2, gpm, kg/m^2,
> 0..9,m,.etc) and there will be two unit types for each precip
threshold.
> The listing indicates MET is using the correct grib record number
for the
> variable. Is this the expected behavior or is something not right.
The
> list of the run is below:
>
>
>
> DEBUG 1: Reading user-defined grib1 MET_GRIB_TABLES file:
> /h/data/global/WXQC/data/met/config/grib2/grib1_af_131_57.txt
>
> DEBUG 1: Reading user-defined grib2 MET_GRIB_TABLES file:
> /h/data/global/WXQC/data/met/config/grib2/grib2_geps.txt
>
> DEBUG 1: Default Config File:
> /h/MET/share/met/config/PointStatConfig_default
>
> DEBUG 1: User Config File:
>
>
/h/data/global/WXQC/data/met/config/met_config/PointStatConfig_meps_tp
> _conus
> _updated
>
> GSL_RNG_TYPE=mt19937
>
> GSL_RNG_SEED=18446744071888718941
>
> DEBUG 1: Forecast File:
>
>
/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.201908040
> 0.0048
>
> DEBUG 1: Observation File:
> /h/data/global/WXQC/data/met/nc_obs/precip/2019080600.nc
>
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match
for
> VarInfo "/A006" in GRIB record 4 of GRIB file
>
>
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.20190804
> 00.004
> 8".
>
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "/A006" in GRIB file
>
>
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.20190804
> 00.004
> 8".
>
> DEBUG 2:
>
> DEBUG 2:
>
>
----------------------------------------------------------------------
> ------
> ----
>
> DEBUG 2:
>
> DEBUG 2: Reading data for /A006.
>
> DEBUG 3: Use the forecast grid.
>
> DEBUG 3: Grid Definition: Projection: Lambert Conformal Nx: 1170 Ny:
> 720
> Lat_LL: 23.159 Lon_LL: 119.272 Lon_orient: 97.000 Alpha: 3171.947
Cone:
> 0.616 Bx: 582.1130 By: 2385.7494
>
> DEBUG 2: Processing masking regions.
>
> DEBUG 3: Processing grid mask: FULL
>
> DEBUG 2: Processing geography data.
>
> DEBUG 3: Rescaling probabilistic field from [0,100] to [0,1].
>
> DEBUG 3: Applying conversion function.
>
> DEBUG 1: Regridding climatology CFSR_APCP_T01_00(*,*) to the
> verification grid.
>
> DEBUG 3: For forecast valid at 20190806_000000, found 1 climatology
> field(s)
> valid at 20190806_000000.
>
> DEBUG 3: Found 1 climatology fields.
>
> DEBUG 2: For /A006 found 1 forecast levels, 1 climatology mean
levels,
> and
> 0
> climatology standard deviation levels.
>
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match
for
> VarInfo "/A006" in GRIB record 7 of GRIB file
>
>
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.20190804
> 00.004
> 8".
>
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "/A006" in GRIB file
>
>
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.20190804
> 00.004
> 8".
>
> DEBUG 2:
>
> DEBUG 2:
>
>
----------------------------------------------------------------------
> ------
> ----
>
> DEBUG 2:
>
> DEBUG 2: Reading data for /A006.
>
> DEBUG 3: Rescaling probabilistic field from [0,100] to [0,1].
>
> DEBUG 3: Applying conversion function.
>
> DEBUG 1: Regridding climatology CFSR_APCP_T10_00(*,*) to the
> verification grid.
>
> DEBUG 3: For forecast valid at 20190806_000000, found 1 climatology
> field(s)
> valid at 20190806_000000.
>
> DEBUG 3: Found 1 climatology fields.
>
> DEBUG 2: For /A006 found 1 forecast levels, 1 climatology mean
levels,
> and
> 0
> climatology standard deviation levels.
>
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match
for
> VarInfo "/A006" in GRIB record 8 of GRIB file
>
>
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.20190804
> 00.004
> 8".
>
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "/A006" in GRIB file
>
>
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.20190804
> 00.004
> 8".
>
> DEBUG 2:
>
> DEBUG 2:
>
>
----------------------------------------------------------------------
> ------
> ----
>
> DEBUG 2:
>
> DEBUG 2: Reading data for /A006.
>
> DEBUG 3: Rescaling probabilistic field from [0,100] to [0,1].
>
> DEBUG 3: Applying conversion function.
>
> DEBUG 1: Regridding climatology CFSR_APCP_T25_00(*,*) to the
> verification grid.
>
> DEBUG 3: For forecast valid at 20190806_000000, found 1 climatology
> field(s)
> valid at 20190806_000000.
>
> DEBUG 3: Found 1 climatology fields.
>
> DEBUG 2: For /A006 found 1 forecast levels, 1 climatology mean
levels,
> and
> 0
> climatology standard deviation levels.
>
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match
for
> VarInfo "/A006" in GRIB record 9 of GRIB file
>
>
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.20190804
> 00.004
> 8".
>
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "/A006" in GRIB file
>
>
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.20190804
> 00.004
> 8".
>
> DEBUG 2:
>
> DEBUG 2:
>
>
----------------------------------------------------------------------
> ------
> ----
>
> DEBUG 2:
>
> DEBUG 2: Reading data for /A006.
>
> DEBUG 3: Rescaling probabilistic field from [0,100] to [0,1].
>
> DEBUG 3: Applying conversion function.
>
> DEBUG 1: Regridding climatology CFSR_APCP_T50_00(*,*) to the
> verification grid.
>
> DEBUG 3: For forecast valid at 20190806_000000, found 1 climatology
> field(s)
> valid at 20190806_000000.
>
> DEBUG 3: Found 1 climatology fields.
>
> DEBUG 2: For /A006 found 1 forecast levels, 1 climatology mean
levels,
> and
> 0
> climatology standard deviation levels.
>
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match
for
> VarInfo "/A006" in GRIB record 11 of GRIB file
>
>
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.20190804
> 00.004
> 8".
>
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "/A006" in GRIB file
>
>
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.20190804
> 00.004
> 8".
>
> DEBUG 2:
>
> DEBUG 2:
>
>
----------------------------------------------------------------------
> ------
> ----
>
> DEBUG 2:
>
> DEBUG 2: Reading data for /A006.
>
> DEBUG 3: Rescaling probabilistic field from [0,100] to [0,1].
>
> DEBUG 3: Applying conversion function.
>
> DEBUG 1: Regridding climatology CFSR_APCP_T100_00(*,*) to the
> verification grid.
>
> DEBUG 3: For forecast valid at 20190806_000000, found 1 climatology
> field(s)
> valid at 20190806_000000.
>
> DEBUG 3: Found 1 climatology fields.
>
> DEBUG 2: For /A006 found 1 forecast levels, 1 climatology mean
levels,
> and
> 0
> climatology standard deviation levels.
>
> DEBUG 2:
>
> DEBUG 2:
>
>
----------------------------------------------------------------------
> ------
> ----
>
> DEBUG 2:
>
> DEBUG 2: Searching 1739 observations from 1739 messages.
>
> DEBUG 2:
>
> DEBUG 2:
>
>
----------------------------------------------------------------------
> ------
> ----
>
> DEBUG 2:
>
> WARNING:
>
> WARNING: check_hdr_str() -> null string!
>
> WARNING:
>
> DEBUG 2: Processing /A006 versus APCP_06/L0, for observation type
> ADPSFC, over region FULL, for interpolation method NEAREST(1), using
611 pairs.
>
> DEBUG 3: Number of matched pairs = 611
>
> DEBUG 3: Observations processed = 1739
>
> DEBUG 3: Rejected: SID exclusion = 0
>
> DEBUG 3: Rejected: obs type = 0
>
> DEBUG 3: Rejected: valid time = 0
>
> DEBUG 3: Rejected: bad obs value = 0
>
> DEBUG 3: Rejected: off the grid = 1128
>
> DEBUG 3: Rejected: topography = 0
>
> DEBUG 3: Rejected: level mismatch = 0
>
> DEBUG 3: Rejected: quality marker = 0
>
> DEBUG 3: Rejected: message type = 0
>
> DEBUG 3: Rejected: masking region = 0
>
> DEBUG 3: Rejected: bad fcst value = 0
>
> DEBUG 3: Rejected: duplicates = 0
>
> DEBUG 2: Computing Probabilistic Statistics.
>
> DEBUG 2:
>
> DEBUG 2:
>
>
----------------------------------------------------------------------
> ------
> ----
>
> DEBUG 2:
>
> WARNING:
>
> WARNING: check_hdr_str() -> null string!
>
> WARNING:
>
> DEBUG 2: Processing /A006 versus APCP_06/L0, for observation type
> ADPSFC, over region FULL, for interpolation method NEAREST(1), using
611 pairs.
>
> DEBUG 3: Number of matched pairs = 611
>
> DEBUG 3: Observations processed = 1739
>
> DEBUG 3: Rejected: SID exclusion = 0
>
> DEBUG 3: Rejected: obs type = 0
>
> DEBUG 3: Rejected: valid time = 0
>
> DEBUG 3: Rejected: bad obs value = 0
>
> DEBUG 3: Rejected: off the grid = 1128
>
> DEBUG 3: Rejected: topography = 0
>
> DEBUG 3: Rejected: level mismatch = 0
>
> DEBUG 3: Rejected: quality marker = 0
>
> DEBUG 3: Rejected: message type = 0
>
> DEBUG 3: Rejected: masking region = 0
>
> DEBUG 3: Rejected: bad fcst value = 0
>
> DEBUG 3: Rejected: duplicates = 0
>
> DEBUG 2: Computing Probabilistic Statistics.
>
> DEBUG 2:
>
> DEBUG 2:
>
>
----------------------------------------------------------------------
> ------
> ----
>
> DEBUG 2:
>
> WARNING:
>
> WARNING: check_hdr_str() -> null string!
>
> WARNING:
>
> DEBUG 2: Processing /A006 versus APCP_06/L0, for observation type
> ADPSFC, over region FULL, for interpolation method NEAREST(1), using
611 pairs.
>
> DEBUG 3: Number of matched pairs = 611
>
> DEBUG 3: Observations processed = 1739
>
> DEBUG 3: Rejected: SID exclusion = 0
>
> DEBUG 3: Rejected: obs type = 0
>
> DEBUG 3: Rejected: valid time = 0
>
> DEBUG 3: Rejected: bad obs value = 0
>
> DEBUG 3: Rejected: off the grid = 1128
>
> DEBUG 3: Rejected: topography = 0
>
> DEBUG 3: Rejected: level mismatch = 0
>
> DEBUG 3: Rejected: quality marker = 0
>
> DEBUG 3: Rejected: message type = 0
>
> DEBUG 3: Rejected: masking region = 0
>
> DEBUG 3: Rejected: bad fcst value = 0
>
> DEBUG 3: Rejected: duplicates = 0
>
> DEBUG 2: Computing Probabilistic Statistics.
>
> DEBUG 2:
>
> DEBUG 2:
>
>
----------------------------------------------------------------------
> ------
> ----
>
> DEBUG 2:
>
> WARNING:
>
> WARNING: check_hdr_str() -> null string!
>
> WARNING:
>
> DEBUG 2: Processing /A006 versus APCP_06/L0, for observation type
> ADPSFC, over region FULL, for interpolation method NEAREST(1), using
611 pairs.
>
> DEBUG 3: Number of matched pairs = 611
>
> DEBUG 3: Observations processed = 1739
>
> DEBUG 3: Rejected: SID exclusion = 0
>
> DEBUG 3: Rejected: obs type = 0
>
> DEBUG 3: Rejected: valid time = 0
>
> DEBUG 3: Rejected: bad obs value = 0
>
> DEBUG 3: Rejected: off the grid = 1128
>
> DEBUG 3: Rejected: topography = 0
>
> DEBUG 3: Rejected: level mismatch = 0
>
> DEBUG 3: Rejected: quality marker = 0
>
> DEBUG 3: Rejected: message type = 0
>
> DEBUG 3: Rejected: masking region = 0
>
> DEBUG 3: Rejected: bad fcst value = 0
>
> DEBUG 3: Rejected: duplicates = 0
>
> DEBUG 2: Computing Probabilistic Statistics.
>
> DEBUG 2:
>
> DEBUG 2:
>
>
----------------------------------------------------------------------
> ------
> ----
>
> DEBUG 2:
>
> WARNING:
>
> WARNING: check_hdr_str() -> null string!
>
> WARNING:
>
> DEBUG 2: Processing /A006 versus APCP_06/L0, for observation type
> ADPSFC, over region FULL, for interpolation method NEAREST(1), using
611 pairs.
>
> DEBUG 3: Number of matched pairs = 611
>
> DEBUG 3: Observations processed = 1739
>
> DEBUG 3: Rejected: SID exclusion = 0
>
> DEBUG 3: Rejected: obs type = 0
>
> DEBUG 3: Rejected: valid time = 0
>
> DEBUG 3: Rejected: bad obs value = 0
>
> DEBUG 3: Rejected: off the grid = 1128
>
> DEBUG 3: Rejected: topography = 0
>
> DEBUG 3: Rejected: level mismatch = 0
>
> DEBUG 3: Rejected: quality marker = 0
>
> DEBUG 3: Rejected: message type = 0
>
> DEBUG 3: Rejected: masking region = 0
>
> DEBUG 3: Rejected: bad fcst value = 0
>
> DEBUG 3: Rejected: duplicates = 0
>
> DEBUG 2: Computing Probabilistic Statistics.
>
> DEBUG 2:
>
> DEBUG 2:
>
>
----------------------------------------------------------------------
> ------
> ----
>
> DEBUG 2:
>
> DEBUG 1: Output file:
>
>
/h/data/global/WXQC/data/met/ptstat/mdlob_pairs/meps/TP/point_stat_mep
> s_conu
> s_480000L_20190806_000000V.stat
>
> DEBUG 1: Output file:
>
>
/h/data/global/WXQC/data/met/ptstat/mdlob_pairs/meps/TP/point_stat_mep
> s_conu
> s_480000L_20190806_000000V_pct.txt
>
> DEBUG 1: Output file:
>
>
/h/data/global/WXQC/data/met/ptstat/mdlob_pairs/meps/TP/point_stat_mep
> s_conu
> s_480000L_20190806_000000V_pstd.txt
>
> Processing 54hour
>
>
>
> Thanks
>
> Bob
>
>
>
------------------------------------------------
Subject: Changing Units in STATS files
From: John Halley Gotway
Time: Fri Aug 09 10:46:27 2019
Bob,
Thanks for sending the additional sample data. Looking in your GRIB
table,
I see this entry:
* 133 131 57 2 "PQP1" "Prob precip >.01" "%"*
To remember what those numbers mean, I found the following in the
data/config/README file...
BUT I FOUND AN ERROR IN THE README FILE WHERE WE HAD TWO COLUMNS
TRANSPOSED. HERE'S THE CORRECT COLUMN DESCRIPTIONS:
*GRIB1 table files begin with "grib1" prefix and end with a ".txt"
suffix.The first line of the file must contain "GRIB1".The following
lines
consist of 4 integers followed by 3 strings:Column 1: GRIB code (e.g.
11
for temperature)Column 2: parameter table version numberColumn 3:
center id
(e.g. 07 for US Weather Service- National Met. Center)Column 4:
subcenter
idColumn 5: variable nameColumn 6: variable descriptionColumn 7:
units*
Next, I checked your original Point-Stat config file and see that
you're
trying to match GRIB code 206 (GRIB1_code=206;). I used wgrib to find
which record contains GRIB code 206... and see that it's record number
4:
*wgrib grib.2019080500.0006 | grep "kpds5=206"Undefined parameter
table
(center 57-4 table 3), using
NCEP-
opn4:4317582:d=19080500:UVI:kpds5=206:kpds6=1:kpds7=0:TR=4:P1=0:P2=6:TimeU=1:sfc:0-
6hr
acc:NAve=0*
So I used wgrib again to dump verbose info about that record:
*wgrib -V -d 4 grib.2019080500.0006...rec 4:4317582:date 2019080500
UVI
kpds5=206 kpds6=1 kpds7=0 levels=(0,0) grid=255 sfc 0-6hr acc:...
center
57 subcenter 4 process 99 Table 3 scan: WE:SN winds(grid) *
*...*
Using this info (center=57, subcenter=4, and table=3), I modified your
PQP1
table entry to be:
* 206 3 57 4 "PQP1" "Prob precip >.01" "%"*
Extending that for the other variables yields (and I just made up
variable
names):
*cat grib1_af_prob_precip.txt*
*GRIB1*
*206 3 57 4 "PQP01" "Prob precip >.01" "%"*
*222 3 57 4 "PQP10" "Prob precip >.10" "%"*
*223 3 57 4 "PQP25" "Prob precip >.25" "%"*
*224 3 57 4 "PQP50" "Prob precip >.50" "%"*
*226 3 57 4 "PQ1P0" "Prob precip >1.0" "%"*
So I set MET_GRIB_TABLES=grib1_af_prob_precip.txt
And that enables me to run Point-Stat configured like this:
fcst = {
field = [
{ name="PQP01"; level = "A006"; prob = TRUE;},
{ name="PQP10"; level = "A006"; prob = TRUE;},
{ name="PQP25"; level = "A006"; prob = TRUE;},
{ name="PQP50"; level = "A006"; prob = TRUE;},
{ name="PQ1P0"; level = "A006"; prob = TRUE;}
];
cat_thresh=[==0.1];
}
I'll correct that documentation mistake in met-8.1.2 and met-9.0.
Thanks,
John
On Fri, Aug 9, 2019 at 10:18 AM robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=91448 >
>
>
> Update, since I sent you the last email, I add PCP1 to code 206 in
our
> default
> grib1 table ( it wasn't in the one I attached on the last email)..
I then
> tried to run with name=PQP1; GRIB1_ptv=2; level = "A006"; prob=TRUE;
but
> MET
> still wasn't finding the grib record.
>
>
> Bob
> -----Original Message-----
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Thursday, August 8, 2019 6:06 PM
> To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
<robert.craig.2 at us.af.mil>
> Cc: SITTEL, MATTHEW C CTR USAF AFMC AFLCMC/HBAW-OL <
> matthew.sittel.ctr at us.af.mil>
> Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #91448] Changing
Units in
> STATS files
>
> Bob,
>
> Thanks for sending the sample data. Looking at the .stat output
file and
> the log messages you sent, I noticed something pretty interesting.
The
> forecast variables have an empty "name". You can see this in the
log
> messages:
> * DEBUG 2: Processing /A006 versus APCP_06/L0, ...*
>
> And you can see this in the FCST_VAR output column in the .stat
file,
> where FCST_VAR=NA. That means that Point-Stat doesn't have a string
to
> assign to your forecast data which makes it pretty difficult to
distinguish
> in the output!
>
> Looking in your Point-Stat config file, I see why...
>
> fcst = {
> field = [
> { GRIB1_code=206; level = "A006"; prob = TRUE;},
> { GRIB1_code=222; level = "A006"; prob = TRUE;},
> { GRIB1_code=223; level = "A006"; prob = TRUE;},
> { GRIB1_code=224; level = "A006"; prob = TRUE;},
> { GRIB1_code=226; level = "A006"; prob = TRUE;}
>
> ];
> cat_thresh=[==0.1];
> }
>
> You're specifying the GRIB1_code instead of a "name" string. Since
I
> don't have your exact input forecast data, I can't exactly replicate
this
> behavior. I did try to mimic with the data we have here, but wasn't
able
> to get the exact behavior. I used TMP and RH, but consistently got
units
> of K and %.
>
> Here's what's going on... you request GRIB1_code = 206...
> Point-Stat does a table lookup to find a matching line in the GRIB
tables.
> Since you didn't specify the parameter table version number, it uses
the
> default value of 2. It finds a GRIB table record for ptv = 2 and
code =
> 206 and uses the units specified on that line.
>
> You'd need to look in your GRIB tables, but perhaps for lines ptv =
2 and
> code = 206, 222, 223, 224, and 226, you don't have the same units
values
> listed? If you're using a different parameter table version number,
please
> specify that... for example...
> { GRIB1_code=206; *GRIB1_ptv=10;* level = "A006"; prob = TRUE;},
>
> So while using these GRIB1 codes to access the data is possible, it
isn't
> sufficient to accurately grab the units string from the GRIB1
tables. A
> better alternative would be to just get the typical option of
specifying
> the "name" to work well.
>
> Thanks,
> John
>
> On Thu, Aug 8, 2019 at 1:52 PM robert.craig.2 at us.af.mil via RT <
> met_help at ucar.edu> wrote:
>
> >
> > Thu Aug 08 13:51:49 2019: Request 91448 was acted upon.
> > Transaction: Ticket created by robert.craig.2 at us.af.mil
> > Queue: met_help
> > Subject: Changing Units in STATS files
> > Owner: Nobody
> > Requestors: robert.craig.2 at us.af.mil
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=91448
> > >
> >
> >
> > John, I am verifying precipitation accumulation probabilities. I
am
> > using a netcdf file with 6 hour accumulations. When I look
through
> > the stat file produced, the units field keeps changing values (ie.
> > w/m^2, gpm, kg/m^2,
> > 0..9,m,.etc) and there will be two unit types for each precip
threshold.
> > The listing indicates MET is using the correct grib record number
for the
> > variable. Is this the expected behavior or is something not
right. The
> > list of the run is below:
> >
> >
> >
> > DEBUG 1: Reading user-defined grib1 MET_GRIB_TABLES file:
> > /h/data/global/WXQC/data/met/config/grib2/grib1_af_131_57.txt
> >
> > DEBUG 1: Reading user-defined grib2 MET_GRIB_TABLES file:
> > /h/data/global/WXQC/data/met/config/grib2/grib2_geps.txt
> >
> > DEBUG 1: Default Config File:
> > /h/MET/share/met/config/PointStatConfig_default
> >
> > DEBUG 1: User Config File:
> >
> >
/h/data/global/WXQC/data/met/config/met_config/PointStatConfig_meps_tp
> > _conus
> > _updated
> >
> > GSL_RNG_TYPE=mt19937
> >
> > GSL_RNG_SEED=18446744071888718941
> >
> > DEBUG 1: Forecast File:
> >
> >
/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.201908040
> > 0.0048
> >
> > DEBUG 1: Observation File:
> > /h/data/global/WXQC/data/met/nc_obs/precip/2019080600.nc
> >
> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match
for
> > VarInfo "/A006" in GRIB record 4 of GRIB file
> >
> >
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.20190804
> > 00.004
> > 8".
> >
> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> > matching VarInfo "/A006" in GRIB file
> >
> >
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.20190804
> > 00.004
> > 8".
> >
> > DEBUG 2:
> >
> > DEBUG 2:
> >
> >
----------------------------------------------------------------------
> > ------
> > ----
> >
> > DEBUG 2:
> >
> > DEBUG 2: Reading data for /A006.
> >
> > DEBUG 3: Use the forecast grid.
> >
> > DEBUG 3: Grid Definition: Projection: Lambert Conformal Nx: 1170
Ny:
> > 720
> > Lat_LL: 23.159 Lon_LL: 119.272 Lon_orient: 97.000 Alpha: 3171.947
Cone:
> > 0.616 Bx: 582.1130 By: 2385.7494
> >
> > DEBUG 2: Processing masking regions.
> >
> > DEBUG 3: Processing grid mask: FULL
> >
> > DEBUG 2: Processing geography data.
> >
> > DEBUG 3: Rescaling probabilistic field from [0,100] to [0,1].
> >
> > DEBUG 3: Applying conversion function.
> >
> > DEBUG 1: Regridding climatology CFSR_APCP_T01_00(*,*) to the
> > verification grid.
> >
> > DEBUG 3: For forecast valid at 20190806_000000, found 1
climatology
> > field(s)
> > valid at 20190806_000000.
> >
> > DEBUG 3: Found 1 climatology fields.
> >
> > DEBUG 2: For /A006 found 1 forecast levels, 1 climatology mean
levels,
> > and
> > 0
> > climatology standard deviation levels.
> >
> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match
for
> > VarInfo "/A006" in GRIB record 7 of GRIB file
> >
> >
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.20190804
> > 00.004
> > 8".
> >
> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> > matching VarInfo "/A006" in GRIB file
> >
> >
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.20190804
> > 00.004
> > 8".
> >
> > DEBUG 2:
> >
> > DEBUG 2:
> >
> >
----------------------------------------------------------------------
> > ------
> > ----
> >
> > DEBUG 2:
> >
> > DEBUG 2: Reading data for /A006.
> >
> > DEBUG 3: Rescaling probabilistic field from [0,100] to [0,1].
> >
> > DEBUG 3: Applying conversion function.
> >
> > DEBUG 1: Regridding climatology CFSR_APCP_T10_00(*,*) to the
> > verification grid.
> >
> > DEBUG 3: For forecast valid at 20190806_000000, found 1
climatology
> > field(s)
> > valid at 20190806_000000.
> >
> > DEBUG 3: Found 1 climatology fields.
> >
> > DEBUG 2: For /A006 found 1 forecast levels, 1 climatology mean
levels,
> > and
> > 0
> > climatology standard deviation levels.
> >
> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match
for
> > VarInfo "/A006" in GRIB record 8 of GRIB file
> >
> >
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.20190804
> > 00.004
> > 8".
> >
> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> > matching VarInfo "/A006" in GRIB file
> >
> >
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.20190804
> > 00.004
> > 8".
> >
> > DEBUG 2:
> >
> > DEBUG 2:
> >
> >
----------------------------------------------------------------------
> > ------
> > ----
> >
> > DEBUG 2:
> >
> > DEBUG 2: Reading data for /A006.
> >
> > DEBUG 3: Rescaling probabilistic field from [0,100] to [0,1].
> >
> > DEBUG 3: Applying conversion function.
> >
> > DEBUG 1: Regridding climatology CFSR_APCP_T25_00(*,*) to the
> > verification grid.
> >
> > DEBUG 3: For forecast valid at 20190806_000000, found 1
climatology
> > field(s)
> > valid at 20190806_000000.
> >
> > DEBUG 3: Found 1 climatology fields.
> >
> > DEBUG 2: For /A006 found 1 forecast levels, 1 climatology mean
levels,
> > and
> > 0
> > climatology standard deviation levels.
> >
> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match
for
> > VarInfo "/A006" in GRIB record 9 of GRIB file
> >
> >
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.20190804
> > 00.004
> > 8".
> >
> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> > matching VarInfo "/A006" in GRIB file
> >
> >
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.20190804
> > 00.004
> > 8".
> >
> > DEBUG 2:
> >
> > DEBUG 2:
> >
> >
----------------------------------------------------------------------
> > ------
> > ----
> >
> > DEBUG 2:
> >
> > DEBUG 2: Reading data for /A006.
> >
> > DEBUG 3: Rescaling probabilistic field from [0,100] to [0,1].
> >
> > DEBUG 3: Applying conversion function.
> >
> > DEBUG 1: Regridding climatology CFSR_APCP_T50_00(*,*) to the
> > verification grid.
> >
> > DEBUG 3: For forecast valid at 20190806_000000, found 1
climatology
> > field(s)
> > valid at 20190806_000000.
> >
> > DEBUG 3: Found 1 climatology fields.
> >
> > DEBUG 2: For /A006 found 1 forecast levels, 1 climatology mean
levels,
> > and
> > 0
> > climatology standard deviation levels.
> >
> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match
for
> > VarInfo "/A006" in GRIB record 11 of GRIB file
> >
> >
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.20190804
> > 00.004
> > 8".
> >
> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> > matching VarInfo "/A006" in GRIB file
> >
> >
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.20190804
> > 00.004
> > 8".
> >
> > DEBUG 2:
> >
> > DEBUG 2:
> >
> >
----------------------------------------------------------------------
> > ------
> > ----
> >
> > DEBUG 2:
> >
> > DEBUG 2: Reading data for /A006.
> >
> > DEBUG 3: Rescaling probabilistic field from [0,100] to [0,1].
> >
> > DEBUG 3: Applying conversion function.
> >
> > DEBUG 1: Regridding climatology CFSR_APCP_T100_00(*,*) to the
> > verification grid.
> >
> > DEBUG 3: For forecast valid at 20190806_000000, found 1
climatology
> > field(s)
> > valid at 20190806_000000.
> >
> > DEBUG 3: Found 1 climatology fields.
> >
> > DEBUG 2: For /A006 found 1 forecast levels, 1 climatology mean
levels,
> > and
> > 0
> > climatology standard deviation levels.
> >
> > DEBUG 2:
> >
> > DEBUG 2:
> >
> >
----------------------------------------------------------------------
> > ------
> > ----
> >
> > DEBUG 2:
> >
> > DEBUG 2: Searching 1739 observations from 1739 messages.
> >
> > DEBUG 2:
> >
> > DEBUG 2:
> >
> >
----------------------------------------------------------------------
> > ------
> > ----
> >
> > DEBUG 2:
> >
> > WARNING:
> >
> > WARNING: check_hdr_str() -> null string!
> >
> > WARNING:
> >
> > DEBUG 2: Processing /A006 versus APCP_06/L0, for observation type
> > ADPSFC, over region FULL, for interpolation method NEAREST(1),
using 611
> pairs.
> >
> > DEBUG 3: Number of matched pairs = 611
> >
> > DEBUG 3: Observations processed = 1739
> >
> > DEBUG 3: Rejected: SID exclusion = 0
> >
> > DEBUG 3: Rejected: obs type = 0
> >
> > DEBUG 3: Rejected: valid time = 0
> >
> > DEBUG 3: Rejected: bad obs value = 0
> >
> > DEBUG 3: Rejected: off the grid = 1128
> >
> > DEBUG 3: Rejected: topography = 0
> >
> > DEBUG 3: Rejected: level mismatch = 0
> >
> > DEBUG 3: Rejected: quality marker = 0
> >
> > DEBUG 3: Rejected: message type = 0
> >
> > DEBUG 3: Rejected: masking region = 0
> >
> > DEBUG 3: Rejected: bad fcst value = 0
> >
> > DEBUG 3: Rejected: duplicates = 0
> >
> > DEBUG 2: Computing Probabilistic Statistics.
> >
> > DEBUG 2:
> >
> > DEBUG 2:
> >
> >
----------------------------------------------------------------------
> > ------
> > ----
> >
> > DEBUG 2:
> >
> > WARNING:
> >
> > WARNING: check_hdr_str() -> null string!
> >
> > WARNING:
> >
> > DEBUG 2: Processing /A006 versus APCP_06/L0, for observation type
> > ADPSFC, over region FULL, for interpolation method NEAREST(1),
using 611
> pairs.
> >
> > DEBUG 3: Number of matched pairs = 611
> >
> > DEBUG 3: Observations processed = 1739
> >
> > DEBUG 3: Rejected: SID exclusion = 0
> >
> > DEBUG 3: Rejected: obs type = 0
> >
> > DEBUG 3: Rejected: valid time = 0
> >
> > DEBUG 3: Rejected: bad obs value = 0
> >
> > DEBUG 3: Rejected: off the grid = 1128
> >
> > DEBUG 3: Rejected: topography = 0
> >
> > DEBUG 3: Rejected: level mismatch = 0
> >
> > DEBUG 3: Rejected: quality marker = 0
> >
> > DEBUG 3: Rejected: message type = 0
> >
> > DEBUG 3: Rejected: masking region = 0
> >
> > DEBUG 3: Rejected: bad fcst value = 0
> >
> > DEBUG 3: Rejected: duplicates = 0
> >
> > DEBUG 2: Computing Probabilistic Statistics.
> >
> > DEBUG 2:
> >
> > DEBUG 2:
> >
> >
----------------------------------------------------------------------
> > ------
> > ----
> >
> > DEBUG 2:
> >
> > WARNING:
> >
> > WARNING: check_hdr_str() -> null string!
> >
> > WARNING:
> >
> > DEBUG 2: Processing /A006 versus APCP_06/L0, for observation type
> > ADPSFC, over region FULL, for interpolation method NEAREST(1),
using 611
> pairs.
> >
> > DEBUG 3: Number of matched pairs = 611
> >
> > DEBUG 3: Observations processed = 1739
> >
> > DEBUG 3: Rejected: SID exclusion = 0
> >
> > DEBUG 3: Rejected: obs type = 0
> >
> > DEBUG 3: Rejected: valid time = 0
> >
> > DEBUG 3: Rejected: bad obs value = 0
> >
> > DEBUG 3: Rejected: off the grid = 1128
> >
> > DEBUG 3: Rejected: topography = 0
> >
> > DEBUG 3: Rejected: level mismatch = 0
> >
> > DEBUG 3: Rejected: quality marker = 0
> >
> > DEBUG 3: Rejected: message type = 0
> >
> > DEBUG 3: Rejected: masking region = 0
> >
> > DEBUG 3: Rejected: bad fcst value = 0
> >
> > DEBUG 3: Rejected: duplicates = 0
> >
> > DEBUG 2: Computing Probabilistic Statistics.
> >
> > DEBUG 2:
> >
> > DEBUG 2:
> >
> >
----------------------------------------------------------------------
> > ------
> > ----
> >
> > DEBUG 2:
> >
> > WARNING:
> >
> > WARNING: check_hdr_str() -> null string!
> >
> > WARNING:
> >
> > DEBUG 2: Processing /A006 versus APCP_06/L0, for observation type
> > ADPSFC, over region FULL, for interpolation method NEAREST(1),
using 611
> pairs.
> >
> > DEBUG 3: Number of matched pairs = 611
> >
> > DEBUG 3: Observations processed = 1739
> >
> > DEBUG 3: Rejected: SID exclusion = 0
> >
> > DEBUG 3: Rejected: obs type = 0
> >
> > DEBUG 3: Rejected: valid time = 0
> >
> > DEBUG 3: Rejected: bad obs value = 0
> >
> > DEBUG 3: Rejected: off the grid = 1128
> >
> > DEBUG 3: Rejected: topography = 0
> >
> > DEBUG 3: Rejected: level mismatch = 0
> >
> > DEBUG 3: Rejected: quality marker = 0
> >
> > DEBUG 3: Rejected: message type = 0
> >
> > DEBUG 3: Rejected: masking region = 0
> >
> > DEBUG 3: Rejected: bad fcst value = 0
> >
> > DEBUG 3: Rejected: duplicates = 0
> >
> > DEBUG 2: Computing Probabilistic Statistics.
> >
> > DEBUG 2:
> >
> > DEBUG 2:
> >
> >
----------------------------------------------------------------------
> > ------
> > ----
> >
> > DEBUG 2:
> >
> > WARNING:
> >
> > WARNING: check_hdr_str() -> null string!
> >
> > WARNING:
> >
> > DEBUG 2: Processing /A006 versus APCP_06/L0, for observation type
> > ADPSFC, over region FULL, for interpolation method NEAREST(1),
using 611
> pairs.
> >
> > DEBUG 3: Number of matched pairs = 611
> >
> > DEBUG 3: Observations processed = 1739
> >
> > DEBUG 3: Rejected: SID exclusion = 0
> >
> > DEBUG 3: Rejected: obs type = 0
> >
> > DEBUG 3: Rejected: valid time = 0
> >
> > DEBUG 3: Rejected: bad obs value = 0
> >
> > DEBUG 3: Rejected: off the grid = 1128
> >
> > DEBUG 3: Rejected: topography = 0
> >
> > DEBUG 3: Rejected: level mismatch = 0
> >
> > DEBUG 3: Rejected: quality marker = 0
> >
> > DEBUG 3: Rejected: message type = 0
> >
> > DEBUG 3: Rejected: masking region = 0
> >
> > DEBUG 3: Rejected: bad fcst value = 0
> >
> > DEBUG 3: Rejected: duplicates = 0
> >
> > DEBUG 2: Computing Probabilistic Statistics.
> >
> > DEBUG 2:
> >
> > DEBUG 2:
> >
> >
----------------------------------------------------------------------
> > ------
> > ----
> >
> > DEBUG 2:
> >
> > DEBUG 1: Output file:
> >
> >
/h/data/global/WXQC/data/met/ptstat/mdlob_pairs/meps/TP/point_stat_mep
> > s_conu
> > s_480000L_20190806_000000V.stat
> >
> > DEBUG 1: Output file:
> >
> >
/h/data/global/WXQC/data/met/ptstat/mdlob_pairs/meps/TP/point_stat_mep
> > s_conu
> > s_480000L_20190806_000000V_pct.txt
> >
> > DEBUG 1: Output file:
> >
> >
/h/data/global/WXQC/data/met/ptstat/mdlob_pairs/meps/TP/point_stat_mep
> > s_conu
> > s_480000L_20190806_000000V_pstd.txt
> >
> > Processing 54hour
> >
> >
> >
> > Thanks
> >
> > Bob
> >
> >
> >
>
>
>
>
------------------------------------------------
Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #91448] Changing Units in STATS files
From: robert.craig.2 at us.af.mil
Time: Fri Aug 09 11:06:48 2019
Thanks John, that fixed the problem.
Bob
-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Friday, August 9, 2019 11:46 AM
To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
<robert.craig.2 at us.af.mil>
Cc: SITTEL, MATTHEW C CTR USAF AFMC AFLCMC/HBAW-OL
<matthew.sittel.ctr at us.af.mil>
Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #91448] Changing
Units in STATS files
Bob,
Thanks for sending the additional sample data. Looking in your GRIB
table, I see this entry:
* 133 131 57 2 "PQP1" "Prob precip >.01" "%"*
To remember what those numbers mean, I found the following in the
data/config/README file...
BUT I FOUND AN ERROR IN THE README FILE WHERE WE HAD TWO COLUMNS
TRANSPOSED. HERE'S THE CORRECT COLUMN DESCRIPTIONS:
*GRIB1 table files begin with "grib1" prefix and end with a ".txt"
suffix.The first line of the file must contain "GRIB1".The following
lines consist of 4 integers followed by 3 strings:Column 1: GRIB code
(e.g. 11 for temperature)Column 2: parameter table version
numberColumn 3: center id (e.g. 07 for US Weather Service- National
Met. Center)Column 4: subcenter idColumn 5: variable nameColumn 6:
variable descriptionColumn 7: units*
Next, I checked your original Point-Stat config file and see that
you're trying to match GRIB code 206 (GRIB1_code=206;). I used wgrib
to find which record contains GRIB code 206... and see that it's
record number 4:
*wgrib grib.2019080500.0006 | grep "kpds5=206"Undefined parameter
table (center 57-4 table 3), using NCEP-
opn4:4317582:d=19080500:UVI:kpds5=206:kpds6=1:kpds7=0:TR=4:P1=0:P2=6:TimeU=1:sfc:0-
6hr
acc:NAve=0*
So I used wgrib again to dump verbose info about that record:
*wgrib -V -d 4 grib.2019080500.0006...rec 4:4317582:date 2019080500
UVI
kpds5=206 kpds6=1 kpds7=0 levels=(0,0) grid=255 sfc 0-6hr acc:...
center
57 subcenter 4 process 99 Table 3 scan: WE:SN winds(grid) *
*...*
Using this info (center=57, subcenter=4, and table=3), I modified your
PQP1 table entry to be:
* 206 3 57 4 "PQP1" "Prob precip >.01" "%"*
Extending that for the other variables yields (and I just made up
variable
names):
*cat grib1_af_prob_precip.txt*
*GRIB1*
*206 3 57 4 "PQP01" "Prob precip >.01" "%"*
*222 3 57 4 "PQP10" "Prob precip >.10" "%"*
*223 3 57 4 "PQP25" "Prob precip >.25" "%"*
*224 3 57 4 "PQP50" "Prob precip >.50" "%"*
*226 3 57 4 "PQ1P0" "Prob precip >1.0" "%"*
So I set MET_GRIB_TABLES=grib1_af_prob_precip.txt
And that enables me to run Point-Stat configured like this:
fcst = {
field = [
{ name="PQP01"; level = "A006"; prob = TRUE;},
{ name="PQP10"; level = "A006"; prob = TRUE;},
{ name="PQP25"; level = "A006"; prob = TRUE;},
{ name="PQP50"; level = "A006"; prob = TRUE;},
{ name="PQ1P0"; level = "A006"; prob = TRUE;}
];
cat_thresh=[==0.1];
}
I'll correct that documentation mistake in met-8.1.2 and met-9.0.
Thanks,
John
On Fri, Aug 9, 2019 at 10:18 AM robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=91448 >
>
>
> Update, since I sent you the last email, I add PCP1 to code 206 in
our
> default
> grib1 table ( it wasn't in the one I attached on the last email)..
I
> then tried to run with name=PQP1; GRIB1_ptv=2; level = "A006";
> prob=TRUE; but MET still wasn't finding the grib record.
>
>
> Bob
> -----Original Message-----
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Thursday, August 8, 2019 6:06 PM
> To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
> <robert.craig.2 at us.af.mil>
> Cc: SITTEL, MATTHEW C CTR USAF AFMC AFLCMC/HBAW-OL <
> matthew.sittel.ctr at us.af.mil>
> Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #91448] Changing
Units
> in STATS files
>
> Bob,
>
> Thanks for sending the sample data. Looking at the .stat output
file
> and the log messages you sent, I noticed something pretty
interesting.
> The forecast variables have an empty "name". You can see this in
the
> log
> messages:
> * DEBUG 2: Processing /A006 versus APCP_06/L0, ...*
>
> And you can see this in the FCST_VAR output column in the .stat
file,
> where FCST_VAR=NA. That means that Point-Stat doesn't have a string
> to assign to your forecast data which makes it pretty difficult to
> distinguish in the output!
>
> Looking in your Point-Stat config file, I see why...
>
> fcst = {
> field = [
> { GRIB1_code=206; level = "A006"; prob = TRUE;},
> { GRIB1_code=222; level = "A006"; prob = TRUE;},
> { GRIB1_code=223; level = "A006"; prob = TRUE;},
> { GRIB1_code=224; level = "A006"; prob = TRUE;},
> { GRIB1_code=226; level = "A006"; prob = TRUE;}
>
> ];
> cat_thresh=[==0.1];
> }
>
> You're specifying the GRIB1_code instead of a "name" string. Since
I
> don't have your exact input forecast data, I can't exactly replicate
> this behavior. I did try to mimic with the data we have here, but
> wasn't able to get the exact behavior. I used TMP and RH, but
> consistently got units of K and %.
>
> Here's what's going on... you request GRIB1_code = 206...
> Point-Stat does a table lookup to find a matching line in the GRIB
tables.
> Since you didn't specify the parameter table version number, it uses
> the default value of 2. It finds a GRIB table record for ptv = 2
and
> code =
> 206 and uses the units specified on that line.
>
> You'd need to look in your GRIB tables, but perhaps for lines ptv =
2
> and code = 206, 222, 223, 224, and 226, you don't have the same
units
> values listed? If you're using a different parameter table version
> number, please specify that... for example...
> { GRIB1_code=206; *GRIB1_ptv=10;* level = "A006"; prob = TRUE;},
>
> So while using these GRIB1 codes to access the data is possible, it
> isn't sufficient to accurately grab the units string from the GRIB1
> tables. A better alternative would be to just get the typical
option
> of specifying the "name" to work well.
>
> Thanks,
> John
>
> On Thu, Aug 8, 2019 at 1:52 PM robert.craig.2 at us.af.mil via RT <
> met_help at ucar.edu> wrote:
>
> >
> > Thu Aug 08 13:51:49 2019: Request 91448 was acted upon.
> > Transaction: Ticket created by robert.craig.2 at us.af.mil
> > Queue: met_help
> > Subject: Changing Units in STATS files
> > Owner: Nobody
> > Requestors: robert.craig.2 at us.af.mil
> > Status: new
> > Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=91448
> > >
> >
> >
> > John, I am verifying precipitation accumulation probabilities. I
am
> > using a netcdf file with 6 hour accumulations. When I look
through
> > the stat file produced, the units field keeps changing values (ie.
> > w/m^2, gpm, kg/m^2,
> > 0..9,m,.etc) and there will be two unit types for each precip
threshold.
> > The listing indicates MET is using the correct grib record number
for the
> > variable. Is this the expected behavior or is something not
right. The
> > list of the run is below:
> >
> >
> >
> > DEBUG 1: Reading user-defined grib1 MET_GRIB_TABLES file:
> > /h/data/global/WXQC/data/met/config/grib2/grib1_af_131_57.txt
> >
> > DEBUG 1: Reading user-defined grib2 MET_GRIB_TABLES file:
> > /h/data/global/WXQC/data/met/config/grib2/grib2_geps.txt
> >
> > DEBUG 1: Default Config File:
> > /h/MET/share/met/config/PointStatConfig_default
> >
> > DEBUG 1: User Config File:
> >
> >
/h/data/global/WXQC/data/met/config/met_config/PointStatConfig_meps_
> > tp
> > _conus
> > _updated
> >
> > GSL_RNG_TYPE=mt19937
> >
> > GSL_RNG_SEED=18446744071888718941
> >
> > DEBUG 1: Forecast File:
> >
> >
/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.2019080
> > 40
> > 0.0048
> >
> > DEBUG 1: Observation File:
> > /h/data/global/WXQC/data/met/nc_obs/precip/2019080600.nc
> >
> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match
> > for VarInfo "/A006" in GRIB record 4 of GRIB file
> >
> >
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.201908
> > 04
> > 00.004
> > 8".
> >
> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
> > records matching VarInfo "/A006" in GRIB file
> >
> >
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.201908
> > 04
> > 00.004
> > 8".
> >
> > DEBUG 2:
> >
> > DEBUG 2:
> >
> >
--------------------------------------------------------------------
> > --
> > ------
> > ----
> >
> > DEBUG 2:
> >
> > DEBUG 2: Reading data for /A006.
> >
> > DEBUG 3: Use the forecast grid.
> >
> > DEBUG 3: Grid Definition: Projection: Lambert Conformal Nx: 1170
Ny:
> > 720
> > Lat_LL: 23.159 Lon_LL: 119.272 Lon_orient: 97.000 Alpha: 3171.947
Cone:
> > 0.616 Bx: 582.1130 By: 2385.7494
> >
> > DEBUG 2: Processing masking regions.
> >
> > DEBUG 3: Processing grid mask: FULL
> >
> > DEBUG 2: Processing geography data.
> >
> > DEBUG 3: Rescaling probabilistic field from [0,100] to [0,1].
> >
> > DEBUG 3: Applying conversion function.
> >
> > DEBUG 1: Regridding climatology CFSR_APCP_T01_00(*,*) to the
> > verification grid.
> >
> > DEBUG 3: For forecast valid at 20190806_000000, found 1
climatology
> > field(s)
> > valid at 20190806_000000.
> >
> > DEBUG 3: Found 1 climatology fields.
> >
> > DEBUG 2: For /A006 found 1 forecast levels, 1 climatology mean
> > levels, and
> > 0
> > climatology standard deviation levels.
> >
> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match
> > for VarInfo "/A006" in GRIB record 7 of GRIB file
> >
> >
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.201908
> > 04
> > 00.004
> > 8".
> >
> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
> > records matching VarInfo "/A006" in GRIB file
> >
> >
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.201908
> > 04
> > 00.004
> > 8".
> >
> > DEBUG 2:
> >
> > DEBUG 2:
> >
> >
--------------------------------------------------------------------
> > --
> > ------
> > ----
> >
> > DEBUG 2:
> >
> > DEBUG 2: Reading data for /A006.
> >
> > DEBUG 3: Rescaling probabilistic field from [0,100] to [0,1].
> >
> > DEBUG 3: Applying conversion function.
> >
> > DEBUG 1: Regridding climatology CFSR_APCP_T10_00(*,*) to the
> > verification grid.
> >
> > DEBUG 3: For forecast valid at 20190806_000000, found 1
climatology
> > field(s)
> > valid at 20190806_000000.
> >
> > DEBUG 3: Found 1 climatology fields.
> >
> > DEBUG 2: For /A006 found 1 forecast levels, 1 climatology mean
> > levels, and
> > 0
> > climatology standard deviation levels.
> >
> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match
> > for VarInfo "/A006" in GRIB record 8 of GRIB file
> >
> >
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.201908
> > 04
> > 00.004
> > 8".
> >
> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
> > records matching VarInfo "/A006" in GRIB file
> >
> >
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.201908
> > 04
> > 00.004
> > 8".
> >
> > DEBUG 2:
> >
> > DEBUG 2:
> >
> >
--------------------------------------------------------------------
> > --
> > ------
> > ----
> >
> > DEBUG 2:
> >
> > DEBUG 2: Reading data for /A006.
> >
> > DEBUG 3: Rescaling probabilistic field from [0,100] to [0,1].
> >
> > DEBUG 3: Applying conversion function.
> >
> > DEBUG 1: Regridding climatology CFSR_APCP_T25_00(*,*) to the
> > verification grid.
> >
> > DEBUG 3: For forecast valid at 20190806_000000, found 1
climatology
> > field(s)
> > valid at 20190806_000000.
> >
> > DEBUG 3: Found 1 climatology fields.
> >
> > DEBUG 2: For /A006 found 1 forecast levels, 1 climatology mean
> > levels, and
> > 0
> > climatology standard deviation levels.
> >
> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match
> > for VarInfo "/A006" in GRIB record 9 of GRIB file
> >
> >
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.201908
> > 04
> > 00.004
> > 8".
> >
> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
> > records matching VarInfo "/A006" in GRIB file
> >
> >
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.201908
> > 04
> > 00.004
> > 8".
> >
> > DEBUG 2:
> >
> > DEBUG 2:
> >
> >
--------------------------------------------------------------------
> > --
> > ------
> > ----
> >
> > DEBUG 2:
> >
> > DEBUG 2: Reading data for /A006.
> >
> > DEBUG 3: Rescaling probabilistic field from [0,100] to [0,1].
> >
> > DEBUG 3: Applying conversion function.
> >
> > DEBUG 1: Regridding climatology CFSR_APCP_T50_00(*,*) to the
> > verification grid.
> >
> > DEBUG 3: For forecast valid at 20190806_000000, found 1
climatology
> > field(s)
> > valid at 20190806_000000.
> >
> > DEBUG 3: Found 1 climatology fields.
> >
> > DEBUG 2: For /A006 found 1 forecast levels, 1 climatology mean
> > levels, and
> > 0
> > climatology standard deviation levels.
> >
> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match
> > for VarInfo "/A006" in GRIB record 11 of GRIB file
> >
> >
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.201908
> > 04
> > 00.004
> > 8".
> >
> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
> > records matching VarInfo "/A006" in GRIB file
> >
> >
"/h/data/global/ENSEMBLE/data/post/MEPS/CONUS/2019080400/grib.201908
> > 04
> > 00.004
> > 8".
> >
> > DEBUG 2:
> >
> > DEBUG 2:
> >
> >
--------------------------------------------------------------------
> > --
> > ------
> > ----
> >
> > DEBUG 2:
> >
> > DEBUG 2: Reading data for /A006.
> >
> > DEBUG 3: Rescaling probabilistic field from [0,100] to [0,1].
> >
> > DEBUG 3: Applying conversion function.
> >
> > DEBUG 1: Regridding climatology CFSR_APCP_T100_00(*,*) to the
> > verification grid.
> >
> > DEBUG 3: For forecast valid at 20190806_000000, found 1
climatology
> > field(s)
> > valid at 20190806_000000.
> >
> > DEBUG 3: Found 1 climatology fields.
> >
> > DEBUG 2: For /A006 found 1 forecast levels, 1 climatology mean
> > levels, and
> > 0
> > climatology standard deviation levels.
> >
> > DEBUG 2:
> >
> > DEBUG 2:
> >
> >
--------------------------------------------------------------------
> > --
> > ------
> > ----
> >
> > DEBUG 2:
> >
> > DEBUG 2: Searching 1739 observations from 1739 messages.
> >
> > DEBUG 2:
> >
> > DEBUG 2:
> >
> >
--------------------------------------------------------------------
> > --
> > ------
> > ----
> >
> > DEBUG 2:
> >
> > WARNING:
> >
> > WARNING: check_hdr_str() -> null string!
> >
> > WARNING:
> >
> > DEBUG 2: Processing /A006 versus APCP_06/L0, for observation type
> > ADPSFC, over region FULL, for interpolation method NEAREST(1),
using
> > 611
> pairs.
> >
> > DEBUG 3: Number of matched pairs = 611
> >
> > DEBUG 3: Observations processed = 1739
> >
> > DEBUG 3: Rejected: SID exclusion = 0
> >
> > DEBUG 3: Rejected: obs type = 0
> >
> > DEBUG 3: Rejected: valid time = 0
> >
> > DEBUG 3: Rejected: bad obs value = 0
> >
> > DEBUG 3: Rejected: off the grid = 1128
> >
> > DEBUG 3: Rejected: topography = 0
> >
> > DEBUG 3: Rejected: level mismatch = 0
> >
> > DEBUG 3: Rejected: quality marker = 0
> >
> > DEBUG 3: Rejected: message type = 0
> >
> > DEBUG 3: Rejected: masking region = 0
> >
> > DEBUG 3: Rejected: bad fcst value = 0
> >
> > DEBUG 3: Rejected: duplicates = 0
> >
> > DEBUG 2: Computing Probabilistic Statistics.
> >
> > DEBUG 2:
> >
> > DEBUG 2:
> >
> >
--------------------------------------------------------------------
> > --
> > ------
> > ----
> >
> > DEBUG 2:
> >
> > WARNING:
> >
> > WARNING: check_hdr_str() -> null string!
> >
> > WARNING:
> >
> > DEBUG 2: Processing /A006 versus APCP_06/L0, for observation type
> > ADPSFC, over region FULL, for interpolation method NEAREST(1),
using
> > 611
> pairs.
> >
> > DEBUG 3: Number of matched pairs = 611
> >
> > DEBUG 3: Observations processed = 1739
> >
> > DEBUG 3: Rejected: SID exclusion = 0
> >
> > DEBUG 3: Rejected: obs type = 0
> >
> > DEBUG 3: Rejected: valid time = 0
> >
> > DEBUG 3: Rejected: bad obs value = 0
> >
> > DEBUG 3: Rejected: off the grid = 1128
> >
> > DEBUG 3: Rejected: topography = 0
> >
> > DEBUG 3: Rejected: level mismatch = 0
> >
> > DEBUG 3: Rejected: quality marker = 0
> >
> > DEBUG 3: Rejected: message type = 0
> >
> > DEBUG 3: Rejected: masking region = 0
> >
> > DEBUG 3: Rejected: bad fcst value = 0
> >
> > DEBUG 3: Rejected: duplicates = 0
> >
> > DEBUG 2: Computing Probabilistic Statistics.
> >
> > DEBUG 2:
> >
> > DEBUG 2:
> >
> >
--------------------------------------------------------------------
> > --
> > ------
> > ----
> >
> > DEBUG 2:
> >
> > WARNING:
> >
> > WARNING: check_hdr_str() -> null string!
> >
> > WARNING:
> >
> > DEBUG 2: Processing /A006 versus APCP_06/L0, for observation type
> > ADPSFC, over region FULL, for interpolation method NEAREST(1),
using
> > 611
> pairs.
> >
> > DEBUG 3: Number of matched pairs = 611
> >
> > DEBUG 3: Observations processed = 1739
> >
> > DEBUG 3: Rejected: SID exclusion = 0
> >
> > DEBUG 3: Rejected: obs type = 0
> >
> > DEBUG 3: Rejected: valid time = 0
> >
> > DEBUG 3: Rejected: bad obs value = 0
> >
> > DEBUG 3: Rejected: off the grid = 1128
> >
> > DEBUG 3: Rejected: topography = 0
> >
> > DEBUG 3: Rejected: level mismatch = 0
> >
> > DEBUG 3: Rejected: quality marker = 0
> >
> > DEBUG 3: Rejected: message type = 0
> >
> > DEBUG 3: Rejected: masking region = 0
> >
> > DEBUG 3: Rejected: bad fcst value = 0
> >
> > DEBUG 3: Rejected: duplicates = 0
> >
> > DEBUG 2: Computing Probabilistic Statistics.
> >
> > DEBUG 2:
> >
> > DEBUG 2:
> >
> >
--------------------------------------------------------------------
> > --
> > ------
> > ----
> >
> > DEBUG 2:
> >
> > WARNING:
> >
> > WARNING: check_hdr_str() -> null string!
> >
> > WARNING:
> >
> > DEBUG 2: Processing /A006 versus APCP_06/L0, for observation type
> > ADPSFC, over region FULL, for interpolation method NEAREST(1),
using
> > 611
> pairs.
> >
> > DEBUG 3: Number of matched pairs = 611
> >
> > DEBUG 3: Observations processed = 1739
> >
> > DEBUG 3: Rejected: SID exclusion = 0
> >
> > DEBUG 3: Rejected: obs type = 0
> >
> > DEBUG 3: Rejected: valid time = 0
> >
> > DEBUG 3: Rejected: bad obs value = 0
> >
> > DEBUG 3: Rejected: off the grid = 1128
> >
> > DEBUG 3: Rejected: topography = 0
> >
> > DEBUG 3: Rejected: level mismatch = 0
> >
> > DEBUG 3: Rejected: quality marker = 0
> >
> > DEBUG 3: Rejected: message type = 0
> >
> > DEBUG 3: Rejected: masking region = 0
> >
> > DEBUG 3: Rejected: bad fcst value = 0
> >
> > DEBUG 3: Rejected: duplicates = 0
> >
> > DEBUG 2: Computing Probabilistic Statistics.
> >
> > DEBUG 2:
> >
> > DEBUG 2:
> >
> >
--------------------------------------------------------------------
> > --
> > ------
> > ----
> >
> > DEBUG 2:
> >
> > WARNING:
> >
> > WARNING: check_hdr_str() -> null string!
> >
> > WARNING:
> >
> > DEBUG 2: Processing /A006 versus APCP_06/L0, for observation type
> > ADPSFC, over region FULL, for interpolation method NEAREST(1),
using
> > 611
> pairs.
> >
> > DEBUG 3: Number of matched pairs = 611
> >
> > DEBUG 3: Observations processed = 1739
> >
> > DEBUG 3: Rejected: SID exclusion = 0
> >
> > DEBUG 3: Rejected: obs type = 0
> >
> > DEBUG 3: Rejected: valid time = 0
> >
> > DEBUG 3: Rejected: bad obs value = 0
> >
> > DEBUG 3: Rejected: off the grid = 1128
> >
> > DEBUG 3: Rejected: topography = 0
> >
> > DEBUG 3: Rejected: level mismatch = 0
> >
> > DEBUG 3: Rejected: quality marker = 0
> >
> > DEBUG 3: Rejected: message type = 0
> >
> > DEBUG 3: Rejected: masking region = 0
> >
> > DEBUG 3: Rejected: bad fcst value = 0
> >
> > DEBUG 3: Rejected: duplicates = 0
> >
> > DEBUG 2: Computing Probabilistic Statistics.
> >
> > DEBUG 2:
> >
> > DEBUG 2:
> >
> >
--------------------------------------------------------------------
> > --
> > ------
> > ----
> >
> > DEBUG 2:
> >
> > DEBUG 1: Output file:
> >
> >
/h/data/global/WXQC/data/met/ptstat/mdlob_pairs/meps/TP/point_stat_m
> > ep
> > s_conu
> > s_480000L_20190806_000000V.stat
> >
> > DEBUG 1: Output file:
> >
> >
/h/data/global/WXQC/data/met/ptstat/mdlob_pairs/meps/TP/point_stat_m
> > ep
> > s_conu
> > s_480000L_20190806_000000V_pct.txt
> >
> > DEBUG 1: Output file:
> >
> >
/h/data/global/WXQC/data/met/ptstat/mdlob_pairs/meps/TP/point_stat_m
> > ep
> > s_conu
> > s_480000L_20190806_000000V_pstd.txt
> >
> > Processing 54hour
> >
> >
> >
> > Thanks
> >
> > Bob
> >
> >
> >
>
>
>
>
------------------------------------------------
More information about the Met_help
mailing list