[Met_help] [rt.rap.ucar.edu #59890] History for Problem with Pairing NetCDF with GriB
John Halley Gotway via RT
met_help at ucar.edu
Fri Jan 18 12:34:31 MST 2013
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hi,
I'm trying to get MET v4.0 to create verification rank histogram output for AFWA's mesoscale ensemble. I'm using in-house, GriB-1 format WRF ensemble member output and verifying it with PREPBUFR-formatted obs that we are pulling via FTP from NCEP. We convert the PREPBUFR format to NetCDF using 'pb2nc'.
The problem I'm having is I cannot figure out what the correct settings are in the config file for ensemble_stat to get the model and observation fields to match up.
I've attached the NetCDF 'dump' of the observation file's contents, the contents of my config file and the output I get when I run ensemble_stat. No matter what settings I pick, it always comes back with 0 pairs of data. The output suggests the model members are read in and that there are records in the observation file as well-both suggesting to me the data are read in correctly, but I'm not designating the correct pairing of them with the config file parameters.
The records I am trying to get are the ones where gc=11 and hgt=0, or simply the surface temperature reports from "ADPSFC" message type. I have read the user's guide and can't figure out what the right settings are for NetCDF; it is discussed briefly but not with enough detail to figure this out on my own, especially the ANNN, RNNN, etc, and the TMP(I,...,J,*,*) wording which is unclear. Can you take a look and advise as to the right settings?
Thanks,
Matt
==
netcdf 2013011412 {
dimensions:
mxstr = 16 ;
hdr_arr_len = 3 ;
obs_arr_len = 5 ;
nobs = UNLIMITED ; // (219303 currently)
nhdr = 40233 ;
variables:
float obs_arr(nobs, obs_arr_len) ;
obs_arr:long_name = "array of observation values" ;
obs_arr:_fill_value = -9999.f ;
obs_arr:columns = "hdr_id gc lvl hgt ob" ;
obs_arr:hdr_id_long_name = "index of matching header data" ;
obs_arr:gc_long_name = "grib code corresponding to the observation type" ;
obs_arr:lvl_long_name = "pressure level (hPa) or accumulation interval (sec)" ;
obs_arr:hgt_long_name = "height in meters above sea level (msl)" ;
obs_arr:ob_long_name = "observation value" ;
char hdr_typ(nhdr, mxstr) ;
hdr_typ:long_name = "message type" ;
char hdr_sid(nhdr, mxstr) ;
hdr_sid:long_name = "station identification" ;
char hdr_vld(nhdr, mxstr) ;
hdr_vld:long_name = "valid time" ;
hdr_vld:units = "YYYYMMDD_HHMMSS" ;
float hdr_arr(nhdr, hdr_arr_len) ;
hdr_arr:long_name = "array of observation station header values" ;
hdr_arr:_fill_value = -9999.f ;
hdr_arr:columns = "lat lon elv" ;
hdr_arr:lat_long_name = "latitude" ;
hdr_arr:lat_units = "degrees_north" ;
hdr_arr:lon_long_name = "longitude" ;
hdr_arr:lon_units = "degrees_east" ;
hdr_arr:elv_long_name = "elevation" ;
hdr_arr:elv_units = "meters above sea level (msl)" ;
// global attributes:
:FileOrigins = "File /h/data/global/WXQC/data//obs_prep/2013011412 generated 20130115_003309 UTC on host dev8dlsn1 by the MET pb2nc tool" ;
:MET_version = "V4.0" ;
:MET_tool = "pb2nc" ;
data:
40013, 7, 1016.4, 0, 0,
40014, 7, 1012.2, 0, 0,
40015, 7, 1024.4, 0, 0,
40016, 11, 1030.6, 0, 236.55,
40016, 7, 1030.6, 0, 0,
40016, 2, 1030.6, 0, 103060,
40017, 11, 1030.6, 0, 236.55,
40017, 7, 1030.6, 0, 0,
40017, 2, 1030.6, 0, 103060,
40018, 11, 1030.9, 0, 235.95,
40018, 7, 1030.9, 0, 0,
40018, 2, 1030.9, 0, 103090,
40019, 11, 1032.9, 0, 239.85,
40019, 7, 1032.9, 0, 0,
40019, 2, 1032.9, 0, 103290,
40020, 11, 1034.1, 0, 240.65,
40020, 7, 1034.1, 0, 0,
40020, 2, 1034.1, 0, 103410,
////////////////////////////////////////////////////////////////////////////////
//
// Ensemble-Stat configuration file.
//
// For additional information, see the MET_BASE/data/config/README file.
//
////////////////////////////////////////////////////////////////////////////////
//
// Output model name to be written
//
model = "WRF";
////////////////////////////////////////////////////////////////////////////////
//
// Ensemble product fields to be processed
//
ens = {
ens_thresh = 1.0;
vld_thresh = 1.0;
field = [
{
name = "TMP";
level = [ "Z2" ];
cat_thresh = [ >=300.0 ];
}
];
}
////////////////////////////////////////////////////////////////////////////////
//
// Forecast and observation fields to be verified
//
fcst = {
message_type = [ "ADPSFC" ];
field = [
{
name = "TMP";
level = "Z2";
}
];
}
obs=fcst;
////////////////////////////////////////////////////////////////////////////////
//
// Point observation time window
//
obs_window = {
beg = -1800;
end = 1800;
}
////////////////////////////////////////////////////////////////////////////////
//
// Verification masking regions
//
mask = {
grid = [ "FULL" ];
poly = [];
sid = "";
};
////////////////////////////////////////////////////////////////////////////////
//
// Interpolation methods
//
interp = {
field = OBS;
vld_thresh = 1.0;
type = [
{
method = BILIN;
width = 2;
}
];
};
////////////////////////////////////////////////////////////////////////////////
//
// Statistical output types
//
output_flag = {
rhist = STAT;
orank = NONE;
};
////////////////////////////////////////////////////////////////////////////////
//
// Ensemble product output types
//
ensemble_flag = {
mean = TRUE;
stdev = TRUE;
minus = FALSE;
plus = FALSE;
min = FALSE;
max = FALSE;
range = FALSE;
vld_count = FALSE;
frequency = FALSE;
rank = TRUE;
};
////////////////////////////////////////////////////////////////////////////////
//
// Random number generator
//
rng = {
type = "mt19937";
seed = "1";
}
////////////////////////////////////////////////////////////////////////////////
duplicate_flag = NONE;
output_prefix = "";
version = "V4.0";
////////////////////////////////////////////////////////////////////////////////
==
Obs file: /h/data/global/WXQC/data/obs_prep/2013011412
DEBUG 1: Default Config File: /home/qcteam/METv4.0/data/config/EnsembleStatConfig_default
DEBUG 1: User Config File: /home/qcteam/METv4.0/cfg/ensemble_stat.cfg
DEBUG 1: Ensemble Files[10]:
DEBUG 1: /prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00
DEBUG 1: /prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00
DEBUG 1: /prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00
DEBUG 1: /prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00
DEBUG 1: /prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00
DEBUG 1: /prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00
DEBUG 1: /prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00
DEBUG 1: /prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00
DEBUG 1: /prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00
DEBUG 1: /prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00
DEBUG 1: Point Observation Files[1]:
DEBUG 1: /h/data/global/WXQC/data/obs_prep/2013011412
DEBUG 2:
DEBUG 2: --------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Processing ensemble field: TMP/Z2
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file "/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB records matching VarInfo "TMP/Z2" in GRIB file "/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file "/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB records matching VarInfo "TMP/Z2" in GRIB file "/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file "/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB records matching VarInfo "TMP/Z2" in GRIB file "/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file "/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB records matching VarInfo "TMP/Z2" in GRIB file "/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file "/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB records matching VarInfo "TMP/Z2" in GRIB file "/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file "/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB records matching VarInfo "TMP/Z2" in GRIB file "/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file "/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB records matching VarInfo "TMP/Z2" in GRIB file "/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file "/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB records matching VarInfo "TMP/Z2" in GRIB file "/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file "/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB records matching VarInfo "TMP/Z2" in GRIB file "/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file "/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB records matching VarInfo "TMP/Z2" in GRIB file "/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
DEBUG 2:
DEBUG 2: --------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Processing point observation file: /h/data/global/WXQC/data/obs_prep/2013011412
DEBUG 2: Searching 219303 observations from 40233 header messages.
DEBUG 2:
DEBUG 2: --------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Processing ensemble file: /prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file "/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB records matching VarInfo "TMP/Z2" in GRIB file "/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
DEBUG 2: For TMP/Z2 found 1 forecast levels.
DEBUG 2:
DEBUG 2: --------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Processing ensemble file: /prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file "/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB records matching VarInfo "TMP/Z2" in GRIB file "/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
DEBUG 2: For TMP/Z2 found 1 forecast levels.
DEBUG 2:
DEBUG 2: --------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Processing ensemble file: /prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file "/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB records matching VarInfo "TMP/Z2" in GRIB file "/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
DEBUG 2: For TMP/Z2 found 1 forecast levels.
DEBUG 2:
DEBUG 2: --------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Processing ensemble file: /prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file "/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB records matching VarInfo "TMP/Z2" in GRIB file "/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
DEBUG 2: For TMP/Z2 found 1 forecast levels.
DEBUG 2:
DEBUG 2: --------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Processing ensemble file: /prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file "/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB records matching VarInfo "TMP/Z2" in GRIB file "/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
DEBUG 2: For TMP/Z2 found 1 forecast levels.
DEBUG 2:
DEBUG 2: --------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Processing ensemble file: /prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file "/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB records matching VarInfo "TMP/Z2" in GRIB file "/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
DEBUG 2: For TMP/Z2 found 1 forecast levels.
DEBUG 2:
DEBUG 2: --------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Processing ensemble file: /prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file "/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB records matching VarInfo "TMP/Z2" in GRIB file "/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
DEBUG 2: For TMP/Z2 found 1 forecast levels.
DEBUG 2:
DEBUG 2: --------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Processing ensemble file: /prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file "/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB records matching VarInfo "TMP/Z2" in GRIB file "/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
DEBUG 2: For TMP/Z2 found 1 forecast levels.
DEBUG 2:
DEBUG 2: --------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Processing ensemble file: /prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file "/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB records matching VarInfo "TMP/Z2" in GRIB file "/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
DEBUG 2: For TMP/Z2 found 1 forecast levels.
DEBUG 2:
DEBUG 2: --------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Processing ensemble file: /prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file "/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB records matching VarInfo "TMP/Z2" in GRIB file "/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
DEBUG 2: For TMP/Z2 found 1 forecast levels.
DEBUG 2:
DEBUG 2: --------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Processing point verification TMP/Z2 versus TMP/Z2, for observation type ADPSFC, over region FULL, for interpolation method BILIN(4), using 0 pairs.
DEBUG 2:
DEBUG 2: --------------------------------------------------------------------------------
DEBUG 2:
DEBUG 1: Output file: /home/qcteam/METv4.0/out/ensemble_stat/ensemble_stat_20130114_120000V.stat
DEBUG 1: Output file: /home/qcteam/METv4.0/out/ensemble_stat/ensemble_stat_20130114_120000V_ens.nc
// SIGNED //
Matthew C. Sittel
Associate Scientist
University Corporation for Atmospheric Research
16WS/WXN, Offutt AFB, NE
(402) 294-3473 DSN: 271-3473
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #59890] Problem with Pairing NetCDF with GriB
From: John Halley Gotway
Time: Tue Jan 15 15:09:51 2013
Matt,
So the question is basically, why are you getting 0 matched pairs from
ensemble-stat. If this were the point-stat tool, the answer would be
easy. In point-stat, when you turn up the verbosity level, for each
verification task, a list of reasons for why observations were not
used is
printed out. And that's often an easy way to diagnose the problem.
But that functionality has not been implemented for ensemble-stat.
Though
the problem you're having demonstrates the need!
There are several possibilities:
- the observations may not fall within the matching time window
- the observations may not fall within the model's grid
- perhaps there are no ADPSFC observations of TMP in your dataset
There are different steps I'd take to rule out each of these cases -
mostly making use of the plot_data_plane and plot_point_obs utilities.
But perhaps the easiest thing to do is to have you run one of your
forecast files and the NetCDF output of PB2NC through point-stat to
verify
TMP/Z2.
Presumably, you'll get 0 matched pairs again. But when you increase
the
verbosity level (-v 3, I think), you'll see a list of reasons why the
obs
weren't used. Send me that list and then we'll know how to proceed.
Alternatively, you could just send me one or two of your GRIB files,
the
NetCDF point observation file, and your config file, and I could debug
it
here. If you choose to go that route, follow the instructions here:
http://www.dtcenter.org/met/users/support/met_help.php#ftp
Your config file settings look fine. The instructions for specifying
NetCDF config file stuff only applies to gridded NetCDF data. But
your
gridded data is in GRIB format. For point observations, we chose the
convention of specifying them as if they were GRIB files. That's why
we
include a GRIB code for each observation value in the data.
Just let me know which route you decide on. And I'm happy to help get
you
up and running.
Thanks,
John
>
> Tue Jan 15 09:22:51 2013: Request 59890 was acted upon.
> Transaction: Ticket created by matthew.sittel.ctr at offutt.af.mil
> Queue: met_help
> Subject: Problem with Pairing NetCDF with GriB
> Owner: Nobody
> Requestors: matthew.sittel.ctr at offutt.af.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59890 >
>
>
> Hi,
>
> I'm trying to get MET v4.0 to create verification rank histogram
output
> for AFWA's mesoscale ensemble. I'm using in-house, GriB-1 format
WRF
> ensemble member output and verifying it with PREPBUFR-formatted obs
that
> we are pulling via FTP from NCEP. We convert the PREPBUFR format to
> NetCDF using 'pb2nc'.
>
> The problem I'm having is I cannot figure out what the correct
settings
> are in the config file for ensemble_stat to get the model and
observation
> fields to match up.
>
> I've attached the NetCDF 'dump' of the observation file's contents,
the
> contents of my config file and the output I get when I run
ensemble_stat.
> No matter what settings I pick, it always comes back with 0 pairs
of
> data. The output suggests the model members are read in and that
there
> are records in the observation file as well-both suggesting to me
the data
> are read in correctly, but I'm not designating the correct pairing
of them
> with the config file parameters.
>
> The records I am trying to get are the ones where gc=11 and hgt=0,
or
> simply the surface temperature reports from "ADPSFC" message type.
I have
> read the user's guide and can't figure out what the right settings
are for
> NetCDF; it is discussed briefly but not with enough detail to figure
this
> out on my own, especially the ANNN, RNNN, etc, and the
TMP(I,...,J,*,*)
> wording which is unclear. Can you take a look and advise as to the
right
> settings?
>
> Thanks,
> Matt
>
> ==
> netcdf 2013011412 {
> dimensions:
> mxstr = 16 ;
> hdr_arr_len = 3 ;
> obs_arr_len = 5 ;
> nobs = UNLIMITED ; // (219303 currently)
> nhdr = 40233 ;
> variables:
> float obs_arr(nobs, obs_arr_len) ;
> obs_arr:long_name = "array of observation values" ;
> obs_arr:_fill_value = -9999.f ;
> obs_arr:columns = "hdr_id gc lvl hgt ob" ;
> obs_arr:hdr_id_long_name = "index of matching header
data"
> ;
> obs_arr:gc_long_name = "grib code corresponding to
the
> observation type" ;
> obs_arr:lvl_long_name = "pressure level (hPa) or
> accumulation interval (sec)" ;
> obs_arr:hgt_long_name = "height in meters above sea
level
> (msl)" ;
> obs_arr:ob_long_name = "observation value" ;
> char hdr_typ(nhdr, mxstr) ;
> hdr_typ:long_name = "message type" ;
> char hdr_sid(nhdr, mxstr) ;
> hdr_sid:long_name = "station identification" ;
> char hdr_vld(nhdr, mxstr) ;
> hdr_vld:long_name = "valid time" ;
> hdr_vld:units = "YYYYMMDD_HHMMSS" ;
> float hdr_arr(nhdr, hdr_arr_len) ;
> hdr_arr:long_name = "array of observation station
header
> values" ;
> hdr_arr:_fill_value = -9999.f ;
> hdr_arr:columns = "lat lon elv" ;
> hdr_arr:lat_long_name = "latitude" ;
> hdr_arr:lat_units = "degrees_north" ;
> hdr_arr:lon_long_name = "longitude" ;
> hdr_arr:lon_units = "degrees_east" ;
> hdr_arr:elv_long_name = "elevation" ;
> hdr_arr:elv_units = "meters above sea level (msl)" ;
>
> // global attributes:
> :FileOrigins = "File
> /h/data/global/WXQC/data//obs_prep/2013011412 generated
> 20130115_003309 UTC on host dev8dlsn1 by the MET pb2nc
> tool" ;
> :MET_version = "V4.0" ;
> :MET_tool = "pb2nc" ;
> data:
> 40013, 7, 1016.4, 0, 0,
> 40014, 7, 1012.2, 0, 0,
> 40015, 7, 1024.4, 0, 0,
> 40016, 11, 1030.6, 0, 236.55,
> 40016, 7, 1030.6, 0, 0,
> 40016, 2, 1030.6, 0, 103060,
> 40017, 11, 1030.6, 0, 236.55,
> 40017, 7, 1030.6, 0, 0,
> 40017, 2, 1030.6, 0, 103060,
> 40018, 11, 1030.9, 0, 235.95,
> 40018, 7, 1030.9, 0, 0,
> 40018, 2, 1030.9, 0, 103090,
> 40019, 11, 1032.9, 0, 239.85,
> 40019, 7, 1032.9, 0, 0,
> 40019, 2, 1032.9, 0, 103290,
> 40020, 11, 1034.1, 0, 240.65,
> 40020, 7, 1034.1, 0, 0,
> 40020, 2, 1034.1, 0, 103410,
>
>
////////////////////////////////////////////////////////////////////////////////
> //
> // Ensemble-Stat configuration file.
> //
> // For additional information, see the MET_BASE/data/config/README
file.
> //
>
////////////////////////////////////////////////////////////////////////////////
>
> //
> // Output model name to be written
> //
> model = "WRF";
>
>
////////////////////////////////////////////////////////////////////////////////
>
> //
> // Ensemble product fields to be processed
> //
> ens = {
> ens_thresh = 1.0;
> vld_thresh = 1.0;
>
> field = [
> {
> name = "TMP";
> level = [ "Z2" ];
> cat_thresh = [ >=300.0 ];
> }
> ];
> }
>
>
////////////////////////////////////////////////////////////////////////////////
>
> //
> // Forecast and observation fields to be verified
> //
> fcst = {
> message_type = [ "ADPSFC" ];
> field = [
> {
> name = "TMP";
> level = "Z2";
> }
> ];
> }
>
> obs=fcst;
>
>
////////////////////////////////////////////////////////////////////////////////
>
> //
> // Point observation time window
> //
> obs_window = {
> beg = -1800;
> end = 1800;
> }
>
>
////////////////////////////////////////////////////////////////////////////////
>
> //
> // Verification masking regions
> //
> mask = {
> grid = [ "FULL" ];
> poly = [];
> sid = "";
> };
>
>
////////////////////////////////////////////////////////////////////////////////
>
> //
> // Interpolation methods
> //
> interp = {
> field = OBS;
> vld_thresh = 1.0;
>
> type = [
> {
> method = BILIN;
> width = 2;
> }
> ];
> };
>
>
////////////////////////////////////////////////////////////////////////////////
>
> //
> // Statistical output types
> //
> output_flag = {
> rhist = STAT;
> orank = NONE;
> };
>
>
////////////////////////////////////////////////////////////////////////////////
>
> //
> // Ensemble product output types
> //
> ensemble_flag = {
> mean = TRUE;
> stdev = TRUE;
> minus = FALSE;
> plus = FALSE;
> min = FALSE;
> max = FALSE;
> range = FALSE;
> vld_count = FALSE;
> frequency = FALSE;
> rank = TRUE;
> };
>
>
////////////////////////////////////////////////////////////////////////////////
>
> //
> // Random number generator
> //
> rng = {
> type = "mt19937";
> seed = "1";
> }
>
>
////////////////////////////////////////////////////////////////////////////////
>
> duplicate_flag = NONE;
> output_prefix = "";
> version = "V4.0";
>
>
////////////////////////////////////////////////////////////////////////////////
> ==
> Obs file: /h/data/global/WXQC/data/obs_prep/2013011412
> DEBUG 1: Default Config File:
> /home/qcteam/METv4.0/data/config/EnsembleStatConfig_default
> DEBUG 1: User Config File:
/home/qcteam/METv4.0/cfg/ensemble_stat.cfg
> DEBUG 1: Ensemble Files[10]:
> DEBUG 1:
>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00
> DEBUG 1:
>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00
> DEBUG 1:
>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00
> DEBUG 1:
>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00
> DEBUG 1:
>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00
> DEBUG 1:
>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00
> DEBUG 1:
>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00
> DEBUG 1:
>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00
> DEBUG 1:
>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00
> DEBUG 1:
>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00
> DEBUG 1: Point Observation Files[1]:
> DEBUG 1: /h/data/global/WXQC/data/obs_prep/2013011412
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Processing ensemble field: TMP/Z2
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "TMP/Z2" in GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "TMP/Z2" in GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "TMP/Z2" in GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "TMP/Z2" in GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "TMP/Z2" in GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "TMP/Z2" in GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "TMP/Z2" in GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "TMP/Z2" in GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "TMP/Z2" in GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "TMP/Z2" in GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Processing point observation file:
> /h/data/global/WXQC/data/obs_prep/2013011412
> DEBUG 2: Searching 219303 observations from 40233 header messages.
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Processing ensemble file:
>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "TMP/Z2" in GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
> DEBUG 2: For TMP/Z2 found 1 forecast levels.
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Processing ensemble file:
>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "TMP/Z2" in GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
> DEBUG 2: For TMP/Z2 found 1 forecast levels.
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Processing ensemble file:
>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "TMP/Z2" in GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
> DEBUG 2: For TMP/Z2 found 1 forecast levels.
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Processing ensemble file:
>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "TMP/Z2" in GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
> DEBUG 2: For TMP/Z2 found 1 forecast levels.
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Processing ensemble file:
>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "TMP/Z2" in GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
> DEBUG 2: For TMP/Z2 found 1 forecast levels.
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Processing ensemble file:
>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "TMP/Z2" in GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
> DEBUG 2: For TMP/Z2 found 1 forecast levels.
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Processing ensemble file:
>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "TMP/Z2" in GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
> DEBUG 2: For TMP/Z2 found 1 forecast levels.
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Processing ensemble file:
>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "TMP/Z2" in GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
> DEBUG 2: For TMP/Z2 found 1 forecast levels.
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Processing ensemble file:
>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "TMP/Z2" in GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
> DEBUG 2: For TMP/Z2 found 1 forecast levels.
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Processing ensemble file:
>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "TMP/Z2" in GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
> DEBUG 2: For TMP/Z2 found 1 forecast levels.
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Processing point verification TMP/Z2 versus TMP/Z2, for
> observation type ADPSFC, over region FULL, for interpolation method
> BILIN(4), using 0 pairs.
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 1: Output file:
>
/home/qcteam/METv4.0/out/ensemble_stat/ensemble_stat_20130114_120000V.stat
> DEBUG 1: Output file:
>
/home/qcteam/METv4.0/out/ensemble_stat/ensemble_stat_20130114_120000V_ens.nc
>
> // SIGNED //
> Matthew C. Sittel
> Associate Scientist
> University Corporation for Atmospheric Research
> 16WS/WXN, Offutt AFB, NE
> (402) 294-3473 DSN: 271-3473
>
>
------------------------------------------------
Subject: Problem with Pairing NetCDF with GriB
From: matthew.sittel.ctr at offutt.af.mil
Time: Wed Jan 16 12:44:03 2013
Hi John,
The plot thickens. When I ran point_stat on one ensemble member vs.
an observation file, MET returned a memory fault when it got to the
observation file read. I tried a second observation file and got the
same result.
I looked using 'ncdump' on the NetCDF observation data output from
'pb2nc', and confirmed there are ADPSFC records, records within the
CONUS grid and in the requested time window.
I used "-v 3" verbosity in the example I sent you... I think 4 is the
highest level of message? I tried rerunning this morning with "-v 6"
and didn't get any additional information as to what is happening with
the observation matching, nor any messages numbered above 4.
I have pushed the 10 ensemble members, observation file and config
file to the directory you listed. I had to create a "met_help"
directory under 'irap' as one was not there. Inside 'sittel' you will
find the files. Please note these were created in a Linux
environment-not sure if that will be an issue for you; it might depend
on your OS. We were forced to go to Linux here.
Thanks,
Matt
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Tuesday, January 15, 2013 4:10 PM
To: Sittel, Matthew C CTR USAF AFWA 16 WS/WXN
Cc: Craig, Robert J Civ USAF AFWA 16 WS/WXN
Subject: Re: [rt.rap.ucar.edu #59890] Problem with Pairing NetCDF with
GriB
Matt,
So the question is basically, why are you getting 0 matched pairs from
ensemble-stat. If this were the point-stat tool, the answer would be
easy. In point-stat, when you turn up the verbosity level, for each
verification task, a list of reasons for why observations were not
used is
printed out. And that's often an easy way to diagnose the problem.
But that functionality has not been implemented for ensemble-stat.
Though
the problem you're having demonstrates the need!
There are several possibilities:
- the observations may not fall within the matching time window
- the observations may not fall within the model's grid
- perhaps there are no ADPSFC observations of TMP in your dataset
There are different steps I'd take to rule out each of these cases -
mostly making use of the plot_data_plane and plot_point_obs utilities.
But perhaps the easiest thing to do is to have you run one of your
forecast files and the NetCDF output of PB2NC through point-stat to
verify
TMP/Z2.
Presumably, you'll get 0 matched pairs again. But when you increase
the
verbosity level (-v 3, I think), you'll see a list of reasons why the
obs
weren't used. Send me that list and then we'll know how to proceed.
Alternatively, you could just send me one or two of your GRIB files,
the
NetCDF point observation file, and your config file, and I could debug
it
here. If you choose to go that route, follow the instructions here:
http://www.dtcenter.org/met/users/support/met_help.php#ftp
Your config file settings look fine. The instructions for specifying
NetCDF config file stuff only applies to gridded NetCDF data. But
your
gridded data is in GRIB format. For point observations, we chose the
convention of specifying them as if they were GRIB files. That's why
we
include a GRIB code for each observation value in the data.
Just let me know which route you decide on. And I'm happy to help get
you
up and running.
Thanks,
John
>
> Tue Jan 15 09:22:51 2013: Request 59890 was acted upon.
> Transaction: Ticket created by matthew.sittel.ctr at offutt.af.mil
> Queue: met_help
> Subject: Problem with Pairing NetCDF with GriB
> Owner: Nobody
> Requestors: matthew.sittel.ctr at offutt.af.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59890 >
>
>
> Hi,
>
> I'm trying to get MET v4.0 to create verification rank histogram
output
> for AFWA's mesoscale ensemble. I'm using in-house, GriB-1 format
WRF
> ensemble member output and verifying it with PREPBUFR-formatted obs
that
> we are pulling via FTP from NCEP. We convert the PREPBUFR format to
> NetCDF using 'pb2nc'.
>
> The problem I'm having is I cannot figure out what the correct
settings
> are in the config file for ensemble_stat to get the model and
observation
> fields to match up.
>
> I've attached the NetCDF 'dump' of the observation file's contents,
the
> contents of my config file and the output I get when I run
ensemble_stat.
> No matter what settings I pick, it always comes back with 0 pairs
of
> data. The output suggests the model members are read in and that
there
> are records in the observation file as well-both suggesting to me
the data
> are read in correctly, but I'm not designating the correct pairing
of them
> with the config file parameters.
>
> The records I am trying to get are the ones where gc=11 and hgt=0,
or
> simply the surface temperature reports from "ADPSFC" message type.
I have
> read the user's guide and can't figure out what the right settings
are for
> NetCDF; it is discussed briefly but not with enough detail to figure
this
> out on my own, especially the ANNN, RNNN, etc, and the
TMP(I,...,J,*,*)
> wording which is unclear. Can you take a look and advise as to the
right
> settings?
>
> Thanks,
> Matt
>
> ==
> netcdf 2013011412 {
> dimensions:
> mxstr = 16 ;
> hdr_arr_len = 3 ;
> obs_arr_len = 5 ;
> nobs = UNLIMITED ; // (219303 currently)
> nhdr = 40233 ;
> variables:
> float obs_arr(nobs, obs_arr_len) ;
> obs_arr:long_name = "array of observation values" ;
> obs_arr:_fill_value = -9999.f ;
> obs_arr:columns = "hdr_id gc lvl hgt ob" ;
> obs_arr:hdr_id_long_name = "index of matching header
data"
> ;
> obs_arr:gc_long_name = "grib code corresponding to
the
> observation type" ;
> obs_arr:lvl_long_name = "pressure level (hPa) or
> accumulation interval (sec)" ;
> obs_arr:hgt_long_name = "height in meters above sea
level
> (msl)" ;
> obs_arr:ob_long_name = "observation value" ;
> char hdr_typ(nhdr, mxstr) ;
> hdr_typ:long_name = "message type" ;
> char hdr_sid(nhdr, mxstr) ;
> hdr_sid:long_name = "station identification" ;
> char hdr_vld(nhdr, mxstr) ;
> hdr_vld:long_name = "valid time" ;
> hdr_vld:units = "YYYYMMDD_HHMMSS" ;
> float hdr_arr(nhdr, hdr_arr_len) ;
> hdr_arr:long_name = "array of observation station
header
> values" ;
> hdr_arr:_fill_value = -9999.f ;
> hdr_arr:columns = "lat lon elv" ;
> hdr_arr:lat_long_name = "latitude" ;
> hdr_arr:lat_units = "degrees_north" ;
> hdr_arr:lon_long_name = "longitude" ;
> hdr_arr:lon_units = "degrees_east" ;
> hdr_arr:elv_long_name = "elevation" ;
> hdr_arr:elv_units = "meters above sea level (msl)" ;
>
> // global attributes:
> :FileOrigins = "File
> /h/data/global/WXQC/data//obs_prep/2013011412 generated
> 20130115_003309 UTC on host dev8dlsn1 by the MET pb2nc
> tool" ;
> :MET_version = "V4.0" ;
> :MET_tool = "pb2nc" ;
> data:
> 40013, 7, 1016.4, 0, 0,
> 40014, 7, 1012.2, 0, 0,
> 40015, 7, 1024.4, 0, 0,
> 40016, 11, 1030.6, 0, 236.55,
> 40016, 7, 1030.6, 0, 0,
> 40016, 2, 1030.6, 0, 103060,
> 40017, 11, 1030.6, 0, 236.55,
> 40017, 7, 1030.6, 0, 0,
> 40017, 2, 1030.6, 0, 103060,
> 40018, 11, 1030.9, 0, 235.95,
> 40018, 7, 1030.9, 0, 0,
> 40018, 2, 1030.9, 0, 103090,
> 40019, 11, 1032.9, 0, 239.85,
> 40019, 7, 1032.9, 0, 0,
> 40019, 2, 1032.9, 0, 103290,
> 40020, 11, 1034.1, 0, 240.65,
> 40020, 7, 1034.1, 0, 0,
> 40020, 2, 1034.1, 0, 103410,
>
>
////////////////////////////////////////////////////////////////////////////////
> //
> // Ensemble-Stat configuration file.
> //
> // For additional information, see the MET_BASE/data/config/README
file.
> //
>
////////////////////////////////////////////////////////////////////////////////
>
> //
> // Output model name to be written
> //
> model = "WRF";
>
>
////////////////////////////////////////////////////////////////////////////////
>
> //
> // Ensemble product fields to be processed
> //
> ens = {
> ens_thresh = 1.0;
> vld_thresh = 1.0;
>
> field = [
> {
> name = "TMP";
> level = [ "Z2" ];
> cat_thresh = [ >=300.0 ];
> }
> ];
> }
>
>
////////////////////////////////////////////////////////////////////////////////
>
> //
> // Forecast and observation fields to be verified
> //
> fcst = {
> message_type = [ "ADPSFC" ];
> field = [
> {
> name = "TMP";
> level = "Z2";
> }
> ];
> }
>
> obs=fcst;
>
>
////////////////////////////////////////////////////////////////////////////////
>
> //
> // Point observation time window
> //
> obs_window = {
> beg = -1800;
> end = 1800;
> }
>
>
////////////////////////////////////////////////////////////////////////////////
>
> //
> // Verification masking regions
> //
> mask = {
> grid = [ "FULL" ];
> poly = [];
> sid = "";
> };
>
>
////////////////////////////////////////////////////////////////////////////////
>
> //
> // Interpolation methods
> //
> interp = {
> field = OBS;
> vld_thresh = 1.0;
>
> type = [
> {
> method = BILIN;
> width = 2;
> }
> ];
> };
>
>
////////////////////////////////////////////////////////////////////////////////
>
> //
> // Statistical output types
> //
> output_flag = {
> rhist = STAT;
> orank = NONE;
> };
>
>
////////////////////////////////////////////////////////////////////////////////
>
> //
> // Ensemble product output types
> //
> ensemble_flag = {
> mean = TRUE;
> stdev = TRUE;
> minus = FALSE;
> plus = FALSE;
> min = FALSE;
> max = FALSE;
> range = FALSE;
> vld_count = FALSE;
> frequency = FALSE;
> rank = TRUE;
> };
>
>
////////////////////////////////////////////////////////////////////////////////
>
> //
> // Random number generator
> //
> rng = {
> type = "mt19937";
> seed = "1";
> }
>
>
////////////////////////////////////////////////////////////////////////////////
>
> duplicate_flag = NONE;
> output_prefix = "";
> version = "V4.0";
>
>
////////////////////////////////////////////////////////////////////////////////
> ==
> Obs file: /h/data/global/WXQC/data/obs_prep/2013011412
> DEBUG 1: Default Config File:
> /home/qcteam/METv4.0/data/config/EnsembleStatConfig_default
> DEBUG 1: User Config File:
/home/qcteam/METv4.0/cfg/ensemble_stat.cfg
> DEBUG 1: Ensemble Files[10]:
> DEBUG 1:
>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00
> DEBUG 1:
>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00
> DEBUG 1:
>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00
> DEBUG 1:
>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00
> DEBUG 1:
>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00
> DEBUG 1:
>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00
> DEBUG 1:
>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00
> DEBUG 1:
>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00
> DEBUG 1:
>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00
> DEBUG 1:
>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00
> DEBUG 1: Point Observation Files[1]:
> DEBUG 1: /h/data/global/WXQC/data/obs_prep/2013011412
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Processing ensemble field: TMP/Z2
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "TMP/Z2" in GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "TMP/Z2" in GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "TMP/Z2" in GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "TMP/Z2" in GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "TMP/Z2" in GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "TMP/Z2" in GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "TMP/Z2" in GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "TMP/Z2" in GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "TMP/Z2" in GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "TMP/Z2" in GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Processing point observation file:
> /h/data/global/WXQC/data/obs_prep/2013011412
> DEBUG 2: Searching 219303 observations from 40233 header messages.
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Processing ensemble file:
>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "TMP/Z2" in GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
> DEBUG 2: For TMP/Z2 found 1 forecast levels.
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Processing ensemble file:
>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "TMP/Z2" in GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
> DEBUG 2: For TMP/Z2 found 1 forecast levels.
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Processing ensemble file:
>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "TMP/Z2" in GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
> DEBUG 2: For TMP/Z2 found 1 forecast levels.
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Processing ensemble file:
>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "TMP/Z2" in GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
> DEBUG 2: For TMP/Z2 found 1 forecast levels.
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Processing ensemble file:
>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "TMP/Z2" in GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
> DEBUG 2: For TMP/Z2 found 1 forecast levels.
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Processing ensemble file:
>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "TMP/Z2" in GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
> DEBUG 2: For TMP/Z2 found 1 forecast levels.
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Processing ensemble file:
>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "TMP/Z2" in GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
> DEBUG 2: For TMP/Z2 found 1 forecast levels.
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Processing ensemble file:
>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "TMP/Z2" in GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
> DEBUG 2: For TMP/Z2 found 1 forecast levels.
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Processing ensemble file:
>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "TMP/Z2" in GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
> DEBUG 2: For TMP/Z2 found 1 forecast levels.
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Processing ensemble file:
>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "TMP/Z2" in GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
> DEBUG 2: For TMP/Z2 found 1 forecast levels.
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Processing point verification TMP/Z2 versus TMP/Z2, for
> observation type ADPSFC, over region FULL, for interpolation method
> BILIN(4), using 0 pairs.
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 1: Output file:
>
/home/qcteam/METv4.0/out/ensemble_stat/ensemble_stat_20130114_120000V.stat
> DEBUG 1: Output file:
>
/home/qcteam/METv4.0/out/ensemble_stat/ensemble_stat_20130114_120000V_ens.nc
>
> // SIGNED //
> Matthew C. Sittel
> Associate Scientist
> University Corporation for Atmospheric Research
> 16WS/WXN, Offutt AFB, NE
> (402) 294-3473 DSN: 271-3473
>
>
------------------------------------------------
Subject: Problem with Pairing NetCDF with GriB
From: John Halley Gotway
Time: Wed Jan 16 15:26:46 2013
Matt,
I ran METv4.0 point_stat for the first ensemble member using the
attached
configuration file and verbosity level 3. I got 0 matched pairs, but
the
following diagnostic information is included in the output:
DEBUG 2: Processing TMP/Z2 versus TMP/Z2, for observation type ADPSFC,
over region FULL, for interpolation method BILIN(4), using 0 pairs.
DEBUG 3: Number of matched pairs = 0
DEBUG 3: Observations processed = 219303
DEBUG 3: Rejected: GRIB code = 190302
DEBUG 3: Rejected: valid time = 29001
DEBUG 3: Rejected: bad obs value = 0
DEBUG 3: Rejected: off the grid = 0
DEBUG 3: Rejected: level mismatch = 0
DEBUG 3: Rejected: message type = 0
DEBUG 3: Rejected: masking region = 0
DEBUG 3: Rejected: bad fcst value = 0
This tells me that there's problem in the matching time window.
Running "wgrib -V" on that forecast file tells me that this is a 24-
hour
forecast initialized at 2013011412. So it's valid at 2013011512.
Your observation file is named 2013011512, but when you actually look
at
the times it contains, they're for 2013011412:
ncdump -v hdr_vld nc2013011512
(note, ncdump doesn't like file names that begin with a number, so I
stuck
an "nc" in front)
I reran point-stat a second time, but first added 3 0's to the "beg"
and
"end" times in the "obs_window" setting. When I did that, I got
further,
but still no matched pairs:
DEBUG 2: Processing TMP/Z2 versus TMP/Z2, for observation type ADPSFC,
over region FULL, for interpolation method BILIN(4), using 0 pairs.
DEBUG 3: Number of matched pairs = 0
DEBUG 3: Observations processed = 219303
DEBUG 3: Rejected: GRIB code = 190302
DEBUG 3: Rejected: valid time = 0
DEBUG 3: Rejected: bad obs value = 0
DEBUG 3: Rejected: off the grid = 22047
DEBUG 3: Rejected: level mismatch = 6177
DEBUG 3: Rejected: message type = 777
DEBUG 3: Rejected: masking region = 0
DEBUG 3: Rejected: bad fcst value = 0
So there just aren't any ADPSFC observations of 2-m temperature that
fall
within your d02 domain. I ran the plot_point_obs utility to verify
that
that's the case.
As for why you got a seg-fault, I don't know. It's possible that you
made
an error in your Point-Stat config file that caused it. I can't
really
debug it unless I can replicate it.
John
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59890 >
>
> Hi John,
>
> The plot thickens. When I ran point_stat on one ensemble member vs.
an
> observation file, MET returned a memory fault when it got to the
> observation file read. I tried a second observation file and got
the same
> result.
>
> I looked using 'ncdump' on the NetCDF observation data output from
> 'pb2nc', and confirmed there are ADPSFC records, records within the
CONUS
> grid and in the requested time window.
>
> I used "-v 3" verbosity in the example I sent you... I think 4 is
the
> highest level of message? I tried rerunning this morning with "-v
6" and
> didn't get any additional information as to what is happening with
the
> observation matching, nor any messages numbered above 4.
>
> I have pushed the 10 ensemble members, observation file and config
file to
> the directory you listed. I had to create a "met_help" directory
under
> 'irap' as one was not there. Inside 'sittel' you will find the
files.
> Please note these were created in a Linux environment-not sure if
that
> will be an issue for you; it might depend on your OS. We were
forced to
> go to Linux here.
>
> Thanks,
> Matt
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Tuesday, January 15, 2013 4:10 PM
> To: Sittel, Matthew C CTR USAF AFWA 16 WS/WXN
> Cc: Craig, Robert J Civ USAF AFWA 16 WS/WXN
> Subject: Re: [rt.rap.ucar.edu #59890] Problem with Pairing NetCDF
with
> GriB
>
> Matt,
>
> So the question is basically, why are you getting 0 matched pairs
from
> ensemble-stat. If this were the point-stat tool, the answer would
be
> easy. In point-stat, when you turn up the verbosity level, for each
> verification task, a list of reasons for why observations were not
used is
> printed out. And that's often an easy way to diagnose the problem.
>
> But that functionality has not been implemented for ensemble-stat.
Though
> the problem you're having demonstrates the need!
>
> There are several possibilities:
> - the observations may not fall within the matching time window
> - the observations may not fall within the model's grid
> - perhaps there are no ADPSFC observations of TMP in your dataset
>
> There are different steps I'd take to rule out each of these cases -
> mostly making use of the plot_data_plane and plot_point_obs
utilities.
> But perhaps the easiest thing to do is to have you run one of your
> forecast files and the NetCDF output of PB2NC through point-stat to
verify
> TMP/Z2.
>
> Presumably, you'll get 0 matched pairs again. But when you increase
the
> verbosity level (-v 3, I think), you'll see a list of reasons why
the obs
> weren't used. Send me that list and then we'll know how to proceed.
>
> Alternatively, you could just send me one or two of your GRIB files,
the
> NetCDF point observation file, and your config file, and I could
debug it
> here. If you choose to go that route, follow the instructions here:
> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>
> Your config file settings look fine. The instructions for
specifying
> NetCDF config file stuff only applies to gridded NetCDF data. But
your
> gridded data is in GRIB format. For point observations, we chose
the
> convention of specifying them as if they were GRIB files. That's
why we
> include a GRIB code for each observation value in the data.
>
> Just let me know which route you decide on. And I'm happy to help
get you
> up and running.
>
> Thanks,
> John
>
>>
>> Tue Jan 15 09:22:51 2013: Request 59890 was acted upon.
>> Transaction: Ticket created by matthew.sittel.ctr at offutt.af.mil
>> Queue: met_help
>> Subject: Problem with Pairing NetCDF with GriB
>> Owner: Nobody
>> Requestors: matthew.sittel.ctr at offutt.af.mil
>> Status: new
>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59890 >
>>
>>
>> Hi,
>>
>> I'm trying to get MET v4.0 to create verification rank histogram
output
>> for AFWA's mesoscale ensemble. I'm using in-house, GriB-1 format
WRF
>> ensemble member output and verifying it with PREPBUFR-formatted obs
that
>> we are pulling via FTP from NCEP. We convert the PREPBUFR format
to
>> NetCDF using 'pb2nc'.
>>
>> The problem I'm having is I cannot figure out what the correct
settings
>> are in the config file for ensemble_stat to get the model and
>> observation
>> fields to match up.
>>
>> I've attached the NetCDF 'dump' of the observation file's contents,
the
>> contents of my config file and the output I get when I run
>> ensemble_stat.
>> No matter what settings I pick, it always comes back with 0 pairs
of
>> data. The output suggests the model members are read in and that
there
>> are records in the observation file as well-both suggesting to me
the
>> data
>> are read in correctly, but I'm not designating the correct pairing
of
>> them
>> with the config file parameters.
>>
>> The records I am trying to get are the ones where gc=11 and hgt=0,
or
>> simply the surface temperature reports from "ADPSFC" message type.
I
>> have
>> read the user's guide and can't figure out what the right settings
are
>> for
>> NetCDF; it is discussed briefly but not with enough detail to
figure
>> this
>> out on my own, especially the ANNN, RNNN, etc, and the
TMP(I,...,J,*,*)
>> wording which is unclear. Can you take a look and advise as to the
>> right
>> settings?
>>
>> Thanks,
>> Matt
>>
>> ==
>> netcdf 2013011412 {
>> dimensions:
>> mxstr = 16 ;
>> hdr_arr_len = 3 ;
>> obs_arr_len = 5 ;
>> nobs = UNLIMITED ; // (219303 currently)
>> nhdr = 40233 ;
>> variables:
>> float obs_arr(nobs, obs_arr_len) ;
>> obs_arr:long_name = "array of observation values" ;
>> obs_arr:_fill_value = -9999.f ;
>> obs_arr:columns = "hdr_id gc lvl hgt ob" ;
>> obs_arr:hdr_id_long_name = "index of matching
header
>> data"
>> ;
>> obs_arr:gc_long_name = "grib code corresponding to
the
>> observation type" ;
>> obs_arr:lvl_long_name = "pressure level (hPa) or
>> accumulation interval (sec)" ;
>> obs_arr:hgt_long_name = "height in meters above sea
>> level
>> (msl)" ;
>> obs_arr:ob_long_name = "observation value" ;
>> char hdr_typ(nhdr, mxstr) ;
>> hdr_typ:long_name = "message type" ;
>> char hdr_sid(nhdr, mxstr) ;
>> hdr_sid:long_name = "station identification" ;
>> char hdr_vld(nhdr, mxstr) ;
>> hdr_vld:long_name = "valid time" ;
>> hdr_vld:units = "YYYYMMDD_HHMMSS" ;
>> float hdr_arr(nhdr, hdr_arr_len) ;
>> hdr_arr:long_name = "array of observation station
header
>> values" ;
>> hdr_arr:_fill_value = -9999.f ;
>> hdr_arr:columns = "lat lon elv" ;
>> hdr_arr:lat_long_name = "latitude" ;
>> hdr_arr:lat_units = "degrees_north" ;
>> hdr_arr:lon_long_name = "longitude" ;
>> hdr_arr:lon_units = "degrees_east" ;
>> hdr_arr:elv_long_name = "elevation" ;
>> hdr_arr:elv_units = "meters above sea level (msl)"
;
>>
>> // global attributes:
>> :FileOrigins = "File
>> /h/data/global/WXQC/data//obs_prep/2013011412 generated
>> 20130115_003309 UTC on host dev8dlsn1 by the MET pb2nc
>> tool" ;
>> :MET_version = "V4.0" ;
>> :MET_tool = "pb2nc" ;
>> data:
>> 40013, 7, 1016.4, 0, 0,
>> 40014, 7, 1012.2, 0, 0,
>> 40015, 7, 1024.4, 0, 0,
>> 40016, 11, 1030.6, 0, 236.55,
>> 40016, 7, 1030.6, 0, 0,
>> 40016, 2, 1030.6, 0, 103060,
>> 40017, 11, 1030.6, 0, 236.55,
>> 40017, 7, 1030.6, 0, 0,
>> 40017, 2, 1030.6, 0, 103060,
>> 40018, 11, 1030.9, 0, 235.95,
>> 40018, 7, 1030.9, 0, 0,
>> 40018, 2, 1030.9, 0, 103090,
>> 40019, 11, 1032.9, 0, 239.85,
>> 40019, 7, 1032.9, 0, 0,
>> 40019, 2, 1032.9, 0, 103290,
>> 40020, 11, 1034.1, 0, 240.65,
>> 40020, 7, 1034.1, 0, 0,
>> 40020, 2, 1034.1, 0, 103410,
>>
>>
////////////////////////////////////////////////////////////////////////////////
>> //
>> // Ensemble-Stat configuration file.
>> //
>> // For additional information, see the MET_BASE/data/config/README
file.
>> //
>>
////////////////////////////////////////////////////////////////////////////////
>>
>> //
>> // Output model name to be written
>> //
>> model = "WRF";
>>
>>
////////////////////////////////////////////////////////////////////////////////
>>
>> //
>> // Ensemble product fields to be processed
>> //
>> ens = {
>> ens_thresh = 1.0;
>> vld_thresh = 1.0;
>>
>> field = [
>> {
>> name = "TMP";
>> level = [ "Z2" ];
>> cat_thresh = [ >=300.0 ];
>> }
>> ];
>> }
>>
>>
////////////////////////////////////////////////////////////////////////////////
>>
>> //
>> // Forecast and observation fields to be verified
>> //
>> fcst = {
>> message_type = [ "ADPSFC" ];
>> field = [
>> {
>> name = "TMP";
>> level = "Z2";
>> }
>> ];
>> }
>>
>> obs=fcst;
>>
>>
////////////////////////////////////////////////////////////////////////////////
>>
>> //
>> // Point observation time window
>> //
>> obs_window = {
>> beg = -1800;
>> end = 1800;
>> }
>>
>>
////////////////////////////////////////////////////////////////////////////////
>>
>> //
>> // Verification masking regions
>> //
>> mask = {
>> grid = [ "FULL" ];
>> poly = [];
>> sid = "";
>> };
>>
>>
////////////////////////////////////////////////////////////////////////////////
>>
>> //
>> // Interpolation methods
>> //
>> interp = {
>> field = OBS;
>> vld_thresh = 1.0;
>>
>> type = [
>> {
>> method = BILIN;
>> width = 2;
>> }
>> ];
>> };
>>
>>
////////////////////////////////////////////////////////////////////////////////
>>
>> //
>> // Statistical output types
>> //
>> output_flag = {
>> rhist = STAT;
>> orank = NONE;
>> };
>>
>>
////////////////////////////////////////////////////////////////////////////////
>>
>> //
>> // Ensemble product output types
>> //
>> ensemble_flag = {
>> mean = TRUE;
>> stdev = TRUE;
>> minus = FALSE;
>> plus = FALSE;
>> min = FALSE;
>> max = FALSE;
>> range = FALSE;
>> vld_count = FALSE;
>> frequency = FALSE;
>> rank = TRUE;
>> };
>>
>>
////////////////////////////////////////////////////////////////////////////////
>>
>> //
>> // Random number generator
>> //
>> rng = {
>> type = "mt19937";
>> seed = "1";
>> }
>>
>>
////////////////////////////////////////////////////////////////////////////////
>>
>> duplicate_flag = NONE;
>> output_prefix = "";
>> version = "V4.0";
>>
>>
////////////////////////////////////////////////////////////////////////////////
>> ==
>> Obs file: /h/data/global/WXQC/data/obs_prep/2013011412
>> DEBUG 1: Default Config File:
>> /home/qcteam/METv4.0/data/config/EnsembleStatConfig_default
>> DEBUG 1: User Config File:
/home/qcteam/METv4.0/cfg/ensemble_stat.cfg
>> DEBUG 1: Ensemble Files[10]:
>> DEBUG 1:
>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00
>> DEBUG 1:
>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00
>> DEBUG 1:
>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00
>> DEBUG 1:
>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00
>> DEBUG 1:
>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00
>> DEBUG 1:
>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00
>> DEBUG 1:
>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00
>> DEBUG 1:
>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00
>> DEBUG 1:
>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00
>> DEBUG 1:
>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00
>> DEBUG 1: Point Observation Files[1]:
>> DEBUG 1: /h/data/global/WXQC/data/obs_prep/2013011412
>> DEBUG 2:
>> DEBUG 2:
>>
--------------------------------------------------------------------------------
>> DEBUG 2:
>> DEBUG 2: Processing ensemble field: TMP/Z2
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
>> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>> matching VarInfo "TMP/Z2" in GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
>> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>> matching VarInfo "TMP/Z2" in GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
>> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>> matching VarInfo "TMP/Z2" in GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
>> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>> matching VarInfo "TMP/Z2" in GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
>> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>> matching VarInfo "TMP/Z2" in GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
>> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>> matching VarInfo "TMP/Z2" in GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
>> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>> matching VarInfo "TMP/Z2" in GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
>> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>> matching VarInfo "TMP/Z2" in GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
>> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>> matching VarInfo "TMP/Z2" in GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
>> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>> matching VarInfo "TMP/Z2" in GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
>> DEBUG 2:
>> DEBUG 2:
>>
--------------------------------------------------------------------------------
>> DEBUG 2:
>> DEBUG 2: Processing point observation file:
>> /h/data/global/WXQC/data/obs_prep/2013011412
>> DEBUG 2: Searching 219303 observations from 40233 header messages.
>> DEBUG 2:
>> DEBUG 2:
>>
--------------------------------------------------------------------------------
>> DEBUG 2:
>> DEBUG 2: Processing ensemble file:
>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
>> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>> matching VarInfo "TMP/Z2" in GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>> DEBUG 2:
>> DEBUG 2:
>>
--------------------------------------------------------------------------------
>> DEBUG 2:
>> DEBUG 2: Processing ensemble file:
>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
>> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>> matching VarInfo "TMP/Z2" in GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>> DEBUG 2:
>> DEBUG 2:
>>
--------------------------------------------------------------------------------
>> DEBUG 2:
>> DEBUG 2: Processing ensemble file:
>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
>> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>> matching VarInfo "TMP/Z2" in GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>> DEBUG 2:
>> DEBUG 2:
>>
--------------------------------------------------------------------------------
>> DEBUG 2:
>> DEBUG 2: Processing ensemble file:
>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
>> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>> matching VarInfo "TMP/Z2" in GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>> DEBUG 2:
>> DEBUG 2:
>>
--------------------------------------------------------------------------------
>> DEBUG 2:
>> DEBUG 2: Processing ensemble file:
>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
>> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>> matching VarInfo "TMP/Z2" in GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>> DEBUG 2:
>> DEBUG 2:
>>
--------------------------------------------------------------------------------
>> DEBUG 2:
>> DEBUG 2: Processing ensemble file:
>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
>> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>> matching VarInfo "TMP/Z2" in GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>> DEBUG 2:
>> DEBUG 2:
>>
--------------------------------------------------------------------------------
>> DEBUG 2:
>> DEBUG 2: Processing ensemble file:
>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
>> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>> matching VarInfo "TMP/Z2" in GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>> DEBUG 2:
>> DEBUG 2:
>>
--------------------------------------------------------------------------------
>> DEBUG 2:
>> DEBUG 2: Processing ensemble file:
>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
>> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>> matching VarInfo "TMP/Z2" in GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>> DEBUG 2:
>> DEBUG 2:
>>
--------------------------------------------------------------------------------
>> DEBUG 2:
>> DEBUG 2: Processing ensemble file:
>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
>> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>> matching VarInfo "TMP/Z2" in GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>> DEBUG 2:
>> DEBUG 2:
>>
--------------------------------------------------------------------------------
>> DEBUG 2:
>> DEBUG 2: Processing ensemble file:
>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
>> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>> matching VarInfo "TMP/Z2" in GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>> DEBUG 2:
>> DEBUG 2:
>>
--------------------------------------------------------------------------------
>> DEBUG 2:
>> DEBUG 2: Processing point verification TMP/Z2 versus TMP/Z2, for
>> observation type ADPSFC, over region FULL, for interpolation method
>> BILIN(4), using 0 pairs.
>> DEBUG 2:
>> DEBUG 2:
>>
--------------------------------------------------------------------------------
>> DEBUG 2:
>> DEBUG 1: Output file:
>>
/home/qcteam/METv4.0/out/ensemble_stat/ensemble_stat_20130114_120000V.stat
>> DEBUG 1: Output file:
>>
/home/qcteam/METv4.0/out/ensemble_stat/ensemble_stat_20130114_120000V_ens.nc
>>
>> // SIGNED //
>> Matthew C. Sittel
>> Associate Scientist
>> University Corporation for Atmospheric Research
>> 16WS/WXN, Offutt AFB, NE
>> (402) 294-3473 DSN: 271-3473
>>
>>
>
>
>
>
>
------------------------------------------------
Subject: Problem with Pairing NetCDF with GriB
From: matthew.sittel.ctr at offutt.af.mil
Time: Thu Jan 17 08:03:01 2013
Hi John,
I put an updated obs file onto the FTP site with correct time stamps-
overwrote the old one. Talk about confusing. The file I sent you was
later overwritten by a new file so when I looked at it, the date
fields were great. Bob will check into why our process pulls data
from one date and saves it as another.
Weird, ncdump accepts numerical file names for me... were you running
this in a Linux environment? Not blaming Linux, but it's always a
good place to start...
Here's the output from my point_stat run of member 1 vs. the
observation file:
=======
/home/qcteam/METv4.0/bin/point_stat
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011412/GRIB_OUT_d02/WRFPRS24_d02.1.tm00
/h/data/global/WXQC/data/obs_prep/2013011512
/home/qcteam/METv4.0/cfg/point_stat.cfg -outdir
/home/qcteam/METv4.0/out -v 4
DEBUG 1: Default Config File:
/home/qcteam/METv4.0/data/config/PointStatConfig_default
DEBUG 1: User Config File: /home/qcteam/METv4.0/cfg/point_stat.cfg
DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created new
Met2dDataFile object of type "FileType_Gb1".
DEBUG 4: VarInfoFactory::new_var_info() -> created new VarInfo object
of type "FileType_Gb1".
GSL_RNG_TYPE=mt19937
GSL_RNG_SEED=1
DEBUG 1: Forecast File:
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011412/GRIB_OUT_d02/WRFPRS24_d02.1.tm00
DEBUG 1: Climatology File: none
DEBUG 1: Observation File:
/h/data/global/WXQC/data/obs_prep/2013011512
DEBUG 2:
DEBUG 2:
--------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Reading data for TMP/Z2.
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011412/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB records
matching VarInfo "TMP/Z2" in GRIB file
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011412/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
Memory fault
=======
I've never tried plot_point_obs-I don't see reference to it in the 4.0
users guide.
Okay, by typing the executable it told me the arguments I needed.
Just ran it... 33208 obs for ADPSFC type, 3102 obs for gc 11, but none
for the intersect of the two... weird.
We'll keep digging and see what we can find... I wonder if PB2NC is
throwing out all the stuff we need based on the settings we put in the
config file.
Thanks,
Matt
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Wednesday, January 16, 2013 4:27 PM
To: Sittel, Matthew C CTR USAF AFWA 16 WS/WXN
Cc: Craig, Robert J Civ USAF AFWA 16 WS/WXN
Subject: RE: [rt.rap.ucar.edu #59890] Problem with Pairing NetCDF with
GriB
Matt,
I ran METv4.0 point_stat for the first ensemble member using the
attached configuration file and verbosity level 3. I got 0 matched
pairs, but the following diagnostic information is included in the
output:
DEBUG 2: Processing TMP/Z2 versus TMP/Z2, for observation type ADPSFC,
over region FULL, for interpolation method BILIN(4), using 0 pairs.
DEBUG 3: Number of matched pairs = 0
DEBUG 3: Observations processed = 219303
DEBUG 3: Rejected: GRIB code = 190302
DEBUG 3: Rejected: valid time = 29001
DEBUG 3: Rejected: bad obs value = 0
DEBUG 3: Rejected: off the grid = 0
DEBUG 3: Rejected: level mismatch = 0
DEBUG 3: Rejected: message type = 0
DEBUG 3: Rejected: masking region = 0
DEBUG 3: Rejected: bad fcst value = 0
This tells me that there's problem in the matching time window.
Running "wgrib -V" on that forecast file tells me that this is a 24-
hour forecast initialized at 2013011412. So it's valid at 2013011512.
Your observation file is named 2013011512, but when you actually look
at the times it contains, they're for 2013011412:
ncdump -v hdr_vld nc2013011512
(note, ncdump doesn't like file names that begin with a number, so I
stuck an "nc" in front)
I reran point-stat a second time, but first added 3 0's to the "beg"
and "end" times in the "obs_window" setting. When I did that, I got
further, but still no matched pairs:
DEBUG 2: Processing TMP/Z2 versus TMP/Z2, for observation type ADPSFC,
over region FULL, for interpolation method BILIN(4), using 0 pairs.
DEBUG 3: Number of matched pairs = 0
DEBUG 3: Observations processed = 219303
DEBUG 3: Rejected: GRIB code = 190302
DEBUG 3: Rejected: valid time = 0
DEBUG 3: Rejected: bad obs value = 0
DEBUG 3: Rejected: off the grid = 22047
DEBUG 3: Rejected: level mismatch = 6177
DEBUG 3: Rejected: message type = 777
DEBUG 3: Rejected: masking region = 0
DEBUG 3: Rejected: bad fcst value = 0
So there just aren't any ADPSFC observations of 2-m temperature that
fall within your d02 domain. I ran the plot_point_obs utility to
verify that that's the case.
As for why you got a seg-fault, I don't know. It's possible that you
made an error in your Point-Stat config file that caused it. I can't
really debug it unless I can replicate it.
John
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59890 >
>
> Hi John,
>
> The plot thickens. When I ran point_stat on one ensemble member vs.
> an observation file, MET returned a memory fault when it got to the
> observation file read. I tried a second observation file and got
the
> same result.
>
> I looked using 'ncdump' on the NetCDF observation data output from
> 'pb2nc', and confirmed there are ADPSFC records, records within the
> CONUS grid and in the requested time window.
>
> I used "-v 3" verbosity in the example I sent you... I think 4 is
the
> highest level of message? I tried rerunning this morning with "-v
6"
> and didn't get any additional information as to what is happening
with
> the observation matching, nor any messages numbered above 4.
>
> I have pushed the 10 ensemble members, observation file and config
> file to the directory you listed. I had to create a "met_help"
> directory under 'irap' as one was not there. Inside 'sittel' you
will find the files.
> Please note these were created in a Linux environment-not sure if
that
> will be an issue for you; it might depend on your OS. We were
forced
> to go to Linux here.
>
> Thanks,
> Matt
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Tuesday, January 15, 2013 4:10 PM
> To: Sittel, Matthew C CTR USAF AFWA 16 WS/WXN
> Cc: Craig, Robert J Civ USAF AFWA 16 WS/WXN
> Subject: Re: [rt.rap.ucar.edu #59890] Problem with Pairing NetCDF
with
> GriB
>
> Matt,
>
> So the question is basically, why are you getting 0 matched pairs
from
> ensemble-stat. If this were the point-stat tool, the answer would
be
> easy. In point-stat, when you turn up the verbosity level, for each
> verification task, a list of reasons for why observations were not
> used is printed out. And that's often an easy way to diagnose the
problem.
>
> But that functionality has not been implemented for ensemble-stat.
> Though the problem you're having demonstrates the need!
>
> There are several possibilities:
> - the observations may not fall within the matching time window
> - the observations may not fall within the model's grid
> - perhaps there are no ADPSFC observations of TMP in your dataset
>
> There are different steps I'd take to rule out each of these cases -
> mostly making use of the plot_data_plane and plot_point_obs
utilities.
> But perhaps the easiest thing to do is to have you run one of your
> forecast files and the NetCDF output of PB2NC through point-stat to
> verify TMP/Z2.
>
> Presumably, you'll get 0 matched pairs again. But when you increase
> the verbosity level (-v 3, I think), you'll see a list of reasons
why
> the obs weren't used. Send me that list and then we'll know how to
proceed.
>
> Alternatively, you could just send me one or two of your GRIB files,
> the NetCDF point observation file, and your config file, and I could
> debug it here. If you choose to go that route, follow the
instructions here:
> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>
> Your config file settings look fine. The instructions for
specifying
> NetCDF config file stuff only applies to gridded NetCDF data. But
> your gridded data is in GRIB format. For point observations, we
chose
> the convention of specifying them as if they were GRIB files.
That's
> why we include a GRIB code for each observation value in the data.
>
> Just let me know which route you decide on. And I'm happy to help
get
> you up and running.
>
> Thanks,
> John
>
>>
>> Tue Jan 15 09:22:51 2013: Request 59890 was acted upon.
>> Transaction: Ticket created by matthew.sittel.ctr at offutt.af.mil
>> Queue: met_help
>> Subject: Problem with Pairing NetCDF with GriB
>> Owner: Nobody
>> Requestors: matthew.sittel.ctr at offutt.af.mil
>> Status: new
>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59890
>> >
>>
>>
>> Hi,
>>
>> I'm trying to get MET v4.0 to create verification rank histogram
>> output for AFWA's mesoscale ensemble. I'm using in-house, GriB-1
>> format WRF ensemble member output and verifying it with
>> PREPBUFR-formatted obs that we are pulling via FTP from NCEP. We
>> convert the PREPBUFR format to NetCDF using 'pb2nc'.
>>
>> The problem I'm having is I cannot figure out what the correct
>> settings are in the config file for ensemble_stat to get the model
>> and observation fields to match up.
>>
>> I've attached the NetCDF 'dump' of the observation file's contents,
>> the contents of my config file and the output I get when I run
>> ensemble_stat.
>> No matter what settings I pick, it always comes back with 0 pairs
of
>> data. The output suggests the model members are read in and that
>> there are records in the observation file as well-both suggesting
to
>> me the data are read in correctly, but I'm not designating the
>> correct pairing of them with the config file parameters.
>>
>> The records I am trying to get are the ones where gc=11 and hgt=0,
or
>> simply the surface temperature reports from "ADPSFC" message type.
I
>> have read the user's guide and can't figure out what the right
>> settings are for NetCDF; it is discussed briefly but not with
enough
>> detail to figure this out on my own, especially the ANNN, RNNN,
etc,
>> and the TMP(I,...,J,*,*) wording which is unclear. Can you take a
>> look and advise as to the right settings?
>>
>> Thanks,
>> Matt
>>
>> ==
>> netcdf 2013011412 {
>> dimensions:
>> mxstr = 16 ;
>> hdr_arr_len = 3 ;
>> obs_arr_len = 5 ;
>> nobs = UNLIMITED ; // (219303 currently)
>> nhdr = 40233 ;
>> variables:
>> float obs_arr(nobs, obs_arr_len) ;
>> obs_arr:long_name = "array of observation values" ;
>> obs_arr:_fill_value = -9999.f ;
>> obs_arr:columns = "hdr_id gc lvl hgt ob" ;
>> obs_arr:hdr_id_long_name = "index of matching
header
>> data"
>> ;
>> obs_arr:gc_long_name = "grib code corresponding to
>> the observation type" ;
>> obs_arr:lvl_long_name = "pressure level (hPa) or
>> accumulation interval (sec)" ;
>> obs_arr:hgt_long_name = "height in meters above sea
>> level (msl)" ;
>> obs_arr:ob_long_name = "observation value" ;
>> char hdr_typ(nhdr, mxstr) ;
>> hdr_typ:long_name = "message type" ;
>> char hdr_sid(nhdr, mxstr) ;
>> hdr_sid:long_name = "station identification" ;
>> char hdr_vld(nhdr, mxstr) ;
>> hdr_vld:long_name = "valid time" ;
>> hdr_vld:units = "YYYYMMDD_HHMMSS" ;
>> float hdr_arr(nhdr, hdr_arr_len) ;
>> hdr_arr:long_name = "array of observation station
>> header values" ;
>> hdr_arr:_fill_value = -9999.f ;
>> hdr_arr:columns = "lat lon elv" ;
>> hdr_arr:lat_long_name = "latitude" ;
>> hdr_arr:lat_units = "degrees_north" ;
>> hdr_arr:lon_long_name = "longitude" ;
>> hdr_arr:lon_units = "degrees_east" ;
>> hdr_arr:elv_long_name = "elevation" ;
>> hdr_arr:elv_units = "meters above sea level (msl)"
;
>>
>> // global attributes:
>> :FileOrigins = "File
>> /h/data/global/WXQC/data//obs_prep/2013011412 generated
>> 20130115_003309 UTC on host dev8dlsn1 by the MET pb2nc tool" ;
>> :MET_version = "V4.0" ;
>> :MET_tool = "pb2nc" ;
>> data:
>> 40013, 7, 1016.4, 0, 0,
>> 40014, 7, 1012.2, 0, 0,
>> 40015, 7, 1024.4, 0, 0,
>> 40016, 11, 1030.6, 0, 236.55,
>> 40016, 7, 1030.6, 0, 0,
>> 40016, 2, 1030.6, 0, 103060,
>> 40017, 11, 1030.6, 0, 236.55,
>> 40017, 7, 1030.6, 0, 0,
>> 40017, 2, 1030.6, 0, 103060,
>> 40018, 11, 1030.9, 0, 235.95,
>> 40018, 7, 1030.9, 0, 0,
>> 40018, 2, 1030.9, 0, 103090,
>> 40019, 11, 1032.9, 0, 239.85,
>> 40019, 7, 1032.9, 0, 0,
>> 40019, 2, 1032.9, 0, 103290,
>> 40020, 11, 1034.1, 0, 240.65,
>> 40020, 7, 1034.1, 0, 0,
>> 40020, 2, 1034.1, 0, 103410,
>>
>>
/////////////////////////////////////////////////////////////////////
>> ///////////
>> //
>> // Ensemble-Stat configuration file.
>> //
>> // For additional information, see the MET_BASE/data/config/README
file.
>> //
>>
/////////////////////////////////////////////////////////////////////
>> ///////////
>>
>> //
>> // Output model name to be written
>> //
>> model = "WRF";
>>
>>
/////////////////////////////////////////////////////////////////////
>> ///////////
>>
>> //
>> // Ensemble product fields to be processed // ens = {
>> ens_thresh = 1.0;
>> vld_thresh = 1.0;
>>
>> field = [
>> {
>> name = "TMP";
>> level = [ "Z2" ];
>> cat_thresh = [ >=300.0 ];
>> }
>> ];
>> }
>>
>>
/////////////////////////////////////////////////////////////////////
>> ///////////
>>
>> //
>> // Forecast and observation fields to be verified // fcst = {
>> message_type = [ "ADPSFC" ];
>> field = [
>> {
>> name = "TMP";
>> level = "Z2";
>> }
>> ];
>> }
>>
>> obs=fcst;
>>
>>
/////////////////////////////////////////////////////////////////////
>> ///////////
>>
>> //
>> // Point observation time window
>> //
>> obs_window = {
>> beg = -1800;
>> end = 1800;
>> }
>>
>>
/////////////////////////////////////////////////////////////////////
>> ///////////
>>
>> //
>> // Verification masking regions
>> //
>> mask = {
>> grid = [ "FULL" ];
>> poly = [];
>> sid = "";
>> };
>>
>>
/////////////////////////////////////////////////////////////////////
>> ///////////
>>
>> //
>> // Interpolation methods
>> //
>> interp = {
>> field = OBS;
>> vld_thresh = 1.0;
>>
>> type = [
>> {
>> method = BILIN;
>> width = 2;
>> }
>> ];
>> };
>>
>>
/////////////////////////////////////////////////////////////////////
>> ///////////
>>
>> //
>> // Statistical output types
>> //
>> output_flag = {
>> rhist = STAT;
>> orank = NONE;
>> };
>>
>>
/////////////////////////////////////////////////////////////////////
>> ///////////
>>
>> //
>> // Ensemble product output types
>> //
>> ensemble_flag = {
>> mean = TRUE;
>> stdev = TRUE;
>> minus = FALSE;
>> plus = FALSE;
>> min = FALSE;
>> max = FALSE;
>> range = FALSE;
>> vld_count = FALSE;
>> frequency = FALSE;
>> rank = TRUE;
>> };
>>
>>
/////////////////////////////////////////////////////////////////////
>> ///////////
>>
>> //
>> // Random number generator
>> //
>> rng = {
>> type = "mt19937";
>> seed = "1";
>> }
>>
>>
/////////////////////////////////////////////////////////////////////
>> ///////////
>>
>> duplicate_flag = NONE;
>> output_prefix = "";
>> version = "V4.0";
>>
>>
/////////////////////////////////////////////////////////////////////
>> ///////////
>> ==
>> Obs file: /h/data/global/WXQC/data/obs_prep/2013011412
>> DEBUG 1: Default Config File:
>> /home/qcteam/METv4.0/data/config/EnsembleStatConfig_default
>> DEBUG 1: User Config File:
/home/qcteam/METv4.0/cfg/ensemble_stat.cfg
>> DEBUG 1: Ensemble Files[10]:
>> DEBUG 1:
>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>> _d02/WRFPRS24_d02.1.tm00
>> DEBUG 1:
>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>> _d02/WRFPRS24_d02.2.tm00
>> DEBUG 1:
>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>> _d02/WRFPRS24_d02.3.tm00
>> DEBUG 1:
>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>> _d02/WRFPRS24_d02.4.tm00
>> DEBUG 1:
>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>> _d02/WRFPRS24_d02.5.tm00
>> DEBUG 1:
>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>> _d02/WRFPRS24_d02.6.tm00
>> DEBUG 1:
>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>> _d02/WRFPRS24_d02.7.tm00
>> DEBUG 1:
>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>> _d02/WRFPRS24_d02.8.tm00
>> DEBUG 1:
>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>> _d02/WRFPRS24_d02.9.tm00
>> DEBUG 1:
>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>> _d02/WRFPRS24_d02.10.tm00
>> DEBUG 1: Point Observation Files[1]:
>> DEBUG 1: /h/data/global/WXQC/data/obs_prep/2013011412
>> DEBUG 2:
>> DEBUG 2:
>>
---------------------------------------------------------------------
>> -----------
>> DEBUG 2:
>> DEBUG 2: Processing ensemble field: TMP/Z2 DEBUG 3:
>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>> "TMP/Z2" in GRIB record 103 of GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>> matching VarInfo "TMP/Z2" in GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>> matching VarInfo "TMP/Z2" in GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>> matching VarInfo "TMP/Z2" in GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>> matching VarInfo "TMP/Z2" in GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>> matching VarInfo "TMP/Z2" in GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>> matching VarInfo "TMP/Z2" in GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>> matching VarInfo "TMP/Z2" in GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>> matching VarInfo "TMP/Z2" in GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>> matching VarInfo "TMP/Z2" in GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>> matching VarInfo "TMP/Z2" in GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
>> DEBUG 2:
>> DEBUG 2:
>>
---------------------------------------------------------------------
>> -----------
>> DEBUG 2:
>> DEBUG 2: Processing point observation file:
>> /h/data/global/WXQC/data/obs_prep/2013011412
>> DEBUG 2: Searching 219303 observations from 40233 header messages.
>> DEBUG 2:
>> DEBUG 2:
>>
---------------------------------------------------------------------
>> -----------
>> DEBUG 2:
>> DEBUG 2: Processing ensemble file:
>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>> _d02/WRFPRS24_d02.1.tm00 DEBUG 3:
>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>> "TMP/Z2" in GRIB record 103 of GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>> matching VarInfo "TMP/Z2" in GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>> DEBUG 2:
>> DEBUG 2:
>>
---------------------------------------------------------------------
>> -----------
>> DEBUG 2:
>> DEBUG 2: Processing ensemble file:
>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>> _d02/WRFPRS24_d02.2.tm00 DEBUG 3:
>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>> "TMP/Z2" in GRIB record 103 of GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>> matching VarInfo "TMP/Z2" in GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>> DEBUG 2:
>> DEBUG 2:
>>
---------------------------------------------------------------------
>> -----------
>> DEBUG 2:
>> DEBUG 2: Processing ensemble file:
>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>> _d02/WRFPRS24_d02.3.tm00 DEBUG 3:
>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>> "TMP/Z2" in GRIB record 103 of GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>> matching VarInfo "TMP/Z2" in GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>> DEBUG 2:
>> DEBUG 2:
>>
---------------------------------------------------------------------
>> -----------
>> DEBUG 2:
>> DEBUG 2: Processing ensemble file:
>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>> _d02/WRFPRS24_d02.4.tm00 DEBUG 3:
>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>> "TMP/Z2" in GRIB record 103 of GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>> matching VarInfo "TMP/Z2" in GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>> DEBUG 2:
>> DEBUG 2:
>>
---------------------------------------------------------------------
>> -----------
>> DEBUG 2:
>> DEBUG 2: Processing ensemble file:
>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>> _d02/WRFPRS24_d02.5.tm00 DEBUG 3:
>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>> "TMP/Z2" in GRIB record 103 of GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>> matching VarInfo "TMP/Z2" in GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>> DEBUG 2:
>> DEBUG 2:
>>
---------------------------------------------------------------------
>> -----------
>> DEBUG 2:
>> DEBUG 2: Processing ensemble file:
>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>> _d02/WRFPRS24_d02.6.tm00 DEBUG 3:
>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>> "TMP/Z2" in GRIB record 103 of GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>> matching VarInfo "TMP/Z2" in GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>> DEBUG 2:
>> DEBUG 2:
>>
---------------------------------------------------------------------
>> -----------
>> DEBUG 2:
>> DEBUG 2: Processing ensemble file:
>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>> _d02/WRFPRS24_d02.7.tm00 DEBUG 3:
>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>> "TMP/Z2" in GRIB record 103 of GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>> matching VarInfo "TMP/Z2" in GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>> DEBUG 2:
>> DEBUG 2:
>>
---------------------------------------------------------------------
>> -----------
>> DEBUG 2:
>> DEBUG 2: Processing ensemble file:
>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>> _d02/WRFPRS24_d02.8.tm00 DEBUG 3:
>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>> "TMP/Z2" in GRIB record 103 of GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>> matching VarInfo "TMP/Z2" in GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>> DEBUG 2:
>> DEBUG 2:
>>
---------------------------------------------------------------------
>> -----------
>> DEBUG 2:
>> DEBUG 2: Processing ensemble file:
>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>> _d02/WRFPRS24_d02.9.tm00 DEBUG 3:
>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>> "TMP/Z2" in GRIB record 103 of GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>> matching VarInfo "TMP/Z2" in GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>> DEBUG 2:
>> DEBUG 2:
>>
---------------------------------------------------------------------
>> -----------
>> DEBUG 2:
>> DEBUG 2: Processing ensemble file:
>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>> _d02/WRFPRS24_d02.10.tm00 DEBUG 3:
>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>> "TMP/Z2" in GRIB record 103 of GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>> matching VarInfo "TMP/Z2" in GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>> DEBUG 2:
>> DEBUG 2:
>>
---------------------------------------------------------------------
>> -----------
>> DEBUG 2:
>> DEBUG 2: Processing point verification TMP/Z2 versus TMP/Z2, for
>> observation type ADPSFC, over region FULL, for interpolation method
>> BILIN(4), using 0 pairs.
>> DEBUG 2:
>> DEBUG 2:
>>
---------------------------------------------------------------------
>> -----------
>> DEBUG 2:
>> DEBUG 1: Output file:
>>
/home/qcteam/METv4.0/out/ensemble_stat/ensemble_stat_20130114_120000V
>> .stat
>> DEBUG 1: Output file:
>>
/home/qcteam/METv4.0/out/ensemble_stat/ensemble_stat_20130114_120000V
>> _ens.nc
>>
>> // SIGNED //
>> Matthew C. Sittel
>> Associate Scientist
>> University Corporation for Atmospheric Research 16WS/WXN, Offutt
AFB,
>> NE
>> (402) 294-3473 DSN: 271-3473
>>
>>
>
>
>
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #59890] Problem with Pairing NetCDF with GriB
From: John Halley Gotway
Time: Thu Jan 17 15:42:16 2013
Matt,
Yeah, the ncdump thing is silly. The older version in my path
complains
about leading numbers, but I believe that later versions allow it.
Guess
I need to update my path!
I'm not at my desk this afternoon, so am unable to debug until
tomorrow
morning. But I'm still puzzled about the source of that seg fault.
I'd
like to be able to replicate it to address it.
Could you please do 2 things:
- Send me the exact point-stat config file you're using:
/home/qcteam/METv4.0/cfg/point_stat.cfg
- Let me know the date of the latest set of patches you're using for
METv4.0. Just take a look in /home/qcteam/METv4.0 to see if there's a
patch file in there with a date in the filename.
Hopefully, if I run your data with your config file and the exact same
version of the code, I'll be able to reproduce it. If not, it's a lot
more difficult problem.
You could always try applying the latest set of METv4.0 patches and
recompiling to see if that helps:
http://www.dtcenter.org/met/users/support/known_issues/METv4.0/index.php
Regarding why I got 0 matched pairs even when accounting for the time
offset, I think looking at the PB2NC configuration is a great idea.
In
particular, look to see if you're filtering on the observation qc
flag.
If you also send me your PB2NC config file and the location of the
PREPBUFR files you're using, I'd be happy to take a look at that too.
Thanks,
John
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59890 >
>
> Hi John,
>
> I put an updated obs file onto the FTP site with correct time
> stamps-overwrote the old one. Talk about confusing. The file I
sent you
> was later overwritten by a new file so when I looked at it, the date
> fields were great. Bob will check into why our process pulls data
from
> one date and saves it as another.
>
> Weird, ncdump accepts numerical file names for me... were you
running this
> in a Linux environment? Not blaming Linux, but it's always a good
place
> to start...
>
> Here's the output from my point_stat run of member 1 vs. the
observation
> file:
>
> =======
> /home/qcteam/METv4.0/bin/point_stat
>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011412/GRIB_OUT_d02/WRFPRS24_d02.1.tm00
> /h/data/global/WXQC/data/obs_prep/2013011512
> /home/qcteam/METv4.0/cfg/point_stat.cfg -outdir
/home/qcteam/METv4.0/out
> -v 4
>
> DEBUG 1: Default Config File:
> /home/qcteam/METv4.0/data/config/PointStatConfig_default
> DEBUG 1: User Config File: /home/qcteam/METv4.0/cfg/point_stat.cfg
> DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created new
> Met2dDataFile object of type "FileType_Gb1".
> DEBUG 4: VarInfoFactory::new_var_info() -> created new VarInfo
object of
> type "FileType_Gb1".
> GSL_RNG_TYPE=mt19937
> GSL_RNG_SEED=1
> DEBUG 1: Forecast File:
>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011412/GRIB_OUT_d02/WRFPRS24_d02.1.tm00
> DEBUG 1: Climatology File: none
> DEBUG 1: Observation File:
/h/data/global/WXQC/data/obs_prep/2013011512
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Reading data for TMP/Z2.
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011412/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "TMP/Z2" in GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011412/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
> Memory fault
> =======
>
> I've never tried plot_point_obs-I don't see reference to it in the
4.0
> users guide.
>
> Okay, by typing the executable it told me the arguments I needed.
Just
> ran it... 33208 obs for ADPSFC type, 3102 obs for gc 11, but none
for the
> intersect of the two... weird.
>
> We'll keep digging and see what we can find... I wonder if PB2NC is
> throwing out all the stuff we need based on the settings we put in
the
> config file.
>
> Thanks,
> Matt
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Wednesday, January 16, 2013 4:27 PM
> To: Sittel, Matthew C CTR USAF AFWA 16 WS/WXN
> Cc: Craig, Robert J Civ USAF AFWA 16 WS/WXN
> Subject: RE: [rt.rap.ucar.edu #59890] Problem with Pairing NetCDF
with
> GriB
>
> Matt,
>
> I ran METv4.0 point_stat for the first ensemble member using the
attached
> configuration file and verbosity level 3. I got 0 matched pairs,
but the
> following diagnostic information is included in the output:
>
> DEBUG 2: Processing TMP/Z2 versus TMP/Z2, for observation type
ADPSFC,
> over region FULL, for interpolation method BILIN(4), using 0 pairs.
> DEBUG 3: Number of matched pairs = 0
> DEBUG 3: Observations processed = 219303
> DEBUG 3: Rejected: GRIB code = 190302
> DEBUG 3: Rejected: valid time = 29001
> DEBUG 3: Rejected: bad obs value = 0
> DEBUG 3: Rejected: off the grid = 0
> DEBUG 3: Rejected: level mismatch = 0
> DEBUG 3: Rejected: message type = 0
> DEBUG 3: Rejected: masking region = 0
> DEBUG 3: Rejected: bad fcst value = 0
>
> This tells me that there's problem in the matching time window.
>
> Running "wgrib -V" on that forecast file tells me that this is a 24-
hour
> forecast initialized at 2013011412. So it's valid at 2013011512.
>
> Your observation file is named 2013011512, but when you actually
look at
> the times it contains, they're for 2013011412:
> ncdump -v hdr_vld nc2013011512
>
> (note, ncdump doesn't like file names that begin with a number, so I
stuck
> an "nc" in front)
>
> I reran point-stat a second time, but first added 3 0's to the "beg"
and
> "end" times in the "obs_window" setting. When I did that, I got
further,
> but still no matched pairs:
>
> DEBUG 2: Processing TMP/Z2 versus TMP/Z2, for observation type
ADPSFC,
> over region FULL, for interpolation method BILIN(4), using 0 pairs.
> DEBUG 3: Number of matched pairs = 0
> DEBUG 3: Observations processed = 219303
> DEBUG 3: Rejected: GRIB code = 190302
> DEBUG 3: Rejected: valid time = 0
> DEBUG 3: Rejected: bad obs value = 0
> DEBUG 3: Rejected: off the grid = 22047
> DEBUG 3: Rejected: level mismatch = 6177
> DEBUG 3: Rejected: message type = 777
> DEBUG 3: Rejected: masking region = 0
> DEBUG 3: Rejected: bad fcst value = 0
>
> So there just aren't any ADPSFC observations of 2-m temperature that
fall
> within your d02 domain. I ran the plot_point_obs utility to verify
that
> that's the case.
>
> As for why you got a seg-fault, I don't know. It's possible that
you made
> an error in your Point-Stat config file that caused it. I can't
really
> debug it unless I can replicate it.
>
> John
>
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59890 >
>>
>> Hi John,
>>
>> The plot thickens. When I ran point_stat on one ensemble member
vs.
>> an observation file, MET returned a memory fault when it got to the
>> observation file read. I tried a second observation file and got
the
>> same result.
>>
>> I looked using 'ncdump' on the NetCDF observation data output from
>> 'pb2nc', and confirmed there are ADPSFC records, records within the
>> CONUS grid and in the requested time window.
>>
>> I used "-v 3" verbosity in the example I sent you... I think 4 is
the
>> highest level of message? I tried rerunning this morning with "-v
6"
>> and didn't get any additional information as to what is happening
with
>> the observation matching, nor any messages numbered above 4.
>>
>> I have pushed the 10 ensemble members, observation file and config
>> file to the directory you listed. I had to create a "met_help"
>> directory under 'irap' as one was not there. Inside 'sittel' you
will
>> find the files.
>> Please note these were created in a Linux environment-not sure if
that
>> will be an issue for you; it might depend on your OS. We were
forced
>> to go to Linux here.
>>
>> Thanks,
>> Matt
>>
>> -----Original Message-----
>> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
>> Sent: Tuesday, January 15, 2013 4:10 PM
>> To: Sittel, Matthew C CTR USAF AFWA 16 WS/WXN
>> Cc: Craig, Robert J Civ USAF AFWA 16 WS/WXN
>> Subject: Re: [rt.rap.ucar.edu #59890] Problem with Pairing NetCDF
with
>> GriB
>>
>> Matt,
>>
>> So the question is basically, why are you getting 0 matched pairs
from
>> ensemble-stat. If this were the point-stat tool, the answer would
be
>> easy. In point-stat, when you turn up the verbosity level, for
each
>> verification task, a list of reasons for why observations were not
>> used is printed out. And that's often an easy way to diagnose the
>> problem.
>>
>> But that functionality has not been implemented for ensemble-stat.
>> Though the problem you're having demonstrates the need!
>>
>> There are several possibilities:
>> - the observations may not fall within the matching time window
>> - the observations may not fall within the model's grid
>> - perhaps there are no ADPSFC observations of TMP in your dataset
>>
>> There are different steps I'd take to rule out each of these cases
-
>> mostly making use of the plot_data_plane and plot_point_obs
utilities.
>> But perhaps the easiest thing to do is to have you run one of your
>> forecast files and the NetCDF output of PB2NC through point-stat to
>> verify TMP/Z2.
>>
>> Presumably, you'll get 0 matched pairs again. But when you
increase
>> the verbosity level (-v 3, I think), you'll see a list of reasons
why
>> the obs weren't used. Send me that list and then we'll know how to
>> proceed.
>>
>> Alternatively, you could just send me one or two of your GRIB
files,
>> the NetCDF point observation file, and your config file, and I
could
>> debug it here. If you choose to go that route, follow the
instructions
>> here:
>> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>>
>> Your config file settings look fine. The instructions for
specifying
>> NetCDF config file stuff only applies to gridded NetCDF data. But
>> your gridded data is in GRIB format. For point observations, we
chose
>> the convention of specifying them as if they were GRIB files.
That's
>> why we include a GRIB code for each observation value in the data.
>>
>> Just let me know which route you decide on. And I'm happy to help
get
>> you up and running.
>>
>> Thanks,
>> John
>>
>>>
>>> Tue Jan 15 09:22:51 2013: Request 59890 was acted upon.
>>> Transaction: Ticket created by matthew.sittel.ctr at offutt.af.mil
>>> Queue: met_help
>>> Subject: Problem with Pairing NetCDF with GriB
>>> Owner: Nobody
>>> Requestors: matthew.sittel.ctr at offutt.af.mil
>>> Status: new
>>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59890
>>> >
>>>
>>>
>>> Hi,
>>>
>>> I'm trying to get MET v4.0 to create verification rank histogram
>>> output for AFWA's mesoscale ensemble. I'm using in-house, GriB-1
>>> format WRF ensemble member output and verifying it with
>>> PREPBUFR-formatted obs that we are pulling via FTP from NCEP. We
>>> convert the PREPBUFR format to NetCDF using 'pb2nc'.
>>>
>>> The problem I'm having is I cannot figure out what the correct
>>> settings are in the config file for ensemble_stat to get the model
>>> and observation fields to match up.
>>>
>>> I've attached the NetCDF 'dump' of the observation file's
contents,
>>> the contents of my config file and the output I get when I run
>>> ensemble_stat.
>>> No matter what settings I pick, it always comes back with 0 pairs
of
>>> data. The output suggests the model members are read in and that
>>> there are records in the observation file as well-both suggesting
to
>>> me the data are read in correctly, but I'm not designating the
>>> correct pairing of them with the config file parameters.
>>>
>>> The records I am trying to get are the ones where gc=11 and hgt=0,
or
>>> simply the surface temperature reports from "ADPSFC" message type.
I
>>> have read the user's guide and can't figure out what the right
>>> settings are for NetCDF; it is discussed briefly but not with
enough
>>> detail to figure this out on my own, especially the ANNN, RNNN,
etc,
>>> and the TMP(I,...,J,*,*) wording which is unclear. Can you take a
>>> look and advise as to the right settings?
>>>
>>> Thanks,
>>> Matt
>>>
>>> ==
>>> netcdf 2013011412 {
>>> dimensions:
>>> mxstr = 16 ;
>>> hdr_arr_len = 3 ;
>>> obs_arr_len = 5 ;
>>> nobs = UNLIMITED ; // (219303 currently)
>>> nhdr = 40233 ;
>>> variables:
>>> float obs_arr(nobs, obs_arr_len) ;
>>> obs_arr:long_name = "array of observation values"
;
>>> obs_arr:_fill_value = -9999.f ;
>>> obs_arr:columns = "hdr_id gc lvl hgt ob" ;
>>> obs_arr:hdr_id_long_name = "index of matching
header
>>> data"
>>> ;
>>> obs_arr:gc_long_name = "grib code corresponding to
>>> the observation type" ;
>>> obs_arr:lvl_long_name = "pressure level (hPa) or
>>> accumulation interval (sec)" ;
>>> obs_arr:hgt_long_name = "height in meters above
sea
>>> level (msl)" ;
>>> obs_arr:ob_long_name = "observation value" ;
>>> char hdr_typ(nhdr, mxstr) ;
>>> hdr_typ:long_name = "message type" ;
>>> char hdr_sid(nhdr, mxstr) ;
>>> hdr_sid:long_name = "station identification" ;
>>> char hdr_vld(nhdr, mxstr) ;
>>> hdr_vld:long_name = "valid time" ;
>>> hdr_vld:units = "YYYYMMDD_HHMMSS" ;
>>> float hdr_arr(nhdr, hdr_arr_len) ;
>>> hdr_arr:long_name = "array of observation station
>>> header values" ;
>>> hdr_arr:_fill_value = -9999.f ;
>>> hdr_arr:columns = "lat lon elv" ;
>>> hdr_arr:lat_long_name = "latitude" ;
>>> hdr_arr:lat_units = "degrees_north" ;
>>> hdr_arr:lon_long_name = "longitude" ;
>>> hdr_arr:lon_units = "degrees_east" ;
>>> hdr_arr:elv_long_name = "elevation" ;
>>> hdr_arr:elv_units = "meters above sea level (msl)"
;
>>>
>>> // global attributes:
>>> :FileOrigins = "File
>>> /h/data/global/WXQC/data//obs_prep/2013011412 generated
>>> 20130115_003309 UTC on host dev8dlsn1 by the MET pb2nc tool" ;
>>> :MET_version = "V4.0" ;
>>> :MET_tool = "pb2nc" ;
>>> data:
>>> 40013, 7, 1016.4, 0, 0,
>>> 40014, 7, 1012.2, 0, 0,
>>> 40015, 7, 1024.4, 0, 0,
>>> 40016, 11, 1030.6, 0, 236.55,
>>> 40016, 7, 1030.6, 0, 0,
>>> 40016, 2, 1030.6, 0, 103060,
>>> 40017, 11, 1030.6, 0, 236.55,
>>> 40017, 7, 1030.6, 0, 0,
>>> 40017, 2, 1030.6, 0, 103060,
>>> 40018, 11, 1030.9, 0, 235.95,
>>> 40018, 7, 1030.9, 0, 0,
>>> 40018, 2, 1030.9, 0, 103090,
>>> 40019, 11, 1032.9, 0, 239.85,
>>> 40019, 7, 1032.9, 0, 0,
>>> 40019, 2, 1032.9, 0, 103290,
>>> 40020, 11, 1034.1, 0, 240.65,
>>> 40020, 7, 1034.1, 0, 0,
>>> 40020, 2, 1034.1, 0, 103410,
>>>
>>>
/////////////////////////////////////////////////////////////////////
>>> ///////////
>>> //
>>> // Ensemble-Stat configuration file.
>>> //
>>> // For additional information, see the MET_BASE/data/config/README
>>> file.
>>> //
>>>
/////////////////////////////////////////////////////////////////////
>>> ///////////
>>>
>>> //
>>> // Output model name to be written
>>> //
>>> model = "WRF";
>>>
>>>
/////////////////////////////////////////////////////////////////////
>>> ///////////
>>>
>>> //
>>> // Ensemble product fields to be processed // ens = {
>>> ens_thresh = 1.0;
>>> vld_thresh = 1.0;
>>>
>>> field = [
>>> {
>>> name = "TMP";
>>> level = [ "Z2" ];
>>> cat_thresh = [ >=300.0 ];
>>> }
>>> ];
>>> }
>>>
>>>
/////////////////////////////////////////////////////////////////////
>>> ///////////
>>>
>>> //
>>> // Forecast and observation fields to be verified // fcst = {
>>> message_type = [ "ADPSFC" ];
>>> field = [
>>> {
>>> name = "TMP";
>>> level = "Z2";
>>> }
>>> ];
>>> }
>>>
>>> obs=fcst;
>>>
>>>
/////////////////////////////////////////////////////////////////////
>>> ///////////
>>>
>>> //
>>> // Point observation time window
>>> //
>>> obs_window = {
>>> beg = -1800;
>>> end = 1800;
>>> }
>>>
>>>
/////////////////////////////////////////////////////////////////////
>>> ///////////
>>>
>>> //
>>> // Verification masking regions
>>> //
>>> mask = {
>>> grid = [ "FULL" ];
>>> poly = [];
>>> sid = "";
>>> };
>>>
>>>
/////////////////////////////////////////////////////////////////////
>>> ///////////
>>>
>>> //
>>> // Interpolation methods
>>> //
>>> interp = {
>>> field = OBS;
>>> vld_thresh = 1.0;
>>>
>>> type = [
>>> {
>>> method = BILIN;
>>> width = 2;
>>> }
>>> ];
>>> };
>>>
>>>
/////////////////////////////////////////////////////////////////////
>>> ///////////
>>>
>>> //
>>> // Statistical output types
>>> //
>>> output_flag = {
>>> rhist = STAT;
>>> orank = NONE;
>>> };
>>>
>>>
/////////////////////////////////////////////////////////////////////
>>> ///////////
>>>
>>> //
>>> // Ensemble product output types
>>> //
>>> ensemble_flag = {
>>> mean = TRUE;
>>> stdev = TRUE;
>>> minus = FALSE;
>>> plus = FALSE;
>>> min = FALSE;
>>> max = FALSE;
>>> range = FALSE;
>>> vld_count = FALSE;
>>> frequency = FALSE;
>>> rank = TRUE;
>>> };
>>>
>>>
/////////////////////////////////////////////////////////////////////
>>> ///////////
>>>
>>> //
>>> // Random number generator
>>> //
>>> rng = {
>>> type = "mt19937";
>>> seed = "1";
>>> }
>>>
>>>
/////////////////////////////////////////////////////////////////////
>>> ///////////
>>>
>>> duplicate_flag = NONE;
>>> output_prefix = "";
>>> version = "V4.0";
>>>
>>>
/////////////////////////////////////////////////////////////////////
>>> ///////////
>>> ==
>>> Obs file: /h/data/global/WXQC/data/obs_prep/2013011412
>>> DEBUG 1: Default Config File:
>>> /home/qcteam/METv4.0/data/config/EnsembleStatConfig_default
>>> DEBUG 1: User Config File:
/home/qcteam/METv4.0/cfg/ensemble_stat.cfg
>>> DEBUG 1: Ensemble Files[10]:
>>> DEBUG 1:
>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>> _d02/WRFPRS24_d02.1.tm00
>>> DEBUG 1:
>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>> _d02/WRFPRS24_d02.2.tm00
>>> DEBUG 1:
>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>> _d02/WRFPRS24_d02.3.tm00
>>> DEBUG 1:
>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>> _d02/WRFPRS24_d02.4.tm00
>>> DEBUG 1:
>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>> _d02/WRFPRS24_d02.5.tm00
>>> DEBUG 1:
>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>> _d02/WRFPRS24_d02.6.tm00
>>> DEBUG 1:
>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>> _d02/WRFPRS24_d02.7.tm00
>>> DEBUG 1:
>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>> _d02/WRFPRS24_d02.8.tm00
>>> DEBUG 1:
>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>> _d02/WRFPRS24_d02.9.tm00
>>> DEBUG 1:
>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>> _d02/WRFPRS24_d02.10.tm00
>>> DEBUG 1: Point Observation Files[1]:
>>> DEBUG 1: /h/data/global/WXQC/data/obs_prep/2013011412
>>> DEBUG 2:
>>> DEBUG 2:
>>>
---------------------------------------------------------------------
>>> -----------
>>> DEBUG 2:
>>> DEBUG 2: Processing ensemble field: TMP/Z2 DEBUG 3:
>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>> matching VarInfo "TMP/Z2" in GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>> matching VarInfo "TMP/Z2" in GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>> matching VarInfo "TMP/Z2" in GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>> matching VarInfo "TMP/Z2" in GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>> matching VarInfo "TMP/Z2" in GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>> matching VarInfo "TMP/Z2" in GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>> matching VarInfo "TMP/Z2" in GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>> matching VarInfo "TMP/Z2" in GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>> matching VarInfo "TMP/Z2" in GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>> matching VarInfo "TMP/Z2" in GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
>>> DEBUG 2:
>>> DEBUG 2:
>>>
---------------------------------------------------------------------
>>> -----------
>>> DEBUG 2:
>>> DEBUG 2: Processing point observation file:
>>> /h/data/global/WXQC/data/obs_prep/2013011412
>>> DEBUG 2: Searching 219303 observations from 40233 header messages.
>>> DEBUG 2:
>>> DEBUG 2:
>>>
---------------------------------------------------------------------
>>> -----------
>>> DEBUG 2:
>>> DEBUG 2: Processing ensemble file:
>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>> _d02/WRFPRS24_d02.1.tm00 DEBUG 3:
>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>> matching VarInfo "TMP/Z2" in GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>> DEBUG 2:
>>> DEBUG 2:
>>>
---------------------------------------------------------------------
>>> -----------
>>> DEBUG 2:
>>> DEBUG 2: Processing ensemble file:
>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>> _d02/WRFPRS24_d02.2.tm00 DEBUG 3:
>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>> matching VarInfo "TMP/Z2" in GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>> DEBUG 2:
>>> DEBUG 2:
>>>
---------------------------------------------------------------------
>>> -----------
>>> DEBUG 2:
>>> DEBUG 2: Processing ensemble file:
>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>> _d02/WRFPRS24_d02.3.tm00 DEBUG 3:
>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>> matching VarInfo "TMP/Z2" in GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>> DEBUG 2:
>>> DEBUG 2:
>>>
---------------------------------------------------------------------
>>> -----------
>>> DEBUG 2:
>>> DEBUG 2: Processing ensemble file:
>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>> _d02/WRFPRS24_d02.4.tm00 DEBUG 3:
>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>> matching VarInfo "TMP/Z2" in GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>> DEBUG 2:
>>> DEBUG 2:
>>>
---------------------------------------------------------------------
>>> -----------
>>> DEBUG 2:
>>> DEBUG 2: Processing ensemble file:
>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>> _d02/WRFPRS24_d02.5.tm00 DEBUG 3:
>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>> matching VarInfo "TMP/Z2" in GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>> DEBUG 2:
>>> DEBUG 2:
>>>
---------------------------------------------------------------------
>>> -----------
>>> DEBUG 2:
>>> DEBUG 2: Processing ensemble file:
>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>> _d02/WRFPRS24_d02.6.tm00 DEBUG 3:
>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>> matching VarInfo "TMP/Z2" in GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>> DEBUG 2:
>>> DEBUG 2:
>>>
---------------------------------------------------------------------
>>> -----------
>>> DEBUG 2:
>>> DEBUG 2: Processing ensemble file:
>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>> _d02/WRFPRS24_d02.7.tm00 DEBUG 3:
>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>> matching VarInfo "TMP/Z2" in GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>> DEBUG 2:
>>> DEBUG 2:
>>>
---------------------------------------------------------------------
>>> -----------
>>> DEBUG 2:
>>> DEBUG 2: Processing ensemble file:
>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>> _d02/WRFPRS24_d02.8.tm00 DEBUG 3:
>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>> matching VarInfo "TMP/Z2" in GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>> DEBUG 2:
>>> DEBUG 2:
>>>
---------------------------------------------------------------------
>>> -----------
>>> DEBUG 2:
>>> DEBUG 2: Processing ensemble file:
>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>> _d02/WRFPRS24_d02.9.tm00 DEBUG 3:
>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>> matching VarInfo "TMP/Z2" in GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>> DEBUG 2:
>>> DEBUG 2:
>>>
---------------------------------------------------------------------
>>> -----------
>>> DEBUG 2:
>>> DEBUG 2: Processing ensemble file:
>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>> _d02/WRFPRS24_d02.10.tm00 DEBUG 3:
>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>> matching VarInfo "TMP/Z2" in GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>> DEBUG 2:
>>> DEBUG 2:
>>>
---------------------------------------------------------------------
>>> -----------
>>> DEBUG 2:
>>> DEBUG 2: Processing point verification TMP/Z2 versus TMP/Z2, for
>>> observation type ADPSFC, over region FULL, for interpolation
method
>>> BILIN(4), using 0 pairs.
>>> DEBUG 2:
>>> DEBUG 2:
>>>
---------------------------------------------------------------------
>>> -----------
>>> DEBUG 2:
>>> DEBUG 1: Output file:
>>>
/home/qcteam/METv4.0/out/ensemble_stat/ensemble_stat_20130114_120000V
>>> .stat
>>> DEBUG 1: Output file:
>>>
/home/qcteam/METv4.0/out/ensemble_stat/ensemble_stat_20130114_120000V
>>> _ens.nc
>>>
>>> // SIGNED //
>>> Matthew C. Sittel
>>> Associate Scientist
>>> University Corporation for Atmospheric Research 16WS/WXN, Offutt
AFB,
>>> NE
>>> (402) 294-3473 DSN: 271-3473
>>>
>>>
>>
>>
>>
>>
>>
>
>
>
------------------------------------------------
Subject: Problem with Pairing NetCDF with GriB
From: matthew.sittel.ctr at offutt.af.mil
Time: Fri Jan 18 07:53:38 2013
Hi John,
I've added the point_stat config file to the FTP directory with the
other files.
I have not downloaded any of the patches for v4.0. I thought I had
but turns out it was for v3.1 back on May 29 '12.
Bob has been looking at the PB2NC process. The obs plotting
capability has been helpful.
I'll recompile and let you know if it helps.
Thanks,
Matt
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Thursday, January 17, 2013 4:42 PM
To: Sittel, Matthew C CTR USAF AFWA 16 WS/WXN
Cc: Craig, Robert J Civ USAF AFWA 16 WS/WXN
Subject: RE: [rt.rap.ucar.edu #59890] Problem with Pairing NetCDF with
GriB
Matt,
Yeah, the ncdump thing is silly. The older version in my path
complains
about leading numbers, but I believe that later versions allow it.
Guess
I need to update my path!
I'm not at my desk this afternoon, so am unable to debug until
tomorrow
morning. But I'm still puzzled about the source of that seg fault.
I'd
like to be able to replicate it to address it.
Could you please do 2 things:
- Send me the exact point-stat config file you're using:
/home/qcteam/METv4.0/cfg/point_stat.cfg
- Let me know the date of the latest set of patches you're using for
METv4.0. Just take a look in /home/qcteam/METv4.0 to see if there's a
patch file in there with a date in the filename.
Hopefully, if I run your data with your config file and the exact same
version of the code, I'll be able to reproduce it. If not, it's a lot
more difficult problem.
You could always try applying the latest set of METv4.0 patches and
recompiling to see if that helps:
http://www.dtcenter.org/met/users/support/known_issues/METv4.0/index.php
Regarding why I got 0 matched pairs even when accounting for the time
offset, I think looking at the PB2NC configuration is a great idea.
In
particular, look to see if you're filtering on the observation qc
flag.
If you also send me your PB2NC config file and the location of the
PREPBUFR files you're using, I'd be happy to take a look at that too.
Thanks,
John
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59890 >
>
> Hi John,
>
> I put an updated obs file onto the FTP site with correct time
> stamps-overwrote the old one. Talk about confusing. The file I
sent you
> was later overwritten by a new file so when I looked at it, the date
> fields were great. Bob will check into why our process pulls data
from
> one date and saves it as another.
>
> Weird, ncdump accepts numerical file names for me... were you
running this
> in a Linux environment? Not blaming Linux, but it's always a good
place
> to start...
>
> Here's the output from my point_stat run of member 1 vs. the
observation
> file:
>
> =======
> /home/qcteam/METv4.0/bin/point_stat
>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011412/GRIB_OUT_d02/WRFPRS24_d02.1.tm00
> /h/data/global/WXQC/data/obs_prep/2013011512
> /home/qcteam/METv4.0/cfg/point_stat.cfg -outdir
/home/qcteam/METv4.0/out
> -v 4
>
> DEBUG 1: Default Config File:
> /home/qcteam/METv4.0/data/config/PointStatConfig_default
> DEBUG 1: User Config File: /home/qcteam/METv4.0/cfg/point_stat.cfg
> DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created new
> Met2dDataFile object of type "FileType_Gb1".
> DEBUG 4: VarInfoFactory::new_var_info() -> created new VarInfo
object of
> type "FileType_Gb1".
> GSL_RNG_TYPE=mt19937
> GSL_RNG_SEED=1
> DEBUG 1: Forecast File:
>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011412/GRIB_OUT_d02/WRFPRS24_d02.1.tm00
> DEBUG 1: Climatology File: none
> DEBUG 1: Observation File:
/h/data/global/WXQC/data/obs_prep/2013011512
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Reading data for TMP/Z2.
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011412/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "TMP/Z2" in GRIB file
>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011412/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
> Memory fault
> =======
>
> I've never tried plot_point_obs-I don't see reference to it in the
4.0
> users guide.
>
> Okay, by typing the executable it told me the arguments I needed.
Just
> ran it... 33208 obs for ADPSFC type, 3102 obs for gc 11, but none
for the
> intersect of the two... weird.
>
> We'll keep digging and see what we can find... I wonder if PB2NC is
> throwing out all the stuff we need based on the settings we put in
the
> config file.
>
> Thanks,
> Matt
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Wednesday, January 16, 2013 4:27 PM
> To: Sittel, Matthew C CTR USAF AFWA 16 WS/WXN
> Cc: Craig, Robert J Civ USAF AFWA 16 WS/WXN
> Subject: RE: [rt.rap.ucar.edu #59890] Problem with Pairing NetCDF
with
> GriB
>
> Matt,
>
> I ran METv4.0 point_stat for the first ensemble member using the
attached
> configuration file and verbosity level 3. I got 0 matched pairs,
but the
> following diagnostic information is included in the output:
>
> DEBUG 2: Processing TMP/Z2 versus TMP/Z2, for observation type
ADPSFC,
> over region FULL, for interpolation method BILIN(4), using 0 pairs.
> DEBUG 3: Number of matched pairs = 0
> DEBUG 3: Observations processed = 219303
> DEBUG 3: Rejected: GRIB code = 190302
> DEBUG 3: Rejected: valid time = 29001
> DEBUG 3: Rejected: bad obs value = 0
> DEBUG 3: Rejected: off the grid = 0
> DEBUG 3: Rejected: level mismatch = 0
> DEBUG 3: Rejected: message type = 0
> DEBUG 3: Rejected: masking region = 0
> DEBUG 3: Rejected: bad fcst value = 0
>
> This tells me that there's problem in the matching time window.
>
> Running "wgrib -V" on that forecast file tells me that this is a 24-
hour
> forecast initialized at 2013011412. So it's valid at 2013011512.
>
> Your observation file is named 2013011512, but when you actually
look at
> the times it contains, they're for 2013011412:
> ncdump -v hdr_vld nc2013011512
>
> (note, ncdump doesn't like file names that begin with a number, so I
stuck
> an "nc" in front)
>
> I reran point-stat a second time, but first added 3 0's to the "beg"
and
> "end" times in the "obs_window" setting. When I did that, I got
further,
> but still no matched pairs:
>
> DEBUG 2: Processing TMP/Z2 versus TMP/Z2, for observation type
ADPSFC,
> over region FULL, for interpolation method BILIN(4), using 0 pairs.
> DEBUG 3: Number of matched pairs = 0
> DEBUG 3: Observations processed = 219303
> DEBUG 3: Rejected: GRIB code = 190302
> DEBUG 3: Rejected: valid time = 0
> DEBUG 3: Rejected: bad obs value = 0
> DEBUG 3: Rejected: off the grid = 22047
> DEBUG 3: Rejected: level mismatch = 6177
> DEBUG 3: Rejected: message type = 777
> DEBUG 3: Rejected: masking region = 0
> DEBUG 3: Rejected: bad fcst value = 0
>
> So there just aren't any ADPSFC observations of 2-m temperature that
fall
> within your d02 domain. I ran the plot_point_obs utility to verify
that
> that's the case.
>
> As for why you got a seg-fault, I don't know. It's possible that
you made
> an error in your Point-Stat config file that caused it. I can't
really
> debug it unless I can replicate it.
>
> John
>
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59890 >
>>
>> Hi John,
>>
>> The plot thickens. When I ran point_stat on one ensemble member
vs.
>> an observation file, MET returned a memory fault when it got to the
>> observation file read. I tried a second observation file and got
the
>> same result.
>>
>> I looked using 'ncdump' on the NetCDF observation data output from
>> 'pb2nc', and confirmed there are ADPSFC records, records within the
>> CONUS grid and in the requested time window.
>>
>> I used "-v 3" verbosity in the example I sent you... I think 4 is
the
>> highest level of message? I tried rerunning this morning with "-v
6"
>> and didn't get any additional information as to what is happening
with
>> the observation matching, nor any messages numbered above 4.
>>
>> I have pushed the 10 ensemble members, observation file and config
>> file to the directory you listed. I had to create a "met_help"
>> directory under 'irap' as one was not there. Inside 'sittel' you
will
>> find the files.
>> Please note these were created in a Linux environment-not sure if
that
>> will be an issue for you; it might depend on your OS. We were
forced
>> to go to Linux here.
>>
>> Thanks,
>> Matt
>>
>> -----Original Message-----
>> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
>> Sent: Tuesday, January 15, 2013 4:10 PM
>> To: Sittel, Matthew C CTR USAF AFWA 16 WS/WXN
>> Cc: Craig, Robert J Civ USAF AFWA 16 WS/WXN
>> Subject: Re: [rt.rap.ucar.edu #59890] Problem with Pairing NetCDF
with
>> GriB
>>
>> Matt,
>>
>> So the question is basically, why are you getting 0 matched pairs
from
>> ensemble-stat. If this were the point-stat tool, the answer would
be
>> easy. In point-stat, when you turn up the verbosity level, for
each
>> verification task, a list of reasons for why observations were not
>> used is printed out. And that's often an easy way to diagnose the
>> problem.
>>
>> But that functionality has not been implemented for ensemble-stat.
>> Though the problem you're having demonstrates the need!
>>
>> There are several possibilities:
>> - the observations may not fall within the matching time window
>> - the observations may not fall within the model's grid
>> - perhaps there are no ADPSFC observations of TMP in your dataset
>>
>> There are different steps I'd take to rule out each of these cases
-
>> mostly making use of the plot_data_plane and plot_point_obs
utilities.
>> But perhaps the easiest thing to do is to have you run one of your
>> forecast files and the NetCDF output of PB2NC through point-stat to
>> verify TMP/Z2.
>>
>> Presumably, you'll get 0 matched pairs again. But when you
increase
>> the verbosity level (-v 3, I think), you'll see a list of reasons
why
>> the obs weren't used. Send me that list and then we'll know how to
>> proceed.
>>
>> Alternatively, you could just send me one or two of your GRIB
files,
>> the NetCDF point observation file, and your config file, and I
could
>> debug it here. If you choose to go that route, follow the
instructions
>> here:
>> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>>
>> Your config file settings look fine. The instructions for
specifying
>> NetCDF config file stuff only applies to gridded NetCDF data. But
>> your gridded data is in GRIB format. For point observations, we
chose
>> the convention of specifying them as if they were GRIB files.
That's
>> why we include a GRIB code for each observation value in the data.
>>
>> Just let me know which route you decide on. And I'm happy to help
get
>> you up and running.
>>
>> Thanks,
>> John
>>
>>>
>>> Tue Jan 15 09:22:51 2013: Request 59890 was acted upon.
>>> Transaction: Ticket created by matthew.sittel.ctr at offutt.af.mil
>>> Queue: met_help
>>> Subject: Problem with Pairing NetCDF with GriB
>>> Owner: Nobody
>>> Requestors: matthew.sittel.ctr at offutt.af.mil
>>> Status: new
>>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59890
>>> >
>>>
>>>
>>> Hi,
>>>
>>> I'm trying to get MET v4.0 to create verification rank histogram
>>> output for AFWA's mesoscale ensemble. I'm using in-house, GriB-1
>>> format WRF ensemble member output and verifying it with
>>> PREPBUFR-formatted obs that we are pulling via FTP from NCEP. We
>>> convert the PREPBUFR format to NetCDF using 'pb2nc'.
>>>
>>> The problem I'm having is I cannot figure out what the correct
>>> settings are in the config file for ensemble_stat to get the model
>>> and observation fields to match up.
>>>
>>> I've attached the NetCDF 'dump' of the observation file's
contents,
>>> the contents of my config file and the output I get when I run
>>> ensemble_stat.
>>> No matter what settings I pick, it always comes back with 0 pairs
of
>>> data. The output suggests the model members are read in and that
>>> there are records in the observation file as well-both suggesting
to
>>> me the data are read in correctly, but I'm not designating the
>>> correct pairing of them with the config file parameters.
>>>
>>> The records I am trying to get are the ones where gc=11 and hgt=0,
or
>>> simply the surface temperature reports from "ADPSFC" message type.
I
>>> have read the user's guide and can't figure out what the right
>>> settings are for NetCDF; it is discussed briefly but not with
enough
>>> detail to figure this out on my own, especially the ANNN, RNNN,
etc,
>>> and the TMP(I,...,J,*,*) wording which is unclear. Can you take a
>>> look and advise as to the right settings?
>>>
>>> Thanks,
>>> Matt
>>>
>>> ==
>>> netcdf 2013011412 {
>>> dimensions:
>>> mxstr = 16 ;
>>> hdr_arr_len = 3 ;
>>> obs_arr_len = 5 ;
>>> nobs = UNLIMITED ; // (219303 currently)
>>> nhdr = 40233 ;
>>> variables:
>>> float obs_arr(nobs, obs_arr_len) ;
>>> obs_arr:long_name = "array of observation values"
;
>>> obs_arr:_fill_value = -9999.f ;
>>> obs_arr:columns = "hdr_id gc lvl hgt ob" ;
>>> obs_arr:hdr_id_long_name = "index of matching
header
>>> data"
>>> ;
>>> obs_arr:gc_long_name = "grib code corresponding to
>>> the observation type" ;
>>> obs_arr:lvl_long_name = "pressure level (hPa) or
>>> accumulation interval (sec)" ;
>>> obs_arr:hgt_long_name = "height in meters above
sea
>>> level (msl)" ;
>>> obs_arr:ob_long_name = "observation value" ;
>>> char hdr_typ(nhdr, mxstr) ;
>>> hdr_typ:long_name = "message type" ;
>>> char hdr_sid(nhdr, mxstr) ;
>>> hdr_sid:long_name = "station identification" ;
>>> char hdr_vld(nhdr, mxstr) ;
>>> hdr_vld:long_name = "valid time" ;
>>> hdr_vld:units = "YYYYMMDD_HHMMSS" ;
>>> float hdr_arr(nhdr, hdr_arr_len) ;
>>> hdr_arr:long_name = "array of observation station
>>> header values" ;
>>> hdr_arr:_fill_value = -9999.f ;
>>> hdr_arr:columns = "lat lon elv" ;
>>> hdr_arr:lat_long_name = "latitude" ;
>>> hdr_arr:lat_units = "degrees_north" ;
>>> hdr_arr:lon_long_name = "longitude" ;
>>> hdr_arr:lon_units = "degrees_east" ;
>>> hdr_arr:elv_long_name = "elevation" ;
>>> hdr_arr:elv_units = "meters above sea level (msl)"
;
>>>
>>> // global attributes:
>>> :FileOrigins = "File
>>> /h/data/global/WXQC/data//obs_prep/2013011412 generated
>>> 20130115_003309 UTC on host dev8dlsn1 by the MET pb2nc tool" ;
>>> :MET_version = "V4.0" ;
>>> :MET_tool = "pb2nc" ;
>>> data:
>>> 40013, 7, 1016.4, 0, 0,
>>> 40014, 7, 1012.2, 0, 0,
>>> 40015, 7, 1024.4, 0, 0,
>>> 40016, 11, 1030.6, 0, 236.55,
>>> 40016, 7, 1030.6, 0, 0,
>>> 40016, 2, 1030.6, 0, 103060,
>>> 40017, 11, 1030.6, 0, 236.55,
>>> 40017, 7, 1030.6, 0, 0,
>>> 40017, 2, 1030.6, 0, 103060,
>>> 40018, 11, 1030.9, 0, 235.95,
>>> 40018, 7, 1030.9, 0, 0,
>>> 40018, 2, 1030.9, 0, 103090,
>>> 40019, 11, 1032.9, 0, 239.85,
>>> 40019, 7, 1032.9, 0, 0,
>>> 40019, 2, 1032.9, 0, 103290,
>>> 40020, 11, 1034.1, 0, 240.65,
>>> 40020, 7, 1034.1, 0, 0,
>>> 40020, 2, 1034.1, 0, 103410,
>>>
>>>
/////////////////////////////////////////////////////////////////////
>>> ///////////
>>> //
>>> // Ensemble-Stat configuration file.
>>> //
>>> // For additional information, see the MET_BASE/data/config/README
>>> file.
>>> //
>>>
/////////////////////////////////////////////////////////////////////
>>> ///////////
>>>
>>> //
>>> // Output model name to be written
>>> //
>>> model = "WRF";
>>>
>>>
/////////////////////////////////////////////////////////////////////
>>> ///////////
>>>
>>> //
>>> // Ensemble product fields to be processed // ens = {
>>> ens_thresh = 1.0;
>>> vld_thresh = 1.0;
>>>
>>> field = [
>>> {
>>> name = "TMP";
>>> level = [ "Z2" ];
>>> cat_thresh = [ >=300.0 ];
>>> }
>>> ];
>>> }
>>>
>>>
/////////////////////////////////////////////////////////////////////
>>> ///////////
>>>
>>> //
>>> // Forecast and observation fields to be verified // fcst = {
>>> message_type = [ "ADPSFC" ];
>>> field = [
>>> {
>>> name = "TMP";
>>> level = "Z2";
>>> }
>>> ];
>>> }
>>>
>>> obs=fcst;
>>>
>>>
/////////////////////////////////////////////////////////////////////
>>> ///////////
>>>
>>> //
>>> // Point observation time window
>>> //
>>> obs_window = {
>>> beg = -1800;
>>> end = 1800;
>>> }
>>>
>>>
/////////////////////////////////////////////////////////////////////
>>> ///////////
>>>
>>> //
>>> // Verification masking regions
>>> //
>>> mask = {
>>> grid = [ "FULL" ];
>>> poly = [];
>>> sid = "";
>>> };
>>>
>>>
/////////////////////////////////////////////////////////////////////
>>> ///////////
>>>
>>> //
>>> // Interpolation methods
>>> //
>>> interp = {
>>> field = OBS;
>>> vld_thresh = 1.0;
>>>
>>> type = [
>>> {
>>> method = BILIN;
>>> width = 2;
>>> }
>>> ];
>>> };
>>>
>>>
/////////////////////////////////////////////////////////////////////
>>> ///////////
>>>
>>> //
>>> // Statistical output types
>>> //
>>> output_flag = {
>>> rhist = STAT;
>>> orank = NONE;
>>> };
>>>
>>>
/////////////////////////////////////////////////////////////////////
>>> ///////////
>>>
>>> //
>>> // Ensemble product output types
>>> //
>>> ensemble_flag = {
>>> mean = TRUE;
>>> stdev = TRUE;
>>> minus = FALSE;
>>> plus = FALSE;
>>> min = FALSE;
>>> max = FALSE;
>>> range = FALSE;
>>> vld_count = FALSE;
>>> frequency = FALSE;
>>> rank = TRUE;
>>> };
>>>
>>>
/////////////////////////////////////////////////////////////////////
>>> ///////////
>>>
>>> //
>>> // Random number generator
>>> //
>>> rng = {
>>> type = "mt19937";
>>> seed = "1";
>>> }
>>>
>>>
/////////////////////////////////////////////////////////////////////
>>> ///////////
>>>
>>> duplicate_flag = NONE;
>>> output_prefix = "";
>>> version = "V4.0";
>>>
>>>
/////////////////////////////////////////////////////////////////////
>>> ///////////
>>> ==
>>> Obs file: /h/data/global/WXQC/data/obs_prep/2013011412
>>> DEBUG 1: Default Config File:
>>> /home/qcteam/METv4.0/data/config/EnsembleStatConfig_default
>>> DEBUG 1: User Config File:
/home/qcteam/METv4.0/cfg/ensemble_stat.cfg
>>> DEBUG 1: Ensemble Files[10]:
>>> DEBUG 1:
>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>> _d02/WRFPRS24_d02.1.tm00
>>> DEBUG 1:
>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>> _d02/WRFPRS24_d02.2.tm00
>>> DEBUG 1:
>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>> _d02/WRFPRS24_d02.3.tm00
>>> DEBUG 1:
>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>> _d02/WRFPRS24_d02.4.tm00
>>> DEBUG 1:
>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>> _d02/WRFPRS24_d02.5.tm00
>>> DEBUG 1:
>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>> _d02/WRFPRS24_d02.6.tm00
>>> DEBUG 1:
>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>> _d02/WRFPRS24_d02.7.tm00
>>> DEBUG 1:
>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>> _d02/WRFPRS24_d02.8.tm00
>>> DEBUG 1:
>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>> _d02/WRFPRS24_d02.9.tm00
>>> DEBUG 1:
>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>> _d02/WRFPRS24_d02.10.tm00
>>> DEBUG 1: Point Observation Files[1]:
>>> DEBUG 1: /h/data/global/WXQC/data/obs_prep/2013011412
>>> DEBUG 2:
>>> DEBUG 2:
>>>
---------------------------------------------------------------------
>>> -----------
>>> DEBUG 2:
>>> DEBUG 2: Processing ensemble field: TMP/Z2 DEBUG 3:
>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>> matching VarInfo "TMP/Z2" in GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>> matching VarInfo "TMP/Z2" in GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>> matching VarInfo "TMP/Z2" in GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>> matching VarInfo "TMP/Z2" in GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>> matching VarInfo "TMP/Z2" in GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>> matching VarInfo "TMP/Z2" in GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>> matching VarInfo "TMP/Z2" in GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>> matching VarInfo "TMP/Z2" in GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>> matching VarInfo "TMP/Z2" in GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>> matching VarInfo "TMP/Z2" in GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
>>> DEBUG 2:
>>> DEBUG 2:
>>>
---------------------------------------------------------------------
>>> -----------
>>> DEBUG 2:
>>> DEBUG 2: Processing point observation file:
>>> /h/data/global/WXQC/data/obs_prep/2013011412
>>> DEBUG 2: Searching 219303 observations from 40233 header messages.
>>> DEBUG 2:
>>> DEBUG 2:
>>>
---------------------------------------------------------------------
>>> -----------
>>> DEBUG 2:
>>> DEBUG 2: Processing ensemble file:
>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>> _d02/WRFPRS24_d02.1.tm00 DEBUG 3:
>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>> matching VarInfo "TMP/Z2" in GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>> DEBUG 2:
>>> DEBUG 2:
>>>
---------------------------------------------------------------------
>>> -----------
>>> DEBUG 2:
>>> DEBUG 2: Processing ensemble file:
>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>> _d02/WRFPRS24_d02.2.tm00 DEBUG 3:
>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>> matching VarInfo "TMP/Z2" in GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>> DEBUG 2:
>>> DEBUG 2:
>>>
---------------------------------------------------------------------
>>> -----------
>>> DEBUG 2:
>>> DEBUG 2: Processing ensemble file:
>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>> _d02/WRFPRS24_d02.3.tm00 DEBUG 3:
>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>> matching VarInfo "TMP/Z2" in GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>> DEBUG 2:
>>> DEBUG 2:
>>>
---------------------------------------------------------------------
>>> -----------
>>> DEBUG 2:
>>> DEBUG 2: Processing ensemble file:
>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>> _d02/WRFPRS24_d02.4.tm00 DEBUG 3:
>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>> matching VarInfo "TMP/Z2" in GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>> DEBUG 2:
>>> DEBUG 2:
>>>
---------------------------------------------------------------------
>>> -----------
>>> DEBUG 2:
>>> DEBUG 2: Processing ensemble file:
>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>> _d02/WRFPRS24_d02.5.tm00 DEBUG 3:
>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>> matching VarInfo "TMP/Z2" in GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>> DEBUG 2:
>>> DEBUG 2:
>>>
---------------------------------------------------------------------
>>> -----------
>>> DEBUG 2:
>>> DEBUG 2: Processing ensemble file:
>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>> _d02/WRFPRS24_d02.6.tm00 DEBUG 3:
>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>> matching VarInfo "TMP/Z2" in GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>> DEBUG 2:
>>> DEBUG 2:
>>>
---------------------------------------------------------------------
>>> -----------
>>> DEBUG 2:
>>> DEBUG 2: Processing ensemble file:
>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>> _d02/WRFPRS24_d02.7.tm00 DEBUG 3:
>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>> matching VarInfo "TMP/Z2" in GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>> DEBUG 2:
>>> DEBUG 2:
>>>
---------------------------------------------------------------------
>>> -----------
>>> DEBUG 2:
>>> DEBUG 2: Processing ensemble file:
>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>> _d02/WRFPRS24_d02.8.tm00 DEBUG 3:
>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>> matching VarInfo "TMP/Z2" in GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>> DEBUG 2:
>>> DEBUG 2:
>>>
---------------------------------------------------------------------
>>> -----------
>>> DEBUG 2:
>>> DEBUG 2: Processing ensemble file:
>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>> _d02/WRFPRS24_d02.9.tm00 DEBUG 3:
>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>> matching VarInfo "TMP/Z2" in GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>> DEBUG 2:
>>> DEBUG 2:
>>>
---------------------------------------------------------------------
>>> -----------
>>> DEBUG 2:
>>> DEBUG 2: Processing ensemble file:
>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>> _d02/WRFPRS24_d02.10.tm00 DEBUG 3:
>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>> matching VarInfo "TMP/Z2" in GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>> DEBUG 2:
>>> DEBUG 2:
>>>
---------------------------------------------------------------------
>>> -----------
>>> DEBUG 2:
>>> DEBUG 2: Processing point verification TMP/Z2 versus TMP/Z2, for
>>> observation type ADPSFC, over region FULL, for interpolation
method
>>> BILIN(4), using 0 pairs.
>>> DEBUG 2:
>>> DEBUG 2:
>>>
---------------------------------------------------------------------
>>> -----------
>>> DEBUG 2:
>>> DEBUG 1: Output file:
>>>
/home/qcteam/METv4.0/out/ensemble_stat/ensemble_stat_20130114_120000V
>>> .stat
>>> DEBUG 1: Output file:
>>>
/home/qcteam/METv4.0/out/ensemble_stat/ensemble_stat_20130114_120000V
>>> _ens.nc
>>>
>>> // SIGNED //
>>> Matthew C. Sittel
>>> Associate Scientist
>>> University Corporation for Atmospheric Research 16WS/WXN, Offutt
AFB,
>>> NE
>>> (402) 294-3473 DSN: 271-3473
>>>
>>>
>>
>>
>>
>>
>>
>
>
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #59890] Problem with Pairing NetCDF with GriB
From: John Halley Gotway
Time: Fri Jan 18 08:19:33 2013
Matt,
Good news. The problem was in your point_stat.cfg file. Diffing it
against the default one in
METv4.0/data/config/PointStatConfig_default, I see the following:
< poly = "";
---
> poly = [];
In the masking section, it expects "poly" to be specified as an array.
Instead of specifying it as an empty array, you specified it as a
string of length 0. So it tried to read a file of that name.
Running METv4.0 with the latest set of patches at verbosity level 4,
gave the following helpful information:
DEBUG 2: Reading data for TMP/Z2.
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
"WRFPRS24_d02.1.tm00".
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB records
matching VarInfo "TMP/Z2" in GRIB file "WRFPRS24_d02.1.tm00".
DEBUG 4: parse_sid_mask() -> parsing station ID mask file "(nul)"
DEBUG 4: parse_grid_mask() -> parsing grid mask "FULL"
DEBUG 4: parse_poly_mask() -> parsing poly mask "(nul)"
ERROR :
ERROR : grd_file_type() -> file does not exist "(nul)"
ERROR :
However, running the same thing for METv4.0 as originally released
results in a seg fault. Yeah, we found it!
Looking at the list of fixes, I'm guessing it's the bug fix posted on
08/01/2012:
"Fixed a bug that caused a seg fault when a config file string
list contains only an empty string"
http://www.dtcenter.org/met/users/support/known_issues/METv4.0/index.php
So just edit your Point-Stat config file to use:
poly = [];
And then recompile METv4.0 after applying the latest set of patches.
Hopefully that'll do the trick. Just let us know if you have more
problems or questions.
Thanks,
John
On 01/18/2013 07:53 AM, matthew.sittel.ctr at offutt.af.mil via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59890 >
>
> Hi John,
>
> I've added the point_stat config file to the FTP directory with the
other files.
>
> I have not downloaded any of the patches for v4.0. I thought I had
but turns out it was for v3.1 back on May 29 '12.
>
> Bob has been looking at the PB2NC process. The obs plotting
capability has been helpful.
>
> I'll recompile and let you know if it helps.
>
> Thanks,
> Matt
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Thursday, January 17, 2013 4:42 PM
> To: Sittel, Matthew C CTR USAF AFWA 16 WS/WXN
> Cc: Craig, Robert J Civ USAF AFWA 16 WS/WXN
> Subject: RE: [rt.rap.ucar.edu #59890] Problem with Pairing NetCDF
with GriB
>
> Matt,
>
> Yeah, the ncdump thing is silly. The older version in my path
complains
> about leading numbers, but I believe that later versions allow it.
Guess
> I need to update my path!
>
> I'm not at my desk this afternoon, so am unable to debug until
tomorrow
> morning. But I'm still puzzled about the source of that seg fault.
I'd
> like to be able to replicate it to address it.
>
> Could you please do 2 things:
> - Send me the exact point-stat config file you're using:
> /home/qcteam/METv4.0/cfg/point_stat.cfg
> - Let me know the date of the latest set of patches you're using for
> METv4.0. Just take a look in /home/qcteam/METv4.0 to see if there's
a
> patch file in there with a date in the filename.
>
> Hopefully, if I run your data with your config file and the exact
same
> version of the code, I'll be able to reproduce it. If not, it's a
lot
> more difficult problem.
>
> You could always try applying the latest set of METv4.0 patches and
> recompiling to see if that helps:
>
http://www.dtcenter.org/met/users/support/known_issues/METv4.0/index.php
>
> Regarding why I got 0 matched pairs even when accounting for the
time
> offset, I think looking at the PB2NC configuration is a great idea.
In
> particular, look to see if you're filtering on the observation qc
flag.
> If you also send me your PB2NC config file and the location of the
> PREPBUFR files you're using, I'd be happy to take a look at that
too.
>
> Thanks,
> John
>
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59890 >
>>
>> Hi John,
>>
>> I put an updated obs file onto the FTP site with correct time
>> stamps-overwrote the old one. Talk about confusing. The file I
sent you
>> was later overwritten by a new file so when I looked at it, the
date
>> fields were great. Bob will check into why our process pulls data
from
>> one date and saves it as another.
>>
>> Weird, ncdump accepts numerical file names for me... were you
running this
>> in a Linux environment? Not blaming Linux, but it's always a good
place
>> to start...
>>
>> Here's the output from my point_stat run of member 1 vs. the
observation
>> file:
>>
>> =======
>> /home/qcteam/METv4.0/bin/point_stat
>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011412/GRIB_OUT_d02/WRFPRS24_d02.1.tm00
>> /h/data/global/WXQC/data/obs_prep/2013011512
>> /home/qcteam/METv4.0/cfg/point_stat.cfg -outdir
/home/qcteam/METv4.0/out
>> -v 4
>>
>> DEBUG 1: Default Config File:
>> /home/qcteam/METv4.0/data/config/PointStatConfig_default
>> DEBUG 1: User Config File: /home/qcteam/METv4.0/cfg/point_stat.cfg
>> DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created
new
>> Met2dDataFile object of type "FileType_Gb1".
>> DEBUG 4: VarInfoFactory::new_var_info() -> created new VarInfo
object of
>> type "FileType_Gb1".
>> GSL_RNG_TYPE=mt19937
>> GSL_RNG_SEED=1
>> DEBUG 1: Forecast File:
>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011412/GRIB_OUT_d02/WRFPRS24_d02.1.tm00
>> DEBUG 1: Climatology File: none
>> DEBUG 1: Observation File:
/h/data/global/WXQC/data/obs_prep/2013011512
>> DEBUG 2:
>> DEBUG 2:
>>
--------------------------------------------------------------------------------
>> DEBUG 2:
>> DEBUG 2: Reading data for TMP/Z2.
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
>> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011412/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>> matching VarInfo "TMP/Z2" in GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011412/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>> Memory fault
>> =======
>>
>> I've never tried plot_point_obs-I don't see reference to it in the
4.0
>> users guide.
>>
>> Okay, by typing the executable it told me the arguments I needed.
Just
>> ran it... 33208 obs for ADPSFC type, 3102 obs for gc 11, but none
for the
>> intersect of the two... weird.
>>
>> We'll keep digging and see what we can find... I wonder if PB2NC is
>> throwing out all the stuff we need based on the settings we put in
the
>> config file.
>>
>> Thanks,
>> Matt
>>
>> -----Original Message-----
>> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
>> Sent: Wednesday, January 16, 2013 4:27 PM
>> To: Sittel, Matthew C CTR USAF AFWA 16 WS/WXN
>> Cc: Craig, Robert J Civ USAF AFWA 16 WS/WXN
>> Subject: RE: [rt.rap.ucar.edu #59890] Problem with Pairing NetCDF
with
>> GriB
>>
>> Matt,
>>
>> I ran METv4.0 point_stat for the first ensemble member using the
attached
>> configuration file and verbosity level 3. I got 0 matched pairs,
but the
>> following diagnostic information is included in the output:
>>
>> DEBUG 2: Processing TMP/Z2 versus TMP/Z2, for observation type
ADPSFC,
>> over region FULL, for interpolation method BILIN(4), using 0 pairs.
>> DEBUG 3: Number of matched pairs = 0
>> DEBUG 3: Observations processed = 219303
>> DEBUG 3: Rejected: GRIB code = 190302
>> DEBUG 3: Rejected: valid time = 29001
>> DEBUG 3: Rejected: bad obs value = 0
>> DEBUG 3: Rejected: off the grid = 0
>> DEBUG 3: Rejected: level mismatch = 0
>> DEBUG 3: Rejected: message type = 0
>> DEBUG 3: Rejected: masking region = 0
>> DEBUG 3: Rejected: bad fcst value = 0
>>
>> This tells me that there's problem in the matching time window.
>>
>> Running "wgrib -V" on that forecast file tells me that this is a
24-hour
>> forecast initialized at 2013011412. So it's valid at 2013011512.
>>
>> Your observation file is named 2013011512, but when you actually
look at
>> the times it contains, they're for 2013011412:
>> ncdump -v hdr_vld nc2013011512
>>
>> (note, ncdump doesn't like file names that begin with a number, so
I stuck
>> an "nc" in front)
>>
>> I reran point-stat a second time, but first added 3 0's to the
"beg" and
>> "end" times in the "obs_window" setting. When I did that, I got
further,
>> but still no matched pairs:
>>
>> DEBUG 2: Processing TMP/Z2 versus TMP/Z2, for observation type
ADPSFC,
>> over region FULL, for interpolation method BILIN(4), using 0 pairs.
>> DEBUG 3: Number of matched pairs = 0
>> DEBUG 3: Observations processed = 219303
>> DEBUG 3: Rejected: GRIB code = 190302
>> DEBUG 3: Rejected: valid time = 0
>> DEBUG 3: Rejected: bad obs value = 0
>> DEBUG 3: Rejected: off the grid = 22047
>> DEBUG 3: Rejected: level mismatch = 6177
>> DEBUG 3: Rejected: message type = 777
>> DEBUG 3: Rejected: masking region = 0
>> DEBUG 3: Rejected: bad fcst value = 0
>>
>> So there just aren't any ADPSFC observations of 2-m temperature
that fall
>> within your d02 domain. I ran the plot_point_obs utility to verify
that
>> that's the case.
>>
>> As for why you got a seg-fault, I don't know. It's possible that
you made
>> an error in your Point-Stat config file that caused it. I can't
really
>> debug it unless I can replicate it.
>>
>> John
>>
>>>
>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59890 >
>>>
>>> Hi John,
>>>
>>> The plot thickens. When I ran point_stat on one ensemble member
vs.
>>> an observation file, MET returned a memory fault when it got to
the
>>> observation file read. I tried a second observation file and got
the
>>> same result.
>>>
>>> I looked using 'ncdump' on the NetCDF observation data output from
>>> 'pb2nc', and confirmed there are ADPSFC records, records within
the
>>> CONUS grid and in the requested time window.
>>>
>>> I used "-v 3" verbosity in the example I sent you... I think 4 is
the
>>> highest level of message? I tried rerunning this morning with "-v
6"
>>> and didn't get any additional information as to what is happening
with
>>> the observation matching, nor any messages numbered above 4.
>>>
>>> I have pushed the 10 ensemble members, observation file and config
>>> file to the directory you listed. I had to create a "met_help"
>>> directory under 'irap' as one was not there. Inside 'sittel' you
will
>>> find the files.
>>> Please note these were created in a Linux environment-not sure if
that
>>> will be an issue for you; it might depend on your OS. We were
forced
>>> to go to Linux here.
>>>
>>> Thanks,
>>> Matt
>>>
>>> -----Original Message-----
>>> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
>>> Sent: Tuesday, January 15, 2013 4:10 PM
>>> To: Sittel, Matthew C CTR USAF AFWA 16 WS/WXN
>>> Cc: Craig, Robert J Civ USAF AFWA 16 WS/WXN
>>> Subject: Re: [rt.rap.ucar.edu #59890] Problem with Pairing NetCDF
with
>>> GriB
>>>
>>> Matt,
>>>
>>> So the question is basically, why are you getting 0 matched pairs
from
>>> ensemble-stat. If this were the point-stat tool, the answer would
be
>>> easy. In point-stat, when you turn up the verbosity level, for
each
>>> verification task, a list of reasons for why observations were not
>>> used is printed out. And that's often an easy way to diagnose the
>>> problem.
>>>
>>> But that functionality has not been implemented for ensemble-stat.
>>> Though the problem you're having demonstrates the need!
>>>
>>> There are several possibilities:
>>> - the observations may not fall within the matching time window
>>> - the observations may not fall within the model's grid
>>> - perhaps there are no ADPSFC observations of TMP in your
dataset
>>>
>>> There are different steps I'd take to rule out each of these cases
-
>>> mostly making use of the plot_data_plane and plot_point_obs
utilities.
>>> But perhaps the easiest thing to do is to have you run one of your
>>> forecast files and the NetCDF output of PB2NC through point-stat
to
>>> verify TMP/Z2.
>>>
>>> Presumably, you'll get 0 matched pairs again. But when you
increase
>>> the verbosity level (-v 3, I think), you'll see a list of reasons
why
>>> the obs weren't used. Send me that list and then we'll know how
to
>>> proceed.
>>>
>>> Alternatively, you could just send me one or two of your GRIB
files,
>>> the NetCDF point observation file, and your config file, and I
could
>>> debug it here. If you choose to go that route, follow the
instructions
>>> here:
>>> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>>>
>>> Your config file settings look fine. The instructions for
specifying
>>> NetCDF config file stuff only applies to gridded NetCDF data. But
>>> your gridded data is in GRIB format. For point observations, we
chose
>>> the convention of specifying them as if they were GRIB files.
That's
>>> why we include a GRIB code for each observation value in the data.
>>>
>>> Just let me know which route you decide on. And I'm happy to help
get
>>> you up and running.
>>>
>>> Thanks,
>>> John
>>>
>>>>
>>>> Tue Jan 15 09:22:51 2013: Request 59890 was acted upon.
>>>> Transaction: Ticket created by matthew.sittel.ctr at offutt.af.mil
>>>> Queue: met_help
>>>> Subject: Problem with Pairing NetCDF with GriB
>>>> Owner: Nobody
>>>> Requestors: matthew.sittel.ctr at offutt.af.mil
>>>> Status: new
>>>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59890
>>>>>
>>>>
>>>>
>>>> Hi,
>>>>
>>>> I'm trying to get MET v4.0 to create verification rank histogram
>>>> output for AFWA's mesoscale ensemble. I'm using in-house, GriB-1
>>>> format WRF ensemble member output and verifying it with
>>>> PREPBUFR-formatted obs that we are pulling via FTP from NCEP. We
>>>> convert the PREPBUFR format to NetCDF using 'pb2nc'.
>>>>
>>>> The problem I'm having is I cannot figure out what the correct
>>>> settings are in the config file for ensemble_stat to get the
model
>>>> and observation fields to match up.
>>>>
>>>> I've attached the NetCDF 'dump' of the observation file's
contents,
>>>> the contents of my config file and the output I get when I run
>>>> ensemble_stat.
>>>> No matter what settings I pick, it always comes back with 0
pairs of
>>>> data. The output suggests the model members are read in and that
>>>> there are records in the observation file as well-both suggesting
to
>>>> me the data are read in correctly, but I'm not designating the
>>>> correct pairing of them with the config file parameters.
>>>>
>>>> The records I am trying to get are the ones where gc=11 and
hgt=0, or
>>>> simply the surface temperature reports from "ADPSFC" message
type. I
>>>> have read the user's guide and can't figure out what the right
>>>> settings are for NetCDF; it is discussed briefly but not with
enough
>>>> detail to figure this out on my own, especially the ANNN, RNNN,
etc,
>>>> and the TMP(I,...,J,*,*) wording which is unclear. Can you take
a
>>>> look and advise as to the right settings?
>>>>
>>>> Thanks,
>>>> Matt
>>>>
>>>> ==
>>>> netcdf 2013011412 {
>>>> dimensions:
>>>> mxstr = 16 ;
>>>> hdr_arr_len = 3 ;
>>>> obs_arr_len = 5 ;
>>>> nobs = UNLIMITED ; // (219303 currently)
>>>> nhdr = 40233 ;
>>>> variables:
>>>> float obs_arr(nobs, obs_arr_len) ;
>>>> obs_arr:long_name = "array of observation
values" ;
>>>> obs_arr:_fill_value = -9999.f ;
>>>> obs_arr:columns = "hdr_id gc lvl hgt ob" ;
>>>> obs_arr:hdr_id_long_name = "index of matching
header
>>>> data"
>>>> ;
>>>> obs_arr:gc_long_name = "grib code corresponding
to
>>>> the observation type" ;
>>>> obs_arr:lvl_long_name = "pressure level (hPa) or
>>>> accumulation interval (sec)" ;
>>>> obs_arr:hgt_long_name = "height in meters above
sea
>>>> level (msl)" ;
>>>> obs_arr:ob_long_name = "observation value" ;
>>>> char hdr_typ(nhdr, mxstr) ;
>>>> hdr_typ:long_name = "message type" ;
>>>> char hdr_sid(nhdr, mxstr) ;
>>>> hdr_sid:long_name = "station identification" ;
>>>> char hdr_vld(nhdr, mxstr) ;
>>>> hdr_vld:long_name = "valid time" ;
>>>> hdr_vld:units = "YYYYMMDD_HHMMSS" ;
>>>> float hdr_arr(nhdr, hdr_arr_len) ;
>>>> hdr_arr:long_name = "array of observation
station
>>>> header values" ;
>>>> hdr_arr:_fill_value = -9999.f ;
>>>> hdr_arr:columns = "lat lon elv" ;
>>>> hdr_arr:lat_long_name = "latitude" ;
>>>> hdr_arr:lat_units = "degrees_north" ;
>>>> hdr_arr:lon_long_name = "longitude" ;
>>>> hdr_arr:lon_units = "degrees_east" ;
>>>> hdr_arr:elv_long_name = "elevation" ;
>>>> hdr_arr:elv_units = "meters above sea level
(msl)" ;
>>>>
>>>> // global attributes:
>>>> :FileOrigins = "File
>>>> /h/data/global/WXQC/data//obs_prep/2013011412 generated
>>>> 20130115_003309 UTC on host dev8dlsn1 by the MET pb2nc tool" ;
>>>> :MET_version = "V4.0" ;
>>>> :MET_tool = "pb2nc" ;
>>>> data:
>>>> 40013, 7, 1016.4, 0, 0,
>>>> 40014, 7, 1012.2, 0, 0,
>>>> 40015, 7, 1024.4, 0, 0,
>>>> 40016, 11, 1030.6, 0, 236.55,
>>>> 40016, 7, 1030.6, 0, 0,
>>>> 40016, 2, 1030.6, 0, 103060,
>>>> 40017, 11, 1030.6, 0, 236.55,
>>>> 40017, 7, 1030.6, 0, 0,
>>>> 40017, 2, 1030.6, 0, 103060,
>>>> 40018, 11, 1030.9, 0, 235.95,
>>>> 40018, 7, 1030.9, 0, 0,
>>>> 40018, 2, 1030.9, 0, 103090,
>>>> 40019, 11, 1032.9, 0, 239.85,
>>>> 40019, 7, 1032.9, 0, 0,
>>>> 40019, 2, 1032.9, 0, 103290,
>>>> 40020, 11, 1034.1, 0, 240.65,
>>>> 40020, 7, 1034.1, 0, 0,
>>>> 40020, 2, 1034.1, 0, 103410,
>>>>
>>>>
/////////////////////////////////////////////////////////////////////
>>>> ///////////
>>>> //
>>>> // Ensemble-Stat configuration file.
>>>> //
>>>> // For additional information, see the
MET_BASE/data/config/README
>>>> file.
>>>> //
>>>>
/////////////////////////////////////////////////////////////////////
>>>> ///////////
>>>>
>>>> //
>>>> // Output model name to be written
>>>> //
>>>> model = "WRF";
>>>>
>>>>
/////////////////////////////////////////////////////////////////////
>>>> ///////////
>>>>
>>>> //
>>>> // Ensemble product fields to be processed // ens = {
>>>> ens_thresh = 1.0;
>>>> vld_thresh = 1.0;
>>>>
>>>> field = [
>>>> {
>>>> name = "TMP";
>>>> level = [ "Z2" ];
>>>> cat_thresh = [ >=300.0 ];
>>>> }
>>>> ];
>>>> }
>>>>
>>>>
/////////////////////////////////////////////////////////////////////
>>>> ///////////
>>>>
>>>> //
>>>> // Forecast and observation fields to be verified // fcst = {
>>>> message_type = [ "ADPSFC" ];
>>>> field = [
>>>> {
>>>> name = "TMP";
>>>> level = "Z2";
>>>> }
>>>> ];
>>>> }
>>>>
>>>> obs=fcst;
>>>>
>>>>
/////////////////////////////////////////////////////////////////////
>>>> ///////////
>>>>
>>>> //
>>>> // Point observation time window
>>>> //
>>>> obs_window = {
>>>> beg = -1800;
>>>> end = 1800;
>>>> }
>>>>
>>>>
/////////////////////////////////////////////////////////////////////
>>>> ///////////
>>>>
>>>> //
>>>> // Verification masking regions
>>>> //
>>>> mask = {
>>>> grid = [ "FULL" ];
>>>> poly = [];
>>>> sid = "";
>>>> };
>>>>
>>>>
/////////////////////////////////////////////////////////////////////
>>>> ///////////
>>>>
>>>> //
>>>> // Interpolation methods
>>>> //
>>>> interp = {
>>>> field = OBS;
>>>> vld_thresh = 1.0;
>>>>
>>>> type = [
>>>> {
>>>> method = BILIN;
>>>> width = 2;
>>>> }
>>>> ];
>>>> };
>>>>
>>>>
/////////////////////////////////////////////////////////////////////
>>>> ///////////
>>>>
>>>> //
>>>> // Statistical output types
>>>> //
>>>> output_flag = {
>>>> rhist = STAT;
>>>> orank = NONE;
>>>> };
>>>>
>>>>
/////////////////////////////////////////////////////////////////////
>>>> ///////////
>>>>
>>>> //
>>>> // Ensemble product output types
>>>> //
>>>> ensemble_flag = {
>>>> mean = TRUE;
>>>> stdev = TRUE;
>>>> minus = FALSE;
>>>> plus = FALSE;
>>>> min = FALSE;
>>>> max = FALSE;
>>>> range = FALSE;
>>>> vld_count = FALSE;
>>>> frequency = FALSE;
>>>> rank = TRUE;
>>>> };
>>>>
>>>>
/////////////////////////////////////////////////////////////////////
>>>> ///////////
>>>>
>>>> //
>>>> // Random number generator
>>>> //
>>>> rng = {
>>>> type = "mt19937";
>>>> seed = "1";
>>>> }
>>>>
>>>>
/////////////////////////////////////////////////////////////////////
>>>> ///////////
>>>>
>>>> duplicate_flag = NONE;
>>>> output_prefix = "";
>>>> version = "V4.0";
>>>>
>>>>
/////////////////////////////////////////////////////////////////////
>>>> ///////////
>>>> ==
>>>> Obs file: /h/data/global/WXQC/data/obs_prep/2013011412
>>>> DEBUG 1: Default Config File:
>>>> /home/qcteam/METv4.0/data/config/EnsembleStatConfig_default
>>>> DEBUG 1: User Config File:
/home/qcteam/METv4.0/cfg/ensemble_stat.cfg
>>>> DEBUG 1: Ensemble Files[10]:
>>>> DEBUG 1:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.1.tm00
>>>> DEBUG 1:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.2.tm00
>>>> DEBUG 1:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.3.tm00
>>>> DEBUG 1:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.4.tm00
>>>> DEBUG 1:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.5.tm00
>>>> DEBUG 1:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.6.tm00
>>>> DEBUG 1:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.7.tm00
>>>> DEBUG 1:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.8.tm00
>>>> DEBUG 1:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.9.tm00
>>>> DEBUG 1:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.10.tm00
>>>> DEBUG 1: Point Observation Files[1]:
>>>> DEBUG 1: /h/data/global/WXQC/data/obs_prep/2013011412
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 2: Processing ensemble field: TMP/Z2 DEBUG 3:
>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 2: Processing point observation file:
>>>> /h/data/global/WXQC/data/obs_prep/2013011412
>>>> DEBUG 2: Searching 219303 observations from 40233 header
messages.
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 2: Processing ensemble file:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.1.tm00 DEBUG 3:
>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 2: Processing ensemble file:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.2.tm00 DEBUG 3:
>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 2: Processing ensemble file:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.3.tm00 DEBUG 3:
>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 2: Processing ensemble file:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.4.tm00 DEBUG 3:
>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 2: Processing ensemble file:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.5.tm00 DEBUG 3:
>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 2: Processing ensemble file:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.6.tm00 DEBUG 3:
>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 2: Processing ensemble file:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.7.tm00 DEBUG 3:
>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 2: Processing ensemble file:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.8.tm00 DEBUG 3:
>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 2: Processing ensemble file:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.9.tm00 DEBUG 3:
>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 2: Processing ensemble file:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.10.tm00 DEBUG 3:
>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 2: Processing point verification TMP/Z2 versus TMP/Z2, for
>>>> observation type ADPSFC, over region FULL, for interpolation
method
>>>> BILIN(4), using 0 pairs.
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 1: Output file:
>>>>
/home/qcteam/METv4.0/out/ensemble_stat/ensemble_stat_20130114_120000V
>>>> .stat
>>>> DEBUG 1: Output file:
>>>>
/home/qcteam/METv4.0/out/ensemble_stat/ensemble_stat_20130114_120000V
>>>> _ens.nc
>>>>
>>>> // SIGNED //
>>>> Matthew C. Sittel
>>>> Associate Scientist
>>>> University Corporation for Atmospheric Research 16WS/WXN, Offutt
AFB,
>>>> NE
>>>> (402) 294-3473 DSN: 271-3473
>>>>
>>>>
>>>
>>>
>>>
>>>
>>>
>>
>>
>>
>
>
>
>
------------------------------------------------
Subject: Problem with Pairing NetCDF with GriB
From: matthew.sittel.ctr at offutt.af.mil
Time: Fri Jan 18 08:41:55 2013
Yeah I noticed the same thing. I made that change and got it to run
to completion. Success! Well, except for the no data in CONUS part.
Bob is now trying the PB2NC recompiled to see if it makes any
difference. I still can't figure out why we'd have nothing in CONUS
for ADPSFC temperature. Why would NCEP not include that?
I'm tempted to use 'ncgen' and create NetCDF files based on our own
observation data pulls, but that's a lot of work to reformat.
I wish ncdump was more user-friendly, where it matches up all fields
for a record rather than listing all the headers together, then all
the lat/lon, etc. I know it would look better in a database and might
be easier to follow... but simply from the command line? Forget it!
Matt
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Friday, January 18, 2013 9:20 AM
To: Sittel, Matthew C CTR USAF AFWA 16 WS/WXN
Cc: Craig, Robert J Civ USAF AFWA 16 WS/WXN
Subject: Re: [rt.rap.ucar.edu #59890] Problem with Pairing NetCDF with
GriB
Matt,
Good news. The problem was in your point_stat.cfg file. Diffing it
against the default one in
METv4.0/data/config/PointStatConfig_default, I see the following:
< poly = "";
---
> poly = [];
In the masking section, it expects "poly" to be specified as an array.
Instead of specifying it as an empty array, you specified it as a
string of length 0. So it tried to read a file of that name.
Running METv4.0 with the latest set of patches at verbosity level 4,
gave the following helpful information:
DEBUG 2: Reading data for TMP/Z2.
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
"WRFPRS24_d02.1.tm00".
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB records
matching VarInfo "TMP/Z2" in GRIB file "WRFPRS24_d02.1.tm00".
DEBUG 4: parse_sid_mask() -> parsing station ID mask file "(nul)"
DEBUG 4: parse_grid_mask() -> parsing grid mask "FULL"
DEBUG 4: parse_poly_mask() -> parsing poly mask "(nul)"
ERROR :
ERROR : grd_file_type() -> file does not exist "(nul)"
ERROR :
However, running the same thing for METv4.0 as originally released
results in a seg fault. Yeah, we found it!
Looking at the list of fixes, I'm guessing it's the bug fix posted on
08/01/2012:
"Fixed a bug that caused a seg fault when a config file string
list contains only an empty string"
http://www.dtcenter.org/met/users/support/known_issues/METv4.0/index.php
So just edit your Point-Stat config file to use:
poly = [];
And then recompile METv4.0 after applying the latest set of patches.
Hopefully that'll do the trick. Just let us know if you have more
problems or questions.
Thanks,
John
On 01/18/2013 07:53 AM, matthew.sittel.ctr at offutt.af.mil via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59890 >
>
> Hi John,
>
> I've added the point_stat config file to the FTP directory with the
other files.
>
> I have not downloaded any of the patches for v4.0. I thought I had
but turns out it was for v3.1 back on May 29 '12.
>
> Bob has been looking at the PB2NC process. The obs plotting
capability has been helpful.
>
> I'll recompile and let you know if it helps.
>
> Thanks,
> Matt
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Thursday, January 17, 2013 4:42 PM
> To: Sittel, Matthew C CTR USAF AFWA 16 WS/WXN
> Cc: Craig, Robert J Civ USAF AFWA 16 WS/WXN
> Subject: RE: [rt.rap.ucar.edu #59890] Problem with Pairing NetCDF
with GriB
>
> Matt,
>
> Yeah, the ncdump thing is silly. The older version in my path
complains
> about leading numbers, but I believe that later versions allow it.
Guess
> I need to update my path!
>
> I'm not at my desk this afternoon, so am unable to debug until
tomorrow
> morning. But I'm still puzzled about the source of that seg fault.
I'd
> like to be able to replicate it to address it.
>
> Could you please do 2 things:
> - Send me the exact point-stat config file you're using:
> /home/qcteam/METv4.0/cfg/point_stat.cfg
> - Let me know the date of the latest set of patches you're using for
> METv4.0. Just take a look in /home/qcteam/METv4.0 to see if there's
a
> patch file in there with a date in the filename.
>
> Hopefully, if I run your data with your config file and the exact
same
> version of the code, I'll be able to reproduce it. If not, it's a
lot
> more difficult problem.
>
> You could always try applying the latest set of METv4.0 patches and
> recompiling to see if that helps:
>
http://www.dtcenter.org/met/users/support/known_issues/METv4.0/index.php
>
> Regarding why I got 0 matched pairs even when accounting for the
time
> offset, I think looking at the PB2NC configuration is a great idea.
In
> particular, look to see if you're filtering on the observation qc
flag.
> If you also send me your PB2NC config file and the location of the
> PREPBUFR files you're using, I'd be happy to take a look at that
too.
>
> Thanks,
> John
>
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59890 >
>>
>> Hi John,
>>
>> I put an updated obs file onto the FTP site with correct time
>> stamps-overwrote the old one. Talk about confusing. The file I
sent you
>> was later overwritten by a new file so when I looked at it, the
date
>> fields were great. Bob will check into why our process pulls data
from
>> one date and saves it as another.
>>
>> Weird, ncdump accepts numerical file names for me... were you
running this
>> in a Linux environment? Not blaming Linux, but it's always a good
place
>> to start...
>>
>> Here's the output from my point_stat run of member 1 vs. the
observation
>> file:
>>
>> =======
>> /home/qcteam/METv4.0/bin/point_stat
>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011412/GRIB_OUT_d02/WRFPRS24_d02.1.tm00
>> /h/data/global/WXQC/data/obs_prep/2013011512
>> /home/qcteam/METv4.0/cfg/point_stat.cfg -outdir
/home/qcteam/METv4.0/out
>> -v 4
>>
>> DEBUG 1: Default Config File:
>> /home/qcteam/METv4.0/data/config/PointStatConfig_default
>> DEBUG 1: User Config File: /home/qcteam/METv4.0/cfg/point_stat.cfg
>> DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created
new
>> Met2dDataFile object of type "FileType_Gb1".
>> DEBUG 4: VarInfoFactory::new_var_info() -> created new VarInfo
object of
>> type "FileType_Gb1".
>> GSL_RNG_TYPE=mt19937
>> GSL_RNG_SEED=1
>> DEBUG 1: Forecast File:
>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011412/GRIB_OUT_d02/WRFPRS24_d02.1.tm00
>> DEBUG 1: Climatology File: none
>> DEBUG 1: Observation File:
/h/data/global/WXQC/data/obs_prep/2013011512
>> DEBUG 2:
>> DEBUG 2:
>>
--------------------------------------------------------------------------------
>> DEBUG 2:
>> DEBUG 2: Reading data for TMP/Z2.
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
>> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011412/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>> matching VarInfo "TMP/Z2" in GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011412/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>> Memory fault
>> =======
>>
>> I've never tried plot_point_obs-I don't see reference to it in the
4.0
>> users guide.
>>
>> Okay, by typing the executable it told me the arguments I needed.
Just
>> ran it... 33208 obs for ADPSFC type, 3102 obs for gc 11, but none
for the
>> intersect of the two... weird.
>>
>> We'll keep digging and see what we can find... I wonder if PB2NC is
>> throwing out all the stuff we need based on the settings we put in
the
>> config file.
>>
>> Thanks,
>> Matt
>>
>> -----Original Message-----
>> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
>> Sent: Wednesday, January 16, 2013 4:27 PM
>> To: Sittel, Matthew C CTR USAF AFWA 16 WS/WXN
>> Cc: Craig, Robert J Civ USAF AFWA 16 WS/WXN
>> Subject: RE: [rt.rap.ucar.edu #59890] Problem with Pairing NetCDF
with
>> GriB
>>
>> Matt,
>>
>> I ran METv4.0 point_stat for the first ensemble member using the
attached
>> configuration file and verbosity level 3. I got 0 matched pairs,
but the
>> following diagnostic information is included in the output:
>>
>> DEBUG 2: Processing TMP/Z2 versus TMP/Z2, for observation type
ADPSFC,
>> over region FULL, for interpolation method BILIN(4), using 0 pairs.
>> DEBUG 3: Number of matched pairs = 0
>> DEBUG 3: Observations processed = 219303
>> DEBUG 3: Rejected: GRIB code = 190302
>> DEBUG 3: Rejected: valid time = 29001
>> DEBUG 3: Rejected: bad obs value = 0
>> DEBUG 3: Rejected: off the grid = 0
>> DEBUG 3: Rejected: level mismatch = 0
>> DEBUG 3: Rejected: message type = 0
>> DEBUG 3: Rejected: masking region = 0
>> DEBUG 3: Rejected: bad fcst value = 0
>>
>> This tells me that there's problem in the matching time window.
>>
>> Running "wgrib -V" on that forecast file tells me that this is a
24-hour
>> forecast initialized at 2013011412. So it's valid at 2013011512.
>>
>> Your observation file is named 2013011512, but when you actually
look at
>> the times it contains, they're for 2013011412:
>> ncdump -v hdr_vld nc2013011512
>>
>> (note, ncdump doesn't like file names that begin with a number, so
I stuck
>> an "nc" in front)
>>
>> I reran point-stat a second time, but first added 3 0's to the
"beg" and
>> "end" times in the "obs_window" setting. When I did that, I got
further,
>> but still no matched pairs:
>>
>> DEBUG 2: Processing TMP/Z2 versus TMP/Z2, for observation type
ADPSFC,
>> over region FULL, for interpolation method BILIN(4), using 0 pairs.
>> DEBUG 3: Number of matched pairs = 0
>> DEBUG 3: Observations processed = 219303
>> DEBUG 3: Rejected: GRIB code = 190302
>> DEBUG 3: Rejected: valid time = 0
>> DEBUG 3: Rejected: bad obs value = 0
>> DEBUG 3: Rejected: off the grid = 22047
>> DEBUG 3: Rejected: level mismatch = 6177
>> DEBUG 3: Rejected: message type = 777
>> DEBUG 3: Rejected: masking region = 0
>> DEBUG 3: Rejected: bad fcst value = 0
>>
>> So there just aren't any ADPSFC observations of 2-m temperature
that fall
>> within your d02 domain. I ran the plot_point_obs utility to verify
that
>> that's the case.
>>
>> As for why you got a seg-fault, I don't know. It's possible that
you made
>> an error in your Point-Stat config file that caused it. I can't
really
>> debug it unless I can replicate it.
>>
>> John
>>
>>>
>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59890 >
>>>
>>> Hi John,
>>>
>>> The plot thickens. When I ran point_stat on one ensemble member
vs.
>>> an observation file, MET returned a memory fault when it got to
the
>>> observation file read. I tried a second observation file and got
the
>>> same result.
>>>
>>> I looked using 'ncdump' on the NetCDF observation data output from
>>> 'pb2nc', and confirmed there are ADPSFC records, records within
the
>>> CONUS grid and in the requested time window.
>>>
>>> I used "-v 3" verbosity in the example I sent you... I think 4 is
the
>>> highest level of message? I tried rerunning this morning with "-v
6"
>>> and didn't get any additional information as to what is happening
with
>>> the observation matching, nor any messages numbered above 4.
>>>
>>> I have pushed the 10 ensemble members, observation file and config
>>> file to the directory you listed. I had to create a "met_help"
>>> directory under 'irap' as one was not there. Inside 'sittel' you
will
>>> find the files.
>>> Please note these were created in a Linux environment-not sure if
that
>>> will be an issue for you; it might depend on your OS. We were
forced
>>> to go to Linux here.
>>>
>>> Thanks,
>>> Matt
>>>
>>> -----Original Message-----
>>> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
>>> Sent: Tuesday, January 15, 2013 4:10 PM
>>> To: Sittel, Matthew C CTR USAF AFWA 16 WS/WXN
>>> Cc: Craig, Robert J Civ USAF AFWA 16 WS/WXN
>>> Subject: Re: [rt.rap.ucar.edu #59890] Problem with Pairing NetCDF
with
>>> GriB
>>>
>>> Matt,
>>>
>>> So the question is basically, why are you getting 0 matched pairs
from
>>> ensemble-stat. If this were the point-stat tool, the answer would
be
>>> easy. In point-stat, when you turn up the verbosity level, for
each
>>> verification task, a list of reasons for why observations were not
>>> used is printed out. And that's often an easy way to diagnose the
>>> problem.
>>>
>>> But that functionality has not been implemented for ensemble-stat.
>>> Though the problem you're having demonstrates the need!
>>>
>>> There are several possibilities:
>>> - the observations may not fall within the matching time window
>>> - the observations may not fall within the model's grid
>>> - perhaps there are no ADPSFC observations of TMP in your
dataset
>>>
>>> There are different steps I'd take to rule out each of these cases
-
>>> mostly making use of the plot_data_plane and plot_point_obs
utilities.
>>> But perhaps the easiest thing to do is to have you run one of your
>>> forecast files and the NetCDF output of PB2NC through point-stat
to
>>> verify TMP/Z2.
>>>
>>> Presumably, you'll get 0 matched pairs again. But when you
increase
>>> the verbosity level (-v 3, I think), you'll see a list of reasons
why
>>> the obs weren't used. Send me that list and then we'll know how
to
>>> proceed.
>>>
>>> Alternatively, you could just send me one or two of your GRIB
files,
>>> the NetCDF point observation file, and your config file, and I
could
>>> debug it here. If you choose to go that route, follow the
instructions
>>> here:
>>> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>>>
>>> Your config file settings look fine. The instructions for
specifying
>>> NetCDF config file stuff only applies to gridded NetCDF data. But
>>> your gridded data is in GRIB format. For point observations, we
chose
>>> the convention of specifying them as if they were GRIB files.
That's
>>> why we include a GRIB code for each observation value in the data.
>>>
>>> Just let me know which route you decide on. And I'm happy to help
get
>>> you up and running.
>>>
>>> Thanks,
>>> John
>>>
>>>>
>>>> Tue Jan 15 09:22:51 2013: Request 59890 was acted upon.
>>>> Transaction: Ticket created by matthew.sittel.ctr at offutt.af.mil
>>>> Queue: met_help
>>>> Subject: Problem with Pairing NetCDF with GriB
>>>> Owner: Nobody
>>>> Requestors: matthew.sittel.ctr at offutt.af.mil
>>>> Status: new
>>>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59890
>>>>>
>>>>
>>>>
>>>> Hi,
>>>>
>>>> I'm trying to get MET v4.0 to create verification rank histogram
>>>> output for AFWA's mesoscale ensemble. I'm using in-house, GriB-1
>>>> format WRF ensemble member output and verifying it with
>>>> PREPBUFR-formatted obs that we are pulling via FTP from NCEP. We
>>>> convert the PREPBUFR format to NetCDF using 'pb2nc'.
>>>>
>>>> The problem I'm having is I cannot figure out what the correct
>>>> settings are in the config file for ensemble_stat to get the
model
>>>> and observation fields to match up.
>>>>
>>>> I've attached the NetCDF 'dump' of the observation file's
contents,
>>>> the contents of my config file and the output I get when I run
>>>> ensemble_stat.
>>>> No matter what settings I pick, it always comes back with 0
pairs of
>>>> data. The output suggests the model members are read in and that
>>>> there are records in the observation file as well-both suggesting
to
>>>> me the data are read in correctly, but I'm not designating the
>>>> correct pairing of them with the config file parameters.
>>>>
>>>> The records I am trying to get are the ones where gc=11 and
hgt=0, or
>>>> simply the surface temperature reports from "ADPSFC" message
type. I
>>>> have read the user's guide and can't figure out what the right
>>>> settings are for NetCDF; it is discussed briefly but not with
enough
>>>> detail to figure this out on my own, especially the ANNN, RNNN,
etc,
>>>> and the TMP(I,...,J,*,*) wording which is unclear. Can you take
a
>>>> look and advise as to the right settings?
>>>>
>>>> Thanks,
>>>> Matt
>>>>
>>>> ==
>>>> netcdf 2013011412 {
>>>> dimensions:
>>>> mxstr = 16 ;
>>>> hdr_arr_len = 3 ;
>>>> obs_arr_len = 5 ;
>>>> nobs = UNLIMITED ; // (219303 currently)
>>>> nhdr = 40233 ;
>>>> variables:
>>>> float obs_arr(nobs, obs_arr_len) ;
>>>> obs_arr:long_name = "array of observation
values" ;
>>>> obs_arr:_fill_value = -9999.f ;
>>>> obs_arr:columns = "hdr_id gc lvl hgt ob" ;
>>>> obs_arr:hdr_id_long_name = "index of matching
header
>>>> data"
>>>> ;
>>>> obs_arr:gc_long_name = "grib code corresponding
to
>>>> the observation type" ;
>>>> obs_arr:lvl_long_name = "pressure level (hPa) or
>>>> accumulation interval (sec)" ;
>>>> obs_arr:hgt_long_name = "height in meters above
sea
>>>> level (msl)" ;
>>>> obs_arr:ob_long_name = "observation value" ;
>>>> char hdr_typ(nhdr, mxstr) ;
>>>> hdr_typ:long_name = "message type" ;
>>>> char hdr_sid(nhdr, mxstr) ;
>>>> hdr_sid:long_name = "station identification" ;
>>>> char hdr_vld(nhdr, mxstr) ;
>>>> hdr_vld:long_name = "valid time" ;
>>>> hdr_vld:units = "YYYYMMDD_HHMMSS" ;
>>>> float hdr_arr(nhdr, hdr_arr_len) ;
>>>> hdr_arr:long_name = "array of observation
station
>>>> header values" ;
>>>> hdr_arr:_fill_value = -9999.f ;
>>>> hdr_arr:columns = "lat lon elv" ;
>>>> hdr_arr:lat_long_name = "latitude" ;
>>>> hdr_arr:lat_units = "degrees_north" ;
>>>> hdr_arr:lon_long_name = "longitude" ;
>>>> hdr_arr:lon_units = "degrees_east" ;
>>>> hdr_arr:elv_long_name = "elevation" ;
>>>> hdr_arr:elv_units = "meters above sea level
(msl)" ;
>>>>
>>>> // global attributes:
>>>> :FileOrigins = "File
>>>> /h/data/global/WXQC/data//obs_prep/2013011412 generated
>>>> 20130115_003309 UTC on host dev8dlsn1 by the MET pb2nc tool" ;
>>>> :MET_version = "V4.0" ;
>>>> :MET_tool = "pb2nc" ;
>>>> data:
>>>> 40013, 7, 1016.4, 0, 0,
>>>> 40014, 7, 1012.2, 0, 0,
>>>> 40015, 7, 1024.4, 0, 0,
>>>> 40016, 11, 1030.6, 0, 236.55,
>>>> 40016, 7, 1030.6, 0, 0,
>>>> 40016, 2, 1030.6, 0, 103060,
>>>> 40017, 11, 1030.6, 0, 236.55,
>>>> 40017, 7, 1030.6, 0, 0,
>>>> 40017, 2, 1030.6, 0, 103060,
>>>> 40018, 11, 1030.9, 0, 235.95,
>>>> 40018, 7, 1030.9, 0, 0,
>>>> 40018, 2, 1030.9, 0, 103090,
>>>> 40019, 11, 1032.9, 0, 239.85,
>>>> 40019, 7, 1032.9, 0, 0,
>>>> 40019, 2, 1032.9, 0, 103290,
>>>> 40020, 11, 1034.1, 0, 240.65,
>>>> 40020, 7, 1034.1, 0, 0,
>>>> 40020, 2, 1034.1, 0, 103410,
>>>>
>>>>
/////////////////////////////////////////////////////////////////////
>>>> ///////////
>>>> //
>>>> // Ensemble-Stat configuration file.
>>>> //
>>>> // For additional information, see the
MET_BASE/data/config/README
>>>> file.
>>>> //
>>>>
/////////////////////////////////////////////////////////////////////
>>>> ///////////
>>>>
>>>> //
>>>> // Output model name to be written
>>>> //
>>>> model = "WRF";
>>>>
>>>>
/////////////////////////////////////////////////////////////////////
>>>> ///////////
>>>>
>>>> //
>>>> // Ensemble product fields to be processed // ens = {
>>>> ens_thresh = 1.0;
>>>> vld_thresh = 1.0;
>>>>
>>>> field = [
>>>> {
>>>> name = "TMP";
>>>> level = [ "Z2" ];
>>>> cat_thresh = [ >=300.0 ];
>>>> }
>>>> ];
>>>> }
>>>>
>>>>
/////////////////////////////////////////////////////////////////////
>>>> ///////////
>>>>
>>>> //
>>>> // Forecast and observation fields to be verified // fcst = {
>>>> message_type = [ "ADPSFC" ];
>>>> field = [
>>>> {
>>>> name = "TMP";
>>>> level = "Z2";
>>>> }
>>>> ];
>>>> }
>>>>
>>>> obs=fcst;
>>>>
>>>>
/////////////////////////////////////////////////////////////////////
>>>> ///////////
>>>>
>>>> //
>>>> // Point observation time window
>>>> //
>>>> obs_window = {
>>>> beg = -1800;
>>>> end = 1800;
>>>> }
>>>>
>>>>
/////////////////////////////////////////////////////////////////////
>>>> ///////////
>>>>
>>>> //
>>>> // Verification masking regions
>>>> //
>>>> mask = {
>>>> grid = [ "FULL" ];
>>>> poly = [];
>>>> sid = "";
>>>> };
>>>>
>>>>
/////////////////////////////////////////////////////////////////////
>>>> ///////////
>>>>
>>>> //
>>>> // Interpolation methods
>>>> //
>>>> interp = {
>>>> field = OBS;
>>>> vld_thresh = 1.0;
>>>>
>>>> type = [
>>>> {
>>>> method = BILIN;
>>>> width = 2;
>>>> }
>>>> ];
>>>> };
>>>>
>>>>
/////////////////////////////////////////////////////////////////////
>>>> ///////////
>>>>
>>>> //
>>>> // Statistical output types
>>>> //
>>>> output_flag = {
>>>> rhist = STAT;
>>>> orank = NONE;
>>>> };
>>>>
>>>>
/////////////////////////////////////////////////////////////////////
>>>> ///////////
>>>>
>>>> //
>>>> // Ensemble product output types
>>>> //
>>>> ensemble_flag = {
>>>> mean = TRUE;
>>>> stdev = TRUE;
>>>> minus = FALSE;
>>>> plus = FALSE;
>>>> min = FALSE;
>>>> max = FALSE;
>>>> range = FALSE;
>>>> vld_count = FALSE;
>>>> frequency = FALSE;
>>>> rank = TRUE;
>>>> };
>>>>
>>>>
/////////////////////////////////////////////////////////////////////
>>>> ///////////
>>>>
>>>> //
>>>> // Random number generator
>>>> //
>>>> rng = {
>>>> type = "mt19937";
>>>> seed = "1";
>>>> }
>>>>
>>>>
/////////////////////////////////////////////////////////////////////
>>>> ///////////
>>>>
>>>> duplicate_flag = NONE;
>>>> output_prefix = "";
>>>> version = "V4.0";
>>>>
>>>>
/////////////////////////////////////////////////////////////////////
>>>> ///////////
>>>> ==
>>>> Obs file: /h/data/global/WXQC/data/obs_prep/2013011412
>>>> DEBUG 1: Default Config File:
>>>> /home/qcteam/METv4.0/data/config/EnsembleStatConfig_default
>>>> DEBUG 1: User Config File:
/home/qcteam/METv4.0/cfg/ensemble_stat.cfg
>>>> DEBUG 1: Ensemble Files[10]:
>>>> DEBUG 1:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.1.tm00
>>>> DEBUG 1:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.2.tm00
>>>> DEBUG 1:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.3.tm00
>>>> DEBUG 1:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.4.tm00
>>>> DEBUG 1:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.5.tm00
>>>> DEBUG 1:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.6.tm00
>>>> DEBUG 1:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.7.tm00
>>>> DEBUG 1:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.8.tm00
>>>> DEBUG 1:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.9.tm00
>>>> DEBUG 1:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.10.tm00
>>>> DEBUG 1: Point Observation Files[1]:
>>>> DEBUG 1: /h/data/global/WXQC/data/obs_prep/2013011412
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 2: Processing ensemble field: TMP/Z2 DEBUG 3:
>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 2: Processing point observation file:
>>>> /h/data/global/WXQC/data/obs_prep/2013011412
>>>> DEBUG 2: Searching 219303 observations from 40233 header
messages.
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 2: Processing ensemble file:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.1.tm00 DEBUG 3:
>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 2: Processing ensemble file:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.2.tm00 DEBUG 3:
>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 2: Processing ensemble file:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.3.tm00 DEBUG 3:
>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 2: Processing ensemble file:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.4.tm00 DEBUG 3:
>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 2: Processing ensemble file:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.5.tm00 DEBUG 3:
>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 2: Processing ensemble file:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.6.tm00 DEBUG 3:
>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 2: Processing ensemble file:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.7.tm00 DEBUG 3:
>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 2: Processing ensemble file:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.8.tm00 DEBUG 3:
>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 2: Processing ensemble file:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.9.tm00 DEBUG 3:
>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 2: Processing ensemble file:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.10.tm00 DEBUG 3:
>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 2: Processing point verification TMP/Z2 versus TMP/Z2, for
>>>> observation type ADPSFC, over region FULL, for interpolation
method
>>>> BILIN(4), using 0 pairs.
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 1: Output file:
>>>>
/home/qcteam/METv4.0/out/ensemble_stat/ensemble_stat_20130114_120000V
>>>> .stat
>>>> DEBUG 1: Output file:
>>>>
/home/qcteam/METv4.0/out/ensemble_stat/ensemble_stat_20130114_120000V
>>>> _ens.nc
>>>>
>>>> // SIGNED //
>>>> Matthew C. Sittel
>>>> Associate Scientist
>>>> University Corporation for Atmospheric Research 16WS/WXN, Offutt
AFB,
>>>> NE
>>>> (402) 294-3473 DSN: 271-3473
>>>>
>>>>
>>>
>>>
>>>
>>>
>>>
>>
>>
>>
>
>
>
>
------------------------------------------------
Subject: Problem with Pairing NetCDF with GriB
From: matthew.sittel.ctr at offutt.af.mil
Time: Fri Jan 18 08:59:48 2013
Interestingly, I went to
http://www.dtcenter.org/met/users/downloads/observation_data.php and
tried pulling the NDAS flavor of the observation data. That works
great-we have data in CONUS! What I can't figure out is if NDAS and
GDAS are mutually exclusive files such that you should read *both* to
get a globe full of data.
Strange... do you know anything about GDAS vs NDAS prepbufr data?
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Friday, January 18, 2013 9:20 AM
To: Sittel, Matthew C CTR USAF AFWA 16 WS/WXN
Cc: Craig, Robert J Civ USAF AFWA 16 WS/WXN
Subject: Re: [rt.rap.ucar.edu #59890] Problem with Pairing NetCDF with
GriB
Matt,
Good news. The problem was in your point_stat.cfg file. Diffing it
against the default one in
METv4.0/data/config/PointStatConfig_default, I see the following:
< poly = "";
---
> poly = [];
In the masking section, it expects "poly" to be specified as an array.
Instead of specifying it as an empty array, you specified it as a
string of length 0. So it tried to read a file of that name.
Running METv4.0 with the latest set of patches at verbosity level 4,
gave the following helpful information:
DEBUG 2: Reading data for TMP/Z2.
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
"WRFPRS24_d02.1.tm00".
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB records
matching VarInfo "TMP/Z2" in GRIB file "WRFPRS24_d02.1.tm00".
DEBUG 4: parse_sid_mask() -> parsing station ID mask file "(nul)"
DEBUG 4: parse_grid_mask() -> parsing grid mask "FULL"
DEBUG 4: parse_poly_mask() -> parsing poly mask "(nul)"
ERROR :
ERROR : grd_file_type() -> file does not exist "(nul)"
ERROR :
However, running the same thing for METv4.0 as originally released
results in a seg fault. Yeah, we found it!
Looking at the list of fixes, I'm guessing it's the bug fix posted on
08/01/2012:
"Fixed a bug that caused a seg fault when a config file string
list contains only an empty string"
http://www.dtcenter.org/met/users/support/known_issues/METv4.0/index.php
So just edit your Point-Stat config file to use:
poly = [];
And then recompile METv4.0 after applying the latest set of patches.
Hopefully that'll do the trick. Just let us know if you have more
problems or questions.
Thanks,
John
On 01/18/2013 07:53 AM, matthew.sittel.ctr at offutt.af.mil via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59890 >
>
> Hi John,
>
> I've added the point_stat config file to the FTP directory with the
other files.
>
> I have not downloaded any of the patches for v4.0. I thought I had
but turns out it was for v3.1 back on May 29 '12.
>
> Bob has been looking at the PB2NC process. The obs plotting
capability has been helpful.
>
> I'll recompile and let you know if it helps.
>
> Thanks,
> Matt
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Thursday, January 17, 2013 4:42 PM
> To: Sittel, Matthew C CTR USAF AFWA 16 WS/WXN
> Cc: Craig, Robert J Civ USAF AFWA 16 WS/WXN
> Subject: RE: [rt.rap.ucar.edu #59890] Problem with Pairing NetCDF
with GriB
>
> Matt,
>
> Yeah, the ncdump thing is silly. The older version in my path
complains
> about leading numbers, but I believe that later versions allow it.
Guess
> I need to update my path!
>
> I'm not at my desk this afternoon, so am unable to debug until
tomorrow
> morning. But I'm still puzzled about the source of that seg fault.
I'd
> like to be able to replicate it to address it.
>
> Could you please do 2 things:
> - Send me the exact point-stat config file you're using:
> /home/qcteam/METv4.0/cfg/point_stat.cfg
> - Let me know the date of the latest set of patches you're using for
> METv4.0. Just take a look in /home/qcteam/METv4.0 to see if there's
a
> patch file in there with a date in the filename.
>
> Hopefully, if I run your data with your config file and the exact
same
> version of the code, I'll be able to reproduce it. If not, it's a
lot
> more difficult problem.
>
> You could always try applying the latest set of METv4.0 patches and
> recompiling to see if that helps:
>
http://www.dtcenter.org/met/users/support/known_issues/METv4.0/index.php
>
> Regarding why I got 0 matched pairs even when accounting for the
time
> offset, I think looking at the PB2NC configuration is a great idea.
In
> particular, look to see if you're filtering on the observation qc
flag.
> If you also send me your PB2NC config file and the location of the
> PREPBUFR files you're using, I'd be happy to take a look at that
too.
>
> Thanks,
> John
>
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59890 >
>>
>> Hi John,
>>
>> I put an updated obs file onto the FTP site with correct time
>> stamps-overwrote the old one. Talk about confusing. The file I
sent you
>> was later overwritten by a new file so when I looked at it, the
date
>> fields were great. Bob will check into why our process pulls data
from
>> one date and saves it as another.
>>
>> Weird, ncdump accepts numerical file names for me... were you
running this
>> in a Linux environment? Not blaming Linux, but it's always a good
place
>> to start...
>>
>> Here's the output from my point_stat run of member 1 vs. the
observation
>> file:
>>
>> =======
>> /home/qcteam/METv4.0/bin/point_stat
>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011412/GRIB_OUT_d02/WRFPRS24_d02.1.tm00
>> /h/data/global/WXQC/data/obs_prep/2013011512
>> /home/qcteam/METv4.0/cfg/point_stat.cfg -outdir
/home/qcteam/METv4.0/out
>> -v 4
>>
>> DEBUG 1: Default Config File:
>> /home/qcteam/METv4.0/data/config/PointStatConfig_default
>> DEBUG 1: User Config File: /home/qcteam/METv4.0/cfg/point_stat.cfg
>> DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created
new
>> Met2dDataFile object of type "FileType_Gb1".
>> DEBUG 4: VarInfoFactory::new_var_info() -> created new VarInfo
object of
>> type "FileType_Gb1".
>> GSL_RNG_TYPE=mt19937
>> GSL_RNG_SEED=1
>> DEBUG 1: Forecast File:
>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011412/GRIB_OUT_d02/WRFPRS24_d02.1.tm00
>> DEBUG 1: Climatology File: none
>> DEBUG 1: Observation File:
/h/data/global/WXQC/data/obs_prep/2013011512
>> DEBUG 2:
>> DEBUG 2:
>>
--------------------------------------------------------------------------------
>> DEBUG 2:
>> DEBUG 2: Reading data for TMP/Z2.
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
>> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011412/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>> matching VarInfo "TMP/Z2" in GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011412/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>> Memory fault
>> =======
>>
>> I've never tried plot_point_obs-I don't see reference to it in the
4.0
>> users guide.
>>
>> Okay, by typing the executable it told me the arguments I needed.
Just
>> ran it... 33208 obs for ADPSFC type, 3102 obs for gc 11, but none
for the
>> intersect of the two... weird.
>>
>> We'll keep digging and see what we can find... I wonder if PB2NC is
>> throwing out all the stuff we need based on the settings we put in
the
>> config file.
>>
>> Thanks,
>> Matt
>>
>> -----Original Message-----
>> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
>> Sent: Wednesday, January 16, 2013 4:27 PM
>> To: Sittel, Matthew C CTR USAF AFWA 16 WS/WXN
>> Cc: Craig, Robert J Civ USAF AFWA 16 WS/WXN
>> Subject: RE: [rt.rap.ucar.edu #59890] Problem with Pairing NetCDF
with
>> GriB
>>
>> Matt,
>>
>> I ran METv4.0 point_stat for the first ensemble member using the
attached
>> configuration file and verbosity level 3. I got 0 matched pairs,
but the
>> following diagnostic information is included in the output:
>>
>> DEBUG 2: Processing TMP/Z2 versus TMP/Z2, for observation type
ADPSFC,
>> over region FULL, for interpolation method BILIN(4), using 0 pairs.
>> DEBUG 3: Number of matched pairs = 0
>> DEBUG 3: Observations processed = 219303
>> DEBUG 3: Rejected: GRIB code = 190302
>> DEBUG 3: Rejected: valid time = 29001
>> DEBUG 3: Rejected: bad obs value = 0
>> DEBUG 3: Rejected: off the grid = 0
>> DEBUG 3: Rejected: level mismatch = 0
>> DEBUG 3: Rejected: message type = 0
>> DEBUG 3: Rejected: masking region = 0
>> DEBUG 3: Rejected: bad fcst value = 0
>>
>> This tells me that there's problem in the matching time window.
>>
>> Running "wgrib -V" on that forecast file tells me that this is a
24-hour
>> forecast initialized at 2013011412. So it's valid at 2013011512.
>>
>> Your observation file is named 2013011512, but when you actually
look at
>> the times it contains, they're for 2013011412:
>> ncdump -v hdr_vld nc2013011512
>>
>> (note, ncdump doesn't like file names that begin with a number, so
I stuck
>> an "nc" in front)
>>
>> I reran point-stat a second time, but first added 3 0's to the
"beg" and
>> "end" times in the "obs_window" setting. When I did that, I got
further,
>> but still no matched pairs:
>>
>> DEBUG 2: Processing TMP/Z2 versus TMP/Z2, for observation type
ADPSFC,
>> over region FULL, for interpolation method BILIN(4), using 0 pairs.
>> DEBUG 3: Number of matched pairs = 0
>> DEBUG 3: Observations processed = 219303
>> DEBUG 3: Rejected: GRIB code = 190302
>> DEBUG 3: Rejected: valid time = 0
>> DEBUG 3: Rejected: bad obs value = 0
>> DEBUG 3: Rejected: off the grid = 22047
>> DEBUG 3: Rejected: level mismatch = 6177
>> DEBUG 3: Rejected: message type = 777
>> DEBUG 3: Rejected: masking region = 0
>> DEBUG 3: Rejected: bad fcst value = 0
>>
>> So there just aren't any ADPSFC observations of 2-m temperature
that fall
>> within your d02 domain. I ran the plot_point_obs utility to verify
that
>> that's the case.
>>
>> As for why you got a seg-fault, I don't know. It's possible that
you made
>> an error in your Point-Stat config file that caused it. I can't
really
>> debug it unless I can replicate it.
>>
>> John
>>
>>>
>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59890 >
>>>
>>> Hi John,
>>>
>>> The plot thickens. When I ran point_stat on one ensemble member
vs.
>>> an observation file, MET returned a memory fault when it got to
the
>>> observation file read. I tried a second observation file and got
the
>>> same result.
>>>
>>> I looked using 'ncdump' on the NetCDF observation data output from
>>> 'pb2nc', and confirmed there are ADPSFC records, records within
the
>>> CONUS grid and in the requested time window.
>>>
>>> I used "-v 3" verbosity in the example I sent you... I think 4 is
the
>>> highest level of message? I tried rerunning this morning with "-v
6"
>>> and didn't get any additional information as to what is happening
with
>>> the observation matching, nor any messages numbered above 4.
>>>
>>> I have pushed the 10 ensemble members, observation file and config
>>> file to the directory you listed. I had to create a "met_help"
>>> directory under 'irap' as one was not there. Inside 'sittel' you
will
>>> find the files.
>>> Please note these were created in a Linux environment-not sure if
that
>>> will be an issue for you; it might depend on your OS. We were
forced
>>> to go to Linux here.
>>>
>>> Thanks,
>>> Matt
>>>
>>> -----Original Message-----
>>> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
>>> Sent: Tuesday, January 15, 2013 4:10 PM
>>> To: Sittel, Matthew C CTR USAF AFWA 16 WS/WXN
>>> Cc: Craig, Robert J Civ USAF AFWA 16 WS/WXN
>>> Subject: Re: [rt.rap.ucar.edu #59890] Problem with Pairing NetCDF
with
>>> GriB
>>>
>>> Matt,
>>>
>>> So the question is basically, why are you getting 0 matched pairs
from
>>> ensemble-stat. If this were the point-stat tool, the answer would
be
>>> easy. In point-stat, when you turn up the verbosity level, for
each
>>> verification task, a list of reasons for why observations were not
>>> used is printed out. And that's often an easy way to diagnose the
>>> problem.
>>>
>>> But that functionality has not been implemented for ensemble-stat.
>>> Though the problem you're having demonstrates the need!
>>>
>>> There are several possibilities:
>>> - the observations may not fall within the matching time window
>>> - the observations may not fall within the model's grid
>>> - perhaps there are no ADPSFC observations of TMP in your
dataset
>>>
>>> There are different steps I'd take to rule out each of these cases
-
>>> mostly making use of the plot_data_plane and plot_point_obs
utilities.
>>> But perhaps the easiest thing to do is to have you run one of your
>>> forecast files and the NetCDF output of PB2NC through point-stat
to
>>> verify TMP/Z2.
>>>
>>> Presumably, you'll get 0 matched pairs again. But when you
increase
>>> the verbosity level (-v 3, I think), you'll see a list of reasons
why
>>> the obs weren't used. Send me that list and then we'll know how
to
>>> proceed.
>>>
>>> Alternatively, you could just send me one or two of your GRIB
files,
>>> the NetCDF point observation file, and your config file, and I
could
>>> debug it here. If you choose to go that route, follow the
instructions
>>> here:
>>> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>>>
>>> Your config file settings look fine. The instructions for
specifying
>>> NetCDF config file stuff only applies to gridded NetCDF data. But
>>> your gridded data is in GRIB format. For point observations, we
chose
>>> the convention of specifying them as if they were GRIB files.
That's
>>> why we include a GRIB code for each observation value in the data.
>>>
>>> Just let me know which route you decide on. And I'm happy to help
get
>>> you up and running.
>>>
>>> Thanks,
>>> John
>>>
>>>>
>>>> Tue Jan 15 09:22:51 2013: Request 59890 was acted upon.
>>>> Transaction: Ticket created by matthew.sittel.ctr at offutt.af.mil
>>>> Queue: met_help
>>>> Subject: Problem with Pairing NetCDF with GriB
>>>> Owner: Nobody
>>>> Requestors: matthew.sittel.ctr at offutt.af.mil
>>>> Status: new
>>>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59890
>>>>>
>>>>
>>>>
>>>> Hi,
>>>>
>>>> I'm trying to get MET v4.0 to create verification rank histogram
>>>> output for AFWA's mesoscale ensemble. I'm using in-house, GriB-1
>>>> format WRF ensemble member output and verifying it with
>>>> PREPBUFR-formatted obs that we are pulling via FTP from NCEP. We
>>>> convert the PREPBUFR format to NetCDF using 'pb2nc'.
>>>>
>>>> The problem I'm having is I cannot figure out what the correct
>>>> settings are in the config file for ensemble_stat to get the
model
>>>> and observation fields to match up.
>>>>
>>>> I've attached the NetCDF 'dump' of the observation file's
contents,
>>>> the contents of my config file and the output I get when I run
>>>> ensemble_stat.
>>>> No matter what settings I pick, it always comes back with 0
pairs of
>>>> data. The output suggests the model members are read in and that
>>>> there are records in the observation file as well-both suggesting
to
>>>> me the data are read in correctly, but I'm not designating the
>>>> correct pairing of them with the config file parameters.
>>>>
>>>> The records I am trying to get are the ones where gc=11 and
hgt=0, or
>>>> simply the surface temperature reports from "ADPSFC" message
type. I
>>>> have read the user's guide and can't figure out what the right
>>>> settings are for NetCDF; it is discussed briefly but not with
enough
>>>> detail to figure this out on my own, especially the ANNN, RNNN,
etc,
>>>> and the TMP(I,...,J,*,*) wording which is unclear. Can you take
a
>>>> look and advise as to the right settings?
>>>>
>>>> Thanks,
>>>> Matt
>>>>
>>>> ==
>>>> netcdf 2013011412 {
>>>> dimensions:
>>>> mxstr = 16 ;
>>>> hdr_arr_len = 3 ;
>>>> obs_arr_len = 5 ;
>>>> nobs = UNLIMITED ; // (219303 currently)
>>>> nhdr = 40233 ;
>>>> variables:
>>>> float obs_arr(nobs, obs_arr_len) ;
>>>> obs_arr:long_name = "array of observation
values" ;
>>>> obs_arr:_fill_value = -9999.f ;
>>>> obs_arr:columns = "hdr_id gc lvl hgt ob" ;
>>>> obs_arr:hdr_id_long_name = "index of matching
header
>>>> data"
>>>> ;
>>>> obs_arr:gc_long_name = "grib code corresponding
to
>>>> the observation type" ;
>>>> obs_arr:lvl_long_name = "pressure level (hPa) or
>>>> accumulation interval (sec)" ;
>>>> obs_arr:hgt_long_name = "height in meters above
sea
>>>> level (msl)" ;
>>>> obs_arr:ob_long_name = "observation value" ;
>>>> char hdr_typ(nhdr, mxstr) ;
>>>> hdr_typ:long_name = "message type" ;
>>>> char hdr_sid(nhdr, mxstr) ;
>>>> hdr_sid:long_name = "station identification" ;
>>>> char hdr_vld(nhdr, mxstr) ;
>>>> hdr_vld:long_name = "valid time" ;
>>>> hdr_vld:units = "YYYYMMDD_HHMMSS" ;
>>>> float hdr_arr(nhdr, hdr_arr_len) ;
>>>> hdr_arr:long_name = "array of observation
station
>>>> header values" ;
>>>> hdr_arr:_fill_value = -9999.f ;
>>>> hdr_arr:columns = "lat lon elv" ;
>>>> hdr_arr:lat_long_name = "latitude" ;
>>>> hdr_arr:lat_units = "degrees_north" ;
>>>> hdr_arr:lon_long_name = "longitude" ;
>>>> hdr_arr:lon_units = "degrees_east" ;
>>>> hdr_arr:elv_long_name = "elevation" ;
>>>> hdr_arr:elv_units = "meters above sea level
(msl)" ;
>>>>
>>>> // global attributes:
>>>> :FileOrigins = "File
>>>> /h/data/global/WXQC/data//obs_prep/2013011412 generated
>>>> 20130115_003309 UTC on host dev8dlsn1 by the MET pb2nc tool" ;
>>>> :MET_version = "V4.0" ;
>>>> :MET_tool = "pb2nc" ;
>>>> data:
>>>> 40013, 7, 1016.4, 0, 0,
>>>> 40014, 7, 1012.2, 0, 0,
>>>> 40015, 7, 1024.4, 0, 0,
>>>> 40016, 11, 1030.6, 0, 236.55,
>>>> 40016, 7, 1030.6, 0, 0,
>>>> 40016, 2, 1030.6, 0, 103060,
>>>> 40017, 11, 1030.6, 0, 236.55,
>>>> 40017, 7, 1030.6, 0, 0,
>>>> 40017, 2, 1030.6, 0, 103060,
>>>> 40018, 11, 1030.9, 0, 235.95,
>>>> 40018, 7, 1030.9, 0, 0,
>>>> 40018, 2, 1030.9, 0, 103090,
>>>> 40019, 11, 1032.9, 0, 239.85,
>>>> 40019, 7, 1032.9, 0, 0,
>>>> 40019, 2, 1032.9, 0, 103290,
>>>> 40020, 11, 1034.1, 0, 240.65,
>>>> 40020, 7, 1034.1, 0, 0,
>>>> 40020, 2, 1034.1, 0, 103410,
>>>>
>>>>
/////////////////////////////////////////////////////////////////////
>>>> ///////////
>>>> //
>>>> // Ensemble-Stat configuration file.
>>>> //
>>>> // For additional information, see the
MET_BASE/data/config/README
>>>> file.
>>>> //
>>>>
/////////////////////////////////////////////////////////////////////
>>>> ///////////
>>>>
>>>> //
>>>> // Output model name to be written
>>>> //
>>>> model = "WRF";
>>>>
>>>>
/////////////////////////////////////////////////////////////////////
>>>> ///////////
>>>>
>>>> //
>>>> // Ensemble product fields to be processed // ens = {
>>>> ens_thresh = 1.0;
>>>> vld_thresh = 1.0;
>>>>
>>>> field = [
>>>> {
>>>> name = "TMP";
>>>> level = [ "Z2" ];
>>>> cat_thresh = [ >=300.0 ];
>>>> }
>>>> ];
>>>> }
>>>>
>>>>
/////////////////////////////////////////////////////////////////////
>>>> ///////////
>>>>
>>>> //
>>>> // Forecast and observation fields to be verified // fcst = {
>>>> message_type = [ "ADPSFC" ];
>>>> field = [
>>>> {
>>>> name = "TMP";
>>>> level = "Z2";
>>>> }
>>>> ];
>>>> }
>>>>
>>>> obs=fcst;
>>>>
>>>>
/////////////////////////////////////////////////////////////////////
>>>> ///////////
>>>>
>>>> //
>>>> // Point observation time window
>>>> //
>>>> obs_window = {
>>>> beg = -1800;
>>>> end = 1800;
>>>> }
>>>>
>>>>
/////////////////////////////////////////////////////////////////////
>>>> ///////////
>>>>
>>>> //
>>>> // Verification masking regions
>>>> //
>>>> mask = {
>>>> grid = [ "FULL" ];
>>>> poly = [];
>>>> sid = "";
>>>> };
>>>>
>>>>
/////////////////////////////////////////////////////////////////////
>>>> ///////////
>>>>
>>>> //
>>>> // Interpolation methods
>>>> //
>>>> interp = {
>>>> field = OBS;
>>>> vld_thresh = 1.0;
>>>>
>>>> type = [
>>>> {
>>>> method = BILIN;
>>>> width = 2;
>>>> }
>>>> ];
>>>> };
>>>>
>>>>
/////////////////////////////////////////////////////////////////////
>>>> ///////////
>>>>
>>>> //
>>>> // Statistical output types
>>>> //
>>>> output_flag = {
>>>> rhist = STAT;
>>>> orank = NONE;
>>>> };
>>>>
>>>>
/////////////////////////////////////////////////////////////////////
>>>> ///////////
>>>>
>>>> //
>>>> // Ensemble product output types
>>>> //
>>>> ensemble_flag = {
>>>> mean = TRUE;
>>>> stdev = TRUE;
>>>> minus = FALSE;
>>>> plus = FALSE;
>>>> min = FALSE;
>>>> max = FALSE;
>>>> range = FALSE;
>>>> vld_count = FALSE;
>>>> frequency = FALSE;
>>>> rank = TRUE;
>>>> };
>>>>
>>>>
/////////////////////////////////////////////////////////////////////
>>>> ///////////
>>>>
>>>> //
>>>> // Random number generator
>>>> //
>>>> rng = {
>>>> type = "mt19937";
>>>> seed = "1";
>>>> }
>>>>
>>>>
/////////////////////////////////////////////////////////////////////
>>>> ///////////
>>>>
>>>> duplicate_flag = NONE;
>>>> output_prefix = "";
>>>> version = "V4.0";
>>>>
>>>>
/////////////////////////////////////////////////////////////////////
>>>> ///////////
>>>> ==
>>>> Obs file: /h/data/global/WXQC/data/obs_prep/2013011412
>>>> DEBUG 1: Default Config File:
>>>> /home/qcteam/METv4.0/data/config/EnsembleStatConfig_default
>>>> DEBUG 1: User Config File:
/home/qcteam/METv4.0/cfg/ensemble_stat.cfg
>>>> DEBUG 1: Ensemble Files[10]:
>>>> DEBUG 1:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.1.tm00
>>>> DEBUG 1:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.2.tm00
>>>> DEBUG 1:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.3.tm00
>>>> DEBUG 1:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.4.tm00
>>>> DEBUG 1:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.5.tm00
>>>> DEBUG 1:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.6.tm00
>>>> DEBUG 1:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.7.tm00
>>>> DEBUG 1:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.8.tm00
>>>> DEBUG 1:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.9.tm00
>>>> DEBUG 1:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.10.tm00
>>>> DEBUG 1: Point Observation Files[1]:
>>>> DEBUG 1: /h/data/global/WXQC/data/obs_prep/2013011412
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 2: Processing ensemble field: TMP/Z2 DEBUG 3:
>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 2: Processing point observation file:
>>>> /h/data/global/WXQC/data/obs_prep/2013011412
>>>> DEBUG 2: Searching 219303 observations from 40233 header
messages.
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 2: Processing ensemble file:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.1.tm00 DEBUG 3:
>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 2: Processing ensemble file:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.2.tm00 DEBUG 3:
>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 2: Processing ensemble file:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.3.tm00 DEBUG 3:
>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 2: Processing ensemble file:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.4.tm00 DEBUG 3:
>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 2: Processing ensemble file:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.5.tm00 DEBUG 3:
>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 2: Processing ensemble file:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.6.tm00 DEBUG 3:
>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 2: Processing ensemble file:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.7.tm00 DEBUG 3:
>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 2: Processing ensemble file:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.8.tm00 DEBUG 3:
>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 2: Processing ensemble file:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.9.tm00 DEBUG 3:
>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 2: Processing ensemble file:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.10.tm00 DEBUG 3:
>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 2: Processing point verification TMP/Z2 versus TMP/Z2, for
>>>> observation type ADPSFC, over region FULL, for interpolation
method
>>>> BILIN(4), using 0 pairs.
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 1: Output file:
>>>>
/home/qcteam/METv4.0/out/ensemble_stat/ensemble_stat_20130114_120000V
>>>> .stat
>>>> DEBUG 1: Output file:
>>>>
/home/qcteam/METv4.0/out/ensemble_stat/ensemble_stat_20130114_120000V
>>>> _ens.nc
>>>>
>>>> // SIGNED //
>>>> Matthew C. Sittel
>>>> Associate Scientist
>>>> University Corporation for Atmospheric Research 16WS/WXN, Offutt
AFB,
>>>> NE
>>>> (402) 294-3473 DSN: 271-3473
>>>>
>>>>
>>>
>>>
>>>
>>>
>>>
>>
>>
>>
>
>
>
>
------------------------------------------------
Subject: Problem with Pairing NetCDF with GriB
From: matthew.sittel.ctr at offutt.af.mil
Time: Fri Jan 18 10:45:13 2013
I pushed a file via FTP to you... it's a prepbufr format file:
gdas1.t06z.prepbufr.nr. Could you run PB2NC on it on your side and
see if you agree with us in that there are no ADPSFC temperature (grib
field 11) observations in the file? For some reason all we get are
grib ID 7, height data, for ADPSFC and nothing else.
Without reading the prepbufr data, we can't be sure if there is
temperature data that gets tossed based on the config file settings,
or it simply isn't in the prepbufr file at all, which means we can't
really use it for our application. That would be an important caveat
for users.
Thanks,
Matt
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Friday, January 18, 2013 9:20 AM
To: Sittel, Matthew C CTR USAF AFWA 16 WS/WXN
Cc: Craig, Robert J Civ USAF AFWA 16 WS/WXN
Subject: Re: [rt.rap.ucar.edu #59890] Problem with Pairing NetCDF with
GriB
Matt,
Good news. The problem was in your point_stat.cfg file. Diffing it
against the default one in
METv4.0/data/config/PointStatConfig_default, I see the following:
< poly = "";
---
> poly = [];
In the masking section, it expects "poly" to be specified as an array.
Instead of specifying it as an empty array, you specified it as a
string of length 0. So it tried to read a file of that name.
Running METv4.0 with the latest set of patches at verbosity level 4,
gave the following helpful information:
DEBUG 2: Reading data for TMP/Z2.
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
"WRFPRS24_d02.1.tm00".
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB records
matching VarInfo "TMP/Z2" in GRIB file "WRFPRS24_d02.1.tm00".
DEBUG 4: parse_sid_mask() -> parsing station ID mask file "(nul)"
DEBUG 4: parse_grid_mask() -> parsing grid mask "FULL"
DEBUG 4: parse_poly_mask() -> parsing poly mask "(nul)"
ERROR :
ERROR : grd_file_type() -> file does not exist "(nul)"
ERROR :
However, running the same thing for METv4.0 as originally released
results in a seg fault. Yeah, we found it!
Looking at the list of fixes, I'm guessing it's the bug fix posted on
08/01/2012:
"Fixed a bug that caused a seg fault when a config file string
list contains only an empty string"
http://www.dtcenter.org/met/users/support/known_issues/METv4.0/index.php
So just edit your Point-Stat config file to use:
poly = [];
And then recompile METv4.0 after applying the latest set of patches.
Hopefully that'll do the trick. Just let us know if you have more
problems or questions.
Thanks,
John
On 01/18/2013 07:53 AM, matthew.sittel.ctr at offutt.af.mil via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59890 >
>
> Hi John,
>
> I've added the point_stat config file to the FTP directory with the
other files.
>
> I have not downloaded any of the patches for v4.0. I thought I had
but turns out it was for v3.1 back on May 29 '12.
>
> Bob has been looking at the PB2NC process. The obs plotting
capability has been helpful.
>
> I'll recompile and let you know if it helps.
>
> Thanks,
> Matt
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Thursday, January 17, 2013 4:42 PM
> To: Sittel, Matthew C CTR USAF AFWA 16 WS/WXN
> Cc: Craig, Robert J Civ USAF AFWA 16 WS/WXN
> Subject: RE: [rt.rap.ucar.edu #59890] Problem with Pairing NetCDF
with GriB
>
> Matt,
>
> Yeah, the ncdump thing is silly. The older version in my path
complains
> about leading numbers, but I believe that later versions allow it.
Guess
> I need to update my path!
>
> I'm not at my desk this afternoon, so am unable to debug until
tomorrow
> morning. But I'm still puzzled about the source of that seg fault.
I'd
> like to be able to replicate it to address it.
>
> Could you please do 2 things:
> - Send me the exact point-stat config file you're using:
> /home/qcteam/METv4.0/cfg/point_stat.cfg
> - Let me know the date of the latest set of patches you're using for
> METv4.0. Just take a look in /home/qcteam/METv4.0 to see if there's
a
> patch file in there with a date in the filename.
>
> Hopefully, if I run your data with your config file and the exact
same
> version of the code, I'll be able to reproduce it. If not, it's a
lot
> more difficult problem.
>
> You could always try applying the latest set of METv4.0 patches and
> recompiling to see if that helps:
>
http://www.dtcenter.org/met/users/support/known_issues/METv4.0/index.php
>
> Regarding why I got 0 matched pairs even when accounting for the
time
> offset, I think looking at the PB2NC configuration is a great idea.
In
> particular, look to see if you're filtering on the observation qc
flag.
> If you also send me your PB2NC config file and the location of the
> PREPBUFR files you're using, I'd be happy to take a look at that
too.
>
> Thanks,
> John
>
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59890 >
>>
>> Hi John,
>>
>> I put an updated obs file onto the FTP site with correct time
>> stamps-overwrote the old one. Talk about confusing. The file I
sent you
>> was later overwritten by a new file so when I looked at it, the
date
>> fields were great. Bob will check into why our process pulls data
from
>> one date and saves it as another.
>>
>> Weird, ncdump accepts numerical file names for me... were you
running this
>> in a Linux environment? Not blaming Linux, but it's always a good
place
>> to start...
>>
>> Here's the output from my point_stat run of member 1 vs. the
observation
>> file:
>>
>> =======
>> /home/qcteam/METv4.0/bin/point_stat
>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011412/GRIB_OUT_d02/WRFPRS24_d02.1.tm00
>> /h/data/global/WXQC/data/obs_prep/2013011512
>> /home/qcteam/METv4.0/cfg/point_stat.cfg -outdir
/home/qcteam/METv4.0/out
>> -v 4
>>
>> DEBUG 1: Default Config File:
>> /home/qcteam/METv4.0/data/config/PointStatConfig_default
>> DEBUG 1: User Config File: /home/qcteam/METv4.0/cfg/point_stat.cfg
>> DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created
new
>> Met2dDataFile object of type "FileType_Gb1".
>> DEBUG 4: VarInfoFactory::new_var_info() -> created new VarInfo
object of
>> type "FileType_Gb1".
>> GSL_RNG_TYPE=mt19937
>> GSL_RNG_SEED=1
>> DEBUG 1: Forecast File:
>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011412/GRIB_OUT_d02/WRFPRS24_d02.1.tm00
>> DEBUG 1: Climatology File: none
>> DEBUG 1: Observation File:
/h/data/global/WXQC/data/obs_prep/2013011512
>> DEBUG 2:
>> DEBUG 2:
>>
--------------------------------------------------------------------------------
>> DEBUG 2:
>> DEBUG 2: Reading data for TMP/Z2.
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
>> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011412/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>> matching VarInfo "TMP/Z2" in GRIB file
>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011412/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>> Memory fault
>> =======
>>
>> I've never tried plot_point_obs-I don't see reference to it in the
4.0
>> users guide.
>>
>> Okay, by typing the executable it told me the arguments I needed.
Just
>> ran it... 33208 obs for ADPSFC type, 3102 obs for gc 11, but none
for the
>> intersect of the two... weird.
>>
>> We'll keep digging and see what we can find... I wonder if PB2NC is
>> throwing out all the stuff we need based on the settings we put in
the
>> config file.
>>
>> Thanks,
>> Matt
>>
>> -----Original Message-----
>> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
>> Sent: Wednesday, January 16, 2013 4:27 PM
>> To: Sittel, Matthew C CTR USAF AFWA 16 WS/WXN
>> Cc: Craig, Robert J Civ USAF AFWA 16 WS/WXN
>> Subject: RE: [rt.rap.ucar.edu #59890] Problem with Pairing NetCDF
with
>> GriB
>>
>> Matt,
>>
>> I ran METv4.0 point_stat for the first ensemble member using the
attached
>> configuration file and verbosity level 3. I got 0 matched pairs,
but the
>> following diagnostic information is included in the output:
>>
>> DEBUG 2: Processing TMP/Z2 versus TMP/Z2, for observation type
ADPSFC,
>> over region FULL, for interpolation method BILIN(4), using 0 pairs.
>> DEBUG 3: Number of matched pairs = 0
>> DEBUG 3: Observations processed = 219303
>> DEBUG 3: Rejected: GRIB code = 190302
>> DEBUG 3: Rejected: valid time = 29001
>> DEBUG 3: Rejected: bad obs value = 0
>> DEBUG 3: Rejected: off the grid = 0
>> DEBUG 3: Rejected: level mismatch = 0
>> DEBUG 3: Rejected: message type = 0
>> DEBUG 3: Rejected: masking region = 0
>> DEBUG 3: Rejected: bad fcst value = 0
>>
>> This tells me that there's problem in the matching time window.
>>
>> Running "wgrib -V" on that forecast file tells me that this is a
24-hour
>> forecast initialized at 2013011412. So it's valid at 2013011512.
>>
>> Your observation file is named 2013011512, but when you actually
look at
>> the times it contains, they're for 2013011412:
>> ncdump -v hdr_vld nc2013011512
>>
>> (note, ncdump doesn't like file names that begin with a number, so
I stuck
>> an "nc" in front)
>>
>> I reran point-stat a second time, but first added 3 0's to the
"beg" and
>> "end" times in the "obs_window" setting. When I did that, I got
further,
>> but still no matched pairs:
>>
>> DEBUG 2: Processing TMP/Z2 versus TMP/Z2, for observation type
ADPSFC,
>> over region FULL, for interpolation method BILIN(4), using 0 pairs.
>> DEBUG 3: Number of matched pairs = 0
>> DEBUG 3: Observations processed = 219303
>> DEBUG 3: Rejected: GRIB code = 190302
>> DEBUG 3: Rejected: valid time = 0
>> DEBUG 3: Rejected: bad obs value = 0
>> DEBUG 3: Rejected: off the grid = 22047
>> DEBUG 3: Rejected: level mismatch = 6177
>> DEBUG 3: Rejected: message type = 777
>> DEBUG 3: Rejected: masking region = 0
>> DEBUG 3: Rejected: bad fcst value = 0
>>
>> So there just aren't any ADPSFC observations of 2-m temperature
that fall
>> within your d02 domain. I ran the plot_point_obs utility to verify
that
>> that's the case.
>>
>> As for why you got a seg-fault, I don't know. It's possible that
you made
>> an error in your Point-Stat config file that caused it. I can't
really
>> debug it unless I can replicate it.
>>
>> John
>>
>>>
>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59890 >
>>>
>>> Hi John,
>>>
>>> The plot thickens. When I ran point_stat on one ensemble member
vs.
>>> an observation file, MET returned a memory fault when it got to
the
>>> observation file read. I tried a second observation file and got
the
>>> same result.
>>>
>>> I looked using 'ncdump' on the NetCDF observation data output from
>>> 'pb2nc', and confirmed there are ADPSFC records, records within
the
>>> CONUS grid and in the requested time window.
>>>
>>> I used "-v 3" verbosity in the example I sent you... I think 4 is
the
>>> highest level of message? I tried rerunning this morning with "-v
6"
>>> and didn't get any additional information as to what is happening
with
>>> the observation matching, nor any messages numbered above 4.
>>>
>>> I have pushed the 10 ensemble members, observation file and config
>>> file to the directory you listed. I had to create a "met_help"
>>> directory under 'irap' as one was not there. Inside 'sittel' you
will
>>> find the files.
>>> Please note these were created in a Linux environment-not sure if
that
>>> will be an issue for you; it might depend on your OS. We were
forced
>>> to go to Linux here.
>>>
>>> Thanks,
>>> Matt
>>>
>>> -----Original Message-----
>>> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
>>> Sent: Tuesday, January 15, 2013 4:10 PM
>>> To: Sittel, Matthew C CTR USAF AFWA 16 WS/WXN
>>> Cc: Craig, Robert J Civ USAF AFWA 16 WS/WXN
>>> Subject: Re: [rt.rap.ucar.edu #59890] Problem with Pairing NetCDF
with
>>> GriB
>>>
>>> Matt,
>>>
>>> So the question is basically, why are you getting 0 matched pairs
from
>>> ensemble-stat. If this were the point-stat tool, the answer would
be
>>> easy. In point-stat, when you turn up the verbosity level, for
each
>>> verification task, a list of reasons for why observations were not
>>> used is printed out. And that's often an easy way to diagnose the
>>> problem.
>>>
>>> But that functionality has not been implemented for ensemble-stat.
>>> Though the problem you're having demonstrates the need!
>>>
>>> There are several possibilities:
>>> - the observations may not fall within the matching time window
>>> - the observations may not fall within the model's grid
>>> - perhaps there are no ADPSFC observations of TMP in your
dataset
>>>
>>> There are different steps I'd take to rule out each of these cases
-
>>> mostly making use of the plot_data_plane and plot_point_obs
utilities.
>>> But perhaps the easiest thing to do is to have you run one of your
>>> forecast files and the NetCDF output of PB2NC through point-stat
to
>>> verify TMP/Z2.
>>>
>>> Presumably, you'll get 0 matched pairs again. But when you
increase
>>> the verbosity level (-v 3, I think), you'll see a list of reasons
why
>>> the obs weren't used. Send me that list and then we'll know how
to
>>> proceed.
>>>
>>> Alternatively, you could just send me one or two of your GRIB
files,
>>> the NetCDF point observation file, and your config file, and I
could
>>> debug it here. If you choose to go that route, follow the
instructions
>>> here:
>>> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>>>
>>> Your config file settings look fine. The instructions for
specifying
>>> NetCDF config file stuff only applies to gridded NetCDF data. But
>>> your gridded data is in GRIB format. For point observations, we
chose
>>> the convention of specifying them as if they were GRIB files.
That's
>>> why we include a GRIB code for each observation value in the data.
>>>
>>> Just let me know which route you decide on. And I'm happy to help
get
>>> you up and running.
>>>
>>> Thanks,
>>> John
>>>
>>>>
>>>> Tue Jan 15 09:22:51 2013: Request 59890 was acted upon.
>>>> Transaction: Ticket created by matthew.sittel.ctr at offutt.af.mil
>>>> Queue: met_help
>>>> Subject: Problem with Pairing NetCDF with GriB
>>>> Owner: Nobody
>>>> Requestors: matthew.sittel.ctr at offutt.af.mil
>>>> Status: new
>>>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59890
>>>>>
>>>>
>>>>
>>>> Hi,
>>>>
>>>> I'm trying to get MET v4.0 to create verification rank histogram
>>>> output for AFWA's mesoscale ensemble. I'm using in-house, GriB-1
>>>> format WRF ensemble member output and verifying it with
>>>> PREPBUFR-formatted obs that we are pulling via FTP from NCEP. We
>>>> convert the PREPBUFR format to NetCDF using 'pb2nc'.
>>>>
>>>> The problem I'm having is I cannot figure out what the correct
>>>> settings are in the config file for ensemble_stat to get the
model
>>>> and observation fields to match up.
>>>>
>>>> I've attached the NetCDF 'dump' of the observation file's
contents,
>>>> the contents of my config file and the output I get when I run
>>>> ensemble_stat.
>>>> No matter what settings I pick, it always comes back with 0
pairs of
>>>> data. The output suggests the model members are read in and that
>>>> there are records in the observation file as well-both suggesting
to
>>>> me the data are read in correctly, but I'm not designating the
>>>> correct pairing of them with the config file parameters.
>>>>
>>>> The records I am trying to get are the ones where gc=11 and
hgt=0, or
>>>> simply the surface temperature reports from "ADPSFC" message
type. I
>>>> have read the user's guide and can't figure out what the right
>>>> settings are for NetCDF; it is discussed briefly but not with
enough
>>>> detail to figure this out on my own, especially the ANNN, RNNN,
etc,
>>>> and the TMP(I,...,J,*,*) wording which is unclear. Can you take
a
>>>> look and advise as to the right settings?
>>>>
>>>> Thanks,
>>>> Matt
>>>>
>>>> ==
>>>> netcdf 2013011412 {
>>>> dimensions:
>>>> mxstr = 16 ;
>>>> hdr_arr_len = 3 ;
>>>> obs_arr_len = 5 ;
>>>> nobs = UNLIMITED ; // (219303 currently)
>>>> nhdr = 40233 ;
>>>> variables:
>>>> float obs_arr(nobs, obs_arr_len) ;
>>>> obs_arr:long_name = "array of observation
values" ;
>>>> obs_arr:_fill_value = -9999.f ;
>>>> obs_arr:columns = "hdr_id gc lvl hgt ob" ;
>>>> obs_arr:hdr_id_long_name = "index of matching
header
>>>> data"
>>>> ;
>>>> obs_arr:gc_long_name = "grib code corresponding
to
>>>> the observation type" ;
>>>> obs_arr:lvl_long_name = "pressure level (hPa) or
>>>> accumulation interval (sec)" ;
>>>> obs_arr:hgt_long_name = "height in meters above
sea
>>>> level (msl)" ;
>>>> obs_arr:ob_long_name = "observation value" ;
>>>> char hdr_typ(nhdr, mxstr) ;
>>>> hdr_typ:long_name = "message type" ;
>>>> char hdr_sid(nhdr, mxstr) ;
>>>> hdr_sid:long_name = "station identification" ;
>>>> char hdr_vld(nhdr, mxstr) ;
>>>> hdr_vld:long_name = "valid time" ;
>>>> hdr_vld:units = "YYYYMMDD_HHMMSS" ;
>>>> float hdr_arr(nhdr, hdr_arr_len) ;
>>>> hdr_arr:long_name = "array of observation
station
>>>> header values" ;
>>>> hdr_arr:_fill_value = -9999.f ;
>>>> hdr_arr:columns = "lat lon elv" ;
>>>> hdr_arr:lat_long_name = "latitude" ;
>>>> hdr_arr:lat_units = "degrees_north" ;
>>>> hdr_arr:lon_long_name = "longitude" ;
>>>> hdr_arr:lon_units = "degrees_east" ;
>>>> hdr_arr:elv_long_name = "elevation" ;
>>>> hdr_arr:elv_units = "meters above sea level
(msl)" ;
>>>>
>>>> // global attributes:
>>>> :FileOrigins = "File
>>>> /h/data/global/WXQC/data//obs_prep/2013011412 generated
>>>> 20130115_003309 UTC on host dev8dlsn1 by the MET pb2nc tool" ;
>>>> :MET_version = "V4.0" ;
>>>> :MET_tool = "pb2nc" ;
>>>> data:
>>>> 40013, 7, 1016.4, 0, 0,
>>>> 40014, 7, 1012.2, 0, 0,
>>>> 40015, 7, 1024.4, 0, 0,
>>>> 40016, 11, 1030.6, 0, 236.55,
>>>> 40016, 7, 1030.6, 0, 0,
>>>> 40016, 2, 1030.6, 0, 103060,
>>>> 40017, 11, 1030.6, 0, 236.55,
>>>> 40017, 7, 1030.6, 0, 0,
>>>> 40017, 2, 1030.6, 0, 103060,
>>>> 40018, 11, 1030.9, 0, 235.95,
>>>> 40018, 7, 1030.9, 0, 0,
>>>> 40018, 2, 1030.9, 0, 103090,
>>>> 40019, 11, 1032.9, 0, 239.85,
>>>> 40019, 7, 1032.9, 0, 0,
>>>> 40019, 2, 1032.9, 0, 103290,
>>>> 40020, 11, 1034.1, 0, 240.65,
>>>> 40020, 7, 1034.1, 0, 0,
>>>> 40020, 2, 1034.1, 0, 103410,
>>>>
>>>>
/////////////////////////////////////////////////////////////////////
>>>> ///////////
>>>> //
>>>> // Ensemble-Stat configuration file.
>>>> //
>>>> // For additional information, see the
MET_BASE/data/config/README
>>>> file.
>>>> //
>>>>
/////////////////////////////////////////////////////////////////////
>>>> ///////////
>>>>
>>>> //
>>>> // Output model name to be written
>>>> //
>>>> model = "WRF";
>>>>
>>>>
/////////////////////////////////////////////////////////////////////
>>>> ///////////
>>>>
>>>> //
>>>> // Ensemble product fields to be processed // ens = {
>>>> ens_thresh = 1.0;
>>>> vld_thresh = 1.0;
>>>>
>>>> field = [
>>>> {
>>>> name = "TMP";
>>>> level = [ "Z2" ];
>>>> cat_thresh = [ >=300.0 ];
>>>> }
>>>> ];
>>>> }
>>>>
>>>>
/////////////////////////////////////////////////////////////////////
>>>> ///////////
>>>>
>>>> //
>>>> // Forecast and observation fields to be verified // fcst = {
>>>> message_type = [ "ADPSFC" ];
>>>> field = [
>>>> {
>>>> name = "TMP";
>>>> level = "Z2";
>>>> }
>>>> ];
>>>> }
>>>>
>>>> obs=fcst;
>>>>
>>>>
/////////////////////////////////////////////////////////////////////
>>>> ///////////
>>>>
>>>> //
>>>> // Point observation time window
>>>> //
>>>> obs_window = {
>>>> beg = -1800;
>>>> end = 1800;
>>>> }
>>>>
>>>>
/////////////////////////////////////////////////////////////////////
>>>> ///////////
>>>>
>>>> //
>>>> // Verification masking regions
>>>> //
>>>> mask = {
>>>> grid = [ "FULL" ];
>>>> poly = [];
>>>> sid = "";
>>>> };
>>>>
>>>>
/////////////////////////////////////////////////////////////////////
>>>> ///////////
>>>>
>>>> //
>>>> // Interpolation methods
>>>> //
>>>> interp = {
>>>> field = OBS;
>>>> vld_thresh = 1.0;
>>>>
>>>> type = [
>>>> {
>>>> method = BILIN;
>>>> width = 2;
>>>> }
>>>> ];
>>>> };
>>>>
>>>>
/////////////////////////////////////////////////////////////////////
>>>> ///////////
>>>>
>>>> //
>>>> // Statistical output types
>>>> //
>>>> output_flag = {
>>>> rhist = STAT;
>>>> orank = NONE;
>>>> };
>>>>
>>>>
/////////////////////////////////////////////////////////////////////
>>>> ///////////
>>>>
>>>> //
>>>> // Ensemble product output types
>>>> //
>>>> ensemble_flag = {
>>>> mean = TRUE;
>>>> stdev = TRUE;
>>>> minus = FALSE;
>>>> plus = FALSE;
>>>> min = FALSE;
>>>> max = FALSE;
>>>> range = FALSE;
>>>> vld_count = FALSE;
>>>> frequency = FALSE;
>>>> rank = TRUE;
>>>> };
>>>>
>>>>
/////////////////////////////////////////////////////////////////////
>>>> ///////////
>>>>
>>>> //
>>>> // Random number generator
>>>> //
>>>> rng = {
>>>> type = "mt19937";
>>>> seed = "1";
>>>> }
>>>>
>>>>
/////////////////////////////////////////////////////////////////////
>>>> ///////////
>>>>
>>>> duplicate_flag = NONE;
>>>> output_prefix = "";
>>>> version = "V4.0";
>>>>
>>>>
/////////////////////////////////////////////////////////////////////
>>>> ///////////
>>>> ==
>>>> Obs file: /h/data/global/WXQC/data/obs_prep/2013011412
>>>> DEBUG 1: Default Config File:
>>>> /home/qcteam/METv4.0/data/config/EnsembleStatConfig_default
>>>> DEBUG 1: User Config File:
/home/qcteam/METv4.0/cfg/ensemble_stat.cfg
>>>> DEBUG 1: Ensemble Files[10]:
>>>> DEBUG 1:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.1.tm00
>>>> DEBUG 1:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.2.tm00
>>>> DEBUG 1:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.3.tm00
>>>> DEBUG 1:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.4.tm00
>>>> DEBUG 1:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.5.tm00
>>>> DEBUG 1:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.6.tm00
>>>> DEBUG 1:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.7.tm00
>>>> DEBUG 1:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.8.tm00
>>>> DEBUG 1:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.9.tm00
>>>> DEBUG 1:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.10.tm00
>>>> DEBUG 1: Point Observation Files[1]:
>>>> DEBUG 1: /h/data/global/WXQC/data/obs_prep/2013011412
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 2: Processing ensemble field: TMP/Z2 DEBUG 3:
>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 2: Processing point observation file:
>>>> /h/data/global/WXQC/data/obs_prep/2013011412
>>>> DEBUG 2: Searching 219303 observations from 40233 header
messages.
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 2: Processing ensemble file:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.1.tm00 DEBUG 3:
>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 2: Processing ensemble file:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.2.tm00 DEBUG 3:
>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 2: Processing ensemble file:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.3.tm00 DEBUG 3:
>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 2: Processing ensemble file:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.4.tm00 DEBUG 3:
>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 2: Processing ensemble file:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.5.tm00 DEBUG 3:
>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 2: Processing ensemble file:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.6.tm00 DEBUG 3:
>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 2: Processing ensemble file:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.7.tm00 DEBUG 3:
>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 2: Processing ensemble file:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.8.tm00 DEBUG 3:
>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 2: Processing ensemble file:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.9.tm00 DEBUG 3:
>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 2: Processing ensemble file:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>> _d02/WRFPRS24_d02.10.tm00 DEBUG 3:
>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 2: Processing point verification TMP/Z2 versus TMP/Z2, for
>>>> observation type ADPSFC, over region FULL, for interpolation
method
>>>> BILIN(4), using 0 pairs.
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
---------------------------------------------------------------------
>>>> -----------
>>>> DEBUG 2:
>>>> DEBUG 1: Output file:
>>>>
/home/qcteam/METv4.0/out/ensemble_stat/ensemble_stat_20130114_120000V
>>>> .stat
>>>> DEBUG 1: Output file:
>>>>
/home/qcteam/METv4.0/out/ensemble_stat/ensemble_stat_20130114_120000V
>>>> _ens.nc
>>>>
>>>> // SIGNED //
>>>> Matthew C. Sittel
>>>> Associate Scientist
>>>> University Corporation for Atmospheric Research 16WS/WXN, Offutt
AFB,
>>>> NE
>>>> (402) 294-3473 DSN: 271-3473
>>>>
>>>>
>>>
>>>
>>>
>>>
>>>
>>
>>
>>
>
>
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #59890] Problem with Pairing NetCDF with GriB
From: John Halley Gotway
Time: Fri Jan 18 11:09:16 2013
Matt,
First, I'm glad you were able to get it working.
Next, if you have point internal observations that you'd like to use,
rather than trying to write a NetCDF point observation file directly,
I'd
suggest using the ASCII2NC tool. As long as you can format your point
observations in the simple ASCII format that it expects, running
ASCII2NC
will result in a NetCDF file the other MET tools can handle.
Regarding NDAS vs GDAS - NDAS PrepBufr files contain point
observations
used in the data assimilation process for NAM. GDAS contains point
observation data used in data assimilation for GFS.
You'd expect NDAS to be limited to North America, but it does actually
contain many obs over Europe and the pacific. Generally speaking
though,
use NDAS for domains over North America and GDAS for domains outside
North
America.
Here's one important issue though - and I suspect this is what you're
encountering - in GDAS, NCEP sets the quality flag for surface obs to
a
value of 9. They do that to prevent their use in the data
assimilation
process because it makes the results of the data assimilation worse.
Try cranking up the quality flag in PB2NC to 9 to see if 2-meter
temperature is retained.
John
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59890 >
>
> I pushed a file via FTP to you... it's a prepbufr format file:
> gdas1.t06z.prepbufr.nr. Could you run PB2NC on it on your side and
see if
> you agree with us in that there are no ADPSFC temperature (grib
field 11)
> observations in the file? For some reason all we get are grib ID 7,
> height data, for ADPSFC and nothing else.
>
> Without reading the prepbufr data, we can't be sure if there is
> temperature data that gets tossed based on the config file settings,
or it
> simply isn't in the prepbufr file at all, which means we can't
really use
> it for our application. That would be an important caveat for
users.
>
> Thanks,
> Matt
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Friday, January 18, 2013 9:20 AM
> To: Sittel, Matthew C CTR USAF AFWA 16 WS/WXN
> Cc: Craig, Robert J Civ USAF AFWA 16 WS/WXN
> Subject: Re: [rt.rap.ucar.edu #59890] Problem with Pairing NetCDF
with
> GriB
>
> Matt,
>
> Good news. The problem was in your point_stat.cfg file. Diffing it
> against the default one in
METv4.0/data/config/PointStatConfig_default, I
> see the following:
> < poly = "";
> ---
> > poly = [];
>
> In the masking section, it expects "poly" to be specified as an
array.
> Instead of specifying it as an empty array, you specified it as a
string
> of length 0. So it tried to read a file of that name.
> Running METv4.0 with the latest set of patches at verbosity level
4,
> gave the following helpful information:
>
> DEBUG 2: Reading data for TMP/Z2.
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
"WRFPRS24_d02.1.tm00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "TMP/Z2" in GRIB file "WRFPRS24_d02.1.tm00".
> DEBUG 4: parse_sid_mask() -> parsing station ID mask file "(nul)"
> DEBUG 4: parse_grid_mask() -> parsing grid mask "FULL"
> DEBUG 4: parse_poly_mask() -> parsing poly mask "(nul)"
> ERROR :
> ERROR : grd_file_type() -> file does not exist "(nul)"
> ERROR :
>
> However, running the same thing for METv4.0 as originally released
results
> in a seg fault. Yeah, we found it!
>
> Looking at the list of fixes, I'm guessing it's the bug fix posted
on
> 08/01/2012:
> "Fixed a bug that caused a seg fault when a config file string
list
> contains only an empty string"
>
http://www.dtcenter.org/met/users/support/known_issues/METv4.0/index.php
>
> So just edit your Point-Stat config file to use:
> poly = [];
>
> And then recompile METv4.0 after applying the latest set of patches.
>
> Hopefully that'll do the trick. Just let us know if you have more
> problems or questions.
>
> Thanks,
> John
>
>
>
> On 01/18/2013 07:53 AM, matthew.sittel.ctr at offutt.af.mil via RT
wrote:
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59890 >
>>
>> Hi John,
>>
>> I've added the point_stat config file to the FTP directory with the
>> other files.
>>
>> I have not downloaded any of the patches for v4.0. I thought I had
but
>> turns out it was for v3.1 back on May 29 '12.
>>
>> Bob has been looking at the PB2NC process. The obs plotting
capability
>> has been helpful.
>>
>> I'll recompile and let you know if it helps.
>>
>> Thanks,
>> Matt
>>
>> -----Original Message-----
>> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
>> Sent: Thursday, January 17, 2013 4:42 PM
>> To: Sittel, Matthew C CTR USAF AFWA 16 WS/WXN
>> Cc: Craig, Robert J Civ USAF AFWA 16 WS/WXN
>> Subject: RE: [rt.rap.ucar.edu #59890] Problem with Pairing NetCDF
with
>> GriB
>>
>> Matt,
>>
>> Yeah, the ncdump thing is silly. The older version in my path
complains
>> about leading numbers, but I believe that later versions allow it.
>> Guess
>> I need to update my path!
>>
>> I'm not at my desk this afternoon, so am unable to debug until
tomorrow
>> morning. But I'm still puzzled about the source of that seg fault.
I'd
>> like to be able to replicate it to address it.
>>
>> Could you please do 2 things:
>> - Send me the exact point-stat config file you're using:
>> /home/qcteam/METv4.0/cfg/point_stat.cfg
>> - Let me know the date of the latest set of patches you're using
for
>> METv4.0. Just take a look in /home/qcteam/METv4.0 to see if
there's a
>> patch file in there with a date in the filename.
>>
>> Hopefully, if I run your data with your config file and the exact
same
>> version of the code, I'll be able to reproduce it. If not, it's a
lot
>> more difficult problem.
>>
>> You could always try applying the latest set of METv4.0 patches and
>> recompiling to see if that helps:
>>
http://www.dtcenter.org/met/users/support/known_issues/METv4.0/index.php
>>
>> Regarding why I got 0 matched pairs even when accounting for the
time
>> offset, I think looking at the PB2NC configuration is a great idea.
In
>> particular, look to see if you're filtering on the observation qc
flag.
>> If you also send me your PB2NC config file and the location of the
>> PREPBUFR files you're using, I'd be happy to take a look at that
too.
>>
>> Thanks,
>> John
>>
>>>
>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59890 >
>>>
>>> Hi John,
>>>
>>> I put an updated obs file onto the FTP site with correct time
>>> stamps-overwrote the old one. Talk about confusing. The file I
sent
>>> you
>>> was later overwritten by a new file so when I looked at it, the
date
>>> fields were great. Bob will check into why our process pulls data
from
>>> one date and saves it as another.
>>>
>>> Weird, ncdump accepts numerical file names for me... were you
running
>>> this
>>> in a Linux environment? Not blaming Linux, but it's always a good
>>> place
>>> to start...
>>>
>>> Here's the output from my point_stat run of member 1 vs. the
>>> observation
>>> file:
>>>
>>> =======
>>> /home/qcteam/METv4.0/bin/point_stat
>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011412/GRIB_OUT_d02/WRFPRS24_d02.1.tm00
>>> /h/data/global/WXQC/data/obs_prep/2013011512
>>> /home/qcteam/METv4.0/cfg/point_stat.cfg -outdir
>>> /home/qcteam/METv4.0/out
>>> -v 4
>>>
>>> DEBUG 1: Default Config File:
>>> /home/qcteam/METv4.0/data/config/PointStatConfig_default
>>> DEBUG 1: User Config File: /home/qcteam/METv4.0/cfg/point_stat.cfg
>>> DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created
new
>>> Met2dDataFile object of type "FileType_Gb1".
>>> DEBUG 4: VarInfoFactory::new_var_info() -> created new VarInfo
object
>>> of
>>> type "FileType_Gb1".
>>> GSL_RNG_TYPE=mt19937
>>> GSL_RNG_SEED=1
>>> DEBUG 1: Forecast File:
>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011412/GRIB_OUT_d02/WRFPRS24_d02.1.tm00
>>> DEBUG 1: Climatology File: none
>>> DEBUG 1: Observation File:
/h/data/global/WXQC/data/obs_prep/2013011512
>>> DEBUG 2:
>>> DEBUG 2:
>>>
--------------------------------------------------------------------------------
>>> DEBUG 2:
>>> DEBUG 2: Reading data for TMP/Z2.
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
>>> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011412/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>> matching VarInfo "TMP/Z2" in GRIB file
>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011412/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>>> Memory fault
>>> =======
>>>
>>> I've never tried plot_point_obs-I don't see reference to it in the
4.0
>>> users guide.
>>>
>>> Okay, by typing the executable it told me the arguments I needed.
Just
>>> ran it... 33208 obs for ADPSFC type, 3102 obs for gc 11, but none
for
>>> the
>>> intersect of the two... weird.
>>>
>>> We'll keep digging and see what we can find... I wonder if PB2NC
is
>>> throwing out all the stuff we need based on the settings we put in
the
>>> config file.
>>>
>>> Thanks,
>>> Matt
>>>
>>> -----Original Message-----
>>> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
>>> Sent: Wednesday, January 16, 2013 4:27 PM
>>> To: Sittel, Matthew C CTR USAF AFWA 16 WS/WXN
>>> Cc: Craig, Robert J Civ USAF AFWA 16 WS/WXN
>>> Subject: RE: [rt.rap.ucar.edu #59890] Problem with Pairing NetCDF
with
>>> GriB
>>>
>>> Matt,
>>>
>>> I ran METv4.0 point_stat for the first ensemble member using the
>>> attached
>>> configuration file and verbosity level 3. I got 0 matched pairs,
but
>>> the
>>> following diagnostic information is included in the output:
>>>
>>> DEBUG 2: Processing TMP/Z2 versus TMP/Z2, for observation type
ADPSFC,
>>> over region FULL, for interpolation method BILIN(4), using 0
pairs.
>>> DEBUG 3: Number of matched pairs = 0
>>> DEBUG 3: Observations processed = 219303
>>> DEBUG 3: Rejected: GRIB code = 190302
>>> DEBUG 3: Rejected: valid time = 29001
>>> DEBUG 3: Rejected: bad obs value = 0
>>> DEBUG 3: Rejected: off the grid = 0
>>> DEBUG 3: Rejected: level mismatch = 0
>>> DEBUG 3: Rejected: message type = 0
>>> DEBUG 3: Rejected: masking region = 0
>>> DEBUG 3: Rejected: bad fcst value = 0
>>>
>>> This tells me that there's problem in the matching time window.
>>>
>>> Running "wgrib -V" on that forecast file tells me that this is a
>>> 24-hour
>>> forecast initialized at 2013011412. So it's valid at 2013011512.
>>>
>>> Your observation file is named 2013011512, but when you actually
look
>>> at
>>> the times it contains, they're for 2013011412:
>>> ncdump -v hdr_vld nc2013011512
>>>
>>> (note, ncdump doesn't like file names that begin with a number, so
I
>>> stuck
>>> an "nc" in front)
>>>
>>> I reran point-stat a second time, but first added 3 0's to the
"beg"
>>> and
>>> "end" times in the "obs_window" setting. When I did that, I got
>>> further,
>>> but still no matched pairs:
>>>
>>> DEBUG 2: Processing TMP/Z2 versus TMP/Z2, for observation type
ADPSFC,
>>> over region FULL, for interpolation method BILIN(4), using 0
pairs.
>>> DEBUG 3: Number of matched pairs = 0
>>> DEBUG 3: Observations processed = 219303
>>> DEBUG 3: Rejected: GRIB code = 190302
>>> DEBUG 3: Rejected: valid time = 0
>>> DEBUG 3: Rejected: bad obs value = 0
>>> DEBUG 3: Rejected: off the grid = 22047
>>> DEBUG 3: Rejected: level mismatch = 6177
>>> DEBUG 3: Rejected: message type = 777
>>> DEBUG 3: Rejected: masking region = 0
>>> DEBUG 3: Rejected: bad fcst value = 0
>>>
>>> So there just aren't any ADPSFC observations of 2-m temperature
that
>>> fall
>>> within your d02 domain. I ran the plot_point_obs utility to
verify
>>> that
>>> that's the case.
>>>
>>> As for why you got a seg-fault, I don't know. It's possible that
you
>>> made
>>> an error in your Point-Stat config file that caused it. I can't
really
>>> debug it unless I can replicate it.
>>>
>>> John
>>>
>>>>
>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59890 >
>>>>
>>>> Hi John,
>>>>
>>>> The plot thickens. When I ran point_stat on one ensemble member
vs.
>>>> an observation file, MET returned a memory fault when it got to
the
>>>> observation file read. I tried a second observation file and got
the
>>>> same result.
>>>>
>>>> I looked using 'ncdump' on the NetCDF observation data output
from
>>>> 'pb2nc', and confirmed there are ADPSFC records, records within
the
>>>> CONUS grid and in the requested time window.
>>>>
>>>> I used "-v 3" verbosity in the example I sent you... I think 4 is
the
>>>> highest level of message? I tried rerunning this morning with "-
v 6"
>>>> and didn't get any additional information as to what is happening
with
>>>> the observation matching, nor any messages numbered above 4.
>>>>
>>>> I have pushed the 10 ensemble members, observation file and
config
>>>> file to the directory you listed. I had to create a "met_help"
>>>> directory under 'irap' as one was not there. Inside 'sittel' you
will
>>>> find the files.
>>>> Please note these were created in a Linux environment-not sure if
that
>>>> will be an issue for you; it might depend on your OS. We were
forced
>>>> to go to Linux here.
>>>>
>>>> Thanks,
>>>> Matt
>>>>
>>>> -----Original Message-----
>>>> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
>>>> Sent: Tuesday, January 15, 2013 4:10 PM
>>>> To: Sittel, Matthew C CTR USAF AFWA 16 WS/WXN
>>>> Cc: Craig, Robert J Civ USAF AFWA 16 WS/WXN
>>>> Subject: Re: [rt.rap.ucar.edu #59890] Problem with Pairing NetCDF
with
>>>> GriB
>>>>
>>>> Matt,
>>>>
>>>> So the question is basically, why are you getting 0 matched pairs
from
>>>> ensemble-stat. If this were the point-stat tool, the answer
would be
>>>> easy. In point-stat, when you turn up the verbosity level, for
each
>>>> verification task, a list of reasons for why observations were
not
>>>> used is printed out. And that's often an easy way to diagnose
the
>>>> problem.
>>>>
>>>> But that functionality has not been implemented for ensemble-
stat.
>>>> Though the problem you're having demonstrates the need!
>>>>
>>>> There are several possibilities:
>>>> - the observations may not fall within the matching time window
>>>> - the observations may not fall within the model's grid
>>>> - perhaps there are no ADPSFC observations of TMP in your
dataset
>>>>
>>>> There are different steps I'd take to rule out each of these
cases -
>>>> mostly making use of the plot_data_plane and plot_point_obs
utilities.
>>>> But perhaps the easiest thing to do is to have you run one of
your
>>>> forecast files and the NetCDF output of PB2NC through point-stat
to
>>>> verify TMP/Z2.
>>>>
>>>> Presumably, you'll get 0 matched pairs again. But when you
increase
>>>> the verbosity level (-v 3, I think), you'll see a list of reasons
why
>>>> the obs weren't used. Send me that list and then we'll know how
to
>>>> proceed.
>>>>
>>>> Alternatively, you could just send me one or two of your GRIB
files,
>>>> the NetCDF point observation file, and your config file, and I
could
>>>> debug it here. If you choose to go that route, follow the
>>>> instructions
>>>> here:
>>>> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>>>>
>>>> Your config file settings look fine. The instructions for
specifying
>>>> NetCDF config file stuff only applies to gridded NetCDF data.
But
>>>> your gridded data is in GRIB format. For point observations, we
chose
>>>> the convention of specifying them as if they were GRIB files.
That's
>>>> why we include a GRIB code for each observation value in the
data.
>>>>
>>>> Just let me know which route you decide on. And I'm happy to
help get
>>>> you up and running.
>>>>
>>>> Thanks,
>>>> John
>>>>
>>>>>
>>>>> Tue Jan 15 09:22:51 2013: Request 59890 was acted upon.
>>>>> Transaction: Ticket created by matthew.sittel.ctr at offutt.af.mil
>>>>> Queue: met_help
>>>>> Subject: Problem with Pairing NetCDF with GriB
>>>>> Owner: Nobody
>>>>> Requestors: matthew.sittel.ctr at offutt.af.mil
>>>>> Status: new
>>>>> Ticket <URL:
>>>>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59890
>>>>>>
>>>>>
>>>>>
>>>>> Hi,
>>>>>
>>>>> I'm trying to get MET v4.0 to create verification rank histogram
>>>>> output for AFWA's mesoscale ensemble. I'm using in-house, GriB-
1
>>>>> format WRF ensemble member output and verifying it with
>>>>> PREPBUFR-formatted obs that we are pulling via FTP from NCEP.
We
>>>>> convert the PREPBUFR format to NetCDF using 'pb2nc'.
>>>>>
>>>>> The problem I'm having is I cannot figure out what the correct
>>>>> settings are in the config file for ensemble_stat to get the
model
>>>>> and observation fields to match up.
>>>>>
>>>>> I've attached the NetCDF 'dump' of the observation file's
contents,
>>>>> the contents of my config file and the output I get when I run
>>>>> ensemble_stat.
>>>>> No matter what settings I pick, it always comes back with 0
pairs
>>>>> of
>>>>> data. The output suggests the model members are read in and
that
>>>>> there are records in the observation file as well-both
suggesting to
>>>>> me the data are read in correctly, but I'm not designating the
>>>>> correct pairing of them with the config file parameters.
>>>>>
>>>>> The records I am trying to get are the ones where gc=11 and
hgt=0, or
>>>>> simply the surface temperature reports from "ADPSFC" message
type. I
>>>>> have read the user's guide and can't figure out what the right
>>>>> settings are for NetCDF; it is discussed briefly but not with
enough
>>>>> detail to figure this out on my own, especially the ANNN, RNNN,
etc,
>>>>> and the TMP(I,...,J,*,*) wording which is unclear. Can you take
a
>>>>> look and advise as to the right settings?
>>>>>
>>>>> Thanks,
>>>>> Matt
>>>>>
>>>>> ==
>>>>> netcdf 2013011412 {
>>>>> dimensions:
>>>>> mxstr = 16 ;
>>>>> hdr_arr_len = 3 ;
>>>>> obs_arr_len = 5 ;
>>>>> nobs = UNLIMITED ; // (219303 currently)
>>>>> nhdr = 40233 ;
>>>>> variables:
>>>>> float obs_arr(nobs, obs_arr_len) ;
>>>>> obs_arr:long_name = "array of observation
values" ;
>>>>> obs_arr:_fill_value = -9999.f ;
>>>>> obs_arr:columns = "hdr_id gc lvl hgt ob" ;
>>>>> obs_arr:hdr_id_long_name = "index of matching
header
>>>>> data"
>>>>> ;
>>>>> obs_arr:gc_long_name = "grib code corresponding
to
>>>>> the observation type" ;
>>>>> obs_arr:lvl_long_name = "pressure level (hPa)
or
>>>>> accumulation interval (sec)" ;
>>>>> obs_arr:hgt_long_name = "height in meters above
sea
>>>>> level (msl)" ;
>>>>> obs_arr:ob_long_name = "observation value" ;
>>>>> char hdr_typ(nhdr, mxstr) ;
>>>>> hdr_typ:long_name = "message type" ;
>>>>> char hdr_sid(nhdr, mxstr) ;
>>>>> hdr_sid:long_name = "station identification" ;
>>>>> char hdr_vld(nhdr, mxstr) ;
>>>>> hdr_vld:long_name = "valid time" ;
>>>>> hdr_vld:units = "YYYYMMDD_HHMMSS" ;
>>>>> float hdr_arr(nhdr, hdr_arr_len) ;
>>>>> hdr_arr:long_name = "array of observation
station
>>>>> header values" ;
>>>>> hdr_arr:_fill_value = -9999.f ;
>>>>> hdr_arr:columns = "lat lon elv" ;
>>>>> hdr_arr:lat_long_name = "latitude" ;
>>>>> hdr_arr:lat_units = "degrees_north" ;
>>>>> hdr_arr:lon_long_name = "longitude" ;
>>>>> hdr_arr:lon_units = "degrees_east" ;
>>>>> hdr_arr:elv_long_name = "elevation" ;
>>>>> hdr_arr:elv_units = "meters above sea level
(msl)" ;
>>>>>
>>>>> // global attributes:
>>>>> :FileOrigins = "File
>>>>> /h/data/global/WXQC/data//obs_prep/2013011412 generated
>>>>> 20130115_003309 UTC on host dev8dlsn1 by the MET pb2nc tool" ;
>>>>> :MET_version = "V4.0" ;
>>>>> :MET_tool = "pb2nc" ;
>>>>> data:
>>>>> 40013, 7, 1016.4, 0, 0,
>>>>> 40014, 7, 1012.2, 0, 0,
>>>>> 40015, 7, 1024.4, 0, 0,
>>>>> 40016, 11, 1030.6, 0, 236.55,
>>>>> 40016, 7, 1030.6, 0, 0,
>>>>> 40016, 2, 1030.6, 0, 103060,
>>>>> 40017, 11, 1030.6, 0, 236.55,
>>>>> 40017, 7, 1030.6, 0, 0,
>>>>> 40017, 2, 1030.6, 0, 103060,
>>>>> 40018, 11, 1030.9, 0, 235.95,
>>>>> 40018, 7, 1030.9, 0, 0,
>>>>> 40018, 2, 1030.9, 0, 103090,
>>>>> 40019, 11, 1032.9, 0, 239.85,
>>>>> 40019, 7, 1032.9, 0, 0,
>>>>> 40019, 2, 1032.9, 0, 103290,
>>>>> 40020, 11, 1034.1, 0, 240.65,
>>>>> 40020, 7, 1034.1, 0, 0,
>>>>> 40020, 2, 1034.1, 0, 103410,
>>>>>
>>>>>
/////////////////////////////////////////////////////////////////////
>>>>> ///////////
>>>>> //
>>>>> // Ensemble-Stat configuration file.
>>>>> //
>>>>> // For additional information, see the
MET_BASE/data/config/README
>>>>> file.
>>>>> //
>>>>>
/////////////////////////////////////////////////////////////////////
>>>>> ///////////
>>>>>
>>>>> //
>>>>> // Output model name to be written
>>>>> //
>>>>> model = "WRF";
>>>>>
>>>>>
/////////////////////////////////////////////////////////////////////
>>>>> ///////////
>>>>>
>>>>> //
>>>>> // Ensemble product fields to be processed // ens = {
>>>>> ens_thresh = 1.0;
>>>>> vld_thresh = 1.0;
>>>>>
>>>>> field = [
>>>>> {
>>>>> name = "TMP";
>>>>> level = [ "Z2" ];
>>>>> cat_thresh = [ >=300.0 ];
>>>>> }
>>>>> ];
>>>>> }
>>>>>
>>>>>
/////////////////////////////////////////////////////////////////////
>>>>> ///////////
>>>>>
>>>>> //
>>>>> // Forecast and observation fields to be verified // fcst = {
>>>>> message_type = [ "ADPSFC" ];
>>>>> field = [
>>>>> {
>>>>> name = "TMP";
>>>>> level = "Z2";
>>>>> }
>>>>> ];
>>>>> }
>>>>>
>>>>> obs=fcst;
>>>>>
>>>>>
/////////////////////////////////////////////////////////////////////
>>>>> ///////////
>>>>>
>>>>> //
>>>>> // Point observation time window
>>>>> //
>>>>> obs_window = {
>>>>> beg = -1800;
>>>>> end = 1800;
>>>>> }
>>>>>
>>>>>
/////////////////////////////////////////////////////////////////////
>>>>> ///////////
>>>>>
>>>>> //
>>>>> // Verification masking regions
>>>>> //
>>>>> mask = {
>>>>> grid = [ "FULL" ];
>>>>> poly = [];
>>>>> sid = "";
>>>>> };
>>>>>
>>>>>
/////////////////////////////////////////////////////////////////////
>>>>> ///////////
>>>>>
>>>>> //
>>>>> // Interpolation methods
>>>>> //
>>>>> interp = {
>>>>> field = OBS;
>>>>> vld_thresh = 1.0;
>>>>>
>>>>> type = [
>>>>> {
>>>>> method = BILIN;
>>>>> width = 2;
>>>>> }
>>>>> ];
>>>>> };
>>>>>
>>>>>
/////////////////////////////////////////////////////////////////////
>>>>> ///////////
>>>>>
>>>>> //
>>>>> // Statistical output types
>>>>> //
>>>>> output_flag = {
>>>>> rhist = STAT;
>>>>> orank = NONE;
>>>>> };
>>>>>
>>>>>
/////////////////////////////////////////////////////////////////////
>>>>> ///////////
>>>>>
>>>>> //
>>>>> // Ensemble product output types
>>>>> //
>>>>> ensemble_flag = {
>>>>> mean = TRUE;
>>>>> stdev = TRUE;
>>>>> minus = FALSE;
>>>>> plus = FALSE;
>>>>> min = FALSE;
>>>>> max = FALSE;
>>>>> range = FALSE;
>>>>> vld_count = FALSE;
>>>>> frequency = FALSE;
>>>>> rank = TRUE;
>>>>> };
>>>>>
>>>>>
/////////////////////////////////////////////////////////////////////
>>>>> ///////////
>>>>>
>>>>> //
>>>>> // Random number generator
>>>>> //
>>>>> rng = {
>>>>> type = "mt19937";
>>>>> seed = "1";
>>>>> }
>>>>>
>>>>>
/////////////////////////////////////////////////////////////////////
>>>>> ///////////
>>>>>
>>>>> duplicate_flag = NONE;
>>>>> output_prefix = "";
>>>>> version = "V4.0";
>>>>>
>>>>>
/////////////////////////////////////////////////////////////////////
>>>>> ///////////
>>>>> ==
>>>>> Obs file: /h/data/global/WXQC/data/obs_prep/2013011412
>>>>> DEBUG 1: Default Config File:
>>>>> /home/qcteam/METv4.0/data/config/EnsembleStatConfig_default
>>>>> DEBUG 1: User Config File:
/home/qcteam/METv4.0/cfg/ensemble_stat.cfg
>>>>> DEBUG 1: Ensemble Files[10]:
>>>>> DEBUG 1:
>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>> _d02/WRFPRS24_d02.1.tm00
>>>>> DEBUG 1:
>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>> _d02/WRFPRS24_d02.2.tm00
>>>>> DEBUG 1:
>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>> _d02/WRFPRS24_d02.3.tm00
>>>>> DEBUG 1:
>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>> _d02/WRFPRS24_d02.4.tm00
>>>>> DEBUG 1:
>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>> _d02/WRFPRS24_d02.5.tm00
>>>>> DEBUG 1:
>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>> _d02/WRFPRS24_d02.6.tm00
>>>>> DEBUG 1:
>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>> _d02/WRFPRS24_d02.7.tm00
>>>>> DEBUG 1:
>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>> _d02/WRFPRS24_d02.8.tm00
>>>>> DEBUG 1:
>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>> _d02/WRFPRS24_d02.9.tm00
>>>>> DEBUG 1:
>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>> _d02/WRFPRS24_d02.10.tm00
>>>>> DEBUG 1: Point Observation Files[1]:
>>>>> DEBUG 1: /h/data/global/WXQC/data/obs_prep/2013011412
>>>>> DEBUG 2:
>>>>> DEBUG 2:
>>>>>
---------------------------------------------------------------------
>>>>> -----------
>>>>> DEBUG 2:
>>>>> DEBUG 2: Processing ensemble field: TMP/Z2 DEBUG 3:
>>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
>>>>> DEBUG 2:
>>>>> DEBUG 2:
>>>>>
---------------------------------------------------------------------
>>>>> -----------
>>>>> DEBUG 2:
>>>>> DEBUG 2: Processing point observation file:
>>>>> /h/data/global/WXQC/data/obs_prep/2013011412
>>>>> DEBUG 2: Searching 219303 observations from 40233 header
messages.
>>>>> DEBUG 2:
>>>>> DEBUG 2:
>>>>>
---------------------------------------------------------------------
>>>>> -----------
>>>>> DEBUG 2:
>>>>> DEBUG 2: Processing ensemble file:
>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>> _d02/WRFPRS24_d02.1.tm00 DEBUG 3:
>>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>>> DEBUG 2:
>>>>> DEBUG 2:
>>>>>
---------------------------------------------------------------------
>>>>> -----------
>>>>> DEBUG 2:
>>>>> DEBUG 2: Processing ensemble file:
>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>> _d02/WRFPRS24_d02.2.tm00 DEBUG 3:
>>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
>>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>>> DEBUG 2:
>>>>> DEBUG 2:
>>>>>
---------------------------------------------------------------------
>>>>> -----------
>>>>> DEBUG 2:
>>>>> DEBUG 2: Processing ensemble file:
>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>> _d02/WRFPRS24_d02.3.tm00 DEBUG 3:
>>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
>>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>>> DEBUG 2:
>>>>> DEBUG 2:
>>>>>
---------------------------------------------------------------------
>>>>> -----------
>>>>> DEBUG 2:
>>>>> DEBUG 2: Processing ensemble file:
>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>> _d02/WRFPRS24_d02.4.tm00 DEBUG 3:
>>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
>>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>>> DEBUG 2:
>>>>> DEBUG 2:
>>>>>
---------------------------------------------------------------------
>>>>> -----------
>>>>> DEBUG 2:
>>>>> DEBUG 2: Processing ensemble file:
>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>> _d02/WRFPRS24_d02.5.tm00 DEBUG 3:
>>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
>>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>>> DEBUG 2:
>>>>> DEBUG 2:
>>>>>
---------------------------------------------------------------------
>>>>> -----------
>>>>> DEBUG 2:
>>>>> DEBUG 2: Processing ensemble file:
>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>> _d02/WRFPRS24_d02.6.tm00 DEBUG 3:
>>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
>>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>>> DEBUG 2:
>>>>> DEBUG 2:
>>>>>
---------------------------------------------------------------------
>>>>> -----------
>>>>> DEBUG 2:
>>>>> DEBUG 2: Processing ensemble file:
>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>> _d02/WRFPRS24_d02.7.tm00 DEBUG 3:
>>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
>>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>>> DEBUG 2:
>>>>> DEBUG 2:
>>>>>
---------------------------------------------------------------------
>>>>> -----------
>>>>> DEBUG 2:
>>>>> DEBUG 2: Processing ensemble file:
>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>> _d02/WRFPRS24_d02.8.tm00 DEBUG 3:
>>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
>>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>>> DEBUG 2:
>>>>> DEBUG 2:
>>>>>
---------------------------------------------------------------------
>>>>> -----------
>>>>> DEBUG 2:
>>>>> DEBUG 2: Processing ensemble file:
>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>> _d02/WRFPRS24_d02.9.tm00 DEBUG 3:
>>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
>>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>>> DEBUG 2:
>>>>> DEBUG 2:
>>>>>
---------------------------------------------------------------------
>>>>> -----------
>>>>> DEBUG 2:
>>>>> DEBUG 2: Processing ensemble file:
>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>> _d02/WRFPRS24_d02.10.tm00 DEBUG 3:
>>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
>>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>>> DEBUG 2:
>>>>> DEBUG 2:
>>>>>
---------------------------------------------------------------------
>>>>> -----------
>>>>> DEBUG 2:
>>>>> DEBUG 2: Processing point verification TMP/Z2 versus TMP/Z2, for
>>>>> observation type ADPSFC, over region FULL, for interpolation
method
>>>>> BILIN(4), using 0 pairs.
>>>>> DEBUG 2:
>>>>> DEBUG 2:
>>>>>
---------------------------------------------------------------------
>>>>> -----------
>>>>> DEBUG 2:
>>>>> DEBUG 1: Output file:
>>>>>
/home/qcteam/METv4.0/out/ensemble_stat/ensemble_stat_20130114_120000V
>>>>> .stat
>>>>> DEBUG 1: Output file:
>>>>>
/home/qcteam/METv4.0/out/ensemble_stat/ensemble_stat_20130114_120000V
>>>>> _ens.nc
>>>>>
>>>>> // SIGNED //
>>>>> Matthew C. Sittel
>>>>> Associate Scientist
>>>>> University Corporation for Atmospheric Research 16WS/WXN, Offutt
AFB,
>>>>> NE
>>>>> (402) 294-3473 DSN: 271-3473
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>
>>
>>
>>
>
>
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #59890] Problem with Pairing NetCDF with GriB
From: John Halley Gotway
Time: Fri Jan 18 12:03:19 2013
Matt,
One other item to note. In METv4.0, we are *not* passing the quality
flag value through PB2NC for each observation.
However, one enhancement in METv4.1, that's currently under
development, is to pass the quality flag value through PB2NC. Then in
Point-Stat and Ensemble-Stat, you can specify in the config file
which quality flags you'd like to use. In the case of this GDAS data,
you may choose to verify your forecast against observations with
quality values of 1, 2, and 9, skipping everything between 2 and 9.
John
On 01/18/2013 11:09 AM, John Halley Gotway wrote:
> Matt,
>
> First, I'm glad you were able to get it working.
>
> Next, if you have point internal observations that you'd like to
use,
> rather than trying to write a NetCDF point observation file
directly, I'd
> suggest using the ASCII2NC tool. As long as you can format your
point
> observations in the simple ASCII format that it expects, running
ASCII2NC
> will result in a NetCDF file the other MET tools can handle.
>
> Regarding NDAS vs GDAS - NDAS PrepBufr files contain point
observations
> used in the data assimilation process for NAM. GDAS contains point
> observation data used in data assimilation for GFS.
>
> You'd expect NDAS to be limited to North America, but it does
actually
> contain many obs over Europe and the pacific. Generally speaking
though,
> use NDAS for domains over North America and GDAS for domains outside
North
> America.
>
> Here's one important issue though - and I suspect this is what
you're
> encountering - in GDAS, NCEP sets the quality flag for surface obs
to a
> value of 9. They do that to prevent their use in the data
assimilation
> process because it makes the results of the data assimilation worse.
>
> Try cranking up the quality flag in PB2NC to 9 to see if 2-meter
> temperature is retained.
>
> John
>
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59890 >
>>
>> I pushed a file via FTP to you... it's a prepbufr format file:
>> gdas1.t06z.prepbufr.nr. Could you run PB2NC on it on your side and
see if
>> you agree with us in that there are no ADPSFC temperature (grib
field 11)
>> observations in the file? For some reason all we get are grib ID
7,
>> height data, for ADPSFC and nothing else.
>>
>> Without reading the prepbufr data, we can't be sure if there is
>> temperature data that gets tossed based on the config file
settings, or it
>> simply isn't in the prepbufr file at all, which means we can't
really use
>> it for our application. That would be an important caveat for
users.
>>
>> Thanks,
>> Matt
>>
>> -----Original Message-----
>> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
>> Sent: Friday, January 18, 2013 9:20 AM
>> To: Sittel, Matthew C CTR USAF AFWA 16 WS/WXN
>> Cc: Craig, Robert J Civ USAF AFWA 16 WS/WXN
>> Subject: Re: [rt.rap.ucar.edu #59890] Problem with Pairing NetCDF
with
>> GriB
>>
>> Matt,
>>
>> Good news. The problem was in your point_stat.cfg file. Diffing
it
>> against the default one in
METv4.0/data/config/PointStatConfig_default, I
>> see the following:
>> < poly = "";
>> ---
>> > poly = [];
>>
>> In the masking section, it expects "poly" to be specified as an
array.
>> Instead of specifying it as an empty array, you specified it as a
string
>> of length 0. So it tried to read a file of that name.
>> Running METv4.0 with the latest set of patches at verbosity
level 4,
>> gave the following helpful information:
>>
>> DEBUG 2: Reading data for TMP/Z2.
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
>> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
"WRFPRS24_d02.1.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>> matching VarInfo "TMP/Z2" in GRIB file "WRFPRS24_d02.1.tm00".
>> DEBUG 4: parse_sid_mask() -> parsing station ID mask file "(nul)"
>> DEBUG 4: parse_grid_mask() -> parsing grid mask "FULL"
>> DEBUG 4: parse_poly_mask() -> parsing poly mask "(nul)"
>> ERROR :
>> ERROR : grd_file_type() -> file does not exist "(nul)"
>> ERROR :
>>
>> However, running the same thing for METv4.0 as originally released
results
>> in a seg fault. Yeah, we found it!
>>
>> Looking at the list of fixes, I'm guessing it's the bug fix posted
on
>> 08/01/2012:
>> "Fixed a bug that caused a seg fault when a config file string
list
>> contains only an empty string"
>>
http://www.dtcenter.org/met/users/support/known_issues/METv4.0/index.php
>>
>> So just edit your Point-Stat config file to use:
>> poly = [];
>>
>> And then recompile METv4.0 after applying the latest set of
patches.
>>
>> Hopefully that'll do the trick. Just let us know if you have more
>> problems or questions.
>>
>> Thanks,
>> John
>>
>>
>>
>> On 01/18/2013 07:53 AM, matthew.sittel.ctr at offutt.af.mil via RT
wrote:
>>>
>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59890 >
>>>
>>> Hi John,
>>>
>>> I've added the point_stat config file to the FTP directory with
the
>>> other files.
>>>
>>> I have not downloaded any of the patches for v4.0. I thought I
had but
>>> turns out it was for v3.1 back on May 29 '12.
>>>
>>> Bob has been looking at the PB2NC process. The obs plotting
capability
>>> has been helpful.
>>>
>>> I'll recompile and let you know if it helps.
>>>
>>> Thanks,
>>> Matt
>>>
>>> -----Original Message-----
>>> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
>>> Sent: Thursday, January 17, 2013 4:42 PM
>>> To: Sittel, Matthew C CTR USAF AFWA 16 WS/WXN
>>> Cc: Craig, Robert J Civ USAF AFWA 16 WS/WXN
>>> Subject: RE: [rt.rap.ucar.edu #59890] Problem with Pairing NetCDF
with
>>> GriB
>>>
>>> Matt,
>>>
>>> Yeah, the ncdump thing is silly. The older version in my path
complains
>>> about leading numbers, but I believe that later versions allow it.
>>> Guess
>>> I need to update my path!
>>>
>>> I'm not at my desk this afternoon, so am unable to debug until
tomorrow
>>> morning. But I'm still puzzled about the source of that seg
fault. I'd
>>> like to be able to replicate it to address it.
>>>
>>> Could you please do 2 things:
>>> - Send me the exact point-stat config file you're using:
>>> /home/qcteam/METv4.0/cfg/point_stat.cfg
>>> - Let me know the date of the latest set of patches you're using
for
>>> METv4.0. Just take a look in /home/qcteam/METv4.0 to see if
there's a
>>> patch file in there with a date in the filename.
>>>
>>> Hopefully, if I run your data with your config file and the exact
same
>>> version of the code, I'll be able to reproduce it. If not, it's a
lot
>>> more difficult problem.
>>>
>>> You could always try applying the latest set of METv4.0 patches
and
>>> recompiling to see if that helps:
>>>
http://www.dtcenter.org/met/users/support/known_issues/METv4.0/index.php
>>>
>>> Regarding why I got 0 matched pairs even when accounting for the
time
>>> offset, I think looking at the PB2NC configuration is a great
idea. In
>>> particular, look to see if you're filtering on the observation qc
flag.
>>> If you also send me your PB2NC config file and the location of the
>>> PREPBUFR files you're using, I'd be happy to take a look at that
too.
>>>
>>> Thanks,
>>> John
>>>
>>>>
>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59890 >
>>>>
>>>> Hi John,
>>>>
>>>> I put an updated obs file onto the FTP site with correct time
>>>> stamps-overwrote the old one. Talk about confusing. The file I
sent
>>>> you
>>>> was later overwritten by a new file so when I looked at it, the
date
>>>> fields were great. Bob will check into why our process pulls
data from
>>>> one date and saves it as another.
>>>>
>>>> Weird, ncdump accepts numerical file names for me... were you
running
>>>> this
>>>> in a Linux environment? Not blaming Linux, but it's always a
good
>>>> place
>>>> to start...
>>>>
>>>> Here's the output from my point_stat run of member 1 vs. the
>>>> observation
>>>> file:
>>>>
>>>> =======
>>>> /home/qcteam/METv4.0/bin/point_stat
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011412/GRIB_OUT_d02/WRFPRS24_d02.1.tm00
>>>> /h/data/global/WXQC/data/obs_prep/2013011512
>>>> /home/qcteam/METv4.0/cfg/point_stat.cfg -outdir
>>>> /home/qcteam/METv4.0/out
>>>> -v 4
>>>>
>>>> DEBUG 1: Default Config File:
>>>> /home/qcteam/METv4.0/data/config/PointStatConfig_default
>>>> DEBUG 1: User Config File:
/home/qcteam/METv4.0/cfg/point_stat.cfg
>>>> DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created
new
>>>> Met2dDataFile object of type "FileType_Gb1".
>>>> DEBUG 4: VarInfoFactory::new_var_info() -> created new VarInfo
object
>>>> of
>>>> type "FileType_Gb1".
>>>> GSL_RNG_TYPE=mt19937
>>>> GSL_RNG_SEED=1
>>>> DEBUG 1: Forecast File:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011412/GRIB_OUT_d02/WRFPRS24_d02.1.tm00
>>>> DEBUG 1: Climatology File: none
>>>> DEBUG 1: Observation File:
/h/data/global/WXQC/data/obs_prep/2013011512
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
--------------------------------------------------------------------------------
>>>> DEBUG 2:
>>>> DEBUG 2: Reading data for TMP/Z2.
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match for
>>>> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011412/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011412/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>>>> Memory fault
>>>> =======
>>>>
>>>> I've never tried plot_point_obs-I don't see reference to it in
the 4.0
>>>> users guide.
>>>>
>>>> Okay, by typing the executable it told me the arguments I needed.
Just
>>>> ran it... 33208 obs for ADPSFC type, 3102 obs for gc 11, but none
for
>>>> the
>>>> intersect of the two... weird.
>>>>
>>>> We'll keep digging and see what we can find... I wonder if PB2NC
is
>>>> throwing out all the stuff we need based on the settings we put
in the
>>>> config file.
>>>>
>>>> Thanks,
>>>> Matt
>>>>
>>>> -----Original Message-----
>>>> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
>>>> Sent: Wednesday, January 16, 2013 4:27 PM
>>>> To: Sittel, Matthew C CTR USAF AFWA 16 WS/WXN
>>>> Cc: Craig, Robert J Civ USAF AFWA 16 WS/WXN
>>>> Subject: RE: [rt.rap.ucar.edu #59890] Problem with Pairing NetCDF
with
>>>> GriB
>>>>
>>>> Matt,
>>>>
>>>> I ran METv4.0 point_stat for the first ensemble member using the
>>>> attached
>>>> configuration file and verbosity level 3. I got 0 matched pairs,
but
>>>> the
>>>> following diagnostic information is included in the output:
>>>>
>>>> DEBUG 2: Processing TMP/Z2 versus TMP/Z2, for observation type
ADPSFC,
>>>> over region FULL, for interpolation method BILIN(4), using 0
pairs.
>>>> DEBUG 3: Number of matched pairs = 0
>>>> DEBUG 3: Observations processed = 219303
>>>> DEBUG 3: Rejected: GRIB code = 190302
>>>> DEBUG 3: Rejected: valid time = 29001
>>>> DEBUG 3: Rejected: bad obs value = 0
>>>> DEBUG 3: Rejected: off the grid = 0
>>>> DEBUG 3: Rejected: level mismatch = 0
>>>> DEBUG 3: Rejected: message type = 0
>>>> DEBUG 3: Rejected: masking region = 0
>>>> DEBUG 3: Rejected: bad fcst value = 0
>>>>
>>>> This tells me that there's problem in the matching time window.
>>>>
>>>> Running "wgrib -V" on that forecast file tells me that this is a
>>>> 24-hour
>>>> forecast initialized at 2013011412. So it's valid at 2013011512.
>>>>
>>>> Your observation file is named 2013011512, but when you actually
look
>>>> at
>>>> the times it contains, they're for 2013011412:
>>>> ncdump -v hdr_vld nc2013011512
>>>>
>>>> (note, ncdump doesn't like file names that begin with a number,
so I
>>>> stuck
>>>> an "nc" in front)
>>>>
>>>> I reran point-stat a second time, but first added 3 0's to the
"beg"
>>>> and
>>>> "end" times in the "obs_window" setting. When I did that, I got
>>>> further,
>>>> but still no matched pairs:
>>>>
>>>> DEBUG 2: Processing TMP/Z2 versus TMP/Z2, for observation type
ADPSFC,
>>>> over region FULL, for interpolation method BILIN(4), using 0
pairs.
>>>> DEBUG 3: Number of matched pairs = 0
>>>> DEBUG 3: Observations processed = 219303
>>>> DEBUG 3: Rejected: GRIB code = 190302
>>>> DEBUG 3: Rejected: valid time = 0
>>>> DEBUG 3: Rejected: bad obs value = 0
>>>> DEBUG 3: Rejected: off the grid = 22047
>>>> DEBUG 3: Rejected: level mismatch = 6177
>>>> DEBUG 3: Rejected: message type = 777
>>>> DEBUG 3: Rejected: masking region = 0
>>>> DEBUG 3: Rejected: bad fcst value = 0
>>>>
>>>> So there just aren't any ADPSFC observations of 2-m temperature
that
>>>> fall
>>>> within your d02 domain. I ran the plot_point_obs utility to
verify
>>>> that
>>>> that's the case.
>>>>
>>>> As for why you got a seg-fault, I don't know. It's possible that
you
>>>> made
>>>> an error in your Point-Stat config file that caused it. I can't
really
>>>> debug it unless I can replicate it.
>>>>
>>>> John
>>>>
>>>>>
>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59890 >
>>>>>
>>>>> Hi John,
>>>>>
>>>>> The plot thickens. When I ran point_stat on one ensemble member
vs.
>>>>> an observation file, MET returned a memory fault when it got to
the
>>>>> observation file read. I tried a second observation file and
got the
>>>>> same result.
>>>>>
>>>>> I looked using 'ncdump' on the NetCDF observation data output
from
>>>>> 'pb2nc', and confirmed there are ADPSFC records, records within
the
>>>>> CONUS grid and in the requested time window.
>>>>>
>>>>> I used "-v 3" verbosity in the example I sent you... I think 4
is the
>>>>> highest level of message? I tried rerunning this morning with
"-v 6"
>>>>> and didn't get any additional information as to what is
happening with
>>>>> the observation matching, nor any messages numbered above 4.
>>>>>
>>>>> I have pushed the 10 ensemble members, observation file and
config
>>>>> file to the directory you listed. I had to create a "met_help"
>>>>> directory under 'irap' as one was not there. Inside 'sittel'
you will
>>>>> find the files.
>>>>> Please note these were created in a Linux environment-not sure
if that
>>>>> will be an issue for you; it might depend on your OS. We were
forced
>>>>> to go to Linux here.
>>>>>
>>>>> Thanks,
>>>>> Matt
>>>>>
>>>>> -----Original Message-----
>>>>> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
>>>>> Sent: Tuesday, January 15, 2013 4:10 PM
>>>>> To: Sittel, Matthew C CTR USAF AFWA 16 WS/WXN
>>>>> Cc: Craig, Robert J Civ USAF AFWA 16 WS/WXN
>>>>> Subject: Re: [rt.rap.ucar.edu #59890] Problem with Pairing
NetCDF with
>>>>> GriB
>>>>>
>>>>> Matt,
>>>>>
>>>>> So the question is basically, why are you getting 0 matched
pairs from
>>>>> ensemble-stat. If this were the point-stat tool, the answer
would be
>>>>> easy. In point-stat, when you turn up the verbosity level, for
each
>>>>> verification task, a list of reasons for why observations were
not
>>>>> used is printed out. And that's often an easy way to diagnose
the
>>>>> problem.
>>>>>
>>>>> But that functionality has not been implemented for ensemble-
stat.
>>>>> Though the problem you're having demonstrates the need!
>>>>>
>>>>> There are several possibilities:
>>>>> - the observations may not fall within the matching time
window
>>>>> - the observations may not fall within the model's grid
>>>>> - perhaps there are no ADPSFC observations of TMP in your
dataset
>>>>>
>>>>> There are different steps I'd take to rule out each of these
cases -
>>>>> mostly making use of the plot_data_plane and plot_point_obs
utilities.
>>>>> But perhaps the easiest thing to do is to have you run one of
your
>>>>> forecast files and the NetCDF output of PB2NC through point-stat
to
>>>>> verify TMP/Z2.
>>>>>
>>>>> Presumably, you'll get 0 matched pairs again. But when you
increase
>>>>> the verbosity level (-v 3, I think), you'll see a list of
reasons why
>>>>> the obs weren't used. Send me that list and then we'll know how
to
>>>>> proceed.
>>>>>
>>>>> Alternatively, you could just send me one or two of your GRIB
files,
>>>>> the NetCDF point observation file, and your config file, and I
could
>>>>> debug it here. If you choose to go that route, follow the
>>>>> instructions
>>>>> here:
>>>>> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>>>>>
>>>>> Your config file settings look fine. The instructions for
specifying
>>>>> NetCDF config file stuff only applies to gridded NetCDF data.
But
>>>>> your gridded data is in GRIB format. For point observations, we
chose
>>>>> the convention of specifying them as if they were GRIB files.
That's
>>>>> why we include a GRIB code for each observation value in the
data.
>>>>>
>>>>> Just let me know which route you decide on. And I'm happy to
help get
>>>>> you up and running.
>>>>>
>>>>> Thanks,
>>>>> John
>>>>>
>>>>>>
>>>>>> Tue Jan 15 09:22:51 2013: Request 59890 was acted upon.
>>>>>> Transaction: Ticket created by matthew.sittel.ctr at offutt.af.mil
>>>>>> Queue: met_help
>>>>>> Subject: Problem with Pairing NetCDF with GriB
>>>>>> Owner: Nobody
>>>>>> Requestors: matthew.sittel.ctr at offutt.af.mil
>>>>>> Status: new
>>>>>> Ticket <URL:
>>>>>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59890
>>>>>>>
>>>>>>
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I'm trying to get MET v4.0 to create verification rank
histogram
>>>>>> output for AFWA's mesoscale ensemble. I'm using in-house,
GriB-1
>>>>>> format WRF ensemble member output and verifying it with
>>>>>> PREPBUFR-formatted obs that we are pulling via FTP from NCEP.
We
>>>>>> convert the PREPBUFR format to NetCDF using 'pb2nc'.
>>>>>>
>>>>>> The problem I'm having is I cannot figure out what the correct
>>>>>> settings are in the config file for ensemble_stat to get the
model
>>>>>> and observation fields to match up.
>>>>>>
>>>>>> I've attached the NetCDF 'dump' of the observation file's
contents,
>>>>>> the contents of my config file and the output I get when I run
>>>>>> ensemble_stat.
>>>>>> No matter what settings I pick, it always comes back with 0
pairs
>>>>>> of
>>>>>> data. The output suggests the model members are read in and
that
>>>>>> there are records in the observation file as well-both
suggesting to
>>>>>> me the data are read in correctly, but I'm not designating the
>>>>>> correct pairing of them with the config file parameters.
>>>>>>
>>>>>> The records I am trying to get are the ones where gc=11 and
hgt=0, or
>>>>>> simply the surface temperature reports from "ADPSFC" message
type. I
>>>>>> have read the user's guide and can't figure out what the right
>>>>>> settings are for NetCDF; it is discussed briefly but not with
enough
>>>>>> detail to figure this out on my own, especially the ANNN, RNNN,
etc,
>>>>>> and the TMP(I,...,J,*,*) wording which is unclear. Can you
take a
>>>>>> look and advise as to the right settings?
>>>>>>
>>>>>> Thanks,
>>>>>> Matt
>>>>>>
>>>>>> ==
>>>>>> netcdf 2013011412 {
>>>>>> dimensions:
>>>>>> mxstr = 16 ;
>>>>>> hdr_arr_len = 3 ;
>>>>>> obs_arr_len = 5 ;
>>>>>> nobs = UNLIMITED ; // (219303 currently)
>>>>>> nhdr = 40233 ;
>>>>>> variables:
>>>>>> float obs_arr(nobs, obs_arr_len) ;
>>>>>> obs_arr:long_name = "array of observation
values" ;
>>>>>> obs_arr:_fill_value = -9999.f ;
>>>>>> obs_arr:columns = "hdr_id gc lvl hgt ob" ;
>>>>>> obs_arr:hdr_id_long_name = "index of matching
header
>>>>>> data"
>>>>>> ;
>>>>>> obs_arr:gc_long_name = "grib code
corresponding to
>>>>>> the observation type" ;
>>>>>> obs_arr:lvl_long_name = "pressure level (hPa)
or
>>>>>> accumulation interval (sec)" ;
>>>>>> obs_arr:hgt_long_name = "height in meters
above sea
>>>>>> level (msl)" ;
>>>>>> obs_arr:ob_long_name = "observation value" ;
>>>>>> char hdr_typ(nhdr, mxstr) ;
>>>>>> hdr_typ:long_name = "message type" ;
>>>>>> char hdr_sid(nhdr, mxstr) ;
>>>>>> hdr_sid:long_name = "station identification"
;
>>>>>> char hdr_vld(nhdr, mxstr) ;
>>>>>> hdr_vld:long_name = "valid time" ;
>>>>>> hdr_vld:units = "YYYYMMDD_HHMMSS" ;
>>>>>> float hdr_arr(nhdr, hdr_arr_len) ;
>>>>>> hdr_arr:long_name = "array of observation
station
>>>>>> header values" ;
>>>>>> hdr_arr:_fill_value = -9999.f ;
>>>>>> hdr_arr:columns = "lat lon elv" ;
>>>>>> hdr_arr:lat_long_name = "latitude" ;
>>>>>> hdr_arr:lat_units = "degrees_north" ;
>>>>>> hdr_arr:lon_long_name = "longitude" ;
>>>>>> hdr_arr:lon_units = "degrees_east" ;
>>>>>> hdr_arr:elv_long_name = "elevation" ;
>>>>>> hdr_arr:elv_units = "meters above sea level
(msl)" ;
>>>>>>
>>>>>> // global attributes:
>>>>>> :FileOrigins = "File
>>>>>> /h/data/global/WXQC/data//obs_prep/2013011412 generated
>>>>>> 20130115_003309 UTC on host dev8dlsn1 by the MET pb2nc tool" ;
>>>>>> :MET_version = "V4.0" ;
>>>>>> :MET_tool = "pb2nc" ;
>>>>>> data:
>>>>>> 40013, 7, 1016.4, 0, 0,
>>>>>> 40014, 7, 1012.2, 0, 0,
>>>>>> 40015, 7, 1024.4, 0, 0,
>>>>>> 40016, 11, 1030.6, 0, 236.55,
>>>>>> 40016, 7, 1030.6, 0, 0,
>>>>>> 40016, 2, 1030.6, 0, 103060,
>>>>>> 40017, 11, 1030.6, 0, 236.55,
>>>>>> 40017, 7, 1030.6, 0, 0,
>>>>>> 40017, 2, 1030.6, 0, 103060,
>>>>>> 40018, 11, 1030.9, 0, 235.95,
>>>>>> 40018, 7, 1030.9, 0, 0,
>>>>>> 40018, 2, 1030.9, 0, 103090,
>>>>>> 40019, 11, 1032.9, 0, 239.85,
>>>>>> 40019, 7, 1032.9, 0, 0,
>>>>>> 40019, 2, 1032.9, 0, 103290,
>>>>>> 40020, 11, 1034.1, 0, 240.65,
>>>>>> 40020, 7, 1034.1, 0, 0,
>>>>>> 40020, 2, 1034.1, 0, 103410,
>>>>>>
>>>>>>
/////////////////////////////////////////////////////////////////////
>>>>>> ///////////
>>>>>> //
>>>>>> // Ensemble-Stat configuration file.
>>>>>> //
>>>>>> // For additional information, see the
MET_BASE/data/config/README
>>>>>> file.
>>>>>> //
>>>>>>
/////////////////////////////////////////////////////////////////////
>>>>>> ///////////
>>>>>>
>>>>>> //
>>>>>> // Output model name to be written
>>>>>> //
>>>>>> model = "WRF";
>>>>>>
>>>>>>
/////////////////////////////////////////////////////////////////////
>>>>>> ///////////
>>>>>>
>>>>>> //
>>>>>> // Ensemble product fields to be processed // ens = {
>>>>>> ens_thresh = 1.0;
>>>>>> vld_thresh = 1.0;
>>>>>>
>>>>>> field = [
>>>>>> {
>>>>>> name = "TMP";
>>>>>> level = [ "Z2" ];
>>>>>> cat_thresh = [ >=300.0 ];
>>>>>> }
>>>>>> ];
>>>>>> }
>>>>>>
>>>>>>
/////////////////////////////////////////////////////////////////////
>>>>>> ///////////
>>>>>>
>>>>>> //
>>>>>> // Forecast and observation fields to be verified // fcst = {
>>>>>> message_type = [ "ADPSFC" ];
>>>>>> field = [
>>>>>> {
>>>>>> name = "TMP";
>>>>>> level = "Z2";
>>>>>> }
>>>>>> ];
>>>>>> }
>>>>>>
>>>>>> obs=fcst;
>>>>>>
>>>>>>
/////////////////////////////////////////////////////////////////////
>>>>>> ///////////
>>>>>>
>>>>>> //
>>>>>> // Point observation time window
>>>>>> //
>>>>>> obs_window = {
>>>>>> beg = -1800;
>>>>>> end = 1800;
>>>>>> }
>>>>>>
>>>>>>
/////////////////////////////////////////////////////////////////////
>>>>>> ///////////
>>>>>>
>>>>>> //
>>>>>> // Verification masking regions
>>>>>> //
>>>>>> mask = {
>>>>>> grid = [ "FULL" ];
>>>>>> poly = [];
>>>>>> sid = "";
>>>>>> };
>>>>>>
>>>>>>
/////////////////////////////////////////////////////////////////////
>>>>>> ///////////
>>>>>>
>>>>>> //
>>>>>> // Interpolation methods
>>>>>> //
>>>>>> interp = {
>>>>>> field = OBS;
>>>>>> vld_thresh = 1.0;
>>>>>>
>>>>>> type = [
>>>>>> {
>>>>>> method = BILIN;
>>>>>> width = 2;
>>>>>> }
>>>>>> ];
>>>>>> };
>>>>>>
>>>>>>
/////////////////////////////////////////////////////////////////////
>>>>>> ///////////
>>>>>>
>>>>>> //
>>>>>> // Statistical output types
>>>>>> //
>>>>>> output_flag = {
>>>>>> rhist = STAT;
>>>>>> orank = NONE;
>>>>>> };
>>>>>>
>>>>>>
/////////////////////////////////////////////////////////////////////
>>>>>> ///////////
>>>>>>
>>>>>> //
>>>>>> // Ensemble product output types
>>>>>> //
>>>>>> ensemble_flag = {
>>>>>> mean = TRUE;
>>>>>> stdev = TRUE;
>>>>>> minus = FALSE;
>>>>>> plus = FALSE;
>>>>>> min = FALSE;
>>>>>> max = FALSE;
>>>>>> range = FALSE;
>>>>>> vld_count = FALSE;
>>>>>> frequency = FALSE;
>>>>>> rank = TRUE;
>>>>>> };
>>>>>>
>>>>>>
/////////////////////////////////////////////////////////////////////
>>>>>> ///////////
>>>>>>
>>>>>> //
>>>>>> // Random number generator
>>>>>> //
>>>>>> rng = {
>>>>>> type = "mt19937";
>>>>>> seed = "1";
>>>>>> }
>>>>>>
>>>>>>
/////////////////////////////////////////////////////////////////////
>>>>>> ///////////
>>>>>>
>>>>>> duplicate_flag = NONE;
>>>>>> output_prefix = "";
>>>>>> version = "V4.0";
>>>>>>
>>>>>>
/////////////////////////////////////////////////////////////////////
>>>>>> ///////////
>>>>>> ==
>>>>>> Obs file: /h/data/global/WXQC/data/obs_prep/2013011412
>>>>>> DEBUG 1: Default Config File:
>>>>>> /home/qcteam/METv4.0/data/config/EnsembleStatConfig_default
>>>>>> DEBUG 1: User Config File:
/home/qcteam/METv4.0/cfg/ensemble_stat.cfg
>>>>>> DEBUG 1: Ensemble Files[10]:
>>>>>> DEBUG 1:
>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>> _d02/WRFPRS24_d02.1.tm00
>>>>>> DEBUG 1:
>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>> _d02/WRFPRS24_d02.2.tm00
>>>>>> DEBUG 1:
>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>> _d02/WRFPRS24_d02.3.tm00
>>>>>> DEBUG 1:
>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>> _d02/WRFPRS24_d02.4.tm00
>>>>>> DEBUG 1:
>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>> _d02/WRFPRS24_d02.5.tm00
>>>>>> DEBUG 1:
>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>> _d02/WRFPRS24_d02.6.tm00
>>>>>> DEBUG 1:
>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>> _d02/WRFPRS24_d02.7.tm00
>>>>>> DEBUG 1:
>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>> _d02/WRFPRS24_d02.8.tm00
>>>>>> DEBUG 1:
>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>> _d02/WRFPRS24_d02.9.tm00
>>>>>> DEBUG 1:
>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>> _d02/WRFPRS24_d02.10.tm00
>>>>>> DEBUG 1: Point Observation Files[1]:
>>>>>> DEBUG 1: /h/data/global/WXQC/data/obs_prep/2013011412
>>>>>> DEBUG 2:
>>>>>> DEBUG 2:
>>>>>>
---------------------------------------------------------------------
>>>>>> -----------
>>>>>> DEBUG 2:
>>>>>> DEBUG 2: Processing ensemble field: TMP/Z2 DEBUG 3:
>>>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
>>>>>> DEBUG 2:
>>>>>> DEBUG 2:
>>>>>>
---------------------------------------------------------------------
>>>>>> -----------
>>>>>> DEBUG 2:
>>>>>> DEBUG 2: Processing point observation file:
>>>>>> /h/data/global/WXQC/data/obs_prep/2013011412
>>>>>> DEBUG 2: Searching 219303 observations from 40233 header
messages.
>>>>>> DEBUG 2:
>>>>>> DEBUG 2:
>>>>>>
---------------------------------------------------------------------
>>>>>> -----------
>>>>>> DEBUG 2:
>>>>>> DEBUG 2: Processing ensemble file:
>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>> _d02/WRFPRS24_d02.1.tm00 DEBUG 3:
>>>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>>>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>>>> DEBUG 2:
>>>>>> DEBUG 2:
>>>>>>
---------------------------------------------------------------------
>>>>>> -----------
>>>>>> DEBUG 2:
>>>>>> DEBUG 2: Processing ensemble file:
>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>> _d02/WRFPRS24_d02.2.tm00 DEBUG 3:
>>>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
>>>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>>>> DEBUG 2:
>>>>>> DEBUG 2:
>>>>>>
---------------------------------------------------------------------
>>>>>> -----------
>>>>>> DEBUG 2:
>>>>>> DEBUG 2: Processing ensemble file:
>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>> _d02/WRFPRS24_d02.3.tm00 DEBUG 3:
>>>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
>>>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>>>> DEBUG 2:
>>>>>> DEBUG 2:
>>>>>>
---------------------------------------------------------------------
>>>>>> -----------
>>>>>> DEBUG 2:
>>>>>> DEBUG 2: Processing ensemble file:
>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>> _d02/WRFPRS24_d02.4.tm00 DEBUG 3:
>>>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
>>>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>>>> DEBUG 2:
>>>>>> DEBUG 2:
>>>>>>
---------------------------------------------------------------------
>>>>>> -----------
>>>>>> DEBUG 2:
>>>>>> DEBUG 2: Processing ensemble file:
>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>> _d02/WRFPRS24_d02.5.tm00 DEBUG 3:
>>>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
>>>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>>>> DEBUG 2:
>>>>>> DEBUG 2:
>>>>>>
---------------------------------------------------------------------
>>>>>> -----------
>>>>>> DEBUG 2:
>>>>>> DEBUG 2: Processing ensemble file:
>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>> _d02/WRFPRS24_d02.6.tm00 DEBUG 3:
>>>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
>>>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>>>> DEBUG 2:
>>>>>> DEBUG 2:
>>>>>>
---------------------------------------------------------------------
>>>>>> -----------
>>>>>> DEBUG 2:
>>>>>> DEBUG 2: Processing ensemble file:
>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>> _d02/WRFPRS24_d02.7.tm00 DEBUG 3:
>>>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
>>>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>>>> DEBUG 2:
>>>>>> DEBUG 2:
>>>>>>
---------------------------------------------------------------------
>>>>>> -----------
>>>>>> DEBUG 2:
>>>>>> DEBUG 2: Processing ensemble file:
>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>> _d02/WRFPRS24_d02.8.tm00 DEBUG 3:
>>>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
>>>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>>>> DEBUG 2:
>>>>>> DEBUG 2:
>>>>>>
---------------------------------------------------------------------
>>>>>> -----------
>>>>>> DEBUG 2:
>>>>>> DEBUG 2: Processing ensemble file:
>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>> _d02/WRFPRS24_d02.9.tm00 DEBUG 3:
>>>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
>>>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>>>> DEBUG 2:
>>>>>> DEBUG 2:
>>>>>>
---------------------------------------------------------------------
>>>>>> -----------
>>>>>> DEBUG 2:
>>>>>> DEBUG 2: Processing ensemble file:
>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>> _d02/WRFPRS24_d02.10.tm00 DEBUG 3:
>>>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
>>>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>>>> DEBUG 2:
>>>>>> DEBUG 2:
>>>>>>
---------------------------------------------------------------------
>>>>>> -----------
>>>>>> DEBUG 2:
>>>>>> DEBUG 2: Processing point verification TMP/Z2 versus TMP/Z2,
for
>>>>>> observation type ADPSFC, over region FULL, for interpolation
method
>>>>>> BILIN(4), using 0 pairs.
>>>>>> DEBUG 2:
>>>>>> DEBUG 2:
>>>>>>
---------------------------------------------------------------------
>>>>>> -----------
>>>>>> DEBUG 2:
>>>>>> DEBUG 1: Output file:
>>>>>>
/home/qcteam/METv4.0/out/ensemble_stat/ensemble_stat_20130114_120000V
>>>>>> .stat
>>>>>> DEBUG 1: Output file:
>>>>>>
/home/qcteam/METv4.0/out/ensemble_stat/ensemble_stat_20130114_120000V
>>>>>> _ens.nc
>>>>>>
>>>>>> // SIGNED //
>>>>>> Matthew C. Sittel
>>>>>> Associate Scientist
>>>>>> University Corporation for Atmospheric Research 16WS/WXN,
Offutt AFB,
>>>>>> NE
>>>>>> (402) 294-3473 DSN: 271-3473
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>>
>>
>>
>>
>
------------------------------------------------
Subject: Problem with Pairing NetCDF with GriB
From: matthew.sittel.ctr at offutt.af.mil
Time: Fri Jan 18 12:13:05 2013
Yep, that was it. Almost 10,000 obs now for 00Z. I'd add that 'set
it to 9' gotcha to the web page, as a heads up to others who might run
into that problem.
Thanks for helping us find the needle in the MET haystack!
Matt
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Friday, January 18, 2013 1:03 PM
To: Sittel, Matthew C CTR USAF AFWA 16 WS/WXN
Cc: Craig, Robert J Civ USAF AFWA 16 WS/WXN
Subject: Re: [rt.rap.ucar.edu #59890] Problem with Pairing NetCDF with
GriB
Matt,
One other item to note. In METv4.0, we are *not* passing the quality
flag value through PB2NC for each observation.
However, one enhancement in METv4.1, that's currently under
development, is to pass the quality flag value through PB2NC. Then in
Point-Stat and Ensemble-Stat, you can specify in the config file
which quality flags you'd like to use. In the case of this GDAS data,
you may choose to verify your forecast against observations with
quality values of 1, 2, and 9, skipping everything between 2 and 9.
John
On 01/18/2013 11:09 AM, John Halley Gotway wrote:
> Matt,
>
> First, I'm glad you were able to get it working.
>
> Next, if you have point internal observations that you'd like to
use,
> rather than trying to write a NetCDF point observation file
directly, I'd
> suggest using the ASCII2NC tool. As long as you can format your
point
> observations in the simple ASCII format that it expects, running
ASCII2NC
> will result in a NetCDF file the other MET tools can handle.
>
> Regarding NDAS vs GDAS - NDAS PrepBufr files contain point
observations
> used in the data assimilation process for NAM. GDAS contains point
> observation data used in data assimilation for GFS.
>
> You'd expect NDAS to be limited to North America, but it does
actually
> contain many obs over Europe and the pacific. Generally speaking
though,
> use NDAS for domains over North America and GDAS for domains outside
North
> America.
>
> Here's one important issue though - and I suspect this is what
you're
> encountering - in GDAS, NCEP sets the quality flag for surface obs
to a
> value of 9. They do that to prevent their use in the data
assimilation
> process because it makes the results of the data assimilation worse.
>
> Try cranking up the quality flag in PB2NC to 9 to see if 2-meter
> temperature is retained.
>
> John
>
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59890 >
>>
>> I pushed a file via FTP to you... it's a prepbufr format file:
>> gdas1.t06z.prepbufr.nr. Could you run PB2NC on it on your side and
see if
>> you agree with us in that there are no ADPSFC temperature (grib
field 11)
>> observations in the file? For some reason all we get are grib ID
7,
>> height data, for ADPSFC and nothing else.
>>
>> Without reading the prepbufr data, we can't be sure if there is
>> temperature data that gets tossed based on the config file
settings, or it
>> simply isn't in the prepbufr file at all, which means we can't
really use
>> it for our application. That would be an important caveat for
users.
>>
>> Thanks,
>> Matt
>>
>> -----Original Message-----
>> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
>> Sent: Friday, January 18, 2013 9:20 AM
>> To: Sittel, Matthew C CTR USAF AFWA 16 WS/WXN
>> Cc: Craig, Robert J Civ USAF AFWA 16 WS/WXN
>> Subject: Re: [rt.rap.ucar.edu #59890] Problem with Pairing NetCDF
with
>> GriB
>>
>> Matt,
>>
>> Good news. The problem was in your point_stat.cfg file. Diffing
it
>> against the default one in
METv4.0/data/config/PointStatConfig_default, I
>> see the following:
>> < poly = "";
>> ---
>> > poly = [];
>>
>> In the masking section, it expects "poly" to be specified as an
array.
>> Instead of specifying it as an empty array, you specified it as a
string
>> of length 0. So it tried to read a file of that name.
>> Running METv4.0 with the latest set of patches at verbosity
level 4,
>> gave the following helpful information:
>>
>> DEBUG 2: Reading data for TMP/Z2.
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
>> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
"WRFPRS24_d02.1.tm00".
>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>> matching VarInfo "TMP/Z2" in GRIB file "WRFPRS24_d02.1.tm00".
>> DEBUG 4: parse_sid_mask() -> parsing station ID mask file "(nul)"
>> DEBUG 4: parse_grid_mask() -> parsing grid mask "FULL"
>> DEBUG 4: parse_poly_mask() -> parsing poly mask "(nul)"
>> ERROR :
>> ERROR : grd_file_type() -> file does not exist "(nul)"
>> ERROR :
>>
>> However, running the same thing for METv4.0 as originally released
results
>> in a seg fault. Yeah, we found it!
>>
>> Looking at the list of fixes, I'm guessing it's the bug fix posted
on
>> 08/01/2012:
>> "Fixed a bug that caused a seg fault when a config file string
list
>> contains only an empty string"
>>
http://www.dtcenter.org/met/users/support/known_issues/METv4.0/index.php
>>
>> So just edit your Point-Stat config file to use:
>> poly = [];
>>
>> And then recompile METv4.0 after applying the latest set of
patches.
>>
>> Hopefully that'll do the trick. Just let us know if you have more
>> problems or questions.
>>
>> Thanks,
>> John
>>
>>
>>
>> On 01/18/2013 07:53 AM, matthew.sittel.ctr at offutt.af.mil via RT
wrote:
>>>
>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59890 >
>>>
>>> Hi John,
>>>
>>> I've added the point_stat config file to the FTP directory with
the
>>> other files.
>>>
>>> I have not downloaded any of the patches for v4.0. I thought I
had but
>>> turns out it was for v3.1 back on May 29 '12.
>>>
>>> Bob has been looking at the PB2NC process. The obs plotting
capability
>>> has been helpful.
>>>
>>> I'll recompile and let you know if it helps.
>>>
>>> Thanks,
>>> Matt
>>>
>>> -----Original Message-----
>>> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
>>> Sent: Thursday, January 17, 2013 4:42 PM
>>> To: Sittel, Matthew C CTR USAF AFWA 16 WS/WXN
>>> Cc: Craig, Robert J Civ USAF AFWA 16 WS/WXN
>>> Subject: RE: [rt.rap.ucar.edu #59890] Problem with Pairing NetCDF
with
>>> GriB
>>>
>>> Matt,
>>>
>>> Yeah, the ncdump thing is silly. The older version in my path
complains
>>> about leading numbers, but I believe that later versions allow it.
>>> Guess
>>> I need to update my path!
>>>
>>> I'm not at my desk this afternoon, so am unable to debug until
tomorrow
>>> morning. But I'm still puzzled about the source of that seg
fault. I'd
>>> like to be able to replicate it to address it.
>>>
>>> Could you please do 2 things:
>>> - Send me the exact point-stat config file you're using:
>>> /home/qcteam/METv4.0/cfg/point_stat.cfg
>>> - Let me know the date of the latest set of patches you're using
for
>>> METv4.0. Just take a look in /home/qcteam/METv4.0 to see if
there's a
>>> patch file in there with a date in the filename.
>>>
>>> Hopefully, if I run your data with your config file and the exact
same
>>> version of the code, I'll be able to reproduce it. If not, it's a
lot
>>> more difficult problem.
>>>
>>> You could always try applying the latest set of METv4.0 patches
and
>>> recompiling to see if that helps:
>>>
http://www.dtcenter.org/met/users/support/known_issues/METv4.0/index.php
>>>
>>> Regarding why I got 0 matched pairs even when accounting for the
time
>>> offset, I think looking at the PB2NC configuration is a great
idea. In
>>> particular, look to see if you're filtering on the observation qc
flag.
>>> If you also send me your PB2NC config file and the location of the
>>> PREPBUFR files you're using, I'd be happy to take a look at that
too.
>>>
>>> Thanks,
>>> John
>>>
>>>>
>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59890 >
>>>>
>>>> Hi John,
>>>>
>>>> I put an updated obs file onto the FTP site with correct time
>>>> stamps-overwrote the old one. Talk about confusing. The file I
sent
>>>> you
>>>> was later overwritten by a new file so when I looked at it, the
date
>>>> fields were great. Bob will check into why our process pulls
data from
>>>> one date and saves it as another.
>>>>
>>>> Weird, ncdump accepts numerical file names for me... were you
running
>>>> this
>>>> in a Linux environment? Not blaming Linux, but it's always a
good
>>>> place
>>>> to start...
>>>>
>>>> Here's the output from my point_stat run of member 1 vs. the
>>>> observation
>>>> file:
>>>>
>>>> =======
>>>> /home/qcteam/METv4.0/bin/point_stat
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011412/GRIB_OUT_d02/WRFPRS24_d02.1.tm00
>>>> /h/data/global/WXQC/data/obs_prep/2013011512
>>>> /home/qcteam/METv4.0/cfg/point_stat.cfg -outdir
>>>> /home/qcteam/METv4.0/out
>>>> -v 4
>>>>
>>>> DEBUG 1: Default Config File:
>>>> /home/qcteam/METv4.0/data/config/PointStatConfig_default
>>>> DEBUG 1: User Config File:
/home/qcteam/METv4.0/cfg/point_stat.cfg
>>>> DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created
new
>>>> Met2dDataFile object of type "FileType_Gb1".
>>>> DEBUG 4: VarInfoFactory::new_var_info() -> created new VarInfo
object
>>>> of
>>>> type "FileType_Gb1".
>>>> GSL_RNG_TYPE=mt19937
>>>> GSL_RNG_SEED=1
>>>> DEBUG 1: Forecast File:
>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011412/GRIB_OUT_d02/WRFPRS24_d02.1.tm00
>>>> DEBUG 1: Climatology File: none
>>>> DEBUG 1: Observation File:
/h/data/global/WXQC/data/obs_prep/2013011512
>>>> DEBUG 2:
>>>> DEBUG 2:
>>>>
--------------------------------------------------------------------------------
>>>> DEBUG 2:
>>>> DEBUG 2: Reading data for TMP/Z2.
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match for
>>>> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011412/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011412/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>>>> Memory fault
>>>> =======
>>>>
>>>> I've never tried plot_point_obs-I don't see reference to it in
the 4.0
>>>> users guide.
>>>>
>>>> Okay, by typing the executable it told me the arguments I needed.
Just
>>>> ran it... 33208 obs for ADPSFC type, 3102 obs for gc 11, but none
for
>>>> the
>>>> intersect of the two... weird.
>>>>
>>>> We'll keep digging and see what we can find... I wonder if PB2NC
is
>>>> throwing out all the stuff we need based on the settings we put
in the
>>>> config file.
>>>>
>>>> Thanks,
>>>> Matt
>>>>
>>>> -----Original Message-----
>>>> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
>>>> Sent: Wednesday, January 16, 2013 4:27 PM
>>>> To: Sittel, Matthew C CTR USAF AFWA 16 WS/WXN
>>>> Cc: Craig, Robert J Civ USAF AFWA 16 WS/WXN
>>>> Subject: RE: [rt.rap.ucar.edu #59890] Problem with Pairing NetCDF
with
>>>> GriB
>>>>
>>>> Matt,
>>>>
>>>> I ran METv4.0 point_stat for the first ensemble member using the
>>>> attached
>>>> configuration file and verbosity level 3. I got 0 matched pairs,
but
>>>> the
>>>> following diagnostic information is included in the output:
>>>>
>>>> DEBUG 2: Processing TMP/Z2 versus TMP/Z2, for observation type
ADPSFC,
>>>> over region FULL, for interpolation method BILIN(4), using 0
pairs.
>>>> DEBUG 3: Number of matched pairs = 0
>>>> DEBUG 3: Observations processed = 219303
>>>> DEBUG 3: Rejected: GRIB code = 190302
>>>> DEBUG 3: Rejected: valid time = 29001
>>>> DEBUG 3: Rejected: bad obs value = 0
>>>> DEBUG 3: Rejected: off the grid = 0
>>>> DEBUG 3: Rejected: level mismatch = 0
>>>> DEBUG 3: Rejected: message type = 0
>>>> DEBUG 3: Rejected: masking region = 0
>>>> DEBUG 3: Rejected: bad fcst value = 0
>>>>
>>>> This tells me that there's problem in the matching time window.
>>>>
>>>> Running "wgrib -V" on that forecast file tells me that this is a
>>>> 24-hour
>>>> forecast initialized at 2013011412. So it's valid at 2013011512.
>>>>
>>>> Your observation file is named 2013011512, but when you actually
look
>>>> at
>>>> the times it contains, they're for 2013011412:
>>>> ncdump -v hdr_vld nc2013011512
>>>>
>>>> (note, ncdump doesn't like file names that begin with a number,
so I
>>>> stuck
>>>> an "nc" in front)
>>>>
>>>> I reran point-stat a second time, but first added 3 0's to the
"beg"
>>>> and
>>>> "end" times in the "obs_window" setting. When I did that, I got
>>>> further,
>>>> but still no matched pairs:
>>>>
>>>> DEBUG 2: Processing TMP/Z2 versus TMP/Z2, for observation type
ADPSFC,
>>>> over region FULL, for interpolation method BILIN(4), using 0
pairs.
>>>> DEBUG 3: Number of matched pairs = 0
>>>> DEBUG 3: Observations processed = 219303
>>>> DEBUG 3: Rejected: GRIB code = 190302
>>>> DEBUG 3: Rejected: valid time = 0
>>>> DEBUG 3: Rejected: bad obs value = 0
>>>> DEBUG 3: Rejected: off the grid = 22047
>>>> DEBUG 3: Rejected: level mismatch = 6177
>>>> DEBUG 3: Rejected: message type = 777
>>>> DEBUG 3: Rejected: masking region = 0
>>>> DEBUG 3: Rejected: bad fcst value = 0
>>>>
>>>> So there just aren't any ADPSFC observations of 2-m temperature
that
>>>> fall
>>>> within your d02 domain. I ran the plot_point_obs utility to
verify
>>>> that
>>>> that's the case.
>>>>
>>>> As for why you got a seg-fault, I don't know. It's possible that
you
>>>> made
>>>> an error in your Point-Stat config file that caused it. I can't
really
>>>> debug it unless I can replicate it.
>>>>
>>>> John
>>>>
>>>>>
>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59890 >
>>>>>
>>>>> Hi John,
>>>>>
>>>>> The plot thickens. When I ran point_stat on one ensemble member
vs.
>>>>> an observation file, MET returned a memory fault when it got to
the
>>>>> observation file read. I tried a second observation file and
got the
>>>>> same result.
>>>>>
>>>>> I looked using 'ncdump' on the NetCDF observation data output
from
>>>>> 'pb2nc', and confirmed there are ADPSFC records, records within
the
>>>>> CONUS grid and in the requested time window.
>>>>>
>>>>> I used "-v 3" verbosity in the example I sent you... I think 4
is the
>>>>> highest level of message? I tried rerunning this morning with
"-v 6"
>>>>> and didn't get any additional information as to what is
happening with
>>>>> the observation matching, nor any messages numbered above 4.
>>>>>
>>>>> I have pushed the 10 ensemble members, observation file and
config
>>>>> file to the directory you listed. I had to create a "met_help"
>>>>> directory under 'irap' as one was not there. Inside 'sittel'
you will
>>>>> find the files.
>>>>> Please note these were created in a Linux environment-not sure
if that
>>>>> will be an issue for you; it might depend on your OS. We were
forced
>>>>> to go to Linux here.
>>>>>
>>>>> Thanks,
>>>>> Matt
>>>>>
>>>>> -----Original Message-----
>>>>> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
>>>>> Sent: Tuesday, January 15, 2013 4:10 PM
>>>>> To: Sittel, Matthew C CTR USAF AFWA 16 WS/WXN
>>>>> Cc: Craig, Robert J Civ USAF AFWA 16 WS/WXN
>>>>> Subject: Re: [rt.rap.ucar.edu #59890] Problem with Pairing
NetCDF with
>>>>> GriB
>>>>>
>>>>> Matt,
>>>>>
>>>>> So the question is basically, why are you getting 0 matched
pairs from
>>>>> ensemble-stat. If this were the point-stat tool, the answer
would be
>>>>> easy. In point-stat, when you turn up the verbosity level, for
each
>>>>> verification task, a list of reasons for why observations were
not
>>>>> used is printed out. And that's often an easy way to diagnose
the
>>>>> problem.
>>>>>
>>>>> But that functionality has not been implemented for ensemble-
stat.
>>>>> Though the problem you're having demonstrates the need!
>>>>>
>>>>> There are several possibilities:
>>>>> - the observations may not fall within the matching time
window
>>>>> - the observations may not fall within the model's grid
>>>>> - perhaps there are no ADPSFC observations of TMP in your
dataset
>>>>>
>>>>> There are different steps I'd take to rule out each of these
cases -
>>>>> mostly making use of the plot_data_plane and plot_point_obs
utilities.
>>>>> But perhaps the easiest thing to do is to have you run one of
your
>>>>> forecast files and the NetCDF output of PB2NC through point-stat
to
>>>>> verify TMP/Z2.
>>>>>
>>>>> Presumably, you'll get 0 matched pairs again. But when you
increase
>>>>> the verbosity level (-v 3, I think), you'll see a list of
reasons why
>>>>> the obs weren't used. Send me that list and then we'll know how
to
>>>>> proceed.
>>>>>
>>>>> Alternatively, you could just send me one or two of your GRIB
files,
>>>>> the NetCDF point observation file, and your config file, and I
could
>>>>> debug it here. If you choose to go that route, follow the
>>>>> instructions
>>>>> here:
>>>>> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>>>>>
>>>>> Your config file settings look fine. The instructions for
specifying
>>>>> NetCDF config file stuff only applies to gridded NetCDF data.
But
>>>>> your gridded data is in GRIB format. For point observations, we
chose
>>>>> the convention of specifying them as if they were GRIB files.
That's
>>>>> why we include a GRIB code for each observation value in the
data.
>>>>>
>>>>> Just let me know which route you decide on. And I'm happy to
help get
>>>>> you up and running.
>>>>>
>>>>> Thanks,
>>>>> John
>>>>>
>>>>>>
>>>>>> Tue Jan 15 09:22:51 2013: Request 59890 was acted upon.
>>>>>> Transaction: Ticket created by matthew.sittel.ctr at offutt.af.mil
>>>>>> Queue: met_help
>>>>>> Subject: Problem with Pairing NetCDF with GriB
>>>>>> Owner: Nobody
>>>>>> Requestors: matthew.sittel.ctr at offutt.af.mil
>>>>>> Status: new
>>>>>> Ticket <URL:
>>>>>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59890
>>>>>>>
>>>>>>
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I'm trying to get MET v4.0 to create verification rank
histogram
>>>>>> output for AFWA's mesoscale ensemble. I'm using in-house,
GriB-1
>>>>>> format WRF ensemble member output and verifying it with
>>>>>> PREPBUFR-formatted obs that we are pulling via FTP from NCEP.
We
>>>>>> convert the PREPBUFR format to NetCDF using 'pb2nc'.
>>>>>>
>>>>>> The problem I'm having is I cannot figure out what the correct
>>>>>> settings are in the config file for ensemble_stat to get the
model
>>>>>> and observation fields to match up.
>>>>>>
>>>>>> I've attached the NetCDF 'dump' of the observation file's
contents,
>>>>>> the contents of my config file and the output I get when I run
>>>>>> ensemble_stat.
>>>>>> No matter what settings I pick, it always comes back with 0
pairs
>>>>>> of
>>>>>> data. The output suggests the model members are read in and
that
>>>>>> there are records in the observation file as well-both
suggesting to
>>>>>> me the data are read in correctly, but I'm not designating the
>>>>>> correct pairing of them with the config file parameters.
>>>>>>
>>>>>> The records I am trying to get are the ones where gc=11 and
hgt=0, or
>>>>>> simply the surface temperature reports from "ADPSFC" message
type. I
>>>>>> have read the user's guide and can't figure out what the right
>>>>>> settings are for NetCDF; it is discussed briefly but not with
enough
>>>>>> detail to figure this out on my own, especially the ANNN, RNNN,
etc,
>>>>>> and the TMP(I,...,J,*,*) wording which is unclear. Can you
take a
>>>>>> look and advise as to the right settings?
>>>>>>
>>>>>> Thanks,
>>>>>> Matt
>>>>>>
>>>>>> ==
>>>>>> netcdf 2013011412 {
>>>>>> dimensions:
>>>>>> mxstr = 16 ;
>>>>>> hdr_arr_len = 3 ;
>>>>>> obs_arr_len = 5 ;
>>>>>> nobs = UNLIMITED ; // (219303 currently)
>>>>>> nhdr = 40233 ;
>>>>>> variables:
>>>>>> float obs_arr(nobs, obs_arr_len) ;
>>>>>> obs_arr:long_name = "array of observation
values" ;
>>>>>> obs_arr:_fill_value = -9999.f ;
>>>>>> obs_arr:columns = "hdr_id gc lvl hgt ob" ;
>>>>>> obs_arr:hdr_id_long_name = "index of matching
header
>>>>>> data"
>>>>>> ;
>>>>>> obs_arr:gc_long_name = "grib code
corresponding to
>>>>>> the observation type" ;
>>>>>> obs_arr:lvl_long_name = "pressure level (hPa)
or
>>>>>> accumulation interval (sec)" ;
>>>>>> obs_arr:hgt_long_name = "height in meters
above sea
>>>>>> level (msl)" ;
>>>>>> obs_arr:ob_long_name = "observation value" ;
>>>>>> char hdr_typ(nhdr, mxstr) ;
>>>>>> hdr_typ:long_name = "message type" ;
>>>>>> char hdr_sid(nhdr, mxstr) ;
>>>>>> hdr_sid:long_name = "station identification"
;
>>>>>> char hdr_vld(nhdr, mxstr) ;
>>>>>> hdr_vld:long_name = "valid time" ;
>>>>>> hdr_vld:units = "YYYYMMDD_HHMMSS" ;
>>>>>> float hdr_arr(nhdr, hdr_arr_len) ;
>>>>>> hdr_arr:long_name = "array of observation
station
>>>>>> header values" ;
>>>>>> hdr_arr:_fill_value = -9999.f ;
>>>>>> hdr_arr:columns = "lat lon elv" ;
>>>>>> hdr_arr:lat_long_name = "latitude" ;
>>>>>> hdr_arr:lat_units = "degrees_north" ;
>>>>>> hdr_arr:lon_long_name = "longitude" ;
>>>>>> hdr_arr:lon_units = "degrees_east" ;
>>>>>> hdr_arr:elv_long_name = "elevation" ;
>>>>>> hdr_arr:elv_units = "meters above sea level
(msl)" ;
>>>>>>
>>>>>> // global attributes:
>>>>>> :FileOrigins = "File
>>>>>> /h/data/global/WXQC/data//obs_prep/2013011412 generated
>>>>>> 20130115_003309 UTC on host dev8dlsn1 by the MET pb2nc tool" ;
>>>>>> :MET_version = "V4.0" ;
>>>>>> :MET_tool = "pb2nc" ;
>>>>>> data:
>>>>>> 40013, 7, 1016.4, 0, 0,
>>>>>> 40014, 7, 1012.2, 0, 0,
>>>>>> 40015, 7, 1024.4, 0, 0,
>>>>>> 40016, 11, 1030.6, 0, 236.55,
>>>>>> 40016, 7, 1030.6, 0, 0,
>>>>>> 40016, 2, 1030.6, 0, 103060,
>>>>>> 40017, 11, 1030.6, 0, 236.55,
>>>>>> 40017, 7, 1030.6, 0, 0,
>>>>>> 40017, 2, 1030.6, 0, 103060,
>>>>>> 40018, 11, 1030.9, 0, 235.95,
>>>>>> 40018, 7, 1030.9, 0, 0,
>>>>>> 40018, 2, 1030.9, 0, 103090,
>>>>>> 40019, 11, 1032.9, 0, 239.85,
>>>>>> 40019, 7, 1032.9, 0, 0,
>>>>>> 40019, 2, 1032.9, 0, 103290,
>>>>>> 40020, 11, 1034.1, 0, 240.65,
>>>>>> 40020, 7, 1034.1, 0, 0,
>>>>>> 40020, 2, 1034.1, 0, 103410,
>>>>>>
>>>>>>
/////////////////////////////////////////////////////////////////////
>>>>>> ///////////
>>>>>> //
>>>>>> // Ensemble-Stat configuration file.
>>>>>> //
>>>>>> // For additional information, see the
MET_BASE/data/config/README
>>>>>> file.
>>>>>> //
>>>>>>
/////////////////////////////////////////////////////////////////////
>>>>>> ///////////
>>>>>>
>>>>>> //
>>>>>> // Output model name to be written
>>>>>> //
>>>>>> model = "WRF";
>>>>>>
>>>>>>
/////////////////////////////////////////////////////////////////////
>>>>>> ///////////
>>>>>>
>>>>>> //
>>>>>> // Ensemble product fields to be processed // ens = {
>>>>>> ens_thresh = 1.0;
>>>>>> vld_thresh = 1.0;
>>>>>>
>>>>>> field = [
>>>>>> {
>>>>>> name = "TMP";
>>>>>> level = [ "Z2" ];
>>>>>> cat_thresh = [ >=300.0 ];
>>>>>> }
>>>>>> ];
>>>>>> }
>>>>>>
>>>>>>
/////////////////////////////////////////////////////////////////////
>>>>>> ///////////
>>>>>>
>>>>>> //
>>>>>> // Forecast and observation fields to be verified // fcst = {
>>>>>> message_type = [ "ADPSFC" ];
>>>>>> field = [
>>>>>> {
>>>>>> name = "TMP";
>>>>>> level = "Z2";
>>>>>> }
>>>>>> ];
>>>>>> }
>>>>>>
>>>>>> obs=fcst;
>>>>>>
>>>>>>
/////////////////////////////////////////////////////////////////////
>>>>>> ///////////
>>>>>>
>>>>>> //
>>>>>> // Point observation time window
>>>>>> //
>>>>>> obs_window = {
>>>>>> beg = -1800;
>>>>>> end = 1800;
>>>>>> }
>>>>>>
>>>>>>
/////////////////////////////////////////////////////////////////////
>>>>>> ///////////
>>>>>>
>>>>>> //
>>>>>> // Verification masking regions
>>>>>> //
>>>>>> mask = {
>>>>>> grid = [ "FULL" ];
>>>>>> poly = [];
>>>>>> sid = "";
>>>>>> };
>>>>>>
>>>>>>
/////////////////////////////////////////////////////////////////////
>>>>>> ///////////
>>>>>>
>>>>>> //
>>>>>> // Interpolation methods
>>>>>> //
>>>>>> interp = {
>>>>>> field = OBS;
>>>>>> vld_thresh = 1.0;
>>>>>>
>>>>>> type = [
>>>>>> {
>>>>>> method = BILIN;
>>>>>> width = 2;
>>>>>> }
>>>>>> ];
>>>>>> };
>>>>>>
>>>>>>
/////////////////////////////////////////////////////////////////////
>>>>>> ///////////
>>>>>>
>>>>>> //
>>>>>> // Statistical output types
>>>>>> //
>>>>>> output_flag = {
>>>>>> rhist = STAT;
>>>>>> orank = NONE;
>>>>>> };
>>>>>>
>>>>>>
/////////////////////////////////////////////////////////////////////
>>>>>> ///////////
>>>>>>
>>>>>> //
>>>>>> // Ensemble product output types
>>>>>> //
>>>>>> ensemble_flag = {
>>>>>> mean = TRUE;
>>>>>> stdev = TRUE;
>>>>>> minus = FALSE;
>>>>>> plus = FALSE;
>>>>>> min = FALSE;
>>>>>> max = FALSE;
>>>>>> range = FALSE;
>>>>>> vld_count = FALSE;
>>>>>> frequency = FALSE;
>>>>>> rank = TRUE;
>>>>>> };
>>>>>>
>>>>>>
/////////////////////////////////////////////////////////////////////
>>>>>> ///////////
>>>>>>
>>>>>> //
>>>>>> // Random number generator
>>>>>> //
>>>>>> rng = {
>>>>>> type = "mt19937";
>>>>>> seed = "1";
>>>>>> }
>>>>>>
>>>>>>
/////////////////////////////////////////////////////////////////////
>>>>>> ///////////
>>>>>>
>>>>>> duplicate_flag = NONE;
>>>>>> output_prefix = "";
>>>>>> version = "V4.0";
>>>>>>
>>>>>>
/////////////////////////////////////////////////////////////////////
>>>>>> ///////////
>>>>>> ==
>>>>>> Obs file: /h/data/global/WXQC/data/obs_prep/2013011412
>>>>>> DEBUG 1: Default Config File:
>>>>>> /home/qcteam/METv4.0/data/config/EnsembleStatConfig_default
>>>>>> DEBUG 1: User Config File:
/home/qcteam/METv4.0/cfg/ensemble_stat.cfg
>>>>>> DEBUG 1: Ensemble Files[10]:
>>>>>> DEBUG 1:
>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>> _d02/WRFPRS24_d02.1.tm00
>>>>>> DEBUG 1:
>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>> _d02/WRFPRS24_d02.2.tm00
>>>>>> DEBUG 1:
>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>> _d02/WRFPRS24_d02.3.tm00
>>>>>> DEBUG 1:
>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>> _d02/WRFPRS24_d02.4.tm00
>>>>>> DEBUG 1:
>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>> _d02/WRFPRS24_d02.5.tm00
>>>>>> DEBUG 1:
>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>> _d02/WRFPRS24_d02.6.tm00
>>>>>> DEBUG 1:
>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>> _d02/WRFPRS24_d02.7.tm00
>>>>>> DEBUG 1:
>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>> _d02/WRFPRS24_d02.8.tm00
>>>>>> DEBUG 1:
>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>> _d02/WRFPRS24_d02.9.tm00
>>>>>> DEBUG 1:
>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>> _d02/WRFPRS24_d02.10.tm00
>>>>>> DEBUG 1: Point Observation Files[1]:
>>>>>> DEBUG 1: /h/data/global/WXQC/data/obs_prep/2013011412
>>>>>> DEBUG 2:
>>>>>> DEBUG 2:
>>>>>>
---------------------------------------------------------------------
>>>>>> -----------
>>>>>> DEBUG 2:
>>>>>> DEBUG 2: Processing ensemble field: TMP/Z2 DEBUG 3:
>>>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
>>>>>> DEBUG 2:
>>>>>> DEBUG 2:
>>>>>>
---------------------------------------------------------------------
>>>>>> -----------
>>>>>> DEBUG 2:
>>>>>> DEBUG 2: Processing point observation file:
>>>>>> /h/data/global/WXQC/data/obs_prep/2013011412
>>>>>> DEBUG 2: Searching 219303 observations from 40233 header
messages.
>>>>>> DEBUG 2:
>>>>>> DEBUG 2:
>>>>>>
---------------------------------------------------------------------
>>>>>> -----------
>>>>>> DEBUG 2:
>>>>>> DEBUG 2: Processing ensemble file:
>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>> _d02/WRFPRS24_d02.1.tm00 DEBUG 3:
>>>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>>>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>>>> DEBUG 2:
>>>>>> DEBUG 2:
>>>>>>
---------------------------------------------------------------------
>>>>>> -----------
>>>>>> DEBUG 2:
>>>>>> DEBUG 2: Processing ensemble file:
>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>> _d02/WRFPRS24_d02.2.tm00 DEBUG 3:
>>>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
>>>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>>>> DEBUG 2:
>>>>>> DEBUG 2:
>>>>>>
---------------------------------------------------------------------
>>>>>> -----------
>>>>>> DEBUG 2:
>>>>>> DEBUG 2: Processing ensemble file:
>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>> _d02/WRFPRS24_d02.3.tm00 DEBUG 3:
>>>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
>>>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>>>> DEBUG 2:
>>>>>> DEBUG 2:
>>>>>>
---------------------------------------------------------------------
>>>>>> -----------
>>>>>> DEBUG 2:
>>>>>> DEBUG 2: Processing ensemble file:
>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>> _d02/WRFPRS24_d02.4.tm00 DEBUG 3:
>>>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
>>>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>>>> DEBUG 2:
>>>>>> DEBUG 2:
>>>>>>
---------------------------------------------------------------------
>>>>>> -----------
>>>>>> DEBUG 2:
>>>>>> DEBUG 2: Processing ensemble file:
>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>> _d02/WRFPRS24_d02.5.tm00 DEBUG 3:
>>>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
>>>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>>>> DEBUG 2:
>>>>>> DEBUG 2:
>>>>>>
---------------------------------------------------------------------
>>>>>> -----------
>>>>>> DEBUG 2:
>>>>>> DEBUG 2: Processing ensemble file:
>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>> _d02/WRFPRS24_d02.6.tm00 DEBUG 3:
>>>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
>>>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>>>> DEBUG 2:
>>>>>> DEBUG 2:
>>>>>>
---------------------------------------------------------------------
>>>>>> -----------
>>>>>> DEBUG 2:
>>>>>> DEBUG 2: Processing ensemble file:
>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>> _d02/WRFPRS24_d02.7.tm00 DEBUG 3:
>>>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
>>>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>>>> DEBUG 2:
>>>>>> DEBUG 2:
>>>>>>
---------------------------------------------------------------------
>>>>>> -----------
>>>>>> DEBUG 2:
>>>>>> DEBUG 2: Processing ensemble file:
>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>> _d02/WRFPRS24_d02.8.tm00 DEBUG 3:
>>>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
>>>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>>>> DEBUG 2:
>>>>>> DEBUG 2:
>>>>>>
---------------------------------------------------------------------
>>>>>> -----------
>>>>>> DEBUG 2:
>>>>>> DEBUG 2: Processing ensemble file:
>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>> _d02/WRFPRS24_d02.9.tm00 DEBUG 3:
>>>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
>>>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>>>> DEBUG 2:
>>>>>> DEBUG 2:
>>>>>>
---------------------------------------------------------------------
>>>>>> -----------
>>>>>> DEBUG 2:
>>>>>> DEBUG 2: Processing ensemble file:
>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>> _d02/WRFPRS24_d02.10.tm00 DEBUG 3:
>>>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
>>>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>>>> DEBUG 2:
>>>>>> DEBUG 2:
>>>>>>
---------------------------------------------------------------------
>>>>>> -----------
>>>>>> DEBUG 2:
>>>>>> DEBUG 2: Processing point verification TMP/Z2 versus TMP/Z2,
for
>>>>>> observation type ADPSFC, over region FULL, for interpolation
method
>>>>>> BILIN(4), using 0 pairs.
>>>>>> DEBUG 2:
>>>>>> DEBUG 2:
>>>>>>
---------------------------------------------------------------------
>>>>>> -----------
>>>>>> DEBUG 2:
>>>>>> DEBUG 1: Output file:
>>>>>>
/home/qcteam/METv4.0/out/ensemble_stat/ensemble_stat_20130114_120000V
>>>>>> .stat
>>>>>> DEBUG 1: Output file:
>>>>>>
/home/qcteam/METv4.0/out/ensemble_stat/ensemble_stat_20130114_120000V
>>>>>> _ens.nc
>>>>>>
>>>>>> // SIGNED //
>>>>>> Matthew C. Sittel
>>>>>> Associate Scientist
>>>>>> University Corporation for Atmospheric Research 16WS/WXN,
Offutt AFB,
>>>>>> NE
>>>>>> (402) 294-3473 DSN: 271-3473
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>>
>>
>>
>>
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #59890] Problem with Pairing NetCDF with GriB
From: John Halley Gotway
Time: Fri Jan 18 12:34:11 2013
Matt,
Thanks for the tip. I've added a note to the website about this
issue:
http://www.dtcenter.org/met/users/downloads/observation_data.php
John
On 01/18/2013 12:13 PM, matthew.sittel.ctr at offutt.af.mil via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59890 >
>
> Yep, that was it. Almost 10,000 obs now for 00Z. I'd add that 'set
it to 9' gotcha to the web page, as a heads up to others who might run
into that problem.
>
> Thanks for helping us find the needle in the MET haystack!
>
> Matt
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Friday, January 18, 2013 1:03 PM
> To: Sittel, Matthew C CTR USAF AFWA 16 WS/WXN
> Cc: Craig, Robert J Civ USAF AFWA 16 WS/WXN
> Subject: Re: [rt.rap.ucar.edu #59890] Problem with Pairing NetCDF
with GriB
>
> Matt,
>
> One other item to note. In METv4.0, we are *not* passing the
quality flag value through PB2NC for each observation.
>
> However, one enhancement in METv4.1, that's currently under
development, is to pass the quality flag value through PB2NC. Then in
Point-Stat and Ensemble-Stat, you can specify in the config file
> which quality flags you'd like to use. In the case of this GDAS
data, you may choose to verify your forecast against observations with
quality values of 1, 2, and 9, skipping everything between 2 and 9.
>
> John
>
> On 01/18/2013 11:09 AM, John Halley Gotway wrote:
>> Matt,
>>
>> First, I'm glad you were able to get it working.
>>
>> Next, if you have point internal observations that you'd like to
use,
>> rather than trying to write a NetCDF point observation file
directly, I'd
>> suggest using the ASCII2NC tool. As long as you can format your
point
>> observations in the simple ASCII format that it expects, running
ASCII2NC
>> will result in a NetCDF file the other MET tools can handle.
>>
>> Regarding NDAS vs GDAS - NDAS PrepBufr files contain point
observations
>> used in the data assimilation process for NAM. GDAS contains point
>> observation data used in data assimilation for GFS.
>>
>> You'd expect NDAS to be limited to North America, but it does
actually
>> contain many obs over Europe and the pacific. Generally speaking
though,
>> use NDAS for domains over North America and GDAS for domains
outside North
>> America.
>>
>> Here's one important issue though - and I suspect this is what
you're
>> encountering - in GDAS, NCEP sets the quality flag for surface obs
to a
>> value of 9. They do that to prevent their use in the data
assimilation
>> process because it makes the results of the data assimilation
worse.
>>
>> Try cranking up the quality flag in PB2NC to 9 to see if 2-meter
>> temperature is retained.
>>
>> John
>>
>>>
>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59890 >
>>>
>>> I pushed a file via FTP to you... it's a prepbufr format file:
>>> gdas1.t06z.prepbufr.nr. Could you run PB2NC on it on your side
and see if
>>> you agree with us in that there are no ADPSFC temperature (grib
field 11)
>>> observations in the file? For some reason all we get are grib ID
7,
>>> height data, for ADPSFC and nothing else.
>>>
>>> Without reading the prepbufr data, we can't be sure if there is
>>> temperature data that gets tossed based on the config file
settings, or it
>>> simply isn't in the prepbufr file at all, which means we can't
really use
>>> it for our application. That would be an important caveat for
users.
>>>
>>> Thanks,
>>> Matt
>>>
>>> -----Original Message-----
>>> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
>>> Sent: Friday, January 18, 2013 9:20 AM
>>> To: Sittel, Matthew C CTR USAF AFWA 16 WS/WXN
>>> Cc: Craig, Robert J Civ USAF AFWA 16 WS/WXN
>>> Subject: Re: [rt.rap.ucar.edu #59890] Problem with Pairing NetCDF
with
>>> GriB
>>>
>>> Matt,
>>>
>>> Good news. The problem was in your point_stat.cfg file. Diffing
it
>>> against the default one in
METv4.0/data/config/PointStatConfig_default, I
>>> see the following:
>>> < poly = "";
>>> ---
>>> > poly = [];
>>>
>>> In the masking section, it expects "poly" to be specified as an
array.
>>> Instead of specifying it as an empty array, you specified it as a
string
>>> of length 0. So it tried to read a file of that name.
>>> Running METv4.0 with the latest set of patches at verbosity
level 4,
>>> gave the following helpful information:
>>>
>>> DEBUG 2: Reading data for TMP/Z2.
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
>>> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
"WRFPRS24_d02.1.tm00".
>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>> matching VarInfo "TMP/Z2" in GRIB file "WRFPRS24_d02.1.tm00".
>>> DEBUG 4: parse_sid_mask() -> parsing station ID mask file "(nul)"
>>> DEBUG 4: parse_grid_mask() -> parsing grid mask "FULL"
>>> DEBUG 4: parse_poly_mask() -> parsing poly mask "(nul)"
>>> ERROR :
>>> ERROR : grd_file_type() -> file does not exist "(nul)"
>>> ERROR :
>>>
>>> However, running the same thing for METv4.0 as originally released
results
>>> in a seg fault. Yeah, we found it!
>>>
>>> Looking at the list of fixes, I'm guessing it's the bug fix posted
on
>>> 08/01/2012:
>>> "Fixed a bug that caused a seg fault when a config file
string list
>>> contains only an empty string"
>>>
http://www.dtcenter.org/met/users/support/known_issues/METv4.0/index.php
>>>
>>> So just edit your Point-Stat config file to use:
>>> poly = [];
>>>
>>> And then recompile METv4.0 after applying the latest set of
patches.
>>>
>>> Hopefully that'll do the trick. Just let us know if you have more
>>> problems or questions.
>>>
>>> Thanks,
>>> John
>>>
>>>
>>>
>>> On 01/18/2013 07:53 AM, matthew.sittel.ctr at offutt.af.mil via RT
wrote:
>>>>
>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59890 >
>>>>
>>>> Hi John,
>>>>
>>>> I've added the point_stat config file to the FTP directory with
the
>>>> other files.
>>>>
>>>> I have not downloaded any of the patches for v4.0. I thought I
had but
>>>> turns out it was for v3.1 back on May 29 '12.
>>>>
>>>> Bob has been looking at the PB2NC process. The obs plotting
capability
>>>> has been helpful.
>>>>
>>>> I'll recompile and let you know if it helps.
>>>>
>>>> Thanks,
>>>> Matt
>>>>
>>>> -----Original Message-----
>>>> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
>>>> Sent: Thursday, January 17, 2013 4:42 PM
>>>> To: Sittel, Matthew C CTR USAF AFWA 16 WS/WXN
>>>> Cc: Craig, Robert J Civ USAF AFWA 16 WS/WXN
>>>> Subject: RE: [rt.rap.ucar.edu #59890] Problem with Pairing NetCDF
with
>>>> GriB
>>>>
>>>> Matt,
>>>>
>>>> Yeah, the ncdump thing is silly. The older version in my path
complains
>>>> about leading numbers, but I believe that later versions allow
it.
>>>> Guess
>>>> I need to update my path!
>>>>
>>>> I'm not at my desk this afternoon, so am unable to debug until
tomorrow
>>>> morning. But I'm still puzzled about the source of that seg
fault. I'd
>>>> like to be able to replicate it to address it.
>>>>
>>>> Could you please do 2 things:
>>>> - Send me the exact point-stat config file you're using:
>>>> /home/qcteam/METv4.0/cfg/point_stat.cfg
>>>> - Let me know the date of the latest set of patches you're using
for
>>>> METv4.0. Just take a look in /home/qcteam/METv4.0 to see if
there's a
>>>> patch file in there with a date in the filename.
>>>>
>>>> Hopefully, if I run your data with your config file and the exact
same
>>>> version of the code, I'll be able to reproduce it. If not, it's
a lot
>>>> more difficult problem.
>>>>
>>>> You could always try applying the latest set of METv4.0 patches
and
>>>> recompiling to see if that helps:
>>>>
http://www.dtcenter.org/met/users/support/known_issues/METv4.0/index.php
>>>>
>>>> Regarding why I got 0 matched pairs even when accounting for the
time
>>>> offset, I think looking at the PB2NC configuration is a great
idea. In
>>>> particular, look to see if you're filtering on the observation qc
flag.
>>>> If you also send me your PB2NC config file and the location of
the
>>>> PREPBUFR files you're using, I'd be happy to take a look at that
too.
>>>>
>>>> Thanks,
>>>> John
>>>>
>>>>>
>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59890 >
>>>>>
>>>>> Hi John,
>>>>>
>>>>> I put an updated obs file onto the FTP site with correct time
>>>>> stamps-overwrote the old one. Talk about confusing. The file I
sent
>>>>> you
>>>>> was later overwritten by a new file so when I looked at it, the
date
>>>>> fields were great. Bob will check into why our process pulls
data from
>>>>> one date and saves it as another.
>>>>>
>>>>> Weird, ncdump accepts numerical file names for me... were you
running
>>>>> this
>>>>> in a Linux environment? Not blaming Linux, but it's always a
good
>>>>> place
>>>>> to start...
>>>>>
>>>>> Here's the output from my point_stat run of member 1 vs. the
>>>>> observation
>>>>> file:
>>>>>
>>>>> =======
>>>>> /home/qcteam/METv4.0/bin/point_stat
>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011412/GRIB_OUT_d02/WRFPRS24_d02.1.tm00
>>>>> /h/data/global/WXQC/data/obs_prep/2013011512
>>>>> /home/qcteam/METv4.0/cfg/point_stat.cfg -outdir
>>>>> /home/qcteam/METv4.0/out
>>>>> -v 4
>>>>>
>>>>> DEBUG 1: Default Config File:
>>>>> /home/qcteam/METv4.0/data/config/PointStatConfig_default
>>>>> DEBUG 1: User Config File:
/home/qcteam/METv4.0/cfg/point_stat.cfg
>>>>> DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created
new
>>>>> Met2dDataFile object of type "FileType_Gb1".
>>>>> DEBUG 4: VarInfoFactory::new_var_info() -> created new VarInfo
object
>>>>> of
>>>>> type "FileType_Gb1".
>>>>> GSL_RNG_TYPE=mt19937
>>>>> GSL_RNG_SEED=1
>>>>> DEBUG 1: Forecast File:
>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011412/GRIB_OUT_d02/WRFPRS24_d02.1.tm00
>>>>> DEBUG 1: Climatology File: none
>>>>> DEBUG 1: Observation File:
/h/data/global/WXQC/data/obs_prep/2013011512
>>>>> DEBUG 2:
>>>>> DEBUG 2:
>>>>>
--------------------------------------------------------------------------------
>>>>> DEBUG 2:
>>>>> DEBUG 2: Reading data for TMP/Z2.
>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match for
>>>>> VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011412/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011412/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>>>>> Memory fault
>>>>> =======
>>>>>
>>>>> I've never tried plot_point_obs-I don't see reference to it in
the 4.0
>>>>> users guide.
>>>>>
>>>>> Okay, by typing the executable it told me the arguments I
needed. Just
>>>>> ran it... 33208 obs for ADPSFC type, 3102 obs for gc 11, but
none for
>>>>> the
>>>>> intersect of the two... weird.
>>>>>
>>>>> We'll keep digging and see what we can find... I wonder if PB2NC
is
>>>>> throwing out all the stuff we need based on the settings we put
in the
>>>>> config file.
>>>>>
>>>>> Thanks,
>>>>> Matt
>>>>>
>>>>> -----Original Message-----
>>>>> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
>>>>> Sent: Wednesday, January 16, 2013 4:27 PM
>>>>> To: Sittel, Matthew C CTR USAF AFWA 16 WS/WXN
>>>>> Cc: Craig, Robert J Civ USAF AFWA 16 WS/WXN
>>>>> Subject: RE: [rt.rap.ucar.edu #59890] Problem with Pairing
NetCDF with
>>>>> GriB
>>>>>
>>>>> Matt,
>>>>>
>>>>> I ran METv4.0 point_stat for the first ensemble member using the
>>>>> attached
>>>>> configuration file and verbosity level 3. I got 0 matched
pairs, but
>>>>> the
>>>>> following diagnostic information is included in the output:
>>>>>
>>>>> DEBUG 2: Processing TMP/Z2 versus TMP/Z2, for observation type
ADPSFC,
>>>>> over region FULL, for interpolation method BILIN(4), using 0
pairs.
>>>>> DEBUG 3: Number of matched pairs = 0
>>>>> DEBUG 3: Observations processed = 219303
>>>>> DEBUG 3: Rejected: GRIB code = 190302
>>>>> DEBUG 3: Rejected: valid time = 29001
>>>>> DEBUG 3: Rejected: bad obs value = 0
>>>>> DEBUG 3: Rejected: off the grid = 0
>>>>> DEBUG 3: Rejected: level mismatch = 0
>>>>> DEBUG 3: Rejected: message type = 0
>>>>> DEBUG 3: Rejected: masking region = 0
>>>>> DEBUG 3: Rejected: bad fcst value = 0
>>>>>
>>>>> This tells me that there's problem in the matching time window.
>>>>>
>>>>> Running "wgrib -V" on that forecast file tells me that this is a
>>>>> 24-hour
>>>>> forecast initialized at 2013011412. So it's valid at
2013011512.
>>>>>
>>>>> Your observation file is named 2013011512, but when you actually
look
>>>>> at
>>>>> the times it contains, they're for 2013011412:
>>>>> ncdump -v hdr_vld nc2013011512
>>>>>
>>>>> (note, ncdump doesn't like file names that begin with a number,
so I
>>>>> stuck
>>>>> an "nc" in front)
>>>>>
>>>>> I reran point-stat a second time, but first added 3 0's to the
"beg"
>>>>> and
>>>>> "end" times in the "obs_window" setting. When I did that, I got
>>>>> further,
>>>>> but still no matched pairs:
>>>>>
>>>>> DEBUG 2: Processing TMP/Z2 versus TMP/Z2, for observation type
ADPSFC,
>>>>> over region FULL, for interpolation method BILIN(4), using 0
pairs.
>>>>> DEBUG 3: Number of matched pairs = 0
>>>>> DEBUG 3: Observations processed = 219303
>>>>> DEBUG 3: Rejected: GRIB code = 190302
>>>>> DEBUG 3: Rejected: valid time = 0
>>>>> DEBUG 3: Rejected: bad obs value = 0
>>>>> DEBUG 3: Rejected: off the grid = 22047
>>>>> DEBUG 3: Rejected: level mismatch = 6177
>>>>> DEBUG 3: Rejected: message type = 777
>>>>> DEBUG 3: Rejected: masking region = 0
>>>>> DEBUG 3: Rejected: bad fcst value = 0
>>>>>
>>>>> So there just aren't any ADPSFC observations of 2-m temperature
that
>>>>> fall
>>>>> within your d02 domain. I ran the plot_point_obs utility to
verify
>>>>> that
>>>>> that's the case.
>>>>>
>>>>> As for why you got a seg-fault, I don't know. It's possible
that you
>>>>> made
>>>>> an error in your Point-Stat config file that caused it. I can't
really
>>>>> debug it unless I can replicate it.
>>>>>
>>>>> John
>>>>>
>>>>>>
>>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59890 >
>>>>>>
>>>>>> Hi John,
>>>>>>
>>>>>> The plot thickens. When I ran point_stat on one ensemble
member vs.
>>>>>> an observation file, MET returned a memory fault when it got to
the
>>>>>> observation file read. I tried a second observation file and
got the
>>>>>> same result.
>>>>>>
>>>>>> I looked using 'ncdump' on the NetCDF observation data output
from
>>>>>> 'pb2nc', and confirmed there are ADPSFC records, records within
the
>>>>>> CONUS grid and in the requested time window.
>>>>>>
>>>>>> I used "-v 3" verbosity in the example I sent you... I think 4
is the
>>>>>> highest level of message? I tried rerunning this morning with
"-v 6"
>>>>>> and didn't get any additional information as to what is
happening with
>>>>>> the observation matching, nor any messages numbered above 4.
>>>>>>
>>>>>> I have pushed the 10 ensemble members, observation file and
config
>>>>>> file to the directory you listed. I had to create a "met_help"
>>>>>> directory under 'irap' as one was not there. Inside 'sittel'
you will
>>>>>> find the files.
>>>>>> Please note these were created in a Linux environment-not sure
if that
>>>>>> will be an issue for you; it might depend on your OS. We were
forced
>>>>>> to go to Linux here.
>>>>>>
>>>>>> Thanks,
>>>>>> Matt
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
>>>>>> Sent: Tuesday, January 15, 2013 4:10 PM
>>>>>> To: Sittel, Matthew C CTR USAF AFWA 16 WS/WXN
>>>>>> Cc: Craig, Robert J Civ USAF AFWA 16 WS/WXN
>>>>>> Subject: Re: [rt.rap.ucar.edu #59890] Problem with Pairing
NetCDF with
>>>>>> GriB
>>>>>>
>>>>>> Matt,
>>>>>>
>>>>>> So the question is basically, why are you getting 0 matched
pairs from
>>>>>> ensemble-stat. If this were the point-stat tool, the answer
would be
>>>>>> easy. In point-stat, when you turn up the verbosity level, for
each
>>>>>> verification task, a list of reasons for why observations were
not
>>>>>> used is printed out. And that's often an easy way to diagnose
the
>>>>>> problem.
>>>>>>
>>>>>> But that functionality has not been implemented for ensemble-
stat.
>>>>>> Though the problem you're having demonstrates the need!
>>>>>>
>>>>>> There are several possibilities:
>>>>>> - the observations may not fall within the matching time
window
>>>>>> - the observations may not fall within the model's grid
>>>>>> - perhaps there are no ADPSFC observations of TMP in your
dataset
>>>>>>
>>>>>> There are different steps I'd take to rule out each of these
cases -
>>>>>> mostly making use of the plot_data_plane and plot_point_obs
utilities.
>>>>>> But perhaps the easiest thing to do is to have you run one of
your
>>>>>> forecast files and the NetCDF output of PB2NC through point-
stat to
>>>>>> verify TMP/Z2.
>>>>>>
>>>>>> Presumably, you'll get 0 matched pairs again. But when you
increase
>>>>>> the verbosity level (-v 3, I think), you'll see a list of
reasons why
>>>>>> the obs weren't used. Send me that list and then we'll know
how to
>>>>>> proceed.
>>>>>>
>>>>>> Alternatively, you could just send me one or two of your GRIB
files,
>>>>>> the NetCDF point observation file, and your config file, and I
could
>>>>>> debug it here. If you choose to go that route, follow the
>>>>>> instructions
>>>>>> here:
>>>>>>
http://www.dtcenter.org/met/users/support/met_help.php#ftp
>>>>>>
>>>>>> Your config file settings look fine. The instructions for
specifying
>>>>>> NetCDF config file stuff only applies to gridded NetCDF data.
But
>>>>>> your gridded data is in GRIB format. For point observations,
we chose
>>>>>> the convention of specifying them as if they were GRIB files.
That's
>>>>>> why we include a GRIB code for each observation value in the
data.
>>>>>>
>>>>>> Just let me know which route you decide on. And I'm happy to
help get
>>>>>> you up and running.
>>>>>>
>>>>>> Thanks,
>>>>>> John
>>>>>>
>>>>>>>
>>>>>>> Tue Jan 15 09:22:51 2013: Request 59890 was acted upon.
>>>>>>> Transaction: Ticket created by
matthew.sittel.ctr at offutt.af.mil
>>>>>>> Queue: met_help
>>>>>>> Subject: Problem with Pairing NetCDF with GriB
>>>>>>> Owner: Nobody
>>>>>>> Requestors: matthew.sittel.ctr at offutt.af.mil
>>>>>>> Status: new
>>>>>>> Ticket <URL:
>>>>>>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59890
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I'm trying to get MET v4.0 to create verification rank
histogram
>>>>>>> output for AFWA's mesoscale ensemble. I'm using in-house,
GriB-1
>>>>>>> format WRF ensemble member output and verifying it with
>>>>>>> PREPBUFR-formatted obs that we are pulling via FTP from NCEP.
We
>>>>>>> convert the PREPBUFR format to NetCDF using 'pb2nc'.
>>>>>>>
>>>>>>> The problem I'm having is I cannot figure out what the correct
>>>>>>> settings are in the config file for ensemble_stat to get the
model
>>>>>>> and observation fields to match up.
>>>>>>>
>>>>>>> I've attached the NetCDF 'dump' of the observation file's
contents,
>>>>>>> the contents of my config file and the output I get when I run
>>>>>>> ensemble_stat.
>>>>>>> No matter what settings I pick, it always comes back with
0 pairs
>>>>>>> of
>>>>>>> data. The output suggests the model members are read in and
that
>>>>>>> there are records in the observation file as well-both
suggesting to
>>>>>>> me the data are read in correctly, but I'm not designating the
>>>>>>> correct pairing of them with the config file parameters.
>>>>>>>
>>>>>>> The records I am trying to get are the ones where gc=11 and
hgt=0, or
>>>>>>> simply the surface temperature reports from "ADPSFC" message
type. I
>>>>>>> have read the user's guide and can't figure out what the right
>>>>>>> settings are for NetCDF; it is discussed briefly but not with
enough
>>>>>>> detail to figure this out on my own, especially the ANNN,
RNNN, etc,
>>>>>>> and the TMP(I,...,J,*,*) wording which is unclear. Can you
take a
>>>>>>> look and advise as to the right settings?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Matt
>>>>>>>
>>>>>>> ==
>>>>>>> netcdf 2013011412 {
>>>>>>> dimensions:
>>>>>>> mxstr = 16 ;
>>>>>>> hdr_arr_len = 3 ;
>>>>>>> obs_arr_len = 5 ;
>>>>>>> nobs = UNLIMITED ; // (219303 currently)
>>>>>>> nhdr = 40233 ;
>>>>>>> variables:
>>>>>>> float obs_arr(nobs, obs_arr_len) ;
>>>>>>> obs_arr:long_name = "array of observation
values" ;
>>>>>>> obs_arr:_fill_value = -9999.f ;
>>>>>>> obs_arr:columns = "hdr_id gc lvl hgt ob" ;
>>>>>>> obs_arr:hdr_id_long_name = "index of
matching header
>>>>>>> data"
>>>>>>> ;
>>>>>>> obs_arr:gc_long_name = "grib code
corresponding to
>>>>>>> the observation type" ;
>>>>>>> obs_arr:lvl_long_name = "pressure level
(hPa) or
>>>>>>> accumulation interval (sec)" ;
>>>>>>> obs_arr:hgt_long_name = "height in meters
above sea
>>>>>>> level (msl)" ;
>>>>>>> obs_arr:ob_long_name = "observation value"
;
>>>>>>> char hdr_typ(nhdr, mxstr) ;
>>>>>>> hdr_typ:long_name = "message type" ;
>>>>>>> char hdr_sid(nhdr, mxstr) ;
>>>>>>> hdr_sid:long_name = "station
identification" ;
>>>>>>> char hdr_vld(nhdr, mxstr) ;
>>>>>>> hdr_vld:long_name = "valid time" ;
>>>>>>> hdr_vld:units = "YYYYMMDD_HHMMSS" ;
>>>>>>> float hdr_arr(nhdr, hdr_arr_len) ;
>>>>>>> hdr_arr:long_name = "array of observation
station
>>>>>>> header values" ;
>>>>>>> hdr_arr:_fill_value = -9999.f ;
>>>>>>> hdr_arr:columns = "lat lon elv" ;
>>>>>>> hdr_arr:lat_long_name = "latitude" ;
>>>>>>> hdr_arr:lat_units = "degrees_north" ;
>>>>>>> hdr_arr:lon_long_name = "longitude" ;
>>>>>>> hdr_arr:lon_units = "degrees_east" ;
>>>>>>> hdr_arr:elv_long_name = "elevation" ;
>>>>>>> hdr_arr:elv_units = "meters above sea level
(msl)" ;
>>>>>>>
>>>>>>> // global attributes:
>>>>>>> :FileOrigins = "File
>>>>>>> /h/data/global/WXQC/data//obs_prep/2013011412 generated
>>>>>>> 20130115_003309 UTC on host dev8dlsn1 by the MET pb2nc tool" ;
>>>>>>> :MET_version = "V4.0" ;
>>>>>>> :MET_tool = "pb2nc" ;
>>>>>>> data:
>>>>>>> 40013, 7, 1016.4, 0, 0,
>>>>>>> 40014, 7, 1012.2, 0, 0,
>>>>>>> 40015, 7, 1024.4, 0, 0,
>>>>>>> 40016, 11, 1030.6, 0, 236.55,
>>>>>>> 40016, 7, 1030.6, 0, 0,
>>>>>>> 40016, 2, 1030.6, 0, 103060,
>>>>>>> 40017, 11, 1030.6, 0, 236.55,
>>>>>>> 40017, 7, 1030.6, 0, 0,
>>>>>>> 40017, 2, 1030.6, 0, 103060,
>>>>>>> 40018, 11, 1030.9, 0, 235.95,
>>>>>>> 40018, 7, 1030.9, 0, 0,
>>>>>>> 40018, 2, 1030.9, 0, 103090,
>>>>>>> 40019, 11, 1032.9, 0, 239.85,
>>>>>>> 40019, 7, 1032.9, 0, 0,
>>>>>>> 40019, 2, 1032.9, 0, 103290,
>>>>>>> 40020, 11, 1034.1, 0, 240.65,
>>>>>>> 40020, 7, 1034.1, 0, 0,
>>>>>>> 40020, 2, 1034.1, 0, 103410,
>>>>>>>
>>>>>>>
/////////////////////////////////////////////////////////////////////
>>>>>>> ///////////
>>>>>>> //
>>>>>>> // Ensemble-Stat configuration file.
>>>>>>> //
>>>>>>> // For additional information, see the
MET_BASE/data/config/README
>>>>>>> file.
>>>>>>> //
>>>>>>>
/////////////////////////////////////////////////////////////////////
>>>>>>> ///////////
>>>>>>>
>>>>>>> //
>>>>>>> // Output model name to be written
>>>>>>> //
>>>>>>> model = "WRF";
>>>>>>>
>>>>>>>
/////////////////////////////////////////////////////////////////////
>>>>>>> ///////////
>>>>>>>
>>>>>>> //
>>>>>>> // Ensemble product fields to be processed // ens = {
>>>>>>> ens_thresh = 1.0;
>>>>>>> vld_thresh = 1.0;
>>>>>>>
>>>>>>> field = [
>>>>>>> {
>>>>>>> name = "TMP";
>>>>>>> level = [ "Z2" ];
>>>>>>> cat_thresh = [ >=300.0 ];
>>>>>>> }
>>>>>>> ];
>>>>>>> }
>>>>>>>
>>>>>>>
/////////////////////////////////////////////////////////////////////
>>>>>>> ///////////
>>>>>>>
>>>>>>> //
>>>>>>> // Forecast and observation fields to be verified // fcst = {
>>>>>>> message_type = [ "ADPSFC" ];
>>>>>>> field = [
>>>>>>> {
>>>>>>> name = "TMP";
>>>>>>> level = "Z2";
>>>>>>> }
>>>>>>> ];
>>>>>>> }
>>>>>>>
>>>>>>> obs=fcst;
>>>>>>>
>>>>>>>
/////////////////////////////////////////////////////////////////////
>>>>>>> ///////////
>>>>>>>
>>>>>>> //
>>>>>>> // Point observation time window
>>>>>>> //
>>>>>>> obs_window = {
>>>>>>> beg = -1800;
>>>>>>> end = 1800;
>>>>>>> }
>>>>>>>
>>>>>>>
/////////////////////////////////////////////////////////////////////
>>>>>>> ///////////
>>>>>>>
>>>>>>> //
>>>>>>> // Verification masking regions
>>>>>>> //
>>>>>>> mask = {
>>>>>>> grid = [ "FULL" ];
>>>>>>> poly = [];
>>>>>>> sid = "";
>>>>>>> };
>>>>>>>
>>>>>>>
/////////////////////////////////////////////////////////////////////
>>>>>>> ///////////
>>>>>>>
>>>>>>> //
>>>>>>> // Interpolation methods
>>>>>>> //
>>>>>>> interp = {
>>>>>>> field = OBS;
>>>>>>> vld_thresh = 1.0;
>>>>>>>
>>>>>>> type = [
>>>>>>> {
>>>>>>> method = BILIN;
>>>>>>> width = 2;
>>>>>>> }
>>>>>>> ];
>>>>>>> };
>>>>>>>
>>>>>>>
/////////////////////////////////////////////////////////////////////
>>>>>>> ///////////
>>>>>>>
>>>>>>> //
>>>>>>> // Statistical output types
>>>>>>> //
>>>>>>> output_flag = {
>>>>>>> rhist = STAT;
>>>>>>> orank = NONE;
>>>>>>> };
>>>>>>>
>>>>>>>
/////////////////////////////////////////////////////////////////////
>>>>>>> ///////////
>>>>>>>
>>>>>>> //
>>>>>>> // Ensemble product output types
>>>>>>> //
>>>>>>> ensemble_flag = {
>>>>>>> mean = TRUE;
>>>>>>> stdev = TRUE;
>>>>>>> minus = FALSE;
>>>>>>> plus = FALSE;
>>>>>>> min = FALSE;
>>>>>>> max = FALSE;
>>>>>>> range = FALSE;
>>>>>>> vld_count = FALSE;
>>>>>>> frequency = FALSE;
>>>>>>> rank = TRUE;
>>>>>>> };
>>>>>>>
>>>>>>>
/////////////////////////////////////////////////////////////////////
>>>>>>> ///////////
>>>>>>>
>>>>>>> //
>>>>>>> // Random number generator
>>>>>>> //
>>>>>>> rng = {
>>>>>>> type = "mt19937";
>>>>>>> seed = "1";
>>>>>>> }
>>>>>>>
>>>>>>>
/////////////////////////////////////////////////////////////////////
>>>>>>> ///////////
>>>>>>>
>>>>>>> duplicate_flag = NONE;
>>>>>>> output_prefix = "";
>>>>>>> version = "V4.0";
>>>>>>>
>>>>>>>
/////////////////////////////////////////////////////////////////////
>>>>>>> ///////////
>>>>>>> ==
>>>>>>> Obs file: /h/data/global/WXQC/data/obs_prep/2013011412
>>>>>>> DEBUG 1: Default Config File:
>>>>>>> /home/qcteam/METv4.0/data/config/EnsembleStatConfig_default
>>>>>>> DEBUG 1: User Config File:
/home/qcteam/METv4.0/cfg/ensemble_stat.cfg
>>>>>>> DEBUG 1: Ensemble Files[10]:
>>>>>>> DEBUG 1:
>>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>>> _d02/WRFPRS24_d02.1.tm00
>>>>>>> DEBUG 1:
>>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>>> _d02/WRFPRS24_d02.2.tm00
>>>>>>> DEBUG 1:
>>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>>> _d02/WRFPRS24_d02.3.tm00
>>>>>>> DEBUG 1:
>>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>>> _d02/WRFPRS24_d02.4.tm00
>>>>>>> DEBUG 1:
>>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>>> _d02/WRFPRS24_d02.5.tm00
>>>>>>> DEBUG 1:
>>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>>> _d02/WRFPRS24_d02.6.tm00
>>>>>>> DEBUG 1:
>>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>>> _d02/WRFPRS24_d02.7.tm00
>>>>>>> DEBUG 1:
>>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>>> _d02/WRFPRS24_d02.8.tm00
>>>>>>> DEBUG 1:
>>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>>> _d02/WRFPRS24_d02.9.tm00
>>>>>>> DEBUG 1:
>>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>>> _d02/WRFPRS24_d02.10.tm00
>>>>>>> DEBUG 1: Point Observation Files[1]:
>>>>>>> DEBUG 1: /h/data/global/WXQC/data/obs_prep/2013011412
>>>>>>> DEBUG 2:
>>>>>>> DEBUG 2:
>>>>>>>
---------------------------------------------------------------------
>>>>>>> -----------
>>>>>>> DEBUG 2:
>>>>>>> DEBUG 2: Processing ensemble field: TMP/Z2 DEBUG 3:
>>>>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
>>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
>>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
>>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
>>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
>>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
>>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
>>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
>>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
>>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
>>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
>>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
>>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
>>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
>>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
>>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
>>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range
match
>>>>>>> for VarInfo "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
>>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
>>>>>>> DEBUG 2:
>>>>>>> DEBUG 2:
>>>>>>>
---------------------------------------------------------------------
>>>>>>> -----------
>>>>>>> DEBUG 2:
>>>>>>> DEBUG 2: Processing point observation file:
>>>>>>> /h/data/global/WXQC/data/obs_prep/2013011412
>>>>>>> DEBUG 2: Searching 219303 observations from 40233 header
messages.
>>>>>>> DEBUG 2:
>>>>>>> DEBUG 2:
>>>>>>>
---------------------------------------------------------------------
>>>>>>> -----------
>>>>>>> DEBUG 2:
>>>>>>> DEBUG 2: Processing ensemble file:
>>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>>> _d02/WRFPRS24_d02.1.tm00 DEBUG 3:
>>>>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.1.tm00".
>>>>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>>>>> DEBUG 2:
>>>>>>> DEBUG 2:
>>>>>>>
---------------------------------------------------------------------
>>>>>>> -----------
>>>>>>> DEBUG 2:
>>>>>>> DEBUG 2: Processing ensemble file:
>>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>>> _d02/WRFPRS24_d02.2.tm00 DEBUG 3:
>>>>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
>>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.2.tm00".
>>>>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>>>>> DEBUG 2:
>>>>>>> DEBUG 2:
>>>>>>>
---------------------------------------------------------------------
>>>>>>> -----------
>>>>>>> DEBUG 2:
>>>>>>> DEBUG 2: Processing ensemble file:
>>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>>> _d02/WRFPRS24_d02.3.tm00 DEBUG 3:
>>>>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
>>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.3.tm00".
>>>>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>>>>> DEBUG 2:
>>>>>>> DEBUG 2:
>>>>>>>
---------------------------------------------------------------------
>>>>>>> -----------
>>>>>>> DEBUG 2:
>>>>>>> DEBUG 2: Processing ensemble file:
>>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>>> _d02/WRFPRS24_d02.4.tm00 DEBUG 3:
>>>>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
>>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.4.tm00".
>>>>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>>>>> DEBUG 2:
>>>>>>> DEBUG 2:
>>>>>>>
---------------------------------------------------------------------
>>>>>>> -----------
>>>>>>> DEBUG 2:
>>>>>>> DEBUG 2: Processing ensemble file:
>>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>>> _d02/WRFPRS24_d02.5.tm00 DEBUG 3:
>>>>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
>>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.5.tm00".
>>>>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>>>>> DEBUG 2:
>>>>>>> DEBUG 2:
>>>>>>>
---------------------------------------------------------------------
>>>>>>> -----------
>>>>>>> DEBUG 2:
>>>>>>> DEBUG 2: Processing ensemble file:
>>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>>> _d02/WRFPRS24_d02.6.tm00 DEBUG 3:
>>>>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
>>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.6.tm00".
>>>>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>>>>> DEBUG 2:
>>>>>>> DEBUG 2:
>>>>>>>
---------------------------------------------------------------------
>>>>>>> -----------
>>>>>>> DEBUG 2:
>>>>>>> DEBUG 2: Processing ensemble file:
>>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>>> _d02/WRFPRS24_d02.7.tm00 DEBUG 3:
>>>>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
>>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.7.tm00".
>>>>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>>>>> DEBUG 2:
>>>>>>> DEBUG 2:
>>>>>>>
---------------------------------------------------------------------
>>>>>>> -----------
>>>>>>> DEBUG 2:
>>>>>>> DEBUG 2: Processing ensemble file:
>>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>>> _d02/WRFPRS24_d02.8.tm00 DEBUG 3:
>>>>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
>>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.8.tm00".
>>>>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>>>>> DEBUG 2:
>>>>>>> DEBUG 2:
>>>>>>>
---------------------------------------------------------------------
>>>>>>> -----------
>>>>>>> DEBUG 2:
>>>>>>> DEBUG 2: Processing ensemble file:
>>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>>> _d02/WRFPRS24_d02.9.tm00 DEBUG 3:
>>>>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
>>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.9.tm00".
>>>>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>>>>> DEBUG 2:
>>>>>>> DEBUG 2:
>>>>>>>
---------------------------------------------------------------------
>>>>>>> -----------
>>>>>>> DEBUG 2:
>>>>>>> DEBUG 2: Processing ensemble file:
>>>>>>>
/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT
>>>>>>> _d02/WRFPRS24_d02.10.tm00 DEBUG 3:
>>>>>>> MetGrib1DataFile::data_plane_array() -> Found range match for
VarInfo
>>>>>>> "TMP/Z2" in GRIB record 103 of GRIB file
>>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
>>>>>>> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
>>>>>>> matching VarInfo "TMP/Z2" in GRIB file
>>>>>>>
"/prod9/data/global/GPOST/data/ensemble/MEPS/conus/2013011312/GRIB_OUT_d02/WRFPRS24_d02.10.tm00".
>>>>>>> DEBUG 2: For TMP/Z2 found 1 forecast levels.
>>>>>>> DEBUG 2:
>>>>>>> DEBUG 2:
>>>>>>>
---------------------------------------------------------------------
>>>>>>> -----------
>>>>>>> DEBUG 2:
>>>>>>> DEBUG 2: Processing point verification TMP/Z2 versus TMP/Z2,
for
>>>>>>> observation type ADPSFC, over region FULL, for interpolation
method
>>>>>>> BILIN(4), using 0 pairs.
>>>>>>> DEBUG 2:
>>>>>>> DEBUG 2:
>>>>>>>
---------------------------------------------------------------------
>>>>>>> -----------
>>>>>>> DEBUG 2:
>>>>>>> DEBUG 1: Output file:
>>>>>>>
/home/qcteam/METv4.0/out/ensemble_stat/ensemble_stat_20130114_120000V
>>>>>>> .stat
>>>>>>> DEBUG 1: Output file:
>>>>>>>
/home/qcteam/METv4.0/out/ensemble_stat/ensemble_stat_20130114_120000V
>>>>>>> _ens.nc
>>>>>>>
>>>>>>> // SIGNED //
>>>>>>> Matthew C. Sittel
>>>>>>> Associate Scientist
>>>>>>> University Corporation for Atmospheric Research 16WS/WXN,
Offutt AFB,
>>>>>>> NE
>>>>>>> (402) 294-3473 DSN: 271-3473
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>
>
>
------------------------------------------------
More information about the Met_help
mailing list