[Met_help] [rt.rap.ucar.edu #90256] History for Point Stat Help
John Halley Gotway via RT
met_help at ucar.edu
Thu May 23 13:37:03 MDT 2019
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Ive got my forecast field in the form of a grib file entitled
GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-c1_02700F0NL2019042612_
0100_010000-000000wnd_ucmp
This is the U Component of wind at 1000 mb. The following is a descriptor
file to read this in GrADS:
1 dset
^US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-c1_03000F0NL2019042612_0100_0100
00-000000wnd_ucmp
2 index
^US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-c1_03000F0NL2019042612_0100_0100
00-000000wnd_ucmp.idx
3 undef 9.999E+20
4 title
US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-c1_03000F0NL2019042612_0100_01000
0-000000wnd_ucmp
5 * produced by grib2ctl v0.9.15
6 * command line options:
US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-c1_03000F0NL2019042612_0100_01000
0-000000wnd_ucmp
7 dtype grib 255
8 ydef 202 levels
9 27.3280 27.4386 27.5492 27.6596 27.7699 27.8801 27.9902
10 28.1002 28.2101 28.3198 28.4295 28.5390 28.6484 28.7577
11 28.8669 28.9759 29.0849 29.1937 29.3025 29.4111 29.5196
12 29.6279 29.7362 29.8443 29.9524 30.0603 30.1681 30.2757
13 30.3833 30.4907 30.5980 30.7052 30.8123 30.9193 31.0261
14 31.1329 31.2395 31.3459 31.4523 31.5585 31.6647 31.7707
15 31.8765 31.9823 32.0879 32.1934 32.2988 32.4041 32.5093
16 32.6143 32.7192 32.8240 32.9286 33.0331 33.1375 33.2418
17 33.3460 33.4500 33.5539 33.6577 33.7614 33.8649 33.9683
18 34.0716 34.1747 34.2778 34.3807 34.4835 34.5861 34.6886
19 34.7910 34.8933 34.9954 35.0975 35.1993 35.3011 35.4027
20 35.5042 35.6056 35.7069 35.8080 35.9090 36.0099 36.1106
21 36.2112 36.3117 36.4120 36.5122 36.6123 36.7123 36.8121
22 36.9118 37.0114 37.1108 37.2101 37.3093 37.4083 37.5073
23 37.6060 37.7047 37.8032 37.9016 37.9999 38.0980 38.1960
24 38.2939 38.3916 38.4892 38.5867 38.6840 38.7812 38.8783
25 38.9752 39.0720 39.1687 39.2653 39.3617 39.4579 39.5541
26 39.6501 39.7460 39.8417 39.9373 40.0328 40.1281 40.2234
27 40.3184 40.4134 40.5082 40.6029 40.6974 40.7918 40.8861
28 40.9802 41.0742 41.1681 41.2618 41.3554 41.4489 41.5422
29 41.6354 41.7285 41.8214 41.9142 42.0069 42.0994 42.1918
30 42.2841 42.3762 42.4682 42.5600 42.6517 42.7433 42.8348
31 42.9261 43.0173 43.1083 43.1992 43.2900 43.3806 43.4711
32 43.5615 43.6517 43.7418 43.8318 43.9216 44.0113 44.1008
33 44.1902 44.2795 44.3687 44.4577 44.5466 44.6353 44.7239
34 44.8124 44.9007 44.9889 45.0770 45.1649 45.2527 45.3403
35 45.4278 45.5152 45.6025 45.6896 45.7766 45.8634 45.9501
36 46.0367 46.1231 46.2094 46.2956 46.3816 46.4675 46.5533
37 46.6389 46.7244 46.8098 46.8950 46.9801 47.0650
38 xdef 202 linear -136.710000 0.124606965174129
39 tdef 1 linear 12Z26apr2019 1mo
40 zdef 1 linear 1 1
41 vars 1
42 UGRD1000mb 0 33,100,1000 ** 1000 mb u wind [m/s]
43 ENDVARS
My ob file is in a netcdf file created using ascii2nc. I named wind speed
obs "ws" in the ascii file that I ran the ascii2nc tool on ( I am aware that
I am trying to compare Wind speed and u component of wind, but I just want
to see if I can get point_stat working).
How should I arrange my fcst and obs arrays? Here is what I have so far:
52 fcst = {
53 field = [
54 {
55 name = "U";
56 level = "P1000";
57 file_type = GRIB2;
58 cat_thresh = [ >80.0 ];
59 }
60 ];
61
62 }
63
64 obs = {
65 field = [
66 {
67 file_type = NETCDF_NCCF;
68 name = "UGRD";
69 level = ["P1000"];
70 }
71 ];
72 }
Running the command as follows gives me this error:
[tsu at maury2 00_POINT_STAT_WORK]$ point_stat
GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-c1_02700F0NL2019042612_
0100_010000-000000wnd_ucmp
NAF_sfc_2019042612.nc config.rc -v 1
DEBUG 1: Default Config File:
/software/depot/met/met-8.0_bugfix/share/met/config/PointStatConfig_default
DEBUG 1: User Config File: config.rc
GSL_RNG_TYPE=mt19937
GSL_RNG_SEED=1726142812
DEBUG 1: Forecast File:
GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-c1_02700F0NL2019042612_
0100_010000-000000wnd_ucmp
DEBUG 1: Observation File: NAF_sfc_2019042612.nc
WARNING:
WARNING: process_fcst_climo_files() -> no fields matching U/P1000 found in
file:
GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-c1_02700F0NL2019042612_
0100_010000-000000wnd_ucmp
WARNING:
ERROR :
ERROR : process_fcst_climo_files() -> no requested forecast data found!
Exiting...
ERROR :
Justin
Justin Tsu
Marine Meteorology Division
Data Assimilation/Mesoscale Modeling
Building 704 Room 212
Naval Research Laboratory, Code 7531
7 Grace Hopper Avenue
Monterey, CA 93943-5502
Ph. (831) 656-4111
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Point Stat Help
From: John Halley Gotway
Time: Mon May 20 13:19:30 2019
Hi Justin,
I see that you're having trouble reading GRIB data into the Point-Stat
tool
in MET. When getting going with a new gridded dataset in MET, I
usually
recommend starting with the plot_data_plane utility. For example, try
running this command:
plot_data_plane
GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_0270F0NL2019042612_0100_010000-000000wnd_ucmp
U_P1000.ps 'name="U"; level="P1000";'
You should get a similar error since Point-Stat wasn't able to read
it.
I'd recommend running this file through the "wgrib2" utility to
inventory
it and see what variable names/levels are present in the file. For
example, is it called "U" or "UGRD"?
If you aren't able to get that working, please just post a sample file
to
our anonymous ftp site. If you do, let me know, and I'll go grab it
and
take a look:
https://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk
Thanks,
John Halley Gotway
On Fri, May 17, 2019 at 3:27 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
wrote:
>
> Fri May 17 15:27:39 2019: Request 90256 was acted upon.
> Transaction: Ticket created by justin.tsu at nrlmry.navy.mil
> Queue: met_help
> Subject: Point Stat Help
> Owner: Nobody
> Requestors: justin.tsu at nrlmry.navy.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90256 >
>
>
> Ive got my forecast field in the form of a grib file entitled
>
>
>
>
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_02700F0NL2019042612_
> 0100_010000-000000wnd_ucmp
>
>
>
> This is the U Component of wind at 1000 mb. The following is a
descriptor
> file to read this in GrADS:
>
>
>
> 1 dset
>
> ^US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_03000F0NL2019042612_0100_0100
> 00-000000wnd_ucmp
>
> 2 index
>
> ^US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_03000F0NL2019042612_0100_0100
> 00-000000wnd_ucmp.idx
>
> 3 undef 9.999E+20
>
> 4 title
>
> US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_03000F0NL2019042612_0100_01000
> 0-000000wnd_ucmp
>
> 5 * produced by grib2ctl v0.9.15
>
> 6 * command line options:
>
> US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_03000F0NL2019042612_0100_01000
> 0-000000wnd_ucmp
>
> 7 dtype grib 255
>
> 8 ydef 202 levels
>
> 9 27.3280 27.4386 27.5492 27.6596 27.7699 27.8801
27.9902
>
> 10 28.1002 28.2101 28.3198 28.4295 28.5390 28.6484
28.7577
>
> 11 28.8669 28.9759 29.0849 29.1937 29.3025 29.4111
29.5196
>
> 12 29.6279 29.7362 29.8443 29.9524 30.0603 30.1681
30.2757
>
> 13 30.3833 30.4907 30.5980 30.7052 30.8123 30.9193
31.0261
>
> 14 31.1329 31.2395 31.3459 31.4523 31.5585 31.6647
31.7707
>
> 15 31.8765 31.9823 32.0879 32.1934 32.2988 32.4041
32.5093
>
> 16 32.6143 32.7192 32.8240 32.9286 33.0331 33.1375
33.2418
>
> 17 33.3460 33.4500 33.5539 33.6577 33.7614 33.8649
33.9683
>
> 18 34.0716 34.1747 34.2778 34.3807 34.4835 34.5861
34.6886
>
> 19 34.7910 34.8933 34.9954 35.0975 35.1993 35.3011
35.4027
>
> 20 35.5042 35.6056 35.7069 35.8080 35.9090 36.0099
36.1106
>
> 21 36.2112 36.3117 36.4120 36.5122 36.6123 36.7123
36.8121
>
> 22 36.9118 37.0114 37.1108 37.2101 37.3093 37.4083
37.5073
>
> 23 37.6060 37.7047 37.8032 37.9016 37.9999 38.0980
38.1960
>
> 24 38.2939 38.3916 38.4892 38.5867 38.6840 38.7812
38.8783
>
> 25 38.9752 39.0720 39.1687 39.2653 39.3617 39.4579
39.5541
>
> 26 39.6501 39.7460 39.8417 39.9373 40.0328 40.1281
40.2234
>
> 27 40.3184 40.4134 40.5082 40.6029 40.6974 40.7918
40.8861
>
> 28 40.9802 41.0742 41.1681 41.2618 41.3554 41.4489
41.5422
>
> 29 41.6354 41.7285 41.8214 41.9142 42.0069 42.0994
42.1918
>
> 30 42.2841 42.3762 42.4682 42.5600 42.6517 42.7433
42.8348
>
> 31 42.9261 43.0173 43.1083 43.1992 43.2900 43.3806
43.4711
>
> 32 43.5615 43.6517 43.7418 43.8318 43.9216 44.0113
44.1008
>
> 33 44.1902 44.2795 44.3687 44.4577 44.5466 44.6353
44.7239
>
> 34 44.8124 44.9007 44.9889 45.0770 45.1649 45.2527
45.3403
>
> 35 45.4278 45.5152 45.6025 45.6896 45.7766 45.8634
45.9501
>
> 36 46.0367 46.1231 46.2094 46.2956 46.3816 46.4675
46.5533
>
> 37 46.6389 46.7244 46.8098 46.8950 46.9801 47.0650
>
> 38 xdef 202 linear -136.710000 0.124606965174129
>
> 39 tdef 1 linear 12Z26apr2019 1mo
>
> 40 zdef 1 linear 1 1
>
> 41 vars 1
>
> 42 UGRD1000mb 0 33,100,1000 ** 1000 mb u wind [m/s]
>
> 43 ENDVARS
>
>
>
> My ob file is in a netcdf file created using ascii2nc. I named wind
speed
> obs "ws" in the ascii file that I ran the ascii2nc tool on ( I am
aware
> that
> I am trying to compare Wind speed and u component of wind, but I
just want
> to see if I can get point_stat working).
>
>
>
> How should I arrange my fcst and obs arrays? Here is what I have
so far:
>
>
>
> 52 fcst = {
>
> 53 field = [
>
> 54 {
>
> 55 name = "U";
>
> 56 level = "P1000";
>
> 57 file_type = GRIB2;
>
> 58 cat_thresh = [ >80.0 ];
>
> 59 }
>
> 60 ];
>
> 61
>
> 62 }
>
> 63
>
> 64 obs = {
>
> 65 field = [
>
> 66 {
>
> 67 file_type = NETCDF_NCCF;
>
> 68 name = "UGRD";
>
> 69 level = ["P1000"];
>
> 70 }
>
> 71 ];
>
> 72 }
>
>
>
> Running the command as follows gives me this error:
>
>
>
> [tsu at maury2 00_POINT_STAT_WORK]$ point_stat
>
>
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_02700F0NL2019042612_
> 0100_010000-000000wnd_ucmp
>
> NAF_sfc_2019042612.nc config.rc -v 1
>
> DEBUG 1: Default Config File:
>
> /software/depot/met/met-
8.0_bugfix/share/met/config/PointStatConfig_default
>
> DEBUG 1: User Config File: config.rc
>
> GSL_RNG_TYPE=mt19937
>
> GSL_RNG_SEED=1726142812
>
> DEBUG 1: Forecast File:
>
>
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_02700F0NL2019042612_
> 0100_010000-000000wnd_ucmp
>
> DEBUG 1: Observation File: NAF_sfc_2019042612.nc
>
> WARNING:
>
> WARNING: process_fcst_climo_files() -> no fields matching U/P1000
found in
>
> file:
>
>
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_02700F0NL2019042612_
> 0100_010000-000000wnd_ucmp
>
> WARNING:
>
> ERROR :
>
> ERROR : process_fcst_climo_files() -> no requested forecast data
found!
>
> Exiting...
>
> ERROR :
>
>
>
> Justin
>
>
>
>
>
> Justin Tsu
>
> Marine Meteorology Division
>
> Data Assimilation/Mesoscale Modeling
>
> Building 704 Room 212
>
> Naval Research Laboratory, Code 7531
>
> 7 Grace Hopper Avenue
>
> Monterey, CA 93943-5502
>
>
>
> Ph. (831) 656-4111
>
>
>
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #90256] Point Stat Help
From: Tsu, Mr. Justin
Time: Tue May 21 22:49:34 2019
Hi John,
So I took your advice and used wgrib to determine that my variable is
indeed named UGRD so I switched this out in my point_Stat config file.
Now that config file looks like this:
52 fcst = {
53 field = [
54 {
55 name = "UGRD";
56 level = ["P1000"];
57 file_type = GRIB1;
58 cat_thresh = [ >80.0 ];
59 }
60 ];
61
62 }
63
64 obs = {
65 field = [
66 {
67 file_type = NETCDF_NCCF;
68 name = "UGRD";
69 level = ["P1000"];
70 }
71 ];
72 }
This config file runs when I execute:
point_stat GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_02700F0NL2019042612_0100_010000-000000wnd_ucmp
NAF_sfc_2019042612.nc config.rc -v 1
However, the resulting contingency table file are empty (aside from
the header). I am guessing this has something to do with the way I
specify my obs file. As I stated previously in my email, the obs file
is named raob_2019042612.nc and is a netcdf file created by using
ascii2nc on a list of point obs arranged in the format specified by
the MET tutorial. I am not sure what the name value in the field
dictionary refers to because the netcdf header for raob_2019042612.nc
looks like:
1 netcdf raob_2019042612 {
2 dimensions:
3 mxstr = 16 ;
4 mxstr2 = 40 ;
5 mxstr3 = 80 ;
6 nobs = 1570630 ;
7 nhdr = 82 ;
8 nhdr_typ = 1 ;
9 nhdr_sid = 82 ;
10 nhdr_vld = 1 ;
11 nobs_qty = 1 ;
12 obs_var_num = 6 ;
13 variables:
14 int hdr_typ(nhdr) ;
15 hdr_typ:long_name = "index of message type" ;
16 hdr_typ:_FillValue = -9999 ;
17 int hdr_sid(nhdr) ;
18 hdr_sid:long_name = "index of station identification" ;
19 hdr_sid:_FillValue = -9999 ;
20 int hdr_vld(nhdr) ;
21 hdr_vld:long_name = "index of valid time" ;
22 hdr_vld:_FillValue = -9999 ;
23 float hdr_lat(nhdr) ;
24 hdr_lat:long_name = "latitude" ;
25 hdr_lat:_FillValue = -9999.f ;
26 hdr_lat:units = "degrees_north" ;
27 float hdr_lon(nhdr) ;
28 hdr_lon:long_name = "longitude" ;
29 hdr_lon:_FillValue = -9999.f ;
30 hdr_lon:units = "degrees_east" ;
31 float hdr_elv(nhdr) ;
32 hdr_elv:long_name = "elevation" ;
33 hdr_elv:_FillValue = -9999.f ;
34 hdr_elv:units = "meters above sea level (msl)" ;
35 int obs_qty(nobs) ;
36 obs_qty:long_name = "index of quality flag" ;
37 int obs_hid(nobs) ;
38 obs_hid:long_name = "index of matching header data" ;
39 obs_hid:_FillValue = -9999 ;
40 int obs_vid(nobs) ;
41 obs_vid:long_name = "index of variable names at var_name"
;
42 obs_vid:_FillValue = -9999 ;
43 float obs_lvl(nobs) ;
44 obs_lvl:long_name = "pressure level (hPa) or accumulation
interval (sec)" ;
45 obs_lvl:_FillValue = -9999.f ;
46 float obs_hgt(nobs) ;
47 obs_hgt:long_name = "height in meters above sea level or
ground level (msl or agl)" ;
48 obs_hgt:_FillValue = -9999.f ;
49 float obs_val(nobs) ;
50 obs_val:long_name = "observation value" ;
51 obs_val:_FillValue = -9999.f ;
52 char hdr_typ_table(nhdr_typ, mxstr2) ;
53 hdr_typ_table:long_name = "message type" ;
54 char hdr_sid_table(nhdr_sid, mxstr2) ;
55 hdr_sid_table:long_name = "station identification" ;
56 char hdr_vld_table(nhdr_vld, mxstr) ;
57 hdr_vld_table:long_name = "valid time" ;
58 hdr_vld_table:units = "YYYYMMDD_HHMMSS UTC" ;
59 char obs_qty_table(nobs_qty, mxstr) ;
60 obs_qty_table:long_name = "quality flag" ;
61 char obs_var(obs_var_num, mxstr2) ;
62 obs_var:long_name = "variable names" ;
63
64 // global attributes:
65 :MET_Obs_version = "1.02" ;
66 :use_var_id = "true" ;
67 :FileOrigins = "File / generated 20190510_235941 UTC on
host maury2.local by the MET ascii2nc tool" ;
68 :MET_version = "V8.0" ;
69 :MET_tool = "ascii2nc" ;
So there isn’t actualy a netcdf variable named UGRD; at this point I'm
not sure what I should call it.
Hope you can shed some light on how this works.
Thanks,
Justin
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Monday, May 20, 2019 12:20 PM
To: Tsu, Mr. Justin
Subject: Re: [rt.rap.ucar.edu #90256] Point Stat Help
Hi Justin,
I see that you're having trouble reading GRIB data into the Point-Stat
tool
in MET. When getting going with a new gridded dataset in MET, I
usually
recommend starting with the plot_data_plane utility. For example, try
running this command:
plot_data_plane
GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_0270F0NL2019042612_0100_010000-000000wnd_ucmp
U_P1000.ps 'name="U"; level="P1000";'
You should get a similar error since Point-Stat wasn't able to read
it.
I'd recommend running this file through the "wgrib2" utility to
inventory
it and see what variable names/levels are present in the file. For
example, is it called "U" or "UGRD"?
If you aren't able to get that working, please just post a sample file
to
our anonymous ftp site. If you do, let me know, and I'll go grab it
and
take a look:
https://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk
Thanks,
John Halley Gotway
On Fri, May 17, 2019 at 3:27 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
wrote:
>
> Fri May 17 15:27:39 2019: Request 90256 was acted upon.
> Transaction: Ticket created by justin.tsu at nrlmry.navy.mil
> Queue: met_help
> Subject: Point Stat Help
> Owner: Nobody
> Requestors: justin.tsu at nrlmry.navy.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90256 >
>
>
> Ive got my forecast field in the form of a grib file entitled
>
>
>
>
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_02700F0NL2019042612_
> 0100_010000-000000wnd_ucmp
>
>
>
> This is the U Component of wind at 1000 mb. The following is a
descriptor
> file to read this in GrADS:
>
>
>
> 1 dset
>
> ^US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_03000F0NL2019042612_0100_0100
> 00-000000wnd_ucmp
>
> 2 index
>
> ^US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_03000F0NL2019042612_0100_0100
> 00-000000wnd_ucmp.idx
>
> 3 undef 9.999E+20
>
> 4 title
>
> US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_03000F0NL2019042612_0100_01000
> 0-000000wnd_ucmp
>
> 5 * produced by grib2ctl v0.9.15
>
> 6 * command line options:
>
> US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_03000F0NL2019042612_0100_01000
> 0-000000wnd_ucmp
>
> 7 dtype grib 255
>
> 8 ydef 202 levels
>
> 9 27.3280 27.4386 27.5492 27.6596 27.7699 27.8801
27.9902
>
> 10 28.1002 28.2101 28.3198 28.4295 28.5390 28.6484
28.7577
>
> 11 28.8669 28.9759 29.0849 29.1937 29.3025 29.4111
29.5196
>
> 12 29.6279 29.7362 29.8443 29.9524 30.0603 30.1681
30.2757
>
> 13 30.3833 30.4907 30.5980 30.7052 30.8123 30.9193
31.0261
>
> 14 31.1329 31.2395 31.3459 31.4523 31.5585 31.6647
31.7707
>
> 15 31.8765 31.9823 32.0879 32.1934 32.2988 32.4041
32.5093
>
> 16 32.6143 32.7192 32.8240 32.9286 33.0331 33.1375
33.2418
>
> 17 33.3460 33.4500 33.5539 33.6577 33.7614 33.8649
33.9683
>
> 18 34.0716 34.1747 34.2778 34.3807 34.4835 34.5861
34.6886
>
> 19 34.7910 34.8933 34.9954 35.0975 35.1993 35.3011
35.4027
>
> 20 35.5042 35.6056 35.7069 35.8080 35.9090 36.0099
36.1106
>
> 21 36.2112 36.3117 36.4120 36.5122 36.6123 36.7123
36.8121
>
> 22 36.9118 37.0114 37.1108 37.2101 37.3093 37.4083
37.5073
>
> 23 37.6060 37.7047 37.8032 37.9016 37.9999 38.0980
38.1960
>
> 24 38.2939 38.3916 38.4892 38.5867 38.6840 38.7812
38.8783
>
> 25 38.9752 39.0720 39.1687 39.2653 39.3617 39.4579
39.5541
>
> 26 39.6501 39.7460 39.8417 39.9373 40.0328 40.1281
40.2234
>
> 27 40.3184 40.4134 40.5082 40.6029 40.6974 40.7918
40.8861
>
> 28 40.9802 41.0742 41.1681 41.2618 41.3554 41.4489
41.5422
>
> 29 41.6354 41.7285 41.8214 41.9142 42.0069 42.0994
42.1918
>
> 30 42.2841 42.3762 42.4682 42.5600 42.6517 42.7433
42.8348
>
> 31 42.9261 43.0173 43.1083 43.1992 43.2900 43.3806
43.4711
>
> 32 43.5615 43.6517 43.7418 43.8318 43.9216 44.0113
44.1008
>
> 33 44.1902 44.2795 44.3687 44.4577 44.5466 44.6353
44.7239
>
> 34 44.8124 44.9007 44.9889 45.0770 45.1649 45.2527
45.3403
>
> 35 45.4278 45.5152 45.6025 45.6896 45.7766 45.8634
45.9501
>
> 36 46.0367 46.1231 46.2094 46.2956 46.3816 46.4675
46.5533
>
> 37 46.6389 46.7244 46.8098 46.8950 46.9801 47.0650
>
> 38 xdef 202 linear -136.710000 0.124606965174129
>
> 39 tdef 1 linear 12Z26apr2019 1mo
>
> 40 zdef 1 linear 1 1
>
> 41 vars 1
>
> 42 UGRD1000mb 0 33,100,1000 ** 1000 mb u wind [m/s]
>
> 43 ENDVARS
>
>
>
> My ob file is in a netcdf file created using ascii2nc. I named wind
speed
> obs "ws" in the ascii file that I ran the ascii2nc tool on ( I am
aware
> that
> I am trying to compare Wind speed and u component of wind, but I
just want
> to see if I can get point_stat working).
>
>
>
> How should I arrange my fcst and obs arrays? Here is what I have
so far:
>
>
>
> 52 fcst = {
>
> 53 field = [
>
> 54 {
>
> 55 name = "U";
>
> 56 level = "P1000";
>
> 57 file_type = GRIB2;
>
> 58 cat_thresh = [ >80.0 ];
>
> 59 }
>
> 60 ];
>
> 61
>
> 62 }
>
> 63
>
> 64 obs = {
>
> 65 field = [
>
> 66 {
>
> 67 file_type = NETCDF_NCCF;
>
> 68 name = "UGRD";
>
> 69 level = ["P1000"];
>
> 70 }
>
> 71 ];
>
> 72 }
>
>
>
> Running the command as follows gives me this error:
>
>
>
> [tsu at maury2 00_POINT_STAT_WORK]$ point_stat
>
>
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_02700F0NL2019042612_
> 0100_010000-000000wnd_ucmp
>
> NAF_sfc_2019042612.nc config.rc -v 1
>
> DEBUG 1: Default Config File:
>
> /software/depot/met/met-
8.0_bugfix/share/met/config/PointStatConfig_default
>
> DEBUG 1: User Config File: config.rc
>
> GSL_RNG_TYPE=mt19937
>
> GSL_RNG_SEED=1726142812
>
> DEBUG 1: Forecast File:
>
>
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_02700F0NL2019042612_
> 0100_010000-000000wnd_ucmp
>
> DEBUG 1: Observation File: NAF_sfc_2019042612.nc
>
> WARNING:
>
> WARNING: process_fcst_climo_files() -> no fields matching U/P1000
found in
>
> file:
>
>
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_02700F0NL2019042612_
> 0100_010000-000000wnd_ucmp
>
> WARNING:
>
> ERROR :
>
> ERROR : process_fcst_climo_files() -> no requested forecast data
found!
>
> Exiting...
>
> ERROR :
>
>
>
> Justin
>
>
>
>
>
> Justin Tsu
>
> Marine Meteorology Division
>
> Data Assimilation/Mesoscale Modeling
>
> Building 704 Room 212
>
> Naval Research Laboratory, Code 7531
>
> 7 Grace Hopper Avenue
>
> Monterey, CA 93943-5502
>
>
>
> Ph. (831) 656-4111
>
>
>
>
>
------------------------------------------------
Subject: Point Stat Help
From: John Halley Gotway
Time: Wed May 22 12:27:11 2019
Justin,
When you run Point-Stat, please increase the verbosity level to at
least 3
(-v 3). That will dump log messages that look something like this:
*DEBUG 2: Processing TMP/P900-750 versus TMP/P900-750, for observation
type
ADPUPA, over region DTC165, for interpolation method NEAREST(1), using
132
pairs.DEBUG 3: Number of matched pairs = 132DEBUG 3: Observations
processed = 89893DEBUG 3: Rejected: SID exclusion = 0DEBUG 3:
Rejected:
obs type = 79788DEBUG 3: Rejected: valid time = 0DEBUG 3:
Rejected: bad obs value = 0DEBUG 3: Rejected: off the grid = 5DEBUG
3:
Rejected: topography = 0DEBUG 3: Rejected: level mismatch =
9219DEBUG
3: Rejected: quality marker = 0DEBUG 3: Rejected: message type =
344DEBUG
3: Rejected: masking region = 405DEBUG 3: Rejected: bad fcst value =
0DEBUG
3: Rejected: duplicates = 0*
For each verification task, these list counts of reasons why
observations
were not used for that task. These counts should help you determine
why
you're getting 0 matched pairs. There are many possible reasons
why...
mismatches in message type, valid time, lat/lon location, vertical
level...
and so on.
Hopefully these counts will point you in the right direction.
Thanks,
John
On Tue, May 21, 2019 at 10:50 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90256 >
>
> Hi John,
>
> So I took your advice and used wgrib to determine that my variable
is
> indeed named UGRD so I switched this out in my point_Stat config
file. Now
> that config file looks like this:
>
> 52 fcst = {
> 53 field = [
> 54 {
> 55 name = "UGRD";
> 56 level = ["P1000"];
> 57 file_type = GRIB1;
> 58 cat_thresh = [ >80.0 ];
> 59 }
> 60 ];
> 61
> 62 }
> 63
> 64 obs = {
> 65 field = [
> 66 {
> 67 file_type = NETCDF_NCCF;
> 68 name = "UGRD";
> 69 level = ["P1000"];
> 70 }
> 71 ];
> 72 }
>
> This config file runs when I execute:
>
> point_stat
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_02700F0NL2019042612_0100_010000-000000wnd_ucmp
> NAF_sfc_2019042612.nc config.rc -v 1
>
> However, the resulting contingency table file are empty (aside from
the
> header). I am guessing this has something to do with the way I
specify my
> obs file. As I stated previously in my email, the obs file is named
> raob_2019042612.nc and is a netcdf file created by using ascii2nc on
a
> list of point obs arranged in the format specified by the MET
tutorial. I
> am not sure what the name value in the field dictionary refers to
because
> the netcdf header for raob_2019042612.nc looks like:
>
> 1 netcdf raob_2019042612 {
> 2 dimensions:
> 3 mxstr = 16 ;
> 4 mxstr2 = 40 ;
> 5 mxstr3 = 80 ;
> 6 nobs = 1570630 ;
> 7 nhdr = 82 ;
> 8 nhdr_typ = 1 ;
> 9 nhdr_sid = 82 ;
> 10 nhdr_vld = 1 ;
> 11 nobs_qty = 1 ;
> 12 obs_var_num = 6 ;
> 13 variables:
> 14 int hdr_typ(nhdr) ;
> 15 hdr_typ:long_name = "index of message type" ;
> 16 hdr_typ:_FillValue = -9999 ;
> 17 int hdr_sid(nhdr) ;
> 18 hdr_sid:long_name = "index of station identification" ;
> 19 hdr_sid:_FillValue = -9999 ;
> 20 int hdr_vld(nhdr) ;
> 21 hdr_vld:long_name = "index of valid time" ;
> 22 hdr_vld:_FillValue = -9999 ;
> 23 float hdr_lat(nhdr) ;
> 24 hdr_lat:long_name = "latitude" ;
> 25 hdr_lat:_FillValue = -9999.f ;
> 26 hdr_lat:units = "degrees_north" ;
> 27 float hdr_lon(nhdr) ;
> 28 hdr_lon:long_name = "longitude" ;
> 29 hdr_lon:_FillValue = -9999.f ;
> 30 hdr_lon:units = "degrees_east" ;
> 31 float hdr_elv(nhdr) ;
> 32 hdr_elv:long_name = "elevation" ;
> 33 hdr_elv:_FillValue = -9999.f ;
> 34 hdr_elv:units = "meters above sea level (msl)" ;
> 35 int obs_qty(nobs) ;
> 36 obs_qty:long_name = "index of quality flag" ;
> 37 int obs_hid(nobs) ;
> 38 obs_hid:long_name = "index of matching header data" ;
> 39 obs_hid:_FillValue = -9999 ;
> 40 int obs_vid(nobs) ;
> 41 obs_vid:long_name = "index of variable names at
var_name" ;
> 42 obs_vid:_FillValue = -9999 ;
> 43 float obs_lvl(nobs) ;
> 44 obs_lvl:long_name = "pressure level (hPa) or
accumulation
> interval (sec)" ;
> 45 obs_lvl:_FillValue = -9999.f ;
> 46 float obs_hgt(nobs) ;
> 47 obs_hgt:long_name = "height in meters above sea level or
> ground level (msl or agl)" ;
> 48 obs_hgt:_FillValue = -9999.f ;
> 49 float obs_val(nobs) ;
> 50 obs_val:long_name = "observation value" ;
> 51 obs_val:_FillValue = -9999.f ;
> 52 char hdr_typ_table(nhdr_typ, mxstr2) ;
> 53 hdr_typ_table:long_name = "message type" ;
> 54 char hdr_sid_table(nhdr_sid, mxstr2) ;
> 55 hdr_sid_table:long_name = "station identification" ;
> 56 char hdr_vld_table(nhdr_vld, mxstr) ;
> 57 hdr_vld_table:long_name = "valid time" ;
> 58 hdr_vld_table:units = "YYYYMMDD_HHMMSS UTC" ;
> 59 char obs_qty_table(nobs_qty, mxstr) ;
> 60 obs_qty_table:long_name = "quality flag" ;
> 61 char obs_var(obs_var_num, mxstr2) ;
> 62 obs_var:long_name = "variable names" ;
> 63
> 64 // global attributes:
> 65 :MET_Obs_version = "1.02" ;
> 66 :use_var_id = "true" ;
> 67 :FileOrigins = "File / generated 20190510_235941 UTC on
host
> maury2.local by the MET ascii2nc tool" ;
> 68 :MET_version = "V8.0" ;
> 69 :MET_tool = "ascii2nc" ;
>
> So there isn’t actualy a netcdf variable named UGRD; at this point
I'm not
> sure what I should call it.
>
> Hope you can shed some light on how this works.
>
> Thanks,
> Justin
>
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Monday, May 20, 2019 12:20 PM
> To: Tsu, Mr. Justin
> Subject: Re: [rt.rap.ucar.edu #90256] Point Stat Help
>
> Hi Justin,
>
> I see that you're having trouble reading GRIB data into the Point-
Stat tool
> in MET. When getting going with a new gridded dataset in MET, I
usually
> recommend starting with the plot_data_plane utility. For example,
try
> running this command:
>
> plot_data_plane
>
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_0270F0NL2019042612_0100_010000-000000wnd_ucmp
> U_P1000.ps 'name="U"; level="P1000";'
>
> You should get a similar error since Point-Stat wasn't able to read
it.
> I'd recommend running this file through the "wgrib2" utility to
inventory
> it and see what variable names/levels are present in the file. For
> example, is it called "U" or "UGRD"?
>
> If you aren't able to get that working, please just post a sample
file to
> our anonymous ftp site. If you do, let me know, and I'll go grab it
and
> take a look:
>
> https://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk
>
> Thanks,
> John Halley Gotway
>
>
>
> On Fri, May 17, 2019 at 3:27 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > Fri May 17 15:27:39 2019: Request 90256 was acted upon.
> > Transaction: Ticket created by justin.tsu at nrlmry.navy.mil
> > Queue: met_help
> > Subject: Point Stat Help
> > Owner: Nobody
> > Requestors: justin.tsu at nrlmry.navy.mil
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90256 >
> >
> >
> > Ive got my forecast field in the form of a grib file entitled
> >
> >
> >
> >
> >
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_02700F0NL2019042612_
> > 0100_010000-000000wnd_ucmp
> >
> >
> >
> > This is the U Component of wind at 1000 mb. The following is a
> descriptor
> > file to read this in GrADS:
> >
> >
> >
> > 1 dset
> >
> >
> ^US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_03000F0NL2019042612_0100_0100
> > 00-000000wnd_ucmp
> >
> > 2 index
> >
> >
> ^US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_03000F0NL2019042612_0100_0100
> > 00-000000wnd_ucmp.idx
> >
> > 3 undef 9.999E+20
> >
> > 4 title
> >
> >
> US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_03000F0NL2019042612_0100_01000
> > 0-000000wnd_ucmp
> >
> > 5 * produced by grib2ctl v0.9.15
> >
> > 6 * command line options:
> >
> >
> US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_03000F0NL2019042612_0100_01000
> > 0-000000wnd_ucmp
> >
> > 7 dtype grib 255
> >
> > 8 ydef 202 levels
> >
> > 9 27.3280 27.4386 27.5492 27.6596 27.7699 27.8801
27.9902
> >
> > 10 28.1002 28.2101 28.3198 28.4295 28.5390 28.6484
28.7577
> >
> > 11 28.8669 28.9759 29.0849 29.1937 29.3025 29.4111
29.5196
> >
> > 12 29.6279 29.7362 29.8443 29.9524 30.0603 30.1681
30.2757
> >
> > 13 30.3833 30.4907 30.5980 30.7052 30.8123 30.9193
31.0261
> >
> > 14 31.1329 31.2395 31.3459 31.4523 31.5585 31.6647
31.7707
> >
> > 15 31.8765 31.9823 32.0879 32.1934 32.2988 32.4041
32.5093
> >
> > 16 32.6143 32.7192 32.8240 32.9286 33.0331 33.1375
33.2418
> >
> > 17 33.3460 33.4500 33.5539 33.6577 33.7614 33.8649
33.9683
> >
> > 18 34.0716 34.1747 34.2778 34.3807 34.4835 34.5861
34.6886
> >
> > 19 34.7910 34.8933 34.9954 35.0975 35.1993 35.3011
35.4027
> >
> > 20 35.5042 35.6056 35.7069 35.8080 35.9090 36.0099
36.1106
> >
> > 21 36.2112 36.3117 36.4120 36.5122 36.6123 36.7123
36.8121
> >
> > 22 36.9118 37.0114 37.1108 37.2101 37.3093 37.4083
37.5073
> >
> > 23 37.6060 37.7047 37.8032 37.9016 37.9999 38.0980
38.1960
> >
> > 24 38.2939 38.3916 38.4892 38.5867 38.6840 38.7812
38.8783
> >
> > 25 38.9752 39.0720 39.1687 39.2653 39.3617 39.4579
39.5541
> >
> > 26 39.6501 39.7460 39.8417 39.9373 40.0328 40.1281
40.2234
> >
> > 27 40.3184 40.4134 40.5082 40.6029 40.6974 40.7918
40.8861
> >
> > 28 40.9802 41.0742 41.1681 41.2618 41.3554 41.4489
41.5422
> >
> > 29 41.6354 41.7285 41.8214 41.9142 42.0069 42.0994
42.1918
> >
> > 30 42.2841 42.3762 42.4682 42.5600 42.6517 42.7433
42.8348
> >
> > 31 42.9261 43.0173 43.1083 43.1992 43.2900 43.3806
43.4711
> >
> > 32 43.5615 43.6517 43.7418 43.8318 43.9216 44.0113
44.1008
> >
> > 33 44.1902 44.2795 44.3687 44.4577 44.5466 44.6353
44.7239
> >
> > 34 44.8124 44.9007 44.9889 45.0770 45.1649 45.2527
45.3403
> >
> > 35 45.4278 45.5152 45.6025 45.6896 45.7766 45.8634
45.9501
> >
> > 36 46.0367 46.1231 46.2094 46.2956 46.3816 46.4675
46.5533
> >
> > 37 46.6389 46.7244 46.8098 46.8950 46.9801 47.0650
> >
> > 38 xdef 202 linear -136.710000 0.124606965174129
> >
> > 39 tdef 1 linear 12Z26apr2019 1mo
> >
> > 40 zdef 1 linear 1 1
> >
> > 41 vars 1
> >
> > 42 UGRD1000mb 0 33,100,1000 ** 1000 mb u wind [m/s]
> >
> > 43 ENDVARS
> >
> >
> >
> > My ob file is in a netcdf file created using ascii2nc. I named
wind
> speed
> > obs "ws" in the ascii file that I ran the ascii2nc tool on ( I am
aware
> > that
> > I am trying to compare Wind speed and u component of wind, but I
just
> want
> > to see if I can get point_stat working).
> >
> >
> >
> > How should I arrange my fcst and obs arrays? Here is what I have
so
> far:
> >
> >
> >
> > 52 fcst = {
> >
> > 53 field = [
> >
> > 54 {
> >
> > 55 name = "U";
> >
> > 56 level = "P1000";
> >
> > 57 file_type = GRIB2;
> >
> > 58 cat_thresh = [ >80.0 ];
> >
> > 59 }
> >
> > 60 ];
> >
> > 61
> >
> > 62 }
> >
> > 63
> >
> > 64 obs = {
> >
> > 65 field = [
> >
> > 66 {
> >
> > 67 file_type = NETCDF_NCCF;
> >
> > 68 name = "UGRD";
> >
> > 69 level = ["P1000"];
> >
> > 70 }
> >
> > 71 ];
> >
> > 72 }
> >
> >
> >
> > Running the command as follows gives me this error:
> >
> >
> >
> > [tsu at maury2 00_POINT_STAT_WORK]$ point_stat
> >
> >
> >
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_02700F0NL2019042612_
> > 0100_010000-000000wnd_ucmp
> >
> > NAF_sfc_2019042612.nc config.rc -v 1
> >
> > DEBUG 1: Default Config File:
> >
> >
> /software/depot/met/met-
8.0_bugfix/share/met/config/PointStatConfig_default
> >
> > DEBUG 1: User Config File: config.rc
> >
> > GSL_RNG_TYPE=mt19937
> >
> > GSL_RNG_SEED=1726142812
> >
> > DEBUG 1: Forecast File:
> >
> >
> >
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_02700F0NL2019042612_
> > 0100_010000-000000wnd_ucmp
> >
> > DEBUG 1: Observation File: NAF_sfc_2019042612.nc
> >
> > WARNING:
> >
> > WARNING: process_fcst_climo_files() -> no fields matching U/P1000
found
> in
> >
> > file:
> >
> >
> >
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_02700F0NL2019042612_
> > 0100_010000-000000wnd_ucmp
> >
> > WARNING:
> >
> > ERROR :
> >
> > ERROR : process_fcst_climo_files() -> no requested forecast data
found!
> >
> > Exiting...
> >
> > ERROR :
> >
> >
> >
> > Justin
> >
> >
> >
> >
> >
> > Justin Tsu
> >
> > Marine Meteorology Division
> >
> > Data Assimilation/Mesoscale Modeling
> >
> > Building 704 Room 212
> >
> > Naval Research Laboratory, Code 7531
> >
> > 7 Grace Hopper Avenue
> >
> > Monterey, CA 93943-5502
> >
> >
> >
> > Ph. (831) 656-4111
> >
> >
> >
> >
> >
>
>
>
>
------------------------------------------------
Subject: Point Stat Help
From: Tsu, Mr. Justin
Time: Wed May 22 13:31:14 2019
Hey John,
Just ran with verbosity set to 3 and discovered that no pairs are
being used:
[tsu at maury2 00_POINT_STAT_WORK]$ point_stat GRIB_UCMP/US058GMET-
GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-c1_00000F0NL2019042612_0100_010000-
000000wnd_ucmp raob_2019042612.nc config.rc -v 3
DEBUG 1: Default Config File: /software/depot/met/met-
8.0_bugfix/share/met/config/PointStatConfig_default
DEBUG 1: User Config File: config.rc
GSL_RNG_TYPE=mt19937
GSL_RNG_SEED=18446744072713528668
DEBUG 1: Forecast File: GRIB_UCMP/US058GMET-GR1dyn.COAMPS-
CENCOOS_CENCOOS-n2-c1_00000F0NL2019042612_0100_010000-000000wnd_ucmp
DEBUG 1: Observation File: raob_2019042612.nc
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match for
VarInfo "UGRD/P1000" in GRIB record 1 of GRIB file
"GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_00000F0NL2019042612_0100_010000-000000wnd_ucmp".
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB records
matching VarInfo "UGRD/P1000" in GRIB file "GRIB_UCMP/US058GMET-
GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-c1_00000F0NL2019042612_0100_010000-
000000wnd_ucmp".
DEBUG 2:
DEBUG 2:
--------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Reading data for UGRD/P1000.
DEBUG 3: Use the matching forecast and observation grids.
DEBUG 3: Grid Definition: Projection: Mercator Nx: 202 Ny: 202
Lat_LL_radians: 0.4770 Lon_LL_radians: 2.3860 Lat_UR_radians: 0.8214
Lon_UR_radians: 1.9489 Mx: 459.8120 My: 459.8003 Bx: 1097.1297 By:
-228.1298
DEBUG 2: Processing masking regions.
DEBUG 3: Processing grid mask: FULL
DEBUG 2: For UGRD/P1000 found 1 forecast levels, 0 climatology mean
levels, and 0 climatology standard deviation levels.
DEBUG 2:
DEBUG 2:
--------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Searching 1570630 observations from 82 messages.
DEBUG 2:
DEBUG 2:
--------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Processing UGRD/P1000 versus UGRD/P1000, for observation type
ADPUPA, over region FULL, for interpolation method NEAREST(1), using 0
pairs.
DEBUG 3: Number of matched pairs = 0
DEBUG 3: Observations processed = 1570630
DEBUG 3: Rejected: SID exclusion = 0
DEBUG 3: Rejected: obs type = 1570630
DEBUG 3: Rejected: valid time = 0
DEBUG 3: Rejected: bad obs value = 0
DEBUG 3: Rejected: off the grid = 0
DEBUG 3: Rejected: level mismatch = 0
DEBUG 3: Rejected: quality marker = 0
DEBUG 3: Rejected: message type = 0
DEBUG 3: Rejected: masking region = 0
DEBUG 3: Rejected: bad fcst value = 0
DEBUG 3: Rejected: duplicates = 0
DEBUG 2:
DEBUG 2:
--------------------------------------------------------------------------------
DEBUG 2:
DEBUG 1: Output file: ./point_stat_000000L_20190426_120000V.stat
DEBUG 1: Output file: ./point_stat_000000L_20190426_120000V_cnt.txt
I had an issue with this in the past using NAVGEM IEEE flatfile format
climatologies that I converted to netcdf using ascii2nc as obs. I
fixed it by using python embedding in the name part of the obs
dictionary. Problem is I cannot do this here because my obs are
originally in hdf5 format and I don’t have a way to use python
embedding to provide MET with the necessary metadata like
read_NRL_binary.py does (I believe somebody from your team provided us
with this python script to print out metadata based on a flatfile
format that we use)
Have you guys had any luck in using ascii2nc generated netcdf files
as either forecast or obs in any of the MET stat utilities?
Justin
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Wednesday, May 22, 2019 11:27 AM
To: Tsu, Mr. Justin
Subject: Re: [rt.rap.ucar.edu #90256] Point Stat Help
Justin,
When you run Point-Stat, please increase the verbosity level to at
least 3
(-v 3). That will dump log messages that look something like this:
*DEBUG 2: Processing TMP/P900-750 versus TMP/P900-750, for observation
type
ADPUPA, over region DTC165, for interpolation method NEAREST(1), using
132
pairs.DEBUG 3: Number of matched pairs = 132DEBUG 3: Observations
processed = 89893DEBUG 3: Rejected: SID exclusion = 0DEBUG 3:
Rejected:
obs type = 79788DEBUG 3: Rejected: valid time = 0DEBUG 3:
Rejected: bad obs value = 0DEBUG 3: Rejected: off the grid = 5DEBUG
3:
Rejected: topography = 0DEBUG 3: Rejected: level mismatch =
9219DEBUG
3: Rejected: quality marker = 0DEBUG 3: Rejected: message type =
344DEBUG
3: Rejected: masking region = 405DEBUG 3: Rejected: bad fcst value =
0DEBUG
3: Rejected: duplicates = 0*
For each verification task, these list counts of reasons why
observations
were not used for that task. These counts should help you determine
why
you're getting 0 matched pairs. There are many possible reasons
why...
mismatches in message type, valid time, lat/lon location, vertical
level...
and so on.
Hopefully these counts will point you in the right direction.
Thanks,
John
On Tue, May 21, 2019 at 10:50 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90256 >
>
> Hi John,
>
> So I took your advice and used wgrib to determine that my variable
is
> indeed named UGRD so I switched this out in my point_Stat config
file. Now
> that config file looks like this:
>
> 52 fcst = {
> 53 field = [
> 54 {
> 55 name = "UGRD";
> 56 level = ["P1000"];
> 57 file_type = GRIB1;
> 58 cat_thresh = [ >80.0 ];
> 59 }
> 60 ];
> 61
> 62 }
> 63
> 64 obs = {
> 65 field = [
> 66 {
> 67 file_type = NETCDF_NCCF;
> 68 name = "UGRD";
> 69 level = ["P1000"];
> 70 }
> 71 ];
> 72 }
>
> This config file runs when I execute:
>
> point_stat
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_02700F0NL2019042612_0100_010000-000000wnd_ucmp
> NAF_sfc_2019042612.nc config.rc -v 1
>
> However, the resulting contingency table file are empty (aside from
the
> header). I am guessing this has something to do with the way I
specify my
> obs file. As I stated previously in my email, the obs file is named
> raob_2019042612.nc and is a netcdf file created by using ascii2nc on
a
> list of point obs arranged in the format specified by the MET
tutorial. I
> am not sure what the name value in the field dictionary refers to
because
> the netcdf header for raob_2019042612.nc looks like:
>
> 1 netcdf raob_2019042612 {
> 2 dimensions:
> 3 mxstr = 16 ;
> 4 mxstr2 = 40 ;
> 5 mxstr3 = 80 ;
> 6 nobs = 1570630 ;
> 7 nhdr = 82 ;
> 8 nhdr_typ = 1 ;
> 9 nhdr_sid = 82 ;
> 10 nhdr_vld = 1 ;
> 11 nobs_qty = 1 ;
> 12 obs_var_num = 6 ;
> 13 variables:
> 14 int hdr_typ(nhdr) ;
> 15 hdr_typ:long_name = "index of message type" ;
> 16 hdr_typ:_FillValue = -9999 ;
> 17 int hdr_sid(nhdr) ;
> 18 hdr_sid:long_name = "index of station identification" ;
> 19 hdr_sid:_FillValue = -9999 ;
> 20 int hdr_vld(nhdr) ;
> 21 hdr_vld:long_name = "index of valid time" ;
> 22 hdr_vld:_FillValue = -9999 ;
> 23 float hdr_lat(nhdr) ;
> 24 hdr_lat:long_name = "latitude" ;
> 25 hdr_lat:_FillValue = -9999.f ;
> 26 hdr_lat:units = "degrees_north" ;
> 27 float hdr_lon(nhdr) ;
> 28 hdr_lon:long_name = "longitude" ;
> 29 hdr_lon:_FillValue = -9999.f ;
> 30 hdr_lon:units = "degrees_east" ;
> 31 float hdr_elv(nhdr) ;
> 32 hdr_elv:long_name = "elevation" ;
> 33 hdr_elv:_FillValue = -9999.f ;
> 34 hdr_elv:units = "meters above sea level (msl)" ;
> 35 int obs_qty(nobs) ;
> 36 obs_qty:long_name = "index of quality flag" ;
> 37 int obs_hid(nobs) ;
> 38 obs_hid:long_name = "index of matching header data" ;
> 39 obs_hid:_FillValue = -9999 ;
> 40 int obs_vid(nobs) ;
> 41 obs_vid:long_name = "index of variable names at
var_name" ;
> 42 obs_vid:_FillValue = -9999 ;
> 43 float obs_lvl(nobs) ;
> 44 obs_lvl:long_name = "pressure level (hPa) or
accumulation
> interval (sec)" ;
> 45 obs_lvl:_FillValue = -9999.f ;
> 46 float obs_hgt(nobs) ;
> 47 obs_hgt:long_name = "height in meters above sea level or
> ground level (msl or agl)" ;
> 48 obs_hgt:_FillValue = -9999.f ;
> 49 float obs_val(nobs) ;
> 50 obs_val:long_name = "observation value" ;
> 51 obs_val:_FillValue = -9999.f ;
> 52 char hdr_typ_table(nhdr_typ, mxstr2) ;
> 53 hdr_typ_table:long_name = "message type" ;
> 54 char hdr_sid_table(nhdr_sid, mxstr2) ;
> 55 hdr_sid_table:long_name = "station identification" ;
> 56 char hdr_vld_table(nhdr_vld, mxstr) ;
> 57 hdr_vld_table:long_name = "valid time" ;
> 58 hdr_vld_table:units = "YYYYMMDD_HHMMSS UTC" ;
> 59 char obs_qty_table(nobs_qty, mxstr) ;
> 60 obs_qty_table:long_name = "quality flag" ;
> 61 char obs_var(obs_var_num, mxstr2) ;
> 62 obs_var:long_name = "variable names" ;
> 63
> 64 // global attributes:
> 65 :MET_Obs_version = "1.02" ;
> 66 :use_var_id = "true" ;
> 67 :FileOrigins = "File / generated 20190510_235941 UTC on
host
> maury2.local by the MET ascii2nc tool" ;
> 68 :MET_version = "V8.0" ;
> 69 :MET_tool = "ascii2nc" ;
>
> So there isn’t actualy a netcdf variable named UGRD; at this point
I'm not
> sure what I should call it.
>
> Hope you can shed some light on how this works.
>
> Thanks,
> Justin
>
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Monday, May 20, 2019 12:20 PM
> To: Tsu, Mr. Justin
> Subject: Re: [rt.rap.ucar.edu #90256] Point Stat Help
>
> Hi Justin,
>
> I see that you're having trouble reading GRIB data into the Point-
Stat tool
> in MET. When getting going with a new gridded dataset in MET, I
usually
> recommend starting with the plot_data_plane utility. For example,
try
> running this command:
>
> plot_data_plane
>
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_0270F0NL2019042612_0100_010000-000000wnd_ucmp
> U_P1000.ps 'name="U"; level="P1000";'
>
> You should get a similar error since Point-Stat wasn't able to read
it.
> I'd recommend running this file through the "wgrib2" utility to
inventory
> it and see what variable names/levels are present in the file. For
> example, is it called "U" or "UGRD"?
>
> If you aren't able to get that working, please just post a sample
file to
> our anonymous ftp site. If you do, let me know, and I'll go grab it
and
> take a look:
>
> https://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk
>
> Thanks,
> John Halley Gotway
>
>
>
> On Fri, May 17, 2019 at 3:27 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > Fri May 17 15:27:39 2019: Request 90256 was acted upon.
> > Transaction: Ticket created by justin.tsu at nrlmry.navy.mil
> > Queue: met_help
> > Subject: Point Stat Help
> > Owner: Nobody
> > Requestors: justin.tsu at nrlmry.navy.mil
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90256 >
> >
> >
> > Ive got my forecast field in the form of a grib file entitled
> >
> >
> >
> >
> >
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_02700F0NL2019042612_
> > 0100_010000-000000wnd_ucmp
> >
> >
> >
> > This is the U Component of wind at 1000 mb. The following is a
> descriptor
> > file to read this in GrADS:
> >
> >
> >
> > 1 dset
> >
> >
> ^US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_03000F0NL2019042612_0100_0100
> > 00-000000wnd_ucmp
> >
> > 2 index
> >
> >
> ^US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_03000F0NL2019042612_0100_0100
> > 00-000000wnd_ucmp.idx
> >
> > 3 undef 9.999E+20
> >
> > 4 title
> >
> >
> US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_03000F0NL2019042612_0100_01000
> > 0-000000wnd_ucmp
> >
> > 5 * produced by grib2ctl v0.9.15
> >
> > 6 * command line options:
> >
> >
> US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_03000F0NL2019042612_0100_01000
> > 0-000000wnd_ucmp
> >
> > 7 dtype grib 255
> >
> > 8 ydef 202 levels
> >
> > 9 27.3280 27.4386 27.5492 27.6596 27.7699 27.8801
27.9902
> >
> > 10 28.1002 28.2101 28.3198 28.4295 28.5390 28.6484
28.7577
> >
> > 11 28.8669 28.9759 29.0849 29.1937 29.3025 29.4111
29.5196
> >
> > 12 29.6279 29.7362 29.8443 29.9524 30.0603 30.1681
30.2757
> >
> > 13 30.3833 30.4907 30.5980 30.7052 30.8123 30.9193
31.0261
> >
> > 14 31.1329 31.2395 31.3459 31.4523 31.5585 31.6647
31.7707
> >
> > 15 31.8765 31.9823 32.0879 32.1934 32.2988 32.4041
32.5093
> >
> > 16 32.6143 32.7192 32.8240 32.9286 33.0331 33.1375
33.2418
> >
> > 17 33.3460 33.4500 33.5539 33.6577 33.7614 33.8649
33.9683
> >
> > 18 34.0716 34.1747 34.2778 34.3807 34.4835 34.5861
34.6886
> >
> > 19 34.7910 34.8933 34.9954 35.0975 35.1993 35.3011
35.4027
> >
> > 20 35.5042 35.6056 35.7069 35.8080 35.9090 36.0099
36.1106
> >
> > 21 36.2112 36.3117 36.4120 36.5122 36.6123 36.7123
36.8121
> >
> > 22 36.9118 37.0114 37.1108 37.2101 37.3093 37.4083
37.5073
> >
> > 23 37.6060 37.7047 37.8032 37.9016 37.9999 38.0980
38.1960
> >
> > 24 38.2939 38.3916 38.4892 38.5867 38.6840 38.7812
38.8783
> >
> > 25 38.9752 39.0720 39.1687 39.2653 39.3617 39.4579
39.5541
> >
> > 26 39.6501 39.7460 39.8417 39.9373 40.0328 40.1281
40.2234
> >
> > 27 40.3184 40.4134 40.5082 40.6029 40.6974 40.7918
40.8861
> >
> > 28 40.9802 41.0742 41.1681 41.2618 41.3554 41.4489
41.5422
> >
> > 29 41.6354 41.7285 41.8214 41.9142 42.0069 42.0994
42.1918
> >
> > 30 42.2841 42.3762 42.4682 42.5600 42.6517 42.7433
42.8348
> >
> > 31 42.9261 43.0173 43.1083 43.1992 43.2900 43.3806
43.4711
> >
> > 32 43.5615 43.6517 43.7418 43.8318 43.9216 44.0113
44.1008
> >
> > 33 44.1902 44.2795 44.3687 44.4577 44.5466 44.6353
44.7239
> >
> > 34 44.8124 44.9007 44.9889 45.0770 45.1649 45.2527
45.3403
> >
> > 35 45.4278 45.5152 45.6025 45.6896 45.7766 45.8634
45.9501
> >
> > 36 46.0367 46.1231 46.2094 46.2956 46.3816 46.4675
46.5533
> >
> > 37 46.6389 46.7244 46.8098 46.8950 46.9801 47.0650
> >
> > 38 xdef 202 linear -136.710000 0.124606965174129
> >
> > 39 tdef 1 linear 12Z26apr2019 1mo
> >
> > 40 zdef 1 linear 1 1
> >
> > 41 vars 1
> >
> > 42 UGRD1000mb 0 33,100,1000 ** 1000 mb u wind [m/s]
> >
> > 43 ENDVARS
> >
> >
> >
> > My ob file is in a netcdf file created using ascii2nc. I named
wind
> speed
> > obs "ws" in the ascii file that I ran the ascii2nc tool on ( I am
aware
> > that
> > I am trying to compare Wind speed and u component of wind, but I
just
> want
> > to see if I can get point_stat working).
> >
> >
> >
> > How should I arrange my fcst and obs arrays? Here is what I have
so
> far:
> >
> >
> >
> > 52 fcst = {
> >
> > 53 field = [
> >
> > 54 {
> >
> > 55 name = "U";
> >
> > 56 level = "P1000";
> >
> > 57 file_type = GRIB2;
> >
> > 58 cat_thresh = [ >80.0 ];
> >
> > 59 }
> >
> > 60 ];
> >
> > 61
> >
> > 62 }
> >
> > 63
> >
> > 64 obs = {
> >
> > 65 field = [
> >
> > 66 {
> >
> > 67 file_type = NETCDF_NCCF;
> >
> > 68 name = "UGRD";
> >
> > 69 level = ["P1000"];
> >
> > 70 }
> >
> > 71 ];
> >
> > 72 }
> >
> >
> >
> > Running the command as follows gives me this error:
> >
> >
> >
> > [tsu at maury2 00_POINT_STAT_WORK]$ point_stat
> >
> >
> >
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_02700F0NL2019042612_
> > 0100_010000-000000wnd_ucmp
> >
> > NAF_sfc_2019042612.nc config.rc -v 1
> >
> > DEBUG 1: Default Config File:
> >
> >
> /software/depot/met/met-
8.0_bugfix/share/met/config/PointStatConfig_default
> >
> > DEBUG 1: User Config File: config.rc
> >
> > GSL_RNG_TYPE=mt19937
> >
> > GSL_RNG_SEED=1726142812
> >
> > DEBUG 1: Forecast File:
> >
> >
> >
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_02700F0NL2019042612_
> > 0100_010000-000000wnd_ucmp
> >
> > DEBUG 1: Observation File: NAF_sfc_2019042612.nc
> >
> > WARNING:
> >
> > WARNING: process_fcst_climo_files() -> no fields matching U/P1000
found
> in
> >
> > file:
> >
> >
> >
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_02700F0NL2019042612_
> > 0100_010000-000000wnd_ucmp
> >
> > WARNING:
> >
> > ERROR :
> >
> > ERROR : process_fcst_climo_files() -> no requested forecast data
found!
> >
> > Exiting...
> >
> > ERROR :
> >
> >
> >
> > Justin
> >
> >
> >
> >
> >
> > Justin Tsu
> >
> > Marine Meteorology Division
> >
> > Data Assimilation/Mesoscale Modeling
> >
> > Building 704 Room 212
> >
> > Naval Research Laboratory, Code 7531
> >
> > 7 Grace Hopper Avenue
> >
> > Monterey, CA 93943-5502
> >
> >
> >
> > Ph. (831) 656-4111
> >
> >
> >
> >
> >
>
>
>
>
------------------------------------------------
Subject: Point Stat Help
From: John Halley Gotway
Time: Wed May 22 15:16:34 2019
Justin,
OK, this is good info. As you point out, the problem is a mismatch in
the
requested observation variable name and the actual observation
variable
name:
DEBUG 3: Rejected: obs type = 1570630
This should be very easy to fix. We just need to figure out how your
observations are named in the file named "raob_2019042612.nc". Just
take a
look at the contents of the "obs_var" variable in that file. For
example,
here's some output from pb2nc:
ncdump -v obs_var out/pb2nc/sample_pb.nc
...
obs_var =
"SPFH",
"TMP",
"HGT",
"UGRD",
"VGRD",
"DPT",
"WIND",
"RH",
"MIXR" ;
}
In this case, the U-component is wind really is called "UGRD". But
obviously in your data it is not. Let say, for example, that it's
called
"UWIND". You just need to edit your Point-Stat config file to tell
it to
use those variables:
*obs = { field = [ { name = "UWIND";
level = ["P1000"]; } ];}*
On Wed, May 22, 2019 at 1:31 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90256 >
>
> Hey John,
>
> Just ran with verbosity set to 3 and discovered that no pairs are
being
> used:
>
> [tsu at maury2 00_POINT_STAT_WORK]$ point_stat
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_00000F0NL2019042612_0100_010000-000000wnd_ucmp
> raob_2019042612.nc config.rc -v 3
> DEBUG 1: Default Config File:
> /software/depot/met/met-
8.0_bugfix/share/met/config/PointStatConfig_default
> DEBUG 1: User Config File: config.rc
> GSL_RNG_TYPE=mt19937
> GSL_RNG_SEED=18446744072713528668
> DEBUG 1: Forecast File:
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_00000F0NL2019042612_0100_010000-000000wnd_ucmp
> DEBUG 1: Observation File: raob_2019042612.nc
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match
for
> VarInfo "UGRD/P1000" in GRIB record 1 of GRIB file
> "GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_00000F0NL2019042612_0100_010000-000000wnd_ucmp".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "UGRD/P1000" in GRIB file
> "GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_00000F0NL2019042612_0100_010000-000000wnd_ucmp".
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Reading data for UGRD/P1000.
> DEBUG 3: Use the matching forecast and observation grids.
> DEBUG 3: Grid Definition: Projection: Mercator Nx: 202 Ny: 202
> Lat_LL_radians: 0.4770 Lon_LL_radians: 2.3860 Lat_UR_radians: 0.8214
> Lon_UR_radians: 1.9489 Mx: 459.8120 My: 459.8003 Bx: 1097.1297 By:
-228.1298
> DEBUG 2: Processing masking regions.
> DEBUG 3: Processing grid mask: FULL
> DEBUG 2: For UGRD/P1000 found 1 forecast levels, 0 climatology mean
> levels, and 0 climatology standard deviation levels.
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Searching 1570630 observations from 82 messages.
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Processing UGRD/P1000 versus UGRD/P1000, for observation
type
> ADPUPA, over region FULL, for interpolation method NEAREST(1), using
0
> pairs.
> DEBUG 3: Number of matched pairs = 0
> DEBUG 3: Observations processed = 1570630
> DEBUG 3: Rejected: SID exclusion = 0
> DEBUG 3: Rejected: obs type = 1570630
> DEBUG 3: Rejected: valid time = 0
> DEBUG 3: Rejected: bad obs value = 0
> DEBUG 3: Rejected: off the grid = 0
> DEBUG 3: Rejected: level mismatch = 0
> DEBUG 3: Rejected: quality marker = 0
> DEBUG 3: Rejected: message type = 0
> DEBUG 3: Rejected: masking region = 0
> DEBUG 3: Rejected: bad fcst value = 0
> DEBUG 3: Rejected: duplicates = 0
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 1: Output file: ./point_stat_000000L_20190426_120000V.stat
> DEBUG 1: Output file: ./point_stat_000000L_20190426_120000V_cnt.txt
>
> I had an issue with this in the past using NAVGEM IEEE flatfile
format
> climatologies that I converted to netcdf using ascii2nc as obs. I
fixed it
> by using python embedding in the name part of the obs dictionary.
Problem
> is I cannot do this here because my obs are originally in hdf5
format and I
> don’t have a way to use python embedding to provide MET with the
necessary
> metadata like read_NRL_binary.py does (I believe somebody from your
team
> provided us with this python script to print out metadata based on a
> flatfile format that we use)
>
> Have you guys had any luck in using ascii2nc generated netcdf files
as
> either forecast or obs in any of the MET stat utilities?
>
> Justin
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Wednesday, May 22, 2019 11:27 AM
> To: Tsu, Mr. Justin
> Subject: Re: [rt.rap.ucar.edu #90256] Point Stat Help
>
> Justin,
>
> When you run Point-Stat, please increase the verbosity level to at
least 3
> (-v 3). That will dump log messages that look something like this:
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> *DEBUG 2: Processing TMP/P900-750 versus TMP/P900-750, for
observation type
> ADPUPA, over region DTC165, for interpolation method NEAREST(1),
using 132
> pairs.DEBUG 3: Number of matched pairs = 132DEBUG 3: Observations
> processed = 89893DEBUG 3: Rejected: SID exclusion = 0DEBUG 3:
Rejected:
> obs type = 79788DEBUG 3: Rejected: valid time = 0DEBUG 3:
> Rejected: bad obs value = 0DEBUG 3: Rejected: off the grid =
5DEBUG 3:
> Rejected: topography = 0DEBUG 3: Rejected: level mismatch =
9219DEBUG
> 3: Rejected: quality marker = 0DEBUG 3: Rejected: message type =
344DEBUG
> 3: Rejected: masking region = 405DEBUG 3: Rejected: bad fcst value =
0DEBUG
> 3: Rejected: duplicates = 0*
>
> For each verification task, these list counts of reasons why
observations
> were not used for that task. These counts should help you determine
why
> you're getting 0 matched pairs. There are many possible reasons
why...
> mismatches in message type, valid time, lat/lon location, vertical
level...
> and so on.
>
> Hopefully these counts will point you in the right direction.
>
> Thanks,
> John
>
> On Tue, May 21, 2019 at 10:50 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu
> >
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90256 >
> >
> > Hi John,
> >
> > So I took your advice and used wgrib to determine that my variable
is
> > indeed named UGRD so I switched this out in my point_Stat config
file.
> Now
> > that config file looks like this:
> >
> > 52 fcst = {
> > 53 field = [
> > 54 {
> > 55 name = "UGRD";
> > 56 level = ["P1000"];
> > 57 file_type = GRIB1;
> > 58 cat_thresh = [ >80.0 ];
> > 59 }
> > 60 ];
> > 61
> > 62 }
> > 63
> > 64 obs = {
> > 65 field = [
> > 66 {
> > 67 file_type = NETCDF_NCCF;
> > 68 name = "UGRD";
> > 69 level = ["P1000"];
> > 70 }
> > 71 ];
> > 72 }
> >
> > This config file runs when I execute:
> >
> > point_stat
> >
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_02700F0NL2019042612_0100_010000-000000wnd_ucmp
> > NAF_sfc_2019042612.nc config.rc -v 1
> >
> > However, the resulting contingency table file are empty (aside
from the
> > header). I am guessing this has something to do with the way I
specify
> my
> > obs file. As I stated previously in my email, the obs file is
named
> > raob_2019042612.nc and is a netcdf file created by using ascii2nc
on a
> > list of point obs arranged in the format specified by the MET
tutorial.
> I
> > am not sure what the name value in the field dictionary refers to
because
> > the netcdf header for raob_2019042612.nc looks like:
> >
> > 1 netcdf raob_2019042612 {
> > 2 dimensions:
> > 3 mxstr = 16 ;
> > 4 mxstr2 = 40 ;
> > 5 mxstr3 = 80 ;
> > 6 nobs = 1570630 ;
> > 7 nhdr = 82 ;
> > 8 nhdr_typ = 1 ;
> > 9 nhdr_sid = 82 ;
> > 10 nhdr_vld = 1 ;
> > 11 nobs_qty = 1 ;
> > 12 obs_var_num = 6 ;
> > 13 variables:
> > 14 int hdr_typ(nhdr) ;
> > 15 hdr_typ:long_name = "index of message type" ;
> > 16 hdr_typ:_FillValue = -9999 ;
> > 17 int hdr_sid(nhdr) ;
> > 18 hdr_sid:long_name = "index of station identification"
;
> > 19 hdr_sid:_FillValue = -9999 ;
> > 20 int hdr_vld(nhdr) ;
> > 21 hdr_vld:long_name = "index of valid time" ;
> > 22 hdr_vld:_FillValue = -9999 ;
> > 23 float hdr_lat(nhdr) ;
> > 24 hdr_lat:long_name = "latitude" ;
> > 25 hdr_lat:_FillValue = -9999.f ;
> > 26 hdr_lat:units = "degrees_north" ;
> > 27 float hdr_lon(nhdr) ;
> > 28 hdr_lon:long_name = "longitude" ;
> > 29 hdr_lon:_FillValue = -9999.f ;
> > 30 hdr_lon:units = "degrees_east" ;
> > 31 float hdr_elv(nhdr) ;
> > 32 hdr_elv:long_name = "elevation" ;
> > 33 hdr_elv:_FillValue = -9999.f ;
> > 34 hdr_elv:units = "meters above sea level (msl)" ;
> > 35 int obs_qty(nobs) ;
> > 36 obs_qty:long_name = "index of quality flag" ;
> > 37 int obs_hid(nobs) ;
> > 38 obs_hid:long_name = "index of matching header data" ;
> > 39 obs_hid:_FillValue = -9999 ;
> > 40 int obs_vid(nobs) ;
> > 41 obs_vid:long_name = "index of variable names at
var_name" ;
> > 42 obs_vid:_FillValue = -9999 ;
> > 43 float obs_lvl(nobs) ;
> > 44 obs_lvl:long_name = "pressure level (hPa) or
accumulation
> > interval (sec)" ;
> > 45 obs_lvl:_FillValue = -9999.f ;
> > 46 float obs_hgt(nobs) ;
> > 47 obs_hgt:long_name = "height in meters above sea level
or
> > ground level (msl or agl)" ;
> > 48 obs_hgt:_FillValue = -9999.f ;
> > 49 float obs_val(nobs) ;
> > 50 obs_val:long_name = "observation value" ;
> > 51 obs_val:_FillValue = -9999.f ;
> > 52 char hdr_typ_table(nhdr_typ, mxstr2) ;
> > 53 hdr_typ_table:long_name = "message type" ;
> > 54 char hdr_sid_table(nhdr_sid, mxstr2) ;
> > 55 hdr_sid_table:long_name = "station identification" ;
> > 56 char hdr_vld_table(nhdr_vld, mxstr) ;
> > 57 hdr_vld_table:long_name = "valid time" ;
> > 58 hdr_vld_table:units = "YYYYMMDD_HHMMSS UTC" ;
> > 59 char obs_qty_table(nobs_qty, mxstr) ;
> > 60 obs_qty_table:long_name = "quality flag" ;
> > 61 char obs_var(obs_var_num, mxstr2) ;
> > 62 obs_var:long_name = "variable names" ;
> > 63
> > 64 // global attributes:
> > 65 :MET_Obs_version = "1.02" ;
> > 66 :use_var_id = "true" ;
> > 67 :FileOrigins = "File / generated 20190510_235941 UTC
on host
> > maury2.local by the MET ascii2nc tool" ;
> > 68 :MET_version = "V8.0" ;
> > 69 :MET_tool = "ascii2nc" ;
> >
> > So there isn’t actualy a netcdf variable named UGRD; at this point
I'm
> not
> > sure what I should call it.
> >
> > Hope you can shed some light on how this works.
> >
> > Thanks,
> > Justin
> >
> >
> > -----Original Message-----
> > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > Sent: Monday, May 20, 2019 12:20 PM
> > To: Tsu, Mr. Justin
> > Subject: Re: [rt.rap.ucar.edu #90256] Point Stat Help
> >
> > Hi Justin,
> >
> > I see that you're having trouble reading GRIB data into the Point-
Stat
> tool
> > in MET. When getting going with a new gridded dataset in MET, I
usually
> > recommend starting with the plot_data_plane utility. For example,
try
> > running this command:
> >
> > plot_data_plane
> >
> >
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_0270F0NL2019042612_0100_010000-000000wnd_ucmp
> > U_P1000.ps 'name="U"; level="P1000";'
> >
> > You should get a similar error since Point-Stat wasn't able to
read it.
> > I'd recommend running this file through the "wgrib2" utility to
inventory
> > it and see what variable names/levels are present in the file.
For
> > example, is it called "U" or "UGRD"?
> >
> > If you aren't able to get that working, please just post a sample
file to
> > our anonymous ftp site. If you do, let me know, and I'll go grab
it and
> > take a look:
> >
> >
> https://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk
> >
> > Thanks,
> > John Halley Gotway
> >
> >
> >
> > On Fri, May 17, 2019 at 3:27 PM Tsu, Mr. Justin via RT <
> met_help at ucar.edu>
> > wrote:
> >
> > >
> > > Fri May 17 15:27:39 2019: Request 90256 was acted upon.
> > > Transaction: Ticket created by justin.tsu at nrlmry.navy.mil
> > > Queue: met_help
> > > Subject: Point Stat Help
> > > Owner: Nobody
> > > Requestors: justin.tsu at nrlmry.navy.mil
> > > Status: new
> > > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90256
> >
> > >
> > >
> > > Ive got my forecast field in the form of a grib file entitled
> > >
> > >
> > >
> > >
> > >
> >
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_02700F0NL2019042612_
> > > 0100_010000-000000wnd_ucmp
> > >
> > >
> > >
> > > This is the U Component of wind at 1000 mb. The following is a
> > descriptor
> > > file to read this in GrADS:
> > >
> > >
> > >
> > > 1 dset
> > >
> > >
> >
> ^US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_03000F0NL2019042612_0100_0100
> > > 00-000000wnd_ucmp
> > >
> > > 2 index
> > >
> > >
> >
> ^US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_03000F0NL2019042612_0100_0100
> > > 00-000000wnd_ucmp.idx
> > >
> > > 3 undef 9.999E+20
> > >
> > > 4 title
> > >
> > >
> >
> US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_03000F0NL2019042612_0100_01000
> > > 0-000000wnd_ucmp
> > >
> > > 5 * produced by grib2ctl v0.9.15
> > >
> > > 6 * command line options:
> > >
> > >
> >
> US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_03000F0NL2019042612_0100_01000
> > > 0-000000wnd_ucmp
> > >
> > > 7 dtype grib 255
> > >
> > > 8 ydef 202 levels
> > >
> > > 9 27.3280 27.4386 27.5492 27.6596 27.7699 27.8801
> 27.9902
> > >
> > > 10 28.1002 28.2101 28.3198 28.4295 28.5390 28.6484
> 28.7577
> > >
> > > 11 28.8669 28.9759 29.0849 29.1937 29.3025 29.4111
> 29.5196
> > >
> > > 12 29.6279 29.7362 29.8443 29.9524 30.0603 30.1681
> 30.2757
> > >
> > > 13 30.3833 30.4907 30.5980 30.7052 30.8123 30.9193
> 31.0261
> > >
> > > 14 31.1329 31.2395 31.3459 31.4523 31.5585 31.6647
> 31.7707
> > >
> > > 15 31.8765 31.9823 32.0879 32.1934 32.2988 32.4041
> 32.5093
> > >
> > > 16 32.6143 32.7192 32.8240 32.9286 33.0331 33.1375
> 33.2418
> > >
> > > 17 33.3460 33.4500 33.5539 33.6577 33.7614 33.8649
> 33.9683
> > >
> > > 18 34.0716 34.1747 34.2778 34.3807 34.4835 34.5861
> 34.6886
> > >
> > > 19 34.7910 34.8933 34.9954 35.0975 35.1993 35.3011
> 35.4027
> > >
> > > 20 35.5042 35.6056 35.7069 35.8080 35.9090 36.0099
> 36.1106
> > >
> > > 21 36.2112 36.3117 36.4120 36.5122 36.6123 36.7123
> 36.8121
> > >
> > > 22 36.9118 37.0114 37.1108 37.2101 37.3093 37.4083
> 37.5073
> > >
> > > 23 37.6060 37.7047 37.8032 37.9016 37.9999 38.0980
> 38.1960
> > >
> > > 24 38.2939 38.3916 38.4892 38.5867 38.6840 38.7812
> 38.8783
> > >
> > > 25 38.9752 39.0720 39.1687 39.2653 39.3617 39.4579
> 39.5541
> > >
> > > 26 39.6501 39.7460 39.8417 39.9373 40.0328 40.1281
> 40.2234
> > >
> > > 27 40.3184 40.4134 40.5082 40.6029 40.6974 40.7918
> 40.8861
> > >
> > > 28 40.9802 41.0742 41.1681 41.2618 41.3554 41.4489
> 41.5422
> > >
> > > 29 41.6354 41.7285 41.8214 41.9142 42.0069 42.0994
> 42.1918
> > >
> > > 30 42.2841 42.3762 42.4682 42.5600 42.6517 42.7433
> 42.8348
> > >
> > > 31 42.9261 43.0173 43.1083 43.1992 43.2900 43.3806
> 43.4711
> > >
> > > 32 43.5615 43.6517 43.7418 43.8318 43.9216 44.0113
> 44.1008
> > >
> > > 33 44.1902 44.2795 44.3687 44.4577 44.5466 44.6353
> 44.7239
> > >
> > > 34 44.8124 44.9007 44.9889 45.0770 45.1649 45.2527
> 45.3403
> > >
> > > 35 45.4278 45.5152 45.6025 45.6896 45.7766 45.8634
> 45.9501
> > >
> > > 36 46.0367 46.1231 46.2094 46.2956 46.3816 46.4675
> 46.5533
> > >
> > > 37 46.6389 46.7244 46.8098 46.8950 46.9801 47.0650
> > >
> > > 38 xdef 202 linear -136.710000 0.124606965174129
> > >
> > > 39 tdef 1 linear 12Z26apr2019 1mo
> > >
> > > 40 zdef 1 linear 1 1
> > >
> > > 41 vars 1
> > >
> > > 42 UGRD1000mb 0 33,100,1000 ** 1000 mb u wind [m/s]
> > >
> > > 43 ENDVARS
> > >
> > >
> > >
> > > My ob file is in a netcdf file created using ascii2nc. I named
wind
> > speed
> > > obs "ws" in the ascii file that I ran the ascii2nc tool on ( I
am aware
> > > that
> > > I am trying to compare Wind speed and u component of wind, but I
just
> > want
> > > to see if I can get point_stat working).
> > >
> > >
> > >
> > > How should I arrange my fcst and obs arrays? Here is what I
have so
> > far:
> > >
> > >
> > >
> > > 52 fcst = {
> > >
> > > 53 field = [
> > >
> > > 54 {
> > >
> > > 55 name = "U";
> > >
> > > 56 level = "P1000";
> > >
> > > 57 file_type = GRIB2;
> > >
> > > 58 cat_thresh = [ >80.0 ];
> > >
> > > 59 }
> > >
> > > 60 ];
> > >
> > > 61
> > >
> > > 62 }
> > >
> > > 63
> > >
> > > 64 obs = {
> > >
> > > 65 field = [
> > >
> > > 66 {
> > >
> > > 67 file_type = NETCDF_NCCF;
> > >
> > > 68 name = "UGRD";
> > >
> > > 69 level = ["P1000"];
> > >
> > > 70 }
> > >
> > > 71 ];
> > >
> > > 72 }
> > >
> > >
> > >
> > > Running the command as follows gives me this error:
> > >
> > >
> > >
> > > [tsu at maury2 00_POINT_STAT_WORK]$ point_stat
> > >
> > >
> > >
> >
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_02700F0NL2019042612_
> > > 0100_010000-000000wnd_ucmp
> > >
> > > NAF_sfc_2019042612.nc config.rc -v 1
> > >
> > > DEBUG 1: Default Config File:
> > >
> > >
> >
> /software/depot/met/met-
8.0_bugfix/share/met/config/PointStatConfig_default
> > >
> > > DEBUG 1: User Config File: config.rc
> > >
> > > GSL_RNG_TYPE=mt19937
> > >
> > > GSL_RNG_SEED=1726142812
> > >
> > > DEBUG 1: Forecast File:
> > >
> > >
> > >
> >
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_02700F0NL2019042612_
> > > 0100_010000-000000wnd_ucmp
> > >
> > > DEBUG 1: Observation File: NAF_sfc_2019042612.nc
> > >
> > > WARNING:
> > >
> > > WARNING: process_fcst_climo_files() -> no fields matching
U/P1000 found
> > in
> > >
> > > file:
> > >
> > >
> > >
> >
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_02700F0NL2019042612_
> > > 0100_010000-000000wnd_ucmp
> > >
> > > WARNING:
> > >
> > > ERROR :
> > >
> > > ERROR : process_fcst_climo_files() -> no requested forecast
data
> found!
> > >
> > > Exiting...
> > >
> > > ERROR :
> > >
> > >
> > >
> > > Justin
> > >
> > >
> > >
> > >
> > >
> > > Justin Tsu
> > >
> > > Marine Meteorology Division
> > >
> > > Data Assimilation/Mesoscale Modeling
> > >
> > > Building 704 Room 212
> > >
> > > Naval Research Laboratory, Code 7531
> > >
> > > 7 Grace Hopper Avenue
> > >
> > > Monterey, CA 93943-5502
> > >
> > >
> > >
> > > Ph. (831) 656-4111
> > >
> > >
> > >
> > >
> > >
> >
> >
> >
> >
>
>
>
------------------------------------------------
Subject: Point Stat Help
From: Tsu, Mr. Justin
Time: Wed May 22 15:35:05 2019
Awesome!
Heres what I get from dumping obv_var
data:
obs_var =
"ws",
"wdir",
"t",
"dptd",
"pres",
"ght" ;
}
I'm assuming I'll need ws here. If I specify my obs to be at P1000,
will point_stat use the interp that I specified to find the obs
nearest to that height? Also is there a way to verify this components
of wind with wind speed since for this example I am only using UGRD
(u component)? I'm guessing I will have to specify a second field or
something.
Justin
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Wednesday, May 22, 2019 2:17 PM
To: Tsu, Mr. Justin
Subject: Re: [rt.rap.ucar.edu #90256] Point Stat Help
Justin,
OK, this is good info. As you point out, the problem is a mismatch in
the
requested observation variable name and the actual observation
variable
name:
DEBUG 3: Rejected: obs type = 1570630
This should be very easy to fix. We just need to figure out how your
observations are named in the file named "raob_2019042612.nc". Just
take a
look at the contents of the "obs_var" variable in that file. For
example,
here's some output from pb2nc:
ncdump -v obs_var out/pb2nc/sample_pb.nc
...
obs_var =
"SPFH",
"TMP",
"HGT",
"UGRD",
"VGRD",
"DPT",
"WIND",
"RH",
"MIXR" ;
}
In this case, the U-component is wind really is called "UGRD". But
obviously in your data it is not. Let say, for example, that it's
called
"UWIND". You just need to edit your Point-Stat config file to tell
it to
use those variables:
*obs = { field = [ { name = "UWIND";
level = ["P1000"]; } ];}*
On Wed, May 22, 2019 at 1:31 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90256 >
>
> Hey John,
>
> Just ran with verbosity set to 3 and discovered that no pairs are
being
> used:
>
> [tsu at maury2 00_POINT_STAT_WORK]$ point_stat
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_00000F0NL2019042612_0100_010000-000000wnd_ucmp
> raob_2019042612.nc config.rc -v 3
> DEBUG 1: Default Config File:
> /software/depot/met/met-
8.0_bugfix/share/met/config/PointStatConfig_default
> DEBUG 1: User Config File: config.rc
> GSL_RNG_TYPE=mt19937
> GSL_RNG_SEED=18446744072713528668
> DEBUG 1: Forecast File:
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_00000F0NL2019042612_0100_010000-000000wnd_ucmp
> DEBUG 1: Observation File: raob_2019042612.nc
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match
for
> VarInfo "UGRD/P1000" in GRIB record 1 of GRIB file
> "GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_00000F0NL2019042612_0100_010000-000000wnd_ucmp".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "UGRD/P1000" in GRIB file
> "GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_00000F0NL2019042612_0100_010000-000000wnd_ucmp".
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Reading data for UGRD/P1000.
> DEBUG 3: Use the matching forecast and observation grids.
> DEBUG 3: Grid Definition: Projection: Mercator Nx: 202 Ny: 202
> Lat_LL_radians: 0.4770 Lon_LL_radians: 2.3860 Lat_UR_radians: 0.8214
> Lon_UR_radians: 1.9489 Mx: 459.8120 My: 459.8003 Bx: 1097.1297 By:
-228.1298
> DEBUG 2: Processing masking regions.
> DEBUG 3: Processing grid mask: FULL
> DEBUG 2: For UGRD/P1000 found 1 forecast levels, 0 climatology mean
> levels, and 0 climatology standard deviation levels.
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Searching 1570630 observations from 82 messages.
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Processing UGRD/P1000 versus UGRD/P1000, for observation
type
> ADPUPA, over region FULL, for interpolation method NEAREST(1), using
0
> pairs.
> DEBUG 3: Number of matched pairs = 0
> DEBUG 3: Observations processed = 1570630
> DEBUG 3: Rejected: SID exclusion = 0
> DEBUG 3: Rejected: obs type = 1570630
> DEBUG 3: Rejected: valid time = 0
> DEBUG 3: Rejected: bad obs value = 0
> DEBUG 3: Rejected: off the grid = 0
> DEBUG 3: Rejected: level mismatch = 0
> DEBUG 3: Rejected: quality marker = 0
> DEBUG 3: Rejected: message type = 0
> DEBUG 3: Rejected: masking region = 0
> DEBUG 3: Rejected: bad fcst value = 0
> DEBUG 3: Rejected: duplicates = 0
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 1: Output file: ./point_stat_000000L_20190426_120000V.stat
> DEBUG 1: Output file: ./point_stat_000000L_20190426_120000V_cnt.txt
>
> I had an issue with this in the past using NAVGEM IEEE flatfile
format
> climatologies that I converted to netcdf using ascii2nc as obs. I
fixed it
> by using python embedding in the name part of the obs dictionary.
Problem
> is I cannot do this here because my obs are originally in hdf5
format and I
> don’t have a way to use python embedding to provide MET with the
necessary
> metadata like read_NRL_binary.py does (I believe somebody from your
team
> provided us with this python script to print out metadata based on a
> flatfile format that we use)
>
> Have you guys had any luck in using ascii2nc generated netcdf files
as
> either forecast or obs in any of the MET stat utilities?
>
> Justin
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Wednesday, May 22, 2019 11:27 AM
> To: Tsu, Mr. Justin
> Subject: Re: [rt.rap.ucar.edu #90256] Point Stat Help
>
> Justin,
>
> When you run Point-Stat, please increase the verbosity level to at
least 3
> (-v 3). That will dump log messages that look something like this:
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> *DEBUG 2: Processing TMP/P900-750 versus TMP/P900-750, for
observation type
> ADPUPA, over region DTC165, for interpolation method NEAREST(1),
using 132
> pairs.DEBUG 3: Number of matched pairs = 132DEBUG 3: Observations
> processed = 89893DEBUG 3: Rejected: SID exclusion = 0DEBUG 3:
Rejected:
> obs type = 79788DEBUG 3: Rejected: valid time = 0DEBUG 3:
> Rejected: bad obs value = 0DEBUG 3: Rejected: off the grid =
5DEBUG 3:
> Rejected: topography = 0DEBUG 3: Rejected: level mismatch =
9219DEBUG
> 3: Rejected: quality marker = 0DEBUG 3: Rejected: message type =
344DEBUG
> 3: Rejected: masking region = 405DEBUG 3: Rejected: bad fcst value =
0DEBUG
> 3: Rejected: duplicates = 0*
>
> For each verification task, these list counts of reasons why
observations
> were not used for that task. These counts should help you determine
why
> you're getting 0 matched pairs. There are many possible reasons
why...
> mismatches in message type, valid time, lat/lon location, vertical
level...
> and so on.
>
> Hopefully these counts will point you in the right direction.
>
> Thanks,
> John
>
> On Tue, May 21, 2019 at 10:50 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu
> >
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90256 >
> >
> > Hi John,
> >
> > So I took your advice and used wgrib to determine that my variable
is
> > indeed named UGRD so I switched this out in my point_Stat config
file.
> Now
> > that config file looks like this:
> >
> > 52 fcst = {
> > 53 field = [
> > 54 {
> > 55 name = "UGRD";
> > 56 level = ["P1000"];
> > 57 file_type = GRIB1;
> > 58 cat_thresh = [ >80.0 ];
> > 59 }
> > 60 ];
> > 61
> > 62 }
> > 63
> > 64 obs = {
> > 65 field = [
> > 66 {
> > 67 file_type = NETCDF_NCCF;
> > 68 name = "UGRD";
> > 69 level = ["P1000"];
> > 70 }
> > 71 ];
> > 72 }
> >
> > This config file runs when I execute:
> >
> > point_stat
> >
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_02700F0NL2019042612_0100_010000-000000wnd_ucmp
> > NAF_sfc_2019042612.nc config.rc -v 1
> >
> > However, the resulting contingency table file are empty (aside
from the
> > header). I am guessing this has something to do with the way I
specify
> my
> > obs file. As I stated previously in my email, the obs file is
named
> > raob_2019042612.nc and is a netcdf file created by using ascii2nc
on a
> > list of point obs arranged in the format specified by the MET
tutorial.
> I
> > am not sure what the name value in the field dictionary refers to
because
> > the netcdf header for raob_2019042612.nc looks like:
> >
> > 1 netcdf raob_2019042612 {
> > 2 dimensions:
> > 3 mxstr = 16 ;
> > 4 mxstr2 = 40 ;
> > 5 mxstr3 = 80 ;
> > 6 nobs = 1570630 ;
> > 7 nhdr = 82 ;
> > 8 nhdr_typ = 1 ;
> > 9 nhdr_sid = 82 ;
> > 10 nhdr_vld = 1 ;
> > 11 nobs_qty = 1 ;
> > 12 obs_var_num = 6 ;
> > 13 variables:
> > 14 int hdr_typ(nhdr) ;
> > 15 hdr_typ:long_name = "index of message type" ;
> > 16 hdr_typ:_FillValue = -9999 ;
> > 17 int hdr_sid(nhdr) ;
> > 18 hdr_sid:long_name = "index of station identification"
;
> > 19 hdr_sid:_FillValue = -9999 ;
> > 20 int hdr_vld(nhdr) ;
> > 21 hdr_vld:long_name = "index of valid time" ;
> > 22 hdr_vld:_FillValue = -9999 ;
> > 23 float hdr_lat(nhdr) ;
> > 24 hdr_lat:long_name = "latitude" ;
> > 25 hdr_lat:_FillValue = -9999.f ;
> > 26 hdr_lat:units = "degrees_north" ;
> > 27 float hdr_lon(nhdr) ;
> > 28 hdr_lon:long_name = "longitude" ;
> > 29 hdr_lon:_FillValue = -9999.f ;
> > 30 hdr_lon:units = "degrees_east" ;
> > 31 float hdr_elv(nhdr) ;
> > 32 hdr_elv:long_name = "elevation" ;
> > 33 hdr_elv:_FillValue = -9999.f ;
> > 34 hdr_elv:units = "meters above sea level (msl)" ;
> > 35 int obs_qty(nobs) ;
> > 36 obs_qty:long_name = "index of quality flag" ;
> > 37 int obs_hid(nobs) ;
> > 38 obs_hid:long_name = "index of matching header data" ;
> > 39 obs_hid:_FillValue = -9999 ;
> > 40 int obs_vid(nobs) ;
> > 41 obs_vid:long_name = "index of variable names at
var_name" ;
> > 42 obs_vid:_FillValue = -9999 ;
> > 43 float obs_lvl(nobs) ;
> > 44 obs_lvl:long_name = "pressure level (hPa) or
accumulation
> > interval (sec)" ;
> > 45 obs_lvl:_FillValue = -9999.f ;
> > 46 float obs_hgt(nobs) ;
> > 47 obs_hgt:long_name = "height in meters above sea level
or
> > ground level (msl or agl)" ;
> > 48 obs_hgt:_FillValue = -9999.f ;
> > 49 float obs_val(nobs) ;
> > 50 obs_val:long_name = "observation value" ;
> > 51 obs_val:_FillValue = -9999.f ;
> > 52 char hdr_typ_table(nhdr_typ, mxstr2) ;
> > 53 hdr_typ_table:long_name = "message type" ;
> > 54 char hdr_sid_table(nhdr_sid, mxstr2) ;
> > 55 hdr_sid_table:long_name = "station identification" ;
> > 56 char hdr_vld_table(nhdr_vld, mxstr) ;
> > 57 hdr_vld_table:long_name = "valid time" ;
> > 58 hdr_vld_table:units = "YYYYMMDD_HHMMSS UTC" ;
> > 59 char obs_qty_table(nobs_qty, mxstr) ;
> > 60 obs_qty_table:long_name = "quality flag" ;
> > 61 char obs_var(obs_var_num, mxstr2) ;
> > 62 obs_var:long_name = "variable names" ;
> > 63
> > 64 // global attributes:
> > 65 :MET_Obs_version = "1.02" ;
> > 66 :use_var_id = "true" ;
> > 67 :FileOrigins = "File / generated 20190510_235941 UTC
on host
> > maury2.local by the MET ascii2nc tool" ;
> > 68 :MET_version = "V8.0" ;
> > 69 :MET_tool = "ascii2nc" ;
> >
> > So there isn’t actualy a netcdf variable named UGRD; at this point
I'm
> not
> > sure what I should call it.
> >
> > Hope you can shed some light on how this works.
> >
> > Thanks,
> > Justin
> >
> >
> > -----Original Message-----
> > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > Sent: Monday, May 20, 2019 12:20 PM
> > To: Tsu, Mr. Justin
> > Subject: Re: [rt.rap.ucar.edu #90256] Point Stat Help
> >
> > Hi Justin,
> >
> > I see that you're having trouble reading GRIB data into the Point-
Stat
> tool
> > in MET. When getting going with a new gridded dataset in MET, I
usually
> > recommend starting with the plot_data_plane utility. For example,
try
> > running this command:
> >
> > plot_data_plane
> >
> >
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_0270F0NL2019042612_0100_010000-000000wnd_ucmp
> > U_P1000.ps 'name="U"; level="P1000";'
> >
> > You should get a similar error since Point-Stat wasn't able to
read it.
> > I'd recommend running this file through the "wgrib2" utility to
inventory
> > it and see what variable names/levels are present in the file.
For
> > example, is it called "U" or "UGRD"?
> >
> > If you aren't able to get that working, please just post a sample
file to
> > our anonymous ftp site. If you do, let me know, and I'll go grab
it and
> > take a look:
> >
> >
> https://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk
> >
> > Thanks,
> > John Halley Gotway
> >
> >
> >
> > On Fri, May 17, 2019 at 3:27 PM Tsu, Mr. Justin via RT <
> met_help at ucar.edu>
> > wrote:
> >
> > >
> > > Fri May 17 15:27:39 2019: Request 90256 was acted upon.
> > > Transaction: Ticket created by justin.tsu at nrlmry.navy.mil
> > > Queue: met_help
> > > Subject: Point Stat Help
> > > Owner: Nobody
> > > Requestors: justin.tsu at nrlmry.navy.mil
> > > Status: new
> > > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90256
> >
> > >
> > >
> > > Ive got my forecast field in the form of a grib file entitled
> > >
> > >
> > >
> > >
> > >
> >
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_02700F0NL2019042612_
> > > 0100_010000-000000wnd_ucmp
> > >
> > >
> > >
> > > This is the U Component of wind at 1000 mb. The following is a
> > descriptor
> > > file to read this in GrADS:
> > >
> > >
> > >
> > > 1 dset
> > >
> > >
> >
> ^US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_03000F0NL2019042612_0100_0100
> > > 00-000000wnd_ucmp
> > >
> > > 2 index
> > >
> > >
> >
> ^US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_03000F0NL2019042612_0100_0100
> > > 00-000000wnd_ucmp.idx
> > >
> > > 3 undef 9.999E+20
> > >
> > > 4 title
> > >
> > >
> >
> US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_03000F0NL2019042612_0100_01000
> > > 0-000000wnd_ucmp
> > >
> > > 5 * produced by grib2ctl v0.9.15
> > >
> > > 6 * command line options:
> > >
> > >
> >
> US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_03000F0NL2019042612_0100_01000
> > > 0-000000wnd_ucmp
> > >
> > > 7 dtype grib 255
> > >
> > > 8 ydef 202 levels
> > >
> > > 9 27.3280 27.4386 27.5492 27.6596 27.7699 27.8801
> 27.9902
> > >
> > > 10 28.1002 28.2101 28.3198 28.4295 28.5390 28.6484
> 28.7577
> > >
> > > 11 28.8669 28.9759 29.0849 29.1937 29.3025 29.4111
> 29.5196
> > >
> > > 12 29.6279 29.7362 29.8443 29.9524 30.0603 30.1681
> 30.2757
> > >
> > > 13 30.3833 30.4907 30.5980 30.7052 30.8123 30.9193
> 31.0261
> > >
> > > 14 31.1329 31.2395 31.3459 31.4523 31.5585 31.6647
> 31.7707
> > >
> > > 15 31.8765 31.9823 32.0879 32.1934 32.2988 32.4041
> 32.5093
> > >
> > > 16 32.6143 32.7192 32.8240 32.9286 33.0331 33.1375
> 33.2418
> > >
> > > 17 33.3460 33.4500 33.5539 33.6577 33.7614 33.8649
> 33.9683
> > >
> > > 18 34.0716 34.1747 34.2778 34.3807 34.4835 34.5861
> 34.6886
> > >
> > > 19 34.7910 34.8933 34.9954 35.0975 35.1993 35.3011
> 35.4027
> > >
> > > 20 35.5042 35.6056 35.7069 35.8080 35.9090 36.0099
> 36.1106
> > >
> > > 21 36.2112 36.3117 36.4120 36.5122 36.6123 36.7123
> 36.8121
> > >
> > > 22 36.9118 37.0114 37.1108 37.2101 37.3093 37.4083
> 37.5073
> > >
> > > 23 37.6060 37.7047 37.8032 37.9016 37.9999 38.0980
> 38.1960
> > >
> > > 24 38.2939 38.3916 38.4892 38.5867 38.6840 38.7812
> 38.8783
> > >
> > > 25 38.9752 39.0720 39.1687 39.2653 39.3617 39.4579
> 39.5541
> > >
> > > 26 39.6501 39.7460 39.8417 39.9373 40.0328 40.1281
> 40.2234
> > >
> > > 27 40.3184 40.4134 40.5082 40.6029 40.6974 40.7918
> 40.8861
> > >
> > > 28 40.9802 41.0742 41.1681 41.2618 41.3554 41.4489
> 41.5422
> > >
> > > 29 41.6354 41.7285 41.8214 41.9142 42.0069 42.0994
> 42.1918
> > >
> > > 30 42.2841 42.3762 42.4682 42.5600 42.6517 42.7433
> 42.8348
> > >
> > > 31 42.9261 43.0173 43.1083 43.1992 43.2900 43.3806
> 43.4711
> > >
> > > 32 43.5615 43.6517 43.7418 43.8318 43.9216 44.0113
> 44.1008
> > >
> > > 33 44.1902 44.2795 44.3687 44.4577 44.5466 44.6353
> 44.7239
> > >
> > > 34 44.8124 44.9007 44.9889 45.0770 45.1649 45.2527
> 45.3403
> > >
> > > 35 45.4278 45.5152 45.6025 45.6896 45.7766 45.8634
> 45.9501
> > >
> > > 36 46.0367 46.1231 46.2094 46.2956 46.3816 46.4675
> 46.5533
> > >
> > > 37 46.6389 46.7244 46.8098 46.8950 46.9801 47.0650
> > >
> > > 38 xdef 202 linear -136.710000 0.124606965174129
> > >
> > > 39 tdef 1 linear 12Z26apr2019 1mo
> > >
> > > 40 zdef 1 linear 1 1
> > >
> > > 41 vars 1
> > >
> > > 42 UGRD1000mb 0 33,100,1000 ** 1000 mb u wind [m/s]
> > >
> > > 43 ENDVARS
> > >
> > >
> > >
> > > My ob file is in a netcdf file created using ascii2nc. I named
wind
> > speed
> > > obs "ws" in the ascii file that I ran the ascii2nc tool on ( I
am aware
> > > that
> > > I am trying to compare Wind speed and u component of wind, but I
just
> > want
> > > to see if I can get point_stat working).
> > >
> > >
> > >
> > > How should I arrange my fcst and obs arrays? Here is what I
have so
> > far:
> > >
> > >
> > >
> > > 52 fcst = {
> > >
> > > 53 field = [
> > >
> > > 54 {
> > >
> > > 55 name = "U";
> > >
> > > 56 level = "P1000";
> > >
> > > 57 file_type = GRIB2;
> > >
> > > 58 cat_thresh = [ >80.0 ];
> > >
> > > 59 }
> > >
> > > 60 ];
> > >
> > > 61
> > >
> > > 62 }
> > >
> > > 63
> > >
> > > 64 obs = {
> > >
> > > 65 field = [
> > >
> > > 66 {
> > >
> > > 67 file_type = NETCDF_NCCF;
> > >
> > > 68 name = "UGRD";
> > >
> > > 69 level = ["P1000"];
> > >
> > > 70 }
> > >
> > > 71 ];
> > >
> > > 72 }
> > >
> > >
> > >
> > > Running the command as follows gives me this error:
> > >
> > >
> > >
> > > [tsu at maury2 00_POINT_STAT_WORK]$ point_stat
> > >
> > >
> > >
> >
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_02700F0NL2019042612_
> > > 0100_010000-000000wnd_ucmp
> > >
> > > NAF_sfc_2019042612.nc config.rc -v 1
> > >
> > > DEBUG 1: Default Config File:
> > >
> > >
> >
> /software/depot/met/met-
8.0_bugfix/share/met/config/PointStatConfig_default
> > >
> > > DEBUG 1: User Config File: config.rc
> > >
> > > GSL_RNG_TYPE=mt19937
> > >
> > > GSL_RNG_SEED=1726142812
> > >
> > > DEBUG 1: Forecast File:
> > >
> > >
> > >
> >
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_02700F0NL2019042612_
> > > 0100_010000-000000wnd_ucmp
> > >
> > > DEBUG 1: Observation File: NAF_sfc_2019042612.nc
> > >
> > > WARNING:
> > >
> > > WARNING: process_fcst_climo_files() -> no fields matching
U/P1000 found
> > in
> > >
> > > file:
> > >
> > >
> > >
> >
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_02700F0NL2019042612_
> > > 0100_010000-000000wnd_ucmp
> > >
> > > WARNING:
> > >
> > > ERROR :
> > >
> > > ERROR : process_fcst_climo_files() -> no requested forecast
data
> found!
> > >
> > > Exiting...
> > >
> > > ERROR :
> > >
> > >
> > >
> > > Justin
> > >
> > >
> > >
> > >
> > >
> > > Justin Tsu
> > >
> > > Marine Meteorology Division
> > >
> > > Data Assimilation/Mesoscale Modeling
> > >
> > > Building 704 Room 212
> > >
> > > Naval Research Laboratory, Code 7531
> > >
> > > 7 Grace Hopper Avenue
> > >
> > > Monterey, CA 93943-5502
> > >
> > >
> > >
> > > Ph. (831) 656-4111
> > >
> > >
> > >
> > >
> > >
> >
> >
> >
> >
>
>
>
------------------------------------------------
Subject: Point Stat Help
From: John Halley Gotway
Time: Wed May 22 15:50:51 2019
Justin,
One nice feature in Point-Stat is that when you request "WIND" (i.e.
the
GRIB abbreviation for wind speed), it will find the corresponding
"UGRD"
and "VGRD" records and derive wind speed for you on the fly.
So please try:
*fcst = {*
* field = [ { name = "WIND"; level = "P1000"; } ];*
*}*
*obs = {*
* field = [ { name ="ws"; level = "P975-1025"; } ];*
*}*
This tells Point-Stat to:
(1) Since WIND/P1000 doesn't exist in the input file, it will read
UGRD/P1000 and VGRD/P1000 and derive wind speed.
(2) Look through all of the point observations and consider the ones
named
"ws". Any of them with a pressure level between 975 and 1025 will be
verified against the forecast wind speed value at P1000.
If you want to do vertical interpolation, you'll need to configure it
a bit
differently. Let's say you have model output every 100 mb. You could
configure it like this:
*fcst = {*
* field = [ { name = "WIND"; level = "P900-1000"; } ];*
*}*
*obs = {*
* field = [ { name ="ws"; level = "P900-1000"; } ];*
*}*
In this case Point-Stat will compute wind speed for both the 900 and
1000
mb levels. It'll only use obs falling between 900mb and 1000mb. For
example, for an obs at 925mb, it'll first interpolate horizontally to
the
lat/lon observation location at the forecast levels above and below.
Then
it'll use those 2 values to interpolate to the obs pressure level,
linear
in the log of pressure.
So the functionality is really controlled by how the fcst/obs levels
are
defined in the config file.
Thanks,
John
On Wed, May 22, 2019 at 3:35 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90256 >
>
> Awesome!
>
> Heres what I get from dumping obv_var
>
> data:
>
> obs_var =
> "ws",
> "wdir",
> "t",
> "dptd",
> "pres",
> "ght" ;
> }
>
> I'm assuming I'll need ws here. If I specify my obs to be at P1000,
will
> point_stat use the interp that I specified to find the obs nearest
to that
> height? Also is there a way to verify this components of wind with
wind
> speed since for this example I am only using UGRD (u component)?
I'm
> guessing I will have to specify a second field or something.
>
> Justin
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Wednesday, May 22, 2019 2:17 PM
> To: Tsu, Mr. Justin
> Subject: Re: [rt.rap.ucar.edu #90256] Point Stat Help
>
> Justin,
>
> OK, this is good info. As you point out, the problem is a mismatch
in the
> requested observation variable name and the actual observation
variable
> name:
> DEBUG 3: Rejected: obs type = 1570630
>
> This should be very easy to fix. We just need to figure out how
your
> observations are named in the file named "raob_2019042612.nc". Just
take
> a
> look at the contents of the "obs_var" variable in that file. For
example,
> here's some output from pb2nc:
>
> ncdump -v obs_var out/pb2nc/sample_pb.nc
> ...
> obs_var =
> "SPFH",
> "TMP",
> "HGT",
> "UGRD",
> "VGRD",
> "DPT",
> "WIND",
> "RH",
> "MIXR" ;
> }
>
> In this case, the U-component is wind really is called "UGRD". But
> obviously in your data it is not. Let say, for example, that it's
called
> "UWIND". You just need to edit your Point-Stat config file to tell
it to
> use those variables:
>
>
>
>
>
>
>
>
> *obs = { field = [ { name = "UWIND";
> level = ["P1000"]; } ];}*
>
> On Wed, May 22, 2019 at 1:31 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90256 >
> >
> > Hey John,
> >
> > Just ran with verbosity set to 3 and discovered that no pairs are
being
> > used:
> >
> > [tsu at maury2 00_POINT_STAT_WORK]$ point_stat
> >
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_00000F0NL2019042612_0100_010000-000000wnd_ucmp
> > raob_2019042612.nc config.rc -v 3
> > DEBUG 1: Default Config File:
> >
> /software/depot/met/met-
8.0_bugfix/share/met/config/PointStatConfig_default
> > DEBUG 1: User Config File: config.rc
> > GSL_RNG_TYPE=mt19937
> > GSL_RNG_SEED=18446744072713528668
> > DEBUG 1: Forecast File:
> >
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_00000F0NL2019042612_0100_010000-000000wnd_ucmp
> > DEBUG 1: Observation File: raob_2019042612.nc
> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match
for
> > VarInfo "UGRD/P1000" in GRIB record 1 of GRIB file
> >
> "GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_00000F0NL2019042612_0100_010000-000000wnd_ucmp".
> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> > matching VarInfo "UGRD/P1000" in GRIB file
> >
> "GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_00000F0NL2019042612_0100_010000-000000wnd_ucmp".
> > DEBUG 2:
> > DEBUG 2:
> >
>
--------------------------------------------------------------------------------
> > DEBUG 2:
> > DEBUG 2: Reading data for UGRD/P1000.
> > DEBUG 3: Use the matching forecast and observation grids.
> > DEBUG 3: Grid Definition: Projection: Mercator Nx: 202 Ny: 202
> > Lat_LL_radians: 0.4770 Lon_LL_radians: 2.3860 Lat_UR_radians:
0.8214
> > Lon_UR_radians: 1.9489 Mx: 459.8120 My: 459.8003 Bx: 1097.1297 By:
> -228.1298
> > DEBUG 2: Processing masking regions.
> > DEBUG 3: Processing grid mask: FULL
> > DEBUG 2: For UGRD/P1000 found 1 forecast levels, 0 climatology
mean
> > levels, and 0 climatology standard deviation levels.
> > DEBUG 2:
> > DEBUG 2:
> >
>
--------------------------------------------------------------------------------
> > DEBUG 2:
> > DEBUG 2: Searching 1570630 observations from 82 messages.
> > DEBUG 2:
> > DEBUG 2:
> >
>
--------------------------------------------------------------------------------
> > DEBUG 2:
> > DEBUG 2: Processing UGRD/P1000 versus UGRD/P1000, for observation
type
> > ADPUPA, over region FULL, for interpolation method NEAREST(1),
using 0
> > pairs.
> > DEBUG 3: Number of matched pairs = 0
> > DEBUG 3: Observations processed = 1570630
> > DEBUG 3: Rejected: SID exclusion = 0
> > DEBUG 3: Rejected: obs type = 1570630
> > DEBUG 3: Rejected: valid time = 0
> > DEBUG 3: Rejected: bad obs value = 0
> > DEBUG 3: Rejected: off the grid = 0
> > DEBUG 3: Rejected: level mismatch = 0
> > DEBUG 3: Rejected: quality marker = 0
> > DEBUG 3: Rejected: message type = 0
> > DEBUG 3: Rejected: masking region = 0
> > DEBUG 3: Rejected: bad fcst value = 0
> > DEBUG 3: Rejected: duplicates = 0
> > DEBUG 2:
> > DEBUG 2:
> >
>
--------------------------------------------------------------------------------
> > DEBUG 2:
> > DEBUG 1: Output file: ./point_stat_000000L_20190426_120000V.stat
> > DEBUG 1: Output file:
./point_stat_000000L_20190426_120000V_cnt.txt
> >
> > I had an issue with this in the past using NAVGEM IEEE flatfile
format
> > climatologies that I converted to netcdf using ascii2nc as obs. I
fixed
> it
> > by using python embedding in the name part of the obs dictionary.
> Problem
> > is I cannot do this here because my obs are originally in hdf5
format
> and I
> > don’t have a way to use python embedding to provide MET with the
> necessary
> > metadata like read_NRL_binary.py does (I believe somebody from
your team
> > provided us with this python script to print out metadata based on
a
> > flatfile format that we use)
> >
> > Have you guys had any luck in using ascii2nc generated netcdf
files as
> > either forecast or obs in any of the MET stat utilities?
> >
> > Justin
> >
> > -----Original Message-----
> > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > Sent: Wednesday, May 22, 2019 11:27 AM
> > To: Tsu, Mr. Justin
> > Subject: Re: [rt.rap.ucar.edu #90256] Point Stat Help
> >
> > Justin,
> >
> > When you run Point-Stat, please increase the verbosity level to at
least
> 3
> > (-v 3). That will dump log messages that look something like
this:
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > *DEBUG 2: Processing TMP/P900-750 versus TMP/P900-750, for
observation
> type
> > ADPUPA, over region DTC165, for interpolation method NEAREST(1),
using
> 132
> > pairs.DEBUG 3: Number of matched pairs = 132DEBUG 3: Observations
> > processed = 89893DEBUG 3: Rejected: SID exclusion = 0DEBUG 3:
> Rejected:
> > obs type = 79788DEBUG 3: Rejected: valid time = 0DEBUG
3:
> > Rejected: bad obs value = 0DEBUG 3: Rejected: off the grid =
5DEBUG 3:
> > Rejected: topography = 0DEBUG 3: Rejected: level mismatch =
9219DEBUG
> > 3: Rejected: quality marker = 0DEBUG 3: Rejected: message type =
> 344DEBUG
> > 3: Rejected: masking region = 405DEBUG 3: Rejected: bad fcst value
=
> 0DEBUG
> > 3: Rejected: duplicates = 0*
> >
> > For each verification task, these list counts of reasons why
observations
> > were not used for that task. These counts should help you
determine why
> > you're getting 0 matched pairs. There are many possible reasons
why...
> > mismatches in message type, valid time, lat/lon location, vertical
> level...
> > and so on.
> >
> > Hopefully these counts will point you in the right direction.
> >
> > Thanks,
> > John
> >
> > On Tue, May 21, 2019 at 10:50 PM Tsu, Mr. Justin via RT <
> met_help at ucar.edu
> > >
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90256 >
> > >
> > > Hi John,
> > >
> > > So I took your advice and used wgrib to determine that my
variable is
> > > indeed named UGRD so I switched this out in my point_Stat config
file.
> > Now
> > > that config file looks like this:
> > >
> > > 52 fcst = {
> > > 53 field = [
> > > 54 {
> > > 55 name = "UGRD";
> > > 56 level = ["P1000"];
> > > 57 file_type = GRIB1;
> > > 58 cat_thresh = [ >80.0 ];
> > > 59 }
> > > 60 ];
> > > 61
> > > 62 }
> > > 63
> > > 64 obs = {
> > > 65 field = [
> > > 66 {
> > > 67 file_type = NETCDF_NCCF;
> > > 68 name = "UGRD";
> > > 69 level = ["P1000"];
> > > 70 }
> > > 71 ];
> > > 72 }
> > >
> > > This config file runs when I execute:
> > >
> > > point_stat
> > >
> >
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_02700F0NL2019042612_0100_010000-000000wnd_ucmp
> > > NAF_sfc_2019042612.nc config.rc -v 1
> > >
> > > However, the resulting contingency table file are empty (aside
from the
> > > header). I am guessing this has something to do with the way I
specify
> > my
> > > obs file. As I stated previously in my email, the obs file is
named
> > > raob_2019042612.nc and is a netcdf file created by using
ascii2nc on a
> > > list of point obs arranged in the format specified by the MET
tutorial.
> > I
> > > am not sure what the name value in the field dictionary refers
to
> because
> > > the netcdf header for raob_2019042612.nc looks like:
> > >
> > > 1 netcdf raob_2019042612 {
> > > 2 dimensions:
> > > 3 mxstr = 16 ;
> > > 4 mxstr2 = 40 ;
> > > 5 mxstr3 = 80 ;
> > > 6 nobs = 1570630 ;
> > > 7 nhdr = 82 ;
> > > 8 nhdr_typ = 1 ;
> > > 9 nhdr_sid = 82 ;
> > > 10 nhdr_vld = 1 ;
> > > 11 nobs_qty = 1 ;
> > > 12 obs_var_num = 6 ;
> > > 13 variables:
> > > 14 int hdr_typ(nhdr) ;
> > > 15 hdr_typ:long_name = "index of message type" ;
> > > 16 hdr_typ:_FillValue = -9999 ;
> > > 17 int hdr_sid(nhdr) ;
> > > 18 hdr_sid:long_name = "index of station
identification" ;
> > > 19 hdr_sid:_FillValue = -9999 ;
> > > 20 int hdr_vld(nhdr) ;
> > > 21 hdr_vld:long_name = "index of valid time" ;
> > > 22 hdr_vld:_FillValue = -9999 ;
> > > 23 float hdr_lat(nhdr) ;
> > > 24 hdr_lat:long_name = "latitude" ;
> > > 25 hdr_lat:_FillValue = -9999.f ;
> > > 26 hdr_lat:units = "degrees_north" ;
> > > 27 float hdr_lon(nhdr) ;
> > > 28 hdr_lon:long_name = "longitude" ;
> > > 29 hdr_lon:_FillValue = -9999.f ;
> > > 30 hdr_lon:units = "degrees_east" ;
> > > 31 float hdr_elv(nhdr) ;
> > > 32 hdr_elv:long_name = "elevation" ;
> > > 33 hdr_elv:_FillValue = -9999.f ;
> > > 34 hdr_elv:units = "meters above sea level (msl)" ;
> > > 35 int obs_qty(nobs) ;
> > > 36 obs_qty:long_name = "index of quality flag" ;
> > > 37 int obs_hid(nobs) ;
> > > 38 obs_hid:long_name = "index of matching header data"
;
> > > 39 obs_hid:_FillValue = -9999 ;
> > > 40 int obs_vid(nobs) ;
> > > 41 obs_vid:long_name = "index of variable names at
var_name" ;
> > > 42 obs_vid:_FillValue = -9999 ;
> > > 43 float obs_lvl(nobs) ;
> > > 44 obs_lvl:long_name = "pressure level (hPa) or
accumulation
> > > interval (sec)" ;
> > > 45 obs_lvl:_FillValue = -9999.f ;
> > > 46 float obs_hgt(nobs) ;
> > > 47 obs_hgt:long_name = "height in meters above sea
level or
> > > ground level (msl or agl)" ;
> > > 48 obs_hgt:_FillValue = -9999.f ;
> > > 49 float obs_val(nobs) ;
> > > 50 obs_val:long_name = "observation value" ;
> > > 51 obs_val:_FillValue = -9999.f ;
> > > 52 char hdr_typ_table(nhdr_typ, mxstr2) ;
> > > 53 hdr_typ_table:long_name = "message type" ;
> > > 54 char hdr_sid_table(nhdr_sid, mxstr2) ;
> > > 55 hdr_sid_table:long_name = "station identification" ;
> > > 56 char hdr_vld_table(nhdr_vld, mxstr) ;
> > > 57 hdr_vld_table:long_name = "valid time" ;
> > > 58 hdr_vld_table:units = "YYYYMMDD_HHMMSS UTC" ;
> > > 59 char obs_qty_table(nobs_qty, mxstr) ;
> > > 60 obs_qty_table:long_name = "quality flag" ;
> > > 61 char obs_var(obs_var_num, mxstr2) ;
> > > 62 obs_var:long_name = "variable names" ;
> > > 63
> > > 64 // global attributes:
> > > 65 :MET_Obs_version = "1.02" ;
> > > 66 :use_var_id = "true" ;
> > > 67 :FileOrigins = "File / generated 20190510_235941 UTC
on
> host
> > > maury2.local by the MET ascii2nc tool" ;
> > > 68 :MET_version = "V8.0" ;
> > > 69 :MET_tool = "ascii2nc" ;
> > >
> > > So there isn’t actualy a netcdf variable named UGRD; at this
point I'm
> > not
> > > sure what I should call it.
> > >
> > > Hope you can shed some light on how this works.
> > >
> > > Thanks,
> > > Justin
> > >
> > >
> > > -----Original Message-----
> > > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > > Sent: Monday, May 20, 2019 12:20 PM
> > > To: Tsu, Mr. Justin
> > > Subject: Re: [rt.rap.ucar.edu #90256] Point Stat Help
> > >
> > > Hi Justin,
> > >
> > > I see that you're having trouble reading GRIB data into the
Point-Stat
> > tool
> > > in MET. When getting going with a new gridded dataset in MET, I
> usually
> > > recommend starting with the plot_data_plane utility. For
example, try
> > > running this command:
> > >
> > > plot_data_plane
> > >
> > >
> >
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_0270F0NL2019042612_0100_010000-000000wnd_ucmp
> > > U_P1000.ps 'name="U"; level="P1000";'
> > >
> > > You should get a similar error since Point-Stat wasn't able to
read it.
> > > I'd recommend running this file through the "wgrib2" utility to
> inventory
> > > it and see what variable names/levels are present in the file.
For
> > > example, is it called "U" or "UGRD"?
> > >
> > > If you aren't able to get that working, please just post a
sample file
> to
> > > our anonymous ftp site. If you do, let me know, and I'll go
grab it
> and
> > > take a look:
> > >
> > >
> >
> https://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk
> > >
> > > Thanks,
> > > John Halley Gotway
> > >
> > >
> > >
> > > On Fri, May 17, 2019 at 3:27 PM Tsu, Mr. Justin via RT <
> > met_help at ucar.edu>
> > > wrote:
> > >
> > > >
> > > > Fri May 17 15:27:39 2019: Request 90256 was acted upon.
> > > > Transaction: Ticket created by justin.tsu at nrlmry.navy.mil
> > > > Queue: met_help
> > > > Subject: Point Stat Help
> > > > Owner: Nobody
> > > > Requestors: justin.tsu at nrlmry.navy.mil
> > > > Status: new
> > > > Ticket <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90256
> > >
> > > >
> > > >
> > > > Ive got my forecast field in the form of a grib file entitled
> > > >
> > > >
> > > >
> > > >
> > > >
> > >
> >
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_02700F0NL2019042612_
> > > > 0100_010000-000000wnd_ucmp
> > > >
> > > >
> > > >
> > > > This is the U Component of wind at 1000 mb. The following is
a
> > > descriptor
> > > > file to read this in GrADS:
> > > >
> > > >
> > > >
> > > > 1 dset
> > > >
> > > >
> > >
> >
> ^US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_03000F0NL2019042612_0100_0100
> > > > 00-000000wnd_ucmp
> > > >
> > > > 2 index
> > > >
> > > >
> > >
> >
> ^US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_03000F0NL2019042612_0100_0100
> > > > 00-000000wnd_ucmp.idx
> > > >
> > > > 3 undef 9.999E+20
> > > >
> > > > 4 title
> > > >
> > > >
> > >
> >
> US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_03000F0NL2019042612_0100_01000
> > > > 0-000000wnd_ucmp
> > > >
> > > > 5 * produced by grib2ctl v0.9.15
> > > >
> > > > 6 * command line options:
> > > >
> > > >
> > >
> >
> US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_03000F0NL2019042612_0100_01000
> > > > 0-000000wnd_ucmp
> > > >
> > > > 7 dtype grib 255
> > > >
> > > > 8 ydef 202 levels
> > > >
> > > > 9 27.3280 27.4386 27.5492 27.6596 27.7699
27.8801
> > 27.9902
> > > >
> > > > 10 28.1002 28.2101 28.3198 28.4295 28.5390 28.6484
> > 28.7577
> > > >
> > > > 11 28.8669 28.9759 29.0849 29.1937 29.3025 29.4111
> > 29.5196
> > > >
> > > > 12 29.6279 29.7362 29.8443 29.9524 30.0603 30.1681
> > 30.2757
> > > >
> > > > 13 30.3833 30.4907 30.5980 30.7052 30.8123 30.9193
> > 31.0261
> > > >
> > > > 14 31.1329 31.2395 31.3459 31.4523 31.5585 31.6647
> > 31.7707
> > > >
> > > > 15 31.8765 31.9823 32.0879 32.1934 32.2988 32.4041
> > 32.5093
> > > >
> > > > 16 32.6143 32.7192 32.8240 32.9286 33.0331 33.1375
> > 33.2418
> > > >
> > > > 17 33.3460 33.4500 33.5539 33.6577 33.7614 33.8649
> > 33.9683
> > > >
> > > > 18 34.0716 34.1747 34.2778 34.3807 34.4835 34.5861
> > 34.6886
> > > >
> > > > 19 34.7910 34.8933 34.9954 35.0975 35.1993 35.3011
> > 35.4027
> > > >
> > > > 20 35.5042 35.6056 35.7069 35.8080 35.9090 36.0099
> > 36.1106
> > > >
> > > > 21 36.2112 36.3117 36.4120 36.5122 36.6123 36.7123
> > 36.8121
> > > >
> > > > 22 36.9118 37.0114 37.1108 37.2101 37.3093 37.4083
> > 37.5073
> > > >
> > > > 23 37.6060 37.7047 37.8032 37.9016 37.9999 38.0980
> > 38.1960
> > > >
> > > > 24 38.2939 38.3916 38.4892 38.5867 38.6840 38.7812
> > 38.8783
> > > >
> > > > 25 38.9752 39.0720 39.1687 39.2653 39.3617 39.4579
> > 39.5541
> > > >
> > > > 26 39.6501 39.7460 39.8417 39.9373 40.0328 40.1281
> > 40.2234
> > > >
> > > > 27 40.3184 40.4134 40.5082 40.6029 40.6974 40.7918
> > 40.8861
> > > >
> > > > 28 40.9802 41.0742 41.1681 41.2618 41.3554 41.4489
> > 41.5422
> > > >
> > > > 29 41.6354 41.7285 41.8214 41.9142 42.0069 42.0994
> > 42.1918
> > > >
> > > > 30 42.2841 42.3762 42.4682 42.5600 42.6517 42.7433
> > 42.8348
> > > >
> > > > 31 42.9261 43.0173 43.1083 43.1992 43.2900 43.3806
> > 43.4711
> > > >
> > > > 32 43.5615 43.6517 43.7418 43.8318 43.9216 44.0113
> > 44.1008
> > > >
> > > > 33 44.1902 44.2795 44.3687 44.4577 44.5466 44.6353
> > 44.7239
> > > >
> > > > 34 44.8124 44.9007 44.9889 45.0770 45.1649 45.2527
> > 45.3403
> > > >
> > > > 35 45.4278 45.5152 45.6025 45.6896 45.7766 45.8634
> > 45.9501
> > > >
> > > > 36 46.0367 46.1231 46.2094 46.2956 46.3816 46.4675
> > 46.5533
> > > >
> > > > 37 46.6389 46.7244 46.8098 46.8950 46.9801 47.0650
> > > >
> > > > 38 xdef 202 linear -136.710000 0.124606965174129
> > > >
> > > > 39 tdef 1 linear 12Z26apr2019 1mo
> > > >
> > > > 40 zdef 1 linear 1 1
> > > >
> > > > 41 vars 1
> > > >
> > > > 42 UGRD1000mb 0 33,100,1000 ** 1000 mb u wind [m/s]
> > > >
> > > > 43 ENDVARS
> > > >
> > > >
> > > >
> > > > My ob file is in a netcdf file created using ascii2nc. I
named wind
> > > speed
> > > > obs "ws" in the ascii file that I ran the ascii2nc tool on ( I
am
> aware
> > > > that
> > > > I am trying to compare Wind speed and u component of wind, but
I just
> > > want
> > > > to see if I can get point_stat working).
> > > >
> > > >
> > > >
> > > > How should I arrange my fcst and obs arrays? Here is what I
have so
> > > far:
> > > >
> > > >
> > > >
> > > > 52 fcst = {
> > > >
> > > > 53 field = [
> > > >
> > > > 54 {
> > > >
> > > > 55 name = "U";
> > > >
> > > > 56 level = "P1000";
> > > >
> > > > 57 file_type = GRIB2;
> > > >
> > > > 58 cat_thresh = [ >80.0 ];
> > > >
> > > > 59 }
> > > >
> > > > 60 ];
> > > >
> > > > 61
> > > >
> > > > 62 }
> > > >
> > > > 63
> > > >
> > > > 64 obs = {
> > > >
> > > > 65 field = [
> > > >
> > > > 66 {
> > > >
> > > > 67 file_type = NETCDF_NCCF;
> > > >
> > > > 68 name = "UGRD";
> > > >
> > > > 69 level = ["P1000"];
> > > >
> > > > 70 }
> > > >
> > > > 71 ];
> > > >
> > > > 72 }
> > > >
> > > >
> > > >
> > > > Running the command as follows gives me this error:
> > > >
> > > >
> > > >
> > > > [tsu at maury2 00_POINT_STAT_WORK]$ point_stat
> > > >
> > > >
> > > >
> > >
> >
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_02700F0NL2019042612_
> > > > 0100_010000-000000wnd_ucmp
> > > >
> > > > NAF_sfc_2019042612.nc config.rc -v 1
> > > >
> > > > DEBUG 1: Default Config File:
> > > >
> > > >
> > >
> >
> /software/depot/met/met-
8.0_bugfix/share/met/config/PointStatConfig_default
> > > >
> > > > DEBUG 1: User Config File: config.rc
> > > >
> > > > GSL_RNG_TYPE=mt19937
> > > >
> > > > GSL_RNG_SEED=1726142812
> > > >
> > > > DEBUG 1: Forecast File:
> > > >
> > > >
> > > >
> > >
> >
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_02700F0NL2019042612_
> > > > 0100_010000-000000wnd_ucmp
> > > >
> > > > DEBUG 1: Observation File: NAF_sfc_2019042612.nc
> > > >
> > > > WARNING:
> > > >
> > > > WARNING: process_fcst_climo_files() -> no fields matching
U/P1000
> found
> > > in
> > > >
> > > > file:
> > > >
> > > >
> > > >
> > >
> >
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_02700F0NL2019042612_
> > > > 0100_010000-000000wnd_ucmp
> > > >
> > > > WARNING:
> > > >
> > > > ERROR :
> > > >
> > > > ERROR : process_fcst_climo_files() -> no requested forecast
data
> > found!
> > > >
> > > > Exiting...
> > > >
> > > > ERROR :
> > > >
> > > >
> > > >
> > > > Justin
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > Justin Tsu
> > > >
> > > > Marine Meteorology Division
> > > >
> > > > Data Assimilation/Mesoscale Modeling
> > > >
> > > > Building 704 Room 212
> > > >
> > > > Naval Research Laboratory, Code 7531
> > > >
> > > > 7 Grace Hopper Avenue
> > > >
> > > > Monterey, CA 93943-5502
> > > >
> > > >
> > > >
> > > > Ph. (831) 656-4111
> > > >
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> > >
> > >
> >
> >
> >
>
>
>
------------------------------------------------
Subject: Point Stat Help
From: Tsu, Mr. Justin
Time: Wed May 22 16:39:28 2019
John,
Thank you for the clear explanation. What do I do if my u component
and v component winds are in separate GRIB files?
I am still getting 0 pairs of matched obs even when I set my pressure
range to look for obs between P700 and P1025.
[tsu at maury2 00_POINT_STAT_WORK]$ point_stat GRIB_UCMP/US058GMET-
GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-c1_00000F0NL2019042612_0100_010000-
000000wnd_ucmp raob_2019042612.nc config.rc -v 3
DEBUG 1: Default Config File: /software/depot/met/met-
8.0_bugfix/share/met/config/PointStatConfig_default
DEBUG 1: User Config File: config.rc
GSL_RNG_TYPE=mt19937
GSL_RNG_SEED=18446744071575299420
DEBUG 1: Forecast File: GRIB_UCMP/US058GMET-GR1dyn.COAMPS-
CENCOOS_CENCOOS-n2-c1_00000F0NL2019042612_0100_010000-000000wnd_ucmp
DEBUG 1: Observation File: raob_2019042612.nc
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match for
VarInfo "UGRD/P1000" in GRIB record 1 of GRIB file
"GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_00000F0NL2019042612_0100_010000-000000wnd_ucmp".
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB records
matching VarInfo "UGRD/P1000" in GRIB file "GRIB_UCMP/US058GMET-
GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-c1_00000F0NL2019042612_0100_010000-
000000wnd_ucmp".
DEBUG 2:
DEBUG 2:
--------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Reading data for UGRD/P1000.
DEBUG 3: Use the matching forecast and observation grids.
DEBUG 3: Grid Definition: Projection: Mercator Nx: 202 Ny: 202
Lat_LL_radians: 0.4770 Lon_LL_radians: 2.3860 Lat_UR_radians: 0.8214
Lon_UR_radians: 1.9489 Mx: 459.8120 My: 459.8003 Bx: 1097.1297 By:
-228.1298
DEBUG 2: Processing masking regions.
DEBUG 3: Processing grid mask: FULL
DEBUG 2: For UGRD/P1000 found 1 forecast levels, 0 climatology mean
levels, and 0 climatology standard deviation levels.
DEBUG 2:
DEBUG 2:
--------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Searching 1570630 observations from 82 messages.
DEBUG 2:
DEBUG 2:
--------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Processing UGRD/P1000 versus ws/P1025-700, for observation
type ADPUPA, over region FULL, for interpolation method NEAREST(1),
using 0 pairs.
DEBUG 3: Number of matched pairs = 0
DEBUG 3: Observations processed = 1570630
DEBUG 3: Rejected: SID exclusion = 0
DEBUG 3: Rejected: obs type = 1310954
DEBUG 3: Rejected: valid time = 0
DEBUG 3: Rejected: bad obs value = 0
DEBUG 3: Rejected: off the grid = 225197
DEBUG 3: Rejected: level mismatch = 32048
DEBUG 3: Rejected: quality marker = 0
DEBUG 3: Rejected: message type = 2431
DEBUG 3: Rejected: masking region = 0
DEBUG 3: Rejected: bad fcst value = 0
DEBUG 3: Rejected: duplicates = 0
DEBUG 2:
DEBUG 2:
--------------------------------------------------------------------------------
DEBUG 2:
DEBUG 1: Output file: ./point_stat_000000L_20190426_120000V.stat
DEBUG 1: Output file: ./point_stat_000000L_20190426_120000V_cnt.txt
Justin
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Wednesday, May 22, 2019 2:51 PM
To: Tsu, Mr. Justin
Subject: Re: [rt.rap.ucar.edu #90256] Point Stat Help
Justin,
One nice feature in Point-Stat is that when you request "WIND" (i.e.
the
GRIB abbreviation for wind speed), it will find the corresponding
"UGRD"
and "VGRD" records and derive wind speed for you on the fly.
So please try:
*fcst = {*
* field = [ { name = "WIND"; level = "P1000"; } ];*
*}*
*obs = {*
* field = [ { name ="ws"; level = "P975-1025"; } ];*
*}*
This tells Point-Stat to:
(1) Since WIND/P1000 doesn't exist in the input file, it will read
UGRD/P1000 and VGRD/P1000 and derive wind speed.
(2) Look through all of the point observations and consider the ones
named
"ws". Any of them with a pressure level between 975 and 1025 will be
verified against the forecast wind speed value at P1000.
If you want to do vertical interpolation, you'll need to configure it
a bit
differently. Let's say you have model output every 100 mb. You could
configure it like this:
*fcst = {*
* field = [ { name = "WIND"; level = "P900-1000"; } ];*
*}*
*obs = {*
* field = [ { name ="ws"; level = "P900-1000"; } ];*
*}*
In this case Point-Stat will compute wind speed for both the 900 and
1000
mb levels. It'll only use obs falling between 900mb and 1000mb. For
example, for an obs at 925mb, it'll first interpolate horizontally to
the
lat/lon observation location at the forecast levels above and below.
Then
it'll use those 2 values to interpolate to the obs pressure level,
linear
in the log of pressure.
So the functionality is really controlled by how the fcst/obs levels
are
defined in the config file.
Thanks,
John
On Wed, May 22, 2019 at 3:35 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90256 >
>
> Awesome!
>
> Heres what I get from dumping obv_var
>
> data:
>
> obs_var =
> "ws",
> "wdir",
> "t",
> "dptd",
> "pres",
> "ght" ;
> }
>
> I'm assuming I'll need ws here. If I specify my obs to be at P1000,
will
> point_stat use the interp that I specified to find the obs nearest
to that
> height? Also is there a way to verify this components of wind with
wind
> speed since for this example I am only using UGRD (u component)?
I'm
> guessing I will have to specify a second field or something.
>
> Justin
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Wednesday, May 22, 2019 2:17 PM
> To: Tsu, Mr. Justin
> Subject: Re: [rt.rap.ucar.edu #90256] Point Stat Help
>
> Justin,
>
> OK, this is good info. As you point out, the problem is a mismatch
in the
> requested observation variable name and the actual observation
variable
> name:
> DEBUG 3: Rejected: obs type = 1570630
>
> This should be very easy to fix. We just need to figure out how
your
> observations are named in the file named "raob_2019042612.nc". Just
take
> a
> look at the contents of the "obs_var" variable in that file. For
example,
> here's some output from pb2nc:
>
> ncdump -v obs_var out/pb2nc/sample_pb.nc
> ...
> obs_var =
> "SPFH",
> "TMP",
> "HGT",
> "UGRD",
> "VGRD",
> "DPT",
> "WIND",
> "RH",
> "MIXR" ;
> }
>
> In this case, the U-component is wind really is called "UGRD". But
> obviously in your data it is not. Let say, for example, that it's
called
> "UWIND". You just need to edit your Point-Stat config file to tell
it to
> use those variables:
>
>
>
>
>
>
>
>
> *obs = { field = [ { name = "UWIND";
> level = ["P1000"]; } ];}*
>
> On Wed, May 22, 2019 at 1:31 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90256 >
> >
> > Hey John,
> >
> > Just ran with verbosity set to 3 and discovered that no pairs are
being
> > used:
> >
> > [tsu at maury2 00_POINT_STAT_WORK]$ point_stat
> >
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_00000F0NL2019042612_0100_010000-000000wnd_ucmp
> > raob_2019042612.nc config.rc -v 3
> > DEBUG 1: Default Config File:
> >
> /software/depot/met/met-
8.0_bugfix/share/met/config/PointStatConfig_default
> > DEBUG 1: User Config File: config.rc
> > GSL_RNG_TYPE=mt19937
> > GSL_RNG_SEED=18446744072713528668
> > DEBUG 1: Forecast File:
> >
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_00000F0NL2019042612_0100_010000-000000wnd_ucmp
> > DEBUG 1: Observation File: raob_2019042612.nc
> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match
for
> > VarInfo "UGRD/P1000" in GRIB record 1 of GRIB file
> >
> "GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_00000F0NL2019042612_0100_010000-000000wnd_ucmp".
> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> > matching VarInfo "UGRD/P1000" in GRIB file
> >
> "GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_00000F0NL2019042612_0100_010000-000000wnd_ucmp".
> > DEBUG 2:
> > DEBUG 2:
> >
>
--------------------------------------------------------------------------------
> > DEBUG 2:
> > DEBUG 2: Reading data for UGRD/P1000.
> > DEBUG 3: Use the matching forecast and observation grids.
> > DEBUG 3: Grid Definition: Projection: Mercator Nx: 202 Ny: 202
> > Lat_LL_radians: 0.4770 Lon_LL_radians: 2.3860 Lat_UR_radians:
0.8214
> > Lon_UR_radians: 1.9489 Mx: 459.8120 My: 459.8003 Bx: 1097.1297 By:
> -228.1298
> > DEBUG 2: Processing masking regions.
> > DEBUG 3: Processing grid mask: FULL
> > DEBUG 2: For UGRD/P1000 found 1 forecast levels, 0 climatology
mean
> > levels, and 0 climatology standard deviation levels.
> > DEBUG 2:
> > DEBUG 2:
> >
>
--------------------------------------------------------------------------------
> > DEBUG 2:
> > DEBUG 2: Searching 1570630 observations from 82 messages.
> > DEBUG 2:
> > DEBUG 2:
> >
>
--------------------------------------------------------------------------------
> > DEBUG 2:
> > DEBUG 2: Processing UGRD/P1000 versus UGRD/P1000, for observation
type
> > ADPUPA, over region FULL, for interpolation method NEAREST(1),
using 0
> > pairs.
> > DEBUG 3: Number of matched pairs = 0
> > DEBUG 3: Observations processed = 1570630
> > DEBUG 3: Rejected: SID exclusion = 0
> > DEBUG 3: Rejected: obs type = 1570630
> > DEBUG 3: Rejected: valid time = 0
> > DEBUG 3: Rejected: bad obs value = 0
> > DEBUG 3: Rejected: off the grid = 0
> > DEBUG 3: Rejected: level mismatch = 0
> > DEBUG 3: Rejected: quality marker = 0
> > DEBUG 3: Rejected: message type = 0
> > DEBUG 3: Rejected: masking region = 0
> > DEBUG 3: Rejected: bad fcst value = 0
> > DEBUG 3: Rejected: duplicates = 0
> > DEBUG 2:
> > DEBUG 2:
> >
>
--------------------------------------------------------------------------------
> > DEBUG 2:
> > DEBUG 1: Output file: ./point_stat_000000L_20190426_120000V.stat
> > DEBUG 1: Output file:
./point_stat_000000L_20190426_120000V_cnt.txt
> >
> > I had an issue with this in the past using NAVGEM IEEE flatfile
format
> > climatologies that I converted to netcdf using ascii2nc as obs. I
fixed
> it
> > by using python embedding in the name part of the obs dictionary.
> Problem
> > is I cannot do this here because my obs are originally in hdf5
format
> and I
> > don’t have a way to use python embedding to provide MET with the
> necessary
> > metadata like read_NRL_binary.py does (I believe somebody from
your team
> > provided us with this python script to print out metadata based on
a
> > flatfile format that we use)
> >
> > Have you guys had any luck in using ascii2nc generated netcdf
files as
> > either forecast or obs in any of the MET stat utilities?
> >
> > Justin
> >
> > -----Original Message-----
> > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > Sent: Wednesday, May 22, 2019 11:27 AM
> > To: Tsu, Mr. Justin
> > Subject: Re: [rt.rap.ucar.edu #90256] Point Stat Help
> >
> > Justin,
> >
> > When you run Point-Stat, please increase the verbosity level to at
least
> 3
> > (-v 3). That will dump log messages that look something like
this:
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > *DEBUG 2: Processing TMP/P900-750 versus TMP/P900-750, for
observation
> type
> > ADPUPA, over region DTC165, for interpolation method NEAREST(1),
using
> 132
> > pairs.DEBUG 3: Number of matched pairs = 132DEBUG 3: Observations
> > processed = 89893DEBUG 3: Rejected: SID exclusion = 0DEBUG 3:
> Rejected:
> > obs type = 79788DEBUG 3: Rejected: valid time = 0DEBUG
3:
> > Rejected: bad obs value = 0DEBUG 3: Rejected: off the grid =
5DEBUG 3:
> > Rejected: topography = 0DEBUG 3: Rejected: level mismatch =
9219DEBUG
> > 3: Rejected: quality marker = 0DEBUG 3: Rejected: message type =
> 344DEBUG
> > 3: Rejected: masking region = 405DEBUG 3: Rejected: bad fcst value
=
> 0DEBUG
> > 3: Rejected: duplicates = 0*
> >
> > For each verification task, these list counts of reasons why
observations
> > were not used for that task. These counts should help you
determine why
> > you're getting 0 matched pairs. There are many possible reasons
why...
> > mismatches in message type, valid time, lat/lon location, vertical
> level...
> > and so on.
> >
> > Hopefully these counts will point you in the right direction.
> >
> > Thanks,
> > John
> >
> > On Tue, May 21, 2019 at 10:50 PM Tsu, Mr. Justin via RT <
> met_help at ucar.edu
> > >
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90256 >
> > >
> > > Hi John,
> > >
> > > So I took your advice and used wgrib to determine that my
variable is
> > > indeed named UGRD so I switched this out in my point_Stat config
file.
> > Now
> > > that config file looks like this:
> > >
> > > 52 fcst = {
> > > 53 field = [
> > > 54 {
> > > 55 name = "UGRD";
> > > 56 level = ["P1000"];
> > > 57 file_type = GRIB1;
> > > 58 cat_thresh = [ >80.0 ];
> > > 59 }
> > > 60 ];
> > > 61
> > > 62 }
> > > 63
> > > 64 obs = {
> > > 65 field = [
> > > 66 {
> > > 67 file_type = NETCDF_NCCF;
> > > 68 name = "UGRD";
> > > 69 level = ["P1000"];
> > > 70 }
> > > 71 ];
> > > 72 }
> > >
> > > This config file runs when I execute:
> > >
> > > point_stat
> > >
> >
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_02700F0NL2019042612_0100_010000-000000wnd_ucmp
> > > NAF_sfc_2019042612.nc config.rc -v 1
> > >
> > > However, the resulting contingency table file are empty (aside
from the
> > > header). I am guessing this has something to do with the way I
specify
> > my
> > > obs file. As I stated previously in my email, the obs file is
named
> > > raob_2019042612.nc and is a netcdf file created by using
ascii2nc on a
> > > list of point obs arranged in the format specified by the MET
tutorial.
> > I
> > > am not sure what the name value in the field dictionary refers
to
> because
> > > the netcdf header for raob_2019042612.nc looks like:
> > >
> > > 1 netcdf raob_2019042612 {
> > > 2 dimensions:
> > > 3 mxstr = 16 ;
> > > 4 mxstr2 = 40 ;
> > > 5 mxstr3 = 80 ;
> > > 6 nobs = 1570630 ;
> > > 7 nhdr = 82 ;
> > > 8 nhdr_typ = 1 ;
> > > 9 nhdr_sid = 82 ;
> > > 10 nhdr_vld = 1 ;
> > > 11 nobs_qty = 1 ;
> > > 12 obs_var_num = 6 ;
> > > 13 variables:
> > > 14 int hdr_typ(nhdr) ;
> > > 15 hdr_typ:long_name = "index of message type" ;
> > > 16 hdr_typ:_FillValue = -9999 ;
> > > 17 int hdr_sid(nhdr) ;
> > > 18 hdr_sid:long_name = "index of station
identification" ;
> > > 19 hdr_sid:_FillValue = -9999 ;
> > > 20 int hdr_vld(nhdr) ;
> > > 21 hdr_vld:long_name = "index of valid time" ;
> > > 22 hdr_vld:_FillValue = -9999 ;
> > > 23 float hdr_lat(nhdr) ;
> > > 24 hdr_lat:long_name = "latitude" ;
> > > 25 hdr_lat:_FillValue = -9999.f ;
> > > 26 hdr_lat:units = "degrees_north" ;
> > > 27 float hdr_lon(nhdr) ;
> > > 28 hdr_lon:long_name = "longitude" ;
> > > 29 hdr_lon:_FillValue = -9999.f ;
> > > 30 hdr_lon:units = "degrees_east" ;
> > > 31 float hdr_elv(nhdr) ;
> > > 32 hdr_elv:long_name = "elevation" ;
> > > 33 hdr_elv:_FillValue = -9999.f ;
> > > 34 hdr_elv:units = "meters above sea level (msl)" ;
> > > 35 int obs_qty(nobs) ;
> > > 36 obs_qty:long_name = "index of quality flag" ;
> > > 37 int obs_hid(nobs) ;
> > > 38 obs_hid:long_name = "index of matching header data"
;
> > > 39 obs_hid:_FillValue = -9999 ;
> > > 40 int obs_vid(nobs) ;
> > > 41 obs_vid:long_name = "index of variable names at
var_name" ;
> > > 42 obs_vid:_FillValue = -9999 ;
> > > 43 float obs_lvl(nobs) ;
> > > 44 obs_lvl:long_name = "pressure level (hPa) or
accumulation
> > > interval (sec)" ;
> > > 45 obs_lvl:_FillValue = -9999.f ;
> > > 46 float obs_hgt(nobs) ;
> > > 47 obs_hgt:long_name = "height in meters above sea
level or
> > > ground level (msl or agl)" ;
> > > 48 obs_hgt:_FillValue = -9999.f ;
> > > 49 float obs_val(nobs) ;
> > > 50 obs_val:long_name = "observation value" ;
> > > 51 obs_val:_FillValue = -9999.f ;
> > > 52 char hdr_typ_table(nhdr_typ, mxstr2) ;
> > > 53 hdr_typ_table:long_name = "message type" ;
> > > 54 char hdr_sid_table(nhdr_sid, mxstr2) ;
> > > 55 hdr_sid_table:long_name = "station identification" ;
> > > 56 char hdr_vld_table(nhdr_vld, mxstr) ;
> > > 57 hdr_vld_table:long_name = "valid time" ;
> > > 58 hdr_vld_table:units = "YYYYMMDD_HHMMSS UTC" ;
> > > 59 char obs_qty_table(nobs_qty, mxstr) ;
> > > 60 obs_qty_table:long_name = "quality flag" ;
> > > 61 char obs_var(obs_var_num, mxstr2) ;
> > > 62 obs_var:long_name = "variable names" ;
> > > 63
> > > 64 // global attributes:
> > > 65 :MET_Obs_version = "1.02" ;
> > > 66 :use_var_id = "true" ;
> > > 67 :FileOrigins = "File / generated 20190510_235941 UTC
on
> host
> > > maury2.local by the MET ascii2nc tool" ;
> > > 68 :MET_version = "V8.0" ;
> > > 69 :MET_tool = "ascii2nc" ;
> > >
> > > So there isn’t actualy a netcdf variable named UGRD; at this
point I'm
> > not
> > > sure what I should call it.
> > >
> > > Hope you can shed some light on how this works.
> > >
> > > Thanks,
> > > Justin
> > >
> > >
> > > -----Original Message-----
> > > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > > Sent: Monday, May 20, 2019 12:20 PM
> > > To: Tsu, Mr. Justin
> > > Subject: Re: [rt.rap.ucar.edu #90256] Point Stat Help
> > >
> > > Hi Justin,
> > >
> > > I see that you're having trouble reading GRIB data into the
Point-Stat
> > tool
> > > in MET. When getting going with a new gridded dataset in MET, I
> usually
> > > recommend starting with the plot_data_plane utility. For
example, try
> > > running this command:
> > >
> > > plot_data_plane
> > >
> > >
> >
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_0270F0NL2019042612_0100_010000-000000wnd_ucmp
> > > U_P1000.ps 'name="U"; level="P1000";'
> > >
> > > You should get a similar error since Point-Stat wasn't able to
read it.
> > > I'd recommend running this file through the "wgrib2" utility to
> inventory
> > > it and see what variable names/levels are present in the file.
For
> > > example, is it called "U" or "UGRD"?
> > >
> > > If you aren't able to get that working, please just post a
sample file
> to
> > > our anonymous ftp site. If you do, let me know, and I'll go
grab it
> and
> > > take a look:
> > >
> > >
> >
> https://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk
> > >
> > > Thanks,
> > > John Halley Gotway
> > >
> > >
> > >
> > > On Fri, May 17, 2019 at 3:27 PM Tsu, Mr. Justin via RT <
> > met_help at ucar.edu>
> > > wrote:
> > >
> > > >
> > > > Fri May 17 15:27:39 2019: Request 90256 was acted upon.
> > > > Transaction: Ticket created by justin.tsu at nrlmry.navy.mil
> > > > Queue: met_help
> > > > Subject: Point Stat Help
> > > > Owner: Nobody
> > > > Requestors: justin.tsu at nrlmry.navy.mil
> > > > Status: new
> > > > Ticket <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90256
> > >
> > > >
> > > >
> > > > Ive got my forecast field in the form of a grib file entitled
> > > >
> > > >
> > > >
> > > >
> > > >
> > >
> >
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_02700F0NL2019042612_
> > > > 0100_010000-000000wnd_ucmp
> > > >
> > > >
> > > >
> > > > This is the U Component of wind at 1000 mb. The following is
a
> > > descriptor
> > > > file to read this in GrADS:
> > > >
> > > >
> > > >
> > > > 1 dset
> > > >
> > > >
> > >
> >
> ^US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_03000F0NL2019042612_0100_0100
> > > > 00-000000wnd_ucmp
> > > >
> > > > 2 index
> > > >
> > > >
> > >
> >
> ^US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_03000F0NL2019042612_0100_0100
> > > > 00-000000wnd_ucmp.idx
> > > >
> > > > 3 undef 9.999E+20
> > > >
> > > > 4 title
> > > >
> > > >
> > >
> >
> US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_03000F0NL2019042612_0100_01000
> > > > 0-000000wnd_ucmp
> > > >
> > > > 5 * produced by grib2ctl v0.9.15
> > > >
> > > > 6 * command line options:
> > > >
> > > >
> > >
> >
> US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_03000F0NL2019042612_0100_01000
> > > > 0-000000wnd_ucmp
> > > >
> > > > 7 dtype grib 255
> > > >
> > > > 8 ydef 202 levels
> > > >
> > > > 9 27.3280 27.4386 27.5492 27.6596 27.7699
27.8801
> > 27.9902
> > > >
> > > > 10 28.1002 28.2101 28.3198 28.4295 28.5390 28.6484
> > 28.7577
> > > >
> > > > 11 28.8669 28.9759 29.0849 29.1937 29.3025 29.4111
> > 29.5196
> > > >
> > > > 12 29.6279 29.7362 29.8443 29.9524 30.0603 30.1681
> > 30.2757
> > > >
> > > > 13 30.3833 30.4907 30.5980 30.7052 30.8123 30.9193
> > 31.0261
> > > >
> > > > 14 31.1329 31.2395 31.3459 31.4523 31.5585 31.6647
> > 31.7707
> > > >
> > > > 15 31.8765 31.9823 32.0879 32.1934 32.2988 32.4041
> > 32.5093
> > > >
> > > > 16 32.6143 32.7192 32.8240 32.9286 33.0331 33.1375
> > 33.2418
> > > >
> > > > 17 33.3460 33.4500 33.5539 33.6577 33.7614 33.8649
> > 33.9683
> > > >
> > > > 18 34.0716 34.1747 34.2778 34.3807 34.4835 34.5861
> > 34.6886
> > > >
> > > > 19 34.7910 34.8933 34.9954 35.0975 35.1993 35.3011
> > 35.4027
> > > >
> > > > 20 35.5042 35.6056 35.7069 35.8080 35.9090 36.0099
> > 36.1106
> > > >
> > > > 21 36.2112 36.3117 36.4120 36.5122 36.6123 36.7123
> > 36.8121
> > > >
> > > > 22 36.9118 37.0114 37.1108 37.2101 37.3093 37.4083
> > 37.5073
> > > >
> > > > 23 37.6060 37.7047 37.8032 37.9016 37.9999 38.0980
> > 38.1960
> > > >
> > > > 24 38.2939 38.3916 38.4892 38.5867 38.6840 38.7812
> > 38.8783
> > > >
> > > > 25 38.9752 39.0720 39.1687 39.2653 39.3617 39.4579
> > 39.5541
> > > >
> > > > 26 39.6501 39.7460 39.8417 39.9373 40.0328 40.1281
> > 40.2234
> > > >
> > > > 27 40.3184 40.4134 40.5082 40.6029 40.6974 40.7918
> > 40.8861
> > > >
> > > > 28 40.9802 41.0742 41.1681 41.2618 41.3554 41.4489
> > 41.5422
> > > >
> > > > 29 41.6354 41.7285 41.8214 41.9142 42.0069 42.0994
> > 42.1918
> > > >
> > > > 30 42.2841 42.3762 42.4682 42.5600 42.6517 42.7433
> > 42.8348
> > > >
> > > > 31 42.9261 43.0173 43.1083 43.1992 43.2900 43.3806
> > 43.4711
> > > >
> > > > 32 43.5615 43.6517 43.7418 43.8318 43.9216 44.0113
> > 44.1008
> > > >
> > > > 33 44.1902 44.2795 44.3687 44.4577 44.5466 44.6353
> > 44.7239
> > > >
> > > > 34 44.8124 44.9007 44.9889 45.0770 45.1649 45.2527
> > 45.3403
> > > >
> > > > 35 45.4278 45.5152 45.6025 45.6896 45.7766 45.8634
> > 45.9501
> > > >
> > > > 36 46.0367 46.1231 46.2094 46.2956 46.3816 46.4675
> > 46.5533
> > > >
> > > > 37 46.6389 46.7244 46.8098 46.8950 46.9801 47.0650
> > > >
> > > > 38 xdef 202 linear -136.710000 0.124606965174129
> > > >
> > > > 39 tdef 1 linear 12Z26apr2019 1mo
> > > >
> > > > 40 zdef 1 linear 1 1
> > > >
> > > > 41 vars 1
> > > >
> > > > 42 UGRD1000mb 0 33,100,1000 ** 1000 mb u wind [m/s]
> > > >
> > > > 43 ENDVARS
> > > >
> > > >
> > > >
> > > > My ob file is in a netcdf file created using ascii2nc. I
named wind
> > > speed
> > > > obs "ws" in the ascii file that I ran the ascii2nc tool on ( I
am
> aware
> > > > that
> > > > I am trying to compare Wind speed and u component of wind, but
I just
> > > want
> > > > to see if I can get point_stat working).
> > > >
> > > >
> > > >
> > > > How should I arrange my fcst and obs arrays? Here is what I
have so
> > > far:
> > > >
> > > >
> > > >
> > > > 52 fcst = {
> > > >
> > > > 53 field = [
> > > >
> > > > 54 {
> > > >
> > > > 55 name = "U";
> > > >
> > > > 56 level = "P1000";
> > > >
> > > > 57 file_type = GRIB2;
> > > >
> > > > 58 cat_thresh = [ >80.0 ];
> > > >
> > > > 59 }
> > > >
> > > > 60 ];
> > > >
> > > > 61
> > > >
> > > > 62 }
> > > >
> > > > 63
> > > >
> > > > 64 obs = {
> > > >
> > > > 65 field = [
> > > >
> > > > 66 {
> > > >
> > > > 67 file_type = NETCDF_NCCF;
> > > >
> > > > 68 name = "UGRD";
> > > >
> > > > 69 level = ["P1000"];
> > > >
> > > > 70 }
> > > >
> > > > 71 ];
> > > >
> > > > 72 }
> > > >
> > > >
> > > >
> > > > Running the command as follows gives me this error:
> > > >
> > > >
> > > >
> > > > [tsu at maury2 00_POINT_STAT_WORK]$ point_stat
> > > >
> > > >
> > > >
> > >
> >
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_02700F0NL2019042612_
> > > > 0100_010000-000000wnd_ucmp
> > > >
> > > > NAF_sfc_2019042612.nc config.rc -v 1
> > > >
> > > > DEBUG 1: Default Config File:
> > > >
> > > >
> > >
> >
> /software/depot/met/met-
8.0_bugfix/share/met/config/PointStatConfig_default
> > > >
> > > > DEBUG 1: User Config File: config.rc
> > > >
> > > > GSL_RNG_TYPE=mt19937
> > > >
> > > > GSL_RNG_SEED=1726142812
> > > >
> > > > DEBUG 1: Forecast File:
> > > >
> > > >
> > > >
> > >
> >
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_02700F0NL2019042612_
> > > > 0100_010000-000000wnd_ucmp
> > > >
> > > > DEBUG 1: Observation File: NAF_sfc_2019042612.nc
> > > >
> > > > WARNING:
> > > >
> > > > WARNING: process_fcst_climo_files() -> no fields matching
U/P1000
> found
> > > in
> > > >
> > > > file:
> > > >
> > > >
> > > >
> > >
> >
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_02700F0NL2019042612_
> > > > 0100_010000-000000wnd_ucmp
> > > >
> > > > WARNING:
> > > >
> > > > ERROR :
> > > >
> > > > ERROR : process_fcst_climo_files() -> no requested forecast
data
> > found!
> > > >
> > > > Exiting...
> > > >
> > > > ERROR :
> > > >
> > > >
> > > >
> > > > Justin
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > Justin Tsu
> > > >
> > > > Marine Meteorology Division
> > > >
> > > > Data Assimilation/Mesoscale Modeling
> > > >
> > > > Building 704 Room 212
> > > >
> > > > Naval Research Laboratory, Code 7531
> > > >
> > > > 7 Grace Hopper Avenue
> > > >
> > > > Monterey, CA 93943-5502
> > > >
> > > >
> > > >
> > > > Ph. (831) 656-4111
> > > >
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> > >
> > >
> >
> >
> >
>
>
>
------------------------------------------------
Subject: Point Stat Help
From: John Halley Gotway
Time: Wed May 22 16:48:29 2019
Justin,
The rejection counts are listed in the order that the logic is
applied.
Looks like the final 2,431 of them were rejected based on message
type:
DEBUG 3: Rejected: message type = 2431
So what message type is in your obs file?
*ncdump -v hdr_typ_table out/pb2nc/sample_pb.nc <http://sample_pb.nc>*
*... hdr_typ_table = "ADPUPA", "AIRCFT", "PROFLR", "VADWND",
"ADPSFC", "SFCSHP", "RASSDA" ;}*
The Point-Stat config file allows you to configure which message types
you'd like to use. Just set it to the one(s) you actually have in
your
data:
*message_type = [ "ADPUPA" ];*
Point-Stat is only set up to read a single input forecast file at each
time. But if you're using GRIB1 or GRIB2 input files, they can
literally
be catt'ed together:
*cat file1.grb2 file2.grb2 > combined.grb2*
So that's the quick and dirty answer.
Thanks,
John
On Wed, May 22, 2019 at 4:39 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90256 >
>
> John,
>
> Thank you for the clear explanation. What do I do if my u component
and v
> component winds are in separate GRIB files?
>
> I am still getting 0 pairs of matched obs even when I set my
pressure
> range to look for obs between P700 and P1025.
>
> [tsu at maury2 00_POINT_STAT_WORK]$ point_stat
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_00000F0NL2019042612_0100_010000-000000wnd_ucmp
> raob_2019042612.nc config.rc -v 3
> DEBUG 1: Default Config File:
> /software/depot/met/met-
8.0_bugfix/share/met/config/PointStatConfig_default
> DEBUG 1: User Config File: config.rc
> GSL_RNG_TYPE=mt19937
> GSL_RNG_SEED=18446744071575299420
> DEBUG 1: Forecast File:
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_00000F0NL2019042612_0100_010000-000000wnd_ucmp
> DEBUG 1: Observation File: raob_2019042612.nc
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match
for
> VarInfo "UGRD/P1000" in GRIB record 1 of GRIB file
> "GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_00000F0NL2019042612_0100_010000-000000wnd_ucmp".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "UGRD/P1000" in GRIB file
> "GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_00000F0NL2019042612_0100_010000-000000wnd_ucmp".
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Reading data for UGRD/P1000.
> DEBUG 3: Use the matching forecast and observation grids.
> DEBUG 3: Grid Definition: Projection: Mercator Nx: 202 Ny: 202
> Lat_LL_radians: 0.4770 Lon_LL_radians: 2.3860 Lat_UR_radians: 0.8214
> Lon_UR_radians: 1.9489 Mx: 459.8120 My: 459.8003 Bx: 1097.1297 By:
-228.1298
> DEBUG 2: Processing masking regions.
> DEBUG 3: Processing grid mask: FULL
> DEBUG 2: For UGRD/P1000 found 1 forecast levels, 0 climatology mean
> levels, and 0 climatology standard deviation levels.
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Searching 1570630 observations from 82 messages.
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Processing UGRD/P1000 versus ws/P1025-700, for observation
type
> ADPUPA, over region FULL, for interpolation method NEAREST(1), using
0
> pairs.
> DEBUG 3: Number of matched pairs = 0
> DEBUG 3: Observations processed = 1570630
> DEBUG 3: Rejected: SID exclusion = 0
> DEBUG 3: Rejected: obs type = 1310954
> DEBUG 3: Rejected: valid time = 0
> DEBUG 3: Rejected: bad obs value = 0
> DEBUG 3: Rejected: off the grid = 225197
> DEBUG 3: Rejected: level mismatch = 32048
> DEBUG 3: Rejected: quality marker = 0
> DEBUG 3: Rejected: message type = 2431
> DEBUG 3: Rejected: masking region = 0
> DEBUG 3: Rejected: bad fcst value = 0
> DEBUG 3: Rejected: duplicates = 0
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 1: Output file: ./point_stat_000000L_20190426_120000V.stat
> DEBUG 1: Output file: ./point_stat_000000L_20190426_120000V_cnt.txt
>
>
> Justin
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Wednesday, May 22, 2019 2:51 PM
> To: Tsu, Mr. Justin
> Subject: Re: [rt.rap.ucar.edu #90256] Point Stat Help
>
> Justin,
>
> One nice feature in Point-Stat is that when you request "WIND" (i.e.
the
> GRIB abbreviation for wind speed), it will find the corresponding
"UGRD"
> and "VGRD" records and derive wind speed for you on the fly.
>
> So please try:
> *fcst = {*
> * field = [ { name = "WIND"; level = "P1000"; } ];*
> *}*
> *obs = {*
> * field = [ { name ="ws"; level = "P975-1025"; } ];*
> *}*
>
> This tells Point-Stat to:
>
> (1) Since WIND/P1000 doesn't exist in the input file, it will read
> UGRD/P1000 and VGRD/P1000 and derive wind speed.
> (2) Look through all of the point observations and consider the ones
named
> "ws". Any of them with a pressure level between 975 and 1025 will
be
> verified against the forecast wind speed value at P1000.
>
>
> If you want to do vertical interpolation, you'll need to configure
it a bit
> differently. Let's say you have model output every 100 mb. You
could
> configure it like this:
>
> *fcst = {*
> * field = [ { name = "WIND"; level = "P900-1000"; } ];*
> *}*
> *obs = {*
> * field = [ { name ="ws"; level = "P900-1000"; } ];*
> *}*
>
> In this case Point-Stat will compute wind speed for both the 900 and
1000
> mb levels. It'll only use obs falling between 900mb and 1000mb.
For
> example, for an obs at 925mb, it'll first interpolate horizontally
to the
> lat/lon observation location at the forecast levels above and below.
Then
> it'll use those 2 values to interpolate to the obs pressure level,
linear
> in the log of pressure.
>
> So the functionality is really controlled by how the fcst/obs levels
are
> defined in the config file.
>
> Thanks,
> John
>
> On Wed, May 22, 2019 at 3:35 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90256 >
> >
> > Awesome!
> >
> > Heres what I get from dumping obv_var
> >
> > data:
> >
> > obs_var =
> > "ws",
> > "wdir",
> > "t",
> > "dptd",
> > "pres",
> > "ght" ;
> > }
> >
> > I'm assuming I'll need ws here. If I specify my obs to be at
P1000, will
> > point_stat use the interp that I specified to find the obs nearest
to
> that
> > height? Also is there a way to verify this components of wind
with wind
> > speed since for this example I am only using UGRD (u component)?
I'm
> > guessing I will have to specify a second field or something.
> >
> > Justin
> >
> > -----Original Message-----
> > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > Sent: Wednesday, May 22, 2019 2:17 PM
> > To: Tsu, Mr. Justin
> > Subject: Re: [rt.rap.ucar.edu #90256] Point Stat Help
> >
> > Justin,
> >
> > OK, this is good info. As you point out, the problem is a
mismatch in
> the
> > requested observation variable name and the actual observation
variable
> > name:
> > DEBUG 3: Rejected: obs type = 1570630
> >
> > This should be very easy to fix. We just need to figure out how
your
> > observations are named in the file named "raob_2019042612.nc".
Just
> take
> > a
> > look at the contents of the "obs_var" variable in that file. For
> example,
> > here's some output from pb2nc:
> >
> > ncdump -v obs_var out/pb2nc/sample_pb.nc
> > ...
> > obs_var =
> > "SPFH",
> > "TMP",
> > "HGT",
> > "UGRD",
> > "VGRD",
> > "DPT",
> > "WIND",
> > "RH",
> > "MIXR" ;
> > }
> >
> > In this case, the U-component is wind really is called "UGRD".
But
> > obviously in your data it is not. Let say, for example, that it's
called
> > "UWIND". You just need to edit your Point-Stat config file to
tell it
> to
> > use those variables:
> >
> >
> >
> >
> >
> >
> >
> >
> > *obs = { field = [ { name = "UWIND";
> > level = ["P1000"]; } ];}*
> >
> > On Wed, May 22, 2019 at 1:31 PM Tsu, Mr. Justin via RT <
> met_help at ucar.edu>
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90256 >
> > >
> > > Hey John,
> > >
> > > Just ran with verbosity set to 3 and discovered that no pairs
are being
> > > used:
> > >
> > > [tsu at maury2 00_POINT_STAT_WORK]$ point_stat
> > >
> >
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_00000F0NL2019042612_0100_010000-000000wnd_ucmp
> > > raob_2019042612.nc config.rc -v 3
> > > DEBUG 1: Default Config File:
> > >
> >
> /software/depot/met/met-
8.0_bugfix/share/met/config/PointStatConfig_default
> > > DEBUG 1: User Config File: config.rc
> > > GSL_RNG_TYPE=mt19937
> > > GSL_RNG_SEED=18446744072713528668
> > > DEBUG 1: Forecast File:
> > >
> >
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_00000F0NL2019042612_0100_010000-000000wnd_ucmp
> > > DEBUG 1: Observation File: raob_2019042612.nc
> > > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact
match for
> > > VarInfo "UGRD/P1000" in GRIB record 1 of GRIB file
> > >
> >
> "GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_00000F0NL2019042612_0100_010000-000000wnd_ucmp".
> > > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> > > matching VarInfo "UGRD/P1000" in GRIB file
> > >
> >
> "GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_00000F0NL2019042612_0100_010000-000000wnd_ucmp".
> > > DEBUG 2:
> > > DEBUG 2:
> > >
> >
>
--------------------------------------------------------------------------------
> > > DEBUG 2:
> > > DEBUG 2: Reading data for UGRD/P1000.
> > > DEBUG 3: Use the matching forecast and observation grids.
> > > DEBUG 3: Grid Definition: Projection: Mercator Nx: 202 Ny: 202
> > > Lat_LL_radians: 0.4770 Lon_LL_radians: 2.3860 Lat_UR_radians:
0.8214
> > > Lon_UR_radians: 1.9489 Mx: 459.8120 My: 459.8003 Bx: 1097.1297
By:
> > -228.1298
> > > DEBUG 2: Processing masking regions.
> > > DEBUG 3: Processing grid mask: FULL
> > > DEBUG 2: For UGRD/P1000 found 1 forecast levels, 0 climatology
mean
> > > levels, and 0 climatology standard deviation levels.
> > > DEBUG 2:
> > > DEBUG 2:
> > >
> >
>
--------------------------------------------------------------------------------
> > > DEBUG 2:
> > > DEBUG 2: Searching 1570630 observations from 82 messages.
> > > DEBUG 2:
> > > DEBUG 2:
> > >
> >
>
--------------------------------------------------------------------------------
> > > DEBUG 2:
> > > DEBUG 2: Processing UGRD/P1000 versus UGRD/P1000, for
observation type
> > > ADPUPA, over region FULL, for interpolation method NEAREST(1),
using 0
> > > pairs.
> > > DEBUG 3: Number of matched pairs = 0
> > > DEBUG 3: Observations processed = 1570630
> > > DEBUG 3: Rejected: SID exclusion = 0
> > > DEBUG 3: Rejected: obs type = 1570630
> > > DEBUG 3: Rejected: valid time = 0
> > > DEBUG 3: Rejected: bad obs value = 0
> > > DEBUG 3: Rejected: off the grid = 0
> > > DEBUG 3: Rejected: level mismatch = 0
> > > DEBUG 3: Rejected: quality marker = 0
> > > DEBUG 3: Rejected: message type = 0
> > > DEBUG 3: Rejected: masking region = 0
> > > DEBUG 3: Rejected: bad fcst value = 0
> > > DEBUG 3: Rejected: duplicates = 0
> > > DEBUG 2:
> > > DEBUG 2:
> > >
> >
>
--------------------------------------------------------------------------------
> > > DEBUG 2:
> > > DEBUG 1: Output file: ./point_stat_000000L_20190426_120000V.stat
> > > DEBUG 1: Output file:
./point_stat_000000L_20190426_120000V_cnt.txt
> > >
> > > I had an issue with this in the past using NAVGEM IEEE flatfile
format
> > > climatologies that I converted to netcdf using ascii2nc as obs.
I
> fixed
> > it
> > > by using python embedding in the name part of the obs
dictionary.
> > Problem
> > > is I cannot do this here because my obs are originally in hdf5
format
> > and I
> > > don’t have a way to use python embedding to provide MET with the
> > necessary
> > > metadata like read_NRL_binary.py does (I believe somebody from
your
> team
> > > provided us with this python script to print out metadata based
on a
> > > flatfile format that we use)
> > >
> > > Have you guys had any luck in using ascii2nc generated netcdf
files as
> > > either forecast or obs in any of the MET stat utilities?
> > >
> > > Justin
> > >
> > > -----Original Message-----
> > > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > > Sent: Wednesday, May 22, 2019 11:27 AM
> > > To: Tsu, Mr. Justin
> > > Subject: Re: [rt.rap.ucar.edu #90256] Point Stat Help
> > >
> > > Justin,
> > >
> > > When you run Point-Stat, please increase the verbosity level to
at
> least
> > 3
> > > (-v 3). That will dump log messages that look something like
this:
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > *DEBUG 2: Processing TMP/P900-750 versus TMP/P900-750, for
observation
> > type
> > > ADPUPA, over region DTC165, for interpolation method NEAREST(1),
using
> > 132
> > > pairs.DEBUG 3: Number of matched pairs = 132DEBUG 3:
Observations
> > > processed = 89893DEBUG 3: Rejected: SID exclusion = 0DEBUG 3:
> > Rejected:
> > > obs type = 79788DEBUG 3: Rejected: valid time = 0DEBUG
3:
> > > Rejected: bad obs value = 0DEBUG 3: Rejected: off the grid =
5DEBUG
> 3:
> > > Rejected: topography = 0DEBUG 3: Rejected: level mismatch =
> 9219DEBUG
> > > 3: Rejected: quality marker = 0DEBUG 3: Rejected: message type
=
> > 344DEBUG
> > > 3: Rejected: masking region = 405DEBUG 3: Rejected: bad fcst
value =
> > 0DEBUG
> > > 3: Rejected: duplicates = 0*
> > >
> > > For each verification task, these list counts of reasons why
> observations
> > > were not used for that task. These counts should help you
determine
> why
> > > you're getting 0 matched pairs. There are many possible reasons
why...
> > > mismatches in message type, valid time, lat/lon location,
vertical
> > level...
> > > and so on.
> > >
> > > Hopefully these counts will point you in the right direction.
> > >
> > > Thanks,
> > > John
> > >
> > > On Tue, May 21, 2019 at 10:50 PM Tsu, Mr. Justin via RT <
> > met_help at ucar.edu
> > > >
> > > wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90256
>
> > > >
> > > > Hi John,
> > > >
> > > > So I took your advice and used wgrib to determine that my
variable is
> > > > indeed named UGRD so I switched this out in my point_Stat
config
> file.
> > > Now
> > > > that config file looks like this:
> > > >
> > > > 52 fcst = {
> > > > 53 field = [
> > > > 54 {
> > > > 55 name = "UGRD";
> > > > 56 level = ["P1000"];
> > > > 57 file_type = GRIB1;
> > > > 58 cat_thresh = [ >80.0 ];
> > > > 59 }
> > > > 60 ];
> > > > 61
> > > > 62 }
> > > > 63
> > > > 64 obs = {
> > > > 65 field = [
> > > > 66 {
> > > > 67 file_type = NETCDF_NCCF;
> > > > 68 name = "UGRD";
> > > > 69 level = ["P1000"];
> > > > 70 }
> > > > 71 ];
> > > > 72 }
> > > >
> > > > This config file runs when I execute:
> > > >
> > > > point_stat
> > > >
> > >
> >
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_02700F0NL2019042612_0100_010000-000000wnd_ucmp
> > > > NAF_sfc_2019042612.nc config.rc -v 1
> > > >
> > > > However, the resulting contingency table file are empty (aside
from
> the
> > > > header). I am guessing this has something to do with the way
I
> specify
> > > my
> > > > obs file. As I stated previously in my email, the obs file is
named
> > > > raob_2019042612.nc and is a netcdf file created by using
ascii2nc
> on a
> > > > list of point obs arranged in the format specified by the MET
> tutorial.
> > > I
> > > > am not sure what the name value in the field dictionary refers
to
> > because
> > > > the netcdf header for raob_2019042612.nc looks like:
> > > >
> > > > 1 netcdf raob_2019042612 {
> > > > 2 dimensions:
> > > > 3 mxstr = 16 ;
> > > > 4 mxstr2 = 40 ;
> > > > 5 mxstr3 = 80 ;
> > > > 6 nobs = 1570630 ;
> > > > 7 nhdr = 82 ;
> > > > 8 nhdr_typ = 1 ;
> > > > 9 nhdr_sid = 82 ;
> > > > 10 nhdr_vld = 1 ;
> > > > 11 nobs_qty = 1 ;
> > > > 12 obs_var_num = 6 ;
> > > > 13 variables:
> > > > 14 int hdr_typ(nhdr) ;
> > > > 15 hdr_typ:long_name = "index of message type" ;
> > > > 16 hdr_typ:_FillValue = -9999 ;
> > > > 17 int hdr_sid(nhdr) ;
> > > > 18 hdr_sid:long_name = "index of station
identification" ;
> > > > 19 hdr_sid:_FillValue = -9999 ;
> > > > 20 int hdr_vld(nhdr) ;
> > > > 21 hdr_vld:long_name = "index of valid time" ;
> > > > 22 hdr_vld:_FillValue = -9999 ;
> > > > 23 float hdr_lat(nhdr) ;
> > > > 24 hdr_lat:long_name = "latitude" ;
> > > > 25 hdr_lat:_FillValue = -9999.f ;
> > > > 26 hdr_lat:units = "degrees_north" ;
> > > > 27 float hdr_lon(nhdr) ;
> > > > 28 hdr_lon:long_name = "longitude" ;
> > > > 29 hdr_lon:_FillValue = -9999.f ;
> > > > 30 hdr_lon:units = "degrees_east" ;
> > > > 31 float hdr_elv(nhdr) ;
> > > > 32 hdr_elv:long_name = "elevation" ;
> > > > 33 hdr_elv:_FillValue = -9999.f ;
> > > > 34 hdr_elv:units = "meters above sea level (msl)" ;
> > > > 35 int obs_qty(nobs) ;
> > > > 36 obs_qty:long_name = "index of quality flag" ;
> > > > 37 int obs_hid(nobs) ;
> > > > 38 obs_hid:long_name = "index of matching header
data" ;
> > > > 39 obs_hid:_FillValue = -9999 ;
> > > > 40 int obs_vid(nobs) ;
> > > > 41 obs_vid:long_name = "index of variable names at
> var_name" ;
> > > > 42 obs_vid:_FillValue = -9999 ;
> > > > 43 float obs_lvl(nobs) ;
> > > > 44 obs_lvl:long_name = "pressure level (hPa) or
accumulation
> > > > interval (sec)" ;
> > > > 45 obs_lvl:_FillValue = -9999.f ;
> > > > 46 float obs_hgt(nobs) ;
> > > > 47 obs_hgt:long_name = "height in meters above sea
level or
> > > > ground level (msl or agl)" ;
> > > > 48 obs_hgt:_FillValue = -9999.f ;
> > > > 49 float obs_val(nobs) ;
> > > > 50 obs_val:long_name = "observation value" ;
> > > > 51 obs_val:_FillValue = -9999.f ;
> > > > 52 char hdr_typ_table(nhdr_typ, mxstr2) ;
> > > > 53 hdr_typ_table:long_name = "message type" ;
> > > > 54 char hdr_sid_table(nhdr_sid, mxstr2) ;
> > > > 55 hdr_sid_table:long_name = "station identification"
;
> > > > 56 char hdr_vld_table(nhdr_vld, mxstr) ;
> > > > 57 hdr_vld_table:long_name = "valid time" ;
> > > > 58 hdr_vld_table:units = "YYYYMMDD_HHMMSS UTC" ;
> > > > 59 char obs_qty_table(nobs_qty, mxstr) ;
> > > > 60 obs_qty_table:long_name = "quality flag" ;
> > > > 61 char obs_var(obs_var_num, mxstr2) ;
> > > > 62 obs_var:long_name = "variable names" ;
> > > > 63
> > > > 64 // global attributes:
> > > > 65 :MET_Obs_version = "1.02" ;
> > > > 66 :use_var_id = "true" ;
> > > > 67 :FileOrigins = "File / generated 20190510_235941
UTC on
> > host
> > > > maury2.local by the MET ascii2nc tool" ;
> > > > 68 :MET_version = "V8.0" ;
> > > > 69 :MET_tool = "ascii2nc" ;
> > > >
> > > > So there isn’t actualy a netcdf variable named UGRD; at this
point
> I'm
> > > not
> > > > sure what I should call it.
> > > >
> > > > Hope you can shed some light on how this works.
> > > >
> > > > Thanks,
> > > > Justin
> > > >
> > > >
> > > > -----Original Message-----
> > > > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > > > Sent: Monday, May 20, 2019 12:20 PM
> > > > To: Tsu, Mr. Justin
> > > > Subject: Re: [rt.rap.ucar.edu #90256] Point Stat Help
> > > >
> > > > Hi Justin,
> > > >
> > > > I see that you're having trouble reading GRIB data into the
> Point-Stat
> > > tool
> > > > in MET. When getting going with a new gridded dataset in MET,
I
> > usually
> > > > recommend starting with the plot_data_plane utility. For
example,
> try
> > > > running this command:
> > > >
> > > > plot_data_plane
> > > >
> > > >
> > >
> >
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_0270F0NL2019042612_0100_010000-000000wnd_ucmp
> > > > U_P1000.ps 'name="U"; level="P1000";'
> > > >
> > > > You should get a similar error since Point-Stat wasn't able to
read
> it.
> > > > I'd recommend running this file through the "wgrib2" utility
to
> > inventory
> > > > it and see what variable names/levels are present in the file.
For
> > > > example, is it called "U" or "UGRD"?
> > > >
> > > > If you aren't able to get that working, please just post a
sample
> file
> > to
> > > > our anonymous ftp site. If you do, let me know, and I'll go
grab it
> > and
> > > > take a look:
> > > >
> > > >
> > >
> >
> https://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk
> > > >
> > > > Thanks,
> > > > John Halley Gotway
> > > >
> > > >
> > > >
> > > > On Fri, May 17, 2019 at 3:27 PM Tsu, Mr. Justin via RT <
> > > met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > >
> > > > > Fri May 17 15:27:39 2019: Request 90256 was acted upon.
> > > > > Transaction: Ticket created by justin.tsu at nrlmry.navy.mil
> > > > > Queue: met_help
> > > > > Subject: Point Stat Help
> > > > > Owner: Nobody
> > > > > Requestors: justin.tsu at nrlmry.navy.mil
> > > > > Status: new
> > > > > Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90256
> > > >
> > > > >
> > > > >
> > > > > Ive got my forecast field in the form of a grib file
entitled
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_02700F0NL2019042612_
> > > > > 0100_010000-000000wnd_ucmp
> > > > >
> > > > >
> > > > >
> > > > > This is the U Component of wind at 1000 mb. The following
is a
> > > > descriptor
> > > > > file to read this in GrADS:
> > > > >
> > > > >
> > > > >
> > > > > 1 dset
> > > > >
> > > > >
> > > >
> > >
> >
> ^US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_03000F0NL2019042612_0100_0100
> > > > > 00-000000wnd_ucmp
> > > > >
> > > > > 2 index
> > > > >
> > > > >
> > > >
> > >
> >
> ^US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_03000F0NL2019042612_0100_0100
> > > > > 00-000000wnd_ucmp.idx
> > > > >
> > > > > 3 undef 9.999E+20
> > > > >
> > > > > 4 title
> > > > >
> > > > >
> > > >
> > >
> >
> US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_03000F0NL2019042612_0100_01000
> > > > > 0-000000wnd_ucmp
> > > > >
> > > > > 5 * produced by grib2ctl v0.9.15
> > > > >
> > > > > 6 * command line options:
> > > > >
> > > > >
> > > >
> > >
> >
> US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_03000F0NL2019042612_0100_01000
> > > > > 0-000000wnd_ucmp
> > > > >
> > > > > 7 dtype grib 255
> > > > >
> > > > > 8 ydef 202 levels
> > > > >
> > > > > 9 27.3280 27.4386 27.5492 27.6596 27.7699
27.8801
> > > 27.9902
> > > > >
> > > > > 10 28.1002 28.2101 28.3198 28.4295 28.5390
28.6484
> > > 28.7577
> > > > >
> > > > > 11 28.8669 28.9759 29.0849 29.1937 29.3025
29.4111
> > > 29.5196
> > > > >
> > > > > 12 29.6279 29.7362 29.8443 29.9524 30.0603
30.1681
> > > 30.2757
> > > > >
> > > > > 13 30.3833 30.4907 30.5980 30.7052 30.8123
30.9193
> > > 31.0261
> > > > >
> > > > > 14 31.1329 31.2395 31.3459 31.4523 31.5585
31.6647
> > > 31.7707
> > > > >
> > > > > 15 31.8765 31.9823 32.0879 32.1934 32.2988
32.4041
> > > 32.5093
> > > > >
> > > > > 16 32.6143 32.7192 32.8240 32.9286 33.0331
33.1375
> > > 33.2418
> > > > >
> > > > > 17 33.3460 33.4500 33.5539 33.6577 33.7614
33.8649
> > > 33.9683
> > > > >
> > > > > 18 34.0716 34.1747 34.2778 34.3807 34.4835
34.5861
> > > 34.6886
> > > > >
> > > > > 19 34.7910 34.8933 34.9954 35.0975 35.1993
35.3011
> > > 35.4027
> > > > >
> > > > > 20 35.5042 35.6056 35.7069 35.8080 35.9090
36.0099
> > > 36.1106
> > > > >
> > > > > 21 36.2112 36.3117 36.4120 36.5122 36.6123
36.7123
> > > 36.8121
> > > > >
> > > > > 22 36.9118 37.0114 37.1108 37.2101 37.3093
37.4083
> > > 37.5073
> > > > >
> > > > > 23 37.6060 37.7047 37.8032 37.9016 37.9999
38.0980
> > > 38.1960
> > > > >
> > > > > 24 38.2939 38.3916 38.4892 38.5867 38.6840
38.7812
> > > 38.8783
> > > > >
> > > > > 25 38.9752 39.0720 39.1687 39.2653 39.3617
39.4579
> > > 39.5541
> > > > >
> > > > > 26 39.6501 39.7460 39.8417 39.9373 40.0328
40.1281
> > > 40.2234
> > > > >
> > > > > 27 40.3184 40.4134 40.5082 40.6029 40.6974
40.7918
> > > 40.8861
> > > > >
> > > > > 28 40.9802 41.0742 41.1681 41.2618 41.3554
41.4489
> > > 41.5422
> > > > >
> > > > > 29 41.6354 41.7285 41.8214 41.9142 42.0069
42.0994
> > > 42.1918
> > > > >
> > > > > 30 42.2841 42.3762 42.4682 42.5600 42.6517
42.7433
> > > 42.8348
> > > > >
> > > > > 31 42.9261 43.0173 43.1083 43.1992 43.2900
43.3806
> > > 43.4711
> > > > >
> > > > > 32 43.5615 43.6517 43.7418 43.8318 43.9216
44.0113
> > > 44.1008
> > > > >
> > > > > 33 44.1902 44.2795 44.3687 44.4577 44.5466
44.6353
> > > 44.7239
> > > > >
> > > > > 34 44.8124 44.9007 44.9889 45.0770 45.1649
45.2527
> > > 45.3403
> > > > >
> > > > > 35 45.4278 45.5152 45.6025 45.6896 45.7766
45.8634
> > > 45.9501
> > > > >
> > > > > 36 46.0367 46.1231 46.2094 46.2956 46.3816
46.4675
> > > 46.5533
> > > > >
> > > > > 37 46.6389 46.7244 46.8098 46.8950 46.9801
47.0650
> > > > >
> > > > > 38 xdef 202 linear -136.710000 0.124606965174129
> > > > >
> > > > > 39 tdef 1 linear 12Z26apr2019 1mo
> > > > >
> > > > > 40 zdef 1 linear 1 1
> > > > >
> > > > > 41 vars 1
> > > > >
> > > > > 42 UGRD1000mb 0 33,100,1000 ** 1000 mb u wind [m/s]
> > > > >
> > > > > 43 ENDVARS
> > > > >
> > > > >
> > > > >
> > > > > My ob file is in a netcdf file created using ascii2nc. I
named
> wind
> > > > speed
> > > > > obs "ws" in the ascii file that I ran the ascii2nc tool on (
I am
> > aware
> > > > > that
> > > > > I am trying to compare Wind speed and u component of wind,
but I
> just
> > > > want
> > > > > to see if I can get point_stat working).
> > > > >
> > > > >
> > > > >
> > > > > How should I arrange my fcst and obs arrays? Here is what
I have
> so
> > > > far:
> > > > >
> > > > >
> > > > >
> > > > > 52 fcst = {
> > > > >
> > > > > 53 field = [
> > > > >
> > > > > 54 {
> > > > >
> > > > > 55 name = "U";
> > > > >
> > > > > 56 level = "P1000";
> > > > >
> > > > > 57 file_type = GRIB2;
> > > > >
> > > > > 58 cat_thresh = [ >80.0 ];
> > > > >
> > > > > 59 }
> > > > >
> > > > > 60 ];
> > > > >
> > > > > 61
> > > > >
> > > > > 62 }
> > > > >
> > > > > 63
> > > > >
> > > > > 64 obs = {
> > > > >
> > > > > 65 field = [
> > > > >
> > > > > 66 {
> > > > >
> > > > > 67 file_type = NETCDF_NCCF;
> > > > >
> > > > > 68 name = "UGRD";
> > > > >
> > > > > 69 level = ["P1000"];
> > > > >
> > > > > 70 }
> > > > >
> > > > > 71 ];
> > > > >
> > > > > 72 }
> > > > >
> > > > >
> > > > >
> > > > > Running the command as follows gives me this error:
> > > > >
> > > > >
> > > > >
> > > > > [tsu at maury2 00_POINT_STAT_WORK]$ point_stat
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_02700F0NL2019042612_
> > > > > 0100_010000-000000wnd_ucmp
> > > > >
> > > > > NAF_sfc_2019042612.nc config.rc -v 1
> > > > >
> > > > > DEBUG 1: Default Config File:
> > > > >
> > > > >
> > > >
> > >
> >
> /software/depot/met/met-
8.0_bugfix/share/met/config/PointStatConfig_default
> > > > >
> > > > > DEBUG 1: User Config File: config.rc
> > > > >
> > > > > GSL_RNG_TYPE=mt19937
> > > > >
> > > > > GSL_RNG_SEED=1726142812
> > > > >
> > > > > DEBUG 1: Forecast File:
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_02700F0NL2019042612_
> > > > > 0100_010000-000000wnd_ucmp
> > > > >
> > > > > DEBUG 1: Observation File: NAF_sfc_2019042612.nc
> > > > >
> > > > > WARNING:
> > > > >
> > > > > WARNING: process_fcst_climo_files() -> no fields matching
U/P1000
> > found
> > > > in
> > > > >
> > > > > file:
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_02700F0NL2019042612_
> > > > > 0100_010000-000000wnd_ucmp
> > > > >
> > > > > WARNING:
> > > > >
> > > > > ERROR :
> > > > >
> > > > > ERROR : process_fcst_climo_files() -> no requested forecast
data
> > > found!
> > > > >
> > > > > Exiting...
> > > > >
> > > > > ERROR :
> > > > >
> > > > >
> > > > >
> > > > > Justin
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Justin Tsu
> > > > >
> > > > > Marine Meteorology Division
> > > > >
> > > > > Data Assimilation/Mesoscale Modeling
> > > > >
> > > > > Building 704 Room 212
> > > > >
> > > > > Naval Research Laboratory, Code 7531
> > > > >
> > > > > 7 Grace Hopper Avenue
> > > > >
> > > > > Monterey, CA 93943-5502
> > > > >
> > > > >
> > > > >
> > > > > Ph. (831) 656-4111
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> > >
> >
> >
> >
>
>
>
------------------------------------------------
Subject: Point Stat Help
From: Tsu, Mr. Justin
Time: Wed May 22 17:06:18 2019
You figured it out! My hdr_typ_table was named radiosonde which I
remember putting into my ascii file before running ascii2nc on it.
This is a very useful thing to know in the future when using nc files
from ascii2nc. And thanks for the tip with the GRIB concatenation.
I've already concatenated my components together and ran a case and am
getting matched pairs. Thanks John!
Justin
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Wednesday, May 22, 2019 3:48 PM
To: Tsu, Mr. Justin
Subject: Re: [rt.rap.ucar.edu #90256] Point Stat Help
Justin,
The rejection counts are listed in the order that the logic is
applied.
Looks like the final 2,431 of them were rejected based on message
type:
DEBUG 3: Rejected: message type = 2431
So what message type is in your obs file?
*ncdump -v hdr_typ_table out/pb2nc/sample_pb.nc <http://sample_pb.nc>*
*... hdr_typ_table = "ADPUPA", "AIRCFT", "PROFLR", "VADWND",
"ADPSFC", "SFCSHP", "RASSDA" ;}*
The Point-Stat config file allows you to configure which message types
you'd like to use. Just set it to the one(s) you actually have in
your
data:
*message_type = [ "ADPUPA" ];*
Point-Stat is only set up to read a single input forecast file at each
time. But if you're using GRIB1 or GRIB2 input files, they can
literally
be catt'ed together:
*cat file1.grb2 file2.grb2 > combined.grb2*
So that's the quick and dirty answer.
Thanks,
John
On Wed, May 22, 2019 at 4:39 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90256 >
>
> John,
>
> Thank you for the clear explanation. What do I do if my u component
and v
> component winds are in separate GRIB files?
>
> I am still getting 0 pairs of matched obs even when I set my
pressure
> range to look for obs between P700 and P1025.
>
> [tsu at maury2 00_POINT_STAT_WORK]$ point_stat
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_00000F0NL2019042612_0100_010000-000000wnd_ucmp
> raob_2019042612.nc config.rc -v 3
> DEBUG 1: Default Config File:
> /software/depot/met/met-
8.0_bugfix/share/met/config/PointStatConfig_default
> DEBUG 1: User Config File: config.rc
> GSL_RNG_TYPE=mt19937
> GSL_RNG_SEED=18446744071575299420
> DEBUG 1: Forecast File:
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_00000F0NL2019042612_0100_010000-000000wnd_ucmp
> DEBUG 1: Observation File: raob_2019042612.nc
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match
for
> VarInfo "UGRD/P1000" in GRIB record 1 of GRIB file
> "GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_00000F0NL2019042612_0100_010000-000000wnd_ucmp".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "UGRD/P1000" in GRIB file
> "GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_00000F0NL2019042612_0100_010000-000000wnd_ucmp".
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Reading data for UGRD/P1000.
> DEBUG 3: Use the matching forecast and observation grids.
> DEBUG 3: Grid Definition: Projection: Mercator Nx: 202 Ny: 202
> Lat_LL_radians: 0.4770 Lon_LL_radians: 2.3860 Lat_UR_radians: 0.8214
> Lon_UR_radians: 1.9489 Mx: 459.8120 My: 459.8003 Bx: 1097.1297 By:
-228.1298
> DEBUG 2: Processing masking regions.
> DEBUG 3: Processing grid mask: FULL
> DEBUG 2: For UGRD/P1000 found 1 forecast levels, 0 climatology mean
> levels, and 0 climatology standard deviation levels.
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Searching 1570630 observations from 82 messages.
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Processing UGRD/P1000 versus ws/P1025-700, for observation
type
> ADPUPA, over region FULL, for interpolation method NEAREST(1), using
0
> pairs.
> DEBUG 3: Number of matched pairs = 0
> DEBUG 3: Observations processed = 1570630
> DEBUG 3: Rejected: SID exclusion = 0
> DEBUG 3: Rejected: obs type = 1310954
> DEBUG 3: Rejected: valid time = 0
> DEBUG 3: Rejected: bad obs value = 0
> DEBUG 3: Rejected: off the grid = 225197
> DEBUG 3: Rejected: level mismatch = 32048
> DEBUG 3: Rejected: quality marker = 0
> DEBUG 3: Rejected: message type = 2431
> DEBUG 3: Rejected: masking region = 0
> DEBUG 3: Rejected: bad fcst value = 0
> DEBUG 3: Rejected: duplicates = 0
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 1: Output file: ./point_stat_000000L_20190426_120000V.stat
> DEBUG 1: Output file: ./point_stat_000000L_20190426_120000V_cnt.txt
>
>
> Justin
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Wednesday, May 22, 2019 2:51 PM
> To: Tsu, Mr. Justin
> Subject: Re: [rt.rap.ucar.edu #90256] Point Stat Help
>
> Justin,
>
> One nice feature in Point-Stat is that when you request "WIND" (i.e.
the
> GRIB abbreviation for wind speed), it will find the corresponding
"UGRD"
> and "VGRD" records and derive wind speed for you on the fly.
>
> So please try:
> *fcst = {*
> * field = [ { name = "WIND"; level = "P1000"; } ];*
> *}*
> *obs = {*
> * field = [ { name ="ws"; level = "P975-1025"; } ];*
> *}*
>
> This tells Point-Stat to:
>
> (1) Since WIND/P1000 doesn't exist in the input file, it will read
> UGRD/P1000 and VGRD/P1000 and derive wind speed.
> (2) Look through all of the point observations and consider the ones
named
> "ws". Any of them with a pressure level between 975 and 1025 will
be
> verified against the forecast wind speed value at P1000.
>
>
> If you want to do vertical interpolation, you'll need to configure
it a bit
> differently. Let's say you have model output every 100 mb. You
could
> configure it like this:
>
> *fcst = {*
> * field = [ { name = "WIND"; level = "P900-1000"; } ];*
> *}*
> *obs = {*
> * field = [ { name ="ws"; level = "P900-1000"; } ];*
> *}*
>
> In this case Point-Stat will compute wind speed for both the 900 and
1000
> mb levels. It'll only use obs falling between 900mb and 1000mb.
For
> example, for an obs at 925mb, it'll first interpolate horizontally
to the
> lat/lon observation location at the forecast levels above and below.
Then
> it'll use those 2 values to interpolate to the obs pressure level,
linear
> in the log of pressure.
>
> So the functionality is really controlled by how the fcst/obs levels
are
> defined in the config file.
>
> Thanks,
> John
>
> On Wed, May 22, 2019 at 3:35 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90256 >
> >
> > Awesome!
> >
> > Heres what I get from dumping obv_var
> >
> > data:
> >
> > obs_var =
> > "ws",
> > "wdir",
> > "t",
> > "dptd",
> > "pres",
> > "ght" ;
> > }
> >
> > I'm assuming I'll need ws here. If I specify my obs to be at
P1000, will
> > point_stat use the interp that I specified to find the obs nearest
to
> that
> > height? Also is there a way to verify this components of wind
with wind
> > speed since for this example I am only using UGRD (u component)?
I'm
> > guessing I will have to specify a second field or something.
> >
> > Justin
> >
> > -----Original Message-----
> > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > Sent: Wednesday, May 22, 2019 2:17 PM
> > To: Tsu, Mr. Justin
> > Subject: Re: [rt.rap.ucar.edu #90256] Point Stat Help
> >
> > Justin,
> >
> > OK, this is good info. As you point out, the problem is a
mismatch in
> the
> > requested observation variable name and the actual observation
variable
> > name:
> > DEBUG 3: Rejected: obs type = 1570630
> >
> > This should be very easy to fix. We just need to figure out how
your
> > observations are named in the file named "raob_2019042612.nc".
Just
> take
> > a
> > look at the contents of the "obs_var" variable in that file. For
> example,
> > here's some output from pb2nc:
> >
> > ncdump -v obs_var out/pb2nc/sample_pb.nc
> > ...
> > obs_var =
> > "SPFH",
> > "TMP",
> > "HGT",
> > "UGRD",
> > "VGRD",
> > "DPT",
> > "WIND",
> > "RH",
> > "MIXR" ;
> > }
> >
> > In this case, the U-component is wind really is called "UGRD".
But
> > obviously in your data it is not. Let say, for example, that it's
called
> > "UWIND". You just need to edit your Point-Stat config file to
tell it
> to
> > use those variables:
> >
> >
> >
> >
> >
> >
> >
> >
> > *obs = { field = [ { name = "UWIND";
> > level = ["P1000"]; } ];}*
> >
> > On Wed, May 22, 2019 at 1:31 PM Tsu, Mr. Justin via RT <
> met_help at ucar.edu>
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90256 >
> > >
> > > Hey John,
> > >
> > > Just ran with verbosity set to 3 and discovered that no pairs
are being
> > > used:
> > >
> > > [tsu at maury2 00_POINT_STAT_WORK]$ point_stat
> > >
> >
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_00000F0NL2019042612_0100_010000-000000wnd_ucmp
> > > raob_2019042612.nc config.rc -v 3
> > > DEBUG 1: Default Config File:
> > >
> >
> /software/depot/met/met-
8.0_bugfix/share/met/config/PointStatConfig_default
> > > DEBUG 1: User Config File: config.rc
> > > GSL_RNG_TYPE=mt19937
> > > GSL_RNG_SEED=18446744072713528668
> > > DEBUG 1: Forecast File:
> > >
> >
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_00000F0NL2019042612_0100_010000-000000wnd_ucmp
> > > DEBUG 1: Observation File: raob_2019042612.nc
> > > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact
match for
> > > VarInfo "UGRD/P1000" in GRIB record 1 of GRIB file
> > >
> >
> "GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_00000F0NL2019042612_0100_010000-000000wnd_ucmp".
> > > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> > > matching VarInfo "UGRD/P1000" in GRIB file
> > >
> >
> "GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_00000F0NL2019042612_0100_010000-000000wnd_ucmp".
> > > DEBUG 2:
> > > DEBUG 2:
> > >
> >
>
--------------------------------------------------------------------------------
> > > DEBUG 2:
> > > DEBUG 2: Reading data for UGRD/P1000.
> > > DEBUG 3: Use the matching forecast and observation grids.
> > > DEBUG 3: Grid Definition: Projection: Mercator Nx: 202 Ny: 202
> > > Lat_LL_radians: 0.4770 Lon_LL_radians: 2.3860 Lat_UR_radians:
0.8214
> > > Lon_UR_radians: 1.9489 Mx: 459.8120 My: 459.8003 Bx: 1097.1297
By:
> > -228.1298
> > > DEBUG 2: Processing masking regions.
> > > DEBUG 3: Processing grid mask: FULL
> > > DEBUG 2: For UGRD/P1000 found 1 forecast levels, 0 climatology
mean
> > > levels, and 0 climatology standard deviation levels.
> > > DEBUG 2:
> > > DEBUG 2:
> > >
> >
>
--------------------------------------------------------------------------------
> > > DEBUG 2:
> > > DEBUG 2: Searching 1570630 observations from 82 messages.
> > > DEBUG 2:
> > > DEBUG 2:
> > >
> >
>
--------------------------------------------------------------------------------
> > > DEBUG 2:
> > > DEBUG 2: Processing UGRD/P1000 versus UGRD/P1000, for
observation type
> > > ADPUPA, over region FULL, for interpolation method NEAREST(1),
using 0
> > > pairs.
> > > DEBUG 3: Number of matched pairs = 0
> > > DEBUG 3: Observations processed = 1570630
> > > DEBUG 3: Rejected: SID exclusion = 0
> > > DEBUG 3: Rejected: obs type = 1570630
> > > DEBUG 3: Rejected: valid time = 0
> > > DEBUG 3: Rejected: bad obs value = 0
> > > DEBUG 3: Rejected: off the grid = 0
> > > DEBUG 3: Rejected: level mismatch = 0
> > > DEBUG 3: Rejected: quality marker = 0
> > > DEBUG 3: Rejected: message type = 0
> > > DEBUG 3: Rejected: masking region = 0
> > > DEBUG 3: Rejected: bad fcst value = 0
> > > DEBUG 3: Rejected: duplicates = 0
> > > DEBUG 2:
> > > DEBUG 2:
> > >
> >
>
--------------------------------------------------------------------------------
> > > DEBUG 2:
> > > DEBUG 1: Output file: ./point_stat_000000L_20190426_120000V.stat
> > > DEBUG 1: Output file:
./point_stat_000000L_20190426_120000V_cnt.txt
> > >
> > > I had an issue with this in the past using NAVGEM IEEE flatfile
format
> > > climatologies that I converted to netcdf using ascii2nc as obs.
I
> fixed
> > it
> > > by using python embedding in the name part of the obs
dictionary.
> > Problem
> > > is I cannot do this here because my obs are originally in hdf5
format
> > and I
> > > don’t have a way to use python embedding to provide MET with the
> > necessary
> > > metadata like read_NRL_binary.py does (I believe somebody from
your
> team
> > > provided us with this python script to print out metadata based
on a
> > > flatfile format that we use)
> > >
> > > Have you guys had any luck in using ascii2nc generated netcdf
files as
> > > either forecast or obs in any of the MET stat utilities?
> > >
> > > Justin
> > >
> > > -----Original Message-----
> > > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > > Sent: Wednesday, May 22, 2019 11:27 AM
> > > To: Tsu, Mr. Justin
> > > Subject: Re: [rt.rap.ucar.edu #90256] Point Stat Help
> > >
> > > Justin,
> > >
> > > When you run Point-Stat, please increase the verbosity level to
at
> least
> > 3
> > > (-v 3). That will dump log messages that look something like
this:
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > *DEBUG 2: Processing TMP/P900-750 versus TMP/P900-750, for
observation
> > type
> > > ADPUPA, over region DTC165, for interpolation method NEAREST(1),
using
> > 132
> > > pairs.DEBUG 3: Number of matched pairs = 132DEBUG 3:
Observations
> > > processed = 89893DEBUG 3: Rejected: SID exclusion = 0DEBUG 3:
> > Rejected:
> > > obs type = 79788DEBUG 3: Rejected: valid time = 0DEBUG
3:
> > > Rejected: bad obs value = 0DEBUG 3: Rejected: off the grid =
5DEBUG
> 3:
> > > Rejected: topography = 0DEBUG 3: Rejected: level mismatch =
> 9219DEBUG
> > > 3: Rejected: quality marker = 0DEBUG 3: Rejected: message type
=
> > 344DEBUG
> > > 3: Rejected: masking region = 405DEBUG 3: Rejected: bad fcst
value =
> > 0DEBUG
> > > 3: Rejected: duplicates = 0*
> > >
> > > For each verification task, these list counts of reasons why
> observations
> > > were not used for that task. These counts should help you
determine
> why
> > > you're getting 0 matched pairs. There are many possible reasons
why...
> > > mismatches in message type, valid time, lat/lon location,
vertical
> > level...
> > > and so on.
> > >
> > > Hopefully these counts will point you in the right direction.
> > >
> > > Thanks,
> > > John
> > >
> > > On Tue, May 21, 2019 at 10:50 PM Tsu, Mr. Justin via RT <
> > met_help at ucar.edu
> > > >
> > > wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90256
>
> > > >
> > > > Hi John,
> > > >
> > > > So I took your advice and used wgrib to determine that my
variable is
> > > > indeed named UGRD so I switched this out in my point_Stat
config
> file.
> > > Now
> > > > that config file looks like this:
> > > >
> > > > 52 fcst = {
> > > > 53 field = [
> > > > 54 {
> > > > 55 name = "UGRD";
> > > > 56 level = ["P1000"];
> > > > 57 file_type = GRIB1;
> > > > 58 cat_thresh = [ >80.0 ];
> > > > 59 }
> > > > 60 ];
> > > > 61
> > > > 62 }
> > > > 63
> > > > 64 obs = {
> > > > 65 field = [
> > > > 66 {
> > > > 67 file_type = NETCDF_NCCF;
> > > > 68 name = "UGRD";
> > > > 69 level = ["P1000"];
> > > > 70 }
> > > > 71 ];
> > > > 72 }
> > > >
> > > > This config file runs when I execute:
> > > >
> > > > point_stat
> > > >
> > >
> >
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_02700F0NL2019042612_0100_010000-000000wnd_ucmp
> > > > NAF_sfc_2019042612.nc config.rc -v 1
> > > >
> > > > However, the resulting contingency table file are empty (aside
from
> the
> > > > header). I am guessing this has something to do with the way
I
> specify
> > > my
> > > > obs file. As I stated previously in my email, the obs file is
named
> > > > raob_2019042612.nc and is a netcdf file created by using
ascii2nc
> on a
> > > > list of point obs arranged in the format specified by the MET
> tutorial.
> > > I
> > > > am not sure what the name value in the field dictionary refers
to
> > because
> > > > the netcdf header for raob_2019042612.nc looks like:
> > > >
> > > > 1 netcdf raob_2019042612 {
> > > > 2 dimensions:
> > > > 3 mxstr = 16 ;
> > > > 4 mxstr2 = 40 ;
> > > > 5 mxstr3 = 80 ;
> > > > 6 nobs = 1570630 ;
> > > > 7 nhdr = 82 ;
> > > > 8 nhdr_typ = 1 ;
> > > > 9 nhdr_sid = 82 ;
> > > > 10 nhdr_vld = 1 ;
> > > > 11 nobs_qty = 1 ;
> > > > 12 obs_var_num = 6 ;
> > > > 13 variables:
> > > > 14 int hdr_typ(nhdr) ;
> > > > 15 hdr_typ:long_name = "index of message type" ;
> > > > 16 hdr_typ:_FillValue = -9999 ;
> > > > 17 int hdr_sid(nhdr) ;
> > > > 18 hdr_sid:long_name = "index of station
identification" ;
> > > > 19 hdr_sid:_FillValue = -9999 ;
> > > > 20 int hdr_vld(nhdr) ;
> > > > 21 hdr_vld:long_name = "index of valid time" ;
> > > > 22 hdr_vld:_FillValue = -9999 ;
> > > > 23 float hdr_lat(nhdr) ;
> > > > 24 hdr_lat:long_name = "latitude" ;
> > > > 25 hdr_lat:_FillValue = -9999.f ;
> > > > 26 hdr_lat:units = "degrees_north" ;
> > > > 27 float hdr_lon(nhdr) ;
> > > > 28 hdr_lon:long_name = "longitude" ;
> > > > 29 hdr_lon:_FillValue = -9999.f ;
> > > > 30 hdr_lon:units = "degrees_east" ;
> > > > 31 float hdr_elv(nhdr) ;
> > > > 32 hdr_elv:long_name = "elevation" ;
> > > > 33 hdr_elv:_FillValue = -9999.f ;
> > > > 34 hdr_elv:units = "meters above sea level (msl)" ;
> > > > 35 int obs_qty(nobs) ;
> > > > 36 obs_qty:long_name = "index of quality flag" ;
> > > > 37 int obs_hid(nobs) ;
> > > > 38 obs_hid:long_name = "index of matching header
data" ;
> > > > 39 obs_hid:_FillValue = -9999 ;
> > > > 40 int obs_vid(nobs) ;
> > > > 41 obs_vid:long_name = "index of variable names at
> var_name" ;
> > > > 42 obs_vid:_FillValue = -9999 ;
> > > > 43 float obs_lvl(nobs) ;
> > > > 44 obs_lvl:long_name = "pressure level (hPa) or
accumulation
> > > > interval (sec)" ;
> > > > 45 obs_lvl:_FillValue = -9999.f ;
> > > > 46 float obs_hgt(nobs) ;
> > > > 47 obs_hgt:long_name = "height in meters above sea
level or
> > > > ground level (msl or agl)" ;
> > > > 48 obs_hgt:_FillValue = -9999.f ;
> > > > 49 float obs_val(nobs) ;
> > > > 50 obs_val:long_name = "observation value" ;
> > > > 51 obs_val:_FillValue = -9999.f ;
> > > > 52 char hdr_typ_table(nhdr_typ, mxstr2) ;
> > > > 53 hdr_typ_table:long_name = "message type" ;
> > > > 54 char hdr_sid_table(nhdr_sid, mxstr2) ;
> > > > 55 hdr_sid_table:long_name = "station identification"
;
> > > > 56 char hdr_vld_table(nhdr_vld, mxstr) ;
> > > > 57 hdr_vld_table:long_name = "valid time" ;
> > > > 58 hdr_vld_table:units = "YYYYMMDD_HHMMSS UTC" ;
> > > > 59 char obs_qty_table(nobs_qty, mxstr) ;
> > > > 60 obs_qty_table:long_name = "quality flag" ;
> > > > 61 char obs_var(obs_var_num, mxstr2) ;
> > > > 62 obs_var:long_name = "variable names" ;
> > > > 63
> > > > 64 // global attributes:
> > > > 65 :MET_Obs_version = "1.02" ;
> > > > 66 :use_var_id = "true" ;
> > > > 67 :FileOrigins = "File / generated 20190510_235941
UTC on
> > host
> > > > maury2.local by the MET ascii2nc tool" ;
> > > > 68 :MET_version = "V8.0" ;
> > > > 69 :MET_tool = "ascii2nc" ;
> > > >
> > > > So there isn’t actualy a netcdf variable named UGRD; at this
point
> I'm
> > > not
> > > > sure what I should call it.
> > > >
> > > > Hope you can shed some light on how this works.
> > > >
> > > > Thanks,
> > > > Justin
> > > >
> > > >
> > > > -----Original Message-----
> > > > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > > > Sent: Monday, May 20, 2019 12:20 PM
> > > > To: Tsu, Mr. Justin
> > > > Subject: Re: [rt.rap.ucar.edu #90256] Point Stat Help
> > > >
> > > > Hi Justin,
> > > >
> > > > I see that you're having trouble reading GRIB data into the
> Point-Stat
> > > tool
> > > > in MET. When getting going with a new gridded dataset in MET,
I
> > usually
> > > > recommend starting with the plot_data_plane utility. For
example,
> try
> > > > running this command:
> > > >
> > > > plot_data_plane
> > > >
> > > >
> > >
> >
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_0270F0NL2019042612_0100_010000-000000wnd_ucmp
> > > > U_P1000.ps 'name="U"; level="P1000";'
> > > >
> > > > You should get a similar error since Point-Stat wasn't able to
read
> it.
> > > > I'd recommend running this file through the "wgrib2" utility
to
> > inventory
> > > > it and see what variable names/levels are present in the file.
For
> > > > example, is it called "U" or "UGRD"?
> > > >
> > > > If you aren't able to get that working, please just post a
sample
> file
> > to
> > > > our anonymous ftp site. If you do, let me know, and I'll go
grab it
> > and
> > > > take a look:
> > > >
> > > >
> > >
> >
> https://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk
> > > >
> > > > Thanks,
> > > > John Halley Gotway
> > > >
> > > >
> > > >
> > > > On Fri, May 17, 2019 at 3:27 PM Tsu, Mr. Justin via RT <
> > > met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > >
> > > > > Fri May 17 15:27:39 2019: Request 90256 was acted upon.
> > > > > Transaction: Ticket created by justin.tsu at nrlmry.navy.mil
> > > > > Queue: met_help
> > > > > Subject: Point Stat Help
> > > > > Owner: Nobody
> > > > > Requestors: justin.tsu at nrlmry.navy.mil
> > > > > Status: new
> > > > > Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90256
> > > >
> > > > >
> > > > >
> > > > > Ive got my forecast field in the form of a grib file
entitled
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_02700F0NL2019042612_
> > > > > 0100_010000-000000wnd_ucmp
> > > > >
> > > > >
> > > > >
> > > > > This is the U Component of wind at 1000 mb. The following
is a
> > > > descriptor
> > > > > file to read this in GrADS:
> > > > >
> > > > >
> > > > >
> > > > > 1 dset
> > > > >
> > > > >
> > > >
> > >
> >
> ^US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_03000F0NL2019042612_0100_0100
> > > > > 00-000000wnd_ucmp
> > > > >
> > > > > 2 index
> > > > >
> > > > >
> > > >
> > >
> >
> ^US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_03000F0NL2019042612_0100_0100
> > > > > 00-000000wnd_ucmp.idx
> > > > >
> > > > > 3 undef 9.999E+20
> > > > >
> > > > > 4 title
> > > > >
> > > > >
> > > >
> > >
> >
> US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_03000F0NL2019042612_0100_01000
> > > > > 0-000000wnd_ucmp
> > > > >
> > > > > 5 * produced by grib2ctl v0.9.15
> > > > >
> > > > > 6 * command line options:
> > > > >
> > > > >
> > > >
> > >
> >
> US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_03000F0NL2019042612_0100_01000
> > > > > 0-000000wnd_ucmp
> > > > >
> > > > > 7 dtype grib 255
> > > > >
> > > > > 8 ydef 202 levels
> > > > >
> > > > > 9 27.3280 27.4386 27.5492 27.6596 27.7699
27.8801
> > > 27.9902
> > > > >
> > > > > 10 28.1002 28.2101 28.3198 28.4295 28.5390
28.6484
> > > 28.7577
> > > > >
> > > > > 11 28.8669 28.9759 29.0849 29.1937 29.3025
29.4111
> > > 29.5196
> > > > >
> > > > > 12 29.6279 29.7362 29.8443 29.9524 30.0603
30.1681
> > > 30.2757
> > > > >
> > > > > 13 30.3833 30.4907 30.5980 30.7052 30.8123
30.9193
> > > 31.0261
> > > > >
> > > > > 14 31.1329 31.2395 31.3459 31.4523 31.5585
31.6647
> > > 31.7707
> > > > >
> > > > > 15 31.8765 31.9823 32.0879 32.1934 32.2988
32.4041
> > > 32.5093
> > > > >
> > > > > 16 32.6143 32.7192 32.8240 32.9286 33.0331
33.1375
> > > 33.2418
> > > > >
> > > > > 17 33.3460 33.4500 33.5539 33.6577 33.7614
33.8649
> > > 33.9683
> > > > >
> > > > > 18 34.0716 34.1747 34.2778 34.3807 34.4835
34.5861
> > > 34.6886
> > > > >
> > > > > 19 34.7910 34.8933 34.9954 35.0975 35.1993
35.3011
> > > 35.4027
> > > > >
> > > > > 20 35.5042 35.6056 35.7069 35.8080 35.9090
36.0099
> > > 36.1106
> > > > >
> > > > > 21 36.2112 36.3117 36.4120 36.5122 36.6123
36.7123
> > > 36.8121
> > > > >
> > > > > 22 36.9118 37.0114 37.1108 37.2101 37.3093
37.4083
> > > 37.5073
> > > > >
> > > > > 23 37.6060 37.7047 37.8032 37.9016 37.9999
38.0980
> > > 38.1960
> > > > >
> > > > > 24 38.2939 38.3916 38.4892 38.5867 38.6840
38.7812
> > > 38.8783
> > > > >
> > > > > 25 38.9752 39.0720 39.1687 39.2653 39.3617
39.4579
> > > 39.5541
> > > > >
> > > > > 26 39.6501 39.7460 39.8417 39.9373 40.0328
40.1281
> > > 40.2234
> > > > >
> > > > > 27 40.3184 40.4134 40.5082 40.6029 40.6974
40.7918
> > > 40.8861
> > > > >
> > > > > 28 40.9802 41.0742 41.1681 41.2618 41.3554
41.4489
> > > 41.5422
> > > > >
> > > > > 29 41.6354 41.7285 41.8214 41.9142 42.0069
42.0994
> > > 42.1918
> > > > >
> > > > > 30 42.2841 42.3762 42.4682 42.5600 42.6517
42.7433
> > > 42.8348
> > > > >
> > > > > 31 42.9261 43.0173 43.1083 43.1992 43.2900
43.3806
> > > 43.4711
> > > > >
> > > > > 32 43.5615 43.6517 43.7418 43.8318 43.9216
44.0113
> > > 44.1008
> > > > >
> > > > > 33 44.1902 44.2795 44.3687 44.4577 44.5466
44.6353
> > > 44.7239
> > > > >
> > > > > 34 44.8124 44.9007 44.9889 45.0770 45.1649
45.2527
> > > 45.3403
> > > > >
> > > > > 35 45.4278 45.5152 45.6025 45.6896 45.7766
45.8634
> > > 45.9501
> > > > >
> > > > > 36 46.0367 46.1231 46.2094 46.2956 46.3816
46.4675
> > > 46.5533
> > > > >
> > > > > 37 46.6389 46.7244 46.8098 46.8950 46.9801
47.0650
> > > > >
> > > > > 38 xdef 202 linear -136.710000 0.124606965174129
> > > > >
> > > > > 39 tdef 1 linear 12Z26apr2019 1mo
> > > > >
> > > > > 40 zdef 1 linear 1 1
> > > > >
> > > > > 41 vars 1
> > > > >
> > > > > 42 UGRD1000mb 0 33,100,1000 ** 1000 mb u wind [m/s]
> > > > >
> > > > > 43 ENDVARS
> > > > >
> > > > >
> > > > >
> > > > > My ob file is in a netcdf file created using ascii2nc. I
named
> wind
> > > > speed
> > > > > obs "ws" in the ascii file that I ran the ascii2nc tool on (
I am
> > aware
> > > > > that
> > > > > I am trying to compare Wind speed and u component of wind,
but I
> just
> > > > want
> > > > > to see if I can get point_stat working).
> > > > >
> > > > >
> > > > >
> > > > > How should I arrange my fcst and obs arrays? Here is what
I have
> so
> > > > far:
> > > > >
> > > > >
> > > > >
> > > > > 52 fcst = {
> > > > >
> > > > > 53 field = [
> > > > >
> > > > > 54 {
> > > > >
> > > > > 55 name = "U";
> > > > >
> > > > > 56 level = "P1000";
> > > > >
> > > > > 57 file_type = GRIB2;
> > > > >
> > > > > 58 cat_thresh = [ >80.0 ];
> > > > >
> > > > > 59 }
> > > > >
> > > > > 60 ];
> > > > >
> > > > > 61
> > > > >
> > > > > 62 }
> > > > >
> > > > > 63
> > > > >
> > > > > 64 obs = {
> > > > >
> > > > > 65 field = [
> > > > >
> > > > > 66 {
> > > > >
> > > > > 67 file_type = NETCDF_NCCF;
> > > > >
> > > > > 68 name = "UGRD";
> > > > >
> > > > > 69 level = ["P1000"];
> > > > >
> > > > > 70 }
> > > > >
> > > > > 71 ];
> > > > >
> > > > > 72 }
> > > > >
> > > > >
> > > > >
> > > > > Running the command as follows gives me this error:
> > > > >
> > > > >
> > > > >
> > > > > [tsu at maury2 00_POINT_STAT_WORK]$ point_stat
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_02700F0NL2019042612_
> > > > > 0100_010000-000000wnd_ucmp
> > > > >
> > > > > NAF_sfc_2019042612.nc config.rc -v 1
> > > > >
> > > > > DEBUG 1: Default Config File:
> > > > >
> > > > >
> > > >
> > >
> >
> /software/depot/met/met-
8.0_bugfix/share/met/config/PointStatConfig_default
> > > > >
> > > > > DEBUG 1: User Config File: config.rc
> > > > >
> > > > > GSL_RNG_TYPE=mt19937
> > > > >
> > > > > GSL_RNG_SEED=1726142812
> > > > >
> > > > > DEBUG 1: Forecast File:
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_02700F0NL2019042612_
> > > > > 0100_010000-000000wnd_ucmp
> > > > >
> > > > > DEBUG 1: Observation File: NAF_sfc_2019042612.nc
> > > > >
> > > > > WARNING:
> > > > >
> > > > > WARNING: process_fcst_climo_files() -> no fields matching
U/P1000
> > found
> > > > in
> > > > >
> > > > > file:
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> GRIB_UCMP/US058GMET-GR1dyn.COAMPS-CENCOOS_CENCOOS-n2-
c1_02700F0NL2019042612_
> > > > > 0100_010000-000000wnd_ucmp
> > > > >
> > > > > WARNING:
> > > > >
> > > > > ERROR :
> > > > >
> > > > > ERROR : process_fcst_climo_files() -> no requested forecast
data
> > > found!
> > > > >
> > > > > Exiting...
> > > > >
> > > > > ERROR :
> > > > >
> > > > >
> > > > >
> > > > > Justin
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Justin Tsu
> > > > >
> > > > > Marine Meteorology Division
> > > > >
> > > > > Data Assimilation/Mesoscale Modeling
> > > > >
> > > > > Building 704 Room 212
> > > > >
> > > > > Naval Research Laboratory, Code 7531
> > > > >
> > > > > 7 Grace Hopper Avenue
> > > > >
> > > > > Monterey, CA 93943-5502
> > > > >
> > > > >
> > > > >
> > > > > Ph. (831) 656-4111
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> > >
> >
> >
> >
>
>
>
------------------------------------------------
More information about the Met_help
mailing list