[Met_help] [rt.rap.ucar.edu #93808] History for Bad Fcst Value
John Halley Gotway via RT
met_help at ucar.edu
Fri Jan 17 14:27:03 MST 2020
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hey John,
For some reason I seem to be having all of my relative humidity obs rejected
due to bad fcst value. This did not happen before. What triggers a bad
fcst value?
DEBUG 1: Forecast File: PYTHON_NUMPY
DEBUG 1: Observation File: sfc_2019072112.nc
DEBUG 2:
DEBUG 2:
----------------------------------------------------------------------------
----
DEBUG 2:
DEBUG 2: Reading data for airtmp/zht_000002.
DEBUG 3: Use the observation grid.
DEBUG 3: Grid Definition: Projection: Mercator Nx: 127 Ny: 127
Lat_LL_radians: 0.0354 Lon_LL_radians: 1.9401 Lat_UR_radians: 0.8649
Lon_UR_radians: 0.9768 Mx: 130.8075 My: 130.8067 Bx: 253.7739 By: -4.6309
DEBUG 2: Processing masking regions.
DEBUG 3: Processing grid mask: FULL
DEBUG 2: Processing geography data.
DEBUG 2: For airtmp/zht_000002 found 1 forecast levels, 0 climatology mean
levels, and 0 climatology standard deviation levels.
DEBUG 2:
DEBUG 2:
----------------------------------------------------------------------------
----
DEBUG 2:
DEBUG 2: Searching 48219 observations from 48219 messages.
DEBUG 2:
DEBUG 2:
----------------------------------------------------------------------------
----
DEBUG 2:
DEBUG 2: Processing airtmp/zht_000002 versus rh/P9999-0, for observation
type SYNOP, over region FULL, for interpolation method NEAREST(1), using 0
pairs.
DEBUG 3: Number of matched pairs = 0
DEBUG 3: Observations processed = 48219
DEBUG 3: Rejected: SID exclusion = 0
DEBUG 3: Rejected: obs type = 40035
DEBUG 3: Rejected: valid time = 0
DEBUG 3: Rejected: bad obs value = 0
DEBUG 3: Rejected: off the grid = 6596
DEBUG 3: Rejected: topography = 0
DEBUG 3: Rejected: level mismatch = 0
DEBUG 3: Rejected: quality marker = 0
DEBUG 3: Rejected: message type = 0
DEBUG 3: Rejected: masking region = 0
DEBUG 3: Rejected: bad fcst value = 1588
DEBUG 3: Rejected: duplicates = 0
DEBUG 2:
DEBUG 2:
----------------------------------------------------------------------------
----
DEBUG 2:
DEBUG 1: Output file:
./out/point_stat/SYNOP/point_stat_airtmprh_000000L_20190721_120000V.stat
DEBUG 1: Output file:
./out/point_stat/SYNOP/point_stat_airtmprh_000000L_20190721_120000V_cnt.txt
DEBUG 1: Output file:
./out/point_stat/SYNOP/point_stat_airtmprh_000000L_20190721_120000V_mpr.txt
Justin Tsu
Marine Meteorology Division
Data Assimilation/Mesoscale Modeling
Building 704 Room 212
Naval Research Laboratory, Code 7531
7 Grace Hopper Avenue
Monterey, CA 93943-5502
Ph. (831) 656-4111
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Bad Fcst Value
From: Julie Prestopnik
Time: Wed Jan 15 12:52:05 2020
Hi Justin.
I see that some of your data is being rejected due to a "bad fcst
value".
John is likely best suited to answer your question, unfortunately he
is out
of the office at a conference. I have assigned this ticket to him.
Please
allow a few business days for a response from him. In the meantime,
you
could try running the plot_data_plane tool on the data to see if that
shows
any obvious problems.
Julie
--
Julie Prestopnik
Software Engineer
National Center for Atmospheric Research
Research Applications Laboratory
Phone: 303.497.8399
Email: jpresto at ucar.edu
My working day may not be your working day. Please do not feel
obliged to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: Bad Fcst Value
From: Tsu, Mr. Justin
Time: Wed Jan 15 17:53:48 2020
Thanks Julie,
I figured out the issue. I do have another question which you may be
able to
help me. From the documentation, it appears that you can set the
level to be
"(*,*)" in the obs dictionary and that should consider any ob at any
vertical
level for verification . When I set level to be "(*,*)", I get a
failed
parsing in point_stat. How do I tell point_stat to ignore the
vertical level
of my ob?
Justin
-----Original Message-----
From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
Sent: Wednesday, January 15, 2020 11:52 AM
To: Tsu, Mr. Justin
Subject: Re: [rt.rap.ucar.edu #93808] Bad Fcst Value
Hi Justin.
I see that some of your data is being rejected due to a "bad fcst
value".
John is likely best suited to answer your question, unfortunately he
is out
of the office at a conference. I have assigned this ticket to him.
Please
allow a few business days for a response from him. In the meantime,
you
could try running the plot_data_plane tool on the data to see if that
shows
any obvious problems.
Julie
--
Julie Prestopnik
Software Engineer
National Center for Atmospheric Research
Research Applications Laboratory
Phone: 303.497.8399
Email: jpresto at ucar.edu
My working day may not be your working day. Please do not feel
obliged to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: Bad Fcst Value
From: Julie Prestopnik
Time: Thu Jan 16 09:46:25 2020
Hi Justin.
I'm glad you were able to get the other issue figured out.
Regarding, setting the level to "(*,*)". Please confirm that you have
a
NetCDF file, as that syntax is for NetCDF files only. Also, I believe
the
two asterisks are for two dimensional fields, so if you had a three
dimensional field, for example, you would use "(*,*,*)". If you still
experience an error, please copy and paste the output, including the
error,
into an email so we can better see what is happening. Thanks!
Julie
On Wed, Jan 15, 2020 at 5:54 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93808 >
>
> Thanks Julie,
>
> I figured out the issue. I do have another question which you may
be able
> to
> help me. From the documentation, it appears that you can set the
level to
> be
> "(*,*)" in the obs dictionary and that should consider any ob at any
> vertical
> level for verification . When I set level to be "(*,*)", I get a
failed
> parsing in point_stat. How do I tell point_stat to ignore the
vertical
> level
> of my ob?
>
> Justin
>
> -----Original Message-----
> From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> Sent: Wednesday, January 15, 2020 11:52 AM
> To: Tsu, Mr. Justin
> Subject: Re: [rt.rap.ucar.edu #93808] Bad Fcst Value
>
> Hi Justin.
>
> I see that some of your data is being rejected due to a "bad fcst
value".
> John is likely best suited to answer your question, unfortunately he
is out
> of the office at a conference. I have assigned this ticket to him.
Please
> allow a few business days for a response from him. In the
meantime, you
> could try running the plot_data_plane tool on the data to see if
that shows
> any obvious problems.
>
> Julie
>
> --
> Julie Prestopnik
> Software Engineer
> National Center for Atmospheric Research
> Research Applications Laboratory
> Phone: 303.497.8399
> Email: jpresto at ucar.edu
>
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>
>
--
Julie Prestopnik
Software Engineer
National Center for Atmospheric Research
Research Applications Laboratory
Phone: 303.497.8399
Email: jpresto at ucar.edu
My working day may not be your working day. Please do not feel
obliged to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: Bad Fcst Value
From: Tsu, Mr. Justin
Time: Thu Jan 16 12:04:34 2020
Thanks Julie,
Here is my command line input and output:
[tsu at lorenz 02_POINT_STAT_WORK]$ !point_stat
point_stat PYTHON_NUMPY sfc_2019072100.nc testConfig -v 3
DEBUG 1: Default Config File:
/software/depot/met-8.1a/met-
8.1a/share/met//config/PointStatConfig_default
DEBUG 1: User Config File: testConfig
ERROR :
ERROR : VarInfo::set_level_info_grib() - failed to parse level string
'(*)'
ERROR :
Sfc_2019072100.nc is a netcdf file. I've attached the header below:
ncdump -h sfc_2019072100.nc
netcdf sfc_2019072100 {
dimensions:
mxstr = 16 ;
mxstr2 = 40 ;
mxstr3 = 80 ;
nobs = 18034 ;
nhdr = 18034 ;
nhdr_typ = 1 ;
nhdr_sid = 3316 ;
nhdr_vld = 1 ;
nobs_qty = 1 ;
obs_var_num = 6 ;
variables:
int hdr_typ(nhdr) ;
hdr_typ:long_name = "index of message type" ;
hdr_typ:_FillValue = -9999 ;
int hdr_sid(nhdr) ;
hdr_sid:long_name = "index of station identification"
;
hdr_sid:_FillValue = -9999 ;
int hdr_vld(nhdr) ;
hdr_vld:long_name = "index of valid time" ;
hdr_vld:_FillValue = -9999 ;
float hdr_lat(nhdr) ;
hdr_lat:long_name = "latitude" ;
hdr_lat:_FillValue = -9999.f ;
hdr_lat:units = "degrees_north" ;
float hdr_lon(nhdr) ;
hdr_lon:long_name = "longitude" ;
hdr_lon:_FillValue = -9999.f ;
hdr_lon:units = "degrees_east" ;
float hdr_elv(nhdr) ;
hdr_elv:long_name = "elevation" ;
hdr_elv:_FillValue = -9999.f ;
hdr_elv:units = "meters above sea level (msl)" ;
int obs_qty(nobs) ;
obs_qty:long_name = "index of quality flag" ;
int obs_hid(nobs) ;
obs_hid:long_name = "index of matching header data" ;
obs_hid:_FillValue = -9999 ;
int obs_vid(nobs) ;
obs_vid:long_name = "index of variable names at
var_name" ;
obs_vid:_FillValue = -9999 ;
float obs_lvl(nobs) ;
obs_lvl:long_name = "pressure level (hPa) or
accumulation
interval (sec)" ;
obs_lvl:_FillValue = -9999.f ;
float obs_hgt(nobs) ;
obs_hgt:long_name = "height in meters above sea level
or
ground level (msl or agl)" ;
obs_hgt:_FillValue = -9999.f ;
float obs_val(nobs) ;
obs_val:long_name = "observation value" ;
obs_val:_FillValue = -9999.f ;
char hdr_typ_table(nhdr_typ, mxstr2) ;
hdr_typ_table:long_name = "message type" ;
char hdr_sid_table(nhdr_sid, mxstr2) ;
hdr_sid_table:long_name = "station identification" ;
char hdr_vld_table(nhdr_vld, mxstr) ;
hdr_vld_table:long_name = "valid time" ;
hdr_vld_table:units = "YYYYMMDD_HHMMSS UTC" ;
char obs_qty_table(nobs_qty, mxstr) ;
obs_qty_table:long_name = "quality flag" ;
char obs_var(obs_var_num, mxstr2) ;
obs_var:long_name = "variable names" ;
// global attributes:
:MET_Obs_version = "1.02" ;
:use_var_id = "true" ;
:FileOrigins = "File / generated 20200116_002940 UTC
on host
lorenz.nrlmry.navy.mil by the MET ascii2nc tool" ;
:MET_version = "V8.1" ;
:MET_tool = "ascii2nc" ;
}
Hope you can help me out!
Justin
-----Original Message-----
From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
Sent: Thursday, January 16, 2020 8:46 AM
To: Tsu, Mr. Justin
Subject: Re: [rt.rap.ucar.edu #93808] Bad Fcst Value
Hi Justin.
I'm glad you were able to get the other issue figured out.
Regarding, setting the level to "(*,*)". Please confirm that you have
a
NetCDF file, as that syntax is for NetCDF files only. Also, I believe
the
two asterisks are for two dimensional fields, so if you had a three
dimensional field, for example, you would use "(*,*,*)". If you still
experience an error, please copy and paste the output, including the
error,
into an email so we can better see what is happening. Thanks!
Julie
On Wed, Jan 15, 2020 at 5:54 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93808 >
>
> Thanks Julie,
>
> I figured out the issue. I do have another question which you may
be able
> to
> help me. From the documentation, it appears that you can set the
level to
> be
> "(*,*)" in the obs dictionary and that should consider any ob at any
> vertical
> level for verification . When I set level to be "(*,*)", I get a
failed
> parsing in point_stat. How do I tell point_stat to ignore the
vertical
> level
> of my ob?
>
> Justin
>
> -----Original Message-----
> From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> Sent: Wednesday, January 15, 2020 11:52 AM
> To: Tsu, Mr. Justin
> Subject: Re: [rt.rap.ucar.edu #93808] Bad Fcst Value
>
> Hi Justin.
>
> I see that some of your data is being rejected due to a "bad fcst
value".
> John is likely best suited to answer your question, unfortunately he
is out
> of the office at a conference. I have assigned this ticket to him.
Please
> allow a few business days for a response from him. In the
meantime, you
> could try running the plot_data_plane tool on the data to see if
that shows
> any obvious problems.
>
> Julie
>
> --
> Julie Prestopnik
> Software Engineer
> National Center for Atmospheric Research
> Research Applications Laboratory
> Phone: 303.497.8399
> Email: jpresto at ucar.edu
>
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>
>
--
Julie Prestopnik
Software Engineer
National Center for Atmospheric Research
Research Applications Laboratory
Phone: 303.497.8399
Email: jpresto at ucar.edu
My working day may not be your working day. Please do not feel
obliged to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: Bad Fcst Value
From: Julie Prestopnik
Time: Thu Jan 16 12:41:31 2020
Hi Justin.
Are you able to send me the config file you are using, along with the
sfc_2019072100.nc file, so that I can try to run it myself and figure
out
what is going on?
There are ftp directions under "How To Send Us Data" on this page:
https://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk
Thanks!
Julie
On Thu, Jan 16, 2020 at 12:05 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93808 >
>
> Thanks Julie,
>
> Here is my command line input and output:
>
> [tsu at lorenz 02_POINT_STAT_WORK]$ !point_stat
> point_stat PYTHON_NUMPY sfc_2019072100.nc testConfig -v 3
> DEBUG 1: Default Config File:
> /software/depot/met-8.1a/met-
8.1a/share/met//config/PointStatConfig_default
> DEBUG 1: User Config File: testConfig
> ERROR :
> ERROR : VarInfo::set_level_info_grib() - failed to parse level
string
> '(*)'
> ERROR :
>
> Sfc_2019072100.nc is a netcdf file. I've attached the header below:
> ncdump -h sfc_2019072100.nc
> netcdf sfc_2019072100 {
> dimensions:
> mxstr = 16 ;
> mxstr2 = 40 ;
> mxstr3 = 80 ;
> nobs = 18034 ;
> nhdr = 18034 ;
> nhdr_typ = 1 ;
> nhdr_sid = 3316 ;
> nhdr_vld = 1 ;
> nobs_qty = 1 ;
> obs_var_num = 6 ;
> variables:
> int hdr_typ(nhdr) ;
> hdr_typ:long_name = "index of message type" ;
> hdr_typ:_FillValue = -9999 ;
> int hdr_sid(nhdr) ;
> hdr_sid:long_name = "index of station
identification" ;
> hdr_sid:_FillValue = -9999 ;
> int hdr_vld(nhdr) ;
> hdr_vld:long_name = "index of valid time" ;
> hdr_vld:_FillValue = -9999 ;
> float hdr_lat(nhdr) ;
> hdr_lat:long_name = "latitude" ;
> hdr_lat:_FillValue = -9999.f ;
> hdr_lat:units = "degrees_north" ;
> float hdr_lon(nhdr) ;
> hdr_lon:long_name = "longitude" ;
> hdr_lon:_FillValue = -9999.f ;
> hdr_lon:units = "degrees_east" ;
> float hdr_elv(nhdr) ;
> hdr_elv:long_name = "elevation" ;
> hdr_elv:_FillValue = -9999.f ;
> hdr_elv:units = "meters above sea level (msl)" ;
> int obs_qty(nobs) ;
> obs_qty:long_name = "index of quality flag" ;
> int obs_hid(nobs) ;
> obs_hid:long_name = "index of matching header data"
;
> obs_hid:_FillValue = -9999 ;
> int obs_vid(nobs) ;
> obs_vid:long_name = "index of variable names at
var_name" ;
> obs_vid:_FillValue = -9999 ;
> float obs_lvl(nobs) ;
> obs_lvl:long_name = "pressure level (hPa) or
accumulation
> interval (sec)" ;
> obs_lvl:_FillValue = -9999.f ;
> float obs_hgt(nobs) ;
> obs_hgt:long_name = "height in meters above sea
level or
> ground level (msl or agl)" ;
> obs_hgt:_FillValue = -9999.f ;
> float obs_val(nobs) ;
> obs_val:long_name = "observation value" ;
> obs_val:_FillValue = -9999.f ;
> char hdr_typ_table(nhdr_typ, mxstr2) ;
> hdr_typ_table:long_name = "message type" ;
> char hdr_sid_table(nhdr_sid, mxstr2) ;
> hdr_sid_table:long_name = "station identification" ;
> char hdr_vld_table(nhdr_vld, mxstr) ;
> hdr_vld_table:long_name = "valid time" ;
> hdr_vld_table:units = "YYYYMMDD_HHMMSS UTC" ;
> char obs_qty_table(nobs_qty, mxstr) ;
> obs_qty_table:long_name = "quality flag" ;
> char obs_var(obs_var_num, mxstr2) ;
> obs_var:long_name = "variable names" ;
>
> // global attributes:
> :MET_Obs_version = "1.02" ;
> :use_var_id = "true" ;
> :FileOrigins = "File / generated 20200116_002940 UTC
on
> host
> lorenz.nrlmry.navy.mil by the MET ascii2nc tool" ;
> :MET_version = "V8.1" ;
> :MET_tool = "ascii2nc" ;
> }
>
> Hope you can help me out!
>
> Justin
>
>
> -----Original Message-----
> From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> Sent: Thursday, January 16, 2020 8:46 AM
> To: Tsu, Mr. Justin
> Subject: Re: [rt.rap.ucar.edu #93808] Bad Fcst Value
>
> Hi Justin.
>
> I'm glad you were able to get the other issue figured out.
>
> Regarding, setting the level to "(*,*)". Please confirm that you
have a
> NetCDF file, as that syntax is for NetCDF files only. Also, I
believe the
> two asterisks are for two dimensional fields, so if you had a three
> dimensional field, for example, you would use "(*,*,*)". If you
still
> experience an error, please copy and paste the output, including the
error,
> into an email so we can better see what is happening. Thanks!
>
> Julie
>
> On Wed, Jan 15, 2020 at 5:54 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93808 >
> >
> > Thanks Julie,
> >
> > I figured out the issue. I do have another question which you may
be
> able
> > to
> > help me. From the documentation, it appears that you can set the
level
> to
> > be
> > "(*,*)" in the obs dictionary and that should consider any ob at
any
> > vertical
> > level for verification . When I set level to be "(*,*)", I get a
failed
> > parsing in point_stat. How do I tell point_stat to ignore the
vertical
> > level
> > of my ob?
> >
> > Justin
> >
> > -----Original Message-----
> > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > Sent: Wednesday, January 15, 2020 11:52 AM
> > To: Tsu, Mr. Justin
> > Subject: Re: [rt.rap.ucar.edu #93808] Bad Fcst Value
> >
> > Hi Justin.
> >
> > I see that some of your data is being rejected due to a "bad fcst
value".
> > John is likely best suited to answer your question, unfortunately
he is
> out
> > of the office at a conference. I have assigned this ticket to
him.
> Please
> > allow a few business days for a response from him. In the
meantime, you
> > could try running the plot_data_plane tool on the data to see if
that
> shows
> > any obvious problems.
> >
> > Julie
> >
> > --
> > Julie Prestopnik
> > Software Engineer
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > Phone: 303.497.8399
> > Email: jpresto at ucar.edu
> >
> > My working day may not be your working day. Please do not feel
obliged
> to
> > reply to this email outside of your normal working hours.
> >
> >
> >
>
> --
> Julie Prestopnik
> Software Engineer
> National Center for Atmospheric Research
> Research Applications Laboratory
> Phone: 303.497.8399
> Email: jpresto at ucar.edu
>
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>
>
--
Julie Prestopnik
Software Engineer
National Center for Atmospheric Research
Research Applications Laboratory
Phone: 303.497.8399
Email: jpresto at ucar.edu
My working day may not be your working day. Please do not feel
obliged to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: Bad Fcst Value
From: Tsu, Mr. Justin
Time: Thu Jan 16 12:51:17 2020
Thanks Julie,
The files are located here:
/incoming/irap/met_help/test
-----Original Message-----
From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
Sent: Thursday, January 16, 2020 11:42 AM
To: Tsu, Mr. Justin
Subject: Re: [rt.rap.ucar.edu #93808] Bad Fcst Value
Hi Justin.
Are you able to send me the config file you are using, along with the
sfc_2019072100.nc file, so that I can try to run it myself and figure
out
what is going on?
There are ftp directions under "How To Send Us Data" on this page:
https://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk
Thanks!
Julie
On Thu, Jan 16, 2020 at 12:05 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93808 >
>
> Thanks Julie,
>
> Here is my command line input and output:
>
> [tsu at lorenz 02_POINT_STAT_WORK]$ !point_stat
> point_stat PYTHON_NUMPY sfc_2019072100.nc testConfig -v 3
> DEBUG 1: Default Config File:
> /software/depot/met-8.1a/met-
8.1a/share/met//config/PointStatConfig_default
> DEBUG 1: User Config File: testConfig
> ERROR :
> ERROR : VarInfo::set_level_info_grib() - failed to parse level
string
> '(*)'
> ERROR :
>
> Sfc_2019072100.nc is a netcdf file. I've attached the header below:
> ncdump -h sfc_2019072100.nc
> netcdf sfc_2019072100 {
> dimensions:
> mxstr = 16 ;
> mxstr2 = 40 ;
> mxstr3 = 80 ;
> nobs = 18034 ;
> nhdr = 18034 ;
> nhdr_typ = 1 ;
> nhdr_sid = 3316 ;
> nhdr_vld = 1 ;
> nobs_qty = 1 ;
> obs_var_num = 6 ;
> variables:
> int hdr_typ(nhdr) ;
> hdr_typ:long_name = "index of message type" ;
> hdr_typ:_FillValue = -9999 ;
> int hdr_sid(nhdr) ;
> hdr_sid:long_name = "index of station
identification" ;
> hdr_sid:_FillValue = -9999 ;
> int hdr_vld(nhdr) ;
> hdr_vld:long_name = "index of valid time" ;
> hdr_vld:_FillValue = -9999 ;
> float hdr_lat(nhdr) ;
> hdr_lat:long_name = "latitude" ;
> hdr_lat:_FillValue = -9999.f ;
> hdr_lat:units = "degrees_north" ;
> float hdr_lon(nhdr) ;
> hdr_lon:long_name = "longitude" ;
> hdr_lon:_FillValue = -9999.f ;
> hdr_lon:units = "degrees_east" ;
> float hdr_elv(nhdr) ;
> hdr_elv:long_name = "elevation" ;
> hdr_elv:_FillValue = -9999.f ;
> hdr_elv:units = "meters above sea level (msl)" ;
> int obs_qty(nobs) ;
> obs_qty:long_name = "index of quality flag" ;
> int obs_hid(nobs) ;
> obs_hid:long_name = "index of matching header data"
;
> obs_hid:_FillValue = -9999 ;
> int obs_vid(nobs) ;
> obs_vid:long_name = "index of variable names at
var_name" ;
> obs_vid:_FillValue = -9999 ;
> float obs_lvl(nobs) ;
> obs_lvl:long_name = "pressure level (hPa) or
accumulation
> interval (sec)" ;
> obs_lvl:_FillValue = -9999.f ;
> float obs_hgt(nobs) ;
> obs_hgt:long_name = "height in meters above sea
level or
> ground level (msl or agl)" ;
> obs_hgt:_FillValue = -9999.f ;
> float obs_val(nobs) ;
> obs_val:long_name = "observation value" ;
> obs_val:_FillValue = -9999.f ;
> char hdr_typ_table(nhdr_typ, mxstr2) ;
> hdr_typ_table:long_name = "message type" ;
> char hdr_sid_table(nhdr_sid, mxstr2) ;
> hdr_sid_table:long_name = "station identification" ;
> char hdr_vld_table(nhdr_vld, mxstr) ;
> hdr_vld_table:long_name = "valid time" ;
> hdr_vld_table:units = "YYYYMMDD_HHMMSS UTC" ;
> char obs_qty_table(nobs_qty, mxstr) ;
> obs_qty_table:long_name = "quality flag" ;
> char obs_var(obs_var_num, mxstr2) ;
> obs_var:long_name = "variable names" ;
>
> // global attributes:
> :MET_Obs_version = "1.02" ;
> :use_var_id = "true" ;
> :FileOrigins = "File / generated 20200116_002940 UTC
on
> host
> lorenz.nrlmry.navy.mil by the MET ascii2nc tool" ;
> :MET_version = "V8.1" ;
> :MET_tool = "ascii2nc" ;
> }
>
> Hope you can help me out!
>
> Justin
>
>
> -----Original Message-----
> From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> Sent: Thursday, January 16, 2020 8:46 AM
> To: Tsu, Mr. Justin
> Subject: Re: [rt.rap.ucar.edu #93808] Bad Fcst Value
>
> Hi Justin.
>
> I'm glad you were able to get the other issue figured out.
>
> Regarding, setting the level to "(*,*)". Please confirm that you
have a
> NetCDF file, as that syntax is for NetCDF files only. Also, I
believe the
> two asterisks are for two dimensional fields, so if you had a three
> dimensional field, for example, you would use "(*,*,*)". If you
still
> experience an error, please copy and paste the output, including the
error,
> into an email so we can better see what is happening. Thanks!
>
> Julie
>
> On Wed, Jan 15, 2020 at 5:54 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93808 >
> >
> > Thanks Julie,
> >
> > I figured out the issue. I do have another question which you may
be
> able
> > to
> > help me. From the documentation, it appears that you can set the
level
> to
> > be
> > "(*,*)" in the obs dictionary and that should consider any ob at
any
> > vertical
> > level for verification . When I set level to be "(*,*)", I get a
failed
> > parsing in point_stat. How do I tell point_stat to ignore the
vertical
> > level
> > of my ob?
> >
> > Justin
> >
> > -----Original Message-----
> > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > Sent: Wednesday, January 15, 2020 11:52 AM
> > To: Tsu, Mr. Justin
> > Subject: Re: [rt.rap.ucar.edu #93808] Bad Fcst Value
> >
> > Hi Justin.
> >
> > I see that some of your data is being rejected due to a "bad fcst
value".
> > John is likely best suited to answer your question, unfortunately
he is
> out
> > of the office at a conference. I have assigned this ticket to
him.
> Please
> > allow a few business days for a response from him. In the
meantime, you
> > could try running the plot_data_plane tool on the data to see if
that
> shows
> > any obvious problems.
> >
> > Julie
> >
> > --
> > Julie Prestopnik
> > Software Engineer
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > Phone: 303.497.8399
> > Email: jpresto at ucar.edu
> >
> > My working day may not be your working day. Please do not feel
obliged
> to
> > reply to this email outside of your normal working hours.
> >
> >
> >
>
> --
> Julie Prestopnik
> Software Engineer
> National Center for Atmospheric Research
> Research Applications Laboratory
> Phone: 303.497.8399
> Email: jpresto at ucar.edu
>
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>
>
--
Julie Prestopnik
Software Engineer
National Center for Atmospheric Research
Research Applications Laboratory
Phone: 303.497.8399
Email: jpresto at ucar.edu
My working day may not be your working day. Please do not feel
obliged to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: Bad Fcst Value
From: Julie Prestopnik
Time: Fri Jan 17 09:01:11 2020
Hi Justin.
Thank you. Could you also please send up the following files?
/users/tsu/MET/work/PY_SRC/read_NRL_binary.py
/DATA/tsu/repos/flds/uuwind_zht_000010_000000_1a0127x0127_2019072100_00000000_fcstfld
Thanks!
Julie
On Thu, Jan 16, 2020 at 12:51 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93808 >
>
> Thanks Julie,
>
> The files are located here:
>
> /incoming/irap/met_help/test
>
> -----Original Message-----
> From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> Sent: Thursday, January 16, 2020 11:42 AM
> To: Tsu, Mr. Justin
> Subject: Re: [rt.rap.ucar.edu #93808] Bad Fcst Value
>
> Hi Justin.
>
> Are you able to send me the config file you are using, along with
the
> sfc_2019072100.nc file, so that I can try to run it myself and
figure out
> what is going on?
>
> There are ftp directions under "How To Send Us Data" on this page:
>
> https://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk
>
> Thanks!
>
> Julie
>
> On Thu, Jan 16, 2020 at 12:05 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu
> >
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93808 >
> >
> > Thanks Julie,
> >
> > Here is my command line input and output:
> >
> > [tsu at lorenz 02_POINT_STAT_WORK]$ !point_stat
> > point_stat PYTHON_NUMPY sfc_2019072100.nc testConfig -v 3
> > DEBUG 1: Default Config File:
> >
> /software/depot/met-8.1a/met-
8.1a/share/met//config/PointStatConfig_default
> > DEBUG 1: User Config File: testConfig
> > ERROR :
> > ERROR : VarInfo::set_level_info_grib() - failed to parse level
string
> > '(*)'
> > ERROR :
> >
> > Sfc_2019072100.nc is a netcdf file. I've attached the header
below:
> > ncdump -h sfc_2019072100.nc
> > netcdf sfc_2019072100 {
> > dimensions:
> > mxstr = 16 ;
> > mxstr2 = 40 ;
> > mxstr3 = 80 ;
> > nobs = 18034 ;
> > nhdr = 18034 ;
> > nhdr_typ = 1 ;
> > nhdr_sid = 3316 ;
> > nhdr_vld = 1 ;
> > nobs_qty = 1 ;
> > obs_var_num = 6 ;
> > variables:
> > int hdr_typ(nhdr) ;
> > hdr_typ:long_name = "index of message type" ;
> > hdr_typ:_FillValue = -9999 ;
> > int hdr_sid(nhdr) ;
> > hdr_sid:long_name = "index of station
identification" ;
> > hdr_sid:_FillValue = -9999 ;
> > int hdr_vld(nhdr) ;
> > hdr_vld:long_name = "index of valid time" ;
> > hdr_vld:_FillValue = -9999 ;
> > float hdr_lat(nhdr) ;
> > hdr_lat:long_name = "latitude" ;
> > hdr_lat:_FillValue = -9999.f ;
> > hdr_lat:units = "degrees_north" ;
> > float hdr_lon(nhdr) ;
> > hdr_lon:long_name = "longitude" ;
> > hdr_lon:_FillValue = -9999.f ;
> > hdr_lon:units = "degrees_east" ;
> > float hdr_elv(nhdr) ;
> > hdr_elv:long_name = "elevation" ;
> > hdr_elv:_FillValue = -9999.f ;
> > hdr_elv:units = "meters above sea level (msl)" ;
> > int obs_qty(nobs) ;
> > obs_qty:long_name = "index of quality flag" ;
> > int obs_hid(nobs) ;
> > obs_hid:long_name = "index of matching header
data" ;
> > obs_hid:_FillValue = -9999 ;
> > int obs_vid(nobs) ;
> > obs_vid:long_name = "index of variable names at
> var_name" ;
> > obs_vid:_FillValue = -9999 ;
> > float obs_lvl(nobs) ;
> > obs_lvl:long_name = "pressure level (hPa) or
accumulation
> > interval (sec)" ;
> > obs_lvl:_FillValue = -9999.f ;
> > float obs_hgt(nobs) ;
> > obs_hgt:long_name = "height in meters above sea
level or
> > ground level (msl or agl)" ;
> > obs_hgt:_FillValue = -9999.f ;
> > float obs_val(nobs) ;
> > obs_val:long_name = "observation value" ;
> > obs_val:_FillValue = -9999.f ;
> > char hdr_typ_table(nhdr_typ, mxstr2) ;
> > hdr_typ_table:long_name = "message type" ;
> > char hdr_sid_table(nhdr_sid, mxstr2) ;
> > hdr_sid_table:long_name = "station identification"
;
> > char hdr_vld_table(nhdr_vld, mxstr) ;
> > hdr_vld_table:long_name = "valid time" ;
> > hdr_vld_table:units = "YYYYMMDD_HHMMSS UTC" ;
> > char obs_qty_table(nobs_qty, mxstr) ;
> > obs_qty_table:long_name = "quality flag" ;
> > char obs_var(obs_var_num, mxstr2) ;
> > obs_var:long_name = "variable names" ;
> >
> > // global attributes:
> > :MET_Obs_version = "1.02" ;
> > :use_var_id = "true" ;
> > :FileOrigins = "File / generated 20200116_002940
UTC on
> > host
> > lorenz.nrlmry.navy.mil by the MET ascii2nc tool" ;
> > :MET_version = "V8.1" ;
> > :MET_tool = "ascii2nc" ;
> > }
> >
> > Hope you can help me out!
> >
> > Justin
> >
> >
> > -----Original Message-----
> > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > Sent: Thursday, January 16, 2020 8:46 AM
> > To: Tsu, Mr. Justin
> > Subject: Re: [rt.rap.ucar.edu #93808] Bad Fcst Value
> >
> > Hi Justin.
> >
> > I'm glad you were able to get the other issue figured out.
> >
> > Regarding, setting the level to "(*,*)". Please confirm that you
have a
> > NetCDF file, as that syntax is for NetCDF files only. Also, I
believe
> the
> > two asterisks are for two dimensional fields, so if you had a
three
> > dimensional field, for example, you would use "(*,*,*)". If you
still
> > experience an error, please copy and paste the output, including
the
> error,
> > into an email so we can better see what is happening. Thanks!
> >
> > Julie
> >
> > On Wed, Jan 15, 2020 at 5:54 PM Tsu, Mr. Justin via RT <
> met_help at ucar.edu>
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93808 >
> > >
> > > Thanks Julie,
> > >
> > > I figured out the issue. I do have another question which you
may be
> > able
> > > to
> > > help me. From the documentation, it appears that you can set
the level
> > to
> > > be
> > > "(*,*)" in the obs dictionary and that should consider any ob at
any
> > > vertical
> > > level for verification . When I set level to be "(*,*)", I get
a
> failed
> > > parsing in point_stat. How do I tell point_stat to ignore the
vertical
> > > level
> > > of my ob?
> > >
> > > Justin
> > >
> > > -----Original Message-----
> > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > Sent: Wednesday, January 15, 2020 11:52 AM
> > > To: Tsu, Mr. Justin
> > > Subject: Re: [rt.rap.ucar.edu #93808] Bad Fcst Value
> > >
> > > Hi Justin.
> > >
> > > I see that some of your data is being rejected due to a "bad
fcst
> value".
> > > John is likely best suited to answer your question,
unfortunately he is
> > out
> > > of the office at a conference. I have assigned this ticket to
him.
> > Please
> > > allow a few business days for a response from him. In the
meantime,
> you
> > > could try running the plot_data_plane tool on the data to see if
that
> > shows
> > > any obvious problems.
> > >
> > > Julie
> > >
> > > --
> > > Julie Prestopnik
> > > Software Engineer
> > > National Center for Atmospheric Research
> > > Research Applications Laboratory
> > > Phone: 303.497.8399
> > > Email: jpresto at ucar.edu
> > >
> > > My working day may not be your working day. Please do not feel
obliged
> > to
> > > reply to this email outside of your normal working hours.
> > >
> > >
> > >
> >
> > --
> > Julie Prestopnik
> > Software Engineer
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > Phone: 303.497.8399
> > Email: jpresto at ucar.edu
> >
> > My working day may not be your working day. Please do not feel
obliged
> to
> > reply to this email outside of your normal working hours.
> >
> >
> >
>
> --
> Julie Prestopnik
> Software Engineer
> National Center for Atmospheric Research
> Research Applications Laboratory
> Phone: 303.497.8399
> Email: jpresto at ucar.edu
>
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>
>
--
Julie Prestopnik
Software Engineer
National Center for Atmospheric Research
Research Applications Laboratory
Phone: 303.497.8399
Email: jpresto at ucar.edu
My working day may not be your working day. Please do not feel
obliged to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: Bad Fcst Value
From: Tsu, Mr. Justin
Time: Fri Jan 17 10:51:45 2020
Julie,
I ended up figuring it out. Thank you for the help!
Justin
-----Original Message-----
From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
Sent: Friday, January 17, 2020 8:01 AM
To: Tsu, Mr. Justin
Subject: Re: [rt.rap.ucar.edu #93808] Bad Fcst Value
Hi Justin.
Thank you. Could you also please send up the following files?
/users/tsu/MET/work/PY_SRC/read_NRL_binary.py
/DATA/tsu/repos/flds/uuwind_zht_000010_000000_1a0127x0127_2019072100_00000000_fcstfld
Thanks!
Julie
On Thu, Jan 16, 2020 at 12:51 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93808 >
>
> Thanks Julie,
>
> The files are located here:
>
> /incoming/irap/met_help/test
>
> -----Original Message-----
> From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> Sent: Thursday, January 16, 2020 11:42 AM
> To: Tsu, Mr. Justin
> Subject: Re: [rt.rap.ucar.edu #93808] Bad Fcst Value
>
> Hi Justin.
>
> Are you able to send me the config file you are using, along with
the
> sfc_2019072100.nc file, so that I can try to run it myself and
figure out
> what is going on?
>
> There are ftp directions under "How To Send Us Data" on this page:
>
> https://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk
>
> Thanks!
>
> Julie
>
> On Thu, Jan 16, 2020 at 12:05 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu
> >
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93808 >
> >
> > Thanks Julie,
> >
> > Here is my command line input and output:
> >
> > [tsu at lorenz 02_POINT_STAT_WORK]$ !point_stat
> > point_stat PYTHON_NUMPY sfc_2019072100.nc testConfig -v 3
> > DEBUG 1: Default Config File:
> >
> /software/depot/met-8.1a/met-
8.1a/share/met//config/PointStatConfig_default
> > DEBUG 1: User Config File: testConfig
> > ERROR :
> > ERROR : VarInfo::set_level_info_grib() - failed to parse level
string
> > '(*)'
> > ERROR :
> >
> > Sfc_2019072100.nc is a netcdf file. I've attached the header
below:
> > ncdump -h sfc_2019072100.nc
> > netcdf sfc_2019072100 {
> > dimensions:
> > mxstr = 16 ;
> > mxstr2 = 40 ;
> > mxstr3 = 80 ;
> > nobs = 18034 ;
> > nhdr = 18034 ;
> > nhdr_typ = 1 ;
> > nhdr_sid = 3316 ;
> > nhdr_vld = 1 ;
> > nobs_qty = 1 ;
> > obs_var_num = 6 ;
> > variables:
> > int hdr_typ(nhdr) ;
> > hdr_typ:long_name = "index of message type" ;
> > hdr_typ:_FillValue = -9999 ;
> > int hdr_sid(nhdr) ;
> > hdr_sid:long_name = "index of station
identification" ;
> > hdr_sid:_FillValue = -9999 ;
> > int hdr_vld(nhdr) ;
> > hdr_vld:long_name = "index of valid time" ;
> > hdr_vld:_FillValue = -9999 ;
> > float hdr_lat(nhdr) ;
> > hdr_lat:long_name = "latitude" ;
> > hdr_lat:_FillValue = -9999.f ;
> > hdr_lat:units = "degrees_north" ;
> > float hdr_lon(nhdr) ;
> > hdr_lon:long_name = "longitude" ;
> > hdr_lon:_FillValue = -9999.f ;
> > hdr_lon:units = "degrees_east" ;
> > float hdr_elv(nhdr) ;
> > hdr_elv:long_name = "elevation" ;
> > hdr_elv:_FillValue = -9999.f ;
> > hdr_elv:units = "meters above sea level (msl)" ;
> > int obs_qty(nobs) ;
> > obs_qty:long_name = "index of quality flag" ;
> > int obs_hid(nobs) ;
> > obs_hid:long_name = "index of matching header
data" ;
> > obs_hid:_FillValue = -9999 ;
> > int obs_vid(nobs) ;
> > obs_vid:long_name = "index of variable names at
> var_name" ;
> > obs_vid:_FillValue = -9999 ;
> > float obs_lvl(nobs) ;
> > obs_lvl:long_name = "pressure level (hPa) or
accumulation
> > interval (sec)" ;
> > obs_lvl:_FillValue = -9999.f ;
> > float obs_hgt(nobs) ;
> > obs_hgt:long_name = "height in meters above sea
level or
> > ground level (msl or agl)" ;
> > obs_hgt:_FillValue = -9999.f ;
> > float obs_val(nobs) ;
> > obs_val:long_name = "observation value" ;
> > obs_val:_FillValue = -9999.f ;
> > char hdr_typ_table(nhdr_typ, mxstr2) ;
> > hdr_typ_table:long_name = "message type" ;
> > char hdr_sid_table(nhdr_sid, mxstr2) ;
> > hdr_sid_table:long_name = "station identification"
;
> > char hdr_vld_table(nhdr_vld, mxstr) ;
> > hdr_vld_table:long_name = "valid time" ;
> > hdr_vld_table:units = "YYYYMMDD_HHMMSS UTC" ;
> > char obs_qty_table(nobs_qty, mxstr) ;
> > obs_qty_table:long_name = "quality flag" ;
> > char obs_var(obs_var_num, mxstr2) ;
> > obs_var:long_name = "variable names" ;
> >
> > // global attributes:
> > :MET_Obs_version = "1.02" ;
> > :use_var_id = "true" ;
> > :FileOrigins = "File / generated 20200116_002940
UTC on
> > host
> > lorenz.nrlmry.navy.mil by the MET ascii2nc tool" ;
> > :MET_version = "V8.1" ;
> > :MET_tool = "ascii2nc" ;
> > }
> >
> > Hope you can help me out!
> >
> > Justin
> >
> >
> > -----Original Message-----
> > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > Sent: Thursday, January 16, 2020 8:46 AM
> > To: Tsu, Mr. Justin
> > Subject: Re: [rt.rap.ucar.edu #93808] Bad Fcst Value
> >
> > Hi Justin.
> >
> > I'm glad you were able to get the other issue figured out.
> >
> > Regarding, setting the level to "(*,*)". Please confirm that you
have a
> > NetCDF file, as that syntax is for NetCDF files only. Also, I
believe
> the
> > two asterisks are for two dimensional fields, so if you had a
three
> > dimensional field, for example, you would use "(*,*,*)". If you
still
> > experience an error, please copy and paste the output, including
the
> error,
> > into an email so we can better see what is happening. Thanks!
> >
> > Julie
> >
> > On Wed, Jan 15, 2020 at 5:54 PM Tsu, Mr. Justin via RT <
> met_help at ucar.edu>
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93808 >
> > >
> > > Thanks Julie,
> > >
> > > I figured out the issue. I do have another question which you
may be
> > able
> > > to
> > > help me. From the documentation, it appears that you can set
the level
> > to
> > > be
> > > "(*,*)" in the obs dictionary and that should consider any ob at
any
> > > vertical
> > > level for verification . When I set level to be "(*,*)", I get
a
> failed
> > > parsing in point_stat. How do I tell point_stat to ignore the
vertical
> > > level
> > > of my ob?
> > >
> > > Justin
> > >
> > > -----Original Message-----
> > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > Sent: Wednesday, January 15, 2020 11:52 AM
> > > To: Tsu, Mr. Justin
> > > Subject: Re: [rt.rap.ucar.edu #93808] Bad Fcst Value
> > >
> > > Hi Justin.
> > >
> > > I see that some of your data is being rejected due to a "bad
fcst
> value".
> > > John is likely best suited to answer your question,
unfortunately he is
> > out
> > > of the office at a conference. I have assigned this ticket to
him.
> > Please
> > > allow a few business days for a response from him. In the
meantime,
> you
> > > could try running the plot_data_plane tool on the data to see if
that
> > shows
> > > any obvious problems.
> > >
> > > Julie
> > >
> > > --
> > > Julie Prestopnik
> > > Software Engineer
> > > National Center for Atmospheric Research
> > > Research Applications Laboratory
> > > Phone: 303.497.8399
> > > Email: jpresto at ucar.edu
> > >
> > > My working day may not be your working day. Please do not feel
obliged
> > to
> > > reply to this email outside of your normal working hours.
> > >
> > >
> > >
> >
> > --
> > Julie Prestopnik
> > Software Engineer
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > Phone: 303.497.8399
> > Email: jpresto at ucar.edu
> >
> > My working day may not be your working day. Please do not feel
obliged
> to
> > reply to this email outside of your normal working hours.
> >
> >
> >
>
> --
> Julie Prestopnik
> Software Engineer
> National Center for Atmospheric Research
> Research Applications Laboratory
> Phone: 303.497.8399
> Email: jpresto at ucar.edu
>
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>
>
--
Julie Prestopnik
Software Engineer
National Center for Atmospheric Research
Research Applications Laboratory
Phone: 303.497.8399
Email: jpresto at ucar.edu
My working day may not be your working day. Please do not feel
obliged to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: Bad Fcst Value
From: Julie Prestopnik
Time: Fri Jan 17 11:03:58 2020
Thank you for letting us know that you figured it out. I discovered
later
that you have point NetCDF data output from ascii2nc, whereas the
'(*,*)'
syntax is for dealing with gridded NetCDF files.
Can you please tell us what part of the user's guide you found
confusing so
that we can fix it for the next release? Thanks for any help you can
give
us in making our documentation more clear.
Julie
On Fri, Jan 17, 2020 at 10:52 AM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93808 >
>
> Julie,
>
> I ended up figuring it out. Thank you for the help!
>
> Justin
>
> -----Original Message-----
> From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> Sent: Friday, January 17, 2020 8:01 AM
> To: Tsu, Mr. Justin
> Subject: Re: [rt.rap.ucar.edu #93808] Bad Fcst Value
>
> Hi Justin.
>
> Thank you. Could you also please send up the following files?
>
> /users/tsu/MET/work/PY_SRC/read_NRL_binary.py
>
>
/DATA/tsu/repos/flds/uuwind_zht_000010_000000_1a0127x0127_2019072100_00000000_fcstfld
>
> Thanks!
>
> Julie
>
> On Thu, Jan 16, 2020 at 12:51 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu
> >
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93808 >
> >
> > Thanks Julie,
> >
> > The files are located here:
> >
> > /incoming/irap/met_help/test
> >
> > -----Original Message-----
> > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > Sent: Thursday, January 16, 2020 11:42 AM
> > To: Tsu, Mr. Justin
> > Subject: Re: [rt.rap.ucar.edu #93808] Bad Fcst Value
> >
> > Hi Justin.
> >
> > Are you able to send me the config file you are using, along with
the
> > sfc_2019072100.nc file, so that I can try to run it myself and
figure
> out
> > what is going on?
> >
> > There are ftp directions under "How To Send Us Data" on this page:
> >
> >
> https://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk
> >
> > Thanks!
> >
> > Julie
> >
> > On Thu, Jan 16, 2020 at 12:05 PM Tsu, Mr. Justin via RT <
> met_help at ucar.edu
> > >
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93808 >
> > >
> > > Thanks Julie,
> > >
> > > Here is my command line input and output:
> > >
> > > [tsu at lorenz 02_POINT_STAT_WORK]$ !point_stat
> > > point_stat PYTHON_NUMPY sfc_2019072100.nc testConfig -v 3
> > > DEBUG 1: Default Config File:
> > >
> >
> /software/depot/met-8.1a/met-
8.1a/share/met//config/PointStatConfig_default
> > > DEBUG 1: User Config File: testConfig
> > > ERROR :
> > > ERROR : VarInfo::set_level_info_grib() - failed to parse level
string
> > > '(*)'
> > > ERROR :
> > >
> > > Sfc_2019072100.nc is a netcdf file. I've attached the header
below:
> > > ncdump -h sfc_2019072100.nc
> > > netcdf sfc_2019072100 {
> > > dimensions:
> > > mxstr = 16 ;
> > > mxstr2 = 40 ;
> > > mxstr3 = 80 ;
> > > nobs = 18034 ;
> > > nhdr = 18034 ;
> > > nhdr_typ = 1 ;
> > > nhdr_sid = 3316 ;
> > > nhdr_vld = 1 ;
> > > nobs_qty = 1 ;
> > > obs_var_num = 6 ;
> > > variables:
> > > int hdr_typ(nhdr) ;
> > > hdr_typ:long_name = "index of message type" ;
> > > hdr_typ:_FillValue = -9999 ;
> > > int hdr_sid(nhdr) ;
> > > hdr_sid:long_name = "index of station
identification" ;
> > > hdr_sid:_FillValue = -9999 ;
> > > int hdr_vld(nhdr) ;
> > > hdr_vld:long_name = "index of valid time" ;
> > > hdr_vld:_FillValue = -9999 ;
> > > float hdr_lat(nhdr) ;
> > > hdr_lat:long_name = "latitude" ;
> > > hdr_lat:_FillValue = -9999.f ;
> > > hdr_lat:units = "degrees_north" ;
> > > float hdr_lon(nhdr) ;
> > > hdr_lon:long_name = "longitude" ;
> > > hdr_lon:_FillValue = -9999.f ;
> > > hdr_lon:units = "degrees_east" ;
> > > float hdr_elv(nhdr) ;
> > > hdr_elv:long_name = "elevation" ;
> > > hdr_elv:_FillValue = -9999.f ;
> > > hdr_elv:units = "meters above sea level (msl)" ;
> > > int obs_qty(nobs) ;
> > > obs_qty:long_name = "index of quality flag" ;
> > > int obs_hid(nobs) ;
> > > obs_hid:long_name = "index of matching header
data" ;
> > > obs_hid:_FillValue = -9999 ;
> > > int obs_vid(nobs) ;
> > > obs_vid:long_name = "index of variable names at
> > var_name" ;
> > > obs_vid:_FillValue = -9999 ;
> > > float obs_lvl(nobs) ;
> > > obs_lvl:long_name = "pressure level (hPa) or
> accumulation
> > > interval (sec)" ;
> > > obs_lvl:_FillValue = -9999.f ;
> > > float obs_hgt(nobs) ;
> > > obs_hgt:long_name = "height in meters above sea
level
> or
> > > ground level (msl or agl)" ;
> > > obs_hgt:_FillValue = -9999.f ;
> > > float obs_val(nobs) ;
> > > obs_val:long_name = "observation value" ;
> > > obs_val:_FillValue = -9999.f ;
> > > char hdr_typ_table(nhdr_typ, mxstr2) ;
> > > hdr_typ_table:long_name = "message type" ;
> > > char hdr_sid_table(nhdr_sid, mxstr2) ;
> > > hdr_sid_table:long_name = "station
identification" ;
> > > char hdr_vld_table(nhdr_vld, mxstr) ;
> > > hdr_vld_table:long_name = "valid time" ;
> > > hdr_vld_table:units = "YYYYMMDD_HHMMSS UTC" ;
> > > char obs_qty_table(nobs_qty, mxstr) ;
> > > obs_qty_table:long_name = "quality flag" ;
> > > char obs_var(obs_var_num, mxstr2) ;
> > > obs_var:long_name = "variable names" ;
> > >
> > > // global attributes:
> > > :MET_Obs_version = "1.02" ;
> > > :use_var_id = "true" ;
> > > :FileOrigins = "File / generated 20200116_002940
UTC on
> > > host
> > > lorenz.nrlmry.navy.mil by the MET ascii2nc tool" ;
> > > :MET_version = "V8.1" ;
> > > :MET_tool = "ascii2nc" ;
> > > }
> > >
> > > Hope you can help me out!
> > >
> > > Justin
> > >
> > >
> > > -----Original Message-----
> > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > Sent: Thursday, January 16, 2020 8:46 AM
> > > To: Tsu, Mr. Justin
> > > Subject: Re: [rt.rap.ucar.edu #93808] Bad Fcst Value
> > >
> > > Hi Justin.
> > >
> > > I'm glad you were able to get the other issue figured out.
> > >
> > > Regarding, setting the level to "(*,*)". Please confirm that
you have
> a
> > > NetCDF file, as that syntax is for NetCDF files only. Also, I
believe
> > the
> > > two asterisks are for two dimensional fields, so if you had a
three
> > > dimensional field, for example, you would use "(*,*,*)". If you
still
> > > experience an error, please copy and paste the output, including
the
> > error,
> > > into an email so we can better see what is happening. Thanks!
> > >
> > > Julie
> > >
> > > On Wed, Jan 15, 2020 at 5:54 PM Tsu, Mr. Justin via RT <
> > met_help at ucar.edu>
> > > wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93808
>
> > > >
> > > > Thanks Julie,
> > > >
> > > > I figured out the issue. I do have another question which you
may be
> > > able
> > > > to
> > > > help me. From the documentation, it appears that you can set
the
> level
> > > to
> > > > be
> > > > "(*,*)" in the obs dictionary and that should consider any ob
at any
> > > > vertical
> > > > level for verification . When I set level to be "(*,*)", I
get a
> > failed
> > > > parsing in point_stat. How do I tell point_stat to ignore the
> vertical
> > > > level
> > > > of my ob?
> > > >
> > > > Justin
> > > >
> > > > -----Original Message-----
> > > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > > Sent: Wednesday, January 15, 2020 11:52 AM
> > > > To: Tsu, Mr. Justin
> > > > Subject: Re: [rt.rap.ucar.edu #93808] Bad Fcst Value
> > > >
> > > > Hi Justin.
> > > >
> > > > I see that some of your data is being rejected due to a "bad
fcst
> > value".
> > > > John is likely best suited to answer your question,
unfortunately he
> is
> > > out
> > > > of the office at a conference. I have assigned this ticket to
him.
> > > Please
> > > > allow a few business days for a response from him. In the
meantime,
> > you
> > > > could try running the plot_data_plane tool on the data to see
if that
> > > shows
> > > > any obvious problems.
> > > >
> > > > Julie
> > > >
> > > > --
> > > > Julie Prestopnik
> > > > Software Engineer
> > > > National Center for Atmospheric Research
> > > > Research Applications Laboratory
> > > > Phone: 303.497.8399
> > > > Email: jpresto at ucar.edu
> > > >
> > > > My working day may not be your working day. Please do not
feel
> obliged
> > > to
> > > > reply to this email outside of your normal working hours.
> > > >
> > > >
> > > >
> > >
> > > --
> > > Julie Prestopnik
> > > Software Engineer
> > > National Center for Atmospheric Research
> > > Research Applications Laboratory
> > > Phone: 303.497.8399
> > > Email: jpresto at ucar.edu
> > >
> > > My working day may not be your working day. Please do not feel
obliged
> > to
> > > reply to this email outside of your normal working hours.
> > >
> > >
> > >
> >
> > --
> > Julie Prestopnik
> > Software Engineer
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > Phone: 303.497.8399
> > Email: jpresto at ucar.edu
> >
> > My working day may not be your working day. Please do not feel
obliged
> to
> > reply to this email outside of your normal working hours.
> >
> >
> >
>
> --
> Julie Prestopnik
> Software Engineer
> National Center for Atmospheric Research
> Research Applications Laboratory
> Phone: 303.497.8399
> Email: jpresto at ucar.edu
>
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>
>
--
Julie Prestopnik
Software Engineer
National Center for Atmospheric Research
Research Applications Laboratory
Phone: 303.497.8399
Email: jpresto at ucar.edu
My working day may not be your working day. Please do not feel
obliged to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: Bad Fcst Value
From: Tsu, Mr. Justin
Time: Fri Jan 17 11:38:14 2020
Perhaps including in the documentation when to use the syntax '(*,*)'
as
you've just stated? The information that I used was on page 63:
// - Simplified vertical level matching logic is applied for surface
message
// types. Observations for the following message types are assumed to
be at
// the surface, as defined by the default message_type_group_map:
// ADPSFC, SFCSHP, MSONET
// field = [
// {
// message_type = [ "ADPUPA" ];
// sid_exc = [];
// name = "TMP";
// level = [ "P250", "P500", "P700", "P850", "P1000" ];
// cat_thresh = [ <=273.0 ];
// },
// {
// message_type = [ "ADPSFC" ];
// sid_exc = [ "KDEN", "KDET" ];
// name = "TMP";
// level = [ "Z2" ];
// cat_thresh = [ <=273.0 ];
// }
// ]
I guess I am sort of confused as to how it works when you use level =
"Z2" and
a message_type that maps to value "SURFACE". How does point_stat take
these
two values and apply them to search for the correct items in your
point output
NETCDF?
Justin
-----Original Message-----
From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
Sent: Friday, January 17, 2020 10:04 AM
To: Tsu, Mr. Justin
Subject: Re: [rt.rap.ucar.edu #93808] Bad Fcst Value
Thank you for letting us know that you figured it out. I discovered
later
that you have point NetCDF data output from ascii2nc, whereas the
'(*,*)'
syntax is for dealing with gridded NetCDF files.
Can you please tell us what part of the user's guide you found
confusing so
that we can fix it for the next release? Thanks for any help you can
give
us in making our documentation more clear.
Julie
On Fri, Jan 17, 2020 at 10:52 AM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93808 >
>
> Julie,
>
> I ended up figuring it out. Thank you for the help!
>
> Justin
>
> -----Original Message-----
> From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> Sent: Friday, January 17, 2020 8:01 AM
> To: Tsu, Mr. Justin
> Subject: Re: [rt.rap.ucar.edu #93808] Bad Fcst Value
>
> Hi Justin.
>
> Thank you. Could you also please send up the following files?
>
> /users/tsu/MET/work/PY_SRC/read_NRL_binary.py
>
>
/DATA/tsu/repos/flds/uuwind_zht_000010_000000_1a0127x0127_2019072100_00000000_fcstfld
>
> Thanks!
>
> Julie
>
> On Thu, Jan 16, 2020 at 12:51 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu
> >
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93808 >
> >
> > Thanks Julie,
> >
> > The files are located here:
> >
> > /incoming/irap/met_help/test
> >
> > -----Original Message-----
> > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > Sent: Thursday, January 16, 2020 11:42 AM
> > To: Tsu, Mr. Justin
> > Subject: Re: [rt.rap.ucar.edu #93808] Bad Fcst Value
> >
> > Hi Justin.
> >
> > Are you able to send me the config file you are using, along with
the
> > sfc_2019072100.nc file, so that I can try to run it myself and
figure
> out
> > what is going on?
> >
> > There are ftp directions under "How To Send Us Data" on this page:
> >
> >
> https://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk
> >
> > Thanks!
> >
> > Julie
> >
> > On Thu, Jan 16, 2020 at 12:05 PM Tsu, Mr. Justin via RT <
> met_help at ucar.edu
> > >
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93808 >
> > >
> > > Thanks Julie,
> > >
> > > Here is my command line input and output:
> > >
> > > [tsu at lorenz 02_POINT_STAT_WORK]$ !point_stat
> > > point_stat PYTHON_NUMPY sfc_2019072100.nc testConfig -v 3
> > > DEBUG 1: Default Config File:
> > >
> >
> /software/depot/met-8.1a/met-
8.1a/share/met//config/PointStatConfig_default
> > > DEBUG 1: User Config File: testConfig
> > > ERROR :
> > > ERROR : VarInfo::set_level_info_grib() - failed to parse level
string
> > > '(*)'
> > > ERROR :
> > >
> > > Sfc_2019072100.nc is a netcdf file. I've attached the header
below:
> > > ncdump -h sfc_2019072100.nc
> > > netcdf sfc_2019072100 {
> > > dimensions:
> > > mxstr = 16 ;
> > > mxstr2 = 40 ;
> > > mxstr3 = 80 ;
> > > nobs = 18034 ;
> > > nhdr = 18034 ;
> > > nhdr_typ = 1 ;
> > > nhdr_sid = 3316 ;
> > > nhdr_vld = 1 ;
> > > nobs_qty = 1 ;
> > > obs_var_num = 6 ;
> > > variables:
> > > int hdr_typ(nhdr) ;
> > > hdr_typ:long_name = "index of message type" ;
> > > hdr_typ:_FillValue = -9999 ;
> > > int hdr_sid(nhdr) ;
> > > hdr_sid:long_name = "index of station
identification" ;
> > > hdr_sid:_FillValue = -9999 ;
> > > int hdr_vld(nhdr) ;
> > > hdr_vld:long_name = "index of valid time" ;
> > > hdr_vld:_FillValue = -9999 ;
> > > float hdr_lat(nhdr) ;
> > > hdr_lat:long_name = "latitude" ;
> > > hdr_lat:_FillValue = -9999.f ;
> > > hdr_lat:units = "degrees_north" ;
> > > float hdr_lon(nhdr) ;
> > > hdr_lon:long_name = "longitude" ;
> > > hdr_lon:_FillValue = -9999.f ;
> > > hdr_lon:units = "degrees_east" ;
> > > float hdr_elv(nhdr) ;
> > > hdr_elv:long_name = "elevation" ;
> > > hdr_elv:_FillValue = -9999.f ;
> > > hdr_elv:units = "meters above sea level (msl)" ;
> > > int obs_qty(nobs) ;
> > > obs_qty:long_name = "index of quality flag" ;
> > > int obs_hid(nobs) ;
> > > obs_hid:long_name = "index of matching header
data" ;
> > > obs_hid:_FillValue = -9999 ;
> > > int obs_vid(nobs) ;
> > > obs_vid:long_name = "index of variable names at
> > var_name" ;
> > > obs_vid:_FillValue = -9999 ;
> > > float obs_lvl(nobs) ;
> > > obs_lvl:long_name = "pressure level (hPa) or
> accumulation
> > > interval (sec)" ;
> > > obs_lvl:_FillValue = -9999.f ;
> > > float obs_hgt(nobs) ;
> > > obs_hgt:long_name = "height in meters above sea
level
> or
> > > ground level (msl or agl)" ;
> > > obs_hgt:_FillValue = -9999.f ;
> > > float obs_val(nobs) ;
> > > obs_val:long_name = "observation value" ;
> > > obs_val:_FillValue = -9999.f ;
> > > char hdr_typ_table(nhdr_typ, mxstr2) ;
> > > hdr_typ_table:long_name = "message type" ;
> > > char hdr_sid_table(nhdr_sid, mxstr2) ;
> > > hdr_sid_table:long_name = "station
identification" ;
> > > char hdr_vld_table(nhdr_vld, mxstr) ;
> > > hdr_vld_table:long_name = "valid time" ;
> > > hdr_vld_table:units = "YYYYMMDD_HHMMSS UTC" ;
> > > char obs_qty_table(nobs_qty, mxstr) ;
> > > obs_qty_table:long_name = "quality flag" ;
> > > char obs_var(obs_var_num, mxstr2) ;
> > > obs_var:long_name = "variable names" ;
> > >
> > > // global attributes:
> > > :MET_Obs_version = "1.02" ;
> > > :use_var_id = "true" ;
> > > :FileOrigins = "File / generated 20200116_002940
UTC on
> > > host
> > > lorenz.nrlmry.navy.mil by the MET ascii2nc tool" ;
> > > :MET_version = "V8.1" ;
> > > :MET_tool = "ascii2nc" ;
> > > }
> > >
> > > Hope you can help me out!
> > >
> > > Justin
> > >
> > >
> > > -----Original Message-----
> > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > Sent: Thursday, January 16, 2020 8:46 AM
> > > To: Tsu, Mr. Justin
> > > Subject: Re: [rt.rap.ucar.edu #93808] Bad Fcst Value
> > >
> > > Hi Justin.
> > >
> > > I'm glad you were able to get the other issue figured out.
> > >
> > > Regarding, setting the level to "(*,*)". Please confirm that
you have
> a
> > > NetCDF file, as that syntax is for NetCDF files only. Also, I
believe
> > the
> > > two asterisks are for two dimensional fields, so if you had a
three
> > > dimensional field, for example, you would use "(*,*,*)". If you
still
> > > experience an error, please copy and paste the output, including
the
> > error,
> > > into an email so we can better see what is happening. Thanks!
> > >
> > > Julie
> > >
> > > On Wed, Jan 15, 2020 at 5:54 PM Tsu, Mr. Justin via RT <
> > met_help at ucar.edu>
> > > wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93808
>
> > > >
> > > > Thanks Julie,
> > > >
> > > > I figured out the issue. I do have another question which you
may be
> > > able
> > > > to
> > > > help me. From the documentation, it appears that you can set
the
> level
> > > to
> > > > be
> > > > "(*,*)" in the obs dictionary and that should consider any ob
at any
> > > > vertical
> > > > level for verification . When I set level to be "(*,*)", I
get a
> > failed
> > > > parsing in point_stat. How do I tell point_stat to ignore the
> vertical
> > > > level
> > > > of my ob?
> > > >
> > > > Justin
> > > >
> > > > -----Original Message-----
> > > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > > Sent: Wednesday, January 15, 2020 11:52 AM
> > > > To: Tsu, Mr. Justin
> > > > Subject: Re: [rt.rap.ucar.edu #93808] Bad Fcst Value
> > > >
> > > > Hi Justin.
> > > >
> > > > I see that some of your data is being rejected due to a "bad
fcst
> > value".
> > > > John is likely best suited to answer your question,
unfortunately he
> is
> > > out
> > > > of the office at a conference. I have assigned this ticket to
him.
> > > Please
> > > > allow a few business days for a response from him. In the
meantime,
> > you
> > > > could try running the plot_data_plane tool on the data to see
if that
> > > shows
> > > > any obvious problems.
> > > >
> > > > Julie
> > > >
> > > > --
> > > > Julie Prestopnik
> > > > Software Engineer
> > > > National Center for Atmospheric Research
> > > > Research Applications Laboratory
> > > > Phone: 303.497.8399
> > > > Email: jpresto at ucar.edu
> > > >
> > > > My working day may not be your working day. Please do not
feel
> obliged
> > > to
> > > > reply to this email outside of your normal working hours.
> > > >
> > > >
> > > >
> > >
> > > --
> > > Julie Prestopnik
> > > Software Engineer
> > > National Center for Atmospheric Research
> > > Research Applications Laboratory
> > > Phone: 303.497.8399
> > > Email: jpresto at ucar.edu
> > >
> > > My working day may not be your working day. Please do not feel
obliged
> > to
> > > reply to this email outside of your normal working hours.
> > >
> > >
> > >
> >
> > --
> > Julie Prestopnik
> > Software Engineer
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > Phone: 303.497.8399
> > Email: jpresto at ucar.edu
> >
> > My working day may not be your working day. Please do not feel
obliged
> to
> > reply to this email outside of your normal working hours.
> >
> >
> >
>
> --
> Julie Prestopnik
> Software Engineer
> National Center for Atmospheric Research
> Research Applications Laboratory
> Phone: 303.497.8399
> Email: jpresto at ucar.edu
>
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>
>
--
Julie Prestopnik
Software Engineer
National Center for Atmospheric Research
Research Applications Laboratory
Phone: 303.497.8399
Email: jpresto at ucar.edu
My working day may not be your working day. Please do not feel
obliged to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: Bad Fcst Value
From: John Halley Gotway
Time: Fri Jan 17 12:22:00 2020
Justin,
While verifying against point observations can sound straight-forward,
there are lots of details to consider, as you're finding!
All of the MET point pre-processing tools (ascii2nc, madis2nc, pb2nc,
and
lidar2nc) write to a common NetCDF point observation file format.
Each
point observation stored in those files is associated with a
message_type.
The Point-Stat configuration file defines which of those message types
are
assumed to exist at the surface:
message_type_group_map = [
{ key = "SURFACE"; val = "ADPSFC,SFCSHP,MSONET"; }
...
}
The logic is this... if verifying against a non-pressure level
observation
field with a "surface" message type, don't actually check that the
vertical
level value match.
For example, you are verifying a forecast of 2-m temperature against
ADPSFC
observations of temperature at 2 meters:
fcst = { field = [ { name="TMP"; level="Z2"; } ]; };
obs = { field = [ { name="TMP"; level="Z2"; message_type = "ADPSFC"; }
]; };
When we search the point observations for matches, we note the
"ADPSFC" is
a surface message type. "Z2" is a height level, not a pressure level.
So
we don't require that the observations actually have an elevation
exactly
equal to 2 meters.
In this way, when verifying against surface observations, it's the
message
type that matters, not the actual observation elevation value.
John
On Fri, Jan 17, 2020 at 11:38 AM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93808 >
>
> Perhaps including in the documentation when to use the syntax
'(*,*)' as
> you've just stated? The information that I used was on page 63:
>
> // - Simplified vertical level matching logic is applied for surface
> message
> // types. Observations for the following message types are assumed
to be at
> // the surface, as defined by the default message_type_group_map:
> // ADPSFC, SFCSHP, MSONET
> // field = [
> // {
> // message_type = [ "ADPUPA" ];
> // sid_exc = [];
> // name = "TMP";
> // level = [ "P250", "P500", "P700", "P850", "P1000" ];
> // cat_thresh = [ <=273.0 ];
> // },
> // {
> // message_type = [ "ADPSFC" ];
> // sid_exc = [ "KDEN", "KDET" ];
> // name = "TMP";
> // level = [ "Z2" ];
> // cat_thresh = [ <=273.0 ];
> // }
> // ]
>
> I guess I am sort of confused as to how it works when you use level
= "Z2"
> and
> a message_type that maps to value "SURFACE". How does point_stat
take
> these
> two values and apply them to search for the correct items in your
point
> output
> NETCDF?
>
> Justin
>
>
> -----Original Message-----
> From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> Sent: Friday, January 17, 2020 10:04 AM
> To: Tsu, Mr. Justin
> Subject: Re: [rt.rap.ucar.edu #93808] Bad Fcst Value
>
> Thank you for letting us know that you figured it out. I discovered
later
> that you have point NetCDF data output from ascii2nc, whereas the
'(*,*)'
> syntax is for dealing with gridded NetCDF files.
>
> Can you please tell us what part of the user's guide you found
confusing so
> that we can fix it for the next release? Thanks for any help you
can give
> us in making our documentation more clear.
>
> Julie
>
>
> On Fri, Jan 17, 2020 at 10:52 AM Tsu, Mr. Justin via RT
<met_help at ucar.edu
> >
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93808 >
> >
> > Julie,
> >
> > I ended up figuring it out. Thank you for the help!
> >
> > Justin
> >
> > -----Original Message-----
> > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > Sent: Friday, January 17, 2020 8:01 AM
> > To: Tsu, Mr. Justin
> > Subject: Re: [rt.rap.ucar.edu #93808] Bad Fcst Value
> >
> > Hi Justin.
> >
> > Thank you. Could you also please send up the following files?
> >
> > /users/tsu/MET/work/PY_SRC/read_NRL_binary.py
> >
> >
>
/DATA/tsu/repos/flds/uuwind_zht_000010_000000_1a0127x0127_2019072100_00000000_fcstfld
> >
> > Thanks!
> >
> > Julie
> >
> > On Thu, Jan 16, 2020 at 12:51 PM Tsu, Mr. Justin via RT <
> met_help at ucar.edu
> > >
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93808 >
> > >
> > > Thanks Julie,
> > >
> > > The files are located here:
> > >
> > > /incoming/irap/met_help/test
> > >
> > > -----Original Message-----
> > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > Sent: Thursday, January 16, 2020 11:42 AM
> > > To: Tsu, Mr. Justin
> > > Subject: Re: [rt.rap.ucar.edu #93808] Bad Fcst Value
> > >
> > > Hi Justin.
> > >
> > > Are you able to send me the config file you are using, along
with the
> > > sfc_2019072100.nc file, so that I can try to run it myself and
figure
> > out
> > > what is going on?
> > >
> > > There are ftp directions under "How To Send Us Data" on this
page:
> > >
> > >
> >
> https://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk
> > >
> > > Thanks!
> > >
> > > Julie
> > >
> > > On Thu, Jan 16, 2020 at 12:05 PM Tsu, Mr. Justin via RT <
> > met_help at ucar.edu
> > > >
> > > wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93808
>
> > > >
> > > > Thanks Julie,
> > > >
> > > > Here is my command line input and output:
> > > >
> > > > [tsu at lorenz 02_POINT_STAT_WORK]$ !point_stat
> > > > point_stat PYTHON_NUMPY sfc_2019072100.nc testConfig -v 3
> > > > DEBUG 1: Default Config File:
> > > >
> > >
> >
> /software/depot/met-8.1a/met-
8.1a/share/met//config/PointStatConfig_default
> > > > DEBUG 1: User Config File: testConfig
> > > > ERROR :
> > > > ERROR : VarInfo::set_level_info_grib() - failed to parse
level
> string
> > > > '(*)'
> > > > ERROR :
> > > >
> > > > Sfc_2019072100.nc is a netcdf file. I've attached the header
below:
> > > > ncdump -h sfc_2019072100.nc
> > > > netcdf sfc_2019072100 {
> > > > dimensions:
> > > > mxstr = 16 ;
> > > > mxstr2 = 40 ;
> > > > mxstr3 = 80 ;
> > > > nobs = 18034 ;
> > > > nhdr = 18034 ;
> > > > nhdr_typ = 1 ;
> > > > nhdr_sid = 3316 ;
> > > > nhdr_vld = 1 ;
> > > > nobs_qty = 1 ;
> > > > obs_var_num = 6 ;
> > > > variables:
> > > > int hdr_typ(nhdr) ;
> > > > hdr_typ:long_name = "index of message type" ;
> > > > hdr_typ:_FillValue = -9999 ;
> > > > int hdr_sid(nhdr) ;
> > > > hdr_sid:long_name = "index of station
> identification" ;
> > > > hdr_sid:_FillValue = -9999 ;
> > > > int hdr_vld(nhdr) ;
> > > > hdr_vld:long_name = "index of valid time" ;
> > > > hdr_vld:_FillValue = -9999 ;
> > > > float hdr_lat(nhdr) ;
> > > > hdr_lat:long_name = "latitude" ;
> > > > hdr_lat:_FillValue = -9999.f ;
> > > > hdr_lat:units = "degrees_north" ;
> > > > float hdr_lon(nhdr) ;
> > > > hdr_lon:long_name = "longitude" ;
> > > > hdr_lon:_FillValue = -9999.f ;
> > > > hdr_lon:units = "degrees_east" ;
> > > > float hdr_elv(nhdr) ;
> > > > hdr_elv:long_name = "elevation" ;
> > > > hdr_elv:_FillValue = -9999.f ;
> > > > hdr_elv:units = "meters above sea level (msl)"
;
> > > > int obs_qty(nobs) ;
> > > > obs_qty:long_name = "index of quality flag" ;
> > > > int obs_hid(nobs) ;
> > > > obs_hid:long_name = "index of matching header
data" ;
> > > > obs_hid:_FillValue = -9999 ;
> > > > int obs_vid(nobs) ;
> > > > obs_vid:long_name = "index of variable names
at
> > > var_name" ;
> > > > obs_vid:_FillValue = -9999 ;
> > > > float obs_lvl(nobs) ;
> > > > obs_lvl:long_name = "pressure level (hPa) or
> > accumulation
> > > > interval (sec)" ;
> > > > obs_lvl:_FillValue = -9999.f ;
> > > > float obs_hgt(nobs) ;
> > > > obs_hgt:long_name = "height in meters above
sea level
> > or
> > > > ground level (msl or agl)" ;
> > > > obs_hgt:_FillValue = -9999.f ;
> > > > float obs_val(nobs) ;
> > > > obs_val:long_name = "observation value" ;
> > > > obs_val:_FillValue = -9999.f ;
> > > > char hdr_typ_table(nhdr_typ, mxstr2) ;
> > > > hdr_typ_table:long_name = "message type" ;
> > > > char hdr_sid_table(nhdr_sid, mxstr2) ;
> > > > hdr_sid_table:long_name = "station
identification" ;
> > > > char hdr_vld_table(nhdr_vld, mxstr) ;
> > > > hdr_vld_table:long_name = "valid time" ;
> > > > hdr_vld_table:units = "YYYYMMDD_HHMMSS UTC" ;
> > > > char obs_qty_table(nobs_qty, mxstr) ;
> > > > obs_qty_table:long_name = "quality flag" ;
> > > > char obs_var(obs_var_num, mxstr2) ;
> > > > obs_var:long_name = "variable names" ;
> > > >
> > > > // global attributes:
> > > > :MET_Obs_version = "1.02" ;
> > > > :use_var_id = "true" ;
> > > > :FileOrigins = "File / generated
20200116_002940 UTC
> on
> > > > host
> > > > lorenz.nrlmry.navy.mil by the MET ascii2nc tool" ;
> > > > :MET_version = "V8.1" ;
> > > > :MET_tool = "ascii2nc" ;
> > > > }
> > > >
> > > > Hope you can help me out!
> > > >
> > > > Justin
> > > >
> > > >
> > > > -----Original Message-----
> > > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > > Sent: Thursday, January 16, 2020 8:46 AM
> > > > To: Tsu, Mr. Justin
> > > > Subject: Re: [rt.rap.ucar.edu #93808] Bad Fcst Value
> > > >
> > > > Hi Justin.
> > > >
> > > > I'm glad you were able to get the other issue figured out.
> > > >
> > > > Regarding, setting the level to "(*,*)". Please confirm that
you
> have
> > a
> > > > NetCDF file, as that syntax is for NetCDF files only. Also, I
> believe
> > > the
> > > > two asterisks are for two dimensional fields, so if you had a
three
> > > > dimensional field, for example, you would use "(*,*,*)". If
you
> still
> > > > experience an error, please copy and paste the output,
including the
> > > error,
> > > > into an email so we can better see what is happening. Thanks!
> > > >
> > > > Julie
> > > >
> > > > On Wed, Jan 15, 2020 at 5:54 PM Tsu, Mr. Justin via RT <
> > > met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > >
> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93808 >
> > > > >
> > > > > Thanks Julie,
> > > > >
> > > > > I figured out the issue. I do have another question which
you may
> be
> > > > able
> > > > > to
> > > > > help me. From the documentation, it appears that you can
set the
> > level
> > > > to
> > > > > be
> > > > > "(*,*)" in the obs dictionary and that should consider any
ob at
> any
> > > > > vertical
> > > > > level for verification . When I set level to be "(*,*)", I
get a
> > > failed
> > > > > parsing in point_stat. How do I tell point_stat to ignore
the
> > vertical
> > > > > level
> > > > > of my ob?
> > > > >
> > > > > Justin
> > > > >
> > > > > -----Original Message-----
> > > > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > > > Sent: Wednesday, January 15, 2020 11:52 AM
> > > > > To: Tsu, Mr. Justin
> > > > > Subject: Re: [rt.rap.ucar.edu #93808] Bad Fcst Value
> > > > >
> > > > > Hi Justin.
> > > > >
> > > > > I see that some of your data is being rejected due to a "bad
fcst
> > > value".
> > > > > John is likely best suited to answer your question,
unfortunately
> he
> > is
> > > > out
> > > > > of the office at a conference. I have assigned this ticket
to him.
> > > > Please
> > > > > allow a few business days for a response from him. In the
> meantime,
> > > you
> > > > > could try running the plot_data_plane tool on the data to
see if
> that
> > > > shows
> > > > > any obvious problems.
> > > > >
> > > > > Julie
> > > > >
> > > > > --
> > > > > Julie Prestopnik
> > > > > Software Engineer
> > > > > National Center for Atmospheric Research
> > > > > Research Applications Laboratory
> > > > > Phone: 303.497.8399
> > > > > Email: jpresto at ucar.edu
> > > > >
> > > > > My working day may not be your working day. Please do not
feel
> > obliged
> > > > to
> > > > > reply to this email outside of your normal working hours.
> > > > >
> > > > >
> > > > >
> > > >
> > > > --
> > > > Julie Prestopnik
> > > > Software Engineer
> > > > National Center for Atmospheric Research
> > > > Research Applications Laboratory
> > > > Phone: 303.497.8399
> > > > Email: jpresto at ucar.edu
> > > >
> > > > My working day may not be your working day. Please do not
feel
> obliged
> > > to
> > > > reply to this email outside of your normal working hours.
> > > >
> > > >
> > > >
> > >
> > > --
> > > Julie Prestopnik
> > > Software Engineer
> > > National Center for Atmospheric Research
> > > Research Applications Laboratory
> > > Phone: 303.497.8399
> > > Email: jpresto at ucar.edu
> > >
> > > My working day may not be your working day. Please do not feel
obliged
> > to
> > > reply to this email outside of your normal working hours.
> > >
> > >
> > >
> >
> > --
> > Julie Prestopnik
> > Software Engineer
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > Phone: 303.497.8399
> > Email: jpresto at ucar.edu
> >
> > My working day may not be your working day. Please do not feel
obliged
> to
> > reply to this email outside of your normal working hours.
> >
> >
> >
>
> --
> Julie Prestopnik
> Software Engineer
> National Center for Atmospheric Research
> Research Applications Laboratory
> Phone: 303.497.8399
> Email: jpresto at ucar.edu
>
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>
>
------------------------------------------------
Subject: Bad Fcst Value
From: Tsu, Mr. Justin
Time: Fri Jan 17 12:34:35 2020
But I had message_type equal to 'SURFACE' and level equal to some
range, say
'P1000-P1015' and point_stat actually took the pressures into
consideration
(i.e. some of my obs were rejected due, because for some reason some
of my obs
have a -1000 pressure level). I wanted to avoid the rejection of
these obs so
needed a way to override this level feature. Does point_stat ignore
the
message_type if using level = "P*-P*" but checks the message_type if
using
level = "Z*"?
Justin
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Friday, January 17, 2020 11:22 AM
To: Tsu, Mr. Justin
Subject: Re: [rt.rap.ucar.edu #93808] Bad Fcst Value
Justin,
While verifying against point observations can sound straight-forward,
there are lots of details to consider, as you're finding!
All of the MET point pre-processing tools (ascii2nc, madis2nc, pb2nc,
and
lidar2nc) write to a common NetCDF point observation file format.
Each
point observation stored in those files is associated with a
message_type.
The Point-Stat configuration file defines which of those message types
are
assumed to exist at the surface:
message_type_group_map = [
{ key = "SURFACE"; val = "ADPSFC,SFCSHP,MSONET"; }
...
}
The logic is this... if verifying against a non-pressure level
observation
field with a "surface" message type, don't actually check that the
vertical
level value match.
For example, you are verifying a forecast of 2-m temperature against
ADPSFC
observations of temperature at 2 meters:
fcst = { field = [ { name="TMP"; level="Z2"; } ]; };
obs = { field = [ { name="TMP"; level="Z2"; message_type = "ADPSFC"; }
]; };
When we search the point observations for matches, we note the
"ADPSFC" is
a surface message type. "Z2" is a height level, not a pressure level.
So
we don't require that the observations actually have an elevation
exactly
equal to 2 meters.
In this way, when verifying against surface observations, it's the
message
type that matters, not the actual observation elevation value.
John
On Fri, Jan 17, 2020 at 11:38 AM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93808 >
>
> Perhaps including in the documentation when to use the syntax
'(*,*)' as
> you've just stated? The information that I used was on page 63:
>
> // - Simplified vertical level matching logic is applied for surface
> message
> // types. Observations for the following message types are assumed
to be at
> // the surface, as defined by the default message_type_group_map:
> // ADPSFC, SFCSHP, MSONET
> // field = [
> // {
> // message_type = [ "ADPUPA" ];
> // sid_exc = [];
> // name = "TMP";
> // level = [ "P250", "P500", "P700", "P850", "P1000" ];
> // cat_thresh = [ <=273.0 ];
> // },
> // {
> // message_type = [ "ADPSFC" ];
> // sid_exc = [ "KDEN", "KDET" ];
> // name = "TMP";
> // level = [ "Z2" ];
> // cat_thresh = [ <=273.0 ];
> // }
> // ]
>
> I guess I am sort of confused as to how it works when you use level
= "Z2"
> and
> a message_type that maps to value "SURFACE". How does point_stat
take
> these
> two values and apply them to search for the correct items in your
point
> output
> NETCDF?
>
> Justin
>
>
> -----Original Message-----
> From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> Sent: Friday, January 17, 2020 10:04 AM
> To: Tsu, Mr. Justin
> Subject: Re: [rt.rap.ucar.edu #93808] Bad Fcst Value
>
> Thank you for letting us know that you figured it out. I discovered
later
> that you have point NetCDF data output from ascii2nc, whereas the
'(*,*)'
> syntax is for dealing with gridded NetCDF files.
>
> Can you please tell us what part of the user's guide you found
confusing so
> that we can fix it for the next release? Thanks for any help you
can give
> us in making our documentation more clear.
>
> Julie
>
>
> On Fri, Jan 17, 2020 at 10:52 AM Tsu, Mr. Justin via RT
<met_help at ucar.edu
> >
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93808 >
> >
> > Julie,
> >
> > I ended up figuring it out. Thank you for the help!
> >
> > Justin
> >
> > -----Original Message-----
> > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > Sent: Friday, January 17, 2020 8:01 AM
> > To: Tsu, Mr. Justin
> > Subject: Re: [rt.rap.ucar.edu #93808] Bad Fcst Value
> >
> > Hi Justin.
> >
> > Thank you. Could you also please send up the following files?
> >
> > /users/tsu/MET/work/PY_SRC/read_NRL_binary.py
> >
> >
>
/DATA/tsu/repos/flds/uuwind_zht_000010_000000_1a0127x0127_2019072100_00000000_fcstfld
> >
> > Thanks!
> >
> > Julie
> >
> > On Thu, Jan 16, 2020 at 12:51 PM Tsu, Mr. Justin via RT <
> met_help at ucar.edu
> > >
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93808 >
> > >
> > > Thanks Julie,
> > >
> > > The files are located here:
> > >
> > > /incoming/irap/met_help/test
> > >
> > > -----Original Message-----
> > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > Sent: Thursday, January 16, 2020 11:42 AM
> > > To: Tsu, Mr. Justin
> > > Subject: Re: [rt.rap.ucar.edu #93808] Bad Fcst Value
> > >
> > > Hi Justin.
> > >
> > > Are you able to send me the config file you are using, along
with the
> > > sfc_2019072100.nc file, so that I can try to run it myself and
figure
> > out
> > > what is going on?
> > >
> > > There are ftp directions under "How To Send Us Data" on this
page:
> > >
> > >
> >
> https://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk
> > >
> > > Thanks!
> > >
> > > Julie
> > >
> > > On Thu, Jan 16, 2020 at 12:05 PM Tsu, Mr. Justin via RT <
> > met_help at ucar.edu
> > > >
> > > wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93808
>
> > > >
> > > > Thanks Julie,
> > > >
> > > > Here is my command line input and output:
> > > >
> > > > [tsu at lorenz 02_POINT_STAT_WORK]$ !point_stat
> > > > point_stat PYTHON_NUMPY sfc_2019072100.nc testConfig -v 3
> > > > DEBUG 1: Default Config File:
> > > >
> > >
> >
> /software/depot/met-8.1a/met-
8.1a/share/met//config/PointStatConfig_default
> > > > DEBUG 1: User Config File: testConfig
> > > > ERROR :
> > > > ERROR : VarInfo::set_level_info_grib() - failed to parse
level
> string
> > > > '(*)'
> > > > ERROR :
> > > >
> > > > Sfc_2019072100.nc is a netcdf file. I've attached the header
below:
> > > > ncdump -h sfc_2019072100.nc
> > > > netcdf sfc_2019072100 {
> > > > dimensions:
> > > > mxstr = 16 ;
> > > > mxstr2 = 40 ;
> > > > mxstr3 = 80 ;
> > > > nobs = 18034 ;
> > > > nhdr = 18034 ;
> > > > nhdr_typ = 1 ;
> > > > nhdr_sid = 3316 ;
> > > > nhdr_vld = 1 ;
> > > > nobs_qty = 1 ;
> > > > obs_var_num = 6 ;
> > > > variables:
> > > > int hdr_typ(nhdr) ;
> > > > hdr_typ:long_name = "index of message type" ;
> > > > hdr_typ:_FillValue = -9999 ;
> > > > int hdr_sid(nhdr) ;
> > > > hdr_sid:long_name = "index of station
> identification" ;
> > > > hdr_sid:_FillValue = -9999 ;
> > > > int hdr_vld(nhdr) ;
> > > > hdr_vld:long_name = "index of valid time" ;
> > > > hdr_vld:_FillValue = -9999 ;
> > > > float hdr_lat(nhdr) ;
> > > > hdr_lat:long_name = "latitude" ;
> > > > hdr_lat:_FillValue = -9999.f ;
> > > > hdr_lat:units = "degrees_north" ;
> > > > float hdr_lon(nhdr) ;
> > > > hdr_lon:long_name = "longitude" ;
> > > > hdr_lon:_FillValue = -9999.f ;
> > > > hdr_lon:units = "degrees_east" ;
> > > > float hdr_elv(nhdr) ;
> > > > hdr_elv:long_name = "elevation" ;
> > > > hdr_elv:_FillValue = -9999.f ;
> > > > hdr_elv:units = "meters above sea level (msl)"
;
> > > > int obs_qty(nobs) ;
> > > > obs_qty:long_name = "index of quality flag" ;
> > > > int obs_hid(nobs) ;
> > > > obs_hid:long_name = "index of matching header
data" ;
> > > > obs_hid:_FillValue = -9999 ;
> > > > int obs_vid(nobs) ;
> > > > obs_vid:long_name = "index of variable names
at
> > > var_name" ;
> > > > obs_vid:_FillValue = -9999 ;
> > > > float obs_lvl(nobs) ;
> > > > obs_lvl:long_name = "pressure level (hPa) or
> > accumulation
> > > > interval (sec)" ;
> > > > obs_lvl:_FillValue = -9999.f ;
> > > > float obs_hgt(nobs) ;
> > > > obs_hgt:long_name = "height in meters above
sea level
> > or
> > > > ground level (msl or agl)" ;
> > > > obs_hgt:_FillValue = -9999.f ;
> > > > float obs_val(nobs) ;
> > > > obs_val:long_name = "observation value" ;
> > > > obs_val:_FillValue = -9999.f ;
> > > > char hdr_typ_table(nhdr_typ, mxstr2) ;
> > > > hdr_typ_table:long_name = "message type" ;
> > > > char hdr_sid_table(nhdr_sid, mxstr2) ;
> > > > hdr_sid_table:long_name = "station
identification" ;
> > > > char hdr_vld_table(nhdr_vld, mxstr) ;
> > > > hdr_vld_table:long_name = "valid time" ;
> > > > hdr_vld_table:units = "YYYYMMDD_HHMMSS UTC" ;
> > > > char obs_qty_table(nobs_qty, mxstr) ;
> > > > obs_qty_table:long_name = "quality flag" ;
> > > > char obs_var(obs_var_num, mxstr2) ;
> > > > obs_var:long_name = "variable names" ;
> > > >
> > > > // global attributes:
> > > > :MET_Obs_version = "1.02" ;
> > > > :use_var_id = "true" ;
> > > > :FileOrigins = "File / generated
20200116_002940 UTC
> on
> > > > host
> > > > lorenz.nrlmry.navy.mil by the MET ascii2nc tool" ;
> > > > :MET_version = "V8.1" ;
> > > > :MET_tool = "ascii2nc" ;
> > > > }
> > > >
> > > > Hope you can help me out!
> > > >
> > > > Justin
> > > >
> > > >
> > > > -----Original Message-----
> > > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > > Sent: Thursday, January 16, 2020 8:46 AM
> > > > To: Tsu, Mr. Justin
> > > > Subject: Re: [rt.rap.ucar.edu #93808] Bad Fcst Value
> > > >
> > > > Hi Justin.
> > > >
> > > > I'm glad you were able to get the other issue figured out.
> > > >
> > > > Regarding, setting the level to "(*,*)". Please confirm that
you
> have
> > a
> > > > NetCDF file, as that syntax is for NetCDF files only. Also, I
> believe
> > > the
> > > > two asterisks are for two dimensional fields, so if you had a
three
> > > > dimensional field, for example, you would use "(*,*,*)". If
you
> still
> > > > experience an error, please copy and paste the output,
including the
> > > error,
> > > > into an email so we can better see what is happening. Thanks!
> > > >
> > > > Julie
> > > >
> > > > On Wed, Jan 15, 2020 at 5:54 PM Tsu, Mr. Justin via RT <
> > > met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > >
> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93808 >
> > > > >
> > > > > Thanks Julie,
> > > > >
> > > > > I figured out the issue. I do have another question which
you may
> be
> > > > able
> > > > > to
> > > > > help me. From the documentation, it appears that you can
set the
> > level
> > > > to
> > > > > be
> > > > > "(*,*)" in the obs dictionary and that should consider any
ob at
> any
> > > > > vertical
> > > > > level for verification . When I set level to be "(*,*)", I
get a
> > > failed
> > > > > parsing in point_stat. How do I tell point_stat to ignore
the
> > vertical
> > > > > level
> > > > > of my ob?
> > > > >
> > > > > Justin
> > > > >
> > > > > -----Original Message-----
> > > > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > > > Sent: Wednesday, January 15, 2020 11:52 AM
> > > > > To: Tsu, Mr. Justin
> > > > > Subject: Re: [rt.rap.ucar.edu #93808] Bad Fcst Value
> > > > >
> > > > > Hi Justin.
> > > > >
> > > > > I see that some of your data is being rejected due to a "bad
fcst
> > > value".
> > > > > John is likely best suited to answer your question,
unfortunately
> he
> > is
> > > > out
> > > > > of the office at a conference. I have assigned this ticket
to him.
> > > > Please
> > > > > allow a few business days for a response from him. In the
> meantime,
> > > you
> > > > > could try running the plot_data_plane tool on the data to
see if
> that
> > > > shows
> > > > > any obvious problems.
> > > > >
> > > > > Julie
> > > > >
> > > > > --
> > > > > Julie Prestopnik
> > > > > Software Engineer
> > > > > National Center for Atmospheric Research
> > > > > Research Applications Laboratory
> > > > > Phone: 303.497.8399
> > > > > Email: jpresto at ucar.edu
> > > > >
> > > > > My working day may not be your working day. Please do not
feel
> > obliged
> > > > to
> > > > > reply to this email outside of your normal working hours.
> > > > >
> > > > >
> > > > >
> > > >
> > > > --
> > > > Julie Prestopnik
> > > > Software Engineer
> > > > National Center for Atmospheric Research
> > > > Research Applications Laboratory
> > > > Phone: 303.497.8399
> > > > Email: jpresto at ucar.edu
> > > >
> > > > My working day may not be your working day. Please do not
feel
> obliged
> > > to
> > > > reply to this email outside of your normal working hours.
> > > >
> > > >
> > > >
> > >
> > > --
> > > Julie Prestopnik
> > > Software Engineer
> > > National Center for Atmospheric Research
> > > Research Applications Laboratory
> > > Phone: 303.497.8399
> > > Email: jpresto at ucar.edu
> > >
> > > My working day may not be your working day. Please do not feel
obliged
> > to
> > > reply to this email outside of your normal working hours.
> > >
> > >
> > >
> >
> > --
> > Julie Prestopnik
> > Software Engineer
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > Phone: 303.497.8399
> > Email: jpresto at ucar.edu
> >
> > My working day may not be your working day. Please do not feel
obliged
> to
> > reply to this email outside of your normal working hours.
> >
> >
> >
>
> --
> Julie Prestopnik
> Software Engineer
> National Center for Atmospheric Research
> Research Applications Laboratory
> Phone: 303.497.8399
> Email: jpresto at ucar.edu
>
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>
>
------------------------------------------------
Subject: Bad Fcst Value
From: John Halley Gotway
Time: Fri Jan 17 13:01:49 2020
Yes, when verifying against surface observations, the vertical level
matching logic IS applied when the user specifies pressure levels
(e.g.
P700-900) but IS NOT applied for other vertical level types (e.g. Z2,
Z10,
or L10).
On Fri, Jan 17, 2020 at 12:34 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93808 >
>
> But I had message_type equal to 'SURFACE' and level equal to some
range,
> say
> 'P1000-P1015' and point_stat actually took the pressures into
> consideration
> (i.e. some of my obs were rejected due, because for some reason some
of my
> obs
> have a -1000 pressure level). I wanted to avoid the rejection of
these
> obs so
> needed a way to override this level feature. Does point_stat ignore
the
> message_type if using level = "P*-P*" but checks the message_type if
using
> level = "Z*"?
>
> Justin
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Friday, January 17, 2020 11:22 AM
> To: Tsu, Mr. Justin
> Subject: Re: [rt.rap.ucar.edu #93808] Bad Fcst Value
>
> Justin,
>
> While verifying against point observations can sound straight-
forward,
> there are lots of details to consider, as you're finding!
>
> All of the MET point pre-processing tools (ascii2nc, madis2nc,
pb2nc, and
> lidar2nc) write to a common NetCDF point observation file format.
Each
> point observation stored in those files is associated with a
message_type.
> The Point-Stat configuration file defines which of those message
types are
> assumed to exist at the surface:
>
> message_type_group_map = [
> { key = "SURFACE"; val = "ADPSFC,SFCSHP,MSONET"; }
> ...
> }
>
> The logic is this... if verifying against a non-pressure level
observation
> field with a "surface" message type, don't actually check that the
vertical
> level value match.
>
> For example, you are verifying a forecast of 2-m temperature against
ADPSFC
> observations of temperature at 2 meters:
>
> fcst = { field = [ { name="TMP"; level="Z2"; } ]; };
> obs = { field = [ { name="TMP"; level="Z2"; message_type = "ADPSFC";
} ];
> };
>
> When we search the point observations for matches, we note the
"ADPSFC" is
> a surface message type. "Z2" is a height level, not a pressure
level. So
> we don't require that the observations actually have an elevation
exactly
> equal to 2 meters.
>
> In this way, when verifying against surface observations, it's the
message
> type that matters, not the actual observation elevation value.
>
> John
>
>
> On Fri, Jan 17, 2020 at 11:38 AM Tsu, Mr. Justin via RT
<met_help at ucar.edu
> >
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93808 >
> >
> > Perhaps including in the documentation when to use the syntax
'(*,*)' as
> > you've just stated? The information that I used was on page 63:
> >
> > // - Simplified vertical level matching logic is applied for
surface
> > message
> > // types. Observations for the following message types are assumed
to be
> at
> > // the surface, as defined by the default message_type_group_map:
> > // ADPSFC, SFCSHP, MSONET
> > // field = [
> > // {
> > // message_type = [ "ADPUPA" ];
> > // sid_exc = [];
> > // name = "TMP";
> > // level = [ "P250", "P500", "P700", "P850", "P1000" ];
> > // cat_thresh = [ <=273.0 ];
> > // },
> > // {
> > // message_type = [ "ADPSFC" ];
> > // sid_exc = [ "KDEN", "KDET" ];
> > // name = "TMP";
> > // level = [ "Z2" ];
> > // cat_thresh = [ <=273.0 ];
> > // }
> > // ]
> >
> > I guess I am sort of confused as to how it works when you use
level =
> "Z2"
> > and
> > a message_type that maps to value "SURFACE". How does point_stat
take
> > these
> > two values and apply them to search for the correct items in your
point
> > output
> > NETCDF?
> >
> > Justin
> >
> >
> > -----Original Message-----
> > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > Sent: Friday, January 17, 2020 10:04 AM
> > To: Tsu, Mr. Justin
> > Subject: Re: [rt.rap.ucar.edu #93808] Bad Fcst Value
> >
> > Thank you for letting us know that you figured it out. I
discovered
> later
> > that you have point NetCDF data output from ascii2nc, whereas the
'(*,*)'
> > syntax is for dealing with gridded NetCDF files.
> >
> > Can you please tell us what part of the user's guide you found
confusing
> so
> > that we can fix it for the next release? Thanks for any help you
can
> give
> > us in making our documentation more clear.
> >
> > Julie
> >
> >
> > On Fri, Jan 17, 2020 at 10:52 AM Tsu, Mr. Justin via RT <
> met_help at ucar.edu
> > >
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93808 >
> > >
> > > Julie,
> > >
> > > I ended up figuring it out. Thank you for the help!
> > >
> > > Justin
> > >
> > > -----Original Message-----
> > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > Sent: Friday, January 17, 2020 8:01 AM
> > > To: Tsu, Mr. Justin
> > > Subject: Re: [rt.rap.ucar.edu #93808] Bad Fcst Value
> > >
> > > Hi Justin.
> > >
> > > Thank you. Could you also please send up the following files?
> > >
> > > /users/tsu/MET/work/PY_SRC/read_NRL_binary.py
> > >
> > >
> >
>
/DATA/tsu/repos/flds/uuwind_zht_000010_000000_1a0127x0127_2019072100_00000000_fcstfld
> > >
> > > Thanks!
> > >
> > > Julie
> > >
> > > On Thu, Jan 16, 2020 at 12:51 PM Tsu, Mr. Justin via RT <
> > met_help at ucar.edu
> > > >
> > > wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93808
>
> > > >
> > > > Thanks Julie,
> > > >
> > > > The files are located here:
> > > >
> > > > /incoming/irap/met_help/test
> > > >
> > > > -----Original Message-----
> > > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > > Sent: Thursday, January 16, 2020 11:42 AM
> > > > To: Tsu, Mr. Justin
> > > > Subject: Re: [rt.rap.ucar.edu #93808] Bad Fcst Value
> > > >
> > > > Hi Justin.
> > > >
> > > > Are you able to send me the config file you are using, along
with the
> > > > sfc_2019072100.nc file, so that I can try to run it myself and
> figure
> > > out
> > > > what is going on?
> > > >
> > > > There are ftp directions under "How To Send Us Data" on this
page:
> > > >
> > > >
> > >
> >
> https://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk
> > > >
> > > > Thanks!
> > > >
> > > > Julie
> > > >
> > > > On Thu, Jan 16, 2020 at 12:05 PM Tsu, Mr. Justin via RT <
> > > met_help at ucar.edu
> > > > >
> > > > wrote:
> > > >
> > > > >
> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93808 >
> > > > >
> > > > > Thanks Julie,
> > > > >
> > > > > Here is my command line input and output:
> > > > >
> > > > > [tsu at lorenz 02_POINT_STAT_WORK]$ !point_stat
> > > > > point_stat PYTHON_NUMPY sfc_2019072100.nc testConfig -v 3
> > > > > DEBUG 1: Default Config File:
> > > > >
> > > >
> > >
> >
> /software/depot/met-8.1a/met-
8.1a/share/met//config/PointStatConfig_default
> > > > > DEBUG 1: User Config File: testConfig
> > > > > ERROR :
> > > > > ERROR : VarInfo::set_level_info_grib() - failed to parse
level
> > string
> > > > > '(*)'
> > > > > ERROR :
> > > > >
> > > > > Sfc_2019072100.nc is a netcdf file. I've attached the
header
> below:
> > > > > ncdump -h sfc_2019072100.nc
> > > > > netcdf sfc_2019072100 {
> > > > > dimensions:
> > > > > mxstr = 16 ;
> > > > > mxstr2 = 40 ;
> > > > > mxstr3 = 80 ;
> > > > > nobs = 18034 ;
> > > > > nhdr = 18034 ;
> > > > > nhdr_typ = 1 ;
> > > > > nhdr_sid = 3316 ;
> > > > > nhdr_vld = 1 ;
> > > > > nobs_qty = 1 ;
> > > > > obs_var_num = 6 ;
> > > > > variables:
> > > > > int hdr_typ(nhdr) ;
> > > > > hdr_typ:long_name = "index of message type"
;
> > > > > hdr_typ:_FillValue = -9999 ;
> > > > > int hdr_sid(nhdr) ;
> > > > > hdr_sid:long_name = "index of station
> > identification" ;
> > > > > hdr_sid:_FillValue = -9999 ;
> > > > > int hdr_vld(nhdr) ;
> > > > > hdr_vld:long_name = "index of valid time" ;
> > > > > hdr_vld:_FillValue = -9999 ;
> > > > > float hdr_lat(nhdr) ;
> > > > > hdr_lat:long_name = "latitude" ;
> > > > > hdr_lat:_FillValue = -9999.f ;
> > > > > hdr_lat:units = "degrees_north" ;
> > > > > float hdr_lon(nhdr) ;
> > > > > hdr_lon:long_name = "longitude" ;
> > > > > hdr_lon:_FillValue = -9999.f ;
> > > > > hdr_lon:units = "degrees_east" ;
> > > > > float hdr_elv(nhdr) ;
> > > > > hdr_elv:long_name = "elevation" ;
> > > > > hdr_elv:_FillValue = -9999.f ;
> > > > > hdr_elv:units = "meters above sea level
(msl)" ;
> > > > > int obs_qty(nobs) ;
> > > > > obs_qty:long_name = "index of quality flag"
;
> > > > > int obs_hid(nobs) ;
> > > > > obs_hid:long_name = "index of matching
header
> data" ;
> > > > > obs_hid:_FillValue = -9999 ;
> > > > > int obs_vid(nobs) ;
> > > > > obs_vid:long_name = "index of variable names
at
> > > > var_name" ;
> > > > > obs_vid:_FillValue = -9999 ;
> > > > > float obs_lvl(nobs) ;
> > > > > obs_lvl:long_name = "pressure level (hPa) or
> > > accumulation
> > > > > interval (sec)" ;
> > > > > obs_lvl:_FillValue = -9999.f ;
> > > > > float obs_hgt(nobs) ;
> > > > > obs_hgt:long_name = "height in meters above
sea
> level
> > > or
> > > > > ground level (msl or agl)" ;
> > > > > obs_hgt:_FillValue = -9999.f ;
> > > > > float obs_val(nobs) ;
> > > > > obs_val:long_name = "observation value" ;
> > > > > obs_val:_FillValue = -9999.f ;
> > > > > char hdr_typ_table(nhdr_typ, mxstr2) ;
> > > > > hdr_typ_table:long_name = "message type" ;
> > > > > char hdr_sid_table(nhdr_sid, mxstr2) ;
> > > > > hdr_sid_table:long_name = "station
identification"
> ;
> > > > > char hdr_vld_table(nhdr_vld, mxstr) ;
> > > > > hdr_vld_table:long_name = "valid time" ;
> > > > > hdr_vld_table:units = "YYYYMMDD_HHMMSS UTC"
;
> > > > > char obs_qty_table(nobs_qty, mxstr) ;
> > > > > obs_qty_table:long_name = "quality flag" ;
> > > > > char obs_var(obs_var_num, mxstr2) ;
> > > > > obs_var:long_name = "variable names" ;
> > > > >
> > > > > // global attributes:
> > > > > :MET_Obs_version = "1.02" ;
> > > > > :use_var_id = "true" ;
> > > > > :FileOrigins = "File / generated
20200116_002940
> UTC
> > on
> > > > > host
> > > > > lorenz.nrlmry.navy.mil by the MET ascii2nc tool" ;
> > > > > :MET_version = "V8.1" ;
> > > > > :MET_tool = "ascii2nc" ;
> > > > > }
> > > > >
> > > > > Hope you can help me out!
> > > > >
> > > > > Justin
> > > > >
> > > > >
> > > > > -----Original Message-----
> > > > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > > > Sent: Thursday, January 16, 2020 8:46 AM
> > > > > To: Tsu, Mr. Justin
> > > > > Subject: Re: [rt.rap.ucar.edu #93808] Bad Fcst Value
> > > > >
> > > > > Hi Justin.
> > > > >
> > > > > I'm glad you were able to get the other issue figured out.
> > > > >
> > > > > Regarding, setting the level to "(*,*)". Please confirm
that you
> > have
> > > a
> > > > > NetCDF file, as that syntax is for NetCDF files only. Also,
I
> > believe
> > > > the
> > > > > two asterisks are for two dimensional fields, so if you had
a three
> > > > > dimensional field, for example, you would use "(*,*,*)". If
you
> > still
> > > > > experience an error, please copy and paste the output,
including
> the
> > > > error,
> > > > > into an email so we can better see what is happening.
Thanks!
> > > > >
> > > > > Julie
> > > > >
> > > > > On Wed, Jan 15, 2020 at 5:54 PM Tsu, Mr. Justin via RT <
> > > > met_help at ucar.edu>
> > > > > wrote:
> > > > >
> > > > > >
> > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93808 >
> > > > > >
> > > > > > Thanks Julie,
> > > > > >
> > > > > > I figured out the issue. I do have another question which
you
> may
> > be
> > > > > able
> > > > > > to
> > > > > > help me. From the documentation, it appears that you can
set the
> > > level
> > > > > to
> > > > > > be
> > > > > > "(*,*)" in the obs dictionary and that should consider any
ob at
> > any
> > > > > > vertical
> > > > > > level for verification . When I set level to be "(*,*)",
I get a
> > > > failed
> > > > > > parsing in point_stat. How do I tell point_stat to ignore
the
> > > vertical
> > > > > > level
> > > > > > of my ob?
> > > > > >
> > > > > > Justin
> > > > > >
> > > > > > -----Original Message-----
> > > > > > From: Julie Prestopnik via RT [mailto:met_help at ucar.edu]
> > > > > > Sent: Wednesday, January 15, 2020 11:52 AM
> > > > > > To: Tsu, Mr. Justin
> > > > > > Subject: Re: [rt.rap.ucar.edu #93808] Bad Fcst Value
> > > > > >
> > > > > > Hi Justin.
> > > > > >
> > > > > > I see that some of your data is being rejected due to a
"bad fcst
> > > > value".
> > > > > > John is likely best suited to answer your question,
unfortunately
> > he
> > > is
> > > > > out
> > > > > > of the office at a conference. I have assigned this
ticket to
> him.
> > > > > Please
> > > > > > allow a few business days for a response from him. In
the
> > meantime,
> > > > you
> > > > > > could try running the plot_data_plane tool on the data to
see if
> > that
> > > > > shows
> > > > > > any obvious problems.
> > > > > >
> > > > > > Julie
> > > > > >
> > > > > > --
> > > > > > Julie Prestopnik
> > > > > > Software Engineer
> > > > > > National Center for Atmospheric Research
> > > > > > Research Applications Laboratory
> > > > > > Phone: 303.497.8399
> > > > > > Email: jpresto at ucar.edu
> > > > > >
> > > > > > My working day may not be your working day. Please do not
feel
> > > obliged
> > > > > to
> > > > > > reply to this email outside of your normal working hours.
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > > --
> > > > > Julie Prestopnik
> > > > > Software Engineer
> > > > > National Center for Atmospheric Research
> > > > > Research Applications Laboratory
> > > > > Phone: 303.497.8399
> > > > > Email: jpresto at ucar.edu
> > > > >
> > > > > My working day may not be your working day. Please do not
feel
> > obliged
> > > > to
> > > > > reply to this email outside of your normal working hours.
> > > > >
> > > > >
> > > > >
> > > >
> > > > --
> > > > Julie Prestopnik
> > > > Software Engineer
> > > > National Center for Atmospheric Research
> > > > Research Applications Laboratory
> > > > Phone: 303.497.8399
> > > > Email: jpresto at ucar.edu
> > > >
> > > > My working day may not be your working day. Please do not
feel
> obliged
> > > to
> > > > reply to this email outside of your normal working hours.
> > > >
> > > >
> > > >
> > >
> > > --
> > > Julie Prestopnik
> > > Software Engineer
> > > National Center for Atmospheric Research
> > > Research Applications Laboratory
> > > Phone: 303.497.8399
> > > Email: jpresto at ucar.edu
> > >
> > > My working day may not be your working day. Please do not feel
obliged
> > to
> > > reply to this email outside of your normal working hours.
> > >
> > >
> > >
> >
> > --
> > Julie Prestopnik
> > Software Engineer
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > Phone: 303.497.8399
> > Email: jpresto at ucar.edu
> >
> > My working day may not be your working day. Please do not feel
obliged
> to
> > reply to this email outside of your normal working hours.
> >
> >
> >
>
>
>
------------------------------------------------
More information about the Met_help
mailing list