[Met_help] [rt.rap.ucar.edu #93039] History for Absurdly high ME2

[John Halley Gotway via RT]

----------------------------------------------------------------
  Initial Request
----------------------------------------------------------------

I am getting some absurdly high mean square errors for every one of my
variables (wind speed, wind direction, relative humidity, dew point
depression, and air temperature) on each vertical pressure level.  These
values range from the thousands to the 10s of millions.  How can I diagnose
the cause of these massive errors?

 

Some context:

.        I am running point_stat

.        The command is           

 point_stat PYTHON_NUMPY ${OBFILE} ${CONFIG}       \

                                  -v 0                      \

                                  -outdir ${PSOUT}          \

                                  -log
${PSOUT}/${VAR_NAME}${WIND_FLAG}_TAU${FCST_HR}_point_stat.log \

                                  -obs_valid_beg 20010101   \

                                  -obs_valid_end 20200101

.        I've tailored my config file so that the vertical levels of both
the fcst and obs are relatively near, so this cannot be a vertical mismatch
issue

Justin

 

Justin Tsu

Marine Meteorology Division

Data Assimilation/Mesoscale Modeling

Building 704 Room 212

Naval Research Laboratory, Code 7531

7 Grace Hopper Avenue

Monterey, CA 93943-5502

 

Ph. (831) 656-4111

 



----------------------------------------------------------------
  Complete Ticket History
----------------------------------------------------------------

Subject: Absurdly high ME2
From: John Halley Gotway
Time: Fri Nov 08 23:14:32 2019

Justin,

The first thing I’d check for is a difference in the units of the fcst
and
obs data which would lead to large error values.

I’d recommend turning on the matched pair (MPR) output line type....
set it
to BOTH.  Then look in the “*_mpr.txt” output file at the FCST and OBS
columns.  Check the values to see if there’s and obvious mismatch in
units.

If so, you can define a “convert(x)” function in either the fcst or
obs
dictionaries of the config file to make the units comparable.

Thanks,
John

On Fri, Nov 8, 2019 at 5:05 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
wrote:

>
> Fri Nov 08 16:04:58 2019: Request 93039 was acted upon.
> Transaction: Ticket created by justin.tsu at nrlmry.navy.mil
>        Queue: met_help
>      Subject: Absurdly high ME2
>        Owner: Nobody
>   Requestors: justin.tsu at nrlmry.navy.mil
>       Status: new
>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039 >
>
>
> I am getting some absurdly high mean square errors for every one of
my
> variables (wind speed, wind direction, relative humidity, dew point
> depression, and air temperature) on each vertical pressure level.
These
> values range from the thousands to the 10s of millions.  How can I
diagnose
> the cause of these massive errors?
>
>
>
> Some context:
>
> .        I am running point_stat
>
> .        The command is
>
>  point_stat PYTHON_NUMPY ${OBFILE} ${CONFIG}       \
>
>                                   -v 0                      \
>
>                                   -outdir ${PSOUT}          \
>
>                                   -log
> ${PSOUT}/${VAR_NAME}${WIND_FLAG}_TAU${FCST_HR}_point_stat.log \
>
>                                   -obs_valid_beg 20010101   \
>
>                                   -obs_valid_end 20200101
>
> .        I've tailored my config file so that the vertical levels of
both
> the fcst and obs are relatively near, so this cannot be a vertical
mismatch
> issue
>
> Justin
>
>
>
> Justin Tsu
>
> Marine Meteorology Division
>
> Data Assimilation/Mesoscale Modeling
>
> Building 704 Room 212
>
> Naval Research Laboratory, Code 7531
>
> 7 Grace Hopper Avenue
>
> Monterey, CA 93943-5502
>
>
>
> Ph. (831) 656-4111
>
>
>
>
>

------------------------------------------------
Subject: Absurdly high ME2
From: Tsu, Mr. Justin
Time: Tue Nov 12 14:17:16 2019

Just discovered that I was only computing dew point depression and not
T_d which is what my fcst is.  Fixed this (which will also fix the
relative humidity).  Also discovered that my raobs airtmp are in
Celsius but my sfc obs are in Kelvin (I assumed sfc was also Celsius
so was doing conversion on that).  Still not sure why my wind
direction and speed have high errors though.

Justin

-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Friday, November 8, 2019 10:15 PM
To: Tsu, Mr. Justin
Subject: Re: [rt.rap.ucar.edu #93039] Absurdly high ME2

Justin,

The first thing I’d check for is a difference in the units of the fcst
and
obs data which would lead to large error values.

I’d recommend turning on the matched pair (MPR) output line type....
set it
to BOTH.  Then look in the “*_mpr.txt” output file at the FCST and OBS
columns.  Check the values to see if there’s and obvious mismatch in
units.

If so, you can define a “convert(x)” function in either the fcst or
obs
dictionaries of the config file to make the units comparable.

Thanks,
John

On Fri, Nov 8, 2019 at 5:05 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
wrote:

>
> Fri Nov 08 16:04:58 2019: Request 93039 was acted upon.
> Transaction: Ticket created by justin.tsu at nrlmry.navy.mil
>        Queue: met_help
>      Subject: Absurdly high ME2
>        Owner: Nobody
>   Requestors: justin.tsu at nrlmry.navy.mil
>       Status: new
>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039 >
>
>
> I am getting some absurdly high mean square errors for every one of
my
> variables (wind speed, wind direction, relative humidity, dew point
> depression, and air temperature) on each vertical pressure level.
These
> values range from the thousands to the 10s of millions.  How can I
diagnose
> the cause of these massive errors?
>
>
>
> Some context:
>
> .        I am running point_stat
>
> .        The command is
>
>  point_stat PYTHON_NUMPY ${OBFILE} ${CONFIG}       \
>
>                                   -v 0                      \
>
>                                   -outdir ${PSOUT}          \
>
>                                   -log
> ${PSOUT}/${VAR_NAME}${WIND_FLAG}_TAU${FCST_HR}_point_stat.log \
>
>                                   -obs_valid_beg 20010101   \
>
>                                   -obs_valid_end 20200101
>
> .        I've tailored my config file so that the vertical levels of
both
> the fcst and obs are relatively near, so this cannot be a vertical
mismatch
> issue
>
> Justin
>
>
>
> Justin Tsu
>
> Marine Meteorology Division
>
> Data Assimilation/Mesoscale Modeling
>
> Building 704 Room 212
>
> Naval Research Laboratory, Code 7531
>
> 7 Grace Hopper Avenue
>
> Monterey, CA 93943-5502
>
>
>
> Ph. (831) 656-4111
>
>
>
>
>


------------------------------------------------
Subject: Absurdly high ME2
From: John Halley Gotway
Time: Tue Nov 12 15:32:02 2019

Justin,

OK, thanks for the updates.  Please let me know if you have any
questions
about using the convert() function in the config file to make those
unit
comparable.  Glad you're figuring out the details.

Thanks,
John

On Tue, Nov 12, 2019 at 3:21 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039 >
>
> Just discovered that I was only computing dew point depression and
not T_d
> which is what my fcst is.  Fixed this (which will also fix the
relative
> humidity).  Also discovered that my raobs airtmp are in Celsius but
my sfc
> obs are in Kelvin (I assumed sfc was also Celsius so was doing
conversion
> on that).  Still not sure why my wind direction and speed have high
errors
> though.
>
> Justin
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Friday, November 8, 2019 10:15 PM
> To: Tsu, Mr. Justin
> Subject: Re: [rt.rap.ucar.edu #93039] Absurdly high ME2
>
> Justin,
>
> The first thing I’d check for is a difference in the units of the
fcst and
> obs data which would lead to large error values.
>
> I’d recommend turning on the matched pair (MPR) output line type....
set it
> to BOTH.  Then look in the “*_mpr.txt” output file at the FCST and
OBS
> columns.  Check the values to see if there’s and obvious mismatch in
units.
>
> If so, you can define a “convert(x)” function in either the fcst or
obs
> dictionaries of the config file to make the units comparable.
>
> Thanks,
> John
>
> On Fri, Nov 8, 2019 at 5:05 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > Fri Nov 08 16:04:58 2019: Request 93039 was acted upon.
> > Transaction: Ticket created by justin.tsu at nrlmry.navy.mil
> >        Queue: met_help
> >      Subject: Absurdly high ME2
> >        Owner: Nobody
> >   Requestors: justin.tsu at nrlmry.navy.mil
> >       Status: new
> >  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039 >
> >
> >
> > I am getting some absurdly high mean square errors for every one
of my
> > variables (wind speed, wind direction, relative humidity, dew
point
> > depression, and air temperature) on each vertical pressure level.
These
> > values range from the thousands to the 10s of millions.  How can I
> diagnose
> > the cause of these massive errors?
> >
> >
> >
> > Some context:
> >
> > .        I am running point_stat
> >
> > .        The command is
> >
> >  point_stat PYTHON_NUMPY ${OBFILE} ${CONFIG}       \
> >
> >                                   -v 0                      \
> >
> >                                   -outdir ${PSOUT}          \
> >
> >                                   -log
> > ${PSOUT}/${VAR_NAME}${WIND_FLAG}_TAU${FCST_HR}_point_stat.log \
> >
> >                                   -obs_valid_beg 20010101   \
> >
> >                                   -obs_valid_end 20200101
> >
> > .        I've tailored my config file so that the vertical levels
of both
> > the fcst and obs are relatively near, so this cannot be a vertical
> mismatch
> > issue
> >
> > Justin
> >
> >
> >
> > Justin Tsu
> >
> > Marine Meteorology Division
> >
> > Data Assimilation/Mesoscale Modeling
> >
> > Building 704 Room 212
> >
> > Naval Research Laboratory, Code 7531
> >
> > 7 Grace Hopper Avenue
> >
> > Monterey, CA 93943-5502
> >
> >
> >
> > Ph. (831) 656-4111
> >
> >
> >
> >
> >
>
>
>

------------------------------------------------
Subject: Absurdly high ME2
From: Tsu, Mr. Justin
Time: Wed Nov 13 12:15:51 2019

Regarding units, I noticed that the units that Elevation of observing
location and height of the actual measurement need to be in MSL for
use in ascii2nc.  What are these units?

Justin

-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Tuesday, November 12, 2019 2:32 PM
To: Tsu, Mr. Justin
Subject: Re: [rt.rap.ucar.edu #93039] Absurdly high ME2

Justin,

OK, thanks for the updates.  Please let me know if you have any
questions
about using the convert() function in the config file to make those
unit
comparable.  Glad you're figuring out the details.

Thanks,
John

On Tue, Nov 12, 2019 at 3:21 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039 >
>
> Just discovered that I was only computing dew point depression and
not T_d
> which is what my fcst is.  Fixed this (which will also fix the
relative
> humidity).  Also discovered that my raobs airtmp are in Celsius but
my sfc
> obs are in Kelvin (I assumed sfc was also Celsius so was doing
conversion
> on that).  Still not sure why my wind direction and speed have high
errors
> though.
>
> Justin
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Friday, November 8, 2019 10:15 PM
> To: Tsu, Mr. Justin
> Subject: Re: [rt.rap.ucar.edu #93039] Absurdly high ME2
>
> Justin,
>
> The first thing I’d check for is a difference in the units of the
fcst and
> obs data which would lead to large error values.
>
> I’d recommend turning on the matched pair (MPR) output line type....
set it
> to BOTH.  Then look in the “*_mpr.txt” output file at the FCST and
OBS
> columns.  Check the values to see if there’s and obvious mismatch in
units.
>
> If so, you can define a “convert(x)” function in either the fcst or
obs
> dictionaries of the config file to make the units comparable.
>
> Thanks,
> John
>
> On Fri, Nov 8, 2019 at 5:05 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > Fri Nov 08 16:04:58 2019: Request 93039 was acted upon.
> > Transaction: Ticket created by justin.tsu at nrlmry.navy.mil
> >        Queue: met_help
> >      Subject: Absurdly high ME2
> >        Owner: Nobody
> >   Requestors: justin.tsu at nrlmry.navy.mil
> >       Status: new
> >  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039 >
> >
> >
> > I am getting some absurdly high mean square errors for every one
of my
> > variables (wind speed, wind direction, relative humidity, dew
point
> > depression, and air temperature) on each vertical pressure level.
These
> > values range from the thousands to the 10s of millions.  How can I
> diagnose
> > the cause of these massive errors?
> >
> >
> >
> > Some context:
> >
> > .        I am running point_stat
> >
> > .        The command is
> >
> >  point_stat PYTHON_NUMPY ${OBFILE} ${CONFIG}       \
> >
> >                                   -v 0                      \
> >
> >                                   -outdir ${PSOUT}          \
> >
> >                                   -log
> > ${PSOUT}/${VAR_NAME}${WIND_FLAG}_TAU${FCST_HR}_point_stat.log \
> >
> >                                   -obs_valid_beg 20010101   \
> >
> >                                   -obs_valid_end 20200101
> >
> > .        I've tailored my config file so that the vertical levels
of both
> > the fcst and obs are relatively near, so this cannot be a vertical
> mismatch
> > issue
> >
> > Justin
> >
> >
> >
> > Justin Tsu
> >
> > Marine Meteorology Division
> >
> > Data Assimilation/Mesoscale Modeling
> >
> > Building 704 Room 212
> >
> > Naval Research Laboratory, Code 7531
> >
> > 7 Grace Hopper Avenue
> >
> > Monterey, CA 93943-5502
> >
> >
> >
> > Ph. (831) 656-4111
> >
> >
> >
> >
> >
>
>
>


------------------------------------------------
Subject: Absurdly high ME2
From: John Halley Gotway
Time: Wed Nov 13 13:25:31 2019

Justin,

Typically elevation in MSL is given in meters.

But MET is not enforcing any unit conventions or doing any unit
conversions
for you.  Point observations do include an elevation of the station
and the
height of the observed values.  I don't think the station elevation is
currently being used in the logic.  But the height of the observations
are
used when vertically interpolating forecasts a specified heights above
ground to the height of the observation.  For example, if your model
output
contains wind speed every 10m above the ground, Point-Stat can
interpolate
that model output to the height of each observation.  It's critical to
make
sure that the definition of the observation heights is consistent with
the
heights of the model output.

They are often either defined relative to MSL or AGL... and could be
in
feet or meters.  So double-check those.

Sorry that the answer isn't more simple!

Thanks,
John

On Wed, Nov 13, 2019 at 12:16 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039 >
>
> Regarding units, I noticed that the units that Elevation of
observing
> location and height of the actual measurement need to be in MSL for
use in
> ascii2nc.  What are these units?
>
> Justin
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Tuesday, November 12, 2019 2:32 PM
> To: Tsu, Mr. Justin
> Subject: Re: [rt.rap.ucar.edu #93039] Absurdly high ME2
>
> Justin,
>
> OK, thanks for the updates.  Please let me know if you have any
questions
> about using the convert() function in the config file to make those
unit
> comparable.  Glad you're figuring out the details.
>
> Thanks,
> John
>
> On Tue, Nov 12, 2019 at 3:21 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039 >
> >
> > Just discovered that I was only computing dew point depression and
not
> T_d
> > which is what my fcst is.  Fixed this (which will also fix the
relative
> > humidity).  Also discovered that my raobs airtmp are in Celsius
but my
> sfc
> > obs are in Kelvin (I assumed sfc was also Celsius so was doing
conversion
> > on that).  Still not sure why my wind direction and speed have
high
> errors
> > though.
> >
> > Justin
> >
> > -----Original Message-----
> > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > Sent: Friday, November 8, 2019 10:15 PM
> > To: Tsu, Mr. Justin
> > Subject: Re: [rt.rap.ucar.edu #93039] Absurdly high ME2
> >
> > Justin,
> >
> > The first thing I’d check for is a difference in the units of the
fcst
> and
> > obs data which would lead to large error values.
> >
> > I’d recommend turning on the matched pair (MPR) output line
type.... set
> it
> > to BOTH.  Then look in the “*_mpr.txt” output file at the FCST and
OBS
> > columns.  Check the values to see if there’s and obvious mismatch
in
> units.
> >
> > If so, you can define a “convert(x)” function in either the fcst
or obs
> > dictionaries of the config file to make the units comparable.
> >
> > Thanks,
> > John
> >
> > On Fri, Nov 8, 2019 at 5:05 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu
> >
> > wrote:
> >
> > >
> > > Fri Nov 08 16:04:58 2019: Request 93039 was acted upon.
> > > Transaction: Ticket created by justin.tsu at nrlmry.navy.mil
> > >        Queue: met_help
> > >      Subject: Absurdly high ME2
> > >        Owner: Nobody
> > >   Requestors: justin.tsu at nrlmry.navy.mil
> > >       Status: new
> > >  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039
> >
> > >
> > >
> > > I am getting some absurdly high mean square errors for every one
of my
> > > variables (wind speed, wind direction, relative humidity, dew
point
> > > depression, and air temperature) on each vertical pressure
level.
> These
> > > values range from the thousands to the 10s of millions.  How can
I
> > diagnose
> > > the cause of these massive errors?
> > >
> > >
> > >
> > > Some context:
> > >
> > > .        I am running point_stat
> > >
> > > .        The command is
> > >
> > >  point_stat PYTHON_NUMPY ${OBFILE} ${CONFIG}       \
> > >
> > >                                   -v 0                      \
> > >
> > >                                   -outdir ${PSOUT}          \
> > >
> > >                                   -log
> > > ${PSOUT}/${VAR_NAME}${WIND_FLAG}_TAU${FCST_HR}_point_stat.log \
> > >
> > >                                   -obs_valid_beg 20010101   \
> > >
> > >                                   -obs_valid_end 20200101
> > >
> > > .        I've tailored my config file so that the vertical
levels of
> both
> > > the fcst and obs are relatively near, so this cannot be a
vertical
> > mismatch
> > > issue
> > >
> > > Justin
> > >
> > >
> > >
> > > Justin Tsu
> > >
> > > Marine Meteorology Division
> > >
> > > Data Assimilation/Mesoscale Modeling
> > >
> > > Building 704 Room 212
> > >
> > > Naval Research Laboratory, Code 7531
> > >
> > > 7 Grace Hopper Avenue
> > >
> > > Monterey, CA 93943-5502
> > >
> > >
> > >
> > > Ph. (831) 656-4111
> > >
> > >
> > >
> > >
> > >
> >
> >
> >
>
>
>

------------------------------------------------
Subject: Absurdly high ME2
From: Tsu, Mr. Justin
Time: Wed Nov 13 13:35:57 2019

It's okay! Just for clarification, all of my model output is on sigma
(hPa) pressure levels.  How does Point-stat know to use the pressure
level column in the observation text file to attach a location value
to the observation when comparing to pressure located model fields?
Justin

-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Wednesday, November 13, 2019 12:26 PM
To: Tsu, Mr. Justin
Subject: Re: [rt.rap.ucar.edu #93039] Absurdly high ME2

Justin,

Typically elevation in MSL is given in meters.

But MET is not enforcing any unit conventions or doing any unit
conversions
for you.  Point observations do include an elevation of the station
and the
height of the observed values.  I don't think the station elevation is
currently being used in the logic.  But the height of the observations
are
used when vertically interpolating forecasts a specified heights above
ground to the height of the observation.  For example, if your model
output
contains wind speed every 10m above the ground, Point-Stat can
interpolate
that model output to the height of each observation.  It's critical to
make
sure that the definition of the observation heights is consistent with
the
heights of the model output.

They are often either defined relative to MSL or AGL... and could be
in
feet or meters.  So double-check those.

Sorry that the answer isn't more simple!

Thanks,
John

On Wed, Nov 13, 2019 at 12:16 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039 >
>
> Regarding units, I noticed that the units that Elevation of
observing
> location and height of the actual measurement need to be in MSL for
use in
> ascii2nc.  What are these units?
>
> Justin
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Tuesday, November 12, 2019 2:32 PM
> To: Tsu, Mr. Justin
> Subject: Re: [rt.rap.ucar.edu #93039] Absurdly high ME2
>
> Justin,
>
> OK, thanks for the updates.  Please let me know if you have any
questions
> about using the convert() function in the config file to make those
unit
> comparable.  Glad you're figuring out the details.
>
> Thanks,
> John
>
> On Tue, Nov 12, 2019 at 3:21 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039 >
> >
> > Just discovered that I was only computing dew point depression and
not
> T_d
> > which is what my fcst is.  Fixed this (which will also fix the
relative
> > humidity).  Also discovered that my raobs airtmp are in Celsius
but my
> sfc
> > obs are in Kelvin (I assumed sfc was also Celsius so was doing
conversion
> > on that).  Still not sure why my wind direction and speed have
high
> errors
> > though.
> >
> > Justin
> >
> > -----Original Message-----
> > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > Sent: Friday, November 8, 2019 10:15 PM
> > To: Tsu, Mr. Justin
> > Subject: Re: [rt.rap.ucar.edu #93039] Absurdly high ME2
> >
> > Justin,
> >
> > The first thing I’d check for is a difference in the units of the
fcst
> and
> > obs data which would lead to large error values.
> >
> > I’d recommend turning on the matched pair (MPR) output line
type.... set
> it
> > to BOTH.  Then look in the “*_mpr.txt” output file at the FCST and
OBS
> > columns.  Check the values to see if there’s and obvious mismatch
in
> units.
> >
> > If so, you can define a “convert(x)” function in either the fcst
or obs
> > dictionaries of the config file to make the units comparable.
> >
> > Thanks,
> > John
> >
> > On Fri, Nov 8, 2019 at 5:05 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu
> >
> > wrote:
> >
> > >
> > > Fri Nov 08 16:04:58 2019: Request 93039 was acted upon.
> > > Transaction: Ticket created by justin.tsu at nrlmry.navy.mil
> > >        Queue: met_help
> > >      Subject: Absurdly high ME2
> > >        Owner: Nobody
> > >   Requestors: justin.tsu at nrlmry.navy.mil
> > >       Status: new
> > >  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039
> >
> > >
> > >
> > > I am getting some absurdly high mean square errors for every one
of my
> > > variables (wind speed, wind direction, relative humidity, dew
point
> > > depression, and air temperature) on each vertical pressure
level.
> These
> > > values range from the thousands to the 10s of millions.  How can
I
> > diagnose
> > > the cause of these massive errors?
> > >
> > >
> > >
> > > Some context:
> > >
> > > .        I am running point_stat
> > >
> > > .        The command is
> > >
> > >  point_stat PYTHON_NUMPY ${OBFILE} ${CONFIG}       \
> > >
> > >                                   -v 0                      \
> > >
> > >                                   -outdir ${PSOUT}          \
> > >
> > >                                   -log
> > > ${PSOUT}/${VAR_NAME}${WIND_FLAG}_TAU${FCST_HR}_point_stat.log \
> > >
> > >                                   -obs_valid_beg 20010101   \
> > >
> > >                                   -obs_valid_end 20200101
> > >
> > > .        I've tailored my config file so that the vertical
levels of
> both
> > > the fcst and obs are relatively near, so this cannot be a
vertical
> > mismatch
> > > issue
> > >
> > > Justin
> > >
> > >
> > >
> > > Justin Tsu
> > >
> > > Marine Meteorology Division
> > >
> > > Data Assimilation/Mesoscale Modeling
> > >
> > > Building 704 Room 212
> > >
> > > Naval Research Laboratory, Code 7531
> > >
> > > 7 Grace Hopper Avenue
> > >
> > > Monterey, CA 93943-5502
> > >
> > >
> > >
> > > Ph. (831) 656-4111
> > >
> > >
> > >
> > >
> > >
> >
> >
> >
>
>
>


------------------------------------------------
Subject: Absurdly high ME2
From: John Halley Gotway
Time: Wed Nov 13 13:59:47 2019

Justin,

Point-Stat looks in the config file at the user-specified level to be
verified.
For example, let's say you're running Point-Stat to verify wind speed
at 2
levels of model output, 850mb pressure level and 100 meters, a shown
below.

fcst = {
   field = [ { name = "WIND"; level = "P850"; },
                { name = "WIND"; level = "Z100"; }
   ];
}

obs = {
   field = [ { name = "WIND"; level = "P840-860"; },
                { name = "WIND"; level = "Z100"; }
   ];
}

For the first task, Point-Stat will check the pressure level of the
obs since the P specifies a pressure level (P840-860).
For the second task, it will check the height of the obs since the Z
specifies a height (Z100).

John

On Wed, Nov 13, 2019 at 1:35 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039 >
>
> It's okay! Just for clarification, all of my model output is on
sigma
> (hPa) pressure levels.  How does Point-stat know to use the pressure
level
> column in the observation text file to attach a location value to
the
> observation when comparing to pressure located model fields?
> Justin
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Wednesday, November 13, 2019 12:26 PM
> To: Tsu, Mr. Justin
> Subject: Re: [rt.rap.ucar.edu #93039] Absurdly high ME2
>
> Justin,
>
> Typically elevation in MSL is given in meters.
>
> But MET is not enforcing any unit conventions or doing any unit
conversions
> for you.  Point observations do include an elevation of the station
and the
> height of the observed values.  I don't think the station elevation
is
> currently being used in the logic.  But the height of the
observations are
> used when vertically interpolating forecasts a specified heights
above
> ground to the height of the observation.  For example, if your model
output
> contains wind speed every 10m above the ground, Point-Stat can
interpolate
> that model output to the height of each observation.  It's critical
to make
> sure that the definition of the observation heights is consistent
with the
> heights of the model output.
>
> They are often either defined relative to MSL or AGL... and could be
in
> feet or meters.  So double-check those.
>
> Sorry that the answer isn't more simple!
>
> Thanks,
> John
>
> On Wed, Nov 13, 2019 at 12:16 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu
> >
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039 >
> >
> > Regarding units, I noticed that the units that Elevation of
observing
> > location and height of the actual measurement need to be in MSL
for use
> in
> > ascii2nc.  What are these units?
> >
> > Justin
> >
> > -----Original Message-----
> > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > Sent: Tuesday, November 12, 2019 2:32 PM
> > To: Tsu, Mr. Justin
> > Subject: Re: [rt.rap.ucar.edu #93039] Absurdly high ME2
> >
> > Justin,
> >
> > OK, thanks for the updates.  Please let me know if you have any
questions
> > about using the convert() function in the config file to make
those unit
> > comparable.  Glad you're figuring out the details.
> >
> > Thanks,
> > John
> >
> > On Tue, Nov 12, 2019 at 3:21 PM Tsu, Mr. Justin via RT <
> met_help at ucar.edu>
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039 >
> > >
> > > Just discovered that I was only computing dew point depression
and not
> > T_d
> > > which is what my fcst is.  Fixed this (which will also fix the
relative
> > > humidity).  Also discovered that my raobs airtmp are in Celsius
but my
> > sfc
> > > obs are in Kelvin (I assumed sfc was also Celsius so was doing
> conversion
> > > on that).  Still not sure why my wind direction and speed have
high
> > errors
> > > though.
> > >
> > > Justin
> > >
> > > -----Original Message-----
> > > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > > Sent: Friday, November 8, 2019 10:15 PM
> > > To: Tsu, Mr. Justin
> > > Subject: Re: [rt.rap.ucar.edu #93039] Absurdly high ME2
> > >
> > > Justin,
> > >
> > > The first thing I’d check for is a difference in the units of
the fcst
> > and
> > > obs data which would lead to large error values.
> > >
> > > I’d recommend turning on the matched pair (MPR) output line
type....
> set
> > it
> > > to BOTH.  Then look in the “*_mpr.txt” output file at the FCST
and OBS
> > > columns.  Check the values to see if there’s and obvious
mismatch in
> > units.
> > >
> > > If so, you can define a “convert(x)” function in either the fcst
or obs
> > > dictionaries of the config file to make the units comparable.
> > >
> > > Thanks,
> > > John
> > >
> > > On Fri, Nov 8, 2019 at 5:05 PM Tsu, Mr. Justin via RT <
> met_help at ucar.edu
> > >
> > > wrote:
> > >
> > > >
> > > > Fri Nov 08 16:04:58 2019: Request 93039 was acted upon.
> > > > Transaction: Ticket created by justin.tsu at nrlmry.navy.mil
> > > >        Queue: met_help
> > > >      Subject: Absurdly high ME2
> > > >        Owner: Nobody
> > > >   Requestors: justin.tsu at nrlmry.navy.mil
> > > >       Status: new
> > > >  Ticket <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039
> > >
> > > >
> > > >
> > > > I am getting some absurdly high mean square errors for every
one of
> my
> > > > variables (wind speed, wind direction, relative humidity, dew
point
> > > > depression, and air temperature) on each vertical pressure
level.
> > These
> > > > values range from the thousands to the 10s of millions.  How
can I
> > > diagnose
> > > > the cause of these massive errors?
> > > >
> > > >
> > > >
> > > > Some context:
> > > >
> > > > .        I am running point_stat
> > > >
> > > > .        The command is
> > > >
> > > >  point_stat PYTHON_NUMPY ${OBFILE} ${CONFIG}       \
> > > >
> > > >                                   -v 0                      \
> > > >
> > > >                                   -outdir ${PSOUT}          \
> > > >
> > > >                                   -log
> > > > ${PSOUT}/${VAR_NAME}${WIND_FLAG}_TAU${FCST_HR}_point_stat.log
\
> > > >
> > > >                                   -obs_valid_beg 20010101   \
> > > >
> > > >                                   -obs_valid_end 20200101
> > > >
> > > > .        I've tailored my config file so that the vertical
levels of
> > both
> > > > the fcst and obs are relatively near, so this cannot be a
vertical
> > > mismatch
> > > > issue
> > > >
> > > > Justin
> > > >
> > > >
> > > >
> > > > Justin Tsu
> > > >
> > > > Marine Meteorology Division
> > > >
> > > > Data Assimilation/Mesoscale Modeling
> > > >
> > > > Building 704 Room 212
> > > >
> > > > Naval Research Laboratory, Code 7531
> > > >
> > > > 7 Grace Hopper Avenue
> > > >
> > > > Monterey, CA 93943-5502
> > > >
> > > >
> > > >
> > > > Ph. (831) 656-4111
> > > >
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> > >
> >
> >
> >
>
>
>

------------------------------------------------
Subject: Absurdly high ME2
From: Tsu, Mr. Justin
Time: Wed Nov 13 14:07:33 2019

Got it,

So since I am using python embedding for my model field, it will
ultimately be up to what vertical level I specify my obs that will
determine the type of comparison.  For example:

fcst = {
   field = [{name = "/users/tsu/MET/work/PY_SRC/read_NRL_binary.py
${DATA_DIR}/${VAR_NAME}_pre_000.10_0000.0_${GRD}_${INIT
_TIME}_${FCST_HR}_fcstfld ${WIND_FLAG}";}];
}

obs = {
    field = [{name = "${GRIB_NAME}";level = ["P0.0-0.15"];}];
}

This comparison is correct because the highlighted portion in my
forecast dictionary indicates this model field is at 0.10 hPa and so I
specify all obs that fall within 0.0 and 0.15 hPA will be compared
against this model field.

The following is incorrect (BUT will still run):

fcst = {
   field = [{name = "/users/tsu/MET/work/PY_SRC/read_NRL_binary.py
${DATA_DIR}/${VAR_NAME}_pre_ 000900_000000_${GRD}_${INIT
_TIME}_${FCST_HR}_fcstfld ${WIND_FLAG}";}];
}

obs = {
    field = [{name = "${GRIB_NAME}";level = ["P0.0-0.15"];}];
}

This is an incorrect comparison but will still run.  It is comparing a
model field at 900 hPa (as seen highlighted) to obs from 0 and 0.15
hPa.


Is everything I just said correct?

Justin

-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Wednesday, November 13, 2019 1:00 PM
To: Tsu, Mr. Justin
Subject: Re: [rt.rap.ucar.edu #93039] Absurdly high ME2

Justin,

Point-Stat looks in the config file at the user-specified level to be
verified.
For example, let's say you're running Point-Stat to verify wind speed
at 2
levels of model output, 850mb pressure level and 100 meters, a shown
below.

fcst = {
   field = [ { name = "WIND"; level = "P850"; },
                { name = "WIND"; level = "Z100"; }
   ];
}

obs = {
   field = [ { name = "WIND"; level = "P840-860"; },
                { name = "WIND"; level = "Z100"; }
   ];
}

For the first task, Point-Stat will check the pressure level of the
obs since the P specifies a pressure level (P840-860).
For the second task, it will check the height of the obs since the Z
specifies a height (Z100).

John

On Wed, Nov 13, 2019 at 1:35 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039 >
>
> It's okay! Just for clarification, all of my model output is on
sigma
> (hPa) pressure levels.  How does Point-stat know to use the pressure
level
> column in the observation text file to attach a location value to
the
> observation when comparing to pressure located model fields?
> Justin
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Wednesday, November 13, 2019 12:26 PM
> To: Tsu, Mr. Justin
> Subject: Re: [rt.rap.ucar.edu #93039] Absurdly high ME2
>
> Justin,
>
> Typically elevation in MSL is given in meters.
>
> But MET is not enforcing any unit conventions or doing any unit
conversions
> for you.  Point observations do include an elevation of the station
and the
> height of the observed values.  I don't think the station elevation
is
> currently being used in the logic.  But the height of the
observations are
> used when vertically interpolating forecasts a specified heights
above
> ground to the height of the observation.  For example, if your model
output
> contains wind speed every 10m above the ground, Point-Stat can
interpolate
> that model output to the height of each observation.  It's critical
to make
> sure that the definition of the observation heights is consistent
with the
> heights of the model output.
>
> They are often either defined relative to MSL or AGL... and could be
in
> feet or meters.  So double-check those.
>
> Sorry that the answer isn't more simple!
>
> Thanks,
> John
>
> On Wed, Nov 13, 2019 at 12:16 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu
> >
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039 >
> >
> > Regarding units, I noticed that the units that Elevation of
observing
> > location and height of the actual measurement need to be in MSL
for use
> in
> > ascii2nc.  What are these units?
> >
> > Justin
> >
> > -----Original Message-----
> > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > Sent: Tuesday, November 12, 2019 2:32 PM
> > To: Tsu, Mr. Justin
> > Subject: Re: [rt.rap.ucar.edu #93039] Absurdly high ME2
> >
> > Justin,
> >
> > OK, thanks for the updates.  Please let me know if you have any
questions
> > about using the convert() function in the config file to make
those unit
> > comparable.  Glad you're figuring out the details.
> >
> > Thanks,
> > John
> >
> > On Tue, Nov 12, 2019 at 3:21 PM Tsu, Mr. Justin via RT <
> met_help at ucar.edu>
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039 >
> > >
> > > Just discovered that I was only computing dew point depression
and not
> > T_d
> > > which is what my fcst is.  Fixed this (which will also fix the
relative
> > > humidity).  Also discovered that my raobs airtmp are in Celsius
but my
> > sfc
> > > obs are in Kelvin (I assumed sfc was also Celsius so was doing
> conversion
> > > on that).  Still not sure why my wind direction and speed have
high
> > errors
> > > though.
> > >
> > > Justin
> > >
> > > -----Original Message-----
> > > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > > Sent: Friday, November 8, 2019 10:15 PM
> > > To: Tsu, Mr. Justin
> > > Subject: Re: [rt.rap.ucar.edu #93039] Absurdly high ME2
> > >
> > > Justin,
> > >
> > > The first thing I’d check for is a difference in the units of
the fcst
> > and
> > > obs data which would lead to large error values.
> > >
> > > I’d recommend turning on the matched pair (MPR) output line
type....
> set
> > it
> > > to BOTH.  Then look in the “*_mpr.txt” output file at the FCST
and OBS
> > > columns.  Check the values to see if there’s and obvious
mismatch in
> > units.
> > >
> > > If so, you can define a “convert(x)” function in either the fcst
or obs
> > > dictionaries of the config file to make the units comparable.
> > >
> > > Thanks,
> > > John
> > >
> > > On Fri, Nov 8, 2019 at 5:05 PM Tsu, Mr. Justin via RT <
> met_help at ucar.edu
> > >
> > > wrote:
> > >
> > > >
> > > > Fri Nov 08 16:04:58 2019: Request 93039 was acted upon.
> > > > Transaction: Ticket created by justin.tsu at nrlmry.navy.mil
> > > >        Queue: met_help
> > > >      Subject: Absurdly high ME2
> > > >        Owner: Nobody
> > > >   Requestors: justin.tsu at nrlmry.navy.mil
> > > >       Status: new
> > > >  Ticket <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039
> > >
> > > >
> > > >
> > > > I am getting some absurdly high mean square errors for every
one of
> my
> > > > variables (wind speed, wind direction, relative humidity, dew
point
> > > > depression, and air temperature) on each vertical pressure
level.
> > These
> > > > values range from the thousands to the 10s of millions.  How
can I
> > > diagnose
> > > > the cause of these massive errors?
> > > >
> > > >
> > > >
> > > > Some context:
> > > >
> > > > .        I am running point_stat
> > > >
> > > > .        The command is
> > > >
> > > >  point_stat PYTHON_NUMPY ${OBFILE} ${CONFIG}       \
> > > >
> > > >                                   -v 0                      \
> > > >
> > > >                                   -outdir ${PSOUT}          \
> > > >
> > > >                                   -log
> > > > ${PSOUT}/${VAR_NAME}${WIND_FLAG}_TAU${FCST_HR}_point_stat.log
\
> > > >
> > > >                                   -obs_valid_beg 20010101   \
> > > >
> > > >                                   -obs_valid_end 20200101
> > > >
> > > > .        I've tailored my config file so that the vertical
levels of
> > both
> > > > the fcst and obs are relatively near, so this cannot be a
vertical
> > > mismatch
> > > > issue
> > > >
> > > > Justin
> > > >
> > > >
> > > >
> > > > Justin Tsu
> > > >
> > > > Marine Meteorology Division
> > > >
> > > > Data Assimilation/Mesoscale Modeling
> > > >
> > > > Building 704 Room 212
> > > >
> > > > Naval Research Laboratory, Code 7531
> > > >
> > > > 7 Grace Hopper Avenue
> > > >
> > > > Monterey, CA 93943-5502
> > > >
> > > >
> > > >
> > > > Ph. (831) 656-4111
> > > >
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> > >
> >
> >
> >
>
>
>


------------------------------------------------
Subject: Absurdly high ME2
From: John Halley Gotway
Time: Wed Nov 13 14:44:09 2019

Justin,

Yes, that's right.  Point-Stat will perform the comparison you request
in
the config file.  And yes, I agree that the first comparison you
listed
makes sense while the second comparison makes very little sense.  You
can
see the level info listed in the FCST_LEVEL and OBS_LEVEL output
columns.

But having the ability to specify the level info separately for the
forecast and observation is very useful.  For example, when verifying
a
forecast at 850mb, you may choose to use observations exactly at that
level
(P850), in a small range around that level (P848-852), or in a large
range
around that level (P800-900).  The user can decide what tolerance
makes
sense to the evaluation at hand.

Thanks,
John

On Wed, Nov 13, 2019 at 2:08 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039 >
>
> Got it,
>
> So since I am using python embedding for my model field, it will
> ultimately be up to what vertical level I specify my obs that will
> determine the type of comparison.  For example:
>
> fcst = {
>    field = [{name = "/users/tsu/MET/work/PY_SRC/read_NRL_binary.py
> ${DATA_DIR}/${VAR_NAME}_pre_000.10_0000.0_${GRD}_${INIT
> _TIME}_${FCST_HR}_fcstfld ${WIND_FLAG}";}];
> }
>
> obs = {
>     field = [{name = "${GRIB_NAME}";level = ["P0.0-0.15"];}];
> }
>
> This comparison is correct because the highlighted portion in my
forecast
> dictionary indicates this model field is at 0.10 hPa and so I
specify all
> obs that fall within 0.0 and 0.15 hPA will be compared against this
model
> field.
>
> The following is incorrect (BUT will still run):
>
> fcst = {
>    field = [{name = "/users/tsu/MET/work/PY_SRC/read_NRL_binary.py
> ${DATA_DIR}/${VAR_NAME}_pre_ 000900_000000_${GRD}_${INIT
> _TIME}_${FCST_HR}_fcstfld ${WIND_FLAG}";}];
> }
>
> obs = {
>     field = [{name = "${GRIB_NAME}";level = ["P0.0-0.15"];}];
> }
>
> This is an incorrect comparison but will still run.  It is comparing
a
> model field at 900 hPa (as seen highlighted) to obs from 0 and 0.15
hPa.
>
>
> Is everything I just said correct?
>
> Justin
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Wednesday, November 13, 2019 1:00 PM
> To: Tsu, Mr. Justin
> Subject: Re: [rt.rap.ucar.edu #93039] Absurdly high ME2
>
> Justin,
>
> Point-Stat looks in the config file at the user-specified level to
be
> verified.
> For example, let's say you're running Point-Stat to verify wind
speed at 2
> levels of model output, 850mb pressure level and 100 meters, a shown
below.
>
> fcst = {
>    field = [ { name = "WIND"; level = "P850"; },
>                 { name = "WIND"; level = "Z100"; }
>    ];
> }
>
> obs = {
>    field = [ { name = "WIND"; level = "P840-860"; },
>                 { name = "WIND"; level = "Z100"; }
>    ];
> }
>
> For the first task, Point-Stat will check the pressure level of the
> obs since the P specifies a pressure level (P840-860).
> For the second task, it will check the height of the obs since the Z
> specifies a height (Z100).
>
> John
>
> On Wed, Nov 13, 2019 at 1:35 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039 >
> >
> > It's okay! Just for clarification, all of my model output is on
sigma
> > (hPa) pressure levels.  How does Point-stat know to use the
pressure
> level
> > column in the observation text file to attach a location value to
the
> > observation when comparing to pressure located model fields?
> > Justin
> >
> > -----Original Message-----
> > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > Sent: Wednesday, November 13, 2019 12:26 PM
> > To: Tsu, Mr. Justin
> > Subject: Re: [rt.rap.ucar.edu #93039] Absurdly high ME2
> >
> > Justin,
> >
> > Typically elevation in MSL is given in meters.
> >
> > But MET is not enforcing any unit conventions or doing any unit
> conversions
> > for you.  Point observations do include an elevation of the
station and
> the
> > height of the observed values.  I don't think the station
elevation is
> > currently being used in the logic.  But the height of the
observations
> are
> > used when vertically interpolating forecasts a specified heights
above
> > ground to the height of the observation.  For example, if your
model
> output
> > contains wind speed every 10m above the ground, Point-Stat can
> interpolate
> > that model output to the height of each observation.  It's
critical to
> make
> > sure that the definition of the observation heights is consistent
with
> the
> > heights of the model output.
> >
> > They are often either defined relative to MSL or AGL... and could
be in
> > feet or meters.  So double-check those.
> >
> > Sorry that the answer isn't more simple!
> >
> > Thanks,
> > John
> >
> > On Wed, Nov 13, 2019 at 12:16 PM Tsu, Mr. Justin via RT <
> met_help at ucar.edu
> > >
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039 >
> > >
> > > Regarding units, I noticed that the units that Elevation of
observing
> > > location and height of the actual measurement need to be in MSL
for use
> > in
> > > ascii2nc.  What are these units?
> > >
> > > Justin
> > >
> > > -----Original Message-----
> > > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > > Sent: Tuesday, November 12, 2019 2:32 PM
> > > To: Tsu, Mr. Justin
> > > Subject: Re: [rt.rap.ucar.edu #93039] Absurdly high ME2
> > >
> > > Justin,
> > >
> > > OK, thanks for the updates.  Please let me know if you have any
> questions
> > > about using the convert() function in the config file to make
those
> unit
> > > comparable.  Glad you're figuring out the details.
> > >
> > > Thanks,
> > > John
> > >
> > > On Tue, Nov 12, 2019 at 3:21 PM Tsu, Mr. Justin via RT <
> > met_help at ucar.edu>
> > > wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039
>
> > > >
> > > > Just discovered that I was only computing dew point depression
and
> not
> > > T_d
> > > > which is what my fcst is.  Fixed this (which will also fix the
> relative
> > > > humidity).  Also discovered that my raobs airtmp are in
Celsius but
> my
> > > sfc
> > > > obs are in Kelvin (I assumed sfc was also Celsius so was doing
> > conversion
> > > > on that).  Still not sure why my wind direction and speed have
high
> > > errors
> > > > though.
> > > >
> > > > Justin
> > > >
> > > > -----Original Message-----
> > > > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > > > Sent: Friday, November 8, 2019 10:15 PM
> > > > To: Tsu, Mr. Justin
> > > > Subject: Re: [rt.rap.ucar.edu #93039] Absurdly high ME2
> > > >
> > > > Justin,
> > > >
> > > > The first thing I’d check for is a difference in the units of
the
> fcst
> > > and
> > > > obs data which would lead to large error values.
> > > >
> > > > I’d recommend turning on the matched pair (MPR) output line
type....
> > set
> > > it
> > > > to BOTH.  Then look in the “*_mpr.txt” output file at the FCST
and
> OBS
> > > > columns.  Check the values to see if there’s and obvious
mismatch in
> > > units.
> > > >
> > > > If so, you can define a “convert(x)” function in either the
fcst or
> obs
> > > > dictionaries of the config file to make the units comparable.
> > > >
> > > > Thanks,
> > > > John
> > > >
> > > > On Fri, Nov 8, 2019 at 5:05 PM Tsu, Mr. Justin via RT <
> > met_help at ucar.edu
> > > >
> > > > wrote:
> > > >
> > > > >
> > > > > Fri Nov 08 16:04:58 2019: Request 93039 was acted upon.
> > > > > Transaction: Ticket created by justin.tsu at nrlmry.navy.mil
> > > > >        Queue: met_help
> > > > >      Subject: Absurdly high ME2
> > > > >        Owner: Nobody
> > > > >   Requestors: justin.tsu at nrlmry.navy.mil
> > > > >       Status: new
> > > > >  Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039
> > > >
> > > > >
> > > > >
> > > > > I am getting some absurdly high mean square errors for every
one of
> > my
> > > > > variables (wind speed, wind direction, relative humidity,
dew point
> > > > > depression, and air temperature) on each vertical pressure
level.
> > > These
> > > > > values range from the thousands to the 10s of millions.  How
can I
> > > > diagnose
> > > > > the cause of these massive errors?
> > > > >
> > > > >
> > > > >
> > > > > Some context:
> > > > >
> > > > > .        I am running point_stat
> > > > >
> > > > > .        The command is
> > > > >
> > > > >  point_stat PYTHON_NUMPY ${OBFILE} ${CONFIG}       \
> > > > >
> > > > >                                   -v 0
\
> > > > >
> > > > >                                   -outdir ${PSOUT}
\
> > > > >
> > > > >                                   -log
> > > > >
${PSOUT}/${VAR_NAME}${WIND_FLAG}_TAU${FCST_HR}_point_stat.log \
> > > > >
> > > > >                                   -obs_valid_beg 20010101
\
> > > > >
> > > > >                                   -obs_valid_end 20200101
> > > > >
> > > > > .        I've tailored my config file so that the vertical
levels
> of
> > > both
> > > > > the fcst and obs are relatively near, so this cannot be a
vertical
> > > > mismatch
> > > > > issue
> > > > >
> > > > > Justin
> > > > >
> > > > >
> > > > >
> > > > > Justin Tsu
> > > > >
> > > > > Marine Meteorology Division
> > > > >
> > > > > Data Assimilation/Mesoscale Modeling
> > > > >
> > > > > Building 704 Room 212
> > > > >
> > > > > Naval Research Laboratory, Code 7531
> > > > >
> > > > > 7 Grace Hopper Avenue
> > > > >
> > > > > Monterey, CA 93943-5502
> > > > >
> > > > >
> > > > >
> > > > > Ph. (831) 656-4111
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > >
> > >
> > >
> > >
> >
> >
> >
>
>
>

------------------------------------------------
Subject: Absurdly high ME2
From: Tsu, Mr. Justin
Time: Tue Nov 19 12:02:15 2019

Thanks John,

I have another question that I just discovered from my MPR files.  I
notice that when verifying raobs, my mpr only produces one sounding. I
get multiple lines (for the different elevations) in the file but they
all have the same ob location, indicating that point_stat only found
one particular observation that matched - but I am not particularly
sure what matched.  If point_stat only looks for obs that match the
time of the fcst field (which if this is the case, I should be getting
A LOT of matches because a majority of my obs are the same date as the
fcst field) then filters out those matches for obs that match the
elevation of the fcst field, then finally interpolates this subset of
matches to the location of the nearest model point, I feel like I
should be getting much more than one sounding.  Any ideas on how to
debug this?

Justin

-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Wednesday, November 13, 2019 1:44 PM
To: Tsu, Mr. Justin
Subject: Re: [rt.rap.ucar.edu #93039] Absurdly high ME2

Justin,

Yes, that's right.  Point-Stat will perform the comparison you request
in
the config file.  And yes, I agree that the first comparison you
listed
makes sense while the second comparison makes very little sense.  You
can
see the level info listed in the FCST_LEVEL and OBS_LEVEL output
columns.

But having the ability to specify the level info separately for the
forecast and observation is very useful.  For example, when verifying
a
forecast at 850mb, you may choose to use observations exactly at that
level
(P850), in a small range around that level (P848-852), or in a large
range
around that level (P800-900).  The user can decide what tolerance
makes
sense to the evaluation at hand.

Thanks,
John

On Wed, Nov 13, 2019 at 2:08 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039 >
>
> Got it,
>
> So since I am using python embedding for my model field, it will
> ultimately be up to what vertical level I specify my obs that will
> determine the type of comparison.  For example:
>
> fcst = {
>    field = [{name = "/users/tsu/MET/work/PY_SRC/read_NRL_binary.py
> ${DATA_DIR}/${VAR_NAME}_pre_000.10_0000.0_${GRD}_${INIT
> _TIME}_${FCST_HR}_fcstfld ${WIND_FLAG}";}];
> }
>
> obs = {
>     field = [{name = "${GRIB_NAME}";level = ["P0.0-0.15"];}];
> }
>
> This comparison is correct because the highlighted portion in my
forecast
> dictionary indicates this model field is at 0.10 hPa and so I
specify all
> obs that fall within 0.0 and 0.15 hPA will be compared against this
model
> field.
>
> The following is incorrect (BUT will still run):
>
> fcst = {
>    field = [{name = "/users/tsu/MET/work/PY_SRC/read_NRL_binary.py
> ${DATA_DIR}/${VAR_NAME}_pre_ 000900_000000_${GRD}_${INIT
> _TIME}_${FCST_HR}_fcstfld ${WIND_FLAG}";}];
> }
>
> obs = {
>     field = [{name = "${GRIB_NAME}";level = ["P0.0-0.15"];}];
> }
>
> This is an incorrect comparison but will still run.  It is comparing
a
> model field at 900 hPa (as seen highlighted) to obs from 0 and 0.15
hPa.
>
>
> Is everything I just said correct?
>
> Justin
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Wednesday, November 13, 2019 1:00 PM
> To: Tsu, Mr. Justin
> Subject: Re: [rt.rap.ucar.edu #93039] Absurdly high ME2
>
> Justin,
>
> Point-Stat looks in the config file at the user-specified level to
be
> verified.
> For example, let's say you're running Point-Stat to verify wind
speed at 2
> levels of model output, 850mb pressure level and 100 meters, a shown
below.
>
> fcst = {
>    field = [ { name = "WIND"; level = "P850"; },
>                 { name = "WIND"; level = "Z100"; }
>    ];
> }
>
> obs = {
>    field = [ { name = "WIND"; level = "P840-860"; },
>                 { name = "WIND"; level = "Z100"; }
>    ];
> }
>
> For the first task, Point-Stat will check the pressure level of the
> obs since the P specifies a pressure level (P840-860).
> For the second task, it will check the height of the obs since the Z
> specifies a height (Z100).
>
> John
>
> On Wed, Nov 13, 2019 at 1:35 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039 >
> >
> > It's okay! Just for clarification, all of my model output is on
sigma
> > (hPa) pressure levels.  How does Point-stat know to use the
pressure
> level
> > column in the observation text file to attach a location value to
the
> > observation when comparing to pressure located model fields?
> > Justin
> >
> > -----Original Message-----
> > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > Sent: Wednesday, November 13, 2019 12:26 PM
> > To: Tsu, Mr. Justin
> > Subject: Re: [rt.rap.ucar.edu #93039] Absurdly high ME2
> >
> > Justin,
> >
> > Typically elevation in MSL is given in meters.
> >
> > But MET is not enforcing any unit conventions or doing any unit
> conversions
> > for you.  Point observations do include an elevation of the
station and
> the
> > height of the observed values.  I don't think the station
elevation is
> > currently being used in the logic.  But the height of the
observations
> are
> > used when vertically interpolating forecasts a specified heights
above
> > ground to the height of the observation.  For example, if your
model
> output
> > contains wind speed every 10m above the ground, Point-Stat can
> interpolate
> > that model output to the height of each observation.  It's
critical to
> make
> > sure that the definition of the observation heights is consistent
with
> the
> > heights of the model output.
> >
> > They are often either defined relative to MSL or AGL... and could
be in
> > feet or meters.  So double-check those.
> >
> > Sorry that the answer isn't more simple!
> >
> > Thanks,
> > John
> >
> > On Wed, Nov 13, 2019 at 12:16 PM Tsu, Mr. Justin via RT <
> met_help at ucar.edu
> > >
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039 >
> > >
> > > Regarding units, I noticed that the units that Elevation of
observing
> > > location and height of the actual measurement need to be in MSL
for use
> > in
> > > ascii2nc.  What are these units?
> > >
> > > Justin
> > >
> > > -----Original Message-----
> > > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > > Sent: Tuesday, November 12, 2019 2:32 PM
> > > To: Tsu, Mr. Justin
> > > Subject: Re: [rt.rap.ucar.edu #93039] Absurdly high ME2
> > >
> > > Justin,
> > >
> > > OK, thanks for the updates.  Please let me know if you have any
> questions
> > > about using the convert() function in the config file to make
those
> unit
> > > comparable.  Glad you're figuring out the details.
> > >
> > > Thanks,
> > > John
> > >
> > > On Tue, Nov 12, 2019 at 3:21 PM Tsu, Mr. Justin via RT <
> > met_help at ucar.edu>
> > > wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039
>
> > > >
> > > > Just discovered that I was only computing dew point depression
and
> not
> > > T_d
> > > > which is what my fcst is.  Fixed this (which will also fix the
> relative
> > > > humidity).  Also discovered that my raobs airtmp are in
Celsius but
> my
> > > sfc
> > > > obs are in Kelvin (I assumed sfc was also Celsius so was doing
> > conversion
> > > > on that).  Still not sure why my wind direction and speed have
high
> > > errors
> > > > though.
> > > >
> > > > Justin
> > > >
> > > > -----Original Message-----
> > > > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > > > Sent: Friday, November 8, 2019 10:15 PM
> > > > To: Tsu, Mr. Justin
> > > > Subject: Re: [rt.rap.ucar.edu #93039] Absurdly high ME2
> > > >
> > > > Justin,
> > > >
> > > > The first thing I’d check for is a difference in the units of
the
> fcst
> > > and
> > > > obs data which would lead to large error values.
> > > >
> > > > I’d recommend turning on the matched pair (MPR) output line
type....
> > set
> > > it
> > > > to BOTH.  Then look in the “*_mpr.txt” output file at the FCST
and
> OBS
> > > > columns.  Check the values to see if there’s and obvious
mismatch in
> > > units.
> > > >
> > > > If so, you can define a “convert(x)” function in either the
fcst or
> obs
> > > > dictionaries of the config file to make the units comparable.
> > > >
> > > > Thanks,
> > > > John
> > > >
> > > > On Fri, Nov 8, 2019 at 5:05 PM Tsu, Mr. Justin via RT <
> > met_help at ucar.edu
> > > >
> > > > wrote:
> > > >
> > > > >
> > > > > Fri Nov 08 16:04:58 2019: Request 93039 was acted upon.
> > > > > Transaction: Ticket created by justin.tsu at nrlmry.navy.mil
> > > > >        Queue: met_help
> > > > >      Subject: Absurdly high ME2
> > > > >        Owner: Nobody
> > > > >   Requestors: justin.tsu at nrlmry.navy.mil
> > > > >       Status: new
> > > > >  Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039
> > > >
> > > > >
> > > > >
> > > > > I am getting some absurdly high mean square errors for every
one of
> > my
> > > > > variables (wind speed, wind direction, relative humidity,
dew point
> > > > > depression, and air temperature) on each vertical pressure
level.
> > > These
> > > > > values range from the thousands to the 10s of millions.  How
can I
> > > > diagnose
> > > > > the cause of these massive errors?
> > > > >
> > > > >
> > > > >
> > > > > Some context:
> > > > >
> > > > > .        I am running point_stat
> > > > >
> > > > > .        The command is
> > > > >
> > > > >  point_stat PYTHON_NUMPY ${OBFILE} ${CONFIG}       \
> > > > >
> > > > >                                   -v 0
\
> > > > >
> > > > >                                   -outdir ${PSOUT}
\
> > > > >
> > > > >                                   -log
> > > > >
${PSOUT}/${VAR_NAME}${WIND_FLAG}_TAU${FCST_HR}_point_stat.log \
> > > > >
> > > > >                                   -obs_valid_beg 20010101
\
> > > > >
> > > > >                                   -obs_valid_end 20200101
> > > > >
> > > > > .        I've tailored my config file so that the vertical
levels
> of
> > > both
> > > > > the fcst and obs are relatively near, so this cannot be a
vertical
> > > > mismatch
> > > > > issue
> > > > >
> > > > > Justin
> > > > >
> > > > >
> > > > >
> > > > > Justin Tsu
> > > > >
> > > > > Marine Meteorology Division
> > > > >
> > > > > Data Assimilation/Mesoscale Modeling
> > > > >
> > > > > Building 704 Room 212
> > > > >
> > > > > Naval Research Laboratory, Code 7531
> > > > >
> > > > > 7 Grace Hopper Avenue
> > > > >
> > > > > Monterey, CA 93943-5502
> > > > >
> > > > >
> > > > >
> > > > > Ph. (831) 656-4111
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > >
> > >
> > >
> > >
> >
> >
> >
>
>
>


------------------------------------------------
Subject: Absurdly high ME2
From: John Halley Gotway
Time: Tue Nov 19 13:28:10 2019

Justin,

It sounds like you are expecting a lot more matches for each sounding
because the observation should be occurring frequently throughout the
day.

Here's the default Point-Stat config file:
https://github.com/NCAR/MET/blob/master_v8.1/met/data/config/PointStatConfig_default

There are 2 options in there you should consider: obs_summary and
obs_window.  Please check to see how you have them set to see if that
sheds
any light on this issue.  Perhaps your matching time window is too
small or
you have obs_summary set to NEAREST?

Below I've cut-and-pasted a description of those options from the
config/README file:
https://github.com/NCAR/MET/blob/master_v8.1/met/data/config/README

Thanks,
John

//
// The "obs_window" entry is a dictionary specifying a beginning
("beg"
// entry) and ending ("end" entry) time offset values in seconds.  It
defines
// the time window over which observations are retained for scoring.
These
time
// offsets are defined relative to a reference time t, as [t+beg,
t+end].
// In PB2NC, the reference time is the PREPBUFR files center time.  In
// Point-Stat and Ensemble-Stat, the reference time is the forecast
valid
time.
//
obs_window = {
   beg = -5400;
   end =  5400;
}

//
// The "obs_summary" entry specifies how to compute statistics on
// observations that appear at a single location (lat,lon,level,elev)
// in Point-Stat and Ensemble-Stat.  Eight techniques are
// currently supported:
//
//    - "NONE" to use all point observations (legacy behavior)
//    - "NEAREST" use only the observation that has the valid
//      time closest to the forecast valid time
//    - "MIN" use only the observation that has the lowest value
//    - "MAX" use only the observation that has the highest value
//    - "UW_MEAN" compute an unweighted mean of the observations
//    - "DW_MEAN" compute a weighted mean of the observations based
//      on the time of the observation
//    - "MEDIAN" use the median observation
//    - "PERC" use the Nth percentile observation where N =
obs_perc_value
//
// The reporting mechanism for this feature can be activated by
specifying
// a verbosity level of three or higher.  The report will show
information
// about where duplicates were detected and which observations were
used
// in those cases.
//
obs_summary = NONE;


On Tue, Nov 19, 2019 at 12:02 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039 >
>
> Thanks John,
>
> I have another question that I just discovered from my MPR files.  I
> notice that when verifying raobs, my mpr only produces one sounding.
I get
> multiple lines (for the different elevations) in the file but they
all have
> the same ob location, indicating that point_stat only found one
particular
> observation that matched - but I am not particularly sure what
matched.  If
> point_stat only looks for obs that match the time of the fcst field
(which
> if this is the case, I should be getting A LOT of matches because a
> majority of my obs are the same date as the fcst field) then filters
out
> those matches for obs that match the elevation of the fcst field,
then
> finally interpolates this subset of matches to the location of the
nearest
> model point, I feel like I should be getting much more than one
sounding.
> Any ideas on how to debug this?
>
> Justin
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Wednesday, November 13, 2019 1:44 PM
> To: Tsu, Mr. Justin
> Subject: Re: [rt.rap.ucar.edu #93039] Absurdly high ME2
>
> Justin,
>
> Yes, that's right.  Point-Stat will perform the comparison you
request in
> the config file.  And yes, I agree that the first comparison you
listed
> makes sense while the second comparison makes very little sense.
You can
> see the level info listed in the FCST_LEVEL and OBS_LEVEL output
columns.
>
> But having the ability to specify the level info separately for the
> forecast and observation is very useful.  For example, when
verifying a
> forecast at 850mb, you may choose to use observations exactly at
that level
> (P850), in a small range around that level (P848-852), or in a large
range
> around that level (P800-900).  The user can decide what tolerance
makes
> sense to the evaluation at hand.
>
> Thanks,
> John
>
> On Wed, Nov 13, 2019 at 2:08 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039 >
> >
> > Got it,
> >
> > So since I am using python embedding for my model field, it will
> > ultimately be up to what vertical level I specify my obs that will
> > determine the type of comparison.  For example:
> >
> > fcst = {
> >    field = [{name = "/users/tsu/MET/work/PY_SRC/read_NRL_binary.py
> > ${DATA_DIR}/${VAR_NAME}_pre_000.10_0000.0_${GRD}_${INIT
> > _TIME}_${FCST_HR}_fcstfld ${WIND_FLAG}";}];
> > }
> >
> > obs = {
> >     field = [{name = "${GRIB_NAME}";level = ["P0.0-0.15"];}];
> > }
> >
> > This comparison is correct because the highlighted portion in my
forecast
> > dictionary indicates this model field is at 0.10 hPa and so I
specify all
> > obs that fall within 0.0 and 0.15 hPA will be compared against
this model
> > field.
> >
> > The following is incorrect (BUT will still run):
> >
> > fcst = {
> >    field = [{name = "/users/tsu/MET/work/PY_SRC/read_NRL_binary.py
> > ${DATA_DIR}/${VAR_NAME}_pre_ 000900_000000_${GRD}_${INIT
> > _TIME}_${FCST_HR}_fcstfld ${WIND_FLAG}";}];
> > }
> >
> > obs = {
> >     field = [{name = "${GRIB_NAME}";level = ["P0.0-0.15"];}];
> > }
> >
> > This is an incorrect comparison but will still run.  It is
comparing a
> > model field at 900 hPa (as seen highlighted) to obs from 0 and
0.15 hPa.
> >
> >
> > Is everything I just said correct?
> >
> > Justin
> >
> > -----Original Message-----
> > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > Sent: Wednesday, November 13, 2019 1:00 PM
> > To: Tsu, Mr. Justin
> > Subject: Re: [rt.rap.ucar.edu #93039] Absurdly high ME2
> >
> > Justin,
> >
> > Point-Stat looks in the config file at the user-specified level to
be
> > verified.
> > For example, let's say you're running Point-Stat to verify wind
speed at
> 2
> > levels of model output, 850mb pressure level and 100 meters, a
shown
> below.
> >
> > fcst = {
> >    field = [ { name = "WIND"; level = "P850"; },
> >                 { name = "WIND"; level = "Z100"; }
> >    ];
> > }
> >
> > obs = {
> >    field = [ { name = "WIND"; level = "P840-860"; },
> >                 { name = "WIND"; level = "Z100"; }
> >    ];
> > }
> >
> > For the first task, Point-Stat will check the pressure level of
the
> > obs since the P specifies a pressure level (P840-860).
> > For the second task, it will check the height of the obs since the
Z
> > specifies a height (Z100).
> >
> > John
> >
> > On Wed, Nov 13, 2019 at 1:35 PM Tsu, Mr. Justin via RT <
> met_help at ucar.edu>
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039 >
> > >
> > > It's okay! Just for clarification, all of my model output is on
sigma
> > > (hPa) pressure levels.  How does Point-stat know to use the
pressure
> > level
> > > column in the observation text file to attach a location value
to the
> > > observation when comparing to pressure located model fields?
> > > Justin
> > >
> > > -----Original Message-----
> > > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > > Sent: Wednesday, November 13, 2019 12:26 PM
> > > To: Tsu, Mr. Justin
> > > Subject: Re: [rt.rap.ucar.edu #93039] Absurdly high ME2
> > >
> > > Justin,
> > >
> > > Typically elevation in MSL is given in meters.
> > >
> > > But MET is not enforcing any unit conventions or doing any unit
> > conversions
> > > for you.  Point observations do include an elevation of the
station and
> > the
> > > height of the observed values.  I don't think the station
elevation is
> > > currently being used in the logic.  But the height of the
observations
> > are
> > > used when vertically interpolating forecasts a specified heights
above
> > > ground to the height of the observation.  For example, if your
model
> > output
> > > contains wind speed every 10m above the ground, Point-Stat can
> > interpolate
> > > that model output to the height of each observation.  It's
critical to
> > make
> > > sure that the definition of the observation heights is
consistent with
> > the
> > > heights of the model output.
> > >
> > > They are often either defined relative to MSL or AGL... and
could be in
> > > feet or meters.  So double-check those.
> > >
> > > Sorry that the answer isn't more simple!
> > >
> > > Thanks,
> > > John
> > >
> > > On Wed, Nov 13, 2019 at 12:16 PM Tsu, Mr. Justin via RT <
> > met_help at ucar.edu
> > > >
> > > wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039
>
> > > >
> > > > Regarding units, I noticed that the units that Elevation of
observing
> > > > location and height of the actual measurement need to be in
MSL for
> use
> > > in
> > > > ascii2nc.  What are these units?
> > > >
> > > > Justin
> > > >
> > > > -----Original Message-----
> > > > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > > > Sent: Tuesday, November 12, 2019 2:32 PM
> > > > To: Tsu, Mr. Justin
> > > > Subject: Re: [rt.rap.ucar.edu #93039] Absurdly high ME2
> > > >
> > > > Justin,
> > > >
> > > > OK, thanks for the updates.  Please let me know if you have
any
> > questions
> > > > about using the convert() function in the config file to make
those
> > unit
> > > > comparable.  Glad you're figuring out the details.
> > > >
> > > > Thanks,
> > > > John
> > > >
> > > > On Tue, Nov 12, 2019 at 3:21 PM Tsu, Mr. Justin via RT <
> > > met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > >
> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039 >
> > > > >
> > > > > Just discovered that I was only computing dew point
depression and
> > not
> > > > T_d
> > > > > which is what my fcst is.  Fixed this (which will also fix
the
> > relative
> > > > > humidity).  Also discovered that my raobs airtmp are in
Celsius but
> > my
> > > > sfc
> > > > > obs are in Kelvin (I assumed sfc was also Celsius so was
doing
> > > conversion
> > > > > on that).  Still not sure why my wind direction and speed
have high
> > > > errors
> > > > > though.
> > > > >
> > > > > Justin
> > > > >
> > > > > -----Original Message-----
> > > > > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > > > > Sent: Friday, November 8, 2019 10:15 PM
> > > > > To: Tsu, Mr. Justin
> > > > > Subject: Re: [rt.rap.ucar.edu #93039] Absurdly high ME2
> > > > >
> > > > > Justin,
> > > > >
> > > > > The first thing I’d check for is a difference in the units
of the
> > fcst
> > > > and
> > > > > obs data which would lead to large error values.
> > > > >
> > > > > I’d recommend turning on the matched pair (MPR) output line
> type....
> > > set
> > > > it
> > > > > to BOTH.  Then look in the “*_mpr.txt” output file at the
FCST and
> > OBS
> > > > > columns.  Check the values to see if there’s and obvious
mismatch
> in
> > > > units.
> > > > >
> > > > > If so, you can define a “convert(x)” function in either the
fcst or
> > obs
> > > > > dictionaries of the config file to make the units
comparable.
> > > > >
> > > > > Thanks,
> > > > > John
> > > > >
> > > > > On Fri, Nov 8, 2019 at 5:05 PM Tsu, Mr. Justin via RT <
> > > met_help at ucar.edu
> > > > >
> > > > > wrote:
> > > > >
> > > > > >
> > > > > > Fri Nov 08 16:04:58 2019: Request 93039 was acted upon.
> > > > > > Transaction: Ticket created by justin.tsu at nrlmry.navy.mil
> > > > > >        Queue: met_help
> > > > > >      Subject: Absurdly high ME2
> > > > > >        Owner: Nobody
> > > > > >   Requestors: justin.tsu at nrlmry.navy.mil
> > > > > >       Status: new
> > > > > >  Ticket <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039
> > > > >
> > > > > >
> > > > > >
> > > > > > I am getting some absurdly high mean square errors for
every one
> of
> > > my
> > > > > > variables (wind speed, wind direction, relative humidity,
dew
> point
> > > > > > depression, and air temperature) on each vertical pressure
level.
> > > > These
> > > > > > values range from the thousands to the 10s of millions.
How can
> I
> > > > > diagnose
> > > > > > the cause of these massive errors?
> > > > > >
> > > > > >
> > > > > >
> > > > > > Some context:
> > > > > >
> > > > > > .        I am running point_stat
> > > > > >
> > > > > > .        The command is
> > > > > >
> > > > > >  point_stat PYTHON_NUMPY ${OBFILE} ${CONFIG}       \
> > > > > >
> > > > > >                                   -v 0
\
> > > > > >
> > > > > >                                   -outdir ${PSOUT}
\
> > > > > >
> > > > > >                                   -log
> > > > > >
${PSOUT}/${VAR_NAME}${WIND_FLAG}_TAU${FCST_HR}_point_stat.log \
> > > > > >
> > > > > >                                   -obs_valid_beg 20010101
\
> > > > > >
> > > > > >                                   -obs_valid_end 20200101
> > > > > >
> > > > > > .        I've tailored my config file so that the vertical
levels
> > of
> > > > both
> > > > > > the fcst and obs are relatively near, so this cannot be a
> vertical
> > > > > mismatch
> > > > > > issue
> > > > > >
> > > > > > Justin
> > > > > >
> > > > > >
> > > > > >
> > > > > > Justin Tsu
> > > > > >
> > > > > > Marine Meteorology Division
> > > > > >
> > > > > > Data Assimilation/Mesoscale Modeling
> > > > > >
> > > > > > Building 704 Room 212
> > > > > >
> > > > > > Naval Research Laboratory, Code 7531
> > > > > >
> > > > > > 7 Grace Hopper Avenue
> > > > > >
> > > > > > Monterey, CA 93943-5502
> > > > > >
> > > > > >
> > > > > >
> > > > > > Ph. (831) 656-4111
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > >
> > >
> > >
> > >
> >
> >
> >
>
>
>

------------------------------------------------
Subject: Absurdly high ME2
From: Tsu, Mr. Justin
Time: Tue Nov 19 13:48:24 2019

I am not sure if these observations have exact timing info stored
correctly because they all have the same date, which is just
YYYYMMDD_HHMMSS, which for the most part is just YYYYMMDD_HH0000.  So:
Ncdump -v hdr_vld_table raob_2019072112.nc

data:

 hdr_vld_table =
  "20190721_120000",
  "20190721_072112" ;
}


But other than that, I run point stat like this:

point_stat PYTHON_NUMPY raob_2019072112.nc PointStatConfig  -v 3
-obs_valid_beg 20010101 -obs_valid_end 20200101

so my obs time window will consider all obs ( I do this because I am
not sure how point_stat reads in the time stamp for my fcst field...
when I leave the obs_valid beg and obs_valid_end parameters out, I get
0 matched pairs because of rejection due to time mismatch). One change
that I applied to my file is that for each pressure level, I am no
longer considering a range of pressures for obs.  So lets say I am
looking at a 700 mb model field, I am only considering obs whose
pressure level is 700.0 mb.  I am currently playing around with
changing the range by setting this to P699.0-701.0 (which actually got
me a few more matched pairs, but again they were at the same obs
lat/lon for some reason).


My obs_summary is set to NONE.

Justin

-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Tuesday, November 19, 2019 12:28 PM
To: Tsu, Mr. Justin
Subject: Re: [rt.rap.ucar.edu #93039] Absurdly high ME2

Justin,

It sounds like you are expecting a lot more matches for each sounding
because the observation should be occurring frequently throughout the
day.

Here's the default Point-Stat config file:
https://github.com/NCAR/MET/blob/master_v8.1/met/data/config/PointStatConfig_default

There are 2 options in there you should consider: obs_summary and
obs_window.  Please check to see how you have them set to see if that
sheds
any light on this issue.  Perhaps your matching time window is too
small or
you have obs_summary set to NEAREST?

Below I've cut-and-pasted a description of those options from the
config/README file:
https://github.com/NCAR/MET/blob/master_v8.1/met/data/config/README

Thanks,
John

//
// The "obs_window" entry is a dictionary specifying a beginning
("beg"
// entry) and ending ("end" entry) time offset values in seconds.  It
defines
// the time window over which observations are retained for scoring.
These
time
// offsets are defined relative to a reference time t, as [t+beg,
t+end].
// In PB2NC, the reference time is the PREPBUFR files center time.  In
// Point-Stat and Ensemble-Stat, the reference time is the forecast
valid
time.
//
obs_window = {
   beg = -5400;
   end =  5400;
}

//
// The "obs_summary" entry specifies how to compute statistics on
// observations that appear at a single location (lat,lon,level,elev)
// in Point-Stat and Ensemble-Stat.  Eight techniques are
// currently supported:
//
//    - "NONE" to use all point observations (legacy behavior)
//    - "NEAREST" use only the observation that has the valid
//      time closest to the forecast valid time
//    - "MIN" use only the observation that has the lowest value
//    - "MAX" use only the observation that has the highest value
//    - "UW_MEAN" compute an unweighted mean of the observations
//    - "DW_MEAN" compute a weighted mean of the observations based
//      on the time of the observation
//    - "MEDIAN" use the median observation
//    - "PERC" use the Nth percentile observation where N =
obs_perc_value
//
// The reporting mechanism for this feature can be activated by
specifying
// a verbosity level of three or higher.  The report will show
information
// about where duplicates were detected and which observations were
used
// in those cases.
//
obs_summary = NONE;


On Tue, Nov 19, 2019 at 12:02 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039 >
>
> Thanks John,
>
> I have another question that I just discovered from my MPR files.  I
> notice that when verifying raobs, my mpr only produces one sounding.
I get
> multiple lines (for the different elevations) in the file but they
all have
> the same ob location, indicating that point_stat only found one
particular
> observation that matched - but I am not particularly sure what
matched.  If
> point_stat only looks for obs that match the time of the fcst field
(which
> if this is the case, I should be getting A LOT of matches because a
> majority of my obs are the same date as the fcst field) then filters
out
> those matches for obs that match the elevation of the fcst field,
then
> finally interpolates this subset of matches to the location of the
nearest
> model point, I feel like I should be getting much more than one
sounding.
> Any ideas on how to debug this?
>
> Justin
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Wednesday, November 13, 2019 1:44 PM
> To: Tsu, Mr. Justin
> Subject: Re: [rt.rap.ucar.edu #93039] Absurdly high ME2
>
> Justin,
>
> Yes, that's right.  Point-Stat will perform the comparison you
request in
> the config file.  And yes, I agree that the first comparison you
listed
> makes sense while the second comparison makes very little sense.
You can
> see the level info listed in the FCST_LEVEL and OBS_LEVEL output
columns.
>
> But having the ability to specify the level info separately for the
> forecast and observation is very useful.  For example, when
verifying a
> forecast at 850mb, you may choose to use observations exactly at
that level
> (P850), in a small range around that level (P848-852), or in a large
range
> around that level (P800-900).  The user can decide what tolerance
makes
> sense to the evaluation at hand.
>
> Thanks,
> John
>
> On Wed, Nov 13, 2019 at 2:08 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039 >
> >
> > Got it,
> >
> > So since I am using python embedding for my model field, it will
> > ultimately be up to what vertical level I specify my obs that will
> > determine the type of comparison.  For example:
> >
> > fcst = {
> >    field = [{name = "/users/tsu/MET/work/PY_SRC/read_NRL_binary.py
> > ${DATA_DIR}/${VAR_NAME}_pre_000.10_0000.0_${GRD}_${INIT
> > _TIME}_${FCST_HR}_fcstfld ${WIND_FLAG}";}];
> > }
> >
> > obs = {
> >     field = [{name = "${GRIB_NAME}";level = ["P0.0-0.15"];}];
> > }
> >
> > This comparison is correct because the highlighted portion in my
forecast
> > dictionary indicates this model field is at 0.10 hPa and so I
specify all
> > obs that fall within 0.0 and 0.15 hPA will be compared against
this model
> > field.
> >
> > The following is incorrect (BUT will still run):
> >
> > fcst = {
> >    field = [{name = "/users/tsu/MET/work/PY_SRC/read_NRL_binary.py
> > ${DATA_DIR}/${VAR_NAME}_pre_ 000900_000000_${GRD}_${INIT
> > _TIME}_${FCST_HR}_fcstfld ${WIND_FLAG}";}];
> > }
> >
> > obs = {
> >     field = [{name = "${GRIB_NAME}";level = ["P0.0-0.15"];}];
> > }
> >
> > This is an incorrect comparison but will still run.  It is
comparing a
> > model field at 900 hPa (as seen highlighted) to obs from 0 and
0.15 hPa.
> >
> >
> > Is everything I just said correct?
> >
> > Justin
> >
> > -----Original Message-----
> > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > Sent: Wednesday, November 13, 2019 1:00 PM
> > To: Tsu, Mr. Justin
> > Subject: Re: [rt.rap.ucar.edu #93039] Absurdly high ME2
> >
> > Justin,
> >
> > Point-Stat looks in the config file at the user-specified level to
be
> > verified.
> > For example, let's say you're running Point-Stat to verify wind
speed at
> 2
> > levels of model output, 850mb pressure level and 100 meters, a
shown
> below.
> >
> > fcst = {
> >    field = [ { name = "WIND"; level = "P850"; },
> >                 { name = "WIND"; level = "Z100"; }
> >    ];
> > }
> >
> > obs = {
> >    field = [ { name = "WIND"; level = "P840-860"; },
> >                 { name = "WIND"; level = "Z100"; }
> >    ];
> > }
> >
> > For the first task, Point-Stat will check the pressure level of
the
> > obs since the P specifies a pressure level (P840-860).
> > For the second task, it will check the height of the obs since the
Z
> > specifies a height (Z100).
> >
> > John
> >
> > On Wed, Nov 13, 2019 at 1:35 PM Tsu, Mr. Justin via RT <
> met_help at ucar.edu>
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039 >
> > >
> > > It's okay! Just for clarification, all of my model output is on
sigma
> > > (hPa) pressure levels.  How does Point-stat know to use the
pressure
> > level
> > > column in the observation text file to attach a location value
to the
> > > observation when comparing to pressure located model fields?
> > > Justin
> > >
> > > -----Original Message-----
> > > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > > Sent: Wednesday, November 13, 2019 12:26 PM
> > > To: Tsu, Mr. Justin
> > > Subject: Re: [rt.rap.ucar.edu #93039] Absurdly high ME2
> > >
> > > Justin,
> > >
> > > Typically elevation in MSL is given in meters.
> > >
> > > But MET is not enforcing any unit conventions or doing any unit
> > conversions
> > > for you.  Point observations do include an elevation of the
station and
> > the
> > > height of the observed values.  I don't think the station
elevation is
> > > currently being used in the logic.  But the height of the
observations
> > are
> > > used when vertically interpolating forecasts a specified heights
above
> > > ground to the height of the observation.  For example, if your
model
> > output
> > > contains wind speed every 10m above the ground, Point-Stat can
> > interpolate
> > > that model output to the height of each observation.  It's
critical to
> > make
> > > sure that the definition of the observation heights is
consistent with
> > the
> > > heights of the model output.
> > >
> > > They are often either defined relative to MSL or AGL... and
could be in
> > > feet or meters.  So double-check those.
> > >
> > > Sorry that the answer isn't more simple!
> > >
> > > Thanks,
> > > John
> > >
> > > On Wed, Nov 13, 2019 at 12:16 PM Tsu, Mr. Justin via RT <
> > met_help at ucar.edu
> > > >
> > > wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039
>
> > > >
> > > > Regarding units, I noticed that the units that Elevation of
observing
> > > > location and height of the actual measurement need to be in
MSL for
> use
> > > in
> > > > ascii2nc.  What are these units?
> > > >
> > > > Justin
> > > >
> > > > -----Original Message-----
> > > > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > > > Sent: Tuesday, November 12, 2019 2:32 PM
> > > > To: Tsu, Mr. Justin
> > > > Subject: Re: [rt.rap.ucar.edu #93039] Absurdly high ME2
> > > >
> > > > Justin,
> > > >
> > > > OK, thanks for the updates.  Please let me know if you have
any
> > questions
> > > > about using the convert() function in the config file to make
those
> > unit
> > > > comparable.  Glad you're figuring out the details.
> > > >
> > > > Thanks,
> > > > John
> > > >
> > > > On Tue, Nov 12, 2019 at 3:21 PM Tsu, Mr. Justin via RT <
> > > met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > >
> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039 >
> > > > >
> > > > > Just discovered that I was only computing dew point
depression and
> > not
> > > > T_d
> > > > > which is what my fcst is.  Fixed this (which will also fix
the
> > relative
> > > > > humidity).  Also discovered that my raobs airtmp are in
Celsius but
> > my
> > > > sfc
> > > > > obs are in Kelvin (I assumed sfc was also Celsius so was
doing
> > > conversion
> > > > > on that).  Still not sure why my wind direction and speed
have high
> > > > errors
> > > > > though.
> > > > >
> > > > > Justin
> > > > >
> > > > > -----Original Message-----
> > > > > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > > > > Sent: Friday, November 8, 2019 10:15 PM
> > > > > To: Tsu, Mr. Justin
> > > > > Subject: Re: [rt.rap.ucar.edu #93039] Absurdly high ME2
> > > > >
> > > > > Justin,
> > > > >
> > > > > The first thing I’d check for is a difference in the units
of the
> > fcst
> > > > and
> > > > > obs data which would lead to large error values.
> > > > >
> > > > > I’d recommend turning on the matched pair (MPR) output line
> type....
> > > set
> > > > it
> > > > > to BOTH.  Then look in the “*_mpr.txt” output file at the
FCST and
> > OBS
> > > > > columns.  Check the values to see if there’s and obvious
mismatch
> in
> > > > units.
> > > > >
> > > > > If so, you can define a “convert(x)” function in either the
fcst or
> > obs
> > > > > dictionaries of the config file to make the units
comparable.
> > > > >
> > > > > Thanks,
> > > > > John
> > > > >
> > > > > On Fri, Nov 8, 2019 at 5:05 PM Tsu, Mr. Justin via RT <
> > > met_help at ucar.edu
> > > > >
> > > > > wrote:
> > > > >
> > > > > >
> > > > > > Fri Nov 08 16:04:58 2019: Request 93039 was acted upon.
> > > > > > Transaction: Ticket created by justin.tsu at nrlmry.navy.mil
> > > > > >        Queue: met_help
> > > > > >      Subject: Absurdly high ME2
> > > > > >        Owner: Nobody
> > > > > >   Requestors: justin.tsu at nrlmry.navy.mil
> > > > > >       Status: new
> > > > > >  Ticket <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039
> > > > >
> > > > > >
> > > > > >
> > > > > > I am getting some absurdly high mean square errors for
every one
> of
> > > my
> > > > > > variables (wind speed, wind direction, relative humidity,
dew
> point
> > > > > > depression, and air temperature) on each vertical pressure
level.
> > > > These
> > > > > > values range from the thousands to the 10s of millions.
How can
> I
> > > > > diagnose
> > > > > > the cause of these massive errors?
> > > > > >
> > > > > >
> > > > > >
> > > > > > Some context:
> > > > > >
> > > > > > .        I am running point_stat
> > > > > >
> > > > > > .        The command is
> > > > > >
> > > > > >  point_stat PYTHON_NUMPY ${OBFILE} ${CONFIG}       \
> > > > > >
> > > > > >                                   -v 0
\
> > > > > >
> > > > > >                                   -outdir ${PSOUT}
\
> > > > > >
> > > > > >                                   -log
> > > > > >
${PSOUT}/${VAR_NAME}${WIND_FLAG}_TAU${FCST_HR}_point_stat.log \
> > > > > >
> > > > > >                                   -obs_valid_beg 20010101
\
> > > > > >
> > > > > >                                   -obs_valid_end 20200101
> > > > > >
> > > > > > .        I've tailored my config file so that the vertical
levels
> > of
> > > > both
> > > > > > the fcst and obs are relatively near, so this cannot be a
> vertical
> > > > > mismatch
> > > > > > issue
> > > > > >
> > > > > > Justin
> > > > > >
> > > > > >
> > > > > >
> > > > > > Justin Tsu
> > > > > >
> > > > > > Marine Meteorology Division
> > > > > >
> > > > > > Data Assimilation/Mesoscale Modeling
> > > > > >
> > > > > > Building 704 Room 212
> > > > > >
> > > > > > Naval Research Laboratory, Code 7531
> > > > > >
> > > > > > 7 Grace Hopper Avenue
> > > > > >
> > > > > > Monterey, CA 93943-5502
> > > > > >
> > > > > >
> > > > > >
> > > > > > Ph. (831) 656-4111
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > >
> > >
> > >
> > >
> >
> >
> >
>
>
>


------------------------------------------------
Subject: Absurdly high ME2
From: Tsu, Mr. Justin
Time: Tue Nov 19 14:19:50 2019

Okay I think I may have identified the issue I am having.  The issue
lies in how I am specifying my model grid.  I just realized that I
never updated my attrs dictionary in my Python embedding script.  I
updated that and now I am getting 0 matched pairs.  I do not think I
am understanding how to correctly set my attrs; that being said, I do
know that I have latitude and longitude gridded model fields in files
named latitu* and longit* for respective dates and nests.  How do I
set my config file regrid dictionary to read in both a latitude and
longitude file in a way that Point Stat can understand?

Justin

-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Tuesday, November 19, 2019 12:28 PM
To: Tsu, Mr. Justin
Subject: Re: [rt.rap.ucar.edu #93039] Absurdly high ME2

Justin,

It sounds like you are expecting a lot more matches for each sounding
because the observation should be occurring frequently throughout the
day.

Here's the default Point-Stat config file:
https://github.com/NCAR/MET/blob/master_v8.1/met/data/config/PointStatConfig_default

There are 2 options in there you should consider: obs_summary and
obs_window.  Please check to see how you have them set to see if that
sheds
any light on this issue.  Perhaps your matching time window is too
small or
you have obs_summary set to NEAREST?

Below I've cut-and-pasted a description of those options from the
config/README file:
https://github.com/NCAR/MET/blob/master_v8.1/met/data/config/README

Thanks,
John

//
// The "obs_window" entry is a dictionary specifying a beginning
("beg"
// entry) and ending ("end" entry) time offset values in seconds.  It
defines
// the time window over which observations are retained for scoring.
These
time
// offsets are defined relative to a reference time t, as [t+beg,
t+end].
// In PB2NC, the reference time is the PREPBUFR files center time.  In
// Point-Stat and Ensemble-Stat, the reference time is the forecast
valid
time.
//
obs_window = {
   beg = -5400;
   end =  5400;
}

//
// The "obs_summary" entry specifies how to compute statistics on
// observations that appear at a single location (lat,lon,level,elev)
// in Point-Stat and Ensemble-Stat.  Eight techniques are
// currently supported:
//
//    - "NONE" to use all point observations (legacy behavior)
//    - "NEAREST" use only the observation that has the valid
//      time closest to the forecast valid time
//    - "MIN" use only the observation that has the lowest value
//    - "MAX" use only the observation that has the highest value
//    - "UW_MEAN" compute an unweighted mean of the observations
//    - "DW_MEAN" compute a weighted mean of the observations based
//      on the time of the observation
//    - "MEDIAN" use the median observation
//    - "PERC" use the Nth percentile observation where N =
obs_perc_value
//
// The reporting mechanism for this feature can be activated by
specifying
// a verbosity level of three or higher.  The report will show
information
// about where duplicates were detected and which observations were
used
// in those cases.
//
obs_summary = NONE;


On Tue, Nov 19, 2019 at 12:02 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039 >
>
> Thanks John,
>
> I have another question that I just discovered from my MPR files.  I
> notice that when verifying raobs, my mpr only produces one sounding.
I get
> multiple lines (for the different elevations) in the file but they
all have
> the same ob location, indicating that point_stat only found one
particular
> observation that matched - but I am not particularly sure what
matched.  If
> point_stat only looks for obs that match the time of the fcst field
(which
> if this is the case, I should be getting A LOT of matches because a
> majority of my obs are the same date as the fcst field) then filters
out
> those matches for obs that match the elevation of the fcst field,
then
> finally interpolates this subset of matches to the location of the
nearest
> model point, I feel like I should be getting much more than one
sounding.
> Any ideas on how to debug this?
>
> Justin
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Wednesday, November 13, 2019 1:44 PM
> To: Tsu, Mr. Justin
> Subject: Re: [rt.rap.ucar.edu #93039] Absurdly high ME2
>
> Justin,
>
> Yes, that's right.  Point-Stat will perform the comparison you
request in
> the config file.  And yes, I agree that the first comparison you
listed
> makes sense while the second comparison makes very little sense.
You can
> see the level info listed in the FCST_LEVEL and OBS_LEVEL output
columns.
>
> But having the ability to specify the level info separately for the
> forecast and observation is very useful.  For example, when
verifying a
> forecast at 850mb, you may choose to use observations exactly at
that level
> (P850), in a small range around that level (P848-852), or in a large
range
> around that level (P800-900).  The user can decide what tolerance
makes
> sense to the evaluation at hand.
>
> Thanks,
> John
>
> On Wed, Nov 13, 2019 at 2:08 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039 >
> >
> > Got it,
> >
> > So since I am using python embedding for my model field, it will
> > ultimately be up to what vertical level I specify my obs that will
> > determine the type of comparison.  For example:
> >
> > fcst = {
> >    field = [{name = "/users/tsu/MET/work/PY_SRC/read_NRL_binary.py
> > ${DATA_DIR}/${VAR_NAME}_pre_000.10_0000.0_${GRD}_${INIT
> > _TIME}_${FCST_HR}_fcstfld ${WIND_FLAG}";}];
> > }
> >
> > obs = {
> >     field = [{name = "${GRIB_NAME}";level = ["P0.0-0.15"];}];
> > }
> >
> > This comparison is correct because the highlighted portion in my
forecast
> > dictionary indicates this model field is at 0.10 hPa and so I
specify all
> > obs that fall within 0.0 and 0.15 hPA will be compared against
this model
> > field.
> >
> > The following is incorrect (BUT will still run):
> >
> > fcst = {
> >    field = [{name = "/users/tsu/MET/work/PY_SRC/read_NRL_binary.py
> > ${DATA_DIR}/${VAR_NAME}_pre_ 000900_000000_${GRD}_${INIT
> > _TIME}_${FCST_HR}_fcstfld ${WIND_FLAG}";}];
> > }
> >
> > obs = {
> >     field = [{name = "${GRIB_NAME}";level = ["P0.0-0.15"];}];
> > }
> >
> > This is an incorrect comparison but will still run.  It is
comparing a
> > model field at 900 hPa (as seen highlighted) to obs from 0 and
0.15 hPa.
> >
> >
> > Is everything I just said correct?
> >
> > Justin
> >
> > -----Original Message-----
> > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > Sent: Wednesday, November 13, 2019 1:00 PM
> > To: Tsu, Mr. Justin
> > Subject: Re: [rt.rap.ucar.edu #93039] Absurdly high ME2
> >
> > Justin,
> >
> > Point-Stat looks in the config file at the user-specified level to
be
> > verified.
> > For example, let's say you're running Point-Stat to verify wind
speed at
> 2
> > levels of model output, 850mb pressure level and 100 meters, a
shown
> below.
> >
> > fcst = {
> >    field = [ { name = "WIND"; level = "P850"; },
> >                 { name = "WIND"; level = "Z100"; }
> >    ];
> > }
> >
> > obs = {
> >    field = [ { name = "WIND"; level = "P840-860"; },
> >                 { name = "WIND"; level = "Z100"; }
> >    ];
> > }
> >
> > For the first task, Point-Stat will check the pressure level of
the
> > obs since the P specifies a pressure level (P840-860).
> > For the second task, it will check the height of the obs since the
Z
> > specifies a height (Z100).
> >
> > John
> >
> > On Wed, Nov 13, 2019 at 1:35 PM Tsu, Mr. Justin via RT <
> met_help at ucar.edu>
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039 >
> > >
> > > It's okay! Just for clarification, all of my model output is on
sigma
> > > (hPa) pressure levels.  How does Point-stat know to use the
pressure
> > level
> > > column in the observation text file to attach a location value
to the
> > > observation when comparing to pressure located model fields?
> > > Justin
> > >
> > > -----Original Message-----
> > > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > > Sent: Wednesday, November 13, 2019 12:26 PM
> > > To: Tsu, Mr. Justin
> > > Subject: Re: [rt.rap.ucar.edu #93039] Absurdly high ME2
> > >
> > > Justin,
> > >
> > > Typically elevation in MSL is given in meters.
> > >
> > > But MET is not enforcing any unit conventions or doing any unit
> > conversions
> > > for you.  Point observations do include an elevation of the
station and
> > the
> > > height of the observed values.  I don't think the station
elevation is
> > > currently being used in the logic.  But the height of the
observations
> > are
> > > used when vertically interpolating forecasts a specified heights
above
> > > ground to the height of the observation.  For example, if your
model
> > output
> > > contains wind speed every 10m above the ground, Point-Stat can
> > interpolate
> > > that model output to the height of each observation.  It's
critical to
> > make
> > > sure that the definition of the observation heights is
consistent with
> > the
> > > heights of the model output.
> > >
> > > They are often either defined relative to MSL or AGL... and
could be in
> > > feet or meters.  So double-check those.
> > >
> > > Sorry that the answer isn't more simple!
> > >
> > > Thanks,
> > > John
> > >
> > > On Wed, Nov 13, 2019 at 12:16 PM Tsu, Mr. Justin via RT <
> > met_help at ucar.edu
> > > >
> > > wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039
>
> > > >
> > > > Regarding units, I noticed that the units that Elevation of
observing
> > > > location and height of the actual measurement need to be in
MSL for
> use
> > > in
> > > > ascii2nc.  What are these units?
> > > >
> > > > Justin
> > > >
> > > > -----Original Message-----
> > > > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > > > Sent: Tuesday, November 12, 2019 2:32 PM
> > > > To: Tsu, Mr. Justin
> > > > Subject: Re: [rt.rap.ucar.edu #93039] Absurdly high ME2
> > > >
> > > > Justin,
> > > >
> > > > OK, thanks for the updates.  Please let me know if you have
any
> > questions
> > > > about using the convert() function in the config file to make
those
> > unit
> > > > comparable.  Glad you're figuring out the details.
> > > >
> > > > Thanks,
> > > > John
> > > >
> > > > On Tue, Nov 12, 2019 at 3:21 PM Tsu, Mr. Justin via RT <
> > > met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > >
> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039 >
> > > > >
> > > > > Just discovered that I was only computing dew point
depression and
> > not
> > > > T_d
> > > > > which is what my fcst is.  Fixed this (which will also fix
the
> > relative
> > > > > humidity).  Also discovered that my raobs airtmp are in
Celsius but
> > my
> > > > sfc
> > > > > obs are in Kelvin (I assumed sfc was also Celsius so was
doing
> > > conversion
> > > > > on that).  Still not sure why my wind direction and speed
have high
> > > > errors
> > > > > though.
> > > > >
> > > > > Justin
> > > > >
> > > > > -----Original Message-----
> > > > > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > > > > Sent: Friday, November 8, 2019 10:15 PM
> > > > > To: Tsu, Mr. Justin
> > > > > Subject: Re: [rt.rap.ucar.edu #93039] Absurdly high ME2
> > > > >
> > > > > Justin,
> > > > >
> > > > > The first thing I’d check for is a difference in the units
of the
> > fcst
> > > > and
> > > > > obs data which would lead to large error values.
> > > > >
> > > > > I’d recommend turning on the matched pair (MPR) output line
> type....
> > > set
> > > > it
> > > > > to BOTH.  Then look in the “*_mpr.txt” output file at the
FCST and
> > OBS
> > > > > columns.  Check the values to see if there’s and obvious
mismatch
> in
> > > > units.
> > > > >
> > > > > If so, you can define a “convert(x)” function in either the
fcst or
> > obs
> > > > > dictionaries of the config file to make the units
comparable.
> > > > >
> > > > > Thanks,
> > > > > John
> > > > >
> > > > > On Fri, Nov 8, 2019 at 5:05 PM Tsu, Mr. Justin via RT <
> > > met_help at ucar.edu
> > > > >
> > > > > wrote:
> > > > >
> > > > > >
> > > > > > Fri Nov 08 16:04:58 2019: Request 93039 was acted upon.
> > > > > > Transaction: Ticket created by justin.tsu at nrlmry.navy.mil
> > > > > >        Queue: met_help
> > > > > >      Subject: Absurdly high ME2
> > > > > >        Owner: Nobody
> > > > > >   Requestors: justin.tsu at nrlmry.navy.mil
> > > > > >       Status: new
> > > > > >  Ticket <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039
> > > > >
> > > > > >
> > > > > >
> > > > > > I am getting some absurdly high mean square errors for
every one
> of
> > > my
> > > > > > variables (wind speed, wind direction, relative humidity,
dew
> point
> > > > > > depression, and air temperature) on each vertical pressure
level.
> > > > These
> > > > > > values range from the thousands to the 10s of millions.
How can
> I
> > > > > diagnose
> > > > > > the cause of these massive errors?
> > > > > >
> > > > > >
> > > > > >
> > > > > > Some context:
> > > > > >
> > > > > > .        I am running point_stat
> > > > > >
> > > > > > .        The command is
> > > > > >
> > > > > >  point_stat PYTHON_NUMPY ${OBFILE} ${CONFIG}       \
> > > > > >
> > > > > >                                   -v 0
\
> > > > > >
> > > > > >                                   -outdir ${PSOUT}
\
> > > > > >
> > > > > >                                   -log
> > > > > >
${PSOUT}/${VAR_NAME}${WIND_FLAG}_TAU${FCST_HR}_point_stat.log \
> > > > > >
> > > > > >                                   -obs_valid_beg 20010101
\
> > > > > >
> > > > > >                                   -obs_valid_end 20200101
> > > > > >
> > > > > > .        I've tailored my config file so that the vertical
levels
> > of
> > > > both
> > > > > > the fcst and obs are relatively near, so this cannot be a
> vertical
> > > > > mismatch
> > > > > > issue
> > > > > >
> > > > > > Justin
> > > > > >
> > > > > >
> > > > > >
> > > > > > Justin Tsu
> > > > > >
> > > > > > Marine Meteorology Division
> > > > > >
> > > > > > Data Assimilation/Mesoscale Modeling
> > > > > >
> > > > > > Building 704 Room 212
> > > > > >
> > > > > > Naval Research Laboratory, Code 7531
> > > > > >
> > > > > > 7 Grace Hopper Avenue
> > > > > >
> > > > > > Monterey, CA 93943-5502
> > > > > >
> > > > > >
> > > > > >
> > > > > > Ph. (831) 656-4111
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > >
> > >
> > >
> > >
> >
> >
> >
>
>
>


------------------------------------------------
Subject: Absurdly high ME2
From: John Halley Gotway
Time: Wed Nov 20 13:33:32 2019

Justin,

I'm not sure. I'd probably need to look at an example to better
understand
what's going on.  But I did look at the sample python scripts from the
MET
website:

https://dtcenter.org/community-code/model-evaluation-tools-met/sample-
analysis-scripts

Here's an excerpt from the NRL one:

https://dtcenter.org/sites/default/files/community-code/met/python-
scripts/read_NRL_binary.py.txt

   'grid': {
       'name': 'Global 1 Degree',
       'type' : 'LatLon',
       'lat_ll' :    -90.0,
       'lon_ll' :      0.0,
       'delta_lat' :   1.0,
       'delta_lon' :   1.0,
       'Nlat' :      181,
       'Nlon' :      360,
   }

This defines a global 1.0 degree (181, 360) grid.  If you're still
using
global data on a lat/lon grid, updating this grid definition should be
pretty easy.  Just figure out the dimensions of the data and switch
(181,
360) with your current (Nlat, Nlon) values.  Then do the math to
figure out
the grid spacing in degrees and replace 1.0 with your values for
delta_lat
and delta_lon.

When you make changes to the python embedding script, it's always a
good
idea to run the plot_data_plane tool using python embedding.  Check
the
output image to confirm that MET is placing your data at the right
location
on the earth.

Thanks,
John

On Tue, Nov 19, 2019 at 2:19 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu>
wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039 >
>
> Okay I think I may have identified the issue I am having.  The issue
lies
> in how I am specifying my model grid.  I just realized that I never
updated
> my attrs dictionary in my Python embedding script.  I updated that
and now
> I am getting 0 matched pairs.  I do not think I am understanding how
to
> correctly set my attrs; that being said, I do know that I have
latitude and
> longitude gridded model fields in files named latitu* and longit*
for
> respective dates and nests.  How do I set my config file regrid
dictionary
> to read in both a latitude and longitude file in a way that Point
Stat can
> understand?
>
> Justin
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Tuesday, November 19, 2019 12:28 PM
> To: Tsu, Mr. Justin
> Subject: Re: [rt.rap.ucar.edu #93039] Absurdly high ME2
>
> Justin,
>
> It sounds like you are expecting a lot more matches for each
sounding
> because the observation should be occurring frequently throughout
the day.
>
> Here's the default Point-Stat config file:
>
>
https://github.com/NCAR/MET/blob/master_v8.1/met/data/config/PointStatConfig_default
>
> There are 2 options in there you should consider: obs_summary and
> obs_window.  Please check to see how you have them set to see if
that sheds
> any light on this issue.  Perhaps your matching time window is too
small or
> you have obs_summary set to NEAREST?
>
> Below I've cut-and-pasted a description of those options from the
> config/README file:
> https://github.com/NCAR/MET/blob/master_v8.1/met/data/config/README
>
> Thanks,
> John
>
> //
> // The "obs_window" entry is a dictionary specifying a beginning
("beg"
> // entry) and ending ("end" entry) time offset values in seconds.
It
> defines
> // the time window over which observations are retained for scoring.
These
> time
> // offsets are defined relative to a reference time t, as [t+beg,
t+end].
> // In PB2NC, the reference time is the PREPBUFR files center time.
In
> // Point-Stat and Ensemble-Stat, the reference time is the forecast
valid
> time.
> //
> obs_window = {
>    beg = -5400;
>    end =  5400;
> }
>
> //
> // The "obs_summary" entry specifies how to compute statistics on
> // observations that appear at a single location
(lat,lon,level,elev)
> // in Point-Stat and Ensemble-Stat.  Eight techniques are
> // currently supported:
> //
> //    - "NONE" to use all point observations (legacy behavior)
> //    - "NEAREST" use only the observation that has the valid
> //      time closest to the forecast valid time
> //    - "MIN" use only the observation that has the lowest value
> //    - "MAX" use only the observation that has the highest value
> //    - "UW_MEAN" compute an unweighted mean of the observations
> //    - "DW_MEAN" compute a weighted mean of the observations based
> //      on the time of the observation
> //    - "MEDIAN" use the median observation
> //    - "PERC" use the Nth percentile observation where N =
obs_perc_value
> //
> // The reporting mechanism for this feature can be activated by
specifying
> // a verbosity level of three or higher.  The report will show
information
> // about where duplicates were detected and which observations were
used
> // in those cases.
> //
> obs_summary = NONE;
>
>
> On Tue, Nov 19, 2019 at 12:02 PM Tsu, Mr. Justin via RT
<met_help at ucar.edu
> >
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039 >
> >
> > Thanks John,
> >
> > I have another question that I just discovered from my MPR files.
I
> > notice that when verifying raobs, my mpr only produces one
sounding. I
> get
> > multiple lines (for the different elevations) in the file but they
all
> have
> > the same ob location, indicating that point_stat only found one
> particular
> > observation that matched - but I am not particularly sure what
matched.
> If
> > point_stat only looks for obs that match the time of the fcst
field
> (which
> > if this is the case, I should be getting A LOT of matches because
a
> > majority of my obs are the same date as the fcst field) then
filters out
> > those matches for obs that match the elevation of the fcst field,
then
> > finally interpolates this subset of matches to the location of the
> nearest
> > model point, I feel like I should be getting much more than one
sounding.
> > Any ideas on how to debug this?
> >
> > Justin
> >
> > -----Original Message-----
> > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > Sent: Wednesday, November 13, 2019 1:44 PM
> > To: Tsu, Mr. Justin
> > Subject: Re: [rt.rap.ucar.edu #93039] Absurdly high ME2
> >
> > Justin,
> >
> > Yes, that's right.  Point-Stat will perform the comparison you
request in
> > the config file.  And yes, I agree that the first comparison you
listed
> > makes sense while the second comparison makes very little sense.
You can
> > see the level info listed in the FCST_LEVEL and OBS_LEVEL output
columns.
> >
> > But having the ability to specify the level info separately for
the
> > forecast and observation is very useful.  For example, when
verifying a
> > forecast at 850mb, you may choose to use observations exactly at
that
> level
> > (P850), in a small range around that level (P848-852), or in a
large
> range
> > around that level (P800-900).  The user can decide what tolerance
makes
> > sense to the evaluation at hand.
> >
> > Thanks,
> > John
> >
> > On Wed, Nov 13, 2019 at 2:08 PM Tsu, Mr. Justin via RT <
> met_help at ucar.edu>
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039 >
> > >
> > > Got it,
> > >
> > > So since I am using python embedding for my model field, it will
> > > ultimately be up to what vertical level I specify my obs that
will
> > > determine the type of comparison.  For example:
> > >
> > > fcst = {
> > >    field = [{name =
"/users/tsu/MET/work/PY_SRC/read_NRL_binary.py
> > > ${DATA_DIR}/${VAR_NAME}_pre_000.10_0000.0_${GRD}_${INIT
> > > _TIME}_${FCST_HR}_fcstfld ${WIND_FLAG}";}];
> > > }
> > >
> > > obs = {
> > >     field = [{name = "${GRIB_NAME}";level = ["P0.0-0.15"];}];
> > > }
> > >
> > > This comparison is correct because the highlighted portion in my
> forecast
> > > dictionary indicates this model field is at 0.10 hPa and so I
specify
> all
> > > obs that fall within 0.0 and 0.15 hPA will be compared against
this
> model
> > > field.
> > >
> > > The following is incorrect (BUT will still run):
> > >
> > > fcst = {
> > >    field = [{name =
"/users/tsu/MET/work/PY_SRC/read_NRL_binary.py
> > > ${DATA_DIR}/${VAR_NAME}_pre_ 000900_000000_${GRD}_${INIT
> > > _TIME}_${FCST_HR}_fcstfld ${WIND_FLAG}";}];
> > > }
> > >
> > > obs = {
> > >     field = [{name = "${GRIB_NAME}";level = ["P0.0-0.15"];}];
> > > }
> > >
> > > This is an incorrect comparison but will still run.  It is
comparing a
> > > model field at 900 hPa (as seen highlighted) to obs from 0 and
0.15
> hPa.
> > >
> > >
> > > Is everything I just said correct?
> > >
> > > Justin
> > >
> > > -----Original Message-----
> > > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > > Sent: Wednesday, November 13, 2019 1:00 PM
> > > To: Tsu, Mr. Justin
> > > Subject: Re: [rt.rap.ucar.edu #93039] Absurdly high ME2
> > >
> > > Justin,
> > >
> > > Point-Stat looks in the config file at the user-specified level
to be
> > > verified.
> > > For example, let's say you're running Point-Stat to verify wind
speed
> at
> > 2
> > > levels of model output, 850mb pressure level and 100 meters, a
shown
> > below.
> > >
> > > fcst = {
> > >    field = [ { name = "WIND"; level = "P850"; },
> > >                 { name = "WIND"; level = "Z100"; }
> > >    ];
> > > }
> > >
> > > obs = {
> > >    field = [ { name = "WIND"; level = "P840-860"; },
> > >                 { name = "WIND"; level = "Z100"; }
> > >    ];
> > > }
> > >
> > > For the first task, Point-Stat will check the pressure level of
the
> > > obs since the P specifies a pressure level (P840-860).
> > > For the second task, it will check the height of the obs since
the Z
> > > specifies a height (Z100).
> > >
> > > John
> > >
> > > On Wed, Nov 13, 2019 at 1:35 PM Tsu, Mr. Justin via RT <
> > met_help at ucar.edu>
> > > wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039
>
> > > >
> > > > It's okay! Just for clarification, all of my model output is
on sigma
> > > > (hPa) pressure levels.  How does Point-stat know to use the
pressure
> > > level
> > > > column in the observation text file to attach a location value
to the
> > > > observation when comparing to pressure located model fields?
> > > > Justin
> > > >
> > > > -----Original Message-----
> > > > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > > > Sent: Wednesday, November 13, 2019 12:26 PM
> > > > To: Tsu, Mr. Justin
> > > > Subject: Re: [rt.rap.ucar.edu #93039] Absurdly high ME2
> > > >
> > > > Justin,
> > > >
> > > > Typically elevation in MSL is given in meters.
> > > >
> > > > But MET is not enforcing any unit conventions or doing any
unit
> > > conversions
> > > > for you.  Point observations do include an elevation of the
station
> and
> > > the
> > > > height of the observed values.  I don't think the station
elevation
> is
> > > > currently being used in the logic.  But the height of the
> observations
> > > are
> > > > used when vertically interpolating forecasts a specified
heights
> above
> > > > ground to the height of the observation.  For example, if your
model
> > > output
> > > > contains wind speed every 10m above the ground, Point-Stat can
> > > interpolate
> > > > that model output to the height of each observation.  It's
critical
> to
> > > make
> > > > sure that the definition of the observation heights is
consistent
> with
> > > the
> > > > heights of the model output.
> > > >
> > > > They are often either defined relative to MSL or AGL... and
could be
> in
> > > > feet or meters.  So double-check those.
> > > >
> > > > Sorry that the answer isn't more simple!
> > > >
> > > > Thanks,
> > > > John
> > > >
> > > > On Wed, Nov 13, 2019 at 12:16 PM Tsu, Mr. Justin via RT <
> > > met_help at ucar.edu
> > > > >
> > > > wrote:
> > > >
> > > > >
> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039 >
> > > > >
> > > > > Regarding units, I noticed that the units that Elevation of
> observing
> > > > > location and height of the actual measurement need to be in
MSL for
> > use
> > > > in
> > > > > ascii2nc.  What are these units?
> > > > >
> > > > > Justin
> > > > >
> > > > > -----Original Message-----
> > > > > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > > > > Sent: Tuesday, November 12, 2019 2:32 PM
> > > > > To: Tsu, Mr. Justin
> > > > > Subject: Re: [rt.rap.ucar.edu #93039] Absurdly high ME2
> > > > >
> > > > > Justin,
> > > > >
> > > > > OK, thanks for the updates.  Please let me know if you have
any
> > > questions
> > > > > about using the convert() function in the config file to
make those
> > > unit
> > > > > comparable.  Glad you're figuring out the details.
> > > > >
> > > > > Thanks,
> > > > > John
> > > > >
> > > > > On Tue, Nov 12, 2019 at 3:21 PM Tsu, Mr. Justin via RT <
> > > > met_help at ucar.edu>
> > > > > wrote:
> > > > >
> > > > > >
> > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039 >
> > > > > >
> > > > > > Just discovered that I was only computing dew point
depression
> and
> > > not
> > > > > T_d
> > > > > > which is what my fcst is.  Fixed this (which will also fix
the
> > > relative
> > > > > > humidity).  Also discovered that my raobs airtmp are in
Celsius
> but
> > > my
> > > > > sfc
> > > > > > obs are in Kelvin (I assumed sfc was also Celsius so was
doing
> > > > conversion
> > > > > > on that).  Still not sure why my wind direction and speed
have
> high
> > > > > errors
> > > > > > though.
> > > > > >
> > > > > > Justin
> > > > > >
> > > > > > -----Original Message-----
> > > > > > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > > > > > Sent: Friday, November 8, 2019 10:15 PM
> > > > > > To: Tsu, Mr. Justin
> > > > > > Subject: Re: [rt.rap.ucar.edu #93039] Absurdly high ME2
> > > > > >
> > > > > > Justin,
> > > > > >
> > > > > > The first thing I’d check for is a difference in the units
of the
> > > fcst
> > > > > and
> > > > > > obs data which would lead to large error values.
> > > > > >
> > > > > > I’d recommend turning on the matched pair (MPR) output
line
> > type....
> > > > set
> > > > > it
> > > > > > to BOTH.  Then look in the “*_mpr.txt” output file at the
FCST
> and
> > > OBS
> > > > > > columns.  Check the values to see if there’s and obvious
mismatch
> > in
> > > > > units.
> > > > > >
> > > > > > If so, you can define a “convert(x)” function in either
the fcst
> or
> > > obs
> > > > > > dictionaries of the config file to make the units
comparable.
> > > > > >
> > > > > > Thanks,
> > > > > > John
> > > > > >
> > > > > > On Fri, Nov 8, 2019 at 5:05 PM Tsu, Mr. Justin via RT <
> > > > met_help at ucar.edu
> > > > > >
> > > > > > wrote:
> > > > > >
> > > > > > >
> > > > > > > Fri Nov 08 16:04:58 2019: Request 93039 was acted upon.
> > > > > > > Transaction: Ticket created by
justin.tsu at nrlmry.navy.mil
> > > > > > >        Queue: met_help
> > > > > > >      Subject: Absurdly high ME2
> > > > > > >        Owner: Nobody
> > > > > > >   Requestors: justin.tsu at nrlmry.navy.mil
> > > > > > >       Status: new
> > > > > > >  Ticket <URL:
> > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93039
> > > > > >
> > > > > > >
> > > > > > >
> > > > > > > I am getting some absurdly high mean square errors for
every
> one
> > of
> > > > my
> > > > > > > variables (wind speed, wind direction, relative
humidity, dew
> > point
> > > > > > > depression, and air temperature) on each vertical
pressure
> level.
> > > > > These
> > > > > > > values range from the thousands to the 10s of millions.
How
> can
> > I
> > > > > > diagnose
> > > > > > > the cause of these massive errors?
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Some context:
> > > > > > >
> > > > > > > .        I am running point_stat
> > > > > > >
> > > > > > > .        The command is
> > > > > > >
> > > > > > >  point_stat PYTHON_NUMPY ${OBFILE} ${CONFIG}       \
> > > > > > >
> > > > > > >                                   -v 0
\
> > > > > > >
> > > > > > >                                   -outdir ${PSOUT}
\
> > > > > > >
> > > > > > >                                   -log
> > > > > > >
${PSOUT}/${VAR_NAME}${WIND_FLAG}_TAU${FCST_HR}_point_stat.log \
> > > > > > >
> > > > > > >                                   -obs_valid_beg
20010101   \
> > > > > > >
> > > > > > >                                   -obs_valid_end
20200101
> > > > > > >
> > > > > > > .        I've tailored my config file so that the
vertical
> levels
> > > of
> > > > > both
> > > > > > > the fcst and obs are relatively near, so this cannot be
a
> > vertical
> > > > > > mismatch
> > > > > > > issue
> > > > > > >
> > > > > > > Justin
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Justin Tsu
> > > > > > >
> > > > > > > Marine Meteorology Division
> > > > > > >
> > > > > > > Data Assimilation/Mesoscale Modeling
> > > > > > >
> > > > > > > Building 704 Room 212
> > > > > > >
> > > > > > > Naval Research Laboratory, Code 7531
> > > > > > >
> > > > > > > 7 Grace Hopper Avenue
> > > > > > >
> > > > > > > Monterey, CA 93943-5502
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Ph. (831) 656-4111
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > >
> > >
> > >
> > >
> >
> >
> >
>
>
>

------------------------------------------------