[Met_help] [rt.rap.ucar.edu #96881] History for Point Stat Warning (MADIS obs)

John Halley Gotway via RT met_help at ucar.edu
Tue Sep 29 09:58:28 MDT 2020 

----------------------------------------------------------------
  Initial Request
----------------------------------------------------------------

Hi all,

Bit of a housekeeping question for you here.  I've recently upgraded to MET 9.1, and while running point_stat to make sure everything is running smooth, I've noticed it kicks out a "WARNING: get_nc_var(NcFile) --> The variable "obs_var" does not exist!"  This happens only when running with MADIS data, which as been converted over to MET format. I do not get the error with NDAS data.

It appears the .stat and txt files are populating, so at first glance it doesn't appear this is fatal?  Is there any information you can provide as to what may be going on?

The full command and output is below.  If you need any additional info, let me know!

As always, thanks!

*******************************************************************************************

/fewxops/software/met/met-9.1/bin/point_stat /fewxops/verification/data/model/grib2/gfs/2020/gfs_verify_2020080100f084.grb2 /fewxops/verification/data/obs/met_format/madis_meso/mesonet_fe_2020080412.nc /fewxops/verification/data/met_output/working/config_files/point_stat/PointStatConfig_wind_categorical_mph -outdir /fewxops/verification/data/met_output/output/stats/gfs/point_stat/madis/PointStatConfig_wind_categorical_mph/2020080100 -point_obs /fewxops/verification/data/obs/met_format/madis_metar/metar_fe_2020080412.nc


DEBUG 1: Default Config File: /usr/local/share/met/config/PointStatConfig_default
DEBUG 1: User Config File: /fewxops/verification/data/met_output/working/config_files/point_stat/PointStatConfig_wind_categorical_mph
GSL_RNG_TYPE=mt19937
GSL_RNG_SEED=18446744072303309663
DEBUG 1: Forecast File: /fewxops/verification/data/model/grib2/gfs/2020/gfs_verify_2020080100f084.grb2
DEBUG 1: Observation File: /fewxops/verification/data/obs/met_format/madis_meso/mesonet_fe_2020080412.nc
DEBUG 1: Observation File: /fewxops/verification/data/obs/met_format/madis_metar/metar_fe_2020080412.nc
DEBUG 2:
DEBUG 2: --------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Reading data for GUST/L0.
DEBUG 2: Processing masking regions.
DEBUG 2: Processing geography data.
DEBUG 2: For GUST/L0 found 1 forecast levels, 0 climatology mean levels, and 0 climatology standard deviation levels.
DEBUG 2:
DEBUG 2: --------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Reading data for WIND/Z10.
DEBUG 2: For WIND/Z10 found 1 forecast levels, 0 climatology mean levels, and 0 climatology standard deviation levels.
DEBUG 2:
DEBUG 2: --------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Reading data for UGRD/Z10.
DEBUG 2: For UGRD/Z10 found 1 forecast levels, 0 climatology mean levels, and 0 climatology standard deviation levels.
DEBUG 2:
DEBUG 2: --------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Reading data for VGRD/Z10.
DEBUG 2: For VGRD/Z10 found 1 forecast levels, 0 climatology mean levels, and 0 climatology standard deviation levels.
DEBUG 2:
DEBUG 2: --------------------------------------------------------------------------------
DEBUG 2:
WARNING:
WARNING: get_nc_var(NcFile) --> The variable "obs_var" does not exist!
WARNING:
DEBUG 2: Searching 214770 observations from 25470 messages.
WARNING:
WARNING: get_nc_var(NcFile) --> The variable "obs_var" does not exist!
WARNING:
DEBUG 2: Searching 25276 observations from 3112 messages.
DEBUG 2:
DEBUG 2: --------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Processing GUST/L0 versus GUST/L0, for observation type ADPSFC, over region G212, for interpolation method NEAREST(1), using 6101 matched pairs.
DEBUG 2: Computing Categorical Statistics.
DEBUG 2: Computing Multi-Category Statistics.
DEBUG 2: Computing Scalar Partial Sums and Continuous Statistics.
DEBUG 2: Processing GUST/L0 versus GUST/L0, for observation type ADPSFC, over region FE, for interpolation method NEAREST(1), using 5998 matched pairs.
DEBUG 2: Computing Categorical Statistics.
DEBUG 2: Computing Multi-Category Statistics.
DEBUG 2: Computing Scalar Partial Sums and Continuous Statistics.
DEBUG 2:
DEBUG 2: --------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Processing WIND/Z10 versus WIND/Z10, for observation type ADPSFC, over region G212, for interpolation method NEAREST(1), using 8575 matched pairs.
DEBUG 2: Computing Categorical Statistics.
DEBUG 2: Computing Multi-Category Statistics.
DEBUG 2: Computing Scalar Partial Sums and Continuous Statistics.
DEBUG 2: Processing WIND/Z10 versus WIND/Z10, for observation type ADPSFC, over region FE, for interpolation method NEAREST(1), using 6943 matched pairs.
DEBUG 2: Computing Categorical Statistics.
DEBUG 2: Computing Multi-Category Statistics.
DEBUG 2:
DEBUG 2: Processing UGRD/Z10 versus UGRD/Z10, for observation type ADPSFC, over region G212, for interpolation method NEAREST(1), using 8314 matched pairs.
DEBUG 2: Computing Categorical Statistics.
DEBUG 2: Computing Multi-Category Statistics.
DEBUG 2: Computing Scalar Partial Sums and Continuous Statistics.
DEBUG 2: Processing UGRD/Z10 versus UGRD/Z10, for observation type ADPSFC, over region FE, for interpolation method NEAREST(1), using 6712 matched pairs.
DEBUG 2: Computing Categorical Statistics.
DEBUG 2: Computing Multi-Category Statistics.
DEBUG 2: Computing Scalar Partial Sums and Continuous Statistics.
DEBUG 2:
DEBUG 2: --------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Processing VGRD/Z10 versus VGRD/Z10, for observation type ADPSFC, over region G212, for interpolation method NEAREST(1), using 8314 matched pairs.
DEBUG 2: Computing Categorical Statistics.
DEBUG 2: Computing Multi-Category Statistics.
DEBUG 2: Computing Scalar Partial Sums and Continuous Statistics.
DEBUG 2: Processing VGRD/Z10 versus VGRD/Z10, for observation type ADPSFC, over region FE, for interpolation method NEAREST(1), using 6712 matched pairs.
DEBUG 2: Computing Categorical Statistics.
DEBUG 2: Computing Multi-Category Statistics.
DEBUG 2: Computing Scalar Partial Sums and Continuous Statistics.
DEBUG 2:
DEBUG 2: --------------------------------------------------------------------------------
DEBUG 2:
DEBUG 1: Output file: /fewxops/verification/data/met_output/output/stats/gfs/point_stat/madis/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_2020080100_840000L_20200804_120000V.stat
DEBUG 1: Output file: /fewxops/verification/data/met_output/output/stats/gfs/point_stat/madis/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_2020080100_840000L_20200804_120000V_fho.txt
DEBUG 1: Output file: /fewxops/verification/data/met_output/output/stats/gfs/point_stat/madis/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_2020080100_840000L_20200804_120000V_ctc.txt
DEBUG 1: Output file: /fewxops/verification/data/met_output/output/stats/gfs/point_stat/madis/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_2020080100_840000L_20200804_120000V_cts.txt
DEBUG 1: Output file: /fewxops/verification/data/met_output/output/stats/gfs/point_stat/madis/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_2020080100_840000L_20200804_120000V_mcts.txt
DEBUG 1: Output file: /fewxops/verification/data/met_output/output/stats/gfs/point_stat/madis/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_2020080100_840000L_20200804_120000V_cnt.txt
DEBUG 1: Output file: /fewxops/verification/data/met_output/output/stats/gfs/point_stat/madis/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_2020080100_840000L_20200804_120000V_vl1l2.txt
DEBUG 1: Output file: /fewxops/verification/data/met_output/output/stats/gfs/point_stat/madis/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_2020080100_840000L_20200804_120000V_val1l2.txt
DEBUG 1: Output file: /fewxops/verification/data/met_output/output/stats/gfs/point_stat/madis/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_2020080100_840000L_20200804_120000V_sal1l2.txt



[https://firstenergycorp.com/content/dam/opcologos/emailsig/FE-logo.png]

Thomas Workoff
Sr Scientist
office: 330-436-1475 (850-1475)
tworkoff at firstenergycorp.com
341 White Pond Drive, Akron, OH 44320 | mailstop: A-WAC-C1 / AK-West Akron Campus

------------------------------------------------------------------------------

The information contained in this message is intended only for the personal and confidential use of the recipient(s) named above. If the reader of this message is not the intended recipient or an agent responsible for delivering it to the intended recipient, you are hereby notified that you have received this document in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify us immediately, and delete the original message.


----------------------------------------------------------------
  Complete Ticket History
----------------------------------------------------------------

Subject: Point Stat Warning (MADIS obs)
From: Julie Prestopnik
Time: Tue Sep 29 09:01:39 2020

Hi Tom.

Did you convert the MADIS data to NetCDF using the ASCII2NC tool? If
so, it
is my understanding that the ASCII2NC tool generates some NetCDF
outputs
with the GRIB code instead of the variable name. It depends on the
input
file. In that case, the obs value is saved into an "obs_val" variable.
 "obs_var" contains the variable names and is added only if the
variable
names are given.  If your NetCDF file contains "obs_val", the warning
can
safely be ignored if the .stat and txt files are populated.

Please let us know if your NetCDF file contains "obs_val" as opposed
to
"obs_var".  If it does not, please upload all of your files using the
"How
to Send Us Data" directions on this page:

https://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk

and we'll take a look and see if we can better figure out what is
going on.

Thanks!

Julie

On Tue, Sep 29, 2020 at 8:28 AM Workoff, Thomas E via RT
<met_help at ucar.edu>
wrote:

>
> Tue Sep 29 07:03:42 2020: Request 96881 was acted upon.
> Transaction: Ticket created by tworkoff at firstenergycorp.com
>        Queue: met_help
>      Subject: Point Stat Warning (MADIS obs)
>        Owner: Nobody
>   Requestors: tworkoff at firstenergycorp.com
>       Status: new
>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96881 >
>
>
> Hi all,
>
> Bit of a housekeeping question for you here.  I've recently upgraded
to
> MET 9.1, and while running point_stat to make sure everything is
running
> smooth, I've noticed it kicks out a "WARNING: get_nc_var(NcFile) -->
The
> variable "obs_var" does not exist!"  This happens only when running
with
> MADIS data, which as been converted over to MET format. I do not get
the
> error with NDAS data.
>
> It appears the .stat and txt files are populating, so at first
glance it
> doesn't appear this is fatal?  Is there any information you can
provide as
> to what may be going on?
>
> The full command and output is below.  If you need any additional
info,
> let me know!
>
> As always, thanks!
>
>
>
*******************************************************************************************
>
> /fewxops/software/met/met-9.1/bin/point_stat
>
/fewxops/verification/data/model/grib2/gfs/2020/gfs_verify_2020080100f084.grb2
> /fewxops/verification/data/obs/met_format/madis_meso/
> mesonet_fe_2020080412.nc
>
/fewxops/verification/data/met_output/working/config_files/point_stat/PointStatConfig_wind_categorical_mph
> -outdir
>
/fewxops/verification/data/met_output/output/stats/gfs/point_stat/madis/PointStatConfig_wind_categorical_mph/2020080100
> -point_obs /fewxops/verification/data/obs/met_format/madis_metar/
> metar_fe_2020080412.nc
>
>
> DEBUG 1: Default Config File:
> /usr/local/share/met/config/PointStatConfig_default
> DEBUG 1: User Config File:
>
/fewxops/verification/data/met_output/working/config_files/point_stat/PointStatConfig_wind_categorical_mph
> GSL_RNG_TYPE=mt19937
> GSL_RNG_SEED=18446744072303309663
> DEBUG 1: Forecast File:
>
/fewxops/verification/data/model/grib2/gfs/2020/gfs_verify_2020080100f084.grb2
> DEBUG 1: Observation File:
> /fewxops/verification/data/obs/met_format/madis_meso/
> mesonet_fe_2020080412.nc
> DEBUG 1: Observation File:
> /fewxops/verification/data/obs/met_format/madis_metar/
> metar_fe_2020080412.nc
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Reading data for GUST/L0.
> DEBUG 2: Processing masking regions.
> DEBUG 2: Processing geography data.
> DEBUG 2: For GUST/L0 found 1 forecast levels, 0 climatology mean
levels,
> and 0 climatology standard deviation levels.
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Reading data for WIND/Z10.
> DEBUG 2: For WIND/Z10 found 1 forecast levels, 0 climatology mean
levels,
> and 0 climatology standard deviation levels.
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Reading data for UGRD/Z10.
> DEBUG 2: For UGRD/Z10 found 1 forecast levels, 0 climatology mean
levels,
> and 0 climatology standard deviation levels.
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Reading data for VGRD/Z10.
> DEBUG 2: For VGRD/Z10 found 1 forecast levels, 0 climatology mean
levels,
> and 0 climatology standard deviation levels.
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> WARNING:
> WARNING: get_nc_var(NcFile) --> The variable "obs_var" does not
exist!
> WARNING:
> DEBUG 2: Searching 214770 observations from 25470 messages.
> WARNING:
> WARNING: get_nc_var(NcFile) --> The variable "obs_var" does not
exist!
> WARNING:
> DEBUG 2: Searching 25276 observations from 3112 messages.
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Processing GUST/L0 versus GUST/L0, for observation type
ADPSFC,
> over region G212, for interpolation method NEAREST(1), using 6101
matched
> pairs.
> DEBUG 2: Computing Categorical Statistics.
> DEBUG 2: Computing Multi-Category Statistics.
> DEBUG 2: Computing Scalar Partial Sums and Continuous Statistics.
> DEBUG 2: Processing GUST/L0 versus GUST/L0, for observation type
ADPSFC,
> over region FE, for interpolation method NEAREST(1), using 5998
matched
> pairs.
> DEBUG 2: Computing Categorical Statistics.
> DEBUG 2: Computing Multi-Category Statistics.
> DEBUG 2: Computing Scalar Partial Sums and Continuous Statistics.
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Processing WIND/Z10 versus WIND/Z10, for observation type
ADPSFC,
> over region G212, for interpolation method NEAREST(1), using 8575
matched
> pairs.
> DEBUG 2: Computing Categorical Statistics.
> DEBUG 2: Computing Multi-Category Statistics.
> DEBUG 2: Computing Scalar Partial Sums and Continuous Statistics.
> DEBUG 2: Processing WIND/Z10 versus WIND/Z10, for observation type
ADPSFC,
> over region FE, for interpolation method NEAREST(1), using 6943
matched
> pairs.
> DEBUG 2: Computing Categorical Statistics.
> DEBUG 2: Computing Multi-Category Statistics.
> DEBUG 2:
> DEBUG 2: Processing UGRD/Z10 versus UGRD/Z10, for observation type
ADPSFC,
> over region G212, for interpolation method NEAREST(1), using 8314
matched
> pairs.
> DEBUG 2: Computing Categorical Statistics.
> DEBUG 2: Computing Multi-Category Statistics.
> DEBUG 2: Computing Scalar Partial Sums and Continuous Statistics.
> DEBUG 2: Processing UGRD/Z10 versus UGRD/Z10, for observation type
ADPSFC,
> over region FE, for interpolation method NEAREST(1), using 6712
matched
> pairs.
> DEBUG 2: Computing Categorical Statistics.
> DEBUG 2: Computing Multi-Category Statistics.
> DEBUG 2: Computing Scalar Partial Sums and Continuous Statistics.
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Processing VGRD/Z10 versus VGRD/Z10, for observation type
ADPSFC,
> over region G212, for interpolation method NEAREST(1), using 8314
matched
> pairs.
> DEBUG 2: Computing Categorical Statistics.
> DEBUG 2: Computing Multi-Category Statistics.
> DEBUG 2: Computing Scalar Partial Sums and Continuous Statistics.
> DEBUG 2: Processing VGRD/Z10 versus VGRD/Z10, for observation type
ADPSFC,
> over region FE, for interpolation method NEAREST(1), using 6712
matched
> pairs.
> DEBUG 2: Computing Categorical Statistics.
> DEBUG 2: Computing Multi-Category Statistics.
> DEBUG 2: Computing Scalar Partial Sums and Continuous Statistics.
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 1: Output file:
>
/fewxops/verification/data/met_output/output/stats/gfs/point_stat/madis/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_2020080100_840000L_20200804_120000V.stat
> DEBUG 1: Output file:
>
/fewxops/verification/data/met_output/output/stats/gfs/point_stat/madis/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_2020080100_840000L_20200804_120000V_fho.txt
> DEBUG 1: Output file:
>
/fewxops/verification/data/met_output/output/stats/gfs/point_stat/madis/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_2020080100_840000L_20200804_120000V_ctc.txt
> DEBUG 1: Output file:
>
/fewxops/verification/data/met_output/output/stats/gfs/point_stat/madis/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_2020080100_840000L_20200804_120000V_cts.txt
> DEBUG 1: Output file:
>
/fewxops/verification/data/met_output/output/stats/gfs/point_stat/madis/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_2020080100_840000L_20200804_120000V_mcts.txt
> DEBUG 1: Output file:
>
/fewxops/verification/data/met_output/output/stats/gfs/point_stat/madis/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_2020080100_840000L_20200804_120000V_cnt.txt
> DEBUG 1: Output file:
>
/fewxops/verification/data/met_output/output/stats/gfs/point_stat/madis/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_2020080100_840000L_20200804_120000V_vl1l2.txt
> DEBUG 1: Output file:
>
/fewxops/verification/data/met_output/output/stats/gfs/point_stat/madis/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_2020080100_840000L_20200804_120000V_val1l2.txt
> DEBUG 1: Output file:
>
/fewxops/verification/data/met_output/output/stats/gfs/point_stat/madis/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_2020080100_840000L_20200804_120000V_sal1l2.txt
>
>
>
> [https://firstenergycorp.com/content/dam/opcologos/emailsig/FE-
logo.png]
>
> Thomas Workoff
> Sr Scientist
> office: 330-436-1475 (850-1475)
> tworkoff at firstenergycorp.com
> 341 White Pond Drive, Akron, OH 44320 | mailstop: A-WAC-C1 / AK-West
Akron
> Campus
>
>
>
------------------------------------------------------------------------------
>
> The information contained in this message is intended only for the
> personal and confidential use of the recipient(s) named above. If
the
> reader of this message is not the intended recipient or an agent
> responsible for delivering it to the intended recipient, you are
hereby
> notified that you have received this document in error and that any
review,
> dissemination, distribution, or copying of this message is strictly
> prohibited. If you have received this communication in error, please
notify
> us immediately, and delete the original message.
>
>

--
Julie Prestopnik (she/her/hers)
Software Engineer
National Center for Atmospheric Research
Research Applications Laboratory
Email: jpresto at ucar.edu

My working day may not be your working day.  Please do not feel
obliged to
reply to this email outside of your normal working hours.

------------------------------------------------
Subject: Point Stat Warning (MADIS obs)
From: John Halley Gotway
Time: Tue Sep 29 09:24:32 2020

Julie and Tom,

This is not a problem. In earlier versions of MET, we identified point
observations by mapping them to GRIB codes. A few years ago, we moved
away
from that and began using strings to describe the observation types
instead
since they are much, much more flexible.

For example, instead of storing temperature with a value of 11 (which
is
the GRIB1 Table 2 id for it), we store it as the string "TMP".
However, the
MET statistics tools should be backward compatible, being able to read
the
older files following the older GRIB code convention or newer files
following the newer string convention.

This warning just tells you that your data follows the older
convention.

Can you tell me... did you pre-process the "MADIS" obs using the
madis2nc
tool or ascii2nc? If it was ascii2nc and your ascii input files
identify
obs using integers, then that's the explanation for this.

John

On Tue, Sep 29, 2020 at 9:02 AM Julie Prestopnik via RT
<met_help at ucar.edu>
wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96881 >
>
> Hi Tom.
>
> Did you convert the MADIS data to NetCDF using the ASCII2NC tool? If
so, it
> is my understanding that the ASCII2NC tool generates some NetCDF
outputs
> with the GRIB code instead of the variable name. It depends on the
input
> file. In that case, the obs value is saved into an "obs_val"
variable.
>  "obs_var" contains the variable names and is added only if the
variable
> names are given.  If your NetCDF file contains "obs_val", the
warning can
> safely be ignored if the .stat and txt files are populated.
>
> Please let us know if your NetCDF file contains "obs_val" as opposed
to
> "obs_var".  If it does not, please upload all of your files using
the "How
> to Send Us Data" directions on this page:
>
>
> https://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk
>
> and we'll take a look and see if we can better figure out what is
going on.
>
> Thanks!
>
> Julie
>
> On Tue, Sep 29, 2020 at 8:28 AM Workoff, Thomas E via RT <
> met_help at ucar.edu>
> wrote:
>
> >
> > Tue Sep 29 07:03:42 2020: Request 96881 was acted upon.
> > Transaction: Ticket created by tworkoff at firstenergycorp.com
> >        Queue: met_help
> >      Subject: Point Stat Warning (MADIS obs)
> >        Owner: Nobody
> >   Requestors: tworkoff at firstenergycorp.com
> >       Status: new
> >  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96881 >
> >
> >
> > Hi all,
> >
> > Bit of a housekeeping question for you here.  I've recently
upgraded to
> > MET 9.1, and while running point_stat to make sure everything is
running
> > smooth, I've noticed it kicks out a "WARNING: get_nc_var(NcFile)
--> The
> > variable "obs_var" does not exist!"  This happens only when
running with
> > MADIS data, which as been converted over to MET format. I do not
get the
> > error with NDAS data.
> >
> > It appears the .stat and txt files are populating, so at first
glance it
> > doesn't appear this is fatal?  Is there any information you can
provide
> as
> > to what may be going on?
> >
> > The full command and output is below.  If you need any additional
info,
> > let me know!
> >
> > As always, thanks!
> >
> >
> >
>
*******************************************************************************************
> >
> > /fewxops/software/met/met-9.1/bin/point_stat
> >
>
/fewxops/verification/data/model/grib2/gfs/2020/gfs_verify_2020080100f084.grb2
> > /fewxops/verification/data/obs/met_format/madis_meso/
> > mesonet_fe_2020080412.nc
> >
>
/fewxops/verification/data/met_output/working/config_files/point_stat/PointStatConfig_wind_categorical_mph
> > -outdir
> >
>
/fewxops/verification/data/met_output/output/stats/gfs/point_stat/madis/PointStatConfig_wind_categorical_mph/2020080100
> > -point_obs /fewxops/verification/data/obs/met_format/madis_metar/
> > metar_fe_2020080412.nc
> >
> >
> > DEBUG 1: Default Config File:
> > /usr/local/share/met/config/PointStatConfig_default
> > DEBUG 1: User Config File:
> >
>
/fewxops/verification/data/met_output/working/config_files/point_stat/PointStatConfig_wind_categorical_mph
> > GSL_RNG_TYPE=mt19937
> > GSL_RNG_SEED=18446744072303309663
> > DEBUG 1: Forecast File:
> >
>
/fewxops/verification/data/model/grib2/gfs/2020/gfs_verify_2020080100f084.grb2
> > DEBUG 1: Observation File:
> > /fewxops/verification/data/obs/met_format/madis_meso/
> > mesonet_fe_2020080412.nc
> > DEBUG 1: Observation File:
> > /fewxops/verification/data/obs/met_format/madis_metar/
> > metar_fe_2020080412.nc
> > DEBUG 2:
> > DEBUG 2:
> >
>
--------------------------------------------------------------------------------
> > DEBUG 2:
> > DEBUG 2: Reading data for GUST/L0.
> > DEBUG 2: Processing masking regions.
> > DEBUG 2: Processing geography data.
> > DEBUG 2: For GUST/L0 found 1 forecast levels, 0 climatology mean
levels,
> > and 0 climatology standard deviation levels.
> > DEBUG 2:
> > DEBUG 2:
> >
>
--------------------------------------------------------------------------------
> > DEBUG 2:
> > DEBUG 2: Reading data for WIND/Z10.
> > DEBUG 2: For WIND/Z10 found 1 forecast levels, 0 climatology mean
levels,
> > and 0 climatology standard deviation levels.
> > DEBUG 2:
> > DEBUG 2:
> >
>
--------------------------------------------------------------------------------
> > DEBUG 2:
> > DEBUG 2: Reading data for UGRD/Z10.
> > DEBUG 2: For UGRD/Z10 found 1 forecast levels, 0 climatology mean
levels,
> > and 0 climatology standard deviation levels.
> > DEBUG 2:
> > DEBUG 2:
> >
>
--------------------------------------------------------------------------------
> > DEBUG 2:
> > DEBUG 2: Reading data for VGRD/Z10.
> > DEBUG 2: For VGRD/Z10 found 1 forecast levels, 0 climatology mean
levels,
> > and 0 climatology standard deviation levels.
> > DEBUG 2:
> > DEBUG 2:
> >
>
--------------------------------------------------------------------------------
> > DEBUG 2:
> > WARNING:
> > WARNING: get_nc_var(NcFile) --> The variable "obs_var" does not
exist!
> > WARNING:
> > DEBUG 2: Searching 214770 observations from 25470 messages.
> > WARNING:
> > WARNING: get_nc_var(NcFile) --> The variable "obs_var" does not
exist!
> > WARNING:
> > DEBUG 2: Searching 25276 observations from 3112 messages.
> > DEBUG 2:
> > DEBUG 2:
> >
>
--------------------------------------------------------------------------------
> > DEBUG 2:
> > DEBUG 2: Processing GUST/L0 versus GUST/L0, for observation type
ADPSFC,
> > over region G212, for interpolation method NEAREST(1), using 6101
matched
> > pairs.
> > DEBUG 2: Computing Categorical Statistics.
> > DEBUG 2: Computing Multi-Category Statistics.
> > DEBUG 2: Computing Scalar Partial Sums and Continuous Statistics.
> > DEBUG 2: Processing GUST/L0 versus GUST/L0, for observation type
ADPSFC,
> > over region FE, for interpolation method NEAREST(1), using 5998
matched
> > pairs.
> > DEBUG 2: Computing Categorical Statistics.
> > DEBUG 2: Computing Multi-Category Statistics.
> > DEBUG 2: Computing Scalar Partial Sums and Continuous Statistics.
> > DEBUG 2:
> > DEBUG 2:
> >
>
--------------------------------------------------------------------------------
> > DEBUG 2:
> > DEBUG 2: Processing WIND/Z10 versus WIND/Z10, for observation type
> ADPSFC,
> > over region G212, for interpolation method NEAREST(1), using 8575
matched
> > pairs.
> > DEBUG 2: Computing Categorical Statistics.
> > DEBUG 2: Computing Multi-Category Statistics.
> > DEBUG 2: Computing Scalar Partial Sums and Continuous Statistics.
> > DEBUG 2: Processing WIND/Z10 versus WIND/Z10, for observation type
> ADPSFC,
> > over region FE, for interpolation method NEAREST(1), using 6943
matched
> > pairs.
> > DEBUG 2: Computing Categorical Statistics.
> > DEBUG 2: Computing Multi-Category Statistics.
> > DEBUG 2:
> > DEBUG 2: Processing UGRD/Z10 versus UGRD/Z10, for observation type
> ADPSFC,
> > over region G212, for interpolation method NEAREST(1), using 8314
matched
> > pairs.
> > DEBUG 2: Computing Categorical Statistics.
> > DEBUG 2: Computing Multi-Category Statistics.
> > DEBUG 2: Computing Scalar Partial Sums and Continuous Statistics.
> > DEBUG 2: Processing UGRD/Z10 versus UGRD/Z10, for observation type
> ADPSFC,
> > over region FE, for interpolation method NEAREST(1), using 6712
matched
> > pairs.
> > DEBUG 2: Computing Categorical Statistics.
> > DEBUG 2: Computing Multi-Category Statistics.
> > DEBUG 2: Computing Scalar Partial Sums and Continuous Statistics.
> > DEBUG 2:
> > DEBUG 2:
> >
>
--------------------------------------------------------------------------------
> > DEBUG 2:
> > DEBUG 2: Processing VGRD/Z10 versus VGRD/Z10, for observation type
> ADPSFC,
> > over region G212, for interpolation method NEAREST(1), using 8314
matched
> > pairs.
> > DEBUG 2: Computing Categorical Statistics.
> > DEBUG 2: Computing Multi-Category Statistics.
> > DEBUG 2: Computing Scalar Partial Sums and Continuous Statistics.
> > DEBUG 2: Processing VGRD/Z10 versus VGRD/Z10, for observation type
> ADPSFC,
> > over region FE, for interpolation method NEAREST(1), using 6712
matched
> > pairs.
> > DEBUG 2: Computing Categorical Statistics.
> > DEBUG 2: Computing Multi-Category Statistics.
> > DEBUG 2: Computing Scalar Partial Sums and Continuous Statistics.
> > DEBUG 2:
> > DEBUG 2:
> >
>
--------------------------------------------------------------------------------
> > DEBUG 2:
> > DEBUG 1: Output file:
> >
>
/fewxops/verification/data/met_output/output/stats/gfs/point_stat/madis/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_2020080100_840000L_20200804_120000V.stat
> > DEBUG 1: Output file:
> >
>
/fewxops/verification/data/met_output/output/stats/gfs/point_stat/madis/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_2020080100_840000L_20200804_120000V_fho.txt
> > DEBUG 1: Output file:
> >
>
/fewxops/verification/data/met_output/output/stats/gfs/point_stat/madis/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_2020080100_840000L_20200804_120000V_ctc.txt
> > DEBUG 1: Output file:
> >
>
/fewxops/verification/data/met_output/output/stats/gfs/point_stat/madis/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_2020080100_840000L_20200804_120000V_cts.txt
> > DEBUG 1: Output file:
> >
>
/fewxops/verification/data/met_output/output/stats/gfs/point_stat/madis/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_2020080100_840000L_20200804_120000V_mcts.txt
> > DEBUG 1: Output file:
> >
>
/fewxops/verification/data/met_output/output/stats/gfs/point_stat/madis/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_2020080100_840000L_20200804_120000V_cnt.txt
> > DEBUG 1: Output file:
> >
>
/fewxops/verification/data/met_output/output/stats/gfs/point_stat/madis/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_2020080100_840000L_20200804_120000V_vl1l2.txt
> > DEBUG 1: Output file:
> >
>
/fewxops/verification/data/met_output/output/stats/gfs/point_stat/madis/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_2020080100_840000L_20200804_120000V_val1l2.txt
> > DEBUG 1: Output file:
> >
>
/fewxops/verification/data/met_output/output/stats/gfs/point_stat/madis/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_2020080100_840000L_20200804_120000V_sal1l2.txt
> >
> >
> >
> > [https://firstenergycorp.com/content/dam/opcologos/emailsig/FE-
logo.png]
> >
> > Thomas Workoff
> > Sr Scientist
> > office: 330-436-1475 (850-1475)
> > tworkoff at firstenergycorp.com
> > 341 White Pond Drive, Akron, OH 44320 | mailstop: A-WAC-C1 / AK-
West
> Akron
> > Campus
> >
> >
> >
>
------------------------------------------------------------------------------
> >
> > The information contained in this message is intended only for the
> > personal and confidential use of the recipient(s) named above. If
the
> > reader of this message is not the intended recipient or an agent
> > responsible for delivering it to the intended recipient, you are
hereby
> > notified that you have received this document in error and that
any
> review,
> > dissemination, distribution, or copying of this message is
strictly
> > prohibited. If you have received this communication in error,
please
> notify
> > us immediately, and delete the original message.
> >
> >
>
> --
> Julie Prestopnik (she/her/hers)
> Software Engineer
> National Center for Atmospheric Research
> Research Applications Laboratory
> Email: jpresto at ucar.edu
>
> My working day may not be your working day.  Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>

------------------------------------------------
Subject: RE: [EXTERNAL] Re: [rt.rap.ucar.edu #96881] Point Stat Warning (MADIS obs)
From: Workoff, Thomas E
Time: Tue Sep 29 09:33:15 2020

Julie and John,

Thanks, as always, for the quick reply!

The data was converted from madis2nc--both mesonet and metar data is
reformatted this way.  I know using the reformatted metar data in
point-stat kicks this warning, but I haven't had time this morning to
test the mesonet data.  I'm assuming it'd throw the warning too.

The obs_val variable is present in the converted .nc file.  A snapshot
of the ncdump is below.  As Julie mentioned, this probably explains
why the code ran successfully?

For what it's worth, I don't remember seeing this warning when I was
running 8.1.2.  I converted to 9.1 over the past few weeks, and this
seems to be when the warning popped up.  I could be mis-remembering
this--but it caught my attention when the warning flashed.  Maybe I'm
just losing my mind.

I originally thought it could be from re-running the point_stat from
9.1 against MADIS data that had been reformatted with  madis2nc from
8.1.2. I'm not sure that makes sense, but it was my first guess.  So I
re-converted the raw MADIS data using madis2nc from 9.1 just to make
sure...but the warning still flashes.

Since the code still runs, I'm not concerned.  I just wanted to make
sure it wasn't potentially affecting the statistics processing behind
the scenes, and that I could still trust the output.

As always, I appreciate the help and discussion!

********************************************
        float obs_hgt(nobs) ;
                obs_hgt:long_name = "height in meters above sea level
(msl)" ;
                obs_hgt:_FillValue = -9999.f ;
        float obs_val(nobs) ;

                obs_val:long_name = "observation value" ;
                obs_val:_FillValue = -9999.f ;
        char hdr_typ_table(nhdr_typ, mxstr2) ;
                hdr_typ_table:long_name = "message type" ;


 


-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Tuesday, September 29, 2020 11:25 AM
To: Workoff, Thomas E <tworkoff at firstenergycorp.com>
Subject: [EXTERNAL] Re: [rt.rap.ucar.edu #96881] Point Stat Warning
(MADIS obs)

Julie and Tom,

This is not a problem. In earlier versions of MET, we identified point
observations by mapping them to GRIB codes. A few years ago, we moved
away from that and began using strings to describe the observation
types instead since they are much, much more flexible.

For example, instead of storing temperature with a value of 11 (which
is the GRIB1 Table 2 id for it), we store it as the string "TMP".
However, the MET statistics tools should be backward compatible, being
able to read the older files following the older GRIB code convention
or newer files following the newer string convention.

This warning just tells you that your data follows the older
convention.

Can you tell me... did you pre-process the "MADIS" obs using the
madis2nc tool or ascii2nc? If it was ascii2nc and your ascii input
files identify obs using integers, then that's the explanation for
this.

John

On Tue, Sep 29, 2020 at 9:02 AM Julie Prestopnik via RT
<met_help at ucar.edu>
wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96881 >
>
> Hi Tom.
>
> Did you convert the MADIS data to NetCDF using the ASCII2NC tool? If
> so, it is my understanding that the ASCII2NC tool generates some
> NetCDF outputs with the GRIB code instead of the variable name. It
> depends on the input file. In that case, the obs value is saved into
an "obs_val" variable.
>  "obs_var" contains the variable names and is added only if the
> variable names are given.  If your NetCDF file contains "obs_val",
the
> warning can safely be ignored if the .stat and txt files are
populated.
>
> Please let us know if your NetCDF file contains "obs_val" as opposed
> to "obs_var".  If it does not, please upload all of your files using
> the "How to Send Us Data" directions on this page:
>
>
> https://secure-web.cisco.com/1jWNrzi5eHU4JGR9SC-
QLRtu6OFVcBAZSfXhFh7Km
> Ob8vvQarQ-
UjjKwoWPWjYfKIwfCB4cPpAmp5YJPubMttQAYw1JLfrlsYbyfd1Pm6xt2Jpf
> 8QR8HHVIsCsdAX7ES9xNZaKxTQnZFWv6stjRErXP7D88Pv-
sFdSrpBRY9uC9LQoWn64rS-
> ZI0fHkOcAtmPosyVK5kyVUgb0mnWK3HhQRt7Klz6tFpGGQw0V04d6f62ai3AA6eQC4y-
1U
> -8jRU5kUkQ0YpGny0D1YnqW3syr87-
Rd3iYVyH0pprk0xKVpnWSGV2c_XakRjZx9Hp_QCT
> llceEZSeEnzfRbTwA5A7Kqx621w2gipQzPJJaXDf-
1c_fm2pnZhgYJ4RWx7YQqNY/https
> %3A%2F%2Fdtcenter.org%2Fcommunity-code%2Fmodel-evaluation-tools-
met%2F
> met-help-desk
>
> and we'll take a look and see if we can better figure out what is
going on.
>
> Thanks!
>
> Julie
>
> On Tue, Sep 29, 2020 at 8:28 AM Workoff, Thomas E via RT <
> met_help at ucar.edu>
> wrote:
>
> >
> > Tue Sep 29 07:03:42 2020: Request 96881 was acted upon.
> > Transaction: Ticket created by tworkoff at firstenergycorp.com
> >        Queue: met_help
> >      Subject: Point Stat Warning (MADIS obs)
> >        Owner: Nobody
> >   Requestors: tworkoff at firstenergycorp.com
> >       Status: new
> >  Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96881 >
> >
> >
> > Hi all,
> >
> > Bit of a housekeeping question for you here.  I've recently
upgraded
> > to MET 9.1, and while running point_stat to make sure everything
is
> > running smooth, I've noticed it kicks out a "WARNING:
> > get_nc_var(NcFile) --> The variable "obs_var" does not exist!"
This
> > happens only when running with MADIS data, which as been converted
> > over to MET format. I do not get the error with NDAS data.
> >
> > It appears the .stat and txt files are populating, so at first
> > glance it doesn't appear this is fatal?  Is there any information
> > you can provide
> as
> > to what may be going on?
> >
> > The full command and output is below.  If you need any additional
> > info, let me know!
> >
> > As always, thanks!
> >
> >
> >
>
**********************************************************************
> *********************
> >
> > /fewxops/software/met/met-9.1/bin/point_stat
> >
>
/fewxops/verification/data/model/grib2/gfs/2020/gfs_verify_2020080100f
> 084.grb2
> > /fewxops/verification/data/obs/met_format/madis_meso/
> > mesonet_fe_2020080412.nc
> >
>
/fewxops/verification/data/met_output/working/config_files/point_stat/
> PointStatConfig_wind_categorical_mph
> > -outdir
> >
>
/fewxops/verification/data/met_output/output/stats/gfs/point_stat/madi
> s/PointStatConfig_wind_categorical_mph/2020080100
> > -point_obs /fewxops/verification/data/obs/met_format/madis_metar/
> > metar_fe_2020080412.nc
> >
> >
> > DEBUG 1: Default Config File:
> > /usr/local/share/met/config/PointStatConfig_default
> > DEBUG 1: User Config File:
> >
>
/fewxops/verification/data/met_output/working/config_files/point_stat/
> PointStatConfig_wind_categorical_mph
> > GSL_RNG_TYPE=mt19937
> > GSL_RNG_SEED=18446744072303309663
> > DEBUG 1: Forecast File:
> >
>
/fewxops/verification/data/model/grib2/gfs/2020/gfs_verify_2020080100f
> 084.grb2
> > DEBUG 1: Observation File:
> > /fewxops/verification/data/obs/met_format/madis_meso/
> > mesonet_fe_2020080412.nc
> > DEBUG 1: Observation File:
> > /fewxops/verification/data/obs/met_format/madis_metar/
> > metar_fe_2020080412.nc
> > DEBUG 2:
> > DEBUG 2:
> >
>
----------------------------------------------------------------------
> ----------
> > DEBUG 2:
> > DEBUG 2: Reading data for GUST/L0.
> > DEBUG 2: Processing masking regions.
> > DEBUG 2: Processing geography data.
> > DEBUG 2: For GUST/L0 found 1 forecast levels, 0 climatology mean
> > levels, and 0 climatology standard deviation levels.
> > DEBUG 2:
> > DEBUG 2:
> >
>
----------------------------------------------------------------------
> ----------
> > DEBUG 2:
> > DEBUG 2: Reading data for WIND/Z10.
> > DEBUG 2: For WIND/Z10 found 1 forecast levels, 0 climatology mean
> > levels, and 0 climatology standard deviation levels.
> > DEBUG 2:
> > DEBUG 2:
> >
>
----------------------------------------------------------------------
> ----------
> > DEBUG 2:
> > DEBUG 2: Reading data for UGRD/Z10.
> > DEBUG 2: For UGRD/Z10 found 1 forecast levels, 0 climatology mean
> > levels, and 0 climatology standard deviation levels.
> > DEBUG 2:
> > DEBUG 2:
> >
>
----------------------------------------------------------------------
> ----------
> > DEBUG 2:
> > DEBUG 2: Reading data for VGRD/Z10.
> > DEBUG 2: For VGRD/Z10 found 1 forecast levels, 0 climatology mean
> > levels, and 0 climatology standard deviation levels.
> > DEBUG 2:
> > DEBUG 2:
> >
>
----------------------------------------------------------------------
> ----------
> > DEBUG 2:
> > WARNING:
> > WARNING: get_nc_var(NcFile) --> The variable "obs_var" does not
exist!
> > WARNING:
> > DEBUG 2: Searching 214770 observations from 25470 messages.
> > WARNING:
> > WARNING: get_nc_var(NcFile) --> The variable "obs_var" does not
exist!
> > WARNING:
> > DEBUG 2: Searching 25276 observations from 3112 messages.
> > DEBUG 2:
> > DEBUG 2:
> >
>
----------------------------------------------------------------------
> ----------
> > DEBUG 2:
> > DEBUG 2: Processing GUST/L0 versus GUST/L0, for observation type
> > ADPSFC, over region G212, for interpolation method NEAREST(1),
using
> > 6101 matched pairs.
> > DEBUG 2: Computing Categorical Statistics.
> > DEBUG 2: Computing Multi-Category Statistics.
> > DEBUG 2: Computing Scalar Partial Sums and Continuous Statistics.
> > DEBUG 2: Processing GUST/L0 versus GUST/L0, for observation type
> > ADPSFC, over region FE, for interpolation method NEAREST(1), using
> > 5998 matched pairs.
> > DEBUG 2: Computing Categorical Statistics.
> > DEBUG 2: Computing Multi-Category Statistics.
> > DEBUG 2: Computing Scalar Partial Sums and Continuous Statistics.
> > DEBUG 2:
> > DEBUG 2:
> >
>
----------------------------------------------------------------------
> ----------
> > DEBUG 2:
> > DEBUG 2: Processing WIND/Z10 versus WIND/Z10, for observation type
> ADPSFC,
> > over region G212, for interpolation method NEAREST(1), using 8575
> > matched pairs.
> > DEBUG 2: Computing Categorical Statistics.
> > DEBUG 2: Computing Multi-Category Statistics.
> > DEBUG 2: Computing Scalar Partial Sums and Continuous Statistics.
> > DEBUG 2: Processing WIND/Z10 versus WIND/Z10, for observation type
> ADPSFC,
> > over region FE, for interpolation method NEAREST(1), using 6943
> > matched pairs.
> > DEBUG 2: Computing Categorical Statistics.
> > DEBUG 2: Computing Multi-Category Statistics.
> > DEBUG 2:
> > DEBUG 2: Processing UGRD/Z10 versus UGRD/Z10, for observation type
> ADPSFC,
> > over region G212, for interpolation method NEAREST(1), using 8314
> > matched pairs.
> > DEBUG 2: Computing Categorical Statistics.
> > DEBUG 2: Computing Multi-Category Statistics.
> > DEBUG 2: Computing Scalar Partial Sums and Continuous Statistics.
> > DEBUG 2: Processing UGRD/Z10 versus UGRD/Z10, for observation type
> ADPSFC,
> > over region FE, for interpolation method NEAREST(1), using 6712
> > matched pairs.
> > DEBUG 2: Computing Categorical Statistics.
> > DEBUG 2: Computing Multi-Category Statistics.
> > DEBUG 2: Computing Scalar Partial Sums and Continuous Statistics.
> > DEBUG 2:
> > DEBUG 2:
> >
>
----------------------------------------------------------------------
> ----------
> > DEBUG 2:
> > DEBUG 2: Processing VGRD/Z10 versus VGRD/Z10, for observation type
> ADPSFC,
> > over region G212, for interpolation method NEAREST(1), using 8314
> > matched pairs.
> > DEBUG 2: Computing Categorical Statistics.
> > DEBUG 2: Computing Multi-Category Statistics.
> > DEBUG 2: Computing Scalar Partial Sums and Continuous Statistics.
> > DEBUG 2: Processing VGRD/Z10 versus VGRD/Z10, for observation type
> ADPSFC,
> > over region FE, for interpolation method NEAREST(1), using 6712
> > matched pairs.
> > DEBUG 2: Computing Categorical Statistics.
> > DEBUG 2: Computing Multi-Category Statistics.
> > DEBUG 2: Computing Scalar Partial Sums and Continuous Statistics.
> > DEBUG 2:
> > DEBUG 2:
> >
>
----------------------------------------------------------------------
> ----------
> > DEBUG 2:
> > DEBUG 1: Output file:
> >
>
/fewxops/verification/data/met_output/output/stats/gfs/point_stat/madi
>
s/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_202008
> 0100_840000L_20200804_120000V.stat
> > DEBUG 1: Output file:
> >
>
/fewxops/verification/data/met_output/output/stats/gfs/point_stat/madi
>
s/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_202008
> 0100_840000L_20200804_120000V_fho.txt
> > DEBUG 1: Output file:
> >
>
/fewxops/verification/data/met_output/output/stats/gfs/point_stat/madi
>
s/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_202008
> 0100_840000L_20200804_120000V_ctc.txt
> > DEBUG 1: Output file:
> >
>
/fewxops/verification/data/met_output/output/stats/gfs/point_stat/madi
>
s/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_202008
> 0100_840000L_20200804_120000V_cts.txt
> > DEBUG 1: Output file:
> >
>
/fewxops/verification/data/met_output/output/stats/gfs/point_stat/madi
>
s/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_202008
> 0100_840000L_20200804_120000V_mcts.txt
> > DEBUG 1: Output file:
> >
>
/fewxops/verification/data/met_output/output/stats/gfs/point_stat/madi
>
s/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_202008
> 0100_840000L_20200804_120000V_cnt.txt
> > DEBUG 1: Output file:
> >
>
/fewxops/verification/data/met_output/output/stats/gfs/point_stat/madi
>
s/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_202008
> 0100_840000L_20200804_120000V_vl1l2.txt
> > DEBUG 1: Output file:
> >
>
/fewxops/verification/data/met_output/output/stats/gfs/point_stat/madi
>
s/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_202008
> 0100_840000L_20200804_120000V_val1l2.txt
> > DEBUG 1: Output file:
> >
>
/fewxops/verification/data/met_output/output/stats/gfs/point_stat/madi
>
s/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_202008
> 0100_840000L_20200804_120000V_sal1l2.txt
> >
> >
> >
> > [https://firstenergycorp.com/content/dam/opcologos/emailsig/FE-
logo.
> > png]
> >
> > Thomas Workoff
> > Sr Scientist
> > office: 330-436-1475 (850-1475)
> > tworkoff at firstenergycorp.com
> > 341 White Pond Drive, Akron, OH 44320 | mailstop: A-WAC-C1 / AK-
West
> Akron
> > Campus
> >
> >
> >
>
----------------------------------------------------------------------
> --------
> >
> > The information contained in this message is intended only for the
> > personal and confidential use of the recipient(s) named above. If
> > the reader of this message is not the intended recipient or an
agent
> > responsible for delivering it to the intended recipient, you are
> > hereby notified that you have received this document in error and
> > that any
> review,
> > dissemination, distribution, or copying of this message is
strictly
> > prohibited. If you have received this communication in error,
please
> notify
> > us immediately, and delete the original message.
> >
> >
>
> --
> Julie Prestopnik (she/her/hers)
> Software Engineer
> National Center for Atmospheric Research Research Applications
> Laboratory
> Email: jpresto at ucar.edu
>
> My working day may not be your working day.  Please do not feel
> obliged to reply to this email outside of your normal working hours.
>
>


------------------------------------------------------------------------------

The information contained in this message is intended only for the
personal and confidential use of the recipient(s) named above. If the
reader of this message is not the intended recipient or an agent
responsible for delivering it to the intended recipient, you are
hereby notified that you have received this document in error and that
any review, dissemination, distribution, or copying of this message is
strictly prohibited. If you have received this communication in error,
please notify us immediately, and delete the original message.


------------------------------------------------
Subject: Point Stat Warning (MADIS obs)
From: John Halley Gotway
Time: Tue Sep 29 09:46:32 2020

Tom,

To be clear, it's "obs_var", not "obs_val".  The "obs_var" variable
stores
the list of unique observation variable names present in the input.
And for
each observation, we store an index into that obs_var array.

So I just checked the output of our nightly build and see that NONE of
the
madis2nc output files have the obs_var variable present in their
output.

We actually have a MET development meeting this morning. I'll ask the
developer who worked on this why we don't.

And if not, we probably shouldn't be printing that warning message!

Thanks for letting us know.

John

On Tue, Sep 29, 2020 at 9:33 AM Workoff, Thomas E via RT
<met_help at ucar.edu>
wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96881 >
>
> Julie and John,
>
> Thanks, as always, for the quick reply!
>
> The data was converted from madis2nc--both mesonet and metar data is
> reformatted this way.  I know using the reformatted metar data in
> point-stat kicks this warning, but I haven't had time this morning
to test
> the mesonet data.  I'm assuming it'd throw the warning too.
>
> The obs_val variable is present in the converted .nc file.  A
snapshot of
> the ncdump is below.  As Julie mentioned, this probably explains why
the
> code ran successfully?
>
> For what it's worth, I don't remember seeing this warning when I was
> running 8.1.2.  I converted to 9.1 over the past few weeks, and this
seems
> to be when the warning popped up.  I could be mis-remembering this--
but it
> caught my attention when the warning flashed.  Maybe I'm just losing
my
> mind.
>
> I originally thought it could be from re-running the point_stat from
9.1
> against MADIS data that had been reformatted with  madis2nc from
8.1.2. I'm
> not sure that makes sense, but it was my first guess.  So I re-
converted
> the raw MADIS data using madis2nc from 9.1 just to make sure...but
the
> warning still flashes.
>
> Since the code still runs, I'm not concerned.  I just wanted to make
sure
> it wasn't potentially affecting the statistics processing behind the
> scenes, and that I could still trust the output.
>
> As always, I appreciate the help and discussion!
>
> ********************************************
>         float obs_hgt(nobs) ;
>                 obs_hgt:long_name = "height in meters above sea
level
> (msl)" ;
>                 obs_hgt:_FillValue = -9999.f ;
>         float obs_val(nobs) ;
>
>                 obs_val:long_name = "observation value" ;
>                 obs_val:_FillValue = -9999.f ;
>         char hdr_typ_table(nhdr_typ, mxstr2) ;
>                 hdr_typ_table:long_name = "message type" ;
>
>
>
>
>
> -----Original Message-----
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Tuesday, September 29, 2020 11:25 AM
> To: Workoff, Thomas E <tworkoff at firstenergycorp.com>
> Subject: [EXTERNAL] Re: [rt.rap.ucar.edu #96881] Point Stat Warning
> (MADIS obs)
>
> Julie and Tom,
>
> This is not a problem. In earlier versions of MET, we identified
point
> observations by mapping them to GRIB codes. A few years ago, we
moved away
> from that and began using strings to describe the observation types
instead
> since they are much, much more flexible.
>
> For example, instead of storing temperature with a value of 11
(which is
> the GRIB1 Table 2 id for it), we store it as the string "TMP".
However, the
> MET statistics tools should be backward compatible, being able to
read the
> older files following the older GRIB code convention or newer files
> following the newer string convention.
>
> This warning just tells you that your data follows the older
convention.
>
> Can you tell me... did you pre-process the "MADIS" obs using the
madis2nc
> tool or ascii2nc? If it was ascii2nc and your ascii input files
identify
> obs using integers, then that's the explanation for this.
>
> John
>
> On Tue, Sep 29, 2020 at 9:02 AM Julie Prestopnik via RT
<met_help at ucar.edu
> >
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96881 >
> >
> > Hi Tom.
> >
> > Did you convert the MADIS data to NetCDF using the ASCII2NC tool?
If
> > so, it is my understanding that the ASCII2NC tool generates some
> > NetCDF outputs with the GRIB code instead of the variable name. It
> > depends on the input file. In that case, the obs value is saved
into an
> "obs_val" variable.
> >  "obs_var" contains the variable names and is added only if the
> > variable names are given.  If your NetCDF file contains "obs_val",
the
> > warning can safely be ignored if the .stat and txt files are
populated.
> >
> > Please let us know if your NetCDF file contains "obs_val" as
opposed
> > to "obs_var".  If it does not, please upload all of your files
using
> > the "How to Send Us Data" directions on this page:
> >
> >
> > https://secure-web.cisco.com/1jWNrzi5eHU4JGR9SC-
QLRtu6OFVcBAZSfXhFh7Km
> > Ob8vvQarQ-
UjjKwoWPWjYfKIwfCB4cPpAmp5YJPubMttQAYw1JLfrlsYbyfd1Pm6xt2Jpf
> > 8QR8HHVIsCsdAX7ES9xNZaKxTQnZFWv6stjRErXP7D88Pv-
sFdSrpBRY9uC9LQoWn64rS-
> >
ZI0fHkOcAtmPosyVK5kyVUgb0mnWK3HhQRt7Klz6tFpGGQw0V04d6f62ai3AA6eQC4y-1U
> > -8jRU5kUkQ0YpGny0D1YnqW3syr87-
Rd3iYVyH0pprk0xKVpnWSGV2c_XakRjZx9Hp_QCT
> > llceEZSeEnzfRbTwA5A7Kqx621w2gipQzPJJaXDf-
1c_fm2pnZhgYJ4RWx7YQqNY/https
> > %3A%2F%2Fdtcenter.org%2Fcommunity-code%2Fmodel-evaluation-tools-
met%2F
> > met-help-desk
> >
> > and we'll take a look and see if we can better figure out what is
going
> on.
> >
> > Thanks!
> >
> > Julie
> >
> > On Tue, Sep 29, 2020 at 8:28 AM Workoff, Thomas E via RT <
> > met_help at ucar.edu>
> > wrote:
> >
> > >
> > > Tue Sep 29 07:03:42 2020: Request 96881 was acted upon.
> > > Transaction: Ticket created by tworkoff at firstenergycorp.com
> > >        Queue: met_help
> > >      Subject: Point Stat Warning (MADIS obs)
> > >        Owner: Nobody
> > >   Requestors: tworkoff at firstenergycorp.com
> > >       Status: new
> > >  Ticket <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96881 >
> > >
> > >
> > > Hi all,
> > >
> > > Bit of a housekeeping question for you here.  I've recently
upgraded
> > > to MET 9.1, and while running point_stat to make sure everything
is
> > > running smooth, I've noticed it kicks out a "WARNING:
> > > get_nc_var(NcFile) --> The variable "obs_var" does not exist!"
This
> > > happens only when running with MADIS data, which as been
converted
> > > over to MET format. I do not get the error with NDAS data.
> > >
> > > It appears the .stat and txt files are populating, so at first
> > > glance it doesn't appear this is fatal?  Is there any
information
> > > you can provide
> > as
> > > to what may be going on?
> > >
> > > The full command and output is below.  If you need any
additional
> > > info, let me know!
> > >
> > > As always, thanks!
> > >
> > >
> > >
> >
**********************************************************************
> > *********************
> > >
> > > /fewxops/software/met/met-9.1/bin/point_stat
> > >
> >
/fewxops/verification/data/model/grib2/gfs/2020/gfs_verify_2020080100f
> > 084.grb2
> > > /fewxops/verification/data/obs/met_format/madis_meso/
> > > mesonet_fe_2020080412.nc
> > >
> >
/fewxops/verification/data/met_output/working/config_files/point_stat/
> > PointStatConfig_wind_categorical_mph
> > > -outdir
> > >
> >
/fewxops/verification/data/met_output/output/stats/gfs/point_stat/madi
> > s/PointStatConfig_wind_categorical_mph/2020080100
> > > -point_obs
/fewxops/verification/data/obs/met_format/madis_metar/
> > > metar_fe_2020080412.nc
> > >
> > >
> > > DEBUG 1: Default Config File:
> > > /usr/local/share/met/config/PointStatConfig_default
> > > DEBUG 1: User Config File:
> > >
> >
/fewxops/verification/data/met_output/working/config_files/point_stat/
> > PointStatConfig_wind_categorical_mph
> > > GSL_RNG_TYPE=mt19937
> > > GSL_RNG_SEED=18446744072303309663
> > > DEBUG 1: Forecast File:
> > >
> >
/fewxops/verification/data/model/grib2/gfs/2020/gfs_verify_2020080100f
> > 084.grb2
> > > DEBUG 1: Observation File:
> > > /fewxops/verification/data/obs/met_format/madis_meso/
> > > mesonet_fe_2020080412.nc
> > > DEBUG 1: Observation File:
> > > /fewxops/verification/data/obs/met_format/madis_metar/
> > > metar_fe_2020080412.nc
> > > DEBUG 2:
> > > DEBUG 2:
> > >
> >
----------------------------------------------------------------------
> > ----------
> > > DEBUG 2:
> > > DEBUG 2: Reading data for GUST/L0.
> > > DEBUG 2: Processing masking regions.
> > > DEBUG 2: Processing geography data.
> > > DEBUG 2: For GUST/L0 found 1 forecast levels, 0 climatology mean
> > > levels, and 0 climatology standard deviation levels.
> > > DEBUG 2:
> > > DEBUG 2:
> > >
> >
----------------------------------------------------------------------
> > ----------
> > > DEBUG 2:
> > > DEBUG 2: Reading data for WIND/Z10.
> > > DEBUG 2: For WIND/Z10 found 1 forecast levels, 0 climatology
mean
> > > levels, and 0 climatology standard deviation levels.
> > > DEBUG 2:
> > > DEBUG 2:
> > >
> >
----------------------------------------------------------------------
> > ----------
> > > DEBUG 2:
> > > DEBUG 2: Reading data for UGRD/Z10.
> > > DEBUG 2: For UGRD/Z10 found 1 forecast levels, 0 climatology
mean
> > > levels, and 0 climatology standard deviation levels.
> > > DEBUG 2:
> > > DEBUG 2:
> > >
> >
----------------------------------------------------------------------
> > ----------
> > > DEBUG 2:
> > > DEBUG 2: Reading data for VGRD/Z10.
> > > DEBUG 2: For VGRD/Z10 found 1 forecast levels, 0 climatology
mean
> > > levels, and 0 climatology standard deviation levels.
> > > DEBUG 2:
> > > DEBUG 2:
> > >
> >
----------------------------------------------------------------------
> > ----------
> > > DEBUG 2:
> > > WARNING:
> > > WARNING: get_nc_var(NcFile) --> The variable "obs_var" does not
exist!
> > > WARNING:
> > > DEBUG 2: Searching 214770 observations from 25470 messages.
> > > WARNING:
> > > WARNING: get_nc_var(NcFile) --> The variable "obs_var" does not
exist!
> > > WARNING:
> > > DEBUG 2: Searching 25276 observations from 3112 messages.
> > > DEBUG 2:
> > > DEBUG 2:
> > >
> >
----------------------------------------------------------------------
> > ----------
> > > DEBUG 2:
> > > DEBUG 2: Processing GUST/L0 versus GUST/L0, for observation type
> > > ADPSFC, over region G212, for interpolation method NEAREST(1),
using
> > > 6101 matched pairs.
> > > DEBUG 2: Computing Categorical Statistics.
> > > DEBUG 2: Computing Multi-Category Statistics.
> > > DEBUG 2: Computing Scalar Partial Sums and Continuous
Statistics.
> > > DEBUG 2: Processing GUST/L0 versus GUST/L0, for observation type
> > > ADPSFC, over region FE, for interpolation method NEAREST(1),
using
> > > 5998 matched pairs.
> > > DEBUG 2: Computing Categorical Statistics.
> > > DEBUG 2: Computing Multi-Category Statistics.
> > > DEBUG 2: Computing Scalar Partial Sums and Continuous
Statistics.
> > > DEBUG 2:
> > > DEBUG 2:
> > >
> >
----------------------------------------------------------------------
> > ----------
> > > DEBUG 2:
> > > DEBUG 2: Processing WIND/Z10 versus WIND/Z10, for observation
type
> > ADPSFC,
> > > over region G212, for interpolation method NEAREST(1), using
8575
> > > matched pairs.
> > > DEBUG 2: Computing Categorical Statistics.
> > > DEBUG 2: Computing Multi-Category Statistics.
> > > DEBUG 2: Computing Scalar Partial Sums and Continuous
Statistics.
> > > DEBUG 2: Processing WIND/Z10 versus WIND/Z10, for observation
type
> > ADPSFC,
> > > over region FE, for interpolation method NEAREST(1), using 6943
> > > matched pairs.
> > > DEBUG 2: Computing Categorical Statistics.
> > > DEBUG 2: Computing Multi-Category Statistics.
> > > DEBUG 2:
> > > DEBUG 2: Processing UGRD/Z10 versus UGRD/Z10, for observation
type
> > ADPSFC,
> > > over region G212, for interpolation method NEAREST(1), using
8314
> > > matched pairs.
> > > DEBUG 2: Computing Categorical Statistics.
> > > DEBUG 2: Computing Multi-Category Statistics.
> > > DEBUG 2: Computing Scalar Partial Sums and Continuous
Statistics.
> > > DEBUG 2: Processing UGRD/Z10 versus UGRD/Z10, for observation
type
> > ADPSFC,
> > > over region FE, for interpolation method NEAREST(1), using 6712
> > > matched pairs.
> > > DEBUG 2: Computing Categorical Statistics.
> > > DEBUG 2: Computing Multi-Category Statistics.
> > > DEBUG 2: Computing Scalar Partial Sums and Continuous
Statistics.
> > > DEBUG 2:
> > > DEBUG 2:
> > >
> >
----------------------------------------------------------------------
> > ----------
> > > DEBUG 2:
> > > DEBUG 2: Processing VGRD/Z10 versus VGRD/Z10, for observation
type
> > ADPSFC,
> > > over region G212, for interpolation method NEAREST(1), using
8314
> > > matched pairs.
> > > DEBUG 2: Computing Categorical Statistics.
> > > DEBUG 2: Computing Multi-Category Statistics.
> > > DEBUG 2: Computing Scalar Partial Sums and Continuous
Statistics.
> > > DEBUG 2: Processing VGRD/Z10 versus VGRD/Z10, for observation
type
> > ADPSFC,
> > > over region FE, for interpolation method NEAREST(1), using 6712
> > > matched pairs.
> > > DEBUG 2: Computing Categorical Statistics.
> > > DEBUG 2: Computing Multi-Category Statistics.
> > > DEBUG 2: Computing Scalar Partial Sums and Continuous
Statistics.
> > > DEBUG 2:
> > > DEBUG 2:
> > >
> >
----------------------------------------------------------------------
> > ----------
> > > DEBUG 2:
> > > DEBUG 1: Output file:
> > >
> >
/fewxops/verification/data/met_output/output/stats/gfs/point_stat/madi
> >
s/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_202008
> > 0100_840000L_20200804_120000V.stat
> > > DEBUG 1: Output file:
> > >
> >
/fewxops/verification/data/met_output/output/stats/gfs/point_stat/madi
> >
s/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_202008
> > 0100_840000L_20200804_120000V_fho.txt
> > > DEBUG 1: Output file:
> > >
> >
/fewxops/verification/data/met_output/output/stats/gfs/point_stat/madi
> >
s/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_202008
> > 0100_840000L_20200804_120000V_ctc.txt
> > > DEBUG 1: Output file:
> > >
> >
/fewxops/verification/data/met_output/output/stats/gfs/point_stat/madi
> >
s/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_202008
> > 0100_840000L_20200804_120000V_cts.txt
> > > DEBUG 1: Output file:
> > >
> >
/fewxops/verification/data/met_output/output/stats/gfs/point_stat/madi
> >
s/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_202008
> > 0100_840000L_20200804_120000V_mcts.txt
> > > DEBUG 1: Output file:
> > >
> >
/fewxops/verification/data/met_output/output/stats/gfs/point_stat/madi
> >
s/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_202008
> > 0100_840000L_20200804_120000V_cnt.txt
> > > DEBUG 1: Output file:
> > >
> >
/fewxops/verification/data/met_output/output/stats/gfs/point_stat/madi
> >
s/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_202008
> > 0100_840000L_20200804_120000V_vl1l2.txt
> > > DEBUG 1: Output file:
> > >
> >
/fewxops/verification/data/met_output/output/stats/gfs/point_stat/madi
> >
s/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_202008
> > 0100_840000L_20200804_120000V_val1l2.txt
> > > DEBUG 1: Output file:
> > >
> >
/fewxops/verification/data/met_output/output/stats/gfs/point_stat/madi
> >
s/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_202008
> > 0100_840000L_20200804_120000V_sal1l2.txt
> > >
> > >
> > >
> > > [https://firstenergycorp.com/content/dam/opcologos/emailsig/FE-
logo.
> > > png]
> > >
> > > Thomas Workoff
> > > Sr Scientist
> > > office: 330-436-1475 (850-1475)
> > > tworkoff at firstenergycorp.com
> > > 341 White Pond Drive, Akron, OH 44320 | mailstop: A-WAC-C1 / AK-
West
> > Akron
> > > Campus
> > >
> > >
> > >
> >
----------------------------------------------------------------------
> > --------
> > >
> > > The information contained in this message is intended only for
the
> > > personal and confidential use of the recipient(s) named above.
If
> > > the reader of this message is not the intended recipient or an
agent
> > > responsible for delivering it to the intended recipient, you are
> > > hereby notified that you have received this document in error
and
> > > that any
> > review,
> > > dissemination, distribution, or copying of this message is
strictly
> > > prohibited. If you have received this communication in error,
please
> > notify
> > > us immediately, and delete the original message.
> > >
> > >
> >
> > --
> > Julie Prestopnik (she/her/hers)
> > Software Engineer
> > National Center for Atmospheric Research Research Applications
> > Laboratory
> > Email: jpresto at ucar.edu
> >
> > My working day may not be your working day.  Please do not feel
> > obliged to reply to this email outside of your normal working
hours.
> >
> >
>
>
>
>
------------------------------------------------------------------------------
>
> The information contained in this message is intended only for the
> personal and confidential use of the recipient(s) named above. If
the
> reader of this message is not the intended recipient or an agent
> responsible for delivering it to the intended recipient, you are
hereby
> notified that you have received this document in error and that any
review,
> dissemination, distribution, or copying of this message is strictly
> prohibited. If you have received this communication in error, please
notify
> us immediately, and delete the original message.
>
>
>

------------------------------------------------
Subject: RE: [EXTERNAL] Re: [rt.rap.ucar.edu #96881] Point Stat Warning (MADIS obs)
From: Workoff, Thomas E
Time: Tue Sep 29 09:50:20 2020

Thanks for the help, John and Julie.

I will move ahead ignoring the warning, unless I hear differently from
the MET team.

Since things are ok on my end, feel free to close this ticket--unless
you prefer to keep it open for the time being.

Thanks again!



-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Tuesday, September 29, 2020 11:47 AM
To: Workoff, Thomas E <tworkoff at firstenergycorp.com>
Subject: Re: [EXTERNAL] Re: [rt.rap.ucar.edu #96881] Point Stat
Warning (MADIS obs)

Tom,

To be clear, it's "obs_var", not "obs_val".  The "obs_var" variable
stores the list of unique observation variable names present in the
input. And for each observation, we store an index into that obs_var
array.

So I just checked the output of our nightly build and see that NONE of
the madis2nc output files have the obs_var variable present in their
output.

We actually have a MET development meeting this morning. I'll ask the
developer who worked on this why we don't.

And if not, we probably shouldn't be printing that warning message!

Thanks for letting us know.

John

On Tue, Sep 29, 2020 at 9:33 AM Workoff, Thomas E via RT
<met_help at ucar.edu>
wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96881 >
>
> Julie and John,
>
> Thanks, as always, for the quick reply!
>
> The data was converted from madis2nc--both mesonet and metar data is
> reformatted this way.  I know using the reformatted metar data in
> point-stat kicks this warning, but I haven't had time this morning
to
> test the mesonet data.  I'm assuming it'd throw the warning too.
>
> The obs_val variable is present in the converted .nc file.  A
snapshot
> of the ncdump is below.  As Julie mentioned, this probably explains
> why the code ran successfully?
>
> For what it's worth, I don't remember seeing this warning when I was
> running 8.1.2.  I converted to 9.1 over the past few weeks, and this
> seems to be when the warning popped up.  I could be mis-remembering
> this--but it caught my attention when the warning flashed.  Maybe
I'm
> just losing my mind.
>
> I originally thought it could be from re-running the point_stat from
> 9.1 against MADIS data that had been reformatted with  madis2nc from
> 8.1.2. I'm not sure that makes sense, but it was my first guess.  So
I
> re-converted the raw MADIS data using madis2nc from 9.1 just to make
> sure...but the warning still flashes.
>
> Since the code still runs, I'm not concerned.  I just wanted to make
> sure it wasn't potentially affecting the statistics processing
behind
> the scenes, and that I could still trust the output.
>
> As always, I appreciate the help and discussion!
>
> ********************************************
>         float obs_hgt(nobs) ;
>                 obs_hgt:long_name = "height in meters above sea
level
> (msl)" ;
>                 obs_hgt:_FillValue = -9999.f ;
>         float obs_val(nobs) ;
>
>                 obs_val:long_name = "observation value" ;
>                 obs_val:_FillValue = -9999.f ;
>         char hdr_typ_table(nhdr_typ, mxstr2) ;
>                 hdr_typ_table:long_name = "message type" ;
>
>
>
>
>
> -----Original Message-----
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Tuesday, September 29, 2020 11:25 AM
> To: Workoff, Thomas E <tworkoff at firstenergycorp.com>
> Subject: [EXTERNAL] Re: [rt.rap.ucar.edu #96881] Point Stat Warning
> (MADIS obs)
>
> Julie and Tom,
>
> This is not a problem. In earlier versions of MET, we identified
point
> observations by mapping them to GRIB codes. A few years ago, we
moved
> away from that and began using strings to describe the observation
> types instead since they are much, much more flexible.
>
> For example, instead of storing temperature with a value of 11
(which
> is the GRIB1 Table 2 id for it), we store it as the string "TMP".
> However, the MET statistics tools should be backward compatible,
being
> able to read the older files following the older GRIB code
convention
> or newer files following the newer string convention.
>
> This warning just tells you that your data follows the older
convention.
>
> Can you tell me... did you pre-process the "MADIS" obs using the
> madis2nc tool or ascii2nc? If it was ascii2nc and your ascii input
> files identify obs using integers, then that's the explanation for
this.
>
> John
>
> On Tue, Sep 29, 2020 at 9:02 AM Julie Prestopnik via RT
> <met_help at ucar.edu
> >
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96881 >
> >
> > Hi Tom.
> >
> > Did you convert the MADIS data to NetCDF using the ASCII2NC tool?
If
> > so, it is my understanding that the ASCII2NC tool generates some
> > NetCDF outputs with the GRIB code instead of the variable name. It
> > depends on the input file. In that case, the obs value is saved
into
> > an
> "obs_val" variable.
> >  "obs_var" contains the variable names and is added only if the
> > variable names are given.  If your NetCDF file contains "obs_val",
> > the warning can safely be ignored if the .stat and txt files are
populated.
> >
> > Please let us know if your NetCDF file contains "obs_val" as
opposed
> > to "obs_var".  If it does not, please upload all of your files
using
> > the "How to Send Us Data" directions on this page:
> >
> >
> > https://secure-web.cisco.com/1jWNrzi5eHU4JGR9SC-
QLRtu6OFVcBAZSfXhFh7
> > Km
> > Ob8vvQarQ-
UjjKwoWPWjYfKIwfCB4cPpAmp5YJPubMttQAYw1JLfrlsYbyfd1Pm6xt2J
> > pf
> > 8QR8HHVIsCsdAX7ES9xNZaKxTQnZFWv6stjRErXP7D88Pv-
sFdSrpBRY9uC9LQoWn64r
> > S-
> >
ZI0fHkOcAtmPosyVK5kyVUgb0mnWK3HhQRt7Klz6tFpGGQw0V04d6f62ai3AA6eQC4y-
> > 1U
> > -8jRU5kUkQ0YpGny0D1YnqW3syr87-
Rd3iYVyH0pprk0xKVpnWSGV2c_XakRjZx9Hp_Q
> > CT
> > llceEZSeEnzfRbTwA5A7Kqx621w2gipQzPJJaXDf-
1c_fm2pnZhgYJ4RWx7YQqNY/htt
> > ps
> > %3A%2F%2Fdtcenter.org%2Fcommunity-code%2Fmodel-evaluation-tools-
met%
> > 2F
> > met-help-desk
> >
> > and we'll take a look and see if we can better figure out what is
> > going
> on.
> >
> > Thanks!
> >
> > Julie
> >
> > On Tue, Sep 29, 2020 at 8:28 AM Workoff, Thomas E via RT <
> > met_help at ucar.edu>
> > wrote:
> >
> > >
> > > Tue Sep 29 07:03:42 2020: Request 96881 was acted upon.
> > > Transaction: Ticket created by tworkoff at firstenergycorp.com
> > >        Queue: met_help
> > >      Subject: Point Stat Warning (MADIS obs)
> > >        Owner: Nobody
> > >   Requestors: tworkoff at firstenergycorp.com
> > >       Status: new
> > >  Ticket <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96881 >
> > >
> > >
> > > Hi all,
> > >
> > > Bit of a housekeeping question for you here.  I've recently
> > > upgraded to MET 9.1, and while running point_stat to make sure
> > > everything is running smooth, I've noticed it kicks out a
"WARNING:
> > > get_nc_var(NcFile) --> The variable "obs_var" does not exist!"
> > > This happens only when running with MADIS data, which as been
> > > converted over to MET format. I do not get the error with NDAS
data.
> > >
> > > It appears the .stat and txt files are populating, so at first
> > > glance it doesn't appear this is fatal?  Is there any
information
> > > you can provide
> > as
> > > to what may be going on?
> > >
> > > The full command and output is below.  If you need any
additional
> > > info, let me know!
> > >
> > > As always, thanks!
> > >
> > >
> > >
> >
********************************************************************
> > **
> > *********************
> > >
> > > /fewxops/software/met/met-9.1/bin/point_stat
> > >
> >
/fewxops/verification/data/model/grib2/gfs/2020/gfs_verify_202008010
> > 0f
> > 084.grb2
> > > /fewxops/verification/data/obs/met_format/madis_meso/
> > > mesonet_fe_2020080412.nc
> > >
> >
/fewxops/verification/data/met_output/working/config_files/point_sta
> > t/ PointStatConfig_wind_categorical_mph
> > > -outdir
> > >
> >
/fewxops/verification/data/met_output/output/stats/gfs/point_stat/ma
> > di
> > s/PointStatConfig_wind_categorical_mph/2020080100
> > > -point_obs
/fewxops/verification/data/obs/met_format/madis_metar/
> > > metar_fe_2020080412.nc
> > >
> > >
> > > DEBUG 1: Default Config File:
> > > /usr/local/share/met/config/PointStatConfig_default
> > > DEBUG 1: User Config File:
> > >
> >
/fewxops/verification/data/met_output/working/config_files/point_sta
> > t/ PointStatConfig_wind_categorical_mph
> > > GSL_RNG_TYPE=mt19937
> > > GSL_RNG_SEED=18446744072303309663
> > > DEBUG 1: Forecast File:
> > >
> >
/fewxops/verification/data/model/grib2/gfs/2020/gfs_verify_202008010
> > 0f
> > 084.grb2
> > > DEBUG 1: Observation File:
> > > /fewxops/verification/data/obs/met_format/madis_meso/
> > > mesonet_fe_2020080412.nc
> > > DEBUG 1: Observation File:
> > > /fewxops/verification/data/obs/met_format/madis_metar/
> > > metar_fe_2020080412.nc
> > > DEBUG 2:
> > > DEBUG 2:
> > >
> >
--------------------------------------------------------------------
> > --
> > ----------
> > > DEBUG 2:
> > > DEBUG 2: Reading data for GUST/L0.
> > > DEBUG 2: Processing masking regions.
> > > DEBUG 2: Processing geography data.
> > > DEBUG 2: For GUST/L0 found 1 forecast levels, 0 climatology mean
> > > levels, and 0 climatology standard deviation levels.
> > > DEBUG 2:
> > > DEBUG 2:
> > >
> >
--------------------------------------------------------------------
> > --
> > ----------
> > > DEBUG 2:
> > > DEBUG 2: Reading data for WIND/Z10.
> > > DEBUG 2: For WIND/Z10 found 1 forecast levels, 0 climatology
mean
> > > levels, and 0 climatology standard deviation levels.
> > > DEBUG 2:
> > > DEBUG 2:
> > >
> >
--------------------------------------------------------------------
> > --
> > ----------
> > > DEBUG 2:
> > > DEBUG 2: Reading data for UGRD/Z10.
> > > DEBUG 2: For UGRD/Z10 found 1 forecast levels, 0 climatology
mean
> > > levels, and 0 climatology standard deviation levels.
> > > DEBUG 2:
> > > DEBUG 2:
> > >
> >
--------------------------------------------------------------------
> > --
> > ----------
> > > DEBUG 2:
> > > DEBUG 2: Reading data for VGRD/Z10.
> > > DEBUG 2: For VGRD/Z10 found 1 forecast levels, 0 climatology
mean
> > > levels, and 0 climatology standard deviation levels.
> > > DEBUG 2:
> > > DEBUG 2:
> > >
> >
--------------------------------------------------------------------
> > --
> > ----------
> > > DEBUG 2:
> > > WARNING:
> > > WARNING: get_nc_var(NcFile) --> The variable "obs_var" does not
exist!
> > > WARNING:
> > > DEBUG 2: Searching 214770 observations from 25470 messages.
> > > WARNING:
> > > WARNING: get_nc_var(NcFile) --> The variable "obs_var" does not
exist!
> > > WARNING:
> > > DEBUG 2: Searching 25276 observations from 3112 messages.
> > > DEBUG 2:
> > > DEBUG 2:
> > >
> >
--------------------------------------------------------------------
> > --
> > ----------
> > > DEBUG 2:
> > > DEBUG 2: Processing GUST/L0 versus GUST/L0, for observation type
> > > ADPSFC, over region G212, for interpolation method NEAREST(1),
> > > using
> > > 6101 matched pairs.
> > > DEBUG 2: Computing Categorical Statistics.
> > > DEBUG 2: Computing Multi-Category Statistics.
> > > DEBUG 2: Computing Scalar Partial Sums and Continuous
Statistics.
> > > DEBUG 2: Processing GUST/L0 versus GUST/L0, for observation type
> > > ADPSFC, over region FE, for interpolation method NEAREST(1),
using
> > > 5998 matched pairs.
> > > DEBUG 2: Computing Categorical Statistics.
> > > DEBUG 2: Computing Multi-Category Statistics.
> > > DEBUG 2: Computing Scalar Partial Sums and Continuous
Statistics.
> > > DEBUG 2:
> > > DEBUG 2:
> > >
> >
--------------------------------------------------------------------
> > --
> > ----------
> > > DEBUG 2:
> > > DEBUG 2: Processing WIND/Z10 versus WIND/Z10, for observation
type
> > ADPSFC,
> > > over region G212, for interpolation method NEAREST(1), using
8575
> > > matched pairs.
> > > DEBUG 2: Computing Categorical Statistics.
> > > DEBUG 2: Computing Multi-Category Statistics.
> > > DEBUG 2: Computing Scalar Partial Sums and Continuous
Statistics.
> > > DEBUG 2: Processing WIND/Z10 versus WIND/Z10, for observation
type
> > ADPSFC,
> > > over region FE, for interpolation method NEAREST(1), using 6943
> > > matched pairs.
> > > DEBUG 2: Computing Categorical Statistics.
> > > DEBUG 2: Computing Multi-Category Statistics.
> > > DEBUG 2:
> > > DEBUG 2: Processing UGRD/Z10 versus UGRD/Z10, for observation
type
> > ADPSFC,
> > > over region G212, for interpolation method NEAREST(1), using
8314
> > > matched pairs.
> > > DEBUG 2: Computing Categorical Statistics.
> > > DEBUG 2: Computing Multi-Category Statistics.
> > > DEBUG 2: Computing Scalar Partial Sums and Continuous
Statistics.
> > > DEBUG 2: Processing UGRD/Z10 versus UGRD/Z10, for observation
type
> > ADPSFC,
> > > over region FE, for interpolation method NEAREST(1), using 6712
> > > matched pairs.
> > > DEBUG 2: Computing Categorical Statistics.
> > > DEBUG 2: Computing Multi-Category Statistics.
> > > DEBUG 2: Computing Scalar Partial Sums and Continuous
Statistics.
> > > DEBUG 2:
> > > DEBUG 2:
> > >
> >
--------------------------------------------------------------------
> > --
> > ----------
> > > DEBUG 2:
> > > DEBUG 2: Processing VGRD/Z10 versus VGRD/Z10, for observation
type
> > ADPSFC,
> > > over region G212, for interpolation method NEAREST(1), using
8314
> > > matched pairs.
> > > DEBUG 2: Computing Categorical Statistics.
> > > DEBUG 2: Computing Multi-Category Statistics.
> > > DEBUG 2: Computing Scalar Partial Sums and Continuous
Statistics.
> > > DEBUG 2: Processing VGRD/Z10 versus VGRD/Z10, for observation
type
> > ADPSFC,
> > > over region FE, for interpolation method NEAREST(1), using 6712
> > > matched pairs.
> > > DEBUG 2: Computing Categorical Statistics.
> > > DEBUG 2: Computing Multi-Category Statistics.
> > > DEBUG 2: Computing Scalar Partial Sums and Continuous
Statistics.
> > > DEBUG 2:
> > > DEBUG 2:
> > >
> >
--------------------------------------------------------------------
> > --
> > ----------
> > > DEBUG 2:
> > > DEBUG 1: Output file:
> > >
> >
/fewxops/verification/data/met_output/output/stats/gfs/point_stat/ma
> > di
> >
s/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_2020
> > 08
> > 0100_840000L_20200804_120000V.stat
> > > DEBUG 1: Output file:
> > >
> >
/fewxops/verification/data/met_output/output/stats/gfs/point_stat/ma
> > di
> >
s/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_2020
> > 08 0100_840000L_20200804_120000V_fho.txt
> > > DEBUG 1: Output file:
> > >
> >
/fewxops/verification/data/met_output/output/stats/gfs/point_stat/ma
> > di
> >
s/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_2020
> > 08 0100_840000L_20200804_120000V_ctc.txt
> > > DEBUG 1: Output file:
> > >
> >
/fewxops/verification/data/met_output/output/stats/gfs/point_stat/ma
> > di
> >
s/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_2020
> > 08 0100_840000L_20200804_120000V_cts.txt
> > > DEBUG 1: Output file:
> > >
> >
/fewxops/verification/data/met_output/output/stats/gfs/point_stat/ma
> > di
> >
s/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_2020
> > 08 0100_840000L_20200804_120000V_mcts.txt
> > > DEBUG 1: Output file:
> > >
> >
/fewxops/verification/data/met_output/output/stats/gfs/point_stat/ma
> > di
> >
s/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_2020
> > 08 0100_840000L_20200804_120000V_cnt.txt
> > > DEBUG 1: Output file:
> > >
> >
/fewxops/verification/data/met_output/output/stats/gfs/point_stat/ma
> > di
> >
s/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_2020
> > 08 0100_840000L_20200804_120000V_vl1l2.txt
> > > DEBUG 1: Output file:
> > >
> >
/fewxops/verification/data/met_output/output/stats/gfs/point_stat/ma
> > di
> >
s/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_2020
> > 08 0100_840000L_20200804_120000V_val1l2.txt
> > > DEBUG 1: Output file:
> > >
> >
/fewxops/verification/data/met_output/output/stats/gfs/point_stat/ma
> > di
> >
s/PointStatConfig_wind_categorical_mph/2020080100/point_stat_FE_2020
> > 08 0100_840000L_20200804_120000V_sal1l2.txt
> > >
> > >
> > >
> > > [https://firstenergycorp.com/content/dam/opcologos/emailsig/FE-
logo.
> > > png]
> > >
> > > Thomas Workoff
> > > Sr Scientist
> > > office: 330-436-1475 (850-1475)
> > > tworkoff at firstenergycorp.com
> > > 341 White Pond Drive, Akron, OH 44320 | mailstop: A-WAC-C1 /
> > > AK-West
> > Akron
> > > Campus
> > >
> > >
> > >
> >
--------------------------------------------------------------------
> > --
> > --------
> > >
> > > The information contained in this message is intended only for
the
> > > personal and confidential use of the recipient(s) named above.
If
> > > the reader of this message is not the intended recipient or an
> > > agent responsible for delivering it to the intended recipient,
you
> > > are hereby notified that you have received this document in
error
> > > and that any
> > review,
> > > dissemination, distribution, or copying of this message is
> > > strictly prohibited. If you have received this communication in
> > > error, please
> > notify
> > > us immediately, and delete the original message.
> > >
> > >
> >
> > --
> > Julie Prestopnik (she/her/hers)
> > Software Engineer
> > National Center for Atmospheric Research Research Applications
> > Laboratory
> > Email: jpresto at ucar.edu
> >
> > My working day may not be your working day.  Please do not feel
> > obliged to reply to this email outside of your normal working
hours.
> >
> >
>
>
>
>
----------------------------------------------------------------------
> --------
>
> The information contained in this message is intended only for the
> personal and confidential use of the recipient(s) named above. If
the
> reader of this message is not the intended recipient or an agent
> responsible for delivering it to the intended recipient, you are
> hereby notified that you have received this document in error and
that
> any review, dissemination, distribution, or copying of this message
is
> strictly prohibited. If you have received this communication in
error,
> please notify us immediately, and delete the original message.
>
>
>

------------------------------------------------------------------------------

The information contained in this message is intended only for the
personal and confidential use of the recipient(s) named above. If the
reader of this message is not the intended recipient or an agent
responsible for delivering it to the intended recipient, you are
hereby notified that you have received this document in error and that
any review, dissemination, distribution, or copying of this message is
strictly prohibited. If you have received this communication in error,
please notify us immediately, and delete the original message.


------------------------------------------------

More information about the Met_help mailing list