[Met_help] [rt.rap.ucar.edu #87432] History for MET execution error while reading longitude from wrf data using point_stat exe

John Halley Gotway via RT met_help at ucar.edu
Tue Oct 23 09:54:13 MDT 2018


----------------------------------------------------------------
  Initial Request
----------------------------------------------------------------

Dear sir

    I have a problem while executing point_stat exe with wrf data processed by upp software and observation data processed by ascii2nc exe. Point_stat read the wrong longitude value from wrf data,the lon_ll from the wrf data is 79.881,but the log shows -79.881。the log and PointStatConfig is below,how to set the config to make point_stat read the right value of wrf grib1 data ?

log:

DEBUG 1: Default Config File: /home/huayunshengda/Downloads/met-7.0/share/met/config/PointStatConfig_default
DEBUG 1: User Config File: PointStatConfig_default
GSL_RNG_TYPE=mt19937
GSL_RNG_SEED=488556635
DEBUG 1: Forecast File: WRFPRS_d01.20180916.00
DEBUG 1: Observation File: 20180916000000.nc
DEBUG 2:
DEBUG 2: --------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Reading data for TMP/L1.
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match for VarInfo "TMP/L1" in GRIB record 5 of GRIB file "WRFPRS_d01.20180916.00".
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB records matching VarInfo "TMP/L1" in GRIB file "WRFPRS_d01.20180916.00".
DEBUG 3: Use the matching forecast and observation grids.
DEBUG 3: Grid Definition: Projection: Lambert Conformal Nx: 570 Ny: 390 Lat_LL: 9.384 Lon_LL: -79.881 Lon_orient: -108.917 Alpha: 1062.189 Cone: 0.591 Bx: 284.5534 By: 920.7310
DEBUG 2: Processing masking regions.
DEBUG 3: Processing grid mask: FULL
DEBUG 2: For TMP/L1 found 1 forecast levels, 0 climatology mean levels, and 0 climatology standard deviation levels.
DEBUG 2:
DEBUG 2: --------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Reading data for UGRD/L2.
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match for VarInfo "UGRD/L2" in GRIB record 17 of GRIB file "WRFPRS_d01.20180916.00".
DEBUG 3: MetGrib1DataFile::rotate_winds() -> Have U-wind record, reading V-wind record.
DEBUG 3: MetGrib1DataFile::data_plane_scalar() -> Found exact match for VarInfo "UGRD/L2" in GRIB record 18 of GRIB file "WRFPRS_d01.20180916.00".
DEBUG 3: Rotating U and V wind components from grid-relative to earth-relative.
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match for VarInfo "UGRD/L2" in GRIB record 363 of GRIB file "WRFPRS_d01.20180916.00".
DEBUG 3: MetGrib1DataFile::rotate_winds() -> Have U-wind record, reading V-wind record.
DEBUG 3: MetGrib1DataFile::data_plane_scalar() -> Found exact match for VarInfo "UGRD/L2" in GRIB record 18 of GRIB file "WRFPRS_d01.20180916.00".
DEBUG 3: Rotating U and V wind components from grid-relative to earth-relative.
DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 2 GRIB records matching VarInfo "UGRD/L2" in GRIB file "WRFPRS_d01.20180916.00".
DEBUG 2: For UGRD/L2 found 2 forecast levels, 0 climatology mean levels, and 0 climatology standard deviation levels.
DEBUG 2: Searching 12400 observations from 2480 messages.
DEBUG 2:
DEBUG 2: --------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Processing TMP/L1 versus TMP/L1, for observation type ADPSFC, over region FULL, for interpolation method NEAREST(1), using 0 pairs.
DEBUG 3: Number of matched pairs  = 0
DEBUG 3: Observations processed   = 12400
DEBUG 3: Rejected: SID exclusion  = 0
DEBUG 3: Rejected: obs type       = 12400
DEBUG 3: Rejected: valid time     = 0
DEBUG 3: Rejected: bad obs value  = 0
DEBUG 3: Rejected: off the grid   = 0
DEBUG 3: Rejected: level mismatch = 0
DEBUG 3: Rejected: quality marker = 0
DEBUG 3: Rejected: message type   = 0
DEBUG 3: Rejected: masking region = 0
DEBUG 3: Rejected: bad fcst value = 0
DEBUG 3: Rejected: duplicates     = 0


config:
////////////////////////////////////////////////////////////////////////////////
//
// Point-Stat configuration file.
//
// For additional information, see the MET_BASE/config/README file.
//
////////////////////////////////////////////////////////////////////////////////

//
// Output model name to be written
//
model = "WRF";

//
// Output description to be written
// May be set separately in each "obs.field" entry
//
desc = "NA";

////////////////////////////////////////////////////////////////////////////////

//
// Verification grid
// May be set separately in each "field" entry
//
 regrid = {
   to_grid    = NONE;
   method     = NEAREST;
   width      = 1;
   vld_thresh = 0.5;
   shape      = SQUARE;
 }

////////////////////////////////////////////////////////////////////////////////

//
// May be set separately in each "field" entry
//
censor_thresh  = [];
censor_val     = [];
cat_thresh     = [ NA ];
cnt_thresh     = [ NA ];
cnt_logic      = UNION;
wind_thresh    = [ NA ];
wind_logic     = UNION;
eclv_points    = 0.05;
rank_corr_flag = FALSE;

//
// Forecast and observation fields to be verified
//
fcst = {
   field = [

      {
        name       = "TMP";
        level      = [ "L001" ];
      },

      {
        name       = "UGRD";
        level      = [ "L002" ];
      },

      {
        name       = "VGRD";
        level      = [ "L002" ];
      }
   ];

}

obs = fcst;


////////////////////////////////////////////////////////////////////////////////

//
// Point observation filtering options
// May be set separately in each "obs.field" entry
//
message_type   = [ "ADPSFC" ];
sid_exc        = [];
obs_quality    = [];
duplicate_flag = NONE;
obs_summary    = NONE;
obs_perc_value = 50;

//
// Mapping of message type group name to comma-separated list of values.
//
message_type_group_map = [
   { key = "SURFACE"; val = "ADPSFC,SFCSHP,MSONET";               }
];

////////////////////////////////////////////////////////////////////////////////

//
// Climatology data
//
//climo_mean = {
//
//   file_name = [];
//   field     = [];
//
//   regrid = {
//      method     = NEAREST;
//      width      = 1;
//      vld_thresh = 0.5;
//      shape      = SQUARE;
//   }
//
//   time_interp_method = DW_MEAN;
//   match_day          = FALSE;
//   time_step          = 21600;
//}
//
//climo_stdev = climo_mean;
//climo_stdev = {
//   file_name = [];
//}
//
//climo_cdf_bins = 1;

////////////////////////////////////////////////////////////////////////////////

//
// Point observation time window
// May be set separately in each "obs.field" entry
//
obs_window = {
   beg = -5400;
   end =  5400;
}

////////////////////////////////////////////////////////////////////////////////

//
// Verification masking regions
// May be set separately in each "obs.field" entry
//
mask = {
   grid    = ["FULL"];
   poly    = [];
   sid     = [];
}

////////////////////////////////////////////////////////////////////////////////

//
// Confidence interval settings
// May be set separately in each "obs.field" entry
//
ci_alpha  = [ 0.05 ];

boot = {
   interval = PCTILE;
   rep_prop = 1.0;
   n_rep    = 0;
   rng      = "mt19937";
   seed     = "";
}

////////////////////////////////////////////////////////////////////////////////

//
// Interpolation methods
// May be set separately in each "obs.field" entry
//
interp = {
   vld_thresh = 1.0;
   shape      = SQUARE;

   type = [
      {
         method = NEAREST;
         width  = 1;
      }
   ];
}

////////////////////////////////////////////////////////////////////////////////

//
// HiRA verification method
// May be set separately in each "obs.field" entry
//
hira = {
   flag       = FALSE;
   width      = [ 2, 3, 4, 5 ];
   vld_thresh = 1.0;
   cov_thresh = [ ==0.25 ];
   shape      = SQUARE;
}

////////////////////////////////////////////////////////////////////////////////

//
// Statistical output types
// May be set separately in each "obs.field" entry
//
output_flag = {
   fho    = BOTH;
   ctc    = BOTH;
   cts    = BOTH;
   mctc   = BOTH;
   mcts   = BOTH;
   cnt    = BOTH;
   sl1l2  = BOTH;
   sal1l2 = BOTH;
   vl1l2  = BOTH;
   val1l2 = BOTH;
   vcnt   = BOTH;
   pct    = BOTH;
   pstd   = BOTH;
   pjc    = BOTH;
   prc    = BOTH;
   eclv   = BOTH;
   mpr    = BOTH;
}

////////////////////////////////////////////////////////////////////////////////

tmp_dir        = "./tmp";
output_prefix  = "";
version        = "V7.0";

////////////////////////////////////////////////////////////////////////////////


----------------------------------------------------------------
  Complete Ticket History
----------------------------------------------------------------

Subject: MET execution error while reading longitude from wrf data using point_stat exe
From: John Halley Gotway
Time: Thu Oct 18 09:40:49 2018

Hello,

I see that you have a question about why Point-Stat is reporting the
longitude of the lower-left corner being off by a factor of -1.

You expect to see "Lon_LL: 79.881" but are seeing the following
instead:
DEBUG 3: Grid Definition: Projection: Lambert Conformal Nx: 570 Ny:
390
Lat_LL: 9.384 Lon_LL: -79.881 Lon_orient: -108.917 Alpha: 1062.189
Cone:
0.591 Bx: 284.5534 By: 920.7310

This behavior is expected and does not indicate a problem.  The MET
grid
library code processes all longitudes in degrees west... instead of
degrees
east, which is the generally accepted convention.  When MET reads a
grid
spec from GRIB1, it multiplies longitudes by -1 to flip from degrees
east
to west.  And when it writes longitudes out, it multiples by -1 to
flip
them back.  But internally, and as shown in log messages, they're
stored as
degrees west.

I realize that's confusing and a headache.  We really should just
switch to
using the standard convention of degrees east throughout the code, but
we
haven't gotten around to changing that.

So it seems like the real issue is that you're getting 0 matched pairs
from
your Point-Stat run.  Running at verbosity level 3 (or higher) is
exactly
the right thing to do because it dumps out counts for reasons why obs
were
skipped, like this...

DEBUG 2: Processing TMP/L1 versus TMP/L1, for observation type ADPSFC,
over
region FULL, for interpolation method NEAREST(1), using 0 pairs.
DEBUG 3: Number of matched pairs  = 0
DEBUG 3: Observations processed   = 12400
DEBUG 3: Rejected: SID exclusion  = 0
DEBUG 3: Rejected: obs type       = 12400

These log messages are telling us that Point-Stat read through 12400
observations looking for observations named "TMP", but it found 0 of
them.
And that's why you got 0 matched pairs.  This should be pretty easy to
fix.  We just need to figure out how to request observations from your
point observation file: 20180916000000.nc

We've had some changes in how MET stores point observation info in the
last
couple years.  We're moving from the limiting convention of
referencing
GRIB code to just using simple strings for observation variable names.
The
formatting of the ascii2nc output file really depends on how you set
up
your ASCII observations.  Can you please post this sample file to our
anonymous ftp site (20180916000000.nc)?  I'll go grab it from there
and
take a closer look.

Once we get this issue resolved, it's possible that you'll still get 0
matched pairs for other reasons.  But we need to fix this one first.

You can follow these instructions for posting to our ftp site:
https://dtcenter.org/met/users/support/met_help.php#ftp

Thanks,
John Halley Gotway


On Thu, Oct 18, 2018 at 12:18 AM 徐 霈霈 via RT <met_help at ucar.edu>
wrote:

>
> Thu Oct 18 00:18:15 2018: Request 87432 was acted upon.
> Transaction: Ticket created by kevin.xpp at hotmail.com
>        Queue: met_help
>      Subject: MET execution error while reading longitude from wrf
data
> using point_stat exe
>        Owner: Nobody
>   Requestors: kevin.xpp at hotmail.com
>       Status: new
>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=87432 >
>
>
> Dear sir
>
>     I have a problem while executing point_stat exe with wrf data
> processed by upp software and observation data processed by ascii2nc
exe.
> Point_stat read the wrong longitude value from wrf data,the lon_ll
from the
> wrf data is 79.881,but the log shows -79.881。the log and
PointStatConfig is
> below,how to set the config to make point_stat read the right value
of wrf
> grib1 data ?
>
> log:
>
> DEBUG 1: Default Config File:
> /home/huayunshengda/Downloads/met-
7.0/share/met/config/PointStatConfig_default
> DEBUG 1: User Config File: PointStatConfig_default
> GSL_RNG_TYPE=mt19937
> GSL_RNG_SEED=488556635
> DEBUG 1: Forecast File: WRFPRS_d01.20180916.00
> DEBUG 1: Observation File: 20180916000000.nc
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Reading data for TMP/L1.
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match
for
> VarInfo "TMP/L1" in GRIB record 5 of GRIB file
"WRFPRS_d01.20180916.00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "TMP/L1" in GRIB file "WRFPRS_d01.20180916.00".
> DEBUG 3: Use the matching forecast and observation grids.
> DEBUG 3: Grid Definition: Projection: Lambert Conformal Nx: 570 Ny:
390
> Lat_LL: 9.384 Lon_LL: -79.881 Lon_orient: -108.917 Alpha: 1062.189
Cone:
> 0.591 Bx: 284.5534 By: 920.7310
> DEBUG 2: Processing masking regions.
> DEBUG 3: Processing grid mask: FULL
> DEBUG 2: For TMP/L1 found 1 forecast levels, 0 climatology mean
levels,
> and 0 climatology standard deviation levels.
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Reading data for UGRD/L2.
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match
for
> VarInfo "UGRD/L2" in GRIB record 17 of GRIB file
"WRFPRS_d01.20180916.00".
> DEBUG 3: MetGrib1DataFile::rotate_winds() -> Have U-wind record,
reading
> V-wind record.
> DEBUG 3: MetGrib1DataFile::data_plane_scalar() -> Found exact match
for
> VarInfo "UGRD/L2" in GRIB record 18 of GRIB file
"WRFPRS_d01.20180916.00".
> DEBUG 3: Rotating U and V wind components from grid-relative to
> earth-relative.
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
> VarInfo "UGRD/L2" in GRIB record 363 of GRIB file
"WRFPRS_d01.20180916.00".
> DEBUG 3: MetGrib1DataFile::rotate_winds() -> Have U-wind record,
reading
> V-wind record.
> DEBUG 3: MetGrib1DataFile::data_plane_scalar() -> Found exact match
for
> VarInfo "UGRD/L2" in GRIB record 18 of GRIB file
"WRFPRS_d01.20180916.00".
> DEBUG 3: Rotating U and V wind components from grid-relative to
> earth-relative.
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 2 GRIB
records
> matching VarInfo "UGRD/L2" in GRIB file "WRFPRS_d01.20180916.00".
> DEBUG 2: For UGRD/L2 found 2 forecast levels, 0 climatology mean
levels,
> and 0 climatology standard deviation levels.
> DEBUG 2: Searching 12400 observations from 2480 messages.
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Processing TMP/L1 versus TMP/L1, for observation type
ADPSFC,
> over region FULL, for interpolation method NEAREST(1), using 0
pairs.
> DEBUG 3: Number of matched pairs  = 0
> DEBUG 3: Observations processed   = 12400
> DEBUG 3: Rejected: SID exclusion  = 0
> DEBUG 3: Rejected: obs type       = 12400
> DEBUG 3: Rejected: valid time     = 0
> DEBUG 3: Rejected: bad obs value  = 0
> DEBUG 3: Rejected: off the grid   = 0
> DEBUG 3: Rejected: level mismatch = 0
> DEBUG 3: Rejected: quality marker = 0
> DEBUG 3: Rejected: message type   = 0
> DEBUG 3: Rejected: masking region = 0
> DEBUG 3: Rejected: bad fcst value = 0
> DEBUG 3: Rejected: duplicates     = 0
>
>
> config:
>
>
////////////////////////////////////////////////////////////////////////////////
> //
> // Point-Stat configuration file.
> //
> // For additional information, see the MET_BASE/config/README file.
> //
>
>
////////////////////////////////////////////////////////////////////////////////
>
> //
> // Output model name to be written
> //
> model = "WRF";
>
> //
> // Output description to be written
> // May be set separately in each "obs.field" entry
> //
> desc = "NA";
>
>
>
////////////////////////////////////////////////////////////////////////////////
>
> //
> // Verification grid
> // May be set separately in each "field" entry
> //
>  regrid = {
>    to_grid    = NONE;
>    method     = NEAREST;
>    width      = 1;
>    vld_thresh = 0.5;
>    shape      = SQUARE;
>  }
>
>
>
////////////////////////////////////////////////////////////////////////////////
>
> //
> // May be set separately in each "field" entry
> //
> censor_thresh  = [];
> censor_val     = [];
> cat_thresh     = [ NA ];
> cnt_thresh     = [ NA ];
> cnt_logic      = UNION;
> wind_thresh    = [ NA ];
> wind_logic     = UNION;
> eclv_points    = 0.05;
> rank_corr_flag = FALSE;
>
> //
> // Forecast and observation fields to be verified
> //
> fcst = {
>    field = [
>
>       {
>         name       = "TMP";
>         level      = [ "L001" ];
>       },
>
>       {
>         name       = "UGRD";
>         level      = [ "L002" ];
>       },
>
>       {
>         name       = "VGRD";
>         level      = [ "L002" ];
>       }
>    ];
>
> }
>
> obs = fcst;
>
>
>
>
////////////////////////////////////////////////////////////////////////////////
>
> //
> // Point observation filtering options
> // May be set separately in each "obs.field" entry
> //
> message_type   = [ "ADPSFC" ];
> sid_exc        = [];
> obs_quality    = [];
> duplicate_flag = NONE;
> obs_summary    = NONE;
> obs_perc_value = 50;
>
> //
> // Mapping of message type group name to comma-separated list of
values.
> //
> message_type_group_map = [
>    { key = "SURFACE"; val = "ADPSFC,SFCSHP,MSONET";               }
> ];
>
>
>
////////////////////////////////////////////////////////////////////////////////
>
> //
> // Climatology data
> //
> //climo_mean = {
> //
> //   file_name = [];
> //   field     = [];
> //
> //   regrid = {
> //      method     = NEAREST;
> //      width      = 1;
> //      vld_thresh = 0.5;
> //      shape      = SQUARE;
> //   }
> //
> //   time_interp_method = DW_MEAN;
> //   match_day          = FALSE;
> //   time_step          = 21600;
> //}
> //
> //climo_stdev = climo_mean;
> //climo_stdev = {
> //   file_name = [];
> //}
> //
> //climo_cdf_bins = 1;
>
>
>
////////////////////////////////////////////////////////////////////////////////
>
> //
> // Point observation time window
> // May be set separately in each "obs.field" entry
> //
> obs_window = {
>    beg = -5400;
>    end =  5400;
> }
>
>
>
////////////////////////////////////////////////////////////////////////////////
>
> //
> // Verification masking regions
> // May be set separately in each "obs.field" entry
> //
> mask = {
>    grid    = ["FULL"];
>    poly    = [];
>    sid     = [];
> }
>
>
>
////////////////////////////////////////////////////////////////////////////////
>
> //
> // Confidence interval settings
> // May be set separately in each "obs.field" entry
> //
> ci_alpha  = [ 0.05 ];
>
> boot = {
>    interval = PCTILE;
>    rep_prop = 1.0;
>    n_rep    = 0;
>    rng      = "mt19937";
>    seed     = "";
> }
>
>
>
////////////////////////////////////////////////////////////////////////////////
>
> //
> // Interpolation methods
> // May be set separately in each "obs.field" entry
> //
> interp = {
>    vld_thresh = 1.0;
>    shape      = SQUARE;
>
>    type = [
>       {
>          method = NEAREST;
>          width  = 1;
>       }
>    ];
> }
>
>
>
////////////////////////////////////////////////////////////////////////////////
>
> //
> // HiRA verification method
> // May be set separately in each "obs.field" entry
> //
> hira = {
>    flag       = FALSE;
>    width      = [ 2, 3, 4, 5 ];
>    vld_thresh = 1.0;
>    cov_thresh = [ ==0.25 ];
>    shape      = SQUARE;
> }
>
>
>
////////////////////////////////////////////////////////////////////////////////
>
> //
> // Statistical output types
> // May be set separately in each "obs.field" entry
> //
> output_flag = {
>    fho    = BOTH;
>    ctc    = BOTH;
>    cts    = BOTH;
>    mctc   = BOTH;
>    mcts   = BOTH;
>    cnt    = BOTH;
>    sl1l2  = BOTH;
>    sal1l2 = BOTH;
>    vl1l2  = BOTH;
>    val1l2 = BOTH;
>    vcnt   = BOTH;
>    pct    = BOTH;
>    pstd   = BOTH;
>    pjc    = BOTH;
>    prc    = BOTH;
>    eclv   = BOTH;
>    mpr    = BOTH;
> }
>
>
>
////////////////////////////////////////////////////////////////////////////////
>
> tmp_dir        = "./tmp";
> output_prefix  = "";
> version        = "V7.0";
>
>
>
////////////////////////////////////////////////////////////////////////////////
>
>

------------------------------------------------
Subject: MET execution error while reading longitude from wrf data using point_stat exe
From: 徐 霈霈
Time: Thu Oct 18 19:51:44 2018

Dear sir

    Thank you for your reply.
    You are right, I found the Point-Stat read the longititude of the
lower-left corner being off by a factor of -1 too , using other wrf
grib1 data ,such as the sample data in met-7.0/data/sample_fcst . I
have upload the data to http://ftp.rap.ucar.edu
in incoming/irap/met_help/peipei_data directory including wrf grib1
data ,ascii data and the PointStatConfig_default.
   I have also changed  the longitude of ascii data to make the
observations fall into the region point_stat read,but there was still
no match pairs.I don't know why.   Please help me check the data and
the config .
   I am looking forward to your reply.
   With best regards.

Peipei Xu
________________________________
发件人: John Halley Gotway via RT <met_help at ucar.edu>
发送时间: 2018年10月18日 15:40
收件人: kevin.xpp at hotmail.com
抄送: met_help at mailman.ucar.edu
主题: Re: [rt.rap.ucar.edu #87432] MET execution error while reading
longitude from wrf data using point_stat exe

Hello,

I see that you have a question about why Point-Stat is reporting the
longitude of the lower-left corner being off by a factor of -1.

You expect to see "Lon_LL: 79.881" but are seeing the following
instead:
DEBUG 3: Grid Definition: Projection: Lambert Conformal Nx: 570 Ny:
390
Lat_LL: 9.384 Lon_LL: -79.881 Lon_orient: -108.917 Alpha: 1062.189
Cone:
0.591 Bx: 284.5534 By: 920.7310

This behavior is expected and does not indicate a problem.  The MET
grid
library code processes all longitudes in degrees west... instead of
degrees
east, which is the generally accepted convention.  When MET reads a
grid
spec from GRIB1, it multiplies longitudes by -1 to flip from degrees
east
to west.  And when it writes longitudes out, it multiples by -1 to
flip
them back.  But internally, and as shown in log messages, they're
stored as
degrees west.

I realize that's confusing and a headache.  We really should just
switch to
using the standard convention of degrees east throughout the code, but
we
haven't gotten around to changing that.

So it seems like the real issue is that you're getting 0 matched pairs
from
your Point-Stat run.  Running at verbosity level 3 (or higher) is
exactly
the right thing to do because it dumps out counts for reasons why obs
were
skipped, like this...

DEBUG 2: Processing TMP/L1 versus TMP/L1, for observation type ADPSFC,
over
region FULL, for interpolation method NEAREST(1), using 0 pairs.
DEBUG 3: Number of matched pairs  = 0
DEBUG 3: Observations processed   = 12400
DEBUG 3: Rejected: SID exclusion  = 0
DEBUG 3: Rejected: obs type       = 12400

These log messages are telling us that Point-Stat read through 12400
observations looking for observations named "TMP", but it found 0 of
them.
And that's why you got 0 matched pairs.  This should be pretty easy to
fix.  We just need to figure out how to request observations from your
point observation file: 20180916000000.nc

We've had some changes in how MET stores point observation info in the
last
couple years.  We're moving from the limiting convention of
referencing
GRIB code to just using simple strings for observation variable names.
The
formatting of the ascii2nc output file really depends on how you set
up
your ASCII observations.  Can you please post this sample file to our
anonymous ftp site (20180916000000.nc)?  I'll go grab it from there
and
take a closer look.

Once we get this issue resolved, it's possible that you'll still get 0
matched pairs for other reasons.  But we need to fix this one first.

You can follow these instructions for posting to our ftp site:
https://dtcenter.org/met/users/support/met_help.php#ftp

Thanks,
John Halley Gotway


On Thu, Oct 18, 2018 at 12:18 AM 徐 霈霈 via RT <met_help at ucar.edu>
wrote:

>
> Thu Oct 18 00:18:15 2018: Request 87432 was acted upon.
> Transaction: Ticket created by kevin.xpp at hotmail.com
>        Queue: met_help
>      Subject: MET execution error while reading longitude from wrf
data
> using point_stat exe
>        Owner: Nobody
>   Requestors: kevin.xpp at hotmail.com
>       Status: new
>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=87432 >
>
>
> Dear sir
>
>     I have a problem while executing point_stat exe with wrf data
> processed by upp software and observation data processed by ascii2nc
exe.
> Point_stat read the wrong longitude value from wrf data,the lon_ll
from the
> wrf data is 79.881,but the log shows -79.881。the log and
PointStatConfig is
> below,how to set the config to make point_stat read the right value
of wrf
> grib1 data ?
>
> log:
>
> DEBUG 1: Default Config File:
> /home/huayunshengda/Downloads/met-
7.0/share/met/config/PointStatConfig_default
> DEBUG 1: User Config File: PointStatConfig_default
> GSL_RNG_TYPE=mt19937
> GSL_RNG_SEED=488556635
> DEBUG 1: Forecast File: WRFPRS_d01.20180916.00
> DEBUG 1: Observation File: 20180916000000.nc
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Reading data for TMP/L1.
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match
for
> VarInfo "TMP/L1" in GRIB record 5 of GRIB file
"WRFPRS_d01.20180916.00".
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> matching VarInfo "TMP/L1" in GRIB file "WRFPRS_d01.20180916.00".
> DEBUG 3: Use the matching forecast and observation grids.
> DEBUG 3: Grid Definition: Projection: Lambert Conformal Nx: 570 Ny:
390
> Lat_LL: 9.384 Lon_LL: -79.881 Lon_orient: -108.917 Alpha: 1062.189
Cone:
> 0.591 Bx: 284.5534 By: 920.7310
> DEBUG 2: Processing masking regions.
> DEBUG 3: Processing grid mask: FULL
> DEBUG 2: For TMP/L1 found 1 forecast levels, 0 climatology mean
levels,
> and 0 climatology standard deviation levels.
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Reading data for UGRD/L2.
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match
for
> VarInfo "UGRD/L2" in GRIB record 17 of GRIB file
"WRFPRS_d01.20180916.00".
> DEBUG 3: MetGrib1DataFile::rotate_winds() -> Have U-wind record,
reading
> V-wind record.
> DEBUG 3: MetGrib1DataFile::data_plane_scalar() -> Found exact match
for
> VarInfo "UGRD/L2" in GRIB record 18 of GRIB file
"WRFPRS_d01.20180916.00".
> DEBUG 3: Rotating U and V wind components from grid-relative to
> earth-relative.
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
> VarInfo "UGRD/L2" in GRIB record 363 of GRIB file
"WRFPRS_d01.20180916.00".
> DEBUG 3: MetGrib1DataFile::rotate_winds() -> Have U-wind record,
reading
> V-wind record.
> DEBUG 3: MetGrib1DataFile::data_plane_scalar() -> Found exact match
for
> VarInfo "UGRD/L2" in GRIB record 18 of GRIB file
"WRFPRS_d01.20180916.00".
> DEBUG 3: Rotating U and V wind components from grid-relative to
> earth-relative.
> DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 2 GRIB
records
> matching VarInfo "UGRD/L2" in GRIB file "WRFPRS_d01.20180916.00".
> DEBUG 2: For UGRD/L2 found 2 forecast levels, 0 climatology mean
levels,
> and 0 climatology standard deviation levels.
> DEBUG 2: Searching 12400 observations from 2480 messages.
> DEBUG 2:
> DEBUG 2:
>
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Processing TMP/L1 versus TMP/L1, for observation type
ADPSFC,
> over region FULL, for interpolation method NEAREST(1), using 0
pairs.
> DEBUG 3: Number of matched pairs  = 0
> DEBUG 3: Observations processed   = 12400
> DEBUG 3: Rejected: SID exclusion  = 0
> DEBUG 3: Rejected: obs type       = 12400
> DEBUG 3: Rejected: valid time     = 0
> DEBUG 3: Rejected: bad obs value  = 0
> DEBUG 3: Rejected: off the grid   = 0
> DEBUG 3: Rejected: level mismatch = 0
> DEBUG 3: Rejected: quality marker = 0
> DEBUG 3: Rejected: message type   = 0
> DEBUG 3: Rejected: masking region = 0
> DEBUG 3: Rejected: bad fcst value = 0
> DEBUG 3: Rejected: duplicates     = 0
>
>
> config:
>
>
////////////////////////////////////////////////////////////////////////////////
> //
> // Point-Stat configuration file.
> //
> // For additional information, see the MET_BASE/config/README file.
> //
>
>
////////////////////////////////////////////////////////////////////////////////
>
> //
> // Output model name to be written
> //
> model = "WRF";
>
> //
> // Output description to be written
> // May be set separately in each "obs.field" entry
> //
> desc = "NA";
>
>
>
////////////////////////////////////////////////////////////////////////////////
>
> //
> // Verification grid
> // May be set separately in each "field" entry
> //
>  regrid = {
>    to_grid    = NONE;
>    method     = NEAREST;
>    width      = 1;
>    vld_thresh = 0.5;
>    shape      = SQUARE;
>  }
>
>
>
////////////////////////////////////////////////////////////////////////////////
>
> //
> // May be set separately in each "field" entry
> //
> censor_thresh  = [];
> censor_val     = [];
> cat_thresh     = [ NA ];
> cnt_thresh     = [ NA ];
> cnt_logic      = UNION;
> wind_thresh    = [ NA ];
> wind_logic     = UNION;
> eclv_points    = 0.05;
> rank_corr_flag = FALSE;
>
> //
> // Forecast and observation fields to be verified
> //
> fcst = {
>    field = [
>
>       {
>         name       = "TMP";
>         level      = [ "L001" ];
>       },
>
>       {
>         name       = "UGRD";
>         level      = [ "L002" ];
>       },
>
>       {
>         name       = "VGRD";
>         level      = [ "L002" ];
>       }
>    ];
>
> }
>
> obs = fcst;
>
>
>
>
////////////////////////////////////////////////////////////////////////////////
>
> //
> // Point observation filtering options
> // May be set separately in each "obs.field" entry
> //
> message_type   = [ "ADPSFC" ];
> sid_exc        = [];
> obs_quality    = [];
> duplicate_flag = NONE;
> obs_summary    = NONE;
> obs_perc_value = 50;
>
> //
> // Mapping of message type group name to comma-separated list of
values.
> //
> message_type_group_map = [
>    { key = "SURFACE"; val = "ADPSFC,SFCSHP,MSONET";               }
> ];
>
>
>
////////////////////////////////////////////////////////////////////////////////
>
> //
> // Climatology data
> //
> //climo_mean = {
> //
> //   file_name = [];
> //   field     = [];
> //
> //   regrid = {
> //      method     = NEAREST;
> //      width      = 1;
> //      vld_thresh = 0.5;
> //      shape      = SQUARE;
> //   }
> //
> //   time_interp_method = DW_MEAN;
> //   match_day          = FALSE;
> //   time_step          = 21600;
> //}
> //
> //climo_stdev = climo_mean;
> //climo_stdev = {
> //   file_name = [];
> //}
> //
> //climo_cdf_bins = 1;
>
>
>
////////////////////////////////////////////////////////////////////////////////
>
> //
> // Point observation time window
> // May be set separately in each "obs.field" entry
> //
> obs_window = {
>    beg = -5400;
>    end =  5400;
> }
>
>
>
////////////////////////////////////////////////////////////////////////////////
>
> //
> // Verification masking regions
> // May be set separately in each "obs.field" entry
> //
> mask = {
>    grid    = ["FULL"];
>    poly    = [];
>    sid     = [];
> }
>
>
>
////////////////////////////////////////////////////////////////////////////////
>
> //
> // Confidence interval settings
> // May be set separately in each "obs.field" entry
> //
> ci_alpha  = [ 0.05 ];
>
> boot = {
>    interval = PCTILE;
>    rep_prop = 1.0;
>    n_rep    = 0;
>    rng      = "mt19937";
>    seed     = "";
> }
>
>
>
////////////////////////////////////////////////////////////////////////////////
>
> //
> // Interpolation methods
> // May be set separately in each "obs.field" entry
> //
> interp = {
>    vld_thresh = 1.0;
>    shape      = SQUARE;
>
>    type = [
>       {
>          method = NEAREST;
>          width  = 1;
>       }
>    ];
> }
>
>
>
////////////////////////////////////////////////////////////////////////////////
>
> //
> // HiRA verification method
> // May be set separately in each "obs.field" entry
> //
> hira = {
>    flag       = FALSE;
>    width      = [ 2, 3, 4, 5 ];
>    vld_thresh = 1.0;
>    cov_thresh = [ ==0.25 ];
>    shape      = SQUARE;
> }
>
>
>
////////////////////////////////////////////////////////////////////////////////
>
> //
> // Statistical output types
> // May be set separately in each "obs.field" entry
> //
> output_flag = {
>    fho    = BOTH;
>    ctc    = BOTH;
>    cts    = BOTH;
>    mctc   = BOTH;
>    mcts   = BOTH;
>    cnt    = BOTH;
>    sl1l2  = BOTH;
>    sal1l2 = BOTH;
>    vl1l2  = BOTH;
>    val1l2 = BOTH;
>    vcnt   = BOTH;
>    pct    = BOTH;
>    pstd   = BOTH;
>    pjc    = BOTH;
>    prc    = BOTH;
>    eclv   = BOTH;
>    mpr    = BOTH;
> }
>
>
>
////////////////////////////////////////////////////////////////////////////////
>
> tmp_dir        = "./tmp";
> output_prefix  = "";
> version        = "V7.0";
>
>
>
////////////////////////////////////////////////////////////////////////////////
>
>


------------------------------------------------
Subject: MET execution error while reading longitude from wrf data using point_stat exe
From: John Halley Gotway
Time: Fri Oct 19 12:00:35 2018

Peipei,

Thanks for sending your sample data.  I grabbed it and was able to
replicate the behavior you describe.

I found the culprit but can't fully explain why it's causing the
behavior.
45 of the 12400 lines in you ascii point obs file contain bad data:
   grep "999999.0" 20180916000000.txt | wc -l
   45

If you exclude those bad lines, then you do get matched pairs:

egrep -v "999999.0" 20180916000000.txt > 20180916000000_NEW.txt
ascii2nc 20180916000000_NEW.txt 20180916000000_NEW.nc
point_stat WRFPRS_d01.20180916.00 20180916000000_NEW.nc
PointStatConfig_default -v 3
...
DEBUG 2: Processing TMP/L1 versus TMP/L1, for observation type ADPSFC,
over
region FULL, for interpolation method NEAREST(1), using 2471 pairs.

On other detail...

When you run Point-Stat, you should set the "level" to request the
GRIB
record you want to use.  You're requesting levels "L001" for TMP and
"L002"
for UGRD and VGRD.  At verbosity level 3, that tell you that it's
using
GRIB records 5, 17, and 18:


DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match for
VarInfo "TMP/L1" in GRIB record 5 of GRIB file
"WRFPRS_d01.20180916.00".

5:1167450:d=18091600:TMP:kpds5=11:kpds6=109:kpds7=1:TR=0:P1=0:P2=0:TimeU=1:hybrid
lev 1:anl:NAve=00
17:5086614:d=18091600:UGRD:kpds5=33:kpds6=109:kpds7=2:TR=0:P1=0:P2=0:TimeU=1:hybrid
lev 2:anl:NAve=0
18:5447946:d=18091600:VGRD:kpds5=34:kpds6=109:kpds7=2:TR=0:P1=0:P2=0:TimeU=1:hybrid
lev 2:anl:NAve=0

So this is getting these "hybrid levels".  Is that what you want or do
you
want 2-m temperature and 10-meter winds?

For those, you'd use:
   field = [
      { name = "TMP";    level = [ "Z2" ];   },
      { name = "UGRD"; level = [ "Z10" ]; },
      { name = "VGRD"; level = [ "Z10" ]; }
   ];

Thanks,
John

On Thu, Oct 18, 2018 at 7:52 PM 徐 霈霈 via RT <met_help at ucar.edu> wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=87432 >
>
> Dear sir
>
>     Thank you for your reply.
>     You are right, I found the Point-Stat read the longititude of
the
> lower-left corner being off by a factor of -1 too , using other wrf
grib1
> data ,such as the sample data in met-7.0/data/sample_fcst . I have
upload
> the data to http://ftp.rap.ucar.edu
> in incoming/irap/met_help/peipei_data directory including wrf grib1
data
> ,ascii data and the PointStatConfig_default.
>    I have also changed  the longitude of ascii data to make the
> observations fall into the region point_stat read,but there was
still no
> match pairs.I don't know why.   Please help me check the data and
the
> config .
>    I am looking forward to your reply.
>    With best regards.
>
> Peipei Xu
> ________________________________
> 发件人: John Halley Gotway via RT <met_help at ucar.edu>
> 发送时间: 2018年10月18日 15:40
> 收件人: kevin.xpp at hotmail.com
> 抄送: met_help at mailman.ucar.edu
> 主题: Re: [rt.rap.ucar.edu #87432] MET execution error while reading
> longitude from wrf data using point_stat exe
>
> Hello,
>
> I see that you have a question about why Point-Stat is reporting the
> longitude of the lower-left corner being off by a factor of -1.
>
> You expect to see "Lon_LL: 79.881" but are seeing the following
instead:
> DEBUG 3: Grid Definition: Projection: Lambert Conformal Nx: 570 Ny:
390
> Lat_LL: 9.384 Lon_LL: -79.881 Lon_orient: -108.917 Alpha: 1062.189
Cone:
> 0.591 Bx: 284.5534 By: 920.7310
>
> This behavior is expected and does not indicate a problem.  The MET
grid
> library code processes all longitudes in degrees west... instead of
degrees
> east, which is the generally accepted convention.  When MET reads a
grid
> spec from GRIB1, it multiplies longitudes by -1 to flip from degrees
east
> to west.  And when it writes longitudes out, it multiples by -1 to
flip
> them back.  But internally, and as shown in log messages, they're
stored as
> degrees west.
>
> I realize that's confusing and a headache.  We really should just
switch to
> using the standard convention of degrees east throughout the code,
but we
> haven't gotten around to changing that.
>
> So it seems like the real issue is that you're getting 0 matched
pairs from
> your Point-Stat run.  Running at verbosity level 3 (or higher) is
exactly
> the right thing to do because it dumps out counts for reasons why
obs were
> skipped, like this...
>
> DEBUG 2: Processing TMP/L1 versus TMP/L1, for observation type
ADPSFC, over
> region FULL, for interpolation method NEAREST(1), using 0 pairs.
> DEBUG 3: Number of matched pairs  = 0
> DEBUG 3: Observations processed   = 12400
> DEBUG 3: Rejected: SID exclusion  = 0
> DEBUG 3: Rejected: obs type       = 12400
>
> These log messages are telling us that Point-Stat read through 12400
> observations looking for observations named "TMP", but it found 0 of
them.
> And that's why you got 0 matched pairs.  This should be pretty easy
to
> fix.  We just need to figure out how to request observations from
your
> point observation file: 20180916000000.nc
>
> We've had some changes in how MET stores point observation info in
the last
> couple years.  We're moving from the limiting convention of
referencing
> GRIB code to just using simple strings for observation variable
names.  The
> formatting of the ascii2nc output file really depends on how you set
up
> your ASCII observations.  Can you please post this sample file to
our
> anonymous ftp site (20180916000000.nc)?  I'll go grab it from there
and
> take a closer look.
>
> Once we get this issue resolved, it's possible that you'll still get
0
> matched pairs for other reasons.  But we need to fix this one first.
>
> You can follow these instructions for posting to our ftp site:
> https://dtcenter.org/met/users/support/met_help.php#ftp
>
> Thanks,
> John Halley Gotway
>
>
> On Thu, Oct 18, 2018 at 12:18 AM 徐 霈霈 via RT <met_help at ucar.edu>
wrote:
>
> >
> > Thu Oct 18 00:18:15 2018: Request 87432 was acted upon.
> > Transaction: Ticket created by kevin.xpp at hotmail.com
> >        Queue: met_help
> >      Subject: MET execution error while reading longitude from wrf
data
> > using point_stat exe
> >        Owner: Nobody
> >   Requestors: kevin.xpp at hotmail.com
> >       Status: new
> >  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=87432 >
> >
> >
> > Dear sir
> >
> >     I have a problem while executing point_stat exe with wrf data
> > processed by upp software and observation data processed by
ascii2nc exe.
> > Point_stat read the wrong longitude value from wrf data,the lon_ll
from
> the
> > wrf data is 79.881,but the log shows -79.881。the log and
PointStatConfig
> is
> > below,how to set the config to make point_stat read the right
value of
> wrf
> > grib1 data ?
> >
> > log:
> >
> > DEBUG 1: Default Config File:
> >
> /home/huayunshengda/Downloads/met-
7.0/share/met/config/PointStatConfig_default
> > DEBUG 1: User Config File: PointStatConfig_default
> > GSL_RNG_TYPE=mt19937
> > GSL_RNG_SEED=488556635
> > DEBUG 1: Forecast File: WRFPRS_d01.20180916.00
> > DEBUG 1: Observation File: 20180916000000.nc
> > DEBUG 2:
> > DEBUG 2:
> >
>
--------------------------------------------------------------------------------
> > DEBUG 2:
> > DEBUG 2: Reading data for TMP/L1.
> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match
for
> > VarInfo "TMP/L1" in GRIB record 5 of GRIB file
"WRFPRS_d01.20180916.00".
> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> > matching VarInfo "TMP/L1" in GRIB file "WRFPRS_d01.20180916.00".
> > DEBUG 3: Use the matching forecast and observation grids.
> > DEBUG 3: Grid Definition: Projection: Lambert Conformal Nx: 570
Ny: 390
> > Lat_LL: 9.384 Lon_LL: -79.881 Lon_orient: -108.917 Alpha: 1062.189
Cone:
> > 0.591 Bx: 284.5534 By: 920.7310
> > DEBUG 2: Processing masking regions.
> > DEBUG 3: Processing grid mask: FULL
> > DEBUG 2: For TMP/L1 found 1 forecast levels, 0 climatology mean
levels,
> > and 0 climatology standard deviation levels.
> > DEBUG 2:
> > DEBUG 2:
> >
>
--------------------------------------------------------------------------------
> > DEBUG 2:
> > DEBUG 2: Reading data for UGRD/L2.
> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match
for
> > VarInfo "UGRD/L2" in GRIB record 17 of GRIB file
> "WRFPRS_d01.20180916.00".
> > DEBUG 3: MetGrib1DataFile::rotate_winds() -> Have U-wind record,
reading
> > V-wind record.
> > DEBUG 3: MetGrib1DataFile::data_plane_scalar() -> Found exact
match for
> > VarInfo "UGRD/L2" in GRIB record 18 of GRIB file
> "WRFPRS_d01.20180916.00".
> > DEBUG 3: Rotating U and V wind components from grid-relative to
> > earth-relative.
> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
> > VarInfo "UGRD/L2" in GRIB record 363 of GRIB file
> "WRFPRS_d01.20180916.00".
> > DEBUG 3: MetGrib1DataFile::rotate_winds() -> Have U-wind record,
reading
> > V-wind record.
> > DEBUG 3: MetGrib1DataFile::data_plane_scalar() -> Found exact
match for
> > VarInfo "UGRD/L2" in GRIB record 18 of GRIB file
> "WRFPRS_d01.20180916.00".
> > DEBUG 3: Rotating U and V wind components from grid-relative to
> > earth-relative.
> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 2 GRIB
records
> > matching VarInfo "UGRD/L2" in GRIB file "WRFPRS_d01.20180916.00".
> > DEBUG 2: For UGRD/L2 found 2 forecast levels, 0 climatology mean
levels,
> > and 0 climatology standard deviation levels.
> > DEBUG 2: Searching 12400 observations from 2480 messages.
> > DEBUG 2:
> > DEBUG 2:
> >
>
--------------------------------------------------------------------------------
> > DEBUG 2:
> > DEBUG 2: Processing TMP/L1 versus TMP/L1, for observation type
ADPSFC,
> > over region FULL, for interpolation method NEAREST(1), using 0
pairs.
> > DEBUG 3: Number of matched pairs  = 0
> > DEBUG 3: Observations processed   = 12400
> > DEBUG 3: Rejected: SID exclusion  = 0
> > DEBUG 3: Rejected: obs type       = 12400
> > DEBUG 3: Rejected: valid time     = 0
> > DEBUG 3: Rejected: bad obs value  = 0
> > DEBUG 3: Rejected: off the grid   = 0
> > DEBUG 3: Rejected: level mismatch = 0
> > DEBUG 3: Rejected: quality marker = 0
> > DEBUG 3: Rejected: message type   = 0
> > DEBUG 3: Rejected: masking region = 0
> > DEBUG 3: Rejected: bad fcst value = 0
> > DEBUG 3: Rejected: duplicates     = 0
> >
> >
> > config:
> >
> >
>
////////////////////////////////////////////////////////////////////////////////
> > //
> > // Point-Stat configuration file.
> > //
> > // For additional information, see the MET_BASE/config/README
file.
> > //
> >
> >
>
////////////////////////////////////////////////////////////////////////////////
> >
> > //
> > // Output model name to be written
> > //
> > model = "WRF";
> >
> > //
> > // Output description to be written
> > // May be set separately in each "obs.field" entry
> > //
> > desc = "NA";
> >
> >
> >
>
////////////////////////////////////////////////////////////////////////////////
> >
> > //
> > // Verification grid
> > // May be set separately in each "field" entry
> > //
> >  regrid = {
> >    to_grid    = NONE;
> >    method     = NEAREST;
> >    width      = 1;
> >    vld_thresh = 0.5;
> >    shape      = SQUARE;
> >  }
> >
> >
> >
>
////////////////////////////////////////////////////////////////////////////////
> >
> > //
> > // May be set separately in each "field" entry
> > //
> > censor_thresh  = [];
> > censor_val     = [];
> > cat_thresh     = [ NA ];
> > cnt_thresh     = [ NA ];
> > cnt_logic      = UNION;
> > wind_thresh    = [ NA ];
> > wind_logic     = UNION;
> > eclv_points    = 0.05;
> > rank_corr_flag = FALSE;
> >
> > //
> > // Forecast and observation fields to be verified
> > //
> > fcst = {
> >    field = [
> >
> >       {
> >         name       = "TMP";
> >         level      = [ "L001" ];
> >       },
> >
> >       {
> >         name       = "UGRD";
> >         level      = [ "L002" ];
> >       },
> >
> >       {
> >         name       = "VGRD";
> >         level      = [ "L002" ];
> >       }
> >    ];
> >
> > }
> >
> > obs = fcst;
> >
> >
> >
> >
>
////////////////////////////////////////////////////////////////////////////////
> >
> > //
> > // Point observation filtering options
> > // May be set separately in each "obs.field" entry
> > //
> > message_type   = [ "ADPSFC" ];
> > sid_exc        = [];
> > obs_quality    = [];
> > duplicate_flag = NONE;
> > obs_summary    = NONE;
> > obs_perc_value = 50;
> >
> > //
> > // Mapping of message type group name to comma-separated list of
values.
> > //
> > message_type_group_map = [
> >    { key = "SURFACE"; val = "ADPSFC,SFCSHP,MSONET";
}
> > ];
> >
> >
> >
>
////////////////////////////////////////////////////////////////////////////////
> >
> > //
> > // Climatology data
> > //
> > //climo_mean = {
> > //
> > //   file_name = [];
> > //   field     = [];
> > //
> > //   regrid = {
> > //      method     = NEAREST;
> > //      width      = 1;
> > //      vld_thresh = 0.5;
> > //      shape      = SQUARE;
> > //   }
> > //
> > //   time_interp_method = DW_MEAN;
> > //   match_day          = FALSE;
> > //   time_step          = 21600;
> > //}
> > //
> > //climo_stdev = climo_mean;
> > //climo_stdev = {
> > //   file_name = [];
> > //}
> > //
> > //climo_cdf_bins = 1;
> >
> >
> >
>
////////////////////////////////////////////////////////////////////////////////
> >
> > //
> > // Point observation time window
> > // May be set separately in each "obs.field" entry
> > //
> > obs_window = {
> >    beg = -5400;
> >    end =  5400;
> > }
> >
> >
> >
>
////////////////////////////////////////////////////////////////////////////////
> >
> > //
> > // Verification masking regions
> > // May be set separately in each "obs.field" entry
> > //
> > mask = {
> >    grid    = ["FULL"];
> >    poly    = [];
> >    sid     = [];
> > }
> >
> >
> >
>
////////////////////////////////////////////////////////////////////////////////
> >
> > //
> > // Confidence interval settings
> > // May be set separately in each "obs.field" entry
> > //
> > ci_alpha  = [ 0.05 ];
> >
> > boot = {
> >    interval = PCTILE;
> >    rep_prop = 1.0;
> >    n_rep    = 0;
> >    rng      = "mt19937";
> >    seed     = "";
> > }
> >
> >
> >
>
////////////////////////////////////////////////////////////////////////////////
> >
> > //
> > // Interpolation methods
> > // May be set separately in each "obs.field" entry
> > //
> > interp = {
> >    vld_thresh = 1.0;
> >    shape      = SQUARE;
> >
> >    type = [
> >       {
> >          method = NEAREST;
> >          width  = 1;
> >       }
> >    ];
> > }
> >
> >
> >
>
////////////////////////////////////////////////////////////////////////////////
> >
> > //
> > // HiRA verification method
> > // May be set separately in each "obs.field" entry
> > //
> > hira = {
> >    flag       = FALSE;
> >    width      = [ 2, 3, 4, 5 ];
> >    vld_thresh = 1.0;
> >    cov_thresh = [ ==0.25 ];
> >    shape      = SQUARE;
> > }
> >
> >
> >
>
////////////////////////////////////////////////////////////////////////////////
> >
> > //
> > // Statistical output types
> > // May be set separately in each "obs.field" entry
> > //
> > output_flag = {
> >    fho    = BOTH;
> >    ctc    = BOTH;
> >    cts    = BOTH;
> >    mctc   = BOTH;
> >    mcts   = BOTH;
> >    cnt    = BOTH;
> >    sl1l2  = BOTH;
> >    sal1l2 = BOTH;
> >    vl1l2  = BOTH;
> >    val1l2 = BOTH;
> >    vcnt   = BOTH;
> >    pct    = BOTH;
> >    pstd   = BOTH;
> >    pjc    = BOTH;
> >    prc    = BOTH;
> >    eclv   = BOTH;
> >    mpr    = BOTH;
> > }
> >
> >
> >
>
////////////////////////////////////////////////////////////////////////////////
> >
> > tmp_dir        = "./tmp";
> > output_prefix  = "";
> > version        = "V7.0";
> >
> >
> >
>
////////////////////////////////////////////////////////////////////////////////
> >
> >
>
>
>

------------------------------------------------
Subject: MET execution error while reading longitude from wrf data using point_stat exe
From: 徐 霈霈
Time: Mon Oct 22 18:55:28 2018

Dear sir

    Thank you so much.
    I will examine the ascii data. And yes, I set the wrong level by
verifying 2-m temperature and 10-meter winds data.
   Sincerely Thanks again.

Peipei
________________________________
发件人: John Halley Gotway via RT <met_help at ucar.edu>
发送时间: 2018年10月19日 18:00
收件人: kevin.xpp at hotmail.com
抄送: met_help at mailman.ucar.edu
主题: Re: [rt.rap.ucar.edu #87432] MET execution error while reading
longitude from wrf data using point_stat exe

Peipei,

Thanks for sending your sample data.  I grabbed it and was able to
replicate the behavior you describe.

I found the culprit but can't fully explain why it's causing the
behavior.
45 of the 12400 lines in you ascii point obs file contain bad data:
   grep "999999.0" 20180916000000.txt | wc -l
   45

If you exclude those bad lines, then you do get matched pairs:

egrep -v "999999.0" 20180916000000.txt > 20180916000000_NEW.txt
ascii2nc 20180916000000_NEW.txt 20180916000000_NEW.nc
point_stat WRFPRS_d01.20180916.00 20180916000000_NEW.nc
PointStatConfig_default -v 3
...
DEBUG 2: Processing TMP/L1 versus TMP/L1, for observation type ADPSFC,
over
region FULL, for interpolation method NEAREST(1), using 2471 pairs.

On other detail...

When you run Point-Stat, you should set the "level" to request the
GRIB
record you want to use.  You're requesting levels "L001" for TMP and
"L002"
for UGRD and VGRD.  At verbosity level 3, that tell you that it's
using
GRIB records 5, 17, and 18:


DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match for
VarInfo "TMP/L1" in GRIB record 5 of GRIB file
"WRFPRS_d01.20180916.00".

5:1167450:d=18091600:TMP:kpds5=11:kpds6=109:kpds7=1:TR=0:P1=0:P2=0:TimeU=1:hybrid
lev 1:anl:NAve=00
17:5086614:d=18091600:UGRD:kpds5=33:kpds6=109:kpds7=2:TR=0:P1=0:P2=0:TimeU=1:hybrid
lev 2:anl:NAve=0
18:5447946:d=18091600:VGRD:kpds5=34:kpds6=109:kpds7=2:TR=0:P1=0:P2=0:TimeU=1:hybrid
lev 2:anl:NAve=0

So this is getting these "hybrid levels".  Is that what you want or do
you
want 2-m temperature and 10-meter winds?

For those, you'd use:
   field = [
      { name = "TMP";    level = [ "Z2" ];   },
      { name = "UGRD"; level = [ "Z10" ]; },
      { name = "VGRD"; level = [ "Z10" ]; }
   ];

Thanks,
John

On Thu, Oct 18, 2018 at 7:52 PM 徐 霈霈 via RT <met_help at ucar.edu> wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=87432 >
>
> Dear sir
>
>     Thank you for your reply.
>     You are right, I found the Point-Stat read the longititude of
the
> lower-left corner being off by a factor of -1 too , using other wrf
grib1
> data ,such as the sample data in met-7.0/data/sample_fcst . I have
upload
> the data to http://ftp.rap.ucar.edu
> in incoming/irap/met_help/peipei_data directory including wrf grib1
data
> ,ascii data and the PointStatConfig_default.
>    I have also changed  the longitude of ascii data to make the
> observations fall into the region point_stat read,but there was
still no
> match pairs.I don't know why.   Please help me check the data and
the
> config .
>    I am looking forward to your reply.
>    With best regards.
>
> Peipei Xu
> ________________________________
> 发件人: John Halley Gotway via RT <met_help at ucar.edu>
> 发送时间: 2018年10月18日 15:40
> 收件人: kevin.xpp at hotmail.com
> 抄送: met_help at mailman.ucar.edu
> 主题: Re: [rt.rap.ucar.edu #87432] MET execution error while reading
> longitude from wrf data using point_stat exe
>
> Hello,
>
> I see that you have a question about why Point-Stat is reporting the
> longitude of the lower-left corner being off by a factor of -1.
>
> You expect to see "Lon_LL: 79.881" but are seeing the following
instead:
> DEBUG 3: Grid Definition: Projection: Lambert Conformal Nx: 570 Ny:
390
> Lat_LL: 9.384 Lon_LL: -79.881 Lon_orient: -108.917 Alpha: 1062.189
Cone:
> 0.591 Bx: 284.5534 By: 920.7310
>
> This behavior is expected and does not indicate a problem.  The MET
grid
> library code processes all longitudes in degrees west... instead of
degrees
> east, which is the generally accepted convention.  When MET reads a
grid
> spec from GRIB1, it multiplies longitudes by -1 to flip from degrees
east
> to west.  And when it writes longitudes out, it multiples by -1 to
flip
> them back.  But internally, and as shown in log messages, they're
stored as
> degrees west.
>
> I realize that's confusing and a headache.  We really should just
switch to
> using the standard convention of degrees east throughout the code,
but we
> haven't gotten around to changing that.
>
> So it seems like the real issue is that you're getting 0 matched
pairs from
> your Point-Stat run.  Running at verbosity level 3 (or higher) is
exactly
> the right thing to do because it dumps out counts for reasons why
obs were
> skipped, like this...
>
> DEBUG 2: Processing TMP/L1 versus TMP/L1, for observation type
ADPSFC, over
> region FULL, for interpolation method NEAREST(1), using 0 pairs.
> DEBUG 3: Number of matched pairs  = 0
> DEBUG 3: Observations processed   = 12400
> DEBUG 3: Rejected: SID exclusion  = 0
> DEBUG 3: Rejected: obs type       = 12400
>
> These log messages are telling us that Point-Stat read through 12400
> observations looking for observations named "TMP", but it found 0 of
them.
> And that's why you got 0 matched pairs.  This should be pretty easy
to
> fix.  We just need to figure out how to request observations from
your
> point observation file: 20180916000000.nc
>
> We've had some changes in how MET stores point observation info in
the last
> couple years.  We're moving from the limiting convention of
referencing
> GRIB code to just using simple strings for observation variable
names.  The
> formatting of the ascii2nc output file really depends on how you set
up
> your ASCII observations.  Can you please post this sample file to
our
> anonymous ftp site (20180916000000.nc)?  I'll go grab it from there
and
> take a closer look.
>
> Once we get this issue resolved, it's possible that you'll still get
0
> matched pairs for other reasons.  But we need to fix this one first.
>
> You can follow these instructions for posting to our ftp site:
> https://dtcenter.org/met/users/support/met_help.php#ftp
>
> Thanks,
> John Halley Gotway
>
>
> On Thu, Oct 18, 2018 at 12:18 AM 徐 霈霈 via RT <met_help at ucar.edu>
wrote:
>
> >
> > Thu Oct 18 00:18:15 2018: Request 87432 was acted upon.
> > Transaction: Ticket created by kevin.xpp at hotmail.com
> >        Queue: met_help
> >      Subject: MET execution error while reading longitude from wrf
data
> > using point_stat exe
> >        Owner: Nobody
> >   Requestors: kevin.xpp at hotmail.com
> >       Status: new
> >  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=87432 >
> >
> >
> > Dear sir
> >
> >     I have a problem while executing point_stat exe with wrf data
> > processed by upp software and observation data processed by
ascii2nc exe.
> > Point_stat read the wrong longitude value from wrf data,the lon_ll
from
> the
> > wrf data is 79.881,but the log shows -79.881。the log and
PointStatConfig
> is
> > below,how to set the config to make point_stat read the right
value of
> wrf
> > grib1 data ?
> >
> > log:
> >
> > DEBUG 1: Default Config File:
> >
> /home/huayunshengda/Downloads/met-
7.0/share/met/config/PointStatConfig_default
> > DEBUG 1: User Config File: PointStatConfig_default
> > GSL_RNG_TYPE=mt19937
> > GSL_RNG_SEED=488556635
> > DEBUG 1: Forecast File: WRFPRS_d01.20180916.00
> > DEBUG 1: Observation File: 20180916000000.nc
> > DEBUG 2:
> > DEBUG 2:
> >
>
--------------------------------------------------------------------------------
> > DEBUG 2:
> > DEBUG 2: Reading data for TMP/L1.
> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match
for
> > VarInfo "TMP/L1" in GRIB record 5 of GRIB file
"WRFPRS_d01.20180916.00".
> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 1 GRIB
records
> > matching VarInfo "TMP/L1" in GRIB file "WRFPRS_d01.20180916.00".
> > DEBUG 3: Use the matching forecast and observation grids.
> > DEBUG 3: Grid Definition: Projection: Lambert Conformal Nx: 570
Ny: 390
> > Lat_LL: 9.384 Lon_LL: -79.881 Lon_orient: -108.917 Alpha: 1062.189
Cone:
> > 0.591 Bx: 284.5534 By: 920.7310
> > DEBUG 2: Processing masking regions.
> > DEBUG 3: Processing grid mask: FULL
> > DEBUG 2: For TMP/L1 found 1 forecast levels, 0 climatology mean
levels,
> > and 0 climatology standard deviation levels.
> > DEBUG 2:
> > DEBUG 2:
> >
>
--------------------------------------------------------------------------------
> > DEBUG 2:
> > DEBUG 2: Reading data for UGRD/L2.
> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found exact match
for
> > VarInfo "UGRD/L2" in GRIB record 17 of GRIB file
> "WRFPRS_d01.20180916.00".
> > DEBUG 3: MetGrib1DataFile::rotate_winds() -> Have U-wind record,
reading
> > V-wind record.
> > DEBUG 3: MetGrib1DataFile::data_plane_scalar() -> Found exact
match for
> > VarInfo "UGRD/L2" in GRIB record 18 of GRIB file
> "WRFPRS_d01.20180916.00".
> > DEBUG 3: Rotating U and V wind components from grid-relative to
> > earth-relative.
> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found range match
for
> > VarInfo "UGRD/L2" in GRIB record 363 of GRIB file
> "WRFPRS_d01.20180916.00".
> > DEBUG 3: MetGrib1DataFile::rotate_winds() -> Have U-wind record,
reading
> > V-wind record.
> > DEBUG 3: MetGrib1DataFile::data_plane_scalar() -> Found exact
match for
> > VarInfo "UGRD/L2" in GRIB record 18 of GRIB file
> "WRFPRS_d01.20180916.00".
> > DEBUG 3: Rotating U and V wind components from grid-relative to
> > earth-relative.
> > DEBUG 3: MetGrib1DataFile::data_plane_array() -> Found 2 GRIB
records
> > matching VarInfo "UGRD/L2" in GRIB file "WRFPRS_d01.20180916.00".
> > DEBUG 2: For UGRD/L2 found 2 forecast levels, 0 climatology mean
levels,
> > and 0 climatology standard deviation levels.
> > DEBUG 2: Searching 12400 observations from 2480 messages.
> > DEBUG 2:
> > DEBUG 2:
> >
>
--------------------------------------------------------------------------------
> > DEBUG 2:
> > DEBUG 2: Processing TMP/L1 versus TMP/L1, for observation type
ADPSFC,
> > over region FULL, for interpolation method NEAREST(1), using 0
pairs.
> > DEBUG 3: Number of matched pairs  = 0
> > DEBUG 3: Observations processed   = 12400
> > DEBUG 3: Rejected: SID exclusion  = 0
> > DEBUG 3: Rejected: obs type       = 12400
> > DEBUG 3: Rejected: valid time     = 0
> > DEBUG 3: Rejected: bad obs value  = 0
> > DEBUG 3: Rejected: off the grid   = 0
> > DEBUG 3: Rejected: level mismatch = 0
> > DEBUG 3: Rejected: quality marker = 0
> > DEBUG 3: Rejected: message type   = 0
> > DEBUG 3: Rejected: masking region = 0
> > DEBUG 3: Rejected: bad fcst value = 0
> > DEBUG 3: Rejected: duplicates     = 0
> >
> >
> > config:
> >
> >
>
////////////////////////////////////////////////////////////////////////////////
> > //
> > // Point-Stat configuration file.
> > //
> > // For additional information, see the MET_BASE/config/README
file.
> > //
> >
> >
>
////////////////////////////////////////////////////////////////////////////////
> >
> > //
> > // Output model name to be written
> > //
> > model = "WRF";
> >
> > //
> > // Output description to be written
> > // May be set separately in each "obs.field" entry
> > //
> > desc = "NA";
> >
> >
> >
>
////////////////////////////////////////////////////////////////////////////////
> >
> > //
> > // Verification grid
> > // May be set separately in each "field" entry
> > //
> >  regrid = {
> >    to_grid    = NONE;
> >    method     = NEAREST;
> >    width      = 1;
> >    vld_thresh = 0.5;
> >    shape      = SQUARE;
> >  }
> >
> >
> >
>
////////////////////////////////////////////////////////////////////////////////
> >
> > //
> > // May be set separately in each "field" entry
> > //
> > censor_thresh  = [];
> > censor_val     = [];
> > cat_thresh     = [ NA ];
> > cnt_thresh     = [ NA ];
> > cnt_logic      = UNION;
> > wind_thresh    = [ NA ];
> > wind_logic     = UNION;
> > eclv_points    = 0.05;
> > rank_corr_flag = FALSE;
> >
> > //
> > // Forecast and observation fields to be verified
> > //
> > fcst = {
> >    field = [
> >
> >       {
> >         name       = "TMP";
> >         level      = [ "L001" ];
> >       },
> >
> >       {
> >         name       = "UGRD";
> >         level      = [ "L002" ];
> >       },
> >
> >       {
> >         name       = "VGRD";
> >         level      = [ "L002" ];
> >       }
> >    ];
> >
> > }
> >
> > obs = fcst;
> >
> >
> >
> >
>
////////////////////////////////////////////////////////////////////////////////
> >
> > //
> > // Point observation filtering options
> > // May be set separately in each "obs.field" entry
> > //
> > message_type   = [ "ADPSFC" ];
> > sid_exc        = [];
> > obs_quality    = [];
> > duplicate_flag = NONE;
> > obs_summary    = NONE;
> > obs_perc_value = 50;
> >
> > //
> > // Mapping of message type group name to comma-separated list of
values.
> > //
> > message_type_group_map = [
> >    { key = "SURFACE"; val = "ADPSFC,SFCSHP,MSONET";
}
> > ];
> >
> >
> >
>
////////////////////////////////////////////////////////////////////////////////
> >
> > //
> > // Climatology data
> > //
> > //climo_mean = {
> > //
> > //   file_name = [];
> > //   field     = [];
> > //
> > //   regrid = {
> > //      method     = NEAREST;
> > //      width      = 1;
> > //      vld_thresh = 0.5;
> > //      shape      = SQUARE;
> > //   }
> > //
> > //   time_interp_method = DW_MEAN;
> > //   match_day          = FALSE;
> > //   time_step          = 21600;
> > //}
> > //
> > //climo_stdev = climo_mean;
> > //climo_stdev = {
> > //   file_name = [];
> > //}
> > //
> > //climo_cdf_bins = 1;
> >
> >
> >
>
////////////////////////////////////////////////////////////////////////////////
> >
> > //
> > // Point observation time window
> > // May be set separately in each "obs.field" entry
> > //
> > obs_window = {
> >    beg = -5400;
> >    end =  5400;
> > }
> >
> >
> >
>
////////////////////////////////////////////////////////////////////////////////
> >
> > //
> > // Verification masking regions
> > // May be set separately in each "obs.field" entry
> > //
> > mask = {
> >    grid    = ["FULL"];
> >    poly    = [];
> >    sid     = [];
> > }
> >
> >
> >
>
////////////////////////////////////////////////////////////////////////////////
> >
> > //
> > // Confidence interval settings
> > // May be set separately in each "obs.field" entry
> > //
> > ci_alpha  = [ 0.05 ];
> >
> > boot = {
> >    interval = PCTILE;
> >    rep_prop = 1.0;
> >    n_rep    = 0;
> >    rng      = "mt19937";
> >    seed     = "";
> > }
> >
> >
> >
>
////////////////////////////////////////////////////////////////////////////////
> >
> > //
> > // Interpolation methods
> > // May be set separately in each "obs.field" entry
> > //
> > interp = {
> >    vld_thresh = 1.0;
> >    shape      = SQUARE;
> >
> >    type = [
> >       {
> >          method = NEAREST;
> >          width  = 1;
> >       }
> >    ];
> > }
> >
> >
> >
>
////////////////////////////////////////////////////////////////////////////////
> >
> > //
> > // HiRA verification method
> > // May be set separately in each "obs.field" entry
> > //
> > hira = {
> >    flag       = FALSE;
> >    width      = [ 2, 3, 4, 5 ];
> >    vld_thresh = 1.0;
> >    cov_thresh = [ ==0.25 ];
> >    shape      = SQUARE;
> > }
> >
> >
> >
>
////////////////////////////////////////////////////////////////////////////////
> >
> > //
> > // Statistical output types
> > // May be set separately in each "obs.field" entry
> > //
> > output_flag = {
> >    fho    = BOTH;
> >    ctc    = BOTH;
> >    cts    = BOTH;
> >    mctc   = BOTH;
> >    mcts   = BOTH;
> >    cnt    = BOTH;
> >    sl1l2  = BOTH;
> >    sal1l2 = BOTH;
> >    vl1l2  = BOTH;
> >    val1l2 = BOTH;
> >    vcnt   = BOTH;
> >    pct    = BOTH;
> >    pstd   = BOTH;
> >    pjc    = BOTH;
> >    prc    = BOTH;
> >    eclv   = BOTH;
> >    mpr    = BOTH;
> > }
> >
> >
> >
>
////////////////////////////////////////////////////////////////////////////////
> >
> > tmp_dir        = "./tmp";
> > output_prefix  = "";
> > version        = "V7.0";
> >
> >
> >
>
////////////////////////////////////////////////////////////////////////////////
> >
> >
>
>
>


------------------------------------------------


More information about the Met_help mailing list