[Met_help] [rt.rap.ucar.edu #69991] History for Rejected: Grid Code problem

Wed Dec 10 09:19:40 MST 2014

----------------------------------------------------------------
  Initial Request
----------------------------------------------------------------

Hello,

   I am trying to run point_stat on a Asia domain,

   The wrfout was processed by unipost firstly, then convert to a China
domain with following command:

   copygb.exe -xg"255 3 450 350 10099 75048 8 101500 13500 13500 0 64 20000
50000" wrfprs06 wrfprs06.china

   The prepbufr data was converted to netCDF format, attached PS file is
the locations of all obs,

   Please see attached log.txt to the point_stat run, the number pair is 0,
I think the reason is the GRIB CODE reject.

   I am not sure what wrong with this , the wrfprs06.china has everything
in it and the observation file also includes the "ADPUPA", "ADPSFC" , and
obs_grib_code = [ "SPFH", "TMP", "HGT", "UGRD", "VGRD", "RH" ] .....

   Maybe you have a quick answer for this .

Many Thanks in advance!

    (See attached file: ps_2014120400.ps)

(See attached file: log.txt)

Regards

 Xin Zhang,                              19 Zhongguancun                 
 Ph.D. (张昕)                           Software Park                    

 Research Staff                          Beijing, 100193                 
 Member                                                                  

 Industry                                China                           
 Solutions                                                               

 IBM                                                                     
 Research-China                                                          

 Phone:         (86-10) 58748142                                         

 Home:          (+1) 3037476490                                          

 Mobile:        (86) 18811446777; (+1)                                   
                720289971                                                

 e-mail:        xin.zhang at cn.ibm.com                                     

----------------------------------------------------------------
  Complete Ticket History
----------------------------------------------------------------

Subject: Rejected: Grid Code problem
From: John Halley Gotway
Time: Fri Dec 05 10:29:42 2014

Xin Zhang,

I see that you're having problems getting matched pair output from
Point-Stat.  Looks like you've already taken the first two steps I'd
recommend... turn up the verbosity level in your call to point_stat
and use
plot_point_obs to plot the location of the point observations.

The order in which the rejection reasons are listed in the point_stat
log
output matches the order in which the logic is applied in the code.
Listed
below is the output for the last verification task listed in that log
file.  There were 5371 observations that made it all the way down to
checking for a level match, but they were discarded for that reason.
When
verifying surface fields (like 10m winds) using the ADPSFC (i.e.
surface)
message type, the code should not be applying any additional level
matching
logic.  So that's why I'm confused.

Have you made any changes to the MET source code?  If so, please try
reverting back to the original met-5.0 release and see if that works.

If not, please send me some sample files to work with... I'd need a
sample
forecast file, observation file, and point_stat configuration file:

   postprd/wrfprs06.small
   2014120400/pb2nc/obs_2014120400.nc

   PointStatConfig_crl

You can post it to our anonymous ftp site following these
instructions:
   http://www.dtcenter.org/met/users/support/met_help.php#ftp
I'll run with your data here and try to figure out the source of the
problem.

Please let me know when you've posted sample data so I can go grab it.

Thanks,
John Halley Gotway
met_help at ucar.edu

DEBUG 2: Processing VGRD/Z10 versus VGRD/Z10, for observation type
ADPSFC, over region FULL, for interpolation method UW_MEAN(25), using
0 pairs.
DEBUG 3: Number of matched pairs  = 0
DEBUG 3: Observations processed   = 153710
DEBUG 3: Rejected: SID exclusion  = 0
DEBUG 3: Rejected: GRIB code      = 119520
DEBUG 3: Rejected: valid time     = 0
DEBUG 3: Rejected: bad obs value  = 0
DEBUG 3: Rejected: off the grid   = 28819
DEBUG 3: Rejected: level mismatch = 5371
DEBUG 3: Rejected: quality marker = 0
DEBUG 3: Rejected: message type   = 0
DEBUG 3: Rejected: masking region = 0
DEBUG 3: Rejected: bad fcst value = 0
DEBUG 3: Rejected: duplicates     = 0

On Fri, Dec 5, 2014 at 1:15 AM, Xin DU Zhang via RT
<met_help at ucar.edu>
wrote:

>
> Fri Dec 05 01:15:06 2014: Request 69991 was acted upon.
> Transaction: Ticket created by xin.zhang at cn.ibm.com
>        Queue: met_help
>      Subject: Rejected: Grid Code problem
>        Owner: Nobody
>   Requestors: xin.zhang at cn.ibm.com
>       Status: new
>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69991 >
>
>
>
> Hello,
>
>    I am trying to run point_stat on a Asia domain,
>
>    The wrfout was processed by unipost firstly, then convert to a
China
> domain with following command:
>
>    copygb.exe -xg"255 3 450 350 10099 75048 8 101500 13500 13500 0
64 20000
> 50000" wrfprs06 wrfprs06.china
>
>    The prepbufr data was converted to netCDF format, attached PS
file is
> the locations of all obs,
>
>    Please see attached log.txt to the point_stat run, the number
pair is 0,
> I think the reason is the GRIB CODE reject.
>
>    I am not sure what wrong with this , the wrfprs06.china has
everything
> in it and the observation file also includes the "ADPUPA", "ADPSFC"
, and
> obs_grib_code = [ "SPFH", "TMP", "HGT", "UGRD", "VGRD", "RH" ] .....
>
>    Maybe you have a quick answer for this .
>
>
> Many Thanks in advance!
>
>     (See attached file: ps_2014120400.ps)
>
>
> (See attached file: log.txt)
>
> Regards
>
>
>
>
>
>  Xin Zhang,                              19 Zhongguancun
>  Ph.D. (张昕)                           Software Park
>
>  Research Staff                          Beijing, 100193
>  Member
>
>  Industry                                China
>  Solutions
>
>  IBM
>  Research-China
>
>  Phone:         (86-10) 58748142
>
>  Home:          (+1) 3037476490
>
>  Mobile:        (86) 18811446777; (+1)
>                 720289971
>
>  e-mail:        xin.zhang at cn.ibm.com
>
>
>
>
>
>
>
>
>
>

------------------------------------------------
Subject: Rejected: Grid Code problem
From: Xin DU Zhang
Time: Fri Dec 05 11:50:01 2014

Hello John,

    Thank you very much for your quick response.

    I did not change anything in source code.

    I can not access google in China, so I uploaded the data to
ftp.ucar.edu:/pub/mmm/xinzhang/xinzhang_met.tgz  (Anonymous ftp)

Regards

 Xin Zhang,                              19 Zhongguancun
 Ph.D. (张昕)                           Software Park

 Research Staff                          Beijing, 100193
 Member

 Industry                                China
 Solutions

 IBM
 Research-China

 Phone:         (86-10) 58748142

 Home:          (+1) 3037476490

 Mobile:        (86) 18811446777; (+1)
                720289971

 e-mail:        xin.zhang at cn.ibm.com

From:	"John Halley Gotway via RT" <met_help at ucar.edu>
To:	Xin DU Zhang/China/IBM at IBMCN
Date:	2014/12/06 01:29
Subject:	Re: [rt.rap.ucar.edu #69991] Rejected: Grid Code problem

Xin Zhang,

I see that you're having problems getting matched pair output from
Point-Stat.  Looks like you've already taken the first two steps I'd
recommend... turn up the verbosity level in your call to point_stat
and use
plot_point_obs to plot the location of the point observations.

The order in which the rejection reasons are listed in the point_stat
log
output matches the order in which the logic is applied in the code.
Listed
below is the output for the last verification task listed in that log
file.  There were 5371 observations that made it all the way down to
checking for a level match, but they were discarded for that reason.
When
verifying surface fields (like 10m winds) using the ADPSFC (i.e.
surface)
message type, the code should not be applying any additional level
matching
logic.  So that's why I'm confused.

Have you made any changes to the MET source code?  If so, please try
reverting back to the original met-5.0 release and see if that works.

If not, please send me some sample files to work with... I'd need a
sample
forecast file, observation file, and point_stat configuration file:

   postprd/wrfprs06.small
   2014120400/pb2nc/obs_2014120400.nc

   PointStatConfig_crl

You can post it to our anonymous ftp site following these
instructions:
   http://www.dtcenter.org/met/users/support/met_help.php#ftp
I'll run with your data here and try to figure out the source of the
problem.

Please let me know when you've posted sample data so I can go grab it.

Thanks,
John Halley Gotway
met_help at ucar.edu

DEBUG 2: Processing VGRD/Z10 versus VGRD/Z10, for observation type
ADPSFC, over region FULL, for interpolation method UW_MEAN(25), using
0 pairs.
DEBUG 3: Number of matched pairs  = 0
DEBUG 3: Observations processed   = 153710
DEBUG 3: Rejected: SID exclusion  = 0
DEBUG 3: Rejected: GRIB code      = 119520
DEBUG 3: Rejected: valid time     = 0
DEBUG 3: Rejected: bad obs value  = 0
DEBUG 3: Rejected: off the grid   = 28819
DEBUG 3: Rejected: level mismatch = 5371
DEBUG 3: Rejected: quality marker = 0
DEBUG 3: Rejected: message type   = 0
DEBUG 3: Rejected: masking region = 0
DEBUG 3: Rejected: bad fcst value = 0
DEBUG 3: Rejected: duplicates     = 0

On Fri, Dec 5, 2014 at 1:15 AM, Xin DU Zhang via RT
<met_help at ucar.edu>
wrote:

>
> Fri Dec 05 01:15:06 2014: Request 69991 was acted upon.
> Transaction: Ticket created by xin.zhang at cn.ibm.com
>        Queue: met_help
>      Subject: Rejected: Grid Code problem
>        Owner: Nobody
>   Requestors: xin.zhang at cn.ibm.com
>       Status: new
>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69991 >
>
>
>
> Hello,
>
>    I am trying to run point_stat on a Asia domain,
>
>    The wrfout was processed by unipost firstly, then convert to a
China
> domain with following command:
>
>    copygb.exe -xg"255 3 450 350 10099 75048 8 101500 13500 13500 0
64
20000
> 50000" wrfprs06 wrfprs06.china
>
>    The prepbufr data was converted to netCDF format, attached PS
file is
> the locations of all obs,
>
>    Please see attached log.txt to the point_stat run, the number
pair is
0,
> I think the reason is the GRIB CODE reject.
>
>    I am not sure what wrong with this , the wrfprs06.china has
everything
> in it and the observation file also includes the "ADPUPA", "ADPSFC"
, and
> obs_grib_code = [ "SPFH", "TMP", "HGT", "UGRD", "VGRD", "RH" ] .....
>
>    Maybe you have a quick answer for this .
>
>
> Many Thanks in advance!
>
>     (See attached file: ps_2014120400.ps)
>
>
> (See attached file: log.txt)
>
> Regards
>
>
>
>
>
>  Xin Zhang,                              19 Zhongguancun
>  Ph.D. (张昕)                           Software Park
>
>  Research Staff                          Beijing, 100193
>  Member
>
>  Industry                                China
>  Solutions
>
>  IBM
>  Research-China
>
>  Phone:         (86-10) 58748142
>
>  Home:          (+1) 3037476490
>
>  Mobile:        (86) 18811446777; (+1)
>                 720289971
>
>  e-mail:        xin.zhang at cn.ibm.com
>
>
>
>
>
>
>
>
>
>

------------------------------------------------
Subject: Rejected: Grid Code problem
From: John Halley Gotway
Time: Fri Dec 05 16:35:53 2014

Xin,

Thanks for sending your data.  I think I see the problem.  I would
expect
the surface variables to be verified by observations of type ADPSFC.
However, there are no ADPSFC observations for the fields you're trying
to
verify in the point NetCDF file you sent me.

This can be seen in a couple of different ways:
 - You can run the following commands to dump that NetCDF file to
ASCII:
    > Rscript met-5.0/scripts/Rscripts/pntnc2ascii.R obs_2014120400.nc
>
obs_2014120400.txt
    > grep ADPSFC obs_2014120400.txt
   While there are some ADPSFC observations, they're only for GRIB
code 7,
geopotential height.

- Alternatively, you can run the plot_point_obs tool, looking only for
ADPSFC and temperature:
   met-5.0/bin/plot_point_obs obs_2014120400.nc
obs_2014120400_ADPSFC.ps
-msg_typ ADPSFC -gc 11 -data_file wrfprs06.small
  That plots 0 points.

Here's my best guess as to what's going on.  I'm guessing you're using
GDAS
PREPBUFR point observations, and you're processing them through PB2NC,
using the default PB2NC configuration file.  That config file uses a
default quality marker value of 2.  Please take a look at the note on
the
bottom of this page:
   http://www.dtcenter.org/met/users/downloads/observation_data.php

Unfortunately, the GDAS system sets most surface observations to a
quality
marker value of 9 to prevent them from being using in the data
assimilation
system.  Therefore, a quality marker threshold of 2 in the PB2NC
effectively discards all surface observations.

I'd suggest rerunning PB2NC, setting quality_mark_thresh = 9.  You
should
get some matched pairs from Point-Stat after that.

Hope that helps.

Thanks,
John Halley Gotway
met_help at ucar.edu

On Fri, Dec 5, 2014 at 11:50 AM, Xin DU Zhang via RT
<met_help at ucar.edu>
wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69991 >
>
> Hello John,
>
>     Thank you very much for your quick response.
>
>     I did not change anything in source code.
>
>     I can not access google in China, so I uploaded the data to
> ftp.ucar.edu:/pub/mmm/xinzhang/xinzhang_met.tgz  (Anonymous ftp)
>
>
> Regards
>
>
>
>
>
>  Xin Zhang,                              19 Zhongguancun
>  Ph.D. (张昕)                           Software Park
>
>  Research Staff                          Beijing, 100193
>  Member
>
>  Industry                                China
>  Solutions
>
>  IBM
>  Research-China
>
>  Phone:         (86-10) 58748142
>
>  Home:          (+1) 3037476490
>
>  Mobile:        (86) 18811446777; (+1)
>                 720289971
>
>  e-mail:        xin.zhang at cn.ibm.com
>
>
>
>
>
>
>
>
>
>
>
>
>
> From:   "John Halley Gotway via RT" <met_help at ucar.edu>
> To:     Xin DU Zhang/China/IBM at IBMCN
> Date:   2014/12/06 01:29
> Subject:        Re: [rt.rap.ucar.edu #69991] Rejected: Grid Code
problem
>
>
>
> Xin Zhang,
>
> I see that you're having problems getting matched pair output from
> Point-Stat.  Looks like you've already taken the first two steps I'd
> recommend... turn up the verbosity level in your call to point_stat
and use
> plot_point_obs to plot the location of the point observations.
>
> The order in which the rejection reasons are listed in the
point_stat log
> output matches the order in which the logic is applied in the code.
Listed
> below is the output for the last verification task listed in that
log
> file.  There were 5371 observations that made it all the way down to
> checking for a level match, but they were discarded for that reason.
When
> verifying surface fields (like 10m winds) using the ADPSFC (i.e.
surface)
> message type, the code should not be applying any additional level
matching
> logic.  So that's why I'm confused.
>
> Have you made any changes to the MET source code?  If so, please try
> reverting back to the original met-5.0 release and see if that
works.
>
> If not, please send me some sample files to work with... I'd need a
sample
> forecast file, observation file, and point_stat configuration file:
>
>    postprd/wrfprs06.small
>    2014120400/pb2nc/obs_2014120400.nc
>
>
>    PointStatConfig_crl
>
>
>
> You can post it to our anonymous ftp site following these
instructions:
>    http://www.dtcenter.org/met/users/support/met_help.php#ftp
> I'll run with your data here and try to figure out the source of the
> problem.
>
> Please let me know when you've posted sample data so I can go grab
it.
>
> Thanks,
> John Halley Gotway
> met_help at ucar.edu
>
>
> DEBUG 2: Processing VGRD/Z10 versus VGRD/Z10, for observation type
> ADPSFC, over region FULL, for interpolation method UW_MEAN(25),
using
> 0 pairs.
> DEBUG 3: Number of matched pairs  = 0
> DEBUG 3: Observations processed   = 153710
> DEBUG 3: Rejected: SID exclusion  = 0
> DEBUG 3: Rejected: GRIB code      = 119520
> DEBUG 3: Rejected: valid time     = 0
> DEBUG 3: Rejected: bad obs value  = 0
> DEBUG 3: Rejected: off the grid   = 28819
> DEBUG 3: Rejected: level mismatch = 5371
> DEBUG 3: Rejected: quality marker = 0
> DEBUG 3: Rejected: message type   = 0
> DEBUG 3: Rejected: masking region = 0
> DEBUG 3: Rejected: bad fcst value = 0
> DEBUG 3: Rejected: duplicates     = 0
>
>
>
> On Fri, Dec 5, 2014 at 1:15 AM, Xin DU Zhang via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > Fri Dec 05 01:15:06 2014: Request 69991 was acted upon.
> > Transaction: Ticket created by xin.zhang at cn.ibm.com
> >        Queue: met_help
> >      Subject: Rejected: Grid Code problem
> >        Owner: Nobody
> >   Requestors: xin.zhang at cn.ibm.com
> >       Status: new
> >  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=69991 >
> >
> >
> >
> > Hello,
> >
> >    I am trying to run point_stat on a Asia domain,
> >
> >    The wrfout was processed by unipost firstly, then convert to a
China
> > domain with following command:
> >
> >    copygb.exe -xg"255 3 450 350 10099 75048 8 101500 13500 13500 0
64
> 20000
> > 50000" wrfprs06 wrfprs06.china
> >
> >    The prepbufr data was converted to netCDF format, attached PS
file is
> > the locations of all obs,
> >
> >    Please see attached log.txt to the point_stat run, the number
pair is
> 0,
> > I think the reason is the GRIB CODE reject.
> >
> >    I am not sure what wrong with this , the wrfprs06.china has
everything
> > in it and the observation file also includes the "ADPUPA",
"ADPSFC" , and
> > obs_grib_code = [ "SPFH", "TMP", "HGT", "UGRD", "VGRD", "RH" ]
.....
> >
> >    Maybe you have a quick answer for this .
> >
> >
> > Many Thanks in advance!
> >
> >     (See attached file: ps_2014120400.ps)
> >
> >
> > (See attached file: log.txt)
> >
> > Regards
> >
> >
> >
> >
> >
> >  Xin Zhang,                              19 Zhongguancun
> >  Ph.D. (张昕)                           Software Park
> >
> >  Research Staff                          Beijing, 100193
> >  Member
> >
> >  Industry                                China
> >  Solutions
> >
> >  IBM
> >  Research-China
> >
> >  Phone:         (86-10) 58748142
> >
> >  Home:          (+1) 3037476490
> >
> >  Mobile:        (86) 18811446777; (+1)
> >                 720289971
> >
> >  e-mail:        xin.zhang at cn.ibm.com
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
>
>
>

------------------------------------------------
Subject: Rejected: Grid Code problem
From: Xin DU Zhang
Time: Tue Dec 09 20:31:48 2014