[Met_help] [rt.rap.ucar.edu #63649] History for compare 10m wind between wrf output and prepbufr data
John Halley Gotway via RT
met_help at ucar.edu
Fri Nov 1 13:06:02 MDT 2013
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hi CS,
I would like to compare 10 m wind between wrf output and prepbufr.
the output from WRF go through postprocess by UPPV2.1, I uploaded two data
files and one point-stat configure file.
ftp -p ftp.rap.ucar.edu/incoming/irap/met_help/zhu_data
Would you please help me figure out why I can not produce output to compare
wind?
Perhaps there is no 10m wind in prepbufr file? I don't know how to check if
there is 10m wind in prepbufr files.
Thank you in advance,
Jiang
**********************************************************
Jiang Zhu, Ph.D.
Geographic Information Network of Alaska
University of Alaska Fairbanks
Phone: 907-474-5689, EMAIL: jiang at gina.alaska.edu
***********************************************************
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #63649] compare 10m wind between wrf output and prepbufr data
From: John Halley Gotway
Time: Thu Oct 31 10:37:21 2013
Jiang,
I see that you're having trouble producing matched pairs when
evaluating
the U and V components of wind. I grabbed the sample data you sent
and
reproduced the behavior you're seeing. Let me step you through the
steps
I took to figure out what's going on...
First, I used wgrib to inventory the forecast file you sent and check
for
UGRD and VGRD at 10-meters. And it looks like it's in there...
[johnhg at rambler]% wgrib WRFPRS_d01_2012-11-05_00_00_00 | egrep
"UGRD|VGRD|WIND" | grep ":10 m"
275:5539450:d=12110418:UGRD:kpds5=33:kpds6=105:kpds7=10:TR=0:P1=6:P2=0:TimeU=1:10
m above gnd:6hr fcst:NAve=0
276:5559028:d=12110418:VGRD:kpds5=34:kpds6=105:kpds7=10:TR=0:P1=6:P2=0:TimeU=1:10
m above gnd:6hr fcst:NAve=0
Second, I ran Point-Stat with a verbosity level of 3 (-v 3). For each
verification task, Point-Stat can dump out reason counts for why
observations were or were not used for that task. Here's the output
for
the UGRD and VGRD tasks...
DEBUG 2: Processing VGRD/Z10 versus VGRD/Z10, for observation type
ADPSFC,
over region FULL, for interpolation method UW_MEAN(1), using 0 pairs.
DEBUG 3: Number of matched pairs = 0
DEBUG 3: Observations processed = 204910
DEBUG 3: Rejected: GRIB code = 170420
DEBUG 3: Rejected: valid time = 0
DEBUG 3: Rejected: bad obs value = 0
DEBUG 3: Rejected: off the grid = 33793
DEBUG 3: Rejected: level mismatch = 697
DEBUG 3: Rejected: quality marker = 0
DEBUG 3: Rejected: message type = 0
DEBUG 3: Rejected: masking region = 0
DEBUG 3: Rejected: bad fcst value = 0
DEBUG 3: Rejected: duplicates = 0
DEBUG 2: Processing VGRD/Z10 versus VGRD/Z10, for observation type
ADPSFC,
over region FULL, for interpolation method UW_MEAN(25), using 0 pairs.
DEBUG 3: Number of matched pairs = 0
DEBUG 3: Observations processed = 204910
DEBUG 3: Rejected: GRIB code = 170420
DEBUG 3: Rejected: valid time = 0
DEBUG 3: Rejected: bad obs value = 0
DEBUG 3: Rejected: off the grid = 33793
DEBUG 3: Rejected: level mismatch = 697
DEBUG 3: Rejected: quality marker = 0
DEBUG 3: Rejected: message type = 0
DEBUG 3: Rejected: masking region = 0
DEBUG 3: Rejected: bad fcst value = 0
DEBUG 3: Rejected: duplicates = 0
Looking at this, I don't see any obvious problems.
Third, I ran the plot_point_obs utility to figure out what
observations of
wind fall in your domain...
[johnhg at rambler]%
/d1/johnhg/MET/MET_releases/METv4.1/bin/plot_point_obs
20121105.gdas1.t00z.prepbufr.nr_le.nc
20121105.gdas1.t00z.prepbufr.nr_le.ps -gc 33 -gc 34 -msg_typ ADPSFC
-data_file WRFPRS_d01_2012-11-05_00_00_00
DEBUG 2: Retrieving grid from file: WRFPRS_d01_2012-11-05_00_00_00
DEBUG 1: Opening netCDF file: 20121105.gdas1.t00z.prepbufr.nr_le.nc
DEBUG 2: Processing 204910 observations at 1339 locations.
DEBUG 2: Observation GRIB codes: 33 34
DEBUG 2: Observation message types: ADPSFC
DEBUG 1: Creating postscript file:
20121105.gdas1.t00z.prepbufr.nr_le.ps
DEBUG 2: Finished plotting 0 locations.
DEBUG 2: Skipped 0 locations off the grid.
Looking at this, there are no wind observations (GRIB code 33 and 34)
for
the ADPSFC message type over your domain for this time.
FYI - if you plot all message types by removing "-msg_typ ADPSFC",
you'll
see that there are 11 non-ADPSFC observation locations.
[johnhg at rambler]%
/d1/johnhg/MET/MET_releases/METv4.1/bin/plot_point_obs
20121105.gdas1.t00z.prepbufr.nr_le.nc
20121105.gdas1.t00z.prepbufr.nr_le.ps -gc 33 -gc 34 -data_file
WRFPRS_d01_2012-11-05_00_00_00
DEBUG 2: Retrieving grid from file: WRFPRS_d01_2012-11-05_00_00_00
DEBUG 1: Opening netCDF file: 20121105.gdas1.t00z.prepbufr.nr_le.nc
DEBUG 2: Processing 204910 observations at 1339 locations.
DEBUG 2: Observation GRIB codes: 33 34
DEBUG 2: Observation message types: ALL
DEBUG 1: Creating postscript file:
20121105.gdas1.t00z.prepbufr.nr_le.ps
DEBUG 2: Finished plotting 11 locations.
DEBUG 2: Skipped 692 locations off the grid.
So there just aren't any observations of wind at this time with
message
type ADPSFC over your domain. Why not?
I suspect it's because of the quality control value used for surface
data
in GDAS files. Please read the "Note" at the bottom of this page...
http://www.dtcenter.org/met/users/downloads/observation_data.php
Please try rerunning PB2NC with "quality_mark_thresh = 9" in the
config
file. After you rerun PB2NC, I suspect you'll get matched pairs in
your
Point-Stat output.
Sorry to be so long-winded on this. I just wanted to step you through
the
"debugging" process to assist you in doing it yourself in the future.
Hope that helps.
Thanks,
John Halley Gotway
met_help at ucar.edu
>
> Tue Oct 29 17:14:40 2013: Request 63649 was acted upon.
> Transaction: Ticket created by jiang at gina.alaska.edu
> Queue: met_help
> Subject: compare 10m wind between wrf output and prepbufr data
> Owner: Nobody
> Requestors: jiang at gina.alaska.edu
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=63649 >
>
>
> Hi CS,
>
> I would like to compare 10 m wind between wrf output and prepbufr.
>
> the output from WRF go through postprocess by UPPV2.1, I uploaded
two data
> files and one point-stat configure file.
> ftp -p ftp.rap.ucar.edu/incoming/irap/met_help/zhu_data
>
> Would you please help me figure out why I can not produce output to
> compare
> wind?
> Perhaps there is no 10m wind in prepbufr file? I don't know how to
check
> if
> there is 10m wind in prepbufr files.
>
> Thank you in advance,
>
> Jiang
>
>
> **********************************************************
> Jiang Zhu, Ph.D.
> Geographic Information Network of Alaska
> University of Alaska Fairbanks
> Phone: 907-474-5689, EMAIL: jiang at gina.alaska.edu
> ***********************************************************
>
------------------------------------------------
Subject: compare 10m wind between wrf output and prepbufr data
From: Jiang Zhu
Time: Thu Oct 31 13:00:12 2013
Hi John,
Thank you very much for your help. I found you have been doing the
support
work for a long time, You do a very good job!
I want to compare my WRF model output with PREPBUFR observation data.
Do
you think use quality_mark_thresh=9 is reasonable? I am kind worry
about
the quality of observation data if I set the quality_mark_thresh=9.
But
like you already saw, if I use quality_mark_thresh=2, there is no U10,
V10
show up.
Could you give me your advice.
Jiang
On Thu, Oct 31, 2013 at 8:37 AM, John Halley Gotway via RT <
met_help at ucar.edu> wrote:
> Jiang,
>
> I see that you're having trouble producing matched pairs when
evaluating
> the U and V components of wind. I grabbed the sample data you sent
and
> reproduced the behavior you're seeing. Let me step you through the
steps
> I took to figure out what's going on...
>
> First, I used wgrib to inventory the forecast file you sent and
check for
> UGRD and VGRD at 10-meters. And it looks like it's in there...
>
> [johnhg at rambler]% wgrib WRFPRS_d01_2012-11-05_00_00_00 | egrep
> "UGRD|VGRD|WIND" | grep ":10 m"
>
>
275:5539450:d=12110418:UGRD:kpds5=33:kpds6=105:kpds7=10:TR=0:P1=6:P2=0:TimeU=1:10
> m above gnd:6hr fcst:NAve=0
>
>
276:5559028:d=12110418:VGRD:kpds5=34:kpds6=105:kpds7=10:TR=0:P1=6:P2=0:TimeU=1:10
> m above gnd:6hr fcst:NAve=0
>
> Second, I ran Point-Stat with a verbosity level of 3 (-v 3). For
each
> verification task, Point-Stat can dump out reason counts for why
> observations were or were not used for that task. Here's the output
for
> the UGRD and VGRD tasks...
>
> DEBUG 2: Processing VGRD/Z10 versus VGRD/Z10, for observation type
ADPSFC,
> over region FULL, for interpolation method UW_MEAN(1), using 0
pairs.
> DEBUG 3: Number of matched pairs = 0
> DEBUG 3: Observations processed = 204910
> DEBUG 3: Rejected: GRIB code = 170420
> DEBUG 3: Rejected: valid time = 0
> DEBUG 3: Rejected: bad obs value = 0
> DEBUG 3: Rejected: off the grid = 33793
> DEBUG 3: Rejected: level mismatch = 697
> DEBUG 3: Rejected: quality marker = 0
> DEBUG 3: Rejected: message type = 0
> DEBUG 3: Rejected: masking region = 0
> DEBUG 3: Rejected: bad fcst value = 0
> DEBUG 3: Rejected: duplicates = 0
> DEBUG 2: Processing VGRD/Z10 versus VGRD/Z10, for observation type
ADPSFC,
> over region FULL, for interpolation method UW_MEAN(25), using 0
pairs.
> DEBUG 3: Number of matched pairs = 0
> DEBUG 3: Observations processed = 204910
> DEBUG 3: Rejected: GRIB code = 170420
> DEBUG 3: Rejected: valid time = 0
> DEBUG 3: Rejected: bad obs value = 0
> DEBUG 3: Rejected: off the grid = 33793
> DEBUG 3: Rejected: level mismatch = 697
> DEBUG 3: Rejected: quality marker = 0
> DEBUG 3: Rejected: message type = 0
> DEBUG 3: Rejected: masking region = 0
> DEBUG 3: Rejected: bad fcst value = 0
> DEBUG 3: Rejected: duplicates = 0
>
> Looking at this, I don't see any obvious problems.
>
> Third, I ran the plot_point_obs utility to figure out what
observations of
> wind fall in your domain...
>
> [johnhg at rambler]%
/d1/johnhg/MET/MET_releases/METv4.1/bin/plot_point_obs
> 20121105.gdas1.t00z.prepbufr.nr_le.nc
> 20121105.gdas1.t00z.prepbufr.nr_le.ps -gc 33 -gc 34 -msg_typ ADPSFC
> -data_file WRFPRS_d01_2012-11-05_00_00_00
> DEBUG 2: Retrieving grid from file: WRFPRS_d01_2012-11-05_00_00_00
> DEBUG 1: Opening netCDF file: 20121105.gdas1.t00z.prepbufr.nr_le.nc
> DEBUG 2: Processing 204910 observations at 1339 locations.
> DEBUG 2: Observation GRIB codes: 33 34
> DEBUG 2: Observation message types: ADPSFC
> DEBUG 1: Creating postscript file:
20121105.gdas1.t00z.prepbufr.nr_le.ps
> DEBUG 2: Finished plotting 0 locations.
> DEBUG 2: Skipped 0 locations off the grid.
>
> Looking at this, there are no wind observations (GRIB code 33 and
34) for
> the ADPSFC message type over your domain for this time.
>
> FYI - if you plot all message types by removing "-msg_typ ADPSFC",
you'll
> see that there are 11 non-ADPSFC observation locations.
>
> [johnhg at rambler]%
/d1/johnhg/MET/MET_releases/METv4.1/bin/plot_point_obs
> 20121105.gdas1.t00z.prepbufr.nr_le.nc
> 20121105.gdas1.t00z.prepbufr.nr_le.ps -gc 33 -gc 34 -data_file
> WRFPRS_d01_2012-11-05_00_00_00
> DEBUG 2: Retrieving grid from file: WRFPRS_d01_2012-11-05_00_00_00
> DEBUG 1: Opening netCDF file: 20121105.gdas1.t00z.prepbufr.nr_le.nc
> DEBUG 2: Processing 204910 observations at 1339 locations.
> DEBUG 2: Observation GRIB codes: 33 34
> DEBUG 2: Observation message types: ALL
> DEBUG 1: Creating postscript file:
20121105.gdas1.t00z.prepbufr.nr_le.ps
> DEBUG 2: Finished plotting 11 locations.
> DEBUG 2: Skipped 692 locations off the grid.
>
> So there just aren't any observations of wind at this time with
message
> type ADPSFC over your domain. Why not?
>
> I suspect it's because of the quality control value used for surface
data
> in GDAS files. Please read the "Note" at the bottom of this page...
> http://www.dtcenter.org/met/users/downloads/observation_data.php
>
> Please try rerunning PB2NC with "quality_mark_thresh = 9" in the
config
> file. After you rerun PB2NC, I suspect you'll get matched pairs in
your
> Point-Stat output.
>
> Sorry to be so long-winded on this. I just wanted to step you
through the
> "debugging" process to assist you in doing it yourself in the
future.
>
> Hope that helps.
>
> Thanks,
> John Halley Gotway
> met_help at ucar.edu
>
> >
> > Tue Oct 29 17:14:40 2013: Request 63649 was acted upon.
> > Transaction: Ticket created by jiang at gina.alaska.edu
> > Queue: met_help
> > Subject: compare 10m wind between wrf output and prepbufr
data
> > Owner: Nobody
> > Requestors: jiang at gina.alaska.edu
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=63649 >
> >
> >
> > Hi CS,
> >
> > I would like to compare 10 m wind between wrf output and prepbufr.
> >
> > the output from WRF go through postprocess by UPPV2.1, I uploaded
two
> data
> > files and one point-stat configure file.
> > ftp -p ftp.rap.ucar.edu/incoming/irap/met_help/zhu_data
> >
> > Would you please help me figure out why I can not produce output
to
> > compare
> > wind?
> > Perhaps there is no 10m wind in prepbufr file? I don't know how to
check
> > if
> > there is 10m wind in prepbufr files.
> >
> > Thank you in advance,
> >
> > Jiang
> >
> >
> > **********************************************************
> > Jiang Zhu, Ph.D.
> > Geographic Information Network of Alaska
> > University of Alaska Fairbanks
> > Phone: 907-474-5689, EMAIL: jiang at gina.alaska.edu
> > ***********************************************************
> >
>
>
>
>
--
**********************************************************
Jiang Zhu, Ph.D.
Geographic Information Network of Alaska
University of Alaska Fairbanks
Phone: 907-474-5689, EMAIL: jiang at gina.alaska.edu
***********************************************************
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #63649] compare 10m wind between wrf output and prepbufr data
From: John Halley Gotway
Time: Thu Oct 31 13:33:20 2013
Jiang,
I understand your hesitation in using the higher quality marker value.
It's really unfortunate that NCEP chose to use the quality marker
threshold as a way to restrict which observations flow into
the global data assimilation system. I don't have a simple answer for
you, but here are a couple of options...
They do not do this for the NDAS (north-american data assimilation
system). So you could try using an NDAS file and comparing it to the
GDAS data.
But if you are using the GDAS data, you could set the
"quality_marker_thresh" to a value of 9 in the PB2NC config file.
Then when you run Point-Stat, you could explicitly list the quality
control
strings you'd like to use. If you'd like to use values of 0 - 2 and 9
(but skip 3 - 8), you'd set it like this:
obs_quality = [ "0", "1", "2", "9" ];
In case you're wondering why these are strings rather than integers,
while qc flags are integers in PREPBUFR, other data sources use
strings. So we just write out the PREPBUFR integers as strings and
use them that way.
Hope that helps.
Thanks,
John
On 10/31/2013 01:00 PM, Jiang Zhu via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=63649 >
>
> Hi John,
>
> Thank you very much for your help. I found you have been doing the
support
> work for a long time, You do a very good job!
>
> I want to compare my WRF model output with PREPBUFR observation
data. Do
> you think use quality_mark_thresh=9 is reasonable? I am kind worry
about
> the quality of observation data if I set the quality_mark_thresh=9.
But
> like you already saw, if I use quality_mark_thresh=2, there is no
U10, V10
> show up.
>
> Could you give me your advice.
>
> Jiang
>
>
>
>
>
> On Thu, Oct 31, 2013 at 8:37 AM, John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
>> Jiang,
>>
>> I see that you're having trouble producing matched pairs when
evaluating
>> the U and V components of wind. I grabbed the sample data you sent
and
>> reproduced the behavior you're seeing. Let me step you through the
steps
>> I took to figure out what's going on...
>>
>> First, I used wgrib to inventory the forecast file you sent and
check for
>> UGRD and VGRD at 10-meters. And it looks like it's in there...
>>
>> [johnhg at rambler]% wgrib WRFPRS_d01_2012-11-05_00_00_00 | egrep
>> "UGRD|VGRD|WIND" | grep ":10 m"
>>
>>
275:5539450:d=12110418:UGRD:kpds5=33:kpds6=105:kpds7=10:TR=0:P1=6:P2=0:TimeU=1:10
>> m above gnd:6hr fcst:NAve=0
>>
>>
276:5559028:d=12110418:VGRD:kpds5=34:kpds6=105:kpds7=10:TR=0:P1=6:P2=0:TimeU=1:10
>> m above gnd:6hr fcst:NAve=0
>>
>> Second, I ran Point-Stat with a verbosity level of 3 (-v 3). For
each
>> verification task, Point-Stat can dump out reason counts for why
>> observations were or were not used for that task. Here's the
output for
>> the UGRD and VGRD tasks...
>>
>> DEBUG 2: Processing VGRD/Z10 versus VGRD/Z10, for observation type
ADPSFC,
>> over region FULL, for interpolation method UW_MEAN(1), using 0
pairs.
>> DEBUG 3: Number of matched pairs = 0
>> DEBUG 3: Observations processed = 204910
>> DEBUG 3: Rejected: GRIB code = 170420
>> DEBUG 3: Rejected: valid time = 0
>> DEBUG 3: Rejected: bad obs value = 0
>> DEBUG 3: Rejected: off the grid = 33793
>> DEBUG 3: Rejected: level mismatch = 697
>> DEBUG 3: Rejected: quality marker = 0
>> DEBUG 3: Rejected: message type = 0
>> DEBUG 3: Rejected: masking region = 0
>> DEBUG 3: Rejected: bad fcst value = 0
>> DEBUG 3: Rejected: duplicates = 0
>> DEBUG 2: Processing VGRD/Z10 versus VGRD/Z10, for observation type
ADPSFC,
>> over region FULL, for interpolation method UW_MEAN(25), using 0
pairs.
>> DEBUG 3: Number of matched pairs = 0
>> DEBUG 3: Observations processed = 204910
>> DEBUG 3: Rejected: GRIB code = 170420
>> DEBUG 3: Rejected: valid time = 0
>> DEBUG 3: Rejected: bad obs value = 0
>> DEBUG 3: Rejected: off the grid = 33793
>> DEBUG 3: Rejected: level mismatch = 697
>> DEBUG 3: Rejected: quality marker = 0
>> DEBUG 3: Rejected: message type = 0
>> DEBUG 3: Rejected: masking region = 0
>> DEBUG 3: Rejected: bad fcst value = 0
>> DEBUG 3: Rejected: duplicates = 0
>>
>> Looking at this, I don't see any obvious problems.
>>
>> Third, I ran the plot_point_obs utility to figure out what
observations of
>> wind fall in your domain...
>>
>> [johnhg at rambler]%
/d1/johnhg/MET/MET_releases/METv4.1/bin/plot_point_obs
>> 20121105.gdas1.t00z.prepbufr.nr_le.nc
>> 20121105.gdas1.t00z.prepbufr.nr_le.ps -gc 33 -gc 34 -msg_typ ADPSFC
>> -data_file WRFPRS_d01_2012-11-05_00_00_00
>> DEBUG 2: Retrieving grid from file: WRFPRS_d01_2012-11-05_00_00_00
>> DEBUG 1: Opening netCDF file: 20121105.gdas1.t00z.prepbufr.nr_le.nc
>> DEBUG 2: Processing 204910 observations at 1339 locations.
>> DEBUG 2: Observation GRIB codes: 33 34
>> DEBUG 2: Observation message types: ADPSFC
>> DEBUG 1: Creating postscript file:
20121105.gdas1.t00z.prepbufr.nr_le.ps
>> DEBUG 2: Finished plotting 0 locations.
>> DEBUG 2: Skipped 0 locations off the grid.
>>
>> Looking at this, there are no wind observations (GRIB code 33 and
34) for
>> the ADPSFC message type over your domain for this time.
>>
>> FYI - if you plot all message types by removing "-msg_typ ADPSFC",
you'll
>> see that there are 11 non-ADPSFC observation locations.
>>
>> [johnhg at rambler]%
/d1/johnhg/MET/MET_releases/METv4.1/bin/plot_point_obs
>> 20121105.gdas1.t00z.prepbufr.nr_le.nc
>> 20121105.gdas1.t00z.prepbufr.nr_le.ps -gc 33 -gc 34 -data_file
>> WRFPRS_d01_2012-11-05_00_00_00
>> DEBUG 2: Retrieving grid from file: WRFPRS_d01_2012-11-05_00_00_00
>> DEBUG 1: Opening netCDF file: 20121105.gdas1.t00z.prepbufr.nr_le.nc
>> DEBUG 2: Processing 204910 observations at 1339 locations.
>> DEBUG 2: Observation GRIB codes: 33 34
>> DEBUG 2: Observation message types: ALL
>> DEBUG 1: Creating postscript file:
20121105.gdas1.t00z.prepbufr.nr_le.ps
>> DEBUG 2: Finished plotting 11 locations.
>> DEBUG 2: Skipped 692 locations off the grid.
>>
>> So there just aren't any observations of wind at this time with
message
>> type ADPSFC over your domain. Why not?
>>
>> I suspect it's because of the quality control value used for
surface data
>> in GDAS files. Please read the "Note" at the bottom of this
page...
>>
http://www.dtcenter.org/met/users/downloads/observation_data.php
>>
>> Please try rerunning PB2NC with "quality_mark_thresh = 9" in the
config
>> file. After you rerun PB2NC, I suspect you'll get matched pairs in
your
>> Point-Stat output.
>>
>> Sorry to be so long-winded on this. I just wanted to step you
through the
>> "debugging" process to assist you in doing it yourself in the
future.
>>
>> Hope that helps.
>>
>> Thanks,
>> John Halley Gotway
>> met_help at ucar.edu
>>
>>>
>>> Tue Oct 29 17:14:40 2013: Request 63649 was acted upon.
>>> Transaction: Ticket created by jiang at gina.alaska.edu
>>> Queue: met_help
>>> Subject: compare 10m wind between wrf output and prepbufr
data
>>> Owner: Nobody
>>> Requestors: jiang at gina.alaska.edu
>>> Status: new
>>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=63649 >
>>>
>>>
>>> Hi CS,
>>>
>>> I would like to compare 10 m wind between wrf output and prepbufr.
>>>
>>> the output from WRF go through postprocess by UPPV2.1, I uploaded
two
>> data
>>> files and one point-stat configure file.
>>> ftp -p ftp.rap.ucar.edu/incoming/irap/met_help/zhu_data
>>>
>>> Would you please help me figure out why I can not produce output
to
>>> compare
>>> wind?
>>> Perhaps there is no 10m wind in prepbufr file? I don't know how to
check
>>> if
>>> there is 10m wind in prepbufr files.
>>>
>>> Thank you in advance,
>>>
>>> Jiang
>>>
>>>
>>> **********************************************************
>>> Jiang Zhu, Ph.D.
>>> Geographic Information Network of Alaska
>>> University of Alaska Fairbanks
>>> Phone: 907-474-5689, EMAIL: jiang at gina.alaska.edu
>>> ***********************************************************
>>>
>>
>>
>>
>>
>
>
------------------------------------------------
Subject: compare 10m wind between wrf output and prepbufr data
From: Jiang Zhu
Time: Thu Oct 31 14:00:59 2013
Hi John,
It is a god idea. But where should I put obs_quality = [ "0", "1",
"2",
"9" ]; do I need put it in configure file for point-stat or do I need
give
it as a command option of point-stat?
Thank you,
Jiang
On Thu, Oct 31, 2013 at 11:33 AM, John Halley Gotway via RT <
met_help at ucar.edu> wrote:
> Jiang,
>
> I understand your hesitation in using the higher quality marker
value.
> It's really unfortunate that NCEP chose to use the quality marker
> threshold as a way to restrict which observations flow into
> the global data assimilation system. I don't have a simple answer
for
> you, but here are a couple of options...
>
> They do not do this for the NDAS (north-american data assimilation
> system). So you could try using an NDAS file and comparing it to
the GDAS
> data.
>
> But if you are using the GDAS data, you could set the
> "quality_marker_thresh" to a value of 9 in the PB2NC config file.
Then
> when you run Point-Stat, you could explicitly list the quality
control
> strings you'd like to use. If you'd like to use values of 0 - 2 and
9
> (but skip 3 - 8), you'd set it like this:
> obs_quality = [ "0", "1", "2", "9" ];
>
> In case you're wondering why these are strings rather than integers,
while
> qc flags are integers in PREPBUFR, other data sources use strings.
So we
> just write out the PREPBUFR integers as strings and
> use them that way.
>
> Hope that helps.
>
> Thanks,
> John
>
> On 10/31/2013 01:00 PM, Jiang Zhu via RT wrote:
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=63649 >
> >
> > Hi John,
> >
> > Thank you very much for your help. I found you have been doing the
> support
> > work for a long time, You do a very good job!
> >
> > I want to compare my WRF model output with PREPBUFR observation
data. Do
> > you think use quality_mark_thresh=9 is reasonable? I am kind worry
about
> > the quality of observation data if I set the
quality_mark_thresh=9. But
> > like you already saw, if I use quality_mark_thresh=2, there is no
U10,
> V10
> > show up.
> >
> > Could you give me your advice.
> >
> > Jiang
> >
> >
> >
> >
> >
> > On Thu, Oct 31, 2013 at 8:37 AM, John Halley Gotway via RT <
> > met_help at ucar.edu> wrote:
> >
> >> Jiang,
> >>
> >> I see that you're having trouble producing matched pairs when
evaluating
> >> the U and V components of wind. I grabbed the sample data you
sent and
> >> reproduced the behavior you're seeing. Let me step you through
the
> steps
> >> I took to figure out what's going on...
> >>
> >> First, I used wgrib to inventory the forecast file you sent and
check
> for
> >> UGRD and VGRD at 10-meters. And it looks like it's in there...
> >>
> >> [johnhg at rambler]% wgrib WRFPRS_d01_2012-11-05_00_00_00 | egrep
> >> "UGRD|VGRD|WIND" | grep ":10 m"
> >>
> >>
>
275:5539450:d=12110418:UGRD:kpds5=33:kpds6=105:kpds7=10:TR=0:P1=6:P2=0:TimeU=1:10
> >> m above gnd:6hr fcst:NAve=0
> >>
> >>
>
276:5559028:d=12110418:VGRD:kpds5=34:kpds6=105:kpds7=10:TR=0:P1=6:P2=0:TimeU=1:10
> >> m above gnd:6hr fcst:NAve=0
> >>
> >> Second, I ran Point-Stat with a verbosity level of 3 (-v 3). For
each
> >> verification task, Point-Stat can dump out reason counts for why
> >> observations were or were not used for that task. Here's the
output for
> >> the UGRD and VGRD tasks...
> >>
> >> DEBUG 2: Processing VGRD/Z10 versus VGRD/Z10, for observation
type
> ADPSFC,
> >> over region FULL, for interpolation method UW_MEAN(1), using 0
pairs.
> >> DEBUG 3: Number of matched pairs = 0
> >> DEBUG 3: Observations processed = 204910
> >> DEBUG 3: Rejected: GRIB code = 170420
> >> DEBUG 3: Rejected: valid time = 0
> >> DEBUG 3: Rejected: bad obs value = 0
> >> DEBUG 3: Rejected: off the grid = 33793
> >> DEBUG 3: Rejected: level mismatch = 697
> >> DEBUG 3: Rejected: quality marker = 0
> >> DEBUG 3: Rejected: message type = 0
> >> DEBUG 3: Rejected: masking region = 0
> >> DEBUG 3: Rejected: bad fcst value = 0
> >> DEBUG 3: Rejected: duplicates = 0
> >> DEBUG 2: Processing VGRD/Z10 versus VGRD/Z10, for observation
type
> ADPSFC,
> >> over region FULL, for interpolation method UW_MEAN(25), using 0
pairs.
> >> DEBUG 3: Number of matched pairs = 0
> >> DEBUG 3: Observations processed = 204910
> >> DEBUG 3: Rejected: GRIB code = 170420
> >> DEBUG 3: Rejected: valid time = 0
> >> DEBUG 3: Rejected: bad obs value = 0
> >> DEBUG 3: Rejected: off the grid = 33793
> >> DEBUG 3: Rejected: level mismatch = 697
> >> DEBUG 3: Rejected: quality marker = 0
> >> DEBUG 3: Rejected: message type = 0
> >> DEBUG 3: Rejected: masking region = 0
> >> DEBUG 3: Rejected: bad fcst value = 0
> >> DEBUG 3: Rejected: duplicates = 0
> >>
> >> Looking at this, I don't see any obvious problems.
> >>
> >> Third, I ran the plot_point_obs utility to figure out what
observations
> of
> >> wind fall in your domain...
> >>
> >> [johnhg at rambler]%
> /d1/johnhg/MET/MET_releases/METv4.1/bin/plot_point_obs
> >> 20121105.gdas1.t00z.prepbufr.nr_le.nc
> >> 20121105.gdas1.t00z.prepbufr.nr_le.ps -gc 33 -gc 34 -msg_typ
ADPSFC
> >> -data_file WRFPRS_d01_2012-11-05_00_00_00
> >> DEBUG 2: Retrieving grid from file: WRFPRS_d01_2012-11-
05_00_00_00
> >> DEBUG 1: Opening netCDF file:
20121105.gdas1.t00z.prepbufr.nr_le.nc
> >> DEBUG 2: Processing 204910 observations at 1339 locations.
> >> DEBUG 2: Observation GRIB codes: 33 34
> >> DEBUG 2: Observation message types: ADPSFC
> >> DEBUG 1: Creating postscript file:
> 20121105.gdas1.t00z.prepbufr.nr_le.ps
> >> DEBUG 2: Finished plotting 0 locations.
> >> DEBUG 2: Skipped 0 locations off the grid.
> >>
> >> Looking at this, there are no wind observations (GRIB code 33 and
34)
> for
> >> the ADPSFC message type over your domain for this time.
> >>
> >> FYI - if you plot all message types by removing "-msg_typ
ADPSFC",
> you'll
> >> see that there are 11 non-ADPSFC observation locations.
> >>
> >> [johnhg at rambler]%
> /d1/johnhg/MET/MET_releases/METv4.1/bin/plot_point_obs
> >> 20121105.gdas1.t00z.prepbufr.nr_le.nc
> >> 20121105.gdas1.t00z.prepbufr.nr_le.ps -gc 33 -gc 34 -data_file
> >> WRFPRS_d01_2012-11-05_00_00_00
> >> DEBUG 2: Retrieving grid from file: WRFPRS_d01_2012-11-
05_00_00_00
> >> DEBUG 1: Opening netCDF file:
20121105.gdas1.t00z.prepbufr.nr_le.nc
> >> DEBUG 2: Processing 204910 observations at 1339 locations.
> >> DEBUG 2: Observation GRIB codes: 33 34
> >> DEBUG 2: Observation message types: ALL
> >> DEBUG 1: Creating postscript file:
> 20121105.gdas1.t00z.prepbufr.nr_le.ps
> >> DEBUG 2: Finished plotting 11 locations.
> >> DEBUG 2: Skipped 692 locations off the grid.
> >>
> >> So there just aren't any observations of wind at this time with
message
> >> type ADPSFC over your domain. Why not?
> >>
> >> I suspect it's because of the quality control value used for
surface
> data
> >> in GDAS files. Please read the "Note" at the bottom of this
page...
> >>
http://www.dtcenter.org/met/users/downloads/observation_data.php
> >>
> >> Please try rerunning PB2NC with "quality_mark_thresh = 9" in the
config
> >> file. After you rerun PB2NC, I suspect you'll get matched pairs
in your
> >> Point-Stat output.
> >>
> >> Sorry to be so long-winded on this. I just wanted to step you
through
> the
> >> "debugging" process to assist you in doing it yourself in the
future.
> >>
> >> Hope that helps.
> >>
> >> Thanks,
> >> John Halley Gotway
> >> met_help at ucar.edu
> >>
> >>>
> >>> Tue Oct 29 17:14:40 2013: Request 63649 was acted upon.
> >>> Transaction: Ticket created by jiang at gina.alaska.edu
> >>> Queue: met_help
> >>> Subject: compare 10m wind between wrf output and prepbufr
data
> >>> Owner: Nobody
> >>> Requestors: jiang at gina.alaska.edu
> >>> Status: new
> >>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=63649>
> >>>
> >>>
> >>> Hi CS,
> >>>
> >>> I would like to compare 10 m wind between wrf output and
prepbufr.
> >>>
> >>> the output from WRF go through postprocess by UPPV2.1, I
uploaded two
> >> data
> >>> files and one point-stat configure file.
> >>> ftp -p ftp.rap.ucar.edu/incoming/irap/met_help/zhu_data
> >>>
> >>> Would you please help me figure out why I can not produce output
to
> >>> compare
> >>> wind?
> >>> Perhaps there is no 10m wind in prepbufr file? I don't know how
to
> check
> >>> if
> >>> there is 10m wind in prepbufr files.
> >>>
> >>> Thank you in advance,
> >>>
> >>> Jiang
> >>>
> >>>
> >>> **********************************************************
> >>> Jiang Zhu, Ph.D.
> >>> Geographic Information Network of Alaska
> >>> University of Alaska Fairbanks
> >>> Phone: 907-474-5689, EMAIL: jiang at gina.alaska.edu
> >>> ***********************************************************
> >>>
> >>
> >>
> >>
> >>
> >
> >
>
>
--
**********************************************************
Jiang Zhu, Ph.D.
Geographic Information Network of Alaska
University of Alaska Fairbanks
Phone: 907-474-5689, EMAIL: jiang at gina.alaska.edu
***********************************************************
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #63649] compare 10m wind between wrf output and prepbufr data
From: John Halley Gotway
Time: Fri Nov 01 09:21:33 2013
Jiang,
It's in the config file for Point-Stat. "obs_quality" is one of the
settings down near the bottom of the config file:
http://www.dtcenter.org/met/users/support/online_tutorial/METv4.1/config/PointStatConfig_default
Thanks,
John
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=63649 >
>
> Hi John,
>
> It is a god idea. But where should I put obs_quality = [ "0", "1",
"2",
> "9" ]; do I need put it in configure file for point-stat or do I
need
> give
> it as a command option of point-stat?
>
>
> Thank you,
>
> Jiang
>
> On Thu, Oct 31, 2013 at 11:33 AM, John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
>> Jiang,
>>
>> I understand your hesitation in using the higher quality marker
value.
>> It's really unfortunate that NCEP chose to use the quality marker
>> threshold as a way to restrict which observations flow into
>> the global data assimilation system. I don't have a simple answer
for
>> you, but here are a couple of options...
>>
>> They do not do this for the NDAS (north-american data assimilation
>> system). So you could try using an NDAS file and comparing it to
the
>> GDAS
>> data.
>>
>> But if you are using the GDAS data, you could set the
>> "quality_marker_thresh" to a value of 9 in the PB2NC config file.
Then
>> when you run Point-Stat, you could explicitly list the quality
control
>> strings you'd like to use. If you'd like to use values of 0 - 2
and 9
>> (but skip 3 - 8), you'd set it like this:
>> obs_quality = [ "0", "1", "2", "9" ];
>>
>> In case you're wondering why these are strings rather than
integers,
>> while
>> qc flags are integers in PREPBUFR, other data sources use strings.
So
>> we
>> just write out the PREPBUFR integers as strings and
>> use them that way.
>>
>> Hope that helps.
>>
>> Thanks,
>> John
>>
>> On 10/31/2013 01:00 PM, Jiang Zhu via RT wrote:
>> >
>> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=63649 >
>> >
>> > Hi John,
>> >
>> > Thank you very much for your help. I found you have been doing
the
>> support
>> > work for a long time, You do a very good job!
>> >
>> > I want to compare my WRF model output with PREPBUFR observation
data.
>> Do
>> > you think use quality_mark_thresh=9 is reasonable? I am kind
worry
>> about
>> > the quality of observation data if I set the
quality_mark_thresh=9.
>> But
>> > like you already saw, if I use quality_mark_thresh=2, there is no
U10,
>> V10
>> > show up.
>> >
>> > Could you give me your advice.
>> >
>> > Jiang
>> >
>> >
>> >
>> >
>> >
>> > On Thu, Oct 31, 2013 at 8:37 AM, John Halley Gotway via RT <
>> > met_help at ucar.edu> wrote:
>> >
>> >> Jiang,
>> >>
>> >> I see that you're having trouble producing matched pairs when
>> evaluating
>> >> the U and V components of wind. I grabbed the sample data you
sent
>> and
>> >> reproduced the behavior you're seeing. Let me step you through
the
>> steps
>> >> I took to figure out what's going on...
>> >>
>> >> First, I used wgrib to inventory the forecast file you sent and
check
>> for
>> >> UGRD and VGRD at 10-meters. And it looks like it's in there...
>> >>
>> >> [johnhg at rambler]% wgrib WRFPRS_d01_2012-11-05_00_00_00 | egrep
>> >> "UGRD|VGRD|WIND" | grep ":10 m"
>> >>
>> >>
>>
275:5539450:d=12110418:UGRD:kpds5=33:kpds6=105:kpds7=10:TR=0:P1=6:P2=0:TimeU=1:10
>> >> m above gnd:6hr fcst:NAve=0
>> >>
>> >>
>>
276:5559028:d=12110418:VGRD:kpds5=34:kpds6=105:kpds7=10:TR=0:P1=6:P2=0:TimeU=1:10
>> >> m above gnd:6hr fcst:NAve=0
>> >>
>> >> Second, I ran Point-Stat with a verbosity level of 3 (-v 3).
For
>> each
>> >> verification task, Point-Stat can dump out reason counts for why
>> >> observations were or were not used for that task. Here's the
output
>> for
>> >> the UGRD and VGRD tasks...
>> >>
>> >> DEBUG 2: Processing VGRD/Z10 versus VGRD/Z10, for observation
type
>> ADPSFC,
>> >> over region FULL, for interpolation method UW_MEAN(1), using 0
pairs.
>> >> DEBUG 3: Number of matched pairs = 0
>> >> DEBUG 3: Observations processed = 204910
>> >> DEBUG 3: Rejected: GRIB code = 170420
>> >> DEBUG 3: Rejected: valid time = 0
>> >> DEBUG 3: Rejected: bad obs value = 0
>> >> DEBUG 3: Rejected: off the grid = 33793
>> >> DEBUG 3: Rejected: level mismatch = 697
>> >> DEBUG 3: Rejected: quality marker = 0
>> >> DEBUG 3: Rejected: message type = 0
>> >> DEBUG 3: Rejected: masking region = 0
>> >> DEBUG 3: Rejected: bad fcst value = 0
>> >> DEBUG 3: Rejected: duplicates = 0
>> >> DEBUG 2: Processing VGRD/Z10 versus VGRD/Z10, for observation
type
>> ADPSFC,
>> >> over region FULL, for interpolation method UW_MEAN(25), using 0
>> pairs.
>> >> DEBUG 3: Number of matched pairs = 0
>> >> DEBUG 3: Observations processed = 204910
>> >> DEBUG 3: Rejected: GRIB code = 170420
>> >> DEBUG 3: Rejected: valid time = 0
>> >> DEBUG 3: Rejected: bad obs value = 0
>> >> DEBUG 3: Rejected: off the grid = 33793
>> >> DEBUG 3: Rejected: level mismatch = 697
>> >> DEBUG 3: Rejected: quality marker = 0
>> >> DEBUG 3: Rejected: message type = 0
>> >> DEBUG 3: Rejected: masking region = 0
>> >> DEBUG 3: Rejected: bad fcst value = 0
>> >> DEBUG 3: Rejected: duplicates = 0
>> >>
>> >> Looking at this, I don't see any obvious problems.
>> >>
>> >> Third, I ran the plot_point_obs utility to figure out what
>> observations
>> of
>> >> wind fall in your domain...
>> >>
>> >> [johnhg at rambler]%
>> /d1/johnhg/MET/MET_releases/METv4.1/bin/plot_point_obs
>> >> 20121105.gdas1.t00z.prepbufr.nr_le.nc
>> >> 20121105.gdas1.t00z.prepbufr.nr_le.ps -gc 33 -gc 34 -msg_typ
ADPSFC
>> >> -data_file WRFPRS_d01_2012-11-05_00_00_00
>> >> DEBUG 2: Retrieving grid from file: WRFPRS_d01_2012-11-
05_00_00_00
>> >> DEBUG 1: Opening netCDF file:
20121105.gdas1.t00z.prepbufr.nr_le.nc
>> >> DEBUG 2: Processing 204910 observations at 1339 locations.
>> >> DEBUG 2: Observation GRIB codes: 33 34
>> >> DEBUG 2: Observation message types: ADPSFC
>> >> DEBUG 1: Creating postscript file:
>> 20121105.gdas1.t00z.prepbufr.nr_le.ps
>> >> DEBUG 2: Finished plotting 0 locations.
>> >> DEBUG 2: Skipped 0 locations off the grid.
>> >>
>> >> Looking at this, there are no wind observations (GRIB code 33
and 34)
>> for
>> >> the ADPSFC message type over your domain for this time.
>> >>
>> >> FYI - if you plot all message types by removing "-msg_typ
ADPSFC",
>> you'll
>> >> see that there are 11 non-ADPSFC observation locations.
>> >>
>> >> [johnhg at rambler]%
>> /d1/johnhg/MET/MET_releases/METv4.1/bin/plot_point_obs
>> >> 20121105.gdas1.t00z.prepbufr.nr_le.nc
>> >> 20121105.gdas1.t00z.prepbufr.nr_le.ps -gc 33 -gc 34 -data_file
>> >> WRFPRS_d01_2012-11-05_00_00_00
>> >> DEBUG 2: Retrieving grid from file: WRFPRS_d01_2012-11-
05_00_00_00
>> >> DEBUG 1: Opening netCDF file:
20121105.gdas1.t00z.prepbufr.nr_le.nc
>> >> DEBUG 2: Processing 204910 observations at 1339 locations.
>> >> DEBUG 2: Observation GRIB codes: 33 34
>> >> DEBUG 2: Observation message types: ALL
>> >> DEBUG 1: Creating postscript file:
>> 20121105.gdas1.t00z.prepbufr.nr_le.ps
>> >> DEBUG 2: Finished plotting 11 locations.
>> >> DEBUG 2: Skipped 692 locations off the grid.
>> >>
>> >> So there just aren't any observations of wind at this time with
>> message
>> >> type ADPSFC over your domain. Why not?
>> >>
>> >> I suspect it's because of the quality control value used for
surface
>> data
>> >> in GDAS files. Please read the "Note" at the bottom of this
page...
>> >>
http://www.dtcenter.org/met/users/downloads/observation_data.php
>> >>
>> >> Please try rerunning PB2NC with "quality_mark_thresh = 9" in the
>> config
>> >> file. After you rerun PB2NC, I suspect you'll get matched pairs
in
>> your
>> >> Point-Stat output.
>> >>
>> >> Sorry to be so long-winded on this. I just wanted to step you
>> through
>> the
>> >> "debugging" process to assist you in doing it yourself in the
future.
>> >>
>> >> Hope that helps.
>> >>
>> >> Thanks,
>> >> John Halley Gotway
>> >> met_help at ucar.edu
>> >>
>> >>>
>> >>> Tue Oct 29 17:14:40 2013: Request 63649 was acted upon.
>> >>> Transaction: Ticket created by jiang at gina.alaska.edu
>> >>> Queue: met_help
>> >>> Subject: compare 10m wind between wrf output and prepbufr
data
>> >>> Owner: Nobody
>> >>> Requestors: jiang at gina.alaska.edu
>> >>> Status: new
>> >>> Ticket <URL:
>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=63649>
>> >>>
>> >>>
>> >>> Hi CS,
>> >>>
>> >>> I would like to compare 10 m wind between wrf output and
prepbufr.
>> >>>
>> >>> the output from WRF go through postprocess by UPPV2.1, I
uploaded
>> two
>> >> data
>> >>> files and one point-stat configure file.
>> >>> ftp -p ftp.rap.ucar.edu/incoming/irap/met_help/zhu_data
>> >>>
>> >>> Would you please help me figure out why I can not produce
output to
>> >>> compare
>> >>> wind?
>> >>> Perhaps there is no 10m wind in prepbufr file? I don't know how
to
>> check
>> >>> if
>> >>> there is 10m wind in prepbufr files.
>> >>>
>> >>> Thank you in advance,
>> >>>
>> >>> Jiang
>> >>>
>> >>>
>> >>> **********************************************************
>> >>> Jiang Zhu, Ph.D.
>> >>> Geographic Information Network of Alaska
>> >>> University of Alaska Fairbanks
>> >>> Phone: 907-474-5689, EMAIL: jiang at gina.alaska.edu
>> >>> ***********************************************************
>> >>>
>> >>
>> >>
>> >>
>> >>
>> >
>> >
>>
>>
>
>
> --
> **********************************************************
> Jiang Zhu, Ph.D.
> Geographic Information Network of Alaska
> University of Alaska Fairbanks
> Phone: 907-474-5689, EMAIL: jiang at gina.alaska.edu
> ***********************************************************
>
------------------------------------------------
Subject: compare 10m wind between wrf output and prepbufr data
From: Jiang Zhu
Time: Fri Nov 01 11:04:59 2013
Hi John,
Thank you very much! You are such a excellent helper!
Have a good weekend!
Jiang
On Fri, Nov 1, 2013 at 7:21 AM, John Halley Gotway via RT
<met_help at ucar.edu
> wrote:
> Jiang,
>
> It's in the config file for Point-Stat. "obs_quality" is one of the
> settings down near the bottom of the config file:
>
>
http://www.dtcenter.org/met/users/support/online_tutorial/METv4.1/config/PointStatConfig_default
>
> Thanks,
> John
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=63649 >
> >
> > Hi John,
> >
> > It is a god idea. But where should I put obs_quality = [ "0",
"1", "2",
> > "9" ]; do I need put it in configure file for point-stat or do I
need
> > give
> > it as a command option of point-stat?
> >
> >
> > Thank you,
> >
> > Jiang
> >
> > On Thu, Oct 31, 2013 at 11:33 AM, John Halley Gotway via RT <
> > met_help at ucar.edu> wrote:
> >
> >> Jiang,
> >>
> >> I understand your hesitation in using the higher quality marker
value.
> >> It's really unfortunate that NCEP chose to use the quality
marker
> >> threshold as a way to restrict which observations flow into
> >> the global data assimilation system. I don't have a simple
answer for
> >> you, but here are a couple of options...
> >>
> >> They do not do this for the NDAS (north-american data
assimilation
> >> system). So you could try using an NDAS file and comparing it to
the
> >> GDAS
> >> data.
> >>
> >> But if you are using the GDAS data, you could set the
> >> "quality_marker_thresh" to a value of 9 in the PB2NC config file.
Then
> >> when you run Point-Stat, you could explicitly list the quality
control
> >> strings you'd like to use. If you'd like to use values of 0 - 2
and 9
> >> (but skip 3 - 8), you'd set it like this:
> >> obs_quality = [ "0", "1", "2", "9" ];
> >>
> >> In case you're wondering why these are strings rather than
integers,
> >> while
> >> qc flags are integers in PREPBUFR, other data sources use
strings. So
> >> we
> >> just write out the PREPBUFR integers as strings and
> >> use them that way.
> >>
> >> Hope that helps.
> >>
> >> Thanks,
> >> John
> >>
> >> On 10/31/2013 01:00 PM, Jiang Zhu via RT wrote:
> >> >
> >> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=63649 >
> >> >
> >> > Hi John,
> >> >
> >> > Thank you very much for your help. I found you have been doing
the
> >> support
> >> > work for a long time, You do a very good job!
> >> >
> >> > I want to compare my WRF model output with PREPBUFR observation
data.
> >> Do
> >> > you think use quality_mark_thresh=9 is reasonable? I am kind
worry
> >> about
> >> > the quality of observation data if I set the
quality_mark_thresh=9.
> >> But
> >> > like you already saw, if I use quality_mark_thresh=2, there is
no U10,
> >> V10
> >> > show up.
> >> >
> >> > Could you give me your advice.
> >> >
> >> > Jiang
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > On Thu, Oct 31, 2013 at 8:37 AM, John Halley Gotway via RT <
> >> > met_help at ucar.edu> wrote:
> >> >
> >> >> Jiang,
> >> >>
> >> >> I see that you're having trouble producing matched pairs when
> >> evaluating
> >> >> the U and V components of wind. I grabbed the sample data you
sent
> >> and
> >> >> reproduced the behavior you're seeing. Let me step you
through the
> >> steps
> >> >> I took to figure out what's going on...
> >> >>
> >> >> First, I used wgrib to inventory the forecast file you sent
and check
> >> for
> >> >> UGRD and VGRD at 10-meters. And it looks like it's in
there...
> >> >>
> >> >> [johnhg at rambler]% wgrib WRFPRS_d01_2012-11-05_00_00_00 | egrep
> >> >> "UGRD|VGRD|WIND" | grep ":10 m"
> >> >>
> >> >>
> >>
>
275:5539450:d=12110418:UGRD:kpds5=33:kpds6=105:kpds7=10:TR=0:P1=6:P2=0:TimeU=1:10
> >> >> m above gnd:6hr fcst:NAve=0
> >> >>
> >> >>
> >>
>
276:5559028:d=12110418:VGRD:kpds5=34:kpds6=105:kpds7=10:TR=0:P1=6:P2=0:TimeU=1:10
> >> >> m above gnd:6hr fcst:NAve=0
> >> >>
> >> >> Second, I ran Point-Stat with a verbosity level of 3 (-v 3).
For
> >> each
> >> >> verification task, Point-Stat can dump out reason counts for
why
> >> >> observations were or were not used for that task. Here's the
output
> >> for
> >> >> the UGRD and VGRD tasks...
> >> >>
> >> >> DEBUG 2: Processing VGRD/Z10 versus VGRD/Z10, for observation
type
> >> ADPSFC,
> >> >> over region FULL, for interpolation method UW_MEAN(1), using 0
pairs.
> >> >> DEBUG 3: Number of matched pairs = 0
> >> >> DEBUG 3: Observations processed = 204910
> >> >> DEBUG 3: Rejected: GRIB code = 170420
> >> >> DEBUG 3: Rejected: valid time = 0
> >> >> DEBUG 3: Rejected: bad obs value = 0
> >> >> DEBUG 3: Rejected: off the grid = 33793
> >> >> DEBUG 3: Rejected: level mismatch = 697
> >> >> DEBUG 3: Rejected: quality marker = 0
> >> >> DEBUG 3: Rejected: message type = 0
> >> >> DEBUG 3: Rejected: masking region = 0
> >> >> DEBUG 3: Rejected: bad fcst value = 0
> >> >> DEBUG 3: Rejected: duplicates = 0
> >> >> DEBUG 2: Processing VGRD/Z10 versus VGRD/Z10, for observation
type
> >> ADPSFC,
> >> >> over region FULL, for interpolation method UW_MEAN(25), using
0
> >> pairs.
> >> >> DEBUG 3: Number of matched pairs = 0
> >> >> DEBUG 3: Observations processed = 204910
> >> >> DEBUG 3: Rejected: GRIB code = 170420
> >> >> DEBUG 3: Rejected: valid time = 0
> >> >> DEBUG 3: Rejected: bad obs value = 0
> >> >> DEBUG 3: Rejected: off the grid = 33793
> >> >> DEBUG 3: Rejected: level mismatch = 697
> >> >> DEBUG 3: Rejected: quality marker = 0
> >> >> DEBUG 3: Rejected: message type = 0
> >> >> DEBUG 3: Rejected: masking region = 0
> >> >> DEBUG 3: Rejected: bad fcst value = 0
> >> >> DEBUG 3: Rejected: duplicates = 0
> >> >>
> >> >> Looking at this, I don't see any obvious problems.
> >> >>
> >> >> Third, I ran the plot_point_obs utility to figure out what
> >> observations
> >> of
> >> >> wind fall in your domain...
> >> >>
> >> >> [johnhg at rambler]%
> >> /d1/johnhg/MET/MET_releases/METv4.1/bin/plot_point_obs
> >> >> 20121105.gdas1.t00z.prepbufr.nr_le.nc
> >> >> 20121105.gdas1.t00z.prepbufr.nr_le.ps -gc 33 -gc 34 -msg_typ
ADPSFC
> >> >> -data_file WRFPRS_d01_2012-11-05_00_00_00
> >> >> DEBUG 2: Retrieving grid from file: WRFPRS_d01_2012-11-
05_00_00_00
> >> >> DEBUG 1: Opening netCDF file:
20121105.gdas1.t00z.prepbufr.nr_le.nc
> >> >> DEBUG 2: Processing 204910 observations at 1339 locations.
> >> >> DEBUG 2: Observation GRIB codes: 33 34
> >> >> DEBUG 2: Observation message types: ADPSFC
> >> >> DEBUG 1: Creating postscript file:
> >> 20121105.gdas1.t00z.prepbufr.nr_le.ps
> >> >> DEBUG 2: Finished plotting 0 locations.
> >> >> DEBUG 2: Skipped 0 locations off the grid.
> >> >>
> >> >> Looking at this, there are no wind observations (GRIB code 33
and 34)
> >> for
> >> >> the ADPSFC message type over your domain for this time.
> >> >>
> >> >> FYI - if you plot all message types by removing "-msg_typ
ADPSFC",
> >> you'll
> >> >> see that there are 11 non-ADPSFC observation locations.
> >> >>
> >> >> [johnhg at rambler]%
> >> /d1/johnhg/MET/MET_releases/METv4.1/bin/plot_point_obs
> >> >> 20121105.gdas1.t00z.prepbufr.nr_le.nc
> >> >> 20121105.gdas1.t00z.prepbufr.nr_le.ps -gc 33 -gc 34 -data_file
> >> >> WRFPRS_d01_2012-11-05_00_00_00
> >> >> DEBUG 2: Retrieving grid from file: WRFPRS_d01_2012-11-
05_00_00_00
> >> >> DEBUG 1: Opening netCDF file:
20121105.gdas1.t00z.prepbufr.nr_le.nc
> >> >> DEBUG 2: Processing 204910 observations at 1339 locations.
> >> >> DEBUG 2: Observation GRIB codes: 33 34
> >> >> DEBUG 2: Observation message types: ALL
> >> >> DEBUG 1: Creating postscript file:
> >> 20121105.gdas1.t00z.prepbufr.nr_le.ps
> >> >> DEBUG 2: Finished plotting 11 locations.
> >> >> DEBUG 2: Skipped 692 locations off the grid.
> >> >>
> >> >> So there just aren't any observations of wind at this time
with
> >> message
> >> >> type ADPSFC over your domain. Why not?
> >> >>
> >> >> I suspect it's because of the quality control value used for
surface
> >> data
> >> >> in GDAS files. Please read the "Note" at the bottom of this
page...
> >> >>
http://www.dtcenter.org/met/users/downloads/observation_data.php
> >> >>
> >> >> Please try rerunning PB2NC with "quality_mark_thresh = 9" in
the
> >> config
> >> >> file. After you rerun PB2NC, I suspect you'll get matched
pairs in
> >> your
> >> >> Point-Stat output.
> >> >>
> >> >> Sorry to be so long-winded on this. I just wanted to step you
> >> through
> >> the
> >> >> "debugging" process to assist you in doing it yourself in the
future.
> >> >>
> >> >> Hope that helps.
> >> >>
> >> >> Thanks,
> >> >> John Halley Gotway
> >> >> met_help at ucar.edu
> >> >>
> >> >>>
> >> >>> Tue Oct 29 17:14:40 2013: Request 63649 was acted upon.
> >> >>> Transaction: Ticket created by jiang at gina.alaska.edu
> >> >>> Queue: met_help
> >> >>> Subject: compare 10m wind between wrf output and
prepbufr data
> >> >>> Owner: Nobody
> >> >>> Requestors: jiang at gina.alaska.edu
> >> >>> Status: new
> >> >>> Ticket <URL:
> >> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=63649>
> >> >>>
> >> >>>
> >> >>> Hi CS,
> >> >>>
> >> >>> I would like to compare 10 m wind between wrf output and
prepbufr.
> >> >>>
> >> >>> the output from WRF go through postprocess by UPPV2.1, I
uploaded
> >> two
> >> >> data
> >> >>> files and one point-stat configure file.
> >> >>> ftp -p ftp.rap.ucar.edu/incoming/irap/met_help/zhu_data
> >> >>>
> >> >>> Would you please help me figure out why I can not produce
output to
> >> >>> compare
> >> >>> wind?
> >> >>> Perhaps there is no 10m wind in prepbufr file? I don't know
how to
> >> check
> >> >>> if
> >> >>> there is 10m wind in prepbufr files.
> >> >>>
> >> >>> Thank you in advance,
> >> >>>
> >> >>> Jiang
> >> >>>
> >> >>>
> >> >>> **********************************************************
> >> >>> Jiang Zhu, Ph.D.
> >> >>> Geographic Information Network of Alaska
> >> >>> University of Alaska Fairbanks
> >> >>> Phone: 907-474-5689, EMAIL: jiang at gina.alaska.edu
> >> >>> ***********************************************************
> >> >>>
> >> >>
> >> >>
> >> >>
> >> >>
> >> >
> >> >
> >>
> >>
> >
> >
> > --
> > **********************************************************
> > Jiang Zhu, Ph.D.
> > Geographic Information Network of Alaska
> > University of Alaska Fairbanks
> > Phone: 907-474-5689, EMAIL: jiang at gina.alaska.edu
> > ***********************************************************
> >
>
>
>
>
--
**********************************************************
Jiang Zhu, Ph.D.
Geographic Information Network of Alaska
University of Alaska Fairbanks
Phone: 907-474-5689, EMAIL: jiang at gina.alaska.edu
***********************************************************
------------------------------------------------
More information about the Met_help
mailing list