[Met_help] [rt.rap.ucar.edu #47129] History for Problem using GDAS ADPSFC observations
RAL HelpDesk {for John Halley Gotway}
met_help at ucar.edu
Fri May 27 15:09:13 MDT 2011
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hi,
I'm having a problem with point_stat in METv3.0 when attempting to use
ADPSFC data from GDAS PREPBUFR files. For example, when I attempt to
verify TMP/Z2 against ADPSFC observations, I get zero matched pairs. If
I use the ONLYSF option, it will use the SFCSHP observations, but not
ADPSFC.
I know the ADPSFC observations exist in the netCDF files that were
created by PB2NC, because there are many ADPSFC obs in the hdr_typ array
when I ncdump any particular obs file.
I looked through the known issues section of the website and saw the
following,But when I differenced the updated pair_data.cc file with the
one I have, they are identical, so I must have made that update previously.
One interesting thing to note is that when I use NDAS observations
instead of GDAS observations, this issue does not occur. The model
fields verify against ADPSFC observations just fine. So I'm not sure
what the issue might be. Unfortunately, I'm using this data for an old
case, so I do not have access to NDAS observations for this period.
Any ideas? Thanks.
- Jeff Z.
Point-Stat Tool
Vertical level matching logic when using a generic level type.
Posted 10/05/2010
*Problem:*When verifying generic level type data (e.g. "RH/L1") against
a surface observation message type (e.g. "ADPSFC"), Point-Stat was
finding zero matched pairs.
*Solution:*The fix is to no longer enforce that the observation level
value exactly match the forecast level value when verifying against
surface observation message types. This makes the behavior in METv3.0
match that of METv2.0.
Update:METv3.0/lib/vx_met_util/pair_data.cc
<http://www.dtcenter.org/met/users/support/known_issues/METv3.0/patches/lib/vx_met_util/pair_data.cc>
And then recompile MET.
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #47129] Problem using GDAS ADPSFC observations
From: Paul Oldenburg
Time: Wed May 25 12:23:17 2011
Jeff,
I would first suggest using the point_stat options -obs_valid_beg and
-obs_valid_end to open the time window for matched
observations to be very large. If this results in matched pairs, then
dump out those matched pairs by setting flag #15
in the point_stat config file setting output_flag to 1 or 2. Then,
you can see what the valid time for the matched
pairs actually is and adjust your setup accordingly.
If that course of action is not fruitful, please send us the following
so that we may try to reproduce and figure out
your problem:
1. Model data
2. Obs data
3. point_stat config file
If these files are too large to include in an email, please upload
them to our FTP site using the instructions here:
http://www.dtcenter.org/met/users/support/met_help.php#ftp
If you have any questions, please let me know.
Paul
On 05/25/2011 12:10 PM, RAL HelpDesk {for Jeff Zielonka} wrote:
>
> Wed May 25 12:10:14 2011: Request 47129 was acted upon.
> Transaction: Ticket created by zielonka at meteo.psu.edu
> Queue: met_help
> Subject: Problem using GDAS ADPSFC observations
> Owner: Nobody
> Requestors: zielonka at meteo.psu.edu
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129 >
>
>
> Hi,
>
> I'm having a problem with point_stat in METv3.0 when attempting to
use
> ADPSFC data from GDAS PREPBUFR files. For example, when I attempt
to
> verify TMP/Z2 against ADPSFC observations, I get zero matched pairs.
If
> I use the ONLYSF option, it will use the SFCSHP observations, but
not
> ADPSFC.
>
> I know the ADPSFC observations exist in the netCDF files that were
> created by PB2NC, because there are many ADPSFC obs in the hdr_typ
array
> when I ncdump any particular obs file.
>
> I looked through the known issues section of the website and saw the
> following,But when I differenced the updated pair_data.cc file with
the
> one I have, they are identical, so I must have made that update
previously.
>
> One interesting thing to note is that when I use NDAS observations
> instead of GDAS observations, this issue does not occur. The model
> fields verify against ADPSFC observations just fine. So I'm not
sure
> what the issue might be. Unfortunately, I'm using this data for an
old
> case, so I do not have access to NDAS observations for this period.
>
> Any ideas? Thanks.
>
>
> - Jeff Z.
>
>
> Point-Stat Tool
>
>
> Vertical level matching logic when using a generic level type.
> Posted 10/05/2010
>
> *Problem:*When verifying generic level type data (e.g. "RH/L1")
against
> a surface observation message type (e.g. "ADPSFC"), Point-Stat was
> finding zero matched pairs.
> *Solution:*The fix is to no longer enforce that the observation
level
> value exactly match the forecast level value when verifying against
> surface observation message types. This makes the behavior in
METv3.0
> match that of METv2.0.
> Update:METv3.0/lib/vx_met_util/pair_data.cc
>
<http://www.dtcenter.org/met/users/support/known_issues/METv3.0/patches/lib/vx_met_util/pair_data.cc>
> And then recompile MET.
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #47129] Problem using GDAS ADPSFC observations
From: Jeff Zielonka
Time: Wed May 25 12:58:21 2011
Hi Paul,
I'm currently using a time window of -1800 to 1800. I increased the
verbosity level when I ran point_stat and it says there are 0 obs that
are being rejected because of the valid time.
I have uploaded a grib file, the corresponding obs file, and the
config
file to the ftp site. Note, when I ran pb2nc for this particular obs
file, I only retained ADPUPA and ADPSFC observations. I also have an
additional masking grid defined in my config file, so you can just get
rid of the PHR2 and keep FULL.
Thanks for your help.
- Jeff
On 5/25/2011 2:23 PM, RAL HelpDesk {for Paul Oldenburg} wrote:
> Jeff,
>
> I would first suggest using the point_stat options -obs_valid_beg
and -obs_valid_end to open the time window for matched
> observations to be very large. If this results in matched pairs,
then dump out those matched pairs by setting flag #15
> in the point_stat config file setting output_flag to 1 or 2. Then,
you can see what the valid time for the matched
> pairs actually is and adjust your setup accordingly.
>
> If that course of action is not fruitful, please send us the
following so that we may try to reproduce and figure out
> your problem:
>
> 1. Model data
> 2. Obs data
> 3. point_stat config file
>
> If these files are too large to include in an email, please upload
them to our FTP site using the instructions here:
>
> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>
> If you have any questions, please let me know.
>
> Paul
>
> On 05/25/2011 12:10 PM, RAL HelpDesk {for Jeff Zielonka} wrote:
>> Wed May 25 12:10:14 2011: Request 47129 was acted upon.
>> Transaction: Ticket created by zielonka at meteo.psu.edu
>> Queue: met_help
>> Subject: Problem using GDAS ADPSFC observations
>> Owner: Nobody
>> Requestors: zielonka at meteo.psu.edu
>> Status: new
>> Ticket<URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129>
>>
>>
>> Hi,
>>
>> I'm having a problem with point_stat in METv3.0 when attempting to
use
>> ADPSFC data from GDAS PREPBUFR files. For example, when I attempt
to
>> verify TMP/Z2 against ADPSFC observations, I get zero matched
pairs. If
>> I use the ONLYSF option, it will use the SFCSHP observations, but
not
>> ADPSFC.
>>
>> I know the ADPSFC observations exist in the netCDF files that were
>> created by PB2NC, because there are many ADPSFC obs in the hdr_typ
array
>> when I ncdump any particular obs file.
>>
>> I looked through the known issues section of the website and saw
the
>> following,But when I differenced the updated pair_data.cc file with
the
>> one I have, they are identical, so I must have made that update
previously.
>>
>> One interesting thing to note is that when I use NDAS observations
>> instead of GDAS observations, this issue does not occur. The model
>> fields verify against ADPSFC observations just fine. So I'm not
sure
>> what the issue might be. Unfortunately, I'm using this data for an
old
>> case, so I do not have access to NDAS observations for this period.
>>
>> Any ideas? Thanks.
>>
>>
>> - Jeff Z.
>>
>>
>> Point-Stat Tool
>>
>>
>> Vertical level matching logic when using a generic level
type.
>> Posted 10/05/2010
>>
>> *Problem:*When verifying generic level type data (e.g. "RH/L1")
against
>> a surface observation message type (e.g. "ADPSFC"), Point-Stat was
>> finding zero matched pairs.
>> *Solution:*The fix is to no longer enforce that the observation
level
>> value exactly match the forecast level value when verifying against
>> surface observation message types. This makes the behavior in
METv3.0
>> match that of METv2.0.
>> Update:METv3.0/lib/vx_met_util/pair_data.cc
>>
<http://www.dtcenter.org/met/users/support/known_issues/METv3.0/patches/lib/vx_met_util/pair_data.cc>
>> And then recompile MET.
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #47129] Problem using GDAS ADPSFC observations
From: Paul Oldenburg
Time: Thu May 26 17:02:40 2011
Jeff,
Sorry on the delay getting back to you. We are working on what
appears to be a bug in pb2nc or some other strangeness
with PrepBUFR observations. I'll let you know when we figure it out.
Thanks,
Paul
On 05/25/2011 12:58 PM, RAL HelpDesk {for Jeff Zielonka} wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129 >
>
> Hi Paul,
>
> I'm currently using a time window of -1800 to 1800. I increased the
> verbosity level when I ran point_stat and it says there are 0 obs
that
> are being rejected because of the valid time.
>
> I have uploaded a grib file, the corresponding obs file, and the
config
> file to the ftp site. Note, when I ran pb2nc for this particular
obs
> file, I only retained ADPUPA and ADPSFC observations. I also have
an
> additional masking grid defined in my config file, so you can just
get
> rid of the PHR2 and keep FULL.
>
> Thanks for your help.
>
> - Jeff
>
>
> On 5/25/2011 2:23 PM, RAL HelpDesk {for Paul Oldenburg} wrote:
>> Jeff,
>>
>> I would first suggest using the point_stat options -obs_valid_beg
and -obs_valid_end to open the time window for matched
>> observations to be very large. If this results in matched pairs,
then dump out those matched pairs by setting flag #15
>> in the point_stat config file setting output_flag to 1 or 2. Then,
you can see what the valid time for the matched
>> pairs actually is and adjust your setup accordingly.
>>
>> If that course of action is not fruitful, please send us the
following so that we may try to reproduce and figure out
>> your problem:
>>
>> 1. Model data
>> 2. Obs data
>> 3. point_stat config file
>>
>> If these files are too large to include in an email, please upload
them to our FTP site using the instructions here:
>>
>> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>>
>> If you have any questions, please let me know.
>>
>> Paul
>>
>> On 05/25/2011 12:10 PM, RAL HelpDesk {for Jeff Zielonka} wrote:
>>> Wed May 25 12:10:14 2011: Request 47129 was acted upon.
>>> Transaction: Ticket created by zielonka at meteo.psu.edu
>>> Queue: met_help
>>> Subject: Problem using GDAS ADPSFC observations
>>> Owner: Nobody
>>> Requestors: zielonka at meteo.psu.edu
>>> Status: new
>>> Ticket<URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129>
>>>
>>>
>>> Hi,
>>>
>>> I'm having a problem with point_stat in METv3.0 when attempting to
use
>>> ADPSFC data from GDAS PREPBUFR files. For example, when I
attempt to
>>> verify TMP/Z2 against ADPSFC observations, I get zero matched
pairs. If
>>> I use the ONLYSF option, it will use the SFCSHP observations, but
not
>>> ADPSFC.
>>>
>>> I know the ADPSFC observations exist in the netCDF files that were
>>> created by PB2NC, because there are many ADPSFC obs in the hdr_typ
array
>>> when I ncdump any particular obs file.
>>>
>>> I looked through the known issues section of the website and saw
the
>>> following,But when I differenced the updated pair_data.cc file
with the
>>> one I have, they are identical, so I must have made that update
previously.
>>>
>>> One interesting thing to note is that when I use NDAS observations
>>> instead of GDAS observations, this issue does not occur. The
model
>>> fields verify against ADPSFC observations just fine. So I'm not
sure
>>> what the issue might be. Unfortunately, I'm using this data for
an old
>>> case, so I do not have access to NDAS observations for this
period.
>>>
>>> Any ideas? Thanks.
>>>
>>>
>>> - Jeff Z.
>>>
>>>
>>> Point-Stat Tool
>>>
>>>
>>> Vertical level matching logic when using a generic level
type.
>>> Posted 10/05/2010
>>>
>>> *Problem:*When verifying generic level type data (e.g. "RH/L1")
against
>>> a surface observation message type (e.g. "ADPSFC"), Point-Stat was
>>> finding zero matched pairs.
>>> *Solution:*The fix is to no longer enforce that the observation
level
>>> value exactly match the forecast level value when verifying
against
>>> surface observation message types. This makes the behavior in
METv3.0
>>> match that of METv2.0.
>>> Update:METv3.0/lib/vx_met_util/pair_data.cc
>>>
<http://www.dtcenter.org/met/users/support/known_issues/METv3.0/patches/lib/vx_met_util/pair_data.cc>
>>> And then recompile MET.
>>
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #47129] Problem using GDAS ADPSFC observations
From: Jeff Zielonka
Time: Fri May 27 07:37:47 2011
Paul,
No worries. I have other things that can keep me occupied while
you're
working on this. Thanks a lot for the help and the update.
- Jeff
On 5/26/2011 7:02 PM, RAL HelpDesk {for Paul Oldenburg} wrote:
> Jeff,
>
> Sorry on the delay getting back to you. We are working on what
appears to be a bug in pb2nc or some other strangeness
> with PrepBUFR observations. I'll let you know when we figure it
out.
>
> Thanks,
>
> Paul
>
>
> On 05/25/2011 12:58 PM, RAL HelpDesk {for Jeff Zielonka} wrote:
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129>
>>
>> Hi Paul,
>>
>> I'm currently using a time window of -1800 to 1800. I increased
the
>> verbosity level when I ran point_stat and it says there are 0 obs
that
>> are being rejected because of the valid time.
>>
>> I have uploaded a grib file, the corresponding obs file, and the
config
>> file to the ftp site. Note, when I ran pb2nc for this particular
obs
>> file, I only retained ADPUPA and ADPSFC observations. I also have
an
>> additional masking grid defined in my config file, so you can just
get
>> rid of the PHR2 and keep FULL.
>>
>> Thanks for your help.
>>
>> - Jeff
>>
>>
>> On 5/25/2011 2:23 PM, RAL HelpDesk {for Paul Oldenburg} wrote:
>>> Jeff,
>>>
>>> I would first suggest using the point_stat options -obs_valid_beg
and -obs_valid_end to open the time window for matched
>>> observations to be very large. If this results in matched pairs,
then dump out those matched pairs by setting flag #15
>>> in the point_stat config file setting output_flag to 1 or 2.
Then, you can see what the valid time for the matched
>>> pairs actually is and adjust your setup accordingly.
>>>
>>> If that course of action is not fruitful, please send us the
following so that we may try to reproduce and figure out
>>> your problem:
>>>
>>> 1. Model data
>>> 2. Obs data
>>> 3. point_stat config file
>>>
>>> If these files are too large to include in an email, please upload
them to our FTP site using the instructions here:
>>>
>>> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>>>
>>> If you have any questions, please let me know.
>>>
>>> Paul
>>>
>>> On 05/25/2011 12:10 PM, RAL HelpDesk {for Jeff Zielonka} wrote:
>>>> Wed May 25 12:10:14 2011: Request 47129 was acted upon.
>>>> Transaction: Ticket created by zielonka at meteo.psu.edu
>>>> Queue: met_help
>>>> Subject: Problem using GDAS ADPSFC observations
>>>> Owner: Nobody
>>>> Requestors: zielonka at meteo.psu.edu
>>>> Status: new
>>>> Ticket<URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129>
>>>>
>>>>
>>>> Hi,
>>>>
>>>> I'm having a problem with point_stat in METv3.0 when attempting
to use
>>>> ADPSFC data from GDAS PREPBUFR files. For example, when I
attempt to
>>>> verify TMP/Z2 against ADPSFC observations, I get zero matched
pairs. If
>>>> I use the ONLYSF option, it will use the SFCSHP observations, but
not
>>>> ADPSFC.
>>>>
>>>> I know the ADPSFC observations exist in the netCDF files that
were
>>>> created by PB2NC, because there are many ADPSFC obs in the
hdr_typ array
>>>> when I ncdump any particular obs file.
>>>>
>>>> I looked through the known issues section of the website and saw
the
>>>> following,But when I differenced the updated pair_data.cc file
with the
>>>> one I have, they are identical, so I must have made that update
previously.
>>>>
>>>> One interesting thing to note is that when I use NDAS
observations
>>>> instead of GDAS observations, this issue does not occur. The
model
>>>> fields verify against ADPSFC observations just fine. So I'm not
sure
>>>> what the issue might be. Unfortunately, I'm using this data for
an old
>>>> case, so I do not have access to NDAS observations for this
period.
>>>>
>>>> Any ideas? Thanks.
>>>>
>>>>
>>>> - Jeff Z.
>>>>
>>>>
>>>> Point-Stat Tool
>>>>
>>>>
>>>> Vertical level matching logic when using a generic level
type.
>>>> Posted 10/05/2010
>>>>
>>>> *Problem:*When verifying generic level type data (e.g. "RH/L1")
against
>>>> a surface observation message type (e.g. "ADPSFC"), Point-Stat
was
>>>> finding zero matched pairs.
>>>> *Solution:*The fix is to no longer enforce that the observation
level
>>>> value exactly match the forecast level value when verifying
against
>>>> surface observation message types. This makes the behavior in
METv3.0
>>>> match that of METv2.0.
>>>> Update:METv3.0/lib/vx_met_util/pair_data.cc
>>>>
<http://www.dtcenter.org/met/users/support/known_issues/METv3.0/patches/lib/vx_met_util/pair_data.cc>
>>>> And then recompile MET.
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #47129] Problem using GDAS ADPSFC observations
From: Paul Oldenburg
Time: Fri May 27 12:36:16 2011
Jeff,
The GDAS obs data contains ADPSFC TMP observations whose quality mark
may be higher than that of other observations.
I'm not totally familiar with quality marks, but I do know that pb2nc
will only pull observations for data whose quality
mark is less than the threshold stored in the setting
quality_mark_thresh. I downloaded the GDAS PrepBUFR obs file for
2007100500 and then ran it through pb2nc. I was able to reproduce the
problem that you reported with 0 matched pairs
found for TMP/Z2. My colleague John suggested re-running pb2nc and
raising the quality_mark_thresh to 9 in the pb2nc
configuration file. When I did so, point_stat found matched pairs
when I ran it with the new pb2nc NetCDF output file.
Please try adjusting your pb2nc configuration file to use this
setting:
quality_mark_thresh = 9
Then, re-run point_stat to see if it finds matched pairs for TMP/Z2.
If not, please let us know.
Thanks,
Paul
On 05/27/2011 07:37 AM, RAL HelpDesk {for Jeff Zielonka} wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129 >
>
> Paul,
>
> No worries. I have other things that can keep me occupied while
you're
> working on this. Thanks a lot for the help and the update.
>
> - Jeff
>
>
> On 5/26/2011 7:02 PM, RAL HelpDesk {for Paul Oldenburg} wrote:
>> Jeff,
>>
>> Sorry on the delay getting back to you. We are working on what
appears to be a bug in pb2nc or some other strangeness
>> with PrepBUFR observations. I'll let you know when we figure it
out.
>>
>> Thanks,
>>
>> Paul
>>
>>
>> On 05/25/2011 12:58 PM, RAL HelpDesk {for Jeff Zielonka} wrote:
>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129>
>>>
>>> Hi Paul,
>>>
>>> I'm currently using a time window of -1800 to 1800. I increased
the
>>> verbosity level when I ran point_stat and it says there are 0 obs
that
>>> are being rejected because of the valid time.
>>>
>>> I have uploaded a grib file, the corresponding obs file, and the
config
>>> file to the ftp site. Note, when I ran pb2nc for this particular
obs
>>> file, I only retained ADPUPA and ADPSFC observations. I also have
an
>>> additional masking grid defined in my config file, so you can just
get
>>> rid of the PHR2 and keep FULL.
>>>
>>> Thanks for your help.
>>>
>>> - Jeff
>>>
>>>
>>> On 5/25/2011 2:23 PM, RAL HelpDesk {for Paul Oldenburg} wrote:
>>>> Jeff,
>>>>
>>>> I would first suggest using the point_stat options -obs_valid_beg
and -obs_valid_end to open the time window for matched
>>>> observations to be very large. If this results in matched pairs,
then dump out those matched pairs by setting flag #15
>>>> in the point_stat config file setting output_flag to 1 or 2.
Then, you can see what the valid time for the matched
>>>> pairs actually is and adjust your setup accordingly.
>>>>
>>>> If that course of action is not fruitful, please send us the
following so that we may try to reproduce and figure out
>>>> your problem:
>>>>
>>>> 1. Model data
>>>> 2. Obs data
>>>> 3. point_stat config file
>>>>
>>>> If these files are too large to include in an email, please
upload them to our FTP site using the instructions here:
>>>>
>>>> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>>>>
>>>> If you have any questions, please let me know.
>>>>
>>>> Paul
>>>>
>>>> On 05/25/2011 12:10 PM, RAL HelpDesk {for Jeff Zielonka} wrote:
>>>>> Wed May 25 12:10:14 2011: Request 47129 was acted upon.
>>>>> Transaction: Ticket created by zielonka at meteo.psu.edu
>>>>> Queue: met_help
>>>>> Subject: Problem using GDAS ADPSFC observations
>>>>> Owner: Nobody
>>>>> Requestors: zielonka at meteo.psu.edu
>>>>> Status: new
>>>>> Ticket<URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129>
>>>>>
>>>>>
>>>>> Hi,
>>>>>
>>>>> I'm having a problem with point_stat in METv3.0 when attempting
to use
>>>>> ADPSFC data from GDAS PREPBUFR files. For example, when I
attempt to
>>>>> verify TMP/Z2 against ADPSFC observations, I get zero matched
pairs. If
>>>>> I use the ONLYSF option, it will use the SFCSHP observations,
but not
>>>>> ADPSFC.
>>>>>
>>>>> I know the ADPSFC observations exist in the netCDF files that
were
>>>>> created by PB2NC, because there are many ADPSFC obs in the
hdr_typ array
>>>>> when I ncdump any particular obs file.
>>>>>
>>>>> I looked through the known issues section of the website and saw
the
>>>>> following,But when I differenced the updated pair_data.cc file
with the
>>>>> one I have, they are identical, so I must have made that update
previously.
>>>>>
>>>>> One interesting thing to note is that when I use NDAS
observations
>>>>> instead of GDAS observations, this issue does not occur. The
model
>>>>> fields verify against ADPSFC observations just fine. So I'm not
sure
>>>>> what the issue might be. Unfortunately, I'm using this data for
an old
>>>>> case, so I do not have access to NDAS observations for this
period.
>>>>>
>>>>> Any ideas? Thanks.
>>>>>
>>>>>
>>>>> - Jeff Z.
>>>>>
>>>>>
>>>>> Point-Stat Tool
>>>>>
>>>>>
>>>>> Vertical level matching logic when using a generic level
type.
>>>>> Posted 10/05/2010
>>>>>
>>>>> *Problem:*When verifying generic level type data (e.g. "RH/L1")
against
>>>>> a surface observation message type (e.g. "ADPSFC"), Point-Stat
was
>>>>> finding zero matched pairs.
>>>>> *Solution:*The fix is to no longer enforce that the observation
level
>>>>> value exactly match the forecast level value when verifying
against
>>>>> surface observation message types. This makes the behavior in
METv3.0
>>>>> match that of METv2.0.
>>>>> Update:METv3.0/lib/vx_met_util/pair_data.cc
>>>>>
<http://www.dtcenter.org/met/users/support/known_issues/METv3.0/patches/lib/vx_met_util/pair_data.cc>
>>>>> And then recompile MET.
>>
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #47129] Problem using GDAS ADPSFC observations
From: Jeff Zielonka
Time: Fri May 27 13:20:22 2011
Paul,
It looks like this did the trick. I re-ran pb2nc and point_stat, and
this resulted in 3655 matched pairs for TMP/Z2 on the full domain, and
320 on the smaller subdomain that you wouldn't have seen. Are these
3655 pairs consistent (within reason) with what you found?
Thanks again for you help with this. I don't think I would have been
able to track down this issue on my own. I've never even thought
about
adjusting the default value for quality_mark_thresh.
I'm assuming this will solve my issues, but I'll let you know if I run
into any snags in the next week.
- Jeff
On 5/27/2011 2:36 PM, RAL HelpDesk {for Paul Oldenburg} wrote:
> Jeff,
>
> The GDAS obs data contains ADPSFC TMP observations whose quality
mark may be higher than that of other observations.
> I'm not totally familiar with quality marks, but I do know that
pb2nc will only pull observations for data whose quality
> mark is less than the threshold stored in the setting
quality_mark_thresh. I downloaded the GDAS PrepBUFR obs file for
> 2007100500 and then ran it through pb2nc. I was able to reproduce
the problem that you reported with 0 matched pairs
> found for TMP/Z2. My colleague John suggested re-running pb2nc and
raising the quality_mark_thresh to 9 in the pb2nc
> configuration file. When I did so, point_stat found matched pairs
when I ran it with the new pb2nc NetCDF output file.
>
> Please try adjusting your pb2nc configuration file to use this
setting:
>
> quality_mark_thresh = 9
>
> Then, re-run point_stat to see if it finds matched pairs for TMP/Z2.
If not, please let us know.
>
> Thanks,
>
> Paul
>
>
> On 05/27/2011 07:37 AM, RAL HelpDesk {for Jeff Zielonka} wrote:
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129>
>>
>> Paul,
>>
>> No worries. I have other things that can keep me occupied while
you're
>> working on this. Thanks a lot for the help and the update.
>>
>> - Jeff
>>
>>
>> On 5/26/2011 7:02 PM, RAL HelpDesk {for Paul Oldenburg} wrote:
>>> Jeff,
>>>
>>> Sorry on the delay getting back to you. We are working on what
appears to be a bug in pb2nc or some other strangeness
>>> with PrepBUFR observations. I'll let you know when we figure it
out.
>>>
>>> Thanks,
>>>
>>> Paul
>>>
>>>
>>> On 05/25/2011 12:58 PM, RAL HelpDesk {for Jeff Zielonka} wrote:
>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129>
>>>>
>>>> Hi Paul,
>>>>
>>>> I'm currently using a time window of -1800 to 1800. I increased
the
>>>> verbosity level when I ran point_stat and it says there are 0 obs
that
>>>> are being rejected because of the valid time.
>>>>
>>>> I have uploaded a grib file, the corresponding obs file, and the
config
>>>> file to the ftp site. Note, when I ran pb2nc for this particular
obs
>>>> file, I only retained ADPUPA and ADPSFC observations. I also
have an
>>>> additional masking grid defined in my config file, so you can
just get
>>>> rid of the PHR2 and keep FULL.
>>>>
>>>> Thanks for your help.
>>>>
>>>> - Jeff
>>>>
>>>>
>>>> On 5/25/2011 2:23 PM, RAL HelpDesk {for Paul Oldenburg} wrote:
>>>>> Jeff,
>>>>>
>>>>> I would first suggest using the point_stat options
-obs_valid_beg and -obs_valid_end to open the time window for matched
>>>>> observations to be very large. If this results in matched
pairs, then dump out those matched pairs by setting flag #15
>>>>> in the point_stat config file setting output_flag to 1 or 2.
Then, you can see what the valid time for the matched
>>>>> pairs actually is and adjust your setup accordingly.
>>>>>
>>>>> If that course of action is not fruitful, please send us the
following so that we may try to reproduce and figure out
>>>>> your problem:
>>>>>
>>>>> 1. Model data
>>>>> 2. Obs data
>>>>> 3. point_stat config file
>>>>>
>>>>> If these files are too large to include in an email, please
upload them to our FTP site using the instructions here:
>>>>>
>>>>> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>>>>>
>>>>> If you have any questions, please let me know.
>>>>>
>>>>> Paul
>>>>>
>>>>> On 05/25/2011 12:10 PM, RAL HelpDesk {for Jeff Zielonka} wrote:
>>>>>> Wed May 25 12:10:14 2011: Request 47129 was acted upon.
>>>>>> Transaction: Ticket created by zielonka at meteo.psu.edu
>>>>>> Queue: met_help
>>>>>> Subject: Problem using GDAS ADPSFC observations
>>>>>> Owner: Nobody
>>>>>> Requestors: zielonka at meteo.psu.edu
>>>>>> Status: new
>>>>>> Ticket<URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129>
>>>>>>
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I'm having a problem with point_stat in METv3.0 when attempting
to use
>>>>>> ADPSFC data from GDAS PREPBUFR files. For example, when I
attempt to
>>>>>> verify TMP/Z2 against ADPSFC observations, I get zero matched
pairs. If
>>>>>> I use the ONLYSF option, it will use the SFCSHP observations,
but not
>>>>>> ADPSFC.
>>>>>>
>>>>>> I know the ADPSFC observations exist in the netCDF files that
were
>>>>>> created by PB2NC, because there are many ADPSFC obs in the
hdr_typ array
>>>>>> when I ncdump any particular obs file.
>>>>>>
>>>>>> I looked through the known issues section of the website and
saw the
>>>>>> following,But when I differenced the updated pair_data.cc file
with the
>>>>>> one I have, they are identical, so I must have made that update
previously.
>>>>>>
>>>>>> One interesting thing to note is that when I use NDAS
observations
>>>>>> instead of GDAS observations, this issue does not occur. The
model
>>>>>> fields verify against ADPSFC observations just fine. So I'm
not sure
>>>>>> what the issue might be. Unfortunately, I'm using this data
for an old
>>>>>> case, so I do not have access to NDAS observations for this
period.
>>>>>>
>>>>>> Any ideas? Thanks.
>>>>>>
>>>>>>
>>>>>> - Jeff Z.
>>>>>>
>>>>>>
>>>>>> Point-Stat Tool
>>>>>>
>>>>>>
>>>>>> Vertical level matching logic when using a generic
level type.
>>>>>> Posted 10/05/2010
>>>>>>
>>>>>> *Problem:*When verifying generic level type data (e.g. "RH/L1")
against
>>>>>> a surface observation message type (e.g. "ADPSFC"), Point-Stat
was
>>>>>> finding zero matched pairs.
>>>>>> *Solution:*The fix is to no longer enforce that the observation
level
>>>>>> value exactly match the forecast level value when verifying
against
>>>>>> surface observation message types. This makes the behavior in
METv3.0
>>>>>> match that of METv2.0.
>>>>>> Update:METv3.0/lib/vx_met_util/pair_data.cc
>>>>>>
<http://www.dtcenter.org/met/users/support/known_issues/METv3.0/patches/lib/vx_met_util/pair_data.cc>
>>>>>> And then recompile MET.
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #47129] Problem using GDAS ADPSFC observations
From: Paul Oldenburg
Time: Fri May 27 13:30:51 2011
Jeff,
During my experiment, point_stat found 3652 matched pairs for TMP/Z2,
which seems commensurate to me. The
quality_mark_threshold is definitely an obscure feature of pb2nc. I
wasn't terribly familiar with it myself before I
worked on this problem.
Paul
On 05/27/2011 01:20 PM, RAL HelpDesk {for Jeff Zielonka} wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129 >
>
> Paul,
>
> It looks like this did the trick. I re-ran pb2nc and point_stat,
and
> this resulted in 3655 matched pairs for TMP/Z2 on the full domain,
and
> 320 on the smaller subdomain that you wouldn't have seen. Are these
> 3655 pairs consistent (within reason) with what you found?
>
> Thanks again for you help with this. I don't think I would have
been
> able to track down this issue on my own. I've never even thought
about
> adjusting the default value for quality_mark_thresh.
>
> I'm assuming this will solve my issues, but I'll let you know if I
run
> into any snags in the next week.
>
> - Jeff
>
>
>
> On 5/27/2011 2:36 PM, RAL HelpDesk {for Paul Oldenburg} wrote:
>> Jeff,
>>
>> The GDAS obs data contains ADPSFC TMP observations whose quality
mark may be higher than that of other observations.
>> I'm not totally familiar with quality marks, but I do know that
pb2nc will only pull observations for data whose quality
>> mark is less than the threshold stored in the setting
quality_mark_thresh. I downloaded the GDAS PrepBUFR obs file for
>> 2007100500 and then ran it through pb2nc. I was able to reproduce
the problem that you reported with 0 matched pairs
>> found for TMP/Z2. My colleague John suggested re-running pb2nc and
raising the quality_mark_thresh to 9 in the pb2nc
>> configuration file. When I did so, point_stat found matched pairs
when I ran it with the new pb2nc NetCDF output file.
>>
>> Please try adjusting your pb2nc configuration file to use this
setting:
>>
>> quality_mark_thresh = 9
>>
>> Then, re-run point_stat to see if it finds matched pairs for
TMP/Z2. If not, please let us know.
>>
>> Thanks,
>>
>> Paul
>>
>>
>> On 05/27/2011 07:37 AM, RAL HelpDesk {for Jeff Zielonka} wrote:
>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129>
>>>
>>> Paul,
>>>
>>> No worries. I have other things that can keep me occupied while
you're
>>> working on this. Thanks a lot for the help and the update.
>>>
>>> - Jeff
>>>
>>>
>>> On 5/26/2011 7:02 PM, RAL HelpDesk {for Paul Oldenburg} wrote:
>>>> Jeff,
>>>>
>>>> Sorry on the delay getting back to you. We are working on what
appears to be a bug in pb2nc or some other strangeness
>>>> with PrepBUFR observations. I'll let you know when we figure it
out.
>>>>
>>>> Thanks,
>>>>
>>>> Paul
>>>>
>>>>
>>>> On 05/25/2011 12:58 PM, RAL HelpDesk {for Jeff Zielonka} wrote:
>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129>
>>>>>
>>>>> Hi Paul,
>>>>>
>>>>> I'm currently using a time window of -1800 to 1800. I increased
the
>>>>> verbosity level when I ran point_stat and it says there are 0
obs that
>>>>> are being rejected because of the valid time.
>>>>>
>>>>> I have uploaded a grib file, the corresponding obs file, and the
config
>>>>> file to the ftp site. Note, when I ran pb2nc for this
particular obs
>>>>> file, I only retained ADPUPA and ADPSFC observations. I also
have an
>>>>> additional masking grid defined in my config file, so you can
just get
>>>>> rid of the PHR2 and keep FULL.
>>>>>
>>>>> Thanks for your help.
>>>>>
>>>>> - Jeff
>>>>>
>>>>>
>>>>> On 5/25/2011 2:23 PM, RAL HelpDesk {for Paul Oldenburg} wrote:
>>>>>> Jeff,
>>>>>>
>>>>>> I would first suggest using the point_stat options
-obs_valid_beg and -obs_valid_end to open the time window for matched
>>>>>> observations to be very large. If this results in matched
pairs, then dump out those matched pairs by setting flag #15
>>>>>> in the point_stat config file setting output_flag to 1 or 2.
Then, you can see what the valid time for the matched
>>>>>> pairs actually is and adjust your setup accordingly.
>>>>>>
>>>>>> If that course of action is not fruitful, please send us the
following so that we may try to reproduce and figure out
>>>>>> your problem:
>>>>>>
>>>>>> 1. Model data
>>>>>> 2. Obs data
>>>>>> 3. point_stat config file
>>>>>>
>>>>>> If these files are too large to include in an email, please
upload them to our FTP site using the instructions here:
>>>>>>
>>>>>> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>>>>>>
>>>>>> If you have any questions, please let me know.
>>>>>>
>>>>>> Paul
>>>>>>
>>>>>> On 05/25/2011 12:10 PM, RAL HelpDesk {for Jeff Zielonka} wrote:
>>>>>>> Wed May 25 12:10:14 2011: Request 47129 was acted upon.
>>>>>>> Transaction: Ticket created by zielonka at meteo.psu.edu
>>>>>>> Queue: met_help
>>>>>>> Subject: Problem using GDAS ADPSFC observations
>>>>>>> Owner: Nobody
>>>>>>> Requestors: zielonka at meteo.psu.edu
>>>>>>> Status: new
>>>>>>> Ticket<URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129>
>>>>>>>
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I'm having a problem with point_stat in METv3.0 when
attempting to use
>>>>>>> ADPSFC data from GDAS PREPBUFR files. For example, when I
attempt to
>>>>>>> verify TMP/Z2 against ADPSFC observations, I get zero matched
pairs. If
>>>>>>> I use the ONLYSF option, it will use the SFCSHP observations,
but not
>>>>>>> ADPSFC.
>>>>>>>
>>>>>>> I know the ADPSFC observations exist in the netCDF files that
were
>>>>>>> created by PB2NC, because there are many ADPSFC obs in the
hdr_typ array
>>>>>>> when I ncdump any particular obs file.
>>>>>>>
>>>>>>> I looked through the known issues section of the website and
saw the
>>>>>>> following,But when I differenced the updated pair_data.cc file
with the
>>>>>>> one I have, they are identical, so I must have made that
update previously.
>>>>>>>
>>>>>>> One interesting thing to note is that when I use NDAS
observations
>>>>>>> instead of GDAS observations, this issue does not occur. The
model
>>>>>>> fields verify against ADPSFC observations just fine. So I'm
not sure
>>>>>>> what the issue might be. Unfortunately, I'm using this data
for an old
>>>>>>> case, so I do not have access to NDAS observations for this
period.
>>>>>>>
>>>>>>> Any ideas? Thanks.
>>>>>>>
>>>>>>>
>>>>>>> - Jeff Z.
>>>>>>>
>>>>>>>
>>>>>>> Point-Stat Tool
>>>>>>>
>>>>>>>
>>>>>>> Vertical level matching logic when using a generic
level type.
>>>>>>> Posted 10/05/2010
>>>>>>>
>>>>>>> *Problem:*When verifying generic level type data (e.g.
"RH/L1") against
>>>>>>> a surface observation message type (e.g. "ADPSFC"), Point-Stat
was
>>>>>>> finding zero matched pairs.
>>>>>>> *Solution:*The fix is to no longer enforce that the
observation level
>>>>>>> value exactly match the forecast level value when verifying
against
>>>>>>> surface observation message types. This makes the behavior in
METv3.0
>>>>>>> match that of METv2.0.
>>>>>>> Update:METv3.0/lib/vx_met_util/pair_data.cc
>>>>>>>
<http://www.dtcenter.org/met/users/support/known_issues/METv3.0/patches/lib/vx_met_util/pair_data.cc>
>>>>>>> And then recompile MET.
>>
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #47129] Problem using GDAS ADPSFC observations
From: John Halley Gotway
Time: Fri May 27 13:45:30 2011
Jeff,
This is John Halley Gotway, and I wanted to chime in here with a
little
background on this issue.
In the DTC, we do reference configuration testing for the various
releases
of the WRF model. We had been using GDAS point observations for the
verification portion of this. However, after we discovered that PB2NC
bug
in the event stack flag (see MET known issues page for details), we
found
the same issue you are having with the GDAS point observations. NCEP
is
setting the quality marker for most of the GDAS surface observations
to a
value of 9. They're doing this to suppress the use of those
observations
in their data assimilation system, which must be having a negative
impact
of the assimilation. The impact on the verification is an unfortunate
consequence. This hadn't shown up before because the event stack flag
bug
was masking the behavior.
We spoke with the folks at NCEP who do verification of the operational
models. And they use the NDAS point observations for verifying over
CONUS
rather than GDAS. This odd quality mark setting does not show up in
NDAS.
So for our reference configuration testing efforts, we've switched
over to
use the NDAS PREPBUFR files. An added advantage is that PB2NC runs
much
faster on NDAS since there's a lot fewer observations to process.
Unless
you have a good reason not to use the NDAS obs for verifying over
CONUS,
I'd suggest doing so.
Right now PB2NC only filters quality marks using a threshold, keeping
all
obs whose quality mark is less than or equal to the threshold.
Perhaps,
it'd be preferable to allow for a list of quality marks to be used.
For
example, you may want to keep obs whose quality marks are 0, 1, 2, or
9.
Right now, there's no support for the in PB2NC. But it'd be a pretty
easy
extension. Would you find this functionality useful?
Hope that helps.
John
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129 >
>
> Jeff,
>
> During my experiment, point_stat found 3652 matched pairs for
TMP/Z2,
> which seems commensurate to me. The
> quality_mark_threshold is definitely an obscure feature of pb2nc. I
> wasn't terribly familiar with it myself before I
> worked on this problem.
>
> Paul
>
>
> On 05/27/2011 01:20 PM, RAL HelpDesk {for Jeff Zielonka} wrote:
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129 >
>>
>> Paul,
>>
>> It looks like this did the trick. I re-ran pb2nc and point_stat,
and
>> this resulted in 3655 matched pairs for TMP/Z2 on the full domain,
and
>> 320 on the smaller subdomain that you wouldn't have seen. Are
these
>> 3655 pairs consistent (within reason) with what you found?
>>
>> Thanks again for you help with this. I don't think I would have
been
>> able to track down this issue on my own. I've never even thought
about
>> adjusting the default value for quality_mark_thresh.
>>
>> I'm assuming this will solve my issues, but I'll let you know if I
run
>> into any snags in the next week.
>>
>> - Jeff
>>
>>
>>
>> On 5/27/2011 2:36 PM, RAL HelpDesk {for Paul Oldenburg} wrote:
>>> Jeff,
>>>
>>> The GDAS obs data contains ADPSFC TMP observations whose quality
mark
>>> may be higher than that of other observations.
>>> I'm not totally familiar with quality marks, but I do know that
pb2nc
>>> will only pull observations for data whose quality
>>> mark is less than the threshold stored in the setting
>>> quality_mark_thresh. I downloaded the GDAS PrepBUFR obs file for
>>> 2007100500 and then ran it through pb2nc. I was able to reproduce
the
>>> problem that you reported with 0 matched pairs
>>> found for TMP/Z2. My colleague John suggested re-running pb2nc
and
>>> raising the quality_mark_thresh to 9 in the pb2nc
>>> configuration file. When I did so, point_stat found matched pairs
when
>>> I ran it with the new pb2nc NetCDF output file.
>>>
>>> Please try adjusting your pb2nc configuration file to use this
setting:
>>>
>>> quality_mark_thresh = 9
>>>
>>> Then, re-run point_stat to see if it finds matched pairs for
TMP/Z2.
>>> If not, please let us know.
>>>
>>> Thanks,
>>>
>>> Paul
>>>
>>>
>>> On 05/27/2011 07:37 AM, RAL HelpDesk {for Jeff Zielonka} wrote:
>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129>
>>>>
>>>> Paul,
>>>>
>>>> No worries. I have other things that can keep me occupied while
>>>> you're
>>>> working on this. Thanks a lot for the help and the update.
>>>>
>>>> - Jeff
>>>>
>>>>
>>>> On 5/26/2011 7:02 PM, RAL HelpDesk {for Paul Oldenburg} wrote:
>>>>> Jeff,
>>>>>
>>>>> Sorry on the delay getting back to you. We are working on what
>>>>> appears to be a bug in pb2nc or some other strangeness
>>>>> with PrepBUFR observations. I'll let you know when we figure it
out.
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Paul
>>>>>
>>>>>
>>>>> On 05/25/2011 12:58 PM, RAL HelpDesk {for Jeff Zielonka} wrote:
>>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129>
>>>>>>
>>>>>> Hi Paul,
>>>>>>
>>>>>> I'm currently using a time window of -1800 to 1800. I
increased the
>>>>>> verbosity level when I ran point_stat and it says there are 0
obs
>>>>>> that
>>>>>> are being rejected because of the valid time.
>>>>>>
>>>>>> I have uploaded a grib file, the corresponding obs file, and
the
>>>>>> config
>>>>>> file to the ftp site. Note, when I ran pb2nc for this
particular
>>>>>> obs
>>>>>> file, I only retained ADPUPA and ADPSFC observations. I also
have
>>>>>> an
>>>>>> additional masking grid defined in my config file, so you can
just
>>>>>> get
>>>>>> rid of the PHR2 and keep FULL.
>>>>>>
>>>>>> Thanks for your help.
>>>>>>
>>>>>> - Jeff
>>>>>>
>>>>>>
>>>>>> On 5/25/2011 2:23 PM, RAL HelpDesk {for Paul Oldenburg} wrote:
>>>>>>> Jeff,
>>>>>>>
>>>>>>> I would first suggest using the point_stat options
-obs_valid_beg
>>>>>>> and -obs_valid_end to open the time window for matched
>>>>>>> observations to be very large. If this results in matched
pairs,
>>>>>>> then dump out those matched pairs by setting flag #15
>>>>>>> in the point_stat config file setting output_flag to 1 or 2.
Then,
>>>>>>> you can see what the valid time for the matched
>>>>>>> pairs actually is and adjust your setup accordingly.
>>>>>>>
>>>>>>> If that course of action is not fruitful, please send us the
>>>>>>> following so that we may try to reproduce and figure out
>>>>>>> your problem:
>>>>>>>
>>>>>>> 1. Model data
>>>>>>> 2. Obs data
>>>>>>> 3. point_stat config file
>>>>>>>
>>>>>>> If these files are too large to include in an email, please
upload
>>>>>>> them to our FTP site using the instructions here:
>>>>>>>
>>>>>>> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>>>>>>>
>>>>>>> If you have any questions, please let me know.
>>>>>>>
>>>>>>> Paul
>>>>>>>
>>>>>>> On 05/25/2011 12:10 PM, RAL HelpDesk {for Jeff Zielonka}
wrote:
>>>>>>>> Wed May 25 12:10:14 2011: Request 47129 was acted upon.
>>>>>>>> Transaction: Ticket created by zielonka at meteo.psu.edu
>>>>>>>> Queue: met_help
>>>>>>>> Subject: Problem using GDAS ADPSFC observations
>>>>>>>> Owner: Nobody
>>>>>>>> Requestors: zielonka at meteo.psu.edu
>>>>>>>> Status: new
>>>>>>>> Ticket<URL:
>>>>>>>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129>
>>>>>>>>
>>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I'm having a problem with point_stat in METv3.0 when
attempting to
>>>>>>>> use
>>>>>>>> ADPSFC data from GDAS PREPBUFR files. For example, when I
>>>>>>>> attempt to
>>>>>>>> verify TMP/Z2 against ADPSFC observations, I get zero matched
>>>>>>>> pairs. If
>>>>>>>> I use the ONLYSF option, it will use the SFCSHP observations,
but
>>>>>>>> not
>>>>>>>> ADPSFC.
>>>>>>>>
>>>>>>>> I know the ADPSFC observations exist in the netCDF files that
were
>>>>>>>> created by PB2NC, because there are many ADPSFC obs in the
hdr_typ
>>>>>>>> array
>>>>>>>> when I ncdump any particular obs file.
>>>>>>>>
>>>>>>>> I looked through the known issues section of the website and
saw
>>>>>>>> the
>>>>>>>> following,But when I differenced the updated pair_data.cc
file
>>>>>>>> with the
>>>>>>>> one I have, they are identical, so I must have made that
update
>>>>>>>> previously.
>>>>>>>>
>>>>>>>> One interesting thing to note is that when I use NDAS
observations
>>>>>>>> instead of GDAS observations, this issue does not occur. The
>>>>>>>> model
>>>>>>>> fields verify against ADPSFC observations just fine. So I'm
not
>>>>>>>> sure
>>>>>>>> what the issue might be. Unfortunately, I'm using this data
for
>>>>>>>> an old
>>>>>>>> case, so I do not have access to NDAS observations for this
>>>>>>>> period.
>>>>>>>>
>>>>>>>> Any ideas? Thanks.
>>>>>>>>
>>>>>>>>
>>>>>>>> - Jeff Z.
>>>>>>>>
>>>>>>>>
>>>>>>>> Point-Stat Tool
>>>>>>>>
>>>>>>>>
>>>>>>>> Vertical level matching logic when using a generic
level
>>>>>>>> type.
>>>>>>>> Posted 10/05/2010
>>>>>>>>
>>>>>>>> *Problem:*When verifying generic level type data (e.g.
"RH/L1")
>>>>>>>> against
>>>>>>>> a surface observation message type (e.g. "ADPSFC"), Point-
Stat was
>>>>>>>> finding zero matched pairs.
>>>>>>>> *Solution:*The fix is to no longer enforce that the
observation
>>>>>>>> level
>>>>>>>> value exactly match the forecast level value when verifying
>>>>>>>> against
>>>>>>>> surface observation message types. This makes the behavior in
>>>>>>>> METv3.0
>>>>>>>> match that of METv2.0.
>>>>>>>> Update:METv3.0/lib/vx_met_util/pair_data.cc
>>>>>>>>
<http://www.dtcenter.org/met/users/support/known_issues/METv3.0/patches/lib/vx_met_util/pair_data.cc>
>>>>>>>> And then recompile MET.
>>>
>>
>
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #47129] Problem using GDAS ADPSFC observations
From: Jeff Zielonka
Time: Fri May 27 14:03:24 2011
John,
I generally use NDAS observations when I'm verifying recent model
runs,
which is why I haven't seen this problem previously. For this
particular set of runs, however, the GDAS observations are what were
downloaded previously for this set of late 2007 to early 2008 cases,
and
I couldn't find an archive of NDAS observations to replace them for
the
given period.
FYI, I'm currently setting up a verification system that will be
running
every hour using a combination of NDAS obs for 12-hourly upper air
verification and MADIS obs (using the converter you gave me) for
hourly
surface verification. I've just found it much easier to run madis2nc
on
hourly obs files than running several iterations of pb2nc to get
hourly
obs files out of the 6-hourly NDAS files.
What I don't quite understand is why the ADPSFC observations in GDAS
would be any different than the ADPSFC obs in NDAS, assuming you're
verifying over the same region within the boundaries of the NDAS
system. You're right that processing NDAS obs is much faster than
GDAS,
but ultimately you should be using the same observation sites (unless
I'm overlooking something), so using GDAS shouldn't degrade your
verification just because they don't work very well in the global data
assimilation system.
It's possible that using a list of quality marks could be useful, but
I
would really have to look at the table referenced in the configuration
file to see what the differences are for the various marks, and how
much
of a difference that would really make.
- Jeff
On 5/27/2011 3:45 PM, RAL HelpDesk {for John Halley Gotway} wrote:
> Jeff,
>
> This is John Halley Gotway, and I wanted to chime in here with a
little
> background on this issue.
>
> In the DTC, we do reference configuration testing for the various
releases
> of the WRF model. We had been using GDAS point observations for the
> verification portion of this. However, after we discovered that
PB2NC bug
> in the event stack flag (see MET known issues page for details), we
found
> the same issue you are having with the GDAS point observations.
NCEP is
> setting the quality marker for most of the GDAS surface observations
to a
> value of 9. They're doing this to suppress the use of those
observations
> in their data assimilation system, which must be having a negative
impact
> of the assimilation. The impact on the verification is an
unfortunate
> consequence. This hadn't shown up before because the event stack
flag bug
> was masking the behavior.
>
> We spoke with the folks at NCEP who do verification of the
operational
> models. And they use the NDAS point observations for verifying over
CONUS
> rather than GDAS. This odd quality mark setting does not show up in
NDAS.
>
> So for our reference configuration testing efforts, we've switched
over to
> use the NDAS PREPBUFR files. An added advantage is that PB2NC runs
much
> faster on NDAS since there's a lot fewer observations to process.
Unless
> you have a good reason not to use the NDAS obs for verifying over
CONUS,
> I'd suggest doing so.
>
> Right now PB2NC only filters quality marks using a threshold,
keeping all
> obs whose quality mark is less than or equal to the threshold.
Perhaps,
> it'd be preferable to allow for a list of quality marks to be used.
For
> example, you may want to keep obs whose quality marks are 0, 1, 2,
or 9.
> Right now, there's no support for the in PB2NC. But it'd be a
pretty easy
> extension. Would you find this functionality useful?
>
> Hope that helps.
>
> John
>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129>
>>
>> Jeff,
>>
>> During my experiment, point_stat found 3652 matched pairs for
TMP/Z2,
>> which seems commensurate to me. The
>> quality_mark_threshold is definitely an obscure feature of pb2nc.
I
>> wasn't terribly familiar with it myself before I
>> worked on this problem.
>>
>> Paul
>>
>>
>> On 05/27/2011 01:20 PM, RAL HelpDesk {for Jeff Zielonka} wrote:
>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129>
>>>
>>> Paul,
>>>
>>> It looks like this did the trick. I re-ran pb2nc and point_stat,
and
>>> this resulted in 3655 matched pairs for TMP/Z2 on the full domain,
and
>>> 320 on the smaller subdomain that you wouldn't have seen. Are
these
>>> 3655 pairs consistent (within reason) with what you found?
>>>
>>> Thanks again for you help with this. I don't think I would have
been
>>> able to track down this issue on my own. I've never even thought
about
>>> adjusting the default value for quality_mark_thresh.
>>>
>>> I'm assuming this will solve my issues, but I'll let you know if I
run
>>> into any snags in the next week.
>>>
>>> - Jeff
>>>
>>>
>>>
>>> On 5/27/2011 2:36 PM, RAL HelpDesk {for Paul Oldenburg} wrote:
>>>> Jeff,
>>>>
>>>> The GDAS obs data contains ADPSFC TMP observations whose quality
mark
>>>> may be higher than that of other observations.
>>>> I'm not totally familiar with quality marks, but I do know that
pb2nc
>>>> will only pull observations for data whose quality
>>>> mark is less than the threshold stored in the setting
>>>> quality_mark_thresh. I downloaded the GDAS PrepBUFR obs file for
>>>> 2007100500 and then ran it through pb2nc. I was able to
reproduce the
>>>> problem that you reported with 0 matched pairs
>>>> found for TMP/Z2. My colleague John suggested re-running pb2nc
and
>>>> raising the quality_mark_thresh to 9 in the pb2nc
>>>> configuration file. When I did so, point_stat found matched
pairs when
>>>> I ran it with the new pb2nc NetCDF output file.
>>>>
>>>> Please try adjusting your pb2nc configuration file to use this
setting:
>>>>
>>>> quality_mark_thresh = 9
>>>>
>>>> Then, re-run point_stat to see if it finds matched pairs for
TMP/Z2.
>>>> If not, please let us know.
>>>>
>>>> Thanks,
>>>>
>>>> Paul
>>>>
>>>>
>>>> On 05/27/2011 07:37 AM, RAL HelpDesk {for Jeff Zielonka} wrote:
>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129>
>>>>>
>>>>> Paul,
>>>>>
>>>>> No worries. I have other things that can keep me occupied while
>>>>> you're
>>>>> working on this. Thanks a lot for the help and the update.
>>>>>
>>>>> - Jeff
>>>>>
>>>>>
>>>>> On 5/26/2011 7:02 PM, RAL HelpDesk {for Paul Oldenburg} wrote:
>>>>>> Jeff,
>>>>>>
>>>>>> Sorry on the delay getting back to you. We are working on what
>>>>>> appears to be a bug in pb2nc or some other strangeness
>>>>>> with PrepBUFR observations. I'll let you know when we figure
it out.
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Paul
>>>>>>
>>>>>>
>>>>>> On 05/25/2011 12:58 PM, RAL HelpDesk {for Jeff Zielonka} wrote:
>>>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129>
>>>>>>>
>>>>>>> Hi Paul,
>>>>>>>
>>>>>>> I'm currently using a time window of -1800 to 1800. I
increased the
>>>>>>> verbosity level when I ran point_stat and it says there are 0
obs
>>>>>>> that
>>>>>>> are being rejected because of the valid time.
>>>>>>>
>>>>>>> I have uploaded a grib file, the corresponding obs file, and
the
>>>>>>> config
>>>>>>> file to the ftp site. Note, when I ran pb2nc for this
particular
>>>>>>> obs
>>>>>>> file, I only retained ADPUPA and ADPSFC observations. I also
have
>>>>>>> an
>>>>>>> additional masking grid defined in my config file, so you can
just
>>>>>>> get
>>>>>>> rid of the PHR2 and keep FULL.
>>>>>>>
>>>>>>> Thanks for your help.
>>>>>>>
>>>>>>> - Jeff
>>>>>>>
>>>>>>>
>>>>>>> On 5/25/2011 2:23 PM, RAL HelpDesk {for Paul Oldenburg} wrote:
>>>>>>>> Jeff,
>>>>>>>>
>>>>>>>> I would first suggest using the point_stat options
-obs_valid_beg
>>>>>>>> and -obs_valid_end to open the time window for matched
>>>>>>>> observations to be very large. If this results in matched
pairs,
>>>>>>>> then dump out those matched pairs by setting flag #15
>>>>>>>> in the point_stat config file setting output_flag to 1 or 2.
Then,
>>>>>>>> you can see what the valid time for the matched
>>>>>>>> pairs actually is and adjust your setup accordingly.
>>>>>>>>
>>>>>>>> If that course of action is not fruitful, please send us the
>>>>>>>> following so that we may try to reproduce and figure out
>>>>>>>> your problem:
>>>>>>>>
>>>>>>>> 1. Model data
>>>>>>>> 2. Obs data
>>>>>>>> 3. point_stat config file
>>>>>>>>
>>>>>>>> If these files are too large to include in an email, please
upload
>>>>>>>> them to our FTP site using the instructions here:
>>>>>>>>
>>>>>>>> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>>>>>>>>
>>>>>>>> If you have any questions, please let me know.
>>>>>>>>
>>>>>>>> Paul
>>>>>>>>
>>>>>>>> On 05/25/2011 12:10 PM, RAL HelpDesk {for Jeff Zielonka}
wrote:
>>>>>>>>> Wed May 25 12:10:14 2011: Request 47129 was acted upon.
>>>>>>>>> Transaction: Ticket created by zielonka at meteo.psu.edu
>>>>>>>>> Queue: met_help
>>>>>>>>> Subject: Problem using GDAS ADPSFC observations
>>>>>>>>> Owner: Nobody
>>>>>>>>> Requestors: zielonka at meteo.psu.edu
>>>>>>>>> Status: new
>>>>>>>>> Ticket<URL:
>>>>>>>>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> I'm having a problem with point_stat in METv3.0 when
attempting to
>>>>>>>>> use
>>>>>>>>> ADPSFC data from GDAS PREPBUFR files. For example, when I
>>>>>>>>> attempt to
>>>>>>>>> verify TMP/Z2 against ADPSFC observations, I get zero
matched
>>>>>>>>> pairs. If
>>>>>>>>> I use the ONLYSF option, it will use the SFCSHP
observations, but
>>>>>>>>> not
>>>>>>>>> ADPSFC.
>>>>>>>>>
>>>>>>>>> I know the ADPSFC observations exist in the netCDF files
that were
>>>>>>>>> created by PB2NC, because there are many ADPSFC obs in the
hdr_typ
>>>>>>>>> array
>>>>>>>>> when I ncdump any particular obs file.
>>>>>>>>>
>>>>>>>>> I looked through the known issues section of the website and
saw
>>>>>>>>> the
>>>>>>>>> following,But when I differenced the updated pair_data.cc
file
>>>>>>>>> with the
>>>>>>>>> one I have, they are identical, so I must have made that
update
>>>>>>>>> previously.
>>>>>>>>>
>>>>>>>>> One interesting thing to note is that when I use NDAS
observations
>>>>>>>>> instead of GDAS observations, this issue does not occur.
The
>>>>>>>>> model
>>>>>>>>> fields verify against ADPSFC observations just fine. So I'm
not
>>>>>>>>> sure
>>>>>>>>> what the issue might be. Unfortunately, I'm using this data
for
>>>>>>>>> an old
>>>>>>>>> case, so I do not have access to NDAS observations for this
>>>>>>>>> period.
>>>>>>>>>
>>>>>>>>> Any ideas? Thanks.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> - Jeff Z.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Point-Stat Tool
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Vertical level matching logic when using a generic
level
>>>>>>>>> type.
>>>>>>>>> Posted 10/05/2010
>>>>>>>>>
>>>>>>>>> *Problem:*When verifying generic level type data (e.g.
"RH/L1")
>>>>>>>>> against
>>>>>>>>> a surface observation message type (e.g. "ADPSFC"), Point-
Stat was
>>>>>>>>> finding zero matched pairs.
>>>>>>>>> *Solution:*The fix is to no longer enforce that the
observation
>>>>>>>>> level
>>>>>>>>> value exactly match the forecast level value when verifying
>>>>>>>>> against
>>>>>>>>> surface observation message types. This makes the behavior
in
>>>>>>>>> METv3.0
>>>>>>>>> match that of METv2.0.
>>>>>>>>> Update:METv3.0/lib/vx_met_util/pair_data.cc
>>>>>>>>>
<http://www.dtcenter.org/met/users/support/known_issues/METv3.0/patches/lib/vx_met_util/pair_data.cc>
>>>>>>>>> And then recompile MET.
>>
>
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #47129] Problem using GDAS ADPSFC observations
From: John Halley Gotway
Time: Fri May 27 14:41:55 2011
Jeff,
I only have a cursory understanding of the differences between the
NDAS and GDAS datasets. When this issue arose in our reference
configuration testing, we had a scientist contact NCEP to discuss the
differences. What she learned is basically that the NDAS and GDAS
data assimilation systems are different. You're right that the
observing locations over CONUS should be the same in both, but the
quality marks attached to those observations differ between the two.
GDAS assigns different quality marks to filter out which observations
should be used in that data assimilation system. So while I
assume the underlying obs values should be the same in both, the
differing quality mark settings will impact which ones are retained by
PB2NC and ultimately used in the verification by Point-Stat.
John
On 05/27/2011 02:03 PM, RAL HelpDesk {for Jeff Zielonka} wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129 >
>
> John,
>
> I generally use NDAS observations when I'm verifying recent model
runs,
> which is why I haven't seen this problem previously. For this
> particular set of runs, however, the GDAS observations are what were
> downloaded previously for this set of late 2007 to early 2008 cases,
and
> I couldn't find an archive of NDAS observations to replace them for
the
> given period.
>
> FYI, I'm currently setting up a verification system that will be
running
> every hour using a combination of NDAS obs for 12-hourly upper air
> verification and MADIS obs (using the converter you gave me) for
hourly
> surface verification. I've just found it much easier to run
madis2nc on
> hourly obs files than running several iterations of pb2nc to get
hourly
> obs files out of the 6-hourly NDAS files.
>
> What I don't quite understand is why the ADPSFC observations in GDAS
> would be any different than the ADPSFC obs in NDAS, assuming you're
> verifying over the same region within the boundaries of the NDAS
> system. You're right that processing NDAS obs is much faster than
GDAS,
> but ultimately you should be using the same observation sites
(unless
> I'm overlooking something), so using GDAS shouldn't degrade your
> verification just because they don't work very well in the global
data
> assimilation system.
>
> It's possible that using a list of quality marks could be useful,
but I
> would really have to look at the table referenced in the
configuration
> file to see what the differences are for the various marks, and how
much
> of a difference that would really make.
>
>
> - Jeff
>
>
> On 5/27/2011 3:45 PM, RAL HelpDesk {for John Halley Gotway} wrote:
>> Jeff,
>>
>> This is John Halley Gotway, and I wanted to chime in here with a
little
>> background on this issue.
>>
>> In the DTC, we do reference configuration testing for the various
releases
>> of the WRF model. We had been using GDAS point observations for
the
>> verification portion of this. However, after we discovered that
PB2NC bug
>> in the event stack flag (see MET known issues page for details), we
found
>> the same issue you are having with the GDAS point observations.
NCEP is
>> setting the quality marker for most of the GDAS surface
observations to a
>> value of 9. They're doing this to suppress the use of those
observations
>> in their data assimilation system, which must be having a negative
impact
>> of the assimilation. The impact on the verification is an
unfortunate
>> consequence. This hadn't shown up before because the event stack
flag bug
>> was masking the behavior.
>>
>> We spoke with the folks at NCEP who do verification of the
operational
>> models. And they use the NDAS point observations for verifying
over CONUS
>> rather than GDAS. This odd quality mark setting does not show up
in NDAS.
>>
>> So for our reference configuration testing efforts, we've switched
over to
>> use the NDAS PREPBUFR files. An added advantage is that PB2NC runs
much
>> faster on NDAS since there's a lot fewer observations to process.
Unless
>> you have a good reason not to use the NDAS obs for verifying over
CONUS,
>> I'd suggest doing so.
>>
>> Right now PB2NC only filters quality marks using a threshold,
keeping all
>> obs whose quality mark is less than or equal to the threshold.
Perhaps,
>> it'd be preferable to allow for a list of quality marks to be used.
For
>> example, you may want to keep obs whose quality marks are 0, 1, 2,
or 9.
>> Right now, there's no support for the in PB2NC. But it'd be a
pretty easy
>> extension. Would you find this functionality useful?
>>
>> Hope that helps.
>>
>> John
>>
>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129>
>>>
>>> Jeff,
>>>
>>> During my experiment, point_stat found 3652 matched pairs for
TMP/Z2,
>>> which seems commensurate to me. The
>>> quality_mark_threshold is definitely an obscure feature of pb2nc.
I
>>> wasn't terribly familiar with it myself before I
>>> worked on this problem.
>>>
>>> Paul
>>>
>>>
>>> On 05/27/2011 01:20 PM, RAL HelpDesk {for Jeff Zielonka} wrote:
>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129>
>>>>
>>>> Paul,
>>>>
>>>> It looks like this did the trick. I re-ran pb2nc and point_stat,
and
>>>> this resulted in 3655 matched pairs for TMP/Z2 on the full
domain, and
>>>> 320 on the smaller subdomain that you wouldn't have seen. Are
these
>>>> 3655 pairs consistent (within reason) with what you found?
>>>>
>>>> Thanks again for you help with this. I don't think I would have
been
>>>> able to track down this issue on my own. I've never even thought
about
>>>> adjusting the default value for quality_mark_thresh.
>>>>
>>>> I'm assuming this will solve my issues, but I'll let you know if
I run
>>>> into any snags in the next week.
>>>>
>>>> - Jeff
>>>>
>>>>
>>>>
>>>> On 5/27/2011 2:36 PM, RAL HelpDesk {for Paul Oldenburg} wrote:
>>>>> Jeff,
>>>>>
>>>>> The GDAS obs data contains ADPSFC TMP observations whose quality
mark
>>>>> may be higher than that of other observations.
>>>>> I'm not totally familiar with quality marks, but I do know that
pb2nc
>>>>> will only pull observations for data whose quality
>>>>> mark is less than the threshold stored in the setting
>>>>> quality_mark_thresh. I downloaded the GDAS PrepBUFR obs file
for
>>>>> 2007100500 and then ran it through pb2nc. I was able to
reproduce the
>>>>> problem that you reported with 0 matched pairs
>>>>> found for TMP/Z2. My colleague John suggested re-running pb2nc
and
>>>>> raising the quality_mark_thresh to 9 in the pb2nc
>>>>> configuration file. When I did so, point_stat found matched
pairs when
>>>>> I ran it with the new pb2nc NetCDF output file.
>>>>>
>>>>> Please try adjusting your pb2nc configuration file to use this
setting:
>>>>>
>>>>> quality_mark_thresh = 9
>>>>>
>>>>> Then, re-run point_stat to see if it finds matched pairs for
TMP/Z2.
>>>>> If not, please let us know.
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Paul
>>>>>
>>>>>
>>>>> On 05/27/2011 07:37 AM, RAL HelpDesk {for Jeff Zielonka} wrote:
>>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129>
>>>>>>
>>>>>> Paul,
>>>>>>
>>>>>> No worries. I have other things that can keep me occupied
while
>>>>>> you're
>>>>>> working on this. Thanks a lot for the help and the update.
>>>>>>
>>>>>> - Jeff
>>>>>>
>>>>>>
>>>>>> On 5/26/2011 7:02 PM, RAL HelpDesk {for Paul Oldenburg} wrote:
>>>>>>> Jeff,
>>>>>>>
>>>>>>> Sorry on the delay getting back to you. We are working on
what
>>>>>>> appears to be a bug in pb2nc or some other strangeness
>>>>>>> with PrepBUFR observations. I'll let you know when we figure
it out.
>>>>>>>
>>>>>>> Thanks,
>>>>>>>
>>>>>>> Paul
>>>>>>>
>>>>>>>
>>>>>>> On 05/25/2011 12:58 PM, RAL HelpDesk {for Jeff Zielonka}
wrote:
>>>>>>>> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129>
>>>>>>>>
>>>>>>>> Hi Paul,
>>>>>>>>
>>>>>>>> I'm currently using a time window of -1800 to 1800. I
increased the
>>>>>>>> verbosity level when I ran point_stat and it says there are 0
obs
>>>>>>>> that
>>>>>>>> are being rejected because of the valid time.
>>>>>>>>
>>>>>>>> I have uploaded a grib file, the corresponding obs file, and
the
>>>>>>>> config
>>>>>>>> file to the ftp site. Note, when I ran pb2nc for this
particular
>>>>>>>> obs
>>>>>>>> file, I only retained ADPUPA and ADPSFC observations. I also
have
>>>>>>>> an
>>>>>>>> additional masking grid defined in my config file, so you can
just
>>>>>>>> get
>>>>>>>> rid of the PHR2 and keep FULL.
>>>>>>>>
>>>>>>>> Thanks for your help.
>>>>>>>>
>>>>>>>> - Jeff
>>>>>>>>
>>>>>>>>
>>>>>>>> On 5/25/2011 2:23 PM, RAL HelpDesk {for Paul Oldenburg}
wrote:
>>>>>>>>> Jeff,
>>>>>>>>>
>>>>>>>>> I would first suggest using the point_stat options
-obs_valid_beg
>>>>>>>>> and -obs_valid_end to open the time window for matched
>>>>>>>>> observations to be very large. If this results in matched
pairs,
>>>>>>>>> then dump out those matched pairs by setting flag #15
>>>>>>>>> in the point_stat config file setting output_flag to 1 or 2.
Then,
>>>>>>>>> you can see what the valid time for the matched
>>>>>>>>> pairs actually is and adjust your setup accordingly.
>>>>>>>>>
>>>>>>>>> If that course of action is not fruitful, please send us the
>>>>>>>>> following so that we may try to reproduce and figure out
>>>>>>>>> your problem:
>>>>>>>>>
>>>>>>>>> 1. Model data
>>>>>>>>> 2. Obs data
>>>>>>>>> 3. point_stat config file
>>>>>>>>>
>>>>>>>>> If these files are too large to include in an email, please
upload
>>>>>>>>> them to our FTP site using the instructions here:
>>>>>>>>>
>>>>>>>>> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>>>>>>>>>
>>>>>>>>> If you have any questions, please let me know.
>>>>>>>>>
>>>>>>>>> Paul
>>>>>>>>>
>>>>>>>>> On 05/25/2011 12:10 PM, RAL HelpDesk {for Jeff Zielonka}
wrote:
>>>>>>>>>> Wed May 25 12:10:14 2011: Request 47129 was acted upon.
>>>>>>>>>> Transaction: Ticket created by zielonka at meteo.psu.edu
>>>>>>>>>> Queue: met_help
>>>>>>>>>> Subject: Problem using GDAS ADPSFC observations
>>>>>>>>>> Owner: Nobody
>>>>>>>>>> Requestors: zielonka at meteo.psu.edu
>>>>>>>>>> Status: new
>>>>>>>>>> Ticket<URL:
>>>>>>>>>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> I'm having a problem with point_stat in METv3.0 when
attempting to
>>>>>>>>>> use
>>>>>>>>>> ADPSFC data from GDAS PREPBUFR files. For example, when I
>>>>>>>>>> attempt to
>>>>>>>>>> verify TMP/Z2 against ADPSFC observations, I get zero
matched
>>>>>>>>>> pairs. If
>>>>>>>>>> I use the ONLYSF option, it will use the SFCSHP
observations, but
>>>>>>>>>> not
>>>>>>>>>> ADPSFC.
>>>>>>>>>>
>>>>>>>>>> I know the ADPSFC observations exist in the netCDF files
that were
>>>>>>>>>> created by PB2NC, because there are many ADPSFC obs in the
hdr_typ
>>>>>>>>>> array
>>>>>>>>>> when I ncdump any particular obs file.
>>>>>>>>>>
>>>>>>>>>> I looked through the known issues section of the website
and saw
>>>>>>>>>> the
>>>>>>>>>> following,But when I differenced the updated pair_data.cc
file
>>>>>>>>>> with the
>>>>>>>>>> one I have, they are identical, so I must have made that
update
>>>>>>>>>> previously.
>>>>>>>>>>
>>>>>>>>>> One interesting thing to note is that when I use NDAS
observations
>>>>>>>>>> instead of GDAS observations, this issue does not occur.
The
>>>>>>>>>> model
>>>>>>>>>> fields verify against ADPSFC observations just fine. So
I'm not
>>>>>>>>>> sure
>>>>>>>>>> what the issue might be. Unfortunately, I'm using this
data for
>>>>>>>>>> an old
>>>>>>>>>> case, so I do not have access to NDAS observations for this
>>>>>>>>>> period.
>>>>>>>>>>
>>>>>>>>>> Any ideas? Thanks.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> - Jeff Z.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Point-Stat Tool
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Vertical level matching logic when using a
generic level
>>>>>>>>>> type.
>>>>>>>>>> Posted 10/05/2010
>>>>>>>>>>
>>>>>>>>>> *Problem:*When verifying generic level type data (e.g.
"RH/L1")
>>>>>>>>>> against
>>>>>>>>>> a surface observation message type (e.g. "ADPSFC"), Point-
Stat was
>>>>>>>>>> finding zero matched pairs.
>>>>>>>>>> *Solution:*The fix is to no longer enforce that the
observation
>>>>>>>>>> level
>>>>>>>>>> value exactly match the forecast level value when verifying
>>>>>>>>>> against
>>>>>>>>>> surface observation message types. This makes the behavior
in
>>>>>>>>>> METv3.0
>>>>>>>>>> match that of METv2.0.
>>>>>>>>>> Update:METv3.0/lib/vx_met_util/pair_data.cc
>>>>>>>>>>
<http://www.dtcenter.org/met/users/support/known_issues/METv3.0/patches/lib/vx_met_util/pair_data.cc>
>>>>>>>>>> And then recompile MET.
>>>
>>
>>
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #47129] Problem using GDAS ADPSFC observations
From: Jeff Zielonka
Time: Fri May 27 14:45:49 2011
John,
This is great to know. So as long as I know which quality marks to
use
for the different data sets, I should be good to go for now.
Thanks for all your help, yet again!
- Jeff
On 5/27/2011 4:41 PM, RAL HelpDesk {for John Halley Gotway} wrote:
> Jeff,
>
> I only have a cursory understanding of the differences between the
NDAS and GDAS datasets. When this issue arose in our reference
configuration testing, we had a scientist contact NCEP to discuss the
> differences. What she learned is basically that the NDAS and GDAS
data assimilation systems are different. You're right that the
observing locations over CONUS should be the same in both, but the
> quality marks attached to those observations differ between the two.
GDAS assigns different quality marks to filter out which observations
should be used in that data assimilation system. So while I
> assume the underlying obs values should be the same in both, the
differing quality mark settings will impact which ones are retained by
PB2NC and ultimately used in the verification by Point-Stat.
>
> John
>
> On 05/27/2011 02:03 PM, RAL HelpDesk {for Jeff Zielonka} wrote:
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129>
>>
>> John,
>>
>> I generally use NDAS observations when I'm verifying recent model
runs,
>> which is why I haven't seen this problem previously. For this
>> particular set of runs, however, the GDAS observations are what
were
>> downloaded previously for this set of late 2007 to early 2008
cases, and
>> I couldn't find an archive of NDAS observations to replace them for
the
>> given period.
>>
>> FYI, I'm currently setting up a verification system that will be
running
>> every hour using a combination of NDAS obs for 12-hourly upper air
>> verification and MADIS obs (using the converter you gave me) for
hourly
>> surface verification. I've just found it much easier to run
madis2nc on
>> hourly obs files than running several iterations of pb2nc to get
hourly
>> obs files out of the 6-hourly NDAS files.
>>
>> What I don't quite understand is why the ADPSFC observations in
GDAS
>> would be any different than the ADPSFC obs in NDAS, assuming you're
>> verifying over the same region within the boundaries of the NDAS
>> system. You're right that processing NDAS obs is much faster than
GDAS,
>> but ultimately you should be using the same observation sites
(unless
>> I'm overlooking something), so using GDAS shouldn't degrade your
>> verification just because they don't work very well in the global
data
>> assimilation system.
>>
>> It's possible that using a list of quality marks could be useful,
but I
>> would really have to look at the table referenced in the
configuration
>> file to see what the differences are for the various marks, and how
much
>> of a difference that would really make.
>>
>>
>> - Jeff
>>
>>
>> On 5/27/2011 3:45 PM, RAL HelpDesk {for John Halley Gotway} wrote:
>>> Jeff,
>>>
>>> This is John Halley Gotway, and I wanted to chime in here with a
little
>>> background on this issue.
>>>
>>> In the DTC, we do reference configuration testing for the various
releases
>>> of the WRF model. We had been using GDAS point observations for
the
>>> verification portion of this. However, after we discovered that
PB2NC bug
>>> in the event stack flag (see MET known issues page for details),
we found
>>> the same issue you are having with the GDAS point observations.
NCEP is
>>> setting the quality marker for most of the GDAS surface
observations to a
>>> value of 9. They're doing this to suppress the use of those
observations
>>> in their data assimilation system, which must be having a negative
impact
>>> of the assimilation. The impact on the verification is an
unfortunate
>>> consequence. This hadn't shown up before because the event stack
flag bug
>>> was masking the behavior.
>>>
>>> We spoke with the folks at NCEP who do verification of the
operational
>>> models. And they use the NDAS point observations for verifying
over CONUS
>>> rather than GDAS. This odd quality mark setting does not show up
in NDAS.
>>>
>>> So for our reference configuration testing efforts, we've switched
over to
>>> use the NDAS PREPBUFR files. An added advantage is that PB2NC
runs much
>>> faster on NDAS since there's a lot fewer observations to process.
Unless
>>> you have a good reason not to use the NDAS obs for verifying over
CONUS,
>>> I'd suggest doing so.
>>>
>>> Right now PB2NC only filters quality marks using a threshold,
keeping all
>>> obs whose quality mark is less than or equal to the threshold.
Perhaps,
>>> it'd be preferable to allow for a list of quality marks to be
used. For
>>> example, you may want to keep obs whose quality marks are 0, 1, 2,
or 9.
>>> Right now, there's no support for the in PB2NC. But it'd be a
pretty easy
>>> extension. Would you find this functionality useful?
>>>
>>> Hope that helps.
>>>
>>> John
>>>
>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129>
>>>>
>>>> Jeff,
>>>>
>>>> During my experiment, point_stat found 3652 matched pairs for
TMP/Z2,
>>>> which seems commensurate to me. The
>>>> quality_mark_threshold is definitely an obscure feature of pb2nc.
I
>>>> wasn't terribly familiar with it myself before I
>>>> worked on this problem.
>>>>
>>>> Paul
>>>>
>>>>
>>>> On 05/27/2011 01:20 PM, RAL HelpDesk {for Jeff Zielonka} wrote:
>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129>
>>>>>
>>>>> Paul,
>>>>>
>>>>> It looks like this did the trick. I re-ran pb2nc and
point_stat, and
>>>>> this resulted in 3655 matched pairs for TMP/Z2 on the full
domain, and
>>>>> 320 on the smaller subdomain that you wouldn't have seen. Are
these
>>>>> 3655 pairs consistent (within reason) with what you found?
>>>>>
>>>>> Thanks again for you help with this. I don't think I would have
been
>>>>> able to track down this issue on my own. I've never even
thought about
>>>>> adjusting the default value for quality_mark_thresh.
>>>>>
>>>>> I'm assuming this will solve my issues, but I'll let you know if
I run
>>>>> into any snags in the next week.
>>>>>
>>>>> - Jeff
>>>>>
>>>>>
>>>>>
>>>>> On 5/27/2011 2:36 PM, RAL HelpDesk {for Paul Oldenburg} wrote:
>>>>>> Jeff,
>>>>>>
>>>>>> The GDAS obs data contains ADPSFC TMP observations whose
quality mark
>>>>>> may be higher than that of other observations.
>>>>>> I'm not totally familiar with quality marks, but I do know that
pb2nc
>>>>>> will only pull observations for data whose quality
>>>>>> mark is less than the threshold stored in the setting
>>>>>> quality_mark_thresh. I downloaded the GDAS PrepBUFR obs file
for
>>>>>> 2007100500 and then ran it through pb2nc. I was able to
reproduce the
>>>>>> problem that you reported with 0 matched pairs
>>>>>> found for TMP/Z2. My colleague John suggested re-running pb2nc
and
>>>>>> raising the quality_mark_thresh to 9 in the pb2nc
>>>>>> configuration file. When I did so, point_stat found matched
pairs when
>>>>>> I ran it with the new pb2nc NetCDF output file.
>>>>>>
>>>>>> Please try adjusting your pb2nc configuration file to use this
setting:
>>>>>>
>>>>>> quality_mark_thresh = 9
>>>>>>
>>>>>> Then, re-run point_stat to see if it finds matched pairs for
TMP/Z2.
>>>>>> If not, please let us know.
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Paul
>>>>>>
>>>>>>
>>>>>> On 05/27/2011 07:37 AM, RAL HelpDesk {for Jeff Zielonka} wrote:
>>>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129>
>>>>>>>
>>>>>>> Paul,
>>>>>>>
>>>>>>> No worries. I have other things that can keep me occupied
while
>>>>>>> you're
>>>>>>> working on this. Thanks a lot for the help and the update.
>>>>>>>
>>>>>>> - Jeff
>>>>>>>
>>>>>>>
>>>>>>> On 5/26/2011 7:02 PM, RAL HelpDesk {for Paul Oldenburg} wrote:
>>>>>>>> Jeff,
>>>>>>>>
>>>>>>>> Sorry on the delay getting back to you. We are working on
what
>>>>>>>> appears to be a bug in pb2nc or some other strangeness
>>>>>>>> with PrepBUFR observations. I'll let you know when we figure
it out.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>>
>>>>>>>> Paul
>>>>>>>>
>>>>>>>>
>>>>>>>> On 05/25/2011 12:58 PM, RAL HelpDesk {for Jeff Zielonka}
wrote:
>>>>>>>>> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129>
>>>>>>>>>
>>>>>>>>> Hi Paul,
>>>>>>>>>
>>>>>>>>> I'm currently using a time window of -1800 to 1800. I
increased the
>>>>>>>>> verbosity level when I ran point_stat and it says there are
0 obs
>>>>>>>>> that
>>>>>>>>> are being rejected because of the valid time.
>>>>>>>>>
>>>>>>>>> I have uploaded a grib file, the corresponding obs file, and
the
>>>>>>>>> config
>>>>>>>>> file to the ftp site. Note, when I ran pb2nc for this
particular
>>>>>>>>> obs
>>>>>>>>> file, I only retained ADPUPA and ADPSFC observations. I
also have
>>>>>>>>> an
>>>>>>>>> additional masking grid defined in my config file, so you
can just
>>>>>>>>> get
>>>>>>>>> rid of the PHR2 and keep FULL.
>>>>>>>>>
>>>>>>>>> Thanks for your help.
>>>>>>>>>
>>>>>>>>> - Jeff
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 5/25/2011 2:23 PM, RAL HelpDesk {for Paul Oldenburg}
wrote:
>>>>>>>>>> Jeff,
>>>>>>>>>>
>>>>>>>>>> I would first suggest using the point_stat options
-obs_valid_beg
>>>>>>>>>> and -obs_valid_end to open the time window for matched
>>>>>>>>>> observations to be very large. If this results in matched
pairs,
>>>>>>>>>> then dump out those matched pairs by setting flag #15
>>>>>>>>>> in the point_stat config file setting output_flag to 1 or
2. Then,
>>>>>>>>>> you can see what the valid time for the matched
>>>>>>>>>> pairs actually is and adjust your setup accordingly.
>>>>>>>>>>
>>>>>>>>>> If that course of action is not fruitful, please send us
the
>>>>>>>>>> following so that we may try to reproduce and figure out
>>>>>>>>>> your problem:
>>>>>>>>>>
>>>>>>>>>> 1. Model data
>>>>>>>>>> 2. Obs data
>>>>>>>>>> 3. point_stat config file
>>>>>>>>>>
>>>>>>>>>> If these files are too large to include in an email, please
upload
>>>>>>>>>> them to our FTP site using the instructions here:
>>>>>>>>>>
>>>>>>>>>> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>>>>>>>>>>
>>>>>>>>>> If you have any questions, please let me know.
>>>>>>>>>>
>>>>>>>>>> Paul
>>>>>>>>>>
>>>>>>>>>> On 05/25/2011 12:10 PM, RAL HelpDesk {for Jeff Zielonka}
wrote:
>>>>>>>>>>> Wed May 25 12:10:14 2011: Request 47129 was acted upon.
>>>>>>>>>>> Transaction: Ticket created by zielonka at meteo.psu.edu
>>>>>>>>>>> Queue: met_help
>>>>>>>>>>> Subject: Problem using GDAS ADPSFC observations
>>>>>>>>>>> Owner: Nobody
>>>>>>>>>>> Requestors: zielonka at meteo.psu.edu
>>>>>>>>>>> Status: new
>>>>>>>>>>> Ticket<URL:
>>>>>>>>>>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> I'm having a problem with point_stat in METv3.0 when
attempting to
>>>>>>>>>>> use
>>>>>>>>>>> ADPSFC data from GDAS PREPBUFR files. For example, when
I
>>>>>>>>>>> attempt to
>>>>>>>>>>> verify TMP/Z2 against ADPSFC observations, I get zero
matched
>>>>>>>>>>> pairs. If
>>>>>>>>>>> I use the ONLYSF option, it will use the SFCSHP
observations, but
>>>>>>>>>>> not
>>>>>>>>>>> ADPSFC.
>>>>>>>>>>>
>>>>>>>>>>> I know the ADPSFC observations exist in the netCDF files
that were
>>>>>>>>>>> created by PB2NC, because there are many ADPSFC obs in the
hdr_typ
>>>>>>>>>>> array
>>>>>>>>>>> when I ncdump any particular obs file.
>>>>>>>>>>>
>>>>>>>>>>> I looked through the known issues section of the website
and saw
>>>>>>>>>>> the
>>>>>>>>>>> following,But when I differenced the updated pair_data.cc
file
>>>>>>>>>>> with the
>>>>>>>>>>> one I have, they are identical, so I must have made that
update
>>>>>>>>>>> previously.
>>>>>>>>>>>
>>>>>>>>>>> One interesting thing to note is that when I use NDAS
observations
>>>>>>>>>>> instead of GDAS observations, this issue does not occur.
The
>>>>>>>>>>> model
>>>>>>>>>>> fields verify against ADPSFC observations just fine. So
I'm not
>>>>>>>>>>> sure
>>>>>>>>>>> what the issue might be. Unfortunately, I'm using this
data for
>>>>>>>>>>> an old
>>>>>>>>>>> case, so I do not have access to NDAS observations for
this
>>>>>>>>>>> period.
>>>>>>>>>>>
>>>>>>>>>>> Any ideas? Thanks.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> - Jeff Z.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Point-Stat Tool
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Vertical level matching logic when using a
generic level
>>>>>>>>>>> type.
>>>>>>>>>>> Posted 10/05/2010
>>>>>>>>>>>
>>>>>>>>>>> *Problem:*When verifying generic level type data (e.g.
"RH/L1")
>>>>>>>>>>> against
>>>>>>>>>>> a surface observation message type (e.g. "ADPSFC"), Point-
Stat was
>>>>>>>>>>> finding zero matched pairs.
>>>>>>>>>>> *Solution:*The fix is to no longer enforce that the
observation
>>>>>>>>>>> level
>>>>>>>>>>> value exactly match the forecast level value when
verifying
>>>>>>>>>>> against
>>>>>>>>>>> surface observation message types. This makes the behavior
in
>>>>>>>>>>> METv3.0
>>>>>>>>>>> match that of METv2.0.
>>>>>>>>>>> Update:METv3.0/lib/vx_met_util/pair_data.cc
>>>>>>>>>>>
<http://www.dtcenter.org/met/users/support/known_issues/METv3.0/patches/lib/vx_met_util/pair_data.cc>
>>>>>>>>>>> And then recompile MET.
>>>
------------------------------------------------
More information about the Met_help
mailing list