[Met_help] [rt.rap.ucar.edu #47129] History for Problem using GDAS ADPSFC observations
RAL HelpDesk {for Paul Oldenburg}
met_help at ucar.edu
Fri May 27 13:31:11 MDT 2011
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hi,
I'm having a problem with point_stat in METv3.0 when attempting to use
ADPSFC data from GDAS PREPBUFR files. For example, when I attempt to
verify TMP/Z2 against ADPSFC observations, I get zero matched pairs. If
I use the ONLYSF option, it will use the SFCSHP observations, but not
ADPSFC.
I know the ADPSFC observations exist in the netCDF files that were
created by PB2NC, because there are many ADPSFC obs in the hdr_typ array
when I ncdump any particular obs file.
I looked through the known issues section of the website and saw the
following,But when I differenced the updated pair_data.cc file with the
one I have, they are identical, so I must have made that update previously.
One interesting thing to note is that when I use NDAS observations
instead of GDAS observations, this issue does not occur. The model
fields verify against ADPSFC observations just fine. So I'm not sure
what the issue might be. Unfortunately, I'm using this data for an old
case, so I do not have access to NDAS observations for this period.
Any ideas? Thanks.
- Jeff Z.
Point-Stat Tool
Vertical level matching logic when using a generic level type.
Posted 10/05/2010
*Problem:*When verifying generic level type data (e.g. "RH/L1") against
a surface observation message type (e.g. "ADPSFC"), Point-Stat was
finding zero matched pairs.
*Solution:*The fix is to no longer enforce that the observation level
value exactly match the forecast level value when verifying against
surface observation message types. This makes the behavior in METv3.0
match that of METv2.0.
Update:METv3.0/lib/vx_met_util/pair_data.cc
<http://www.dtcenter.org/met/users/support/known_issues/METv3.0/patches/lib/vx_met_util/pair_data.cc>
And then recompile MET.
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #47129] Problem using GDAS ADPSFC observations
From: Paul Oldenburg
Time: Wed May 25 12:23:17 2011
Jeff,
I would first suggest using the point_stat options -obs_valid_beg and
-obs_valid_end to open the time window for matched
observations to be very large. If this results in matched pairs, then
dump out those matched pairs by setting flag #15
in the point_stat config file setting output_flag to 1 or 2. Then,
you can see what the valid time for the matched
pairs actually is and adjust your setup accordingly.
If that course of action is not fruitful, please send us the following
so that we may try to reproduce and figure out
your problem:
1. Model data
2. Obs data
3. point_stat config file
If these files are too large to include in an email, please upload
them to our FTP site using the instructions here:
http://www.dtcenter.org/met/users/support/met_help.php#ftp
If you have any questions, please let me know.
Paul
On 05/25/2011 12:10 PM, RAL HelpDesk {for Jeff Zielonka} wrote:
>
> Wed May 25 12:10:14 2011: Request 47129 was acted upon.
> Transaction: Ticket created by zielonka at meteo.psu.edu
> Queue: met_help
> Subject: Problem using GDAS ADPSFC observations
> Owner: Nobody
> Requestors: zielonka at meteo.psu.edu
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129 >
>
>
> Hi,
>
> I'm having a problem with point_stat in METv3.0 when attempting to
use
> ADPSFC data from GDAS PREPBUFR files. For example, when I attempt
to
> verify TMP/Z2 against ADPSFC observations, I get zero matched pairs.
If
> I use the ONLYSF option, it will use the SFCSHP observations, but
not
> ADPSFC.
>
> I know the ADPSFC observations exist in the netCDF files that were
> created by PB2NC, because there are many ADPSFC obs in the hdr_typ
array
> when I ncdump any particular obs file.
>
> I looked through the known issues section of the website and saw the
> following,But when I differenced the updated pair_data.cc file with
the
> one I have, they are identical, so I must have made that update
previously.
>
> One interesting thing to note is that when I use NDAS observations
> instead of GDAS observations, this issue does not occur. The model
> fields verify against ADPSFC observations just fine. So I'm not
sure
> what the issue might be. Unfortunately, I'm using this data for an
old
> case, so I do not have access to NDAS observations for this period.
>
> Any ideas? Thanks.
>
>
> - Jeff Z.
>
>
> Point-Stat Tool
>
>
> Vertical level matching logic when using a generic level type.
> Posted 10/05/2010
>
> *Problem:*When verifying generic level type data (e.g. "RH/L1")
against
> a surface observation message type (e.g. "ADPSFC"), Point-Stat was
> finding zero matched pairs.
> *Solution:*The fix is to no longer enforce that the observation
level
> value exactly match the forecast level value when verifying against
> surface observation message types. This makes the behavior in
METv3.0
> match that of METv2.0.
> Update:METv3.0/lib/vx_met_util/pair_data.cc
>
<http://www.dtcenter.org/met/users/support/known_issues/METv3.0/patches/lib/vx_met_util/pair_data.cc>
> And then recompile MET.
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #47129] Problem using GDAS ADPSFC observations
From: Jeff Zielonka
Time: Wed May 25 12:58:21 2011
Hi Paul,
I'm currently using a time window of -1800 to 1800. I increased the
verbosity level when I ran point_stat and it says there are 0 obs that
are being rejected because of the valid time.
I have uploaded a grib file, the corresponding obs file, and the
config
file to the ftp site. Note, when I ran pb2nc for this particular obs
file, I only retained ADPUPA and ADPSFC observations. I also have an
additional masking grid defined in my config file, so you can just get
rid of the PHR2 and keep FULL.
Thanks for your help.
- Jeff
On 5/25/2011 2:23 PM, RAL HelpDesk {for Paul Oldenburg} wrote:
> Jeff,
>
> I would first suggest using the point_stat options -obs_valid_beg
and -obs_valid_end to open the time window for matched
> observations to be very large. If this results in matched pairs,
then dump out those matched pairs by setting flag #15
> in the point_stat config file setting output_flag to 1 or 2. Then,
you can see what the valid time for the matched
> pairs actually is and adjust your setup accordingly.
>
> If that course of action is not fruitful, please send us the
following so that we may try to reproduce and figure out
> your problem:
>
> 1. Model data
> 2. Obs data
> 3. point_stat config file
>
> If these files are too large to include in an email, please upload
them to our FTP site using the instructions here:
>
> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>
> If you have any questions, please let me know.
>
> Paul
>
> On 05/25/2011 12:10 PM, RAL HelpDesk {for Jeff Zielonka} wrote:
>> Wed May 25 12:10:14 2011: Request 47129 was acted upon.
>> Transaction: Ticket created by zielonka at meteo.psu.edu
>> Queue: met_help
>> Subject: Problem using GDAS ADPSFC observations
>> Owner: Nobody
>> Requestors: zielonka at meteo.psu.edu
>> Status: new
>> Ticket<URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129>
>>
>>
>> Hi,
>>
>> I'm having a problem with point_stat in METv3.0 when attempting to
use
>> ADPSFC data from GDAS PREPBUFR files. For example, when I attempt
to
>> verify TMP/Z2 against ADPSFC observations, I get zero matched
pairs. If
>> I use the ONLYSF option, it will use the SFCSHP observations, but
not
>> ADPSFC.
>>
>> I know the ADPSFC observations exist in the netCDF files that were
>> created by PB2NC, because there are many ADPSFC obs in the hdr_typ
array
>> when I ncdump any particular obs file.
>>
>> I looked through the known issues section of the website and saw
the
>> following,But when I differenced the updated pair_data.cc file with
the
>> one I have, they are identical, so I must have made that update
previously.
>>
>> One interesting thing to note is that when I use NDAS observations
>> instead of GDAS observations, this issue does not occur. The model
>> fields verify against ADPSFC observations just fine. So I'm not
sure
>> what the issue might be. Unfortunately, I'm using this data for an
old
>> case, so I do not have access to NDAS observations for this period.
>>
>> Any ideas? Thanks.
>>
>>
>> - Jeff Z.
>>
>>
>> Point-Stat Tool
>>
>>
>> Vertical level matching logic when using a generic level
type.
>> Posted 10/05/2010
>>
>> *Problem:*When verifying generic level type data (e.g. "RH/L1")
against
>> a surface observation message type (e.g. "ADPSFC"), Point-Stat was
>> finding zero matched pairs.
>> *Solution:*The fix is to no longer enforce that the observation
level
>> value exactly match the forecast level value when verifying against
>> surface observation message types. This makes the behavior in
METv3.0
>> match that of METv2.0.
>> Update:METv3.0/lib/vx_met_util/pair_data.cc
>>
<http://www.dtcenter.org/met/users/support/known_issues/METv3.0/patches/lib/vx_met_util/pair_data.cc>
>> And then recompile MET.
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #47129] Problem using GDAS ADPSFC observations
From: Paul Oldenburg
Time: Thu May 26 17:02:40 2011
Jeff,
Sorry on the delay getting back to you. We are working on what
appears to be a bug in pb2nc or some other strangeness
with PrepBUFR observations. I'll let you know when we figure it out.
Thanks,
Paul
On 05/25/2011 12:58 PM, RAL HelpDesk {for Jeff Zielonka} wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129 >
>
> Hi Paul,
>
> I'm currently using a time window of -1800 to 1800. I increased the
> verbosity level when I ran point_stat and it says there are 0 obs
that
> are being rejected because of the valid time.
>
> I have uploaded a grib file, the corresponding obs file, and the
config
> file to the ftp site. Note, when I ran pb2nc for this particular
obs
> file, I only retained ADPUPA and ADPSFC observations. I also have
an
> additional masking grid defined in my config file, so you can just
get
> rid of the PHR2 and keep FULL.
>
> Thanks for your help.
>
> - Jeff
>
>
> On 5/25/2011 2:23 PM, RAL HelpDesk {for Paul Oldenburg} wrote:
>> Jeff,
>>
>> I would first suggest using the point_stat options -obs_valid_beg
and -obs_valid_end to open the time window for matched
>> observations to be very large. If this results in matched pairs,
then dump out those matched pairs by setting flag #15
>> in the point_stat config file setting output_flag to 1 or 2. Then,
you can see what the valid time for the matched
>> pairs actually is and adjust your setup accordingly.
>>
>> If that course of action is not fruitful, please send us the
following so that we may try to reproduce and figure out
>> your problem:
>>
>> 1. Model data
>> 2. Obs data
>> 3. point_stat config file
>>
>> If these files are too large to include in an email, please upload
them to our FTP site using the instructions here:
>>
>> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>>
>> If you have any questions, please let me know.
>>
>> Paul
>>
>> On 05/25/2011 12:10 PM, RAL HelpDesk {for Jeff Zielonka} wrote:
>>> Wed May 25 12:10:14 2011: Request 47129 was acted upon.
>>> Transaction: Ticket created by zielonka at meteo.psu.edu
>>> Queue: met_help
>>> Subject: Problem using GDAS ADPSFC observations
>>> Owner: Nobody
>>> Requestors: zielonka at meteo.psu.edu
>>> Status: new
>>> Ticket<URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129>
>>>
>>>
>>> Hi,
>>>
>>> I'm having a problem with point_stat in METv3.0 when attempting to
use
>>> ADPSFC data from GDAS PREPBUFR files. For example, when I
attempt to
>>> verify TMP/Z2 against ADPSFC observations, I get zero matched
pairs. If
>>> I use the ONLYSF option, it will use the SFCSHP observations, but
not
>>> ADPSFC.
>>>
>>> I know the ADPSFC observations exist in the netCDF files that were
>>> created by PB2NC, because there are many ADPSFC obs in the hdr_typ
array
>>> when I ncdump any particular obs file.
>>>
>>> I looked through the known issues section of the website and saw
the
>>> following,But when I differenced the updated pair_data.cc file
with the
>>> one I have, they are identical, so I must have made that update
previously.
>>>
>>> One interesting thing to note is that when I use NDAS observations
>>> instead of GDAS observations, this issue does not occur. The
model
>>> fields verify against ADPSFC observations just fine. So I'm not
sure
>>> what the issue might be. Unfortunately, I'm using this data for
an old
>>> case, so I do not have access to NDAS observations for this
period.
>>>
>>> Any ideas? Thanks.
>>>
>>>
>>> - Jeff Z.
>>>
>>>
>>> Point-Stat Tool
>>>
>>>
>>> Vertical level matching logic when using a generic level
type.
>>> Posted 10/05/2010
>>>
>>> *Problem:*When verifying generic level type data (e.g. "RH/L1")
against
>>> a surface observation message type (e.g. "ADPSFC"), Point-Stat was
>>> finding zero matched pairs.
>>> *Solution:*The fix is to no longer enforce that the observation
level
>>> value exactly match the forecast level value when verifying
against
>>> surface observation message types. This makes the behavior in
METv3.0
>>> match that of METv2.0.
>>> Update:METv3.0/lib/vx_met_util/pair_data.cc
>>>
<http://www.dtcenter.org/met/users/support/known_issues/METv3.0/patches/lib/vx_met_util/pair_data.cc>
>>> And then recompile MET.
>>
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #47129] Problem using GDAS ADPSFC observations
From: Jeff Zielonka
Time: Fri May 27 07:37:47 2011
Paul,
No worries. I have other things that can keep me occupied while
you're
working on this. Thanks a lot for the help and the update.
- Jeff
On 5/26/2011 7:02 PM, RAL HelpDesk {for Paul Oldenburg} wrote:
> Jeff,
>
> Sorry on the delay getting back to you. We are working on what
appears to be a bug in pb2nc or some other strangeness
> with PrepBUFR observations. I'll let you know when we figure it
out.
>
> Thanks,
>
> Paul
>
>
> On 05/25/2011 12:58 PM, RAL HelpDesk {for Jeff Zielonka} wrote:
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129>
>>
>> Hi Paul,
>>
>> I'm currently using a time window of -1800 to 1800. I increased
the
>> verbosity level when I ran point_stat and it says there are 0 obs
that
>> are being rejected because of the valid time.
>>
>> I have uploaded a grib file, the corresponding obs file, and the
config
>> file to the ftp site. Note, when I ran pb2nc for this particular
obs
>> file, I only retained ADPUPA and ADPSFC observations. I also have
an
>> additional masking grid defined in my config file, so you can just
get
>> rid of the PHR2 and keep FULL.
>>
>> Thanks for your help.
>>
>> - Jeff
>>
>>
>> On 5/25/2011 2:23 PM, RAL HelpDesk {for Paul Oldenburg} wrote:
>>> Jeff,
>>>
>>> I would first suggest using the point_stat options -obs_valid_beg
and -obs_valid_end to open the time window for matched
>>> observations to be very large. If this results in matched pairs,
then dump out those matched pairs by setting flag #15
>>> in the point_stat config file setting output_flag to 1 or 2.
Then, you can see what the valid time for the matched
>>> pairs actually is and adjust your setup accordingly.
>>>
>>> If that course of action is not fruitful, please send us the
following so that we may try to reproduce and figure out
>>> your problem:
>>>
>>> 1. Model data
>>> 2. Obs data
>>> 3. point_stat config file
>>>
>>> If these files are too large to include in an email, please upload
them to our FTP site using the instructions here:
>>>
>>> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>>>
>>> If you have any questions, please let me know.
>>>
>>> Paul
>>>
>>> On 05/25/2011 12:10 PM, RAL HelpDesk {for Jeff Zielonka} wrote:
>>>> Wed May 25 12:10:14 2011: Request 47129 was acted upon.
>>>> Transaction: Ticket created by zielonka at meteo.psu.edu
>>>> Queue: met_help
>>>> Subject: Problem using GDAS ADPSFC observations
>>>> Owner: Nobody
>>>> Requestors: zielonka at meteo.psu.edu
>>>> Status: new
>>>> Ticket<URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129>
>>>>
>>>>
>>>> Hi,
>>>>
>>>> I'm having a problem with point_stat in METv3.0 when attempting
to use
>>>> ADPSFC data from GDAS PREPBUFR files. For example, when I
attempt to
>>>> verify TMP/Z2 against ADPSFC observations, I get zero matched
pairs. If
>>>> I use the ONLYSF option, it will use the SFCSHP observations, but
not
>>>> ADPSFC.
>>>>
>>>> I know the ADPSFC observations exist in the netCDF files that
were
>>>> created by PB2NC, because there are many ADPSFC obs in the
hdr_typ array
>>>> when I ncdump any particular obs file.
>>>>
>>>> I looked through the known issues section of the website and saw
the
>>>> following,But when I differenced the updated pair_data.cc file
with the
>>>> one I have, they are identical, so I must have made that update
previously.
>>>>
>>>> One interesting thing to note is that when I use NDAS
observations
>>>> instead of GDAS observations, this issue does not occur. The
model
>>>> fields verify against ADPSFC observations just fine. So I'm not
sure
>>>> what the issue might be. Unfortunately, I'm using this data for
an old
>>>> case, so I do not have access to NDAS observations for this
period.
>>>>
>>>> Any ideas? Thanks.
>>>>
>>>>
>>>> - Jeff Z.
>>>>
>>>>
>>>> Point-Stat Tool
>>>>
>>>>
>>>> Vertical level matching logic when using a generic level
type.
>>>> Posted 10/05/2010
>>>>
>>>> *Problem:*When verifying generic level type data (e.g. "RH/L1")
against
>>>> a surface observation message type (e.g. "ADPSFC"), Point-Stat
was
>>>> finding zero matched pairs.
>>>> *Solution:*The fix is to no longer enforce that the observation
level
>>>> value exactly match the forecast level value when verifying
against
>>>> surface observation message types. This makes the behavior in
METv3.0
>>>> match that of METv2.0.
>>>> Update:METv3.0/lib/vx_met_util/pair_data.cc
>>>>
<http://www.dtcenter.org/met/users/support/known_issues/METv3.0/patches/lib/vx_met_util/pair_data.cc>
>>>> And then recompile MET.
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #47129] Problem using GDAS ADPSFC observations
From: Paul Oldenburg
Time: Fri May 27 12:36:16 2011
Jeff,
The GDAS obs data contains ADPSFC TMP observations whose quality mark
may be higher than that of other observations.
I'm not totally familiar with quality marks, but I do know that pb2nc
will only pull observations for data whose quality
mark is less than the threshold stored in the setting
quality_mark_thresh. I downloaded the GDAS PrepBUFR obs file for
2007100500 and then ran it through pb2nc. I was able to reproduce the
problem that you reported with 0 matched pairs
found for TMP/Z2. My colleague John suggested re-running pb2nc and
raising the quality_mark_thresh to 9 in the pb2nc
configuration file. When I did so, point_stat found matched pairs
when I ran it with the new pb2nc NetCDF output file.
Please try adjusting your pb2nc configuration file to use this
setting:
quality_mark_thresh = 9
Then, re-run point_stat to see if it finds matched pairs for TMP/Z2.
If not, please let us know.
Thanks,
Paul
On 05/27/2011 07:37 AM, RAL HelpDesk {for Jeff Zielonka} wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129 >
>
> Paul,
>
> No worries. I have other things that can keep me occupied while
you're
> working on this. Thanks a lot for the help and the update.
>
> - Jeff
>
>
> On 5/26/2011 7:02 PM, RAL HelpDesk {for Paul Oldenburg} wrote:
>> Jeff,
>>
>> Sorry on the delay getting back to you. We are working on what
appears to be a bug in pb2nc or some other strangeness
>> with PrepBUFR observations. I'll let you know when we figure it
out.
>>
>> Thanks,
>>
>> Paul
>>
>>
>> On 05/25/2011 12:58 PM, RAL HelpDesk {for Jeff Zielonka} wrote:
>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129>
>>>
>>> Hi Paul,
>>>
>>> I'm currently using a time window of -1800 to 1800. I increased
the
>>> verbosity level when I ran point_stat and it says there are 0 obs
that
>>> are being rejected because of the valid time.
>>>
>>> I have uploaded a grib file, the corresponding obs file, and the
config
>>> file to the ftp site. Note, when I ran pb2nc for this particular
obs
>>> file, I only retained ADPUPA and ADPSFC observations. I also have
an
>>> additional masking grid defined in my config file, so you can just
get
>>> rid of the PHR2 and keep FULL.
>>>
>>> Thanks for your help.
>>>
>>> - Jeff
>>>
>>>
>>> On 5/25/2011 2:23 PM, RAL HelpDesk {for Paul Oldenburg} wrote:
>>>> Jeff,
>>>>
>>>> I would first suggest using the point_stat options -obs_valid_beg
and -obs_valid_end to open the time window for matched
>>>> observations to be very large. If this results in matched pairs,
then dump out those matched pairs by setting flag #15
>>>> in the point_stat config file setting output_flag to 1 or 2.
Then, you can see what the valid time for the matched
>>>> pairs actually is and adjust your setup accordingly.
>>>>
>>>> If that course of action is not fruitful, please send us the
following so that we may try to reproduce and figure out
>>>> your problem:
>>>>
>>>> 1. Model data
>>>> 2. Obs data
>>>> 3. point_stat config file
>>>>
>>>> If these files are too large to include in an email, please
upload them to our FTP site using the instructions here:
>>>>
>>>> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>>>>
>>>> If you have any questions, please let me know.
>>>>
>>>> Paul
>>>>
>>>> On 05/25/2011 12:10 PM, RAL HelpDesk {for Jeff Zielonka} wrote:
>>>>> Wed May 25 12:10:14 2011: Request 47129 was acted upon.
>>>>> Transaction: Ticket created by zielonka at meteo.psu.edu
>>>>> Queue: met_help
>>>>> Subject: Problem using GDAS ADPSFC observations
>>>>> Owner: Nobody
>>>>> Requestors: zielonka at meteo.psu.edu
>>>>> Status: new
>>>>> Ticket<URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129>
>>>>>
>>>>>
>>>>> Hi,
>>>>>
>>>>> I'm having a problem with point_stat in METv3.0 when attempting
to use
>>>>> ADPSFC data from GDAS PREPBUFR files. For example, when I
attempt to
>>>>> verify TMP/Z2 against ADPSFC observations, I get zero matched
pairs. If
>>>>> I use the ONLYSF option, it will use the SFCSHP observations,
but not
>>>>> ADPSFC.
>>>>>
>>>>> I know the ADPSFC observations exist in the netCDF files that
were
>>>>> created by PB2NC, because there are many ADPSFC obs in the
hdr_typ array
>>>>> when I ncdump any particular obs file.
>>>>>
>>>>> I looked through the known issues section of the website and saw
the
>>>>> following,But when I differenced the updated pair_data.cc file
with the
>>>>> one I have, they are identical, so I must have made that update
previously.
>>>>>
>>>>> One interesting thing to note is that when I use NDAS
observations
>>>>> instead of GDAS observations, this issue does not occur. The
model
>>>>> fields verify against ADPSFC observations just fine. So I'm not
sure
>>>>> what the issue might be. Unfortunately, I'm using this data for
an old
>>>>> case, so I do not have access to NDAS observations for this
period.
>>>>>
>>>>> Any ideas? Thanks.
>>>>>
>>>>>
>>>>> - Jeff Z.
>>>>>
>>>>>
>>>>> Point-Stat Tool
>>>>>
>>>>>
>>>>> Vertical level matching logic when using a generic level
type.
>>>>> Posted 10/05/2010
>>>>>
>>>>> *Problem:*When verifying generic level type data (e.g. "RH/L1")
against
>>>>> a surface observation message type (e.g. "ADPSFC"), Point-Stat
was
>>>>> finding zero matched pairs.
>>>>> *Solution:*The fix is to no longer enforce that the observation
level
>>>>> value exactly match the forecast level value when verifying
against
>>>>> surface observation message types. This makes the behavior in
METv3.0
>>>>> match that of METv2.0.
>>>>> Update:METv3.0/lib/vx_met_util/pair_data.cc
>>>>>
<http://www.dtcenter.org/met/users/support/known_issues/METv3.0/patches/lib/vx_met_util/pair_data.cc>
>>>>> And then recompile MET.
>>
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #47129] Problem using GDAS ADPSFC observations
From: Jeff Zielonka
Time: Fri May 27 13:20:22 2011
Paul,
It looks like this did the trick. I re-ran pb2nc and point_stat, and
this resulted in 3655 matched pairs for TMP/Z2 on the full domain, and
320 on the smaller subdomain that you wouldn't have seen. Are these
3655 pairs consistent (within reason) with what you found?
Thanks again for you help with this. I don't think I would have been
able to track down this issue on my own. I've never even thought
about
adjusting the default value for quality_mark_thresh.
I'm assuming this will solve my issues, but I'll let you know if I run
into any snags in the next week.
- Jeff
On 5/27/2011 2:36 PM, RAL HelpDesk {for Paul Oldenburg} wrote:
> Jeff,
>
> The GDAS obs data contains ADPSFC TMP observations whose quality
mark may be higher than that of other observations.
> I'm not totally familiar with quality marks, but I do know that
pb2nc will only pull observations for data whose quality
> mark is less than the threshold stored in the setting
quality_mark_thresh. I downloaded the GDAS PrepBUFR obs file for
> 2007100500 and then ran it through pb2nc. I was able to reproduce
the problem that you reported with 0 matched pairs
> found for TMP/Z2. My colleague John suggested re-running pb2nc and
raising the quality_mark_thresh to 9 in the pb2nc
> configuration file. When I did so, point_stat found matched pairs
when I ran it with the new pb2nc NetCDF output file.
>
> Please try adjusting your pb2nc configuration file to use this
setting:
>
> quality_mark_thresh = 9
>
> Then, re-run point_stat to see if it finds matched pairs for TMP/Z2.
If not, please let us know.
>
> Thanks,
>
> Paul
>
>
> On 05/27/2011 07:37 AM, RAL HelpDesk {for Jeff Zielonka} wrote:
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129>
>>
>> Paul,
>>
>> No worries. I have other things that can keep me occupied while
you're
>> working on this. Thanks a lot for the help and the update.
>>
>> - Jeff
>>
>>
>> On 5/26/2011 7:02 PM, RAL HelpDesk {for Paul Oldenburg} wrote:
>>> Jeff,
>>>
>>> Sorry on the delay getting back to you. We are working on what
appears to be a bug in pb2nc or some other strangeness
>>> with PrepBUFR observations. I'll let you know when we figure it
out.
>>>
>>> Thanks,
>>>
>>> Paul
>>>
>>>
>>> On 05/25/2011 12:58 PM, RAL HelpDesk {for Jeff Zielonka} wrote:
>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129>
>>>>
>>>> Hi Paul,
>>>>
>>>> I'm currently using a time window of -1800 to 1800. I increased
the
>>>> verbosity level when I ran point_stat and it says there are 0 obs
that
>>>> are being rejected because of the valid time.
>>>>
>>>> I have uploaded a grib file, the corresponding obs file, and the
config
>>>> file to the ftp site. Note, when I ran pb2nc for this particular
obs
>>>> file, I only retained ADPUPA and ADPSFC observations. I also
have an
>>>> additional masking grid defined in my config file, so you can
just get
>>>> rid of the PHR2 and keep FULL.
>>>>
>>>> Thanks for your help.
>>>>
>>>> - Jeff
>>>>
>>>>
>>>> On 5/25/2011 2:23 PM, RAL HelpDesk {for Paul Oldenburg} wrote:
>>>>> Jeff,
>>>>>
>>>>> I would first suggest using the point_stat options
-obs_valid_beg and -obs_valid_end to open the time window for matched
>>>>> observations to be very large. If this results in matched
pairs, then dump out those matched pairs by setting flag #15
>>>>> in the point_stat config file setting output_flag to 1 or 2.
Then, you can see what the valid time for the matched
>>>>> pairs actually is and adjust your setup accordingly.
>>>>>
>>>>> If that course of action is not fruitful, please send us the
following so that we may try to reproduce and figure out
>>>>> your problem:
>>>>>
>>>>> 1. Model data
>>>>> 2. Obs data
>>>>> 3. point_stat config file
>>>>>
>>>>> If these files are too large to include in an email, please
upload them to our FTP site using the instructions here:
>>>>>
>>>>> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>>>>>
>>>>> If you have any questions, please let me know.
>>>>>
>>>>> Paul
>>>>>
>>>>> On 05/25/2011 12:10 PM, RAL HelpDesk {for Jeff Zielonka} wrote:
>>>>>> Wed May 25 12:10:14 2011: Request 47129 was acted upon.
>>>>>> Transaction: Ticket created by zielonka at meteo.psu.edu
>>>>>> Queue: met_help
>>>>>> Subject: Problem using GDAS ADPSFC observations
>>>>>> Owner: Nobody
>>>>>> Requestors: zielonka at meteo.psu.edu
>>>>>> Status: new
>>>>>> Ticket<URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129>
>>>>>>
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I'm having a problem with point_stat in METv3.0 when attempting
to use
>>>>>> ADPSFC data from GDAS PREPBUFR files. For example, when I
attempt to
>>>>>> verify TMP/Z2 against ADPSFC observations, I get zero matched
pairs. If
>>>>>> I use the ONLYSF option, it will use the SFCSHP observations,
but not
>>>>>> ADPSFC.
>>>>>>
>>>>>> I know the ADPSFC observations exist in the netCDF files that
were
>>>>>> created by PB2NC, because there are many ADPSFC obs in the
hdr_typ array
>>>>>> when I ncdump any particular obs file.
>>>>>>
>>>>>> I looked through the known issues section of the website and
saw the
>>>>>> following,But when I differenced the updated pair_data.cc file
with the
>>>>>> one I have, they are identical, so I must have made that update
previously.
>>>>>>
>>>>>> One interesting thing to note is that when I use NDAS
observations
>>>>>> instead of GDAS observations, this issue does not occur. The
model
>>>>>> fields verify against ADPSFC observations just fine. So I'm
not sure
>>>>>> what the issue might be. Unfortunately, I'm using this data
for an old
>>>>>> case, so I do not have access to NDAS observations for this
period.
>>>>>>
>>>>>> Any ideas? Thanks.
>>>>>>
>>>>>>
>>>>>> - Jeff Z.
>>>>>>
>>>>>>
>>>>>> Point-Stat Tool
>>>>>>
>>>>>>
>>>>>> Vertical level matching logic when using a generic
level type.
>>>>>> Posted 10/05/2010
>>>>>>
>>>>>> *Problem:*When verifying generic level type data (e.g. "RH/L1")
against
>>>>>> a surface observation message type (e.g. "ADPSFC"), Point-Stat
was
>>>>>> finding zero matched pairs.
>>>>>> *Solution:*The fix is to no longer enforce that the observation
level
>>>>>> value exactly match the forecast level value when verifying
against
>>>>>> surface observation message types. This makes the behavior in
METv3.0
>>>>>> match that of METv2.0.
>>>>>> Update:METv3.0/lib/vx_met_util/pair_data.cc
>>>>>>
<http://www.dtcenter.org/met/users/support/known_issues/METv3.0/patches/lib/vx_met_util/pair_data.cc>
>>>>>> And then recompile MET.
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #47129] Problem using GDAS ADPSFC observations
From: Paul Oldenburg
Time: Fri May 27 13:30:51 2011
Jeff,
During my experiment, point_stat found 3652 matched pairs for TMP/Z2,
which seems commensurate to me. The
quality_mark_threshold is definitely an obscure feature of pb2nc. I
wasn't terribly familiar with it myself before I
worked on this problem.
Paul
On 05/27/2011 01:20 PM, RAL HelpDesk {for Jeff Zielonka} wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129 >
>
> Paul,
>
> It looks like this did the trick. I re-ran pb2nc and point_stat,
and
> this resulted in 3655 matched pairs for TMP/Z2 on the full domain,
and
> 320 on the smaller subdomain that you wouldn't have seen. Are these
> 3655 pairs consistent (within reason) with what you found?
>
> Thanks again for you help with this. I don't think I would have
been
> able to track down this issue on my own. I've never even thought
about
> adjusting the default value for quality_mark_thresh.
>
> I'm assuming this will solve my issues, but I'll let you know if I
run
> into any snags in the next week.
>
> - Jeff
>
>
>
> On 5/27/2011 2:36 PM, RAL HelpDesk {for Paul Oldenburg} wrote:
>> Jeff,
>>
>> The GDAS obs data contains ADPSFC TMP observations whose quality
mark may be higher than that of other observations.
>> I'm not totally familiar with quality marks, but I do know that
pb2nc will only pull observations for data whose quality
>> mark is less than the threshold stored in the setting
quality_mark_thresh. I downloaded the GDAS PrepBUFR obs file for
>> 2007100500 and then ran it through pb2nc. I was able to reproduce
the problem that you reported with 0 matched pairs
>> found for TMP/Z2. My colleague John suggested re-running pb2nc and
raising the quality_mark_thresh to 9 in the pb2nc
>> configuration file. When I did so, point_stat found matched pairs
when I ran it with the new pb2nc NetCDF output file.
>>
>> Please try adjusting your pb2nc configuration file to use this
setting:
>>
>> quality_mark_thresh = 9
>>
>> Then, re-run point_stat to see if it finds matched pairs for
TMP/Z2. If not, please let us know.
>>
>> Thanks,
>>
>> Paul
>>
>>
>> On 05/27/2011 07:37 AM, RAL HelpDesk {for Jeff Zielonka} wrote:
>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129>
>>>
>>> Paul,
>>>
>>> No worries. I have other things that can keep me occupied while
you're
>>> working on this. Thanks a lot for the help and the update.
>>>
>>> - Jeff
>>>
>>>
>>> On 5/26/2011 7:02 PM, RAL HelpDesk {for Paul Oldenburg} wrote:
>>>> Jeff,
>>>>
>>>> Sorry on the delay getting back to you. We are working on what
appears to be a bug in pb2nc or some other strangeness
>>>> with PrepBUFR observations. I'll let you know when we figure it
out.
>>>>
>>>> Thanks,
>>>>
>>>> Paul
>>>>
>>>>
>>>> On 05/25/2011 12:58 PM, RAL HelpDesk {for Jeff Zielonka} wrote:
>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129>
>>>>>
>>>>> Hi Paul,
>>>>>
>>>>> I'm currently using a time window of -1800 to 1800. I increased
the
>>>>> verbosity level when I ran point_stat and it says there are 0
obs that
>>>>> are being rejected because of the valid time.
>>>>>
>>>>> I have uploaded a grib file, the corresponding obs file, and the
config
>>>>> file to the ftp site. Note, when I ran pb2nc for this
particular obs
>>>>> file, I only retained ADPUPA and ADPSFC observations. I also
have an
>>>>> additional masking grid defined in my config file, so you can
just get
>>>>> rid of the PHR2 and keep FULL.
>>>>>
>>>>> Thanks for your help.
>>>>>
>>>>> - Jeff
>>>>>
>>>>>
>>>>> On 5/25/2011 2:23 PM, RAL HelpDesk {for Paul Oldenburg} wrote:
>>>>>> Jeff,
>>>>>>
>>>>>> I would first suggest using the point_stat options
-obs_valid_beg and -obs_valid_end to open the time window for matched
>>>>>> observations to be very large. If this results in matched
pairs, then dump out those matched pairs by setting flag #15
>>>>>> in the point_stat config file setting output_flag to 1 or 2.
Then, you can see what the valid time for the matched
>>>>>> pairs actually is and adjust your setup accordingly.
>>>>>>
>>>>>> If that course of action is not fruitful, please send us the
following so that we may try to reproduce and figure out
>>>>>> your problem:
>>>>>>
>>>>>> 1. Model data
>>>>>> 2. Obs data
>>>>>> 3. point_stat config file
>>>>>>
>>>>>> If these files are too large to include in an email, please
upload them to our FTP site using the instructions here:
>>>>>>
>>>>>> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>>>>>>
>>>>>> If you have any questions, please let me know.
>>>>>>
>>>>>> Paul
>>>>>>
>>>>>> On 05/25/2011 12:10 PM, RAL HelpDesk {for Jeff Zielonka} wrote:
>>>>>>> Wed May 25 12:10:14 2011: Request 47129 was acted upon.
>>>>>>> Transaction: Ticket created by zielonka at meteo.psu.edu
>>>>>>> Queue: met_help
>>>>>>> Subject: Problem using GDAS ADPSFC observations
>>>>>>> Owner: Nobody
>>>>>>> Requestors: zielonka at meteo.psu.edu
>>>>>>> Status: new
>>>>>>> Ticket<URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=47129>
>>>>>>>
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I'm having a problem with point_stat in METv3.0 when
attempting to use
>>>>>>> ADPSFC data from GDAS PREPBUFR files. For example, when I
attempt to
>>>>>>> verify TMP/Z2 against ADPSFC observations, I get zero matched
pairs. If
>>>>>>> I use the ONLYSF option, it will use the SFCSHP observations,
but not
>>>>>>> ADPSFC.
>>>>>>>
>>>>>>> I know the ADPSFC observations exist in the netCDF files that
were
>>>>>>> created by PB2NC, because there are many ADPSFC obs in the
hdr_typ array
>>>>>>> when I ncdump any particular obs file.
>>>>>>>
>>>>>>> I looked through the known issues section of the website and
saw the
>>>>>>> following,But when I differenced the updated pair_data.cc file
with the
>>>>>>> one I have, they are identical, so I must have made that
update previously.
>>>>>>>
>>>>>>> One interesting thing to note is that when I use NDAS
observations
>>>>>>> instead of GDAS observations, this issue does not occur. The
model
>>>>>>> fields verify against ADPSFC observations just fine. So I'm
not sure
>>>>>>> what the issue might be. Unfortunately, I'm using this data
for an old
>>>>>>> case, so I do not have access to NDAS observations for this
period.
>>>>>>>
>>>>>>> Any ideas? Thanks.
>>>>>>>
>>>>>>>
>>>>>>> - Jeff Z.
>>>>>>>
>>>>>>>
>>>>>>> Point-Stat Tool
>>>>>>>
>>>>>>>
>>>>>>> Vertical level matching logic when using a generic
level type.
>>>>>>> Posted 10/05/2010
>>>>>>>
>>>>>>> *Problem:*When verifying generic level type data (e.g.
"RH/L1") against
>>>>>>> a surface observation message type (e.g. "ADPSFC"), Point-Stat
was
>>>>>>> finding zero matched pairs.
>>>>>>> *Solution:*The fix is to no longer enforce that the
observation level
>>>>>>> value exactly match the forecast level value when verifying
against
>>>>>>> surface observation message types. This makes the behavior in
METv3.0
>>>>>>> match that of METv2.0.
>>>>>>> Update:METv3.0/lib/vx_met_util/pair_data.cc
>>>>>>>
<http://www.dtcenter.org/met/users/support/known_issues/METv3.0/patches/lib/vx_met_util/pair_data.cc>
>>>>>>> And then recompile MET.
>>
>
------------------------------------------------
More information about the Met_help
mailing list