[Met_help] [rt.rap.ucar.edu #62407] History for MET with GDAS
John Halley Gotway via RT
met_help at ucar.edu
Thu Aug 1 13:52:57 MDT 2013
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hi,
I'm using MET with GDAS data and I'm confused about the output because of the six hourly nature of the GDAS files. My WRF output is hourly (each hour is a separate output file) and I'm using GDAS data from the CISL archive, which is six hourly:
http://rda.ucar.edu/datasets/ds337.0/#access
I've been using p_interp to interpolate one WRF time/file (ex: 7-30-2006 12 UTC) and then running point_stats with the corresponding GDAS file for that time (ex: 200607301200.nc ). From what I understand, the GDAS six hourly files contain data for three hours on either side of the title hour; so for example, the 12 UTC file contains data from 09 - 15 UTC. Does that mean that the output produced by point_stats is a comparison of the WRF output at one time (12 UTC) to the average of 09 - 15 UTC GDAS observations?
Ideally, I would like to compare my hourly WRF output files to hourly GDAS data using MET. Is there a way to further discretize the six hourly GDAS files to extract the hourly data instead?
Thank you,
Tricia
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #62407] MET with GDAS
From: John Halley Gotway
Time: Mon Jul 29 10:23:12 2013
Tricia,
Your understanding of the structure of the pinterp and GDAS output
files is correct. However, your assumption about the behavior of
Point-Stat is not.
Here's how the logic works in Point-Stat...
- It reads the fields you've specified in the configuration file
from the gridded forecast file you pass it.
- For each forecast field, it extracts the valid time.
- It constructs a time window around the forecast valid time using
the "obs_window" setting from the configuration file. The default
value is +/- 5400 seconds, so +/- 90 minutes.
- Any point observations falling within that time window are used in
the verification.
Since you have hourly model output, I would say that +/- 90 minutes is
way too big. If you really only want the observations right near the
top of the hour, perhaps +/- 5 minutes (300 seconds) would
be good.
Another point to make is regarding the "duplicate_flag" option which
defaults to off to preserve earlier functionality. You can read about
the details of this setting in METv4.1/data/config/README.
You may want to set it to "SINGLE" - which means that Point-Stat
should only use a single observation from each station when there are
multiple observations that fall within the time window. When
duplicate_flag is set to NONE, all observations falling within the
time window are used - even if they're at the same station.
For scripting up your calls to Point-Stat, you just pass it the
forecast file to be evaluated and the point observation file with the
observations for that time. Since the observations come in 6-hour
chunks, every six hours half the observations you want to use will be
in one file and the other half will be in the next file. In that
case, use the "-point_obs" option to pass a second point
observation file.
Hope that helps.
Thanks,
John Halley Gotway
On 07/26/2013 04:58 PM, Lawston, Patricia Marie.[GSFC INTERNS] via RT
wrote:
>
> Fri Jul 26 16:58:52 2013: Request 62407 was acted upon.
> Transaction: Ticket created by patricia.m.lawston at nasa.gov
> Queue: met_help
> Subject: MET with GDAS
> Owner: Nobody
> Requestors: patricia.m.lawston at nasa.gov
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62407 >
>
>
> Hi,
>
> I'm using MET with GDAS data and I'm confused about the output
because of the six hourly nature of the GDAS files. My WRF output is
hourly (each hour is a separate output file) and I'm using GDAS data
from the CISL archive, which is six hourly:
> http://rda.ucar.edu/datasets/ds337.0/#access
>
> I've been using p_interp to interpolate one WRF time/file (ex: 7-30-
2006 12 UTC) and then running point_stats with the corresponding GDAS
file for that time (ex: 200607301200.nc ). From what I understand,
the GDAS six hourly files contain data for three hours on either side
of the title hour; so for example, the 12 UTC file contains data from
09 - 15 UTC. Does that mean that the output produced by point_stats
is a comparison of the WRF output at one time (12 UTC) to the average
of 09 - 15 UTC GDAS observations?
>
> Ideally, I would like to compare my hourly WRF output files to
hourly GDAS data using MET. Is there a way to further discretize the
six hourly GDAS files to extract the hourly data instead?
>
> Thank you,
> Tricia
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #62407] MET with GDAS
From: John Halley Gotway
Time: Mon Jul 29 10:28:50 2013
Tricia,
One more note about GDAS data. If you're evaluating forecasts at the
surface, you may notice that you don't have many matched pairs. In
the GDAS processing, they assign most of the surface
observations a quality mark of 9. They do that to prevent them from
being used in the data assimilation process, since they tend not to
have a positive impact on the DA. If you're not getting many
matched pairs for surface verification, try increasing the
"quality_mark_thresh" configuration setting in PB2NC to 9.
Thanks,
John
On 07/26/2013 04:58 PM, Lawston, Patricia Marie.[GSFC INTERNS] via RT
wrote:
>
> Fri Jul 26 16:58:52 2013: Request 62407 was acted upon.
> Transaction: Ticket created by patricia.m.lawston at nasa.gov
> Queue: met_help
> Subject: MET with GDAS
> Owner: Nobody
> Requestors: patricia.m.lawston at nasa.gov
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62407 >
>
>
> Hi,
>
> I'm using MET with GDAS data and I'm confused about the output
because of the six hourly nature of the GDAS files. My WRF output is
hourly (each hour is a separate output file) and I'm using GDAS data
from the CISL archive, which is six hourly:
> http://rda.ucar.edu/datasets/ds337.0/#access
>
> I've been using p_interp to interpolate one WRF time/file (ex: 7-30-
2006 12 UTC) and then running point_stats with the corresponding GDAS
file for that time (ex: 200607301200.nc ). From what I understand,
the GDAS six hourly files contain data for three hours on either side
of the title hour; so for example, the 12 UTC file contains data from
09 - 15 UTC. Does that mean that the output produced by point_stats
is a comparison of the WRF output at one time (12 UTC) to the average
of 09 - 15 UTC GDAS observations?
>
> Ideally, I would like to compare my hourly WRF output files to
hourly GDAS data using MET. Is there a way to further discretize the
six hourly GDAS files to extract the hourly data instead?
>
> Thank you,
> Tricia
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #62407] MET with GDAS
From: Lawston, Patricia Marie.[GSFC INTERNS]
Time: Mon Jul 29 12:23:00 2013
Hi John,
Thank you very much for your clear and detailed response. Your
explanation is very helpful. One other question, if you don't mind -
do you have any information about the T2/Tv fix for PB2NC? Has it
been released yet?
Thanks again,
Tricia
________________________________________
From: John Halley Gotway via RT [met_help at ucar.edu]
Sent: Monday, July 29, 2013 11:23 AM
To: Lawston, Patricia Marie. (GSFC-617.0)[GSFC INTERNS]
Subject: Re: [rt.rap.ucar.edu #62407] MET with GDAS
Tricia,
Your understanding of the structure of the pinterp and GDAS output
files is correct. However, your assumption about the behavior of
Point-Stat is not.
Here's how the logic works in Point-Stat...
- It reads the fields you've specified in the configuration file
from the gridded forecast file you pass it.
- For each forecast field, it extracts the valid time.
- It constructs a time window around the forecast valid time using
the "obs_window" setting from the configuration file. The default
value is +/- 5400 seconds, so +/- 90 minutes.
- Any point observations falling within that time window are used in
the verification.
Since you have hourly model output, I would say that +/- 90 minutes is
way too big. If you really only want the observations right near the
top of the hour, perhaps +/- 5 minutes (300 seconds) would
be good.
Another point to make is regarding the "duplicate_flag" option which
defaults to off to preserve earlier functionality. You can read about
the details of this setting in METv4.1/data/config/README.
You may want to set it to "SINGLE" - which means that Point-Stat
should only use a single observation from each station when there are
multiple observations that fall within the time window. When
duplicate_flag is set to NONE, all observations falling within the
time window are used - even if they're at the same station.
For scripting up your calls to Point-Stat, you just pass it the
forecast file to be evaluated and the point observation file with the
observations for that time. Since the observations come in 6-hour
chunks, every six hours half the observations you want to use will be
in one file and the other half will be in the next file. In that
case, use the "-point_obs" option to pass a second point
observation file.
Hope that helps.
Thanks,
John Halley Gotway
On 07/26/2013 04:58 PM, Lawston, Patricia Marie.[GSFC INTERNS] via RT
wrote:
>
> Fri Jul 26 16:58:52 2013: Request 62407 was acted upon.
> Transaction: Ticket created by patricia.m.lawston at nasa.gov
> Queue: met_help
> Subject: MET with GDAS
> Owner: Nobody
> Requestors: patricia.m.lawston at nasa.gov
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62407 >
>
>
> Hi,
>
> I'm using MET with GDAS data and I'm confused about the output
because of the six hourly nature of the GDAS files. My WRF output is
hourly (each hour is a separate output file) and I'm using GDAS data
from the CISL archive, which is six hourly:
> http://rda.ucar.edu/datasets/ds337.0/#access
>
> I've been using p_interp to interpolate one WRF time/file (ex: 7-30-
2006 12 UTC) and then running point_stats with the corresponding GDAS
file for that time (ex: 200607301200.nc ). From what I understand,
the GDAS six hourly files contain data for three hours on either side
of the title hour; so for example, the 12 UTC file contains data from
09 - 15 UTC. Does that mean that the output produced by point_stats
is a comparison of the WRF output at one time (12 UTC) to the average
of 09 - 15 UTC GDAS observations?
>
> Ideally, I would like to compare my hourly WRF output files to
hourly GDAS data using MET. Is there a way to further discretize the
six hourly GDAS files to extract the hourly data instead?
>
> Thank you,
> Tricia
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #62407] MET with GDAS
From: John Halley Gotway
Time: Mon Jul 29 13:46:28 2013
Yep, you can download the latest set of patches here:
http://www.dtcenter.org/met/users/support/known_issues/METv4.1/index.php
Just follow the instructions at the top of the page to apply the
patches and recompile.
We'll be sending out an email about this to all registered MET users
shortly. So when you see that, you can ignore it.
Thanks,
John
On 07/29/2013 12:23 PM, Lawston, Patricia Marie.[GSFC INTERNS] via RT
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62407 >
>
> Hi John,
>
> Thank you very much for your clear and detailed response. Your
explanation is very helpful. One other question, if you don't mind -
do you have any information about the T2/Tv fix for PB2NC? Has it
been released yet?
>
> Thanks again,
> Tricia
>
>
> ________________________________________
> From: John Halley Gotway via RT [met_help at ucar.edu]
> Sent: Monday, July 29, 2013 11:23 AM
> To: Lawston, Patricia Marie. (GSFC-617.0)[GSFC INTERNS]
> Subject: Re: [rt.rap.ucar.edu #62407] MET with GDAS
>
> Tricia,
>
> Your understanding of the structure of the pinterp and GDAS output
files is correct. However, your assumption about the behavior of
Point-Stat is not.
>
> Here's how the logic works in Point-Stat...
> - It reads the fields you've specified in the configuration file
from the gridded forecast file you pass it.
> - For each forecast field, it extracts the valid time.
> - It constructs a time window around the forecast valid time
using the "obs_window" setting from the configuration file. The
default value is +/- 5400 seconds, so +/- 90 minutes.
> - Any point observations falling within that time window are used
in the verification.
>
> Since you have hourly model output, I would say that +/- 90 minutes
is way too big. If you really only want the observations right near
the top of the hour, perhaps +/- 5 minutes (300 seconds) would
> be good.
>
> Another point to make is regarding the "duplicate_flag" option which
defaults to off to preserve earlier functionality. You can read about
the details of this setting in METv4.1/data/config/README.
> You may want to set it to "SINGLE" - which means that Point-Stat
should only use a single observation from each station when there are
multiple observations that fall within the time window. When
> duplicate_flag is set to NONE, all observations falling within the
time window are used - even if they're at the same station.
>
> For scripting up your calls to Point-Stat, you just pass it the
forecast file to be evaluated and the point observation file with the
observations for that time. Since the observations come in 6-hour
> chunks, every six hours half the observations you want to use will
be in one file and the other half will be in the next file. In that
case, use the "-point_obs" option to pass a second point
> observation file.
>
> Hope that helps.
>
> Thanks,
> John Halley Gotway
>
>
> On 07/26/2013 04:58 PM, Lawston, Patricia Marie.[GSFC INTERNS] via
RT wrote:
>>
>> Fri Jul 26 16:58:52 2013: Request 62407 was acted upon.
>> Transaction: Ticket created by patricia.m.lawston at nasa.gov
>> Queue: met_help
>> Subject: MET with GDAS
>> Owner: Nobody
>> Requestors: patricia.m.lawston at nasa.gov
>> Status: new
>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62407 >
>>
>>
>> Hi,
>>
>> I'm using MET with GDAS data and I'm confused about the output
because of the six hourly nature of the GDAS files. My WRF output is
hourly (each hour is a separate output file) and I'm using GDAS data
from the CISL archive, which is six hourly:
>> http://rda.ucar.edu/datasets/ds337.0/#access
>>
>> I've been using p_interp to interpolate one WRF time/file (ex: 7-
30-2006 12 UTC) and then running point_stats with the corresponding
GDAS file for that time (ex: 200607301200.nc ). From what I
understand, the GDAS six hourly files contain data for three hours on
either side of the title hour; so for example, the 12 UTC file
contains data from 09 - 15 UTC. Does that mean that the output
produced by point_stats is a comparison of the WRF output at one time
(12 UTC) to the average of 09 - 15 UTC GDAS observations?
>>
>> Ideally, I would like to compare my hourly WRF output files to
hourly GDAS data using MET. Is there a way to further discretize the
six hourly GDAS files to extract the hourly data instead?
>>
>> Thank you,
>> Tricia
>>
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #62407] MET with GDAS
From: Lawston, Patricia Marie.[GSFC INTERNS]
Time: Mon Jul 29 14:17:20 2013
Ok great! Thank you for your help, John!
-Tricia
________________________________________
From: John Halley Gotway via RT [met_help at ucar.edu]
Sent: Monday, July 29, 2013 2:46 PM
To: Lawston, Patricia Marie. (GSFC-617.0)[GSFC INTERNS]
Subject: Re: [rt.rap.ucar.edu #62407] MET with GDAS
Yep, you can download the latest set of patches here:
http://www.dtcenter.org/met/users/support/known_issues/METv4.1/index.php
Just follow the instructions at the top of the page to apply the
patches and recompile.
We'll be sending out an email about this to all registered MET users
shortly. So when you see that, you can ignore it.
Thanks,
John
On 07/29/2013 12:23 PM, Lawston, Patricia Marie.[GSFC INTERNS] via RT
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62407 >
>
> Hi John,
>
> Thank you very much for your clear and detailed response. Your
explanation is very helpful. One other question, if you don't mind -
do you have any information about the T2/Tv fix for PB2NC? Has it
been released yet?
>
> Thanks again,
> Tricia
>
>
> ________________________________________
> From: John Halley Gotway via RT [met_help at ucar.edu]
> Sent: Monday, July 29, 2013 11:23 AM
> To: Lawston, Patricia Marie. (GSFC-617.0)[GSFC INTERNS]
> Subject: Re: [rt.rap.ucar.edu #62407] MET with GDAS
>
> Tricia,
>
> Your understanding of the structure of the pinterp and GDAS output
files is correct. However, your assumption about the behavior of
Point-Stat is not.
>
> Here's how the logic works in Point-Stat...
> - It reads the fields you've specified in the configuration file
from the gridded forecast file you pass it.
> - For each forecast field, it extracts the valid time.
> - It constructs a time window around the forecast valid time
using the "obs_window" setting from the configuration file. The
default value is +/- 5400 seconds, so +/- 90 minutes.
> - Any point observations falling within that time window are used
in the verification.
>
> Since you have hourly model output, I would say that +/- 90 minutes
is way too big. If you really only want the observations right near
the top of the hour, perhaps +/- 5 minutes (300 seconds) would
> be good.
>
> Another point to make is regarding the "duplicate_flag" option which
defaults to off to preserve earlier functionality. You can read about
the details of this setting in METv4.1/data/config/README.
> You may want to set it to "SINGLE" - which means that Point-Stat
should only use a single observation from each station when there are
multiple observations that fall within the time window. When
> duplicate_flag is set to NONE, all observations falling within the
time window are used - even if they're at the same station.
>
> For scripting up your calls to Point-Stat, you just pass it the
forecast file to be evaluated and the point observation file with the
observations for that time. Since the observations come in 6-hour
> chunks, every six hours half the observations you want to use will
be in one file and the other half will be in the next file. In that
case, use the "-point_obs" option to pass a second point
> observation file.
>
> Hope that helps.
>
> Thanks,
> John Halley Gotway
>
>
> On 07/26/2013 04:58 PM, Lawston, Patricia Marie.[GSFC INTERNS] via
RT wrote:
>>
>> Fri Jul 26 16:58:52 2013: Request 62407 was acted upon.
>> Transaction: Ticket created by patricia.m.lawston at nasa.gov
>> Queue: met_help
>> Subject: MET with GDAS
>> Owner: Nobody
>> Requestors: patricia.m.lawston at nasa.gov
>> Status: new
>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62407 >
>>
>>
>> Hi,
>>
>> I'm using MET with GDAS data and I'm confused about the output
because of the six hourly nature of the GDAS files. My WRF output is
hourly (each hour is a separate output file) and I'm using GDAS data
from the CISL archive, which is six hourly:
>> http://rda.ucar.edu/datasets/ds337.0/#access
>>
>> I've been using p_interp to interpolate one WRF time/file (ex: 7-
30-2006 12 UTC) and then running point_stats with the corresponding
GDAS file for that time (ex: 200607301200.nc ). From what I
understand, the GDAS six hourly files contain data for three hours on
either side of the title hour; so for example, the 12 UTC file
contains data from 09 - 15 UTC. Does that mean that the output
produced by point_stats is a comparison of the WRF output at one time
(12 UTC) to the average of 09 - 15 UTC GDAS observations?
>>
>> Ideally, I would like to compare my hourly WRF output files to
hourly GDAS data using MET. Is there a way to further discretize the
six hourly GDAS files to extract the hourly data instead?
>>
>> Thank you,
>> Tricia
>>
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #62407] Resolved: MET with GDAS
From: Lawston, Patricia Marie.[GSFC INTERNS]
Time: Thu Aug 01 10:25:02 2013
Hi,
Thank you for your previous help. I have a new question/problem
related to the GDAS data and point stats.
Point stats runs successfully with 20 to 25 matched pairs for 00, 06,
12, 18 UTC. However, any other hour results in zero matched pairs.
For now, I'm not running point stats for hours where the GDAS
observations are split between two input files (3, 9, 15, 21z), but I
would like the point stats output for all other hours. I ran PB2NC
with quality_mark_thresh = 9. I set the duplicate_flag = SINGLE in
the PointStatsConfig and my window is +/- 300 seconds. My message
type is "ANYSFC".
Below is an excerpt of the terminal messages, showing a successful
hour and a subsequent hour where there are no matched pairs.
Please let me know if you have any suggestions or if you need any
additional information from me.
Thank you!
-Tricia
DEBUG 1: Forecast File: wrfout_d01_2008-05-26_06:00:00_PLEV
DEBUG 1: Climatology File: none
DEBUG 1: Observation File: gdas.2008052606_new.nc
DEBUG 2:
DEBUG 2:
--------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Reading data for T2(0,*,*).
DEBUG 2: For T2(0,*,*) found 1 forecast levels and 0 climatology
levels.
DEBUG 2:
DEBUG 2:
--------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Reading data for Q2(0,*,*).
DEBUG 2: For Q2(0,*,*) found 1 forecast levels and 0 climatology
levels.
DEBUG 2:
DEBUG 2:
--------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Searching 478 observations from 57 messages.
DEBUG 2:
DEBUG 2:
--------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Processing T2(0,*,*) versus TMP/Z2, for observation type
ANYSFC, over region ARMSGP, for interpolation method UW_MEAN(1), using
25 pairs.
DEBUG 2: Computing Categorical Statistics.
DEBUG 2: Computing Continuous Statistics.
DEBUG 2: Computing Scalar Partial Sums.
DEBUG 2: Processing T2(0,*,*) versus TMP/Z2, for observation type
ANYSFC, over region ARMSGP, for interpolation method MEDIAN(9), using
25 pairs.
DEBUG 2: Computing Categorical Statistics.
DEBUG 2: Computing Continuous Statistics.
DEBUG 2: Computing Scalar Partial Sums.
DEBUG 2: Processing T2(0,*,*) versus TMP/Z2, for observation type
ANYSFC, over region ARMSGP, for interpolation method DW_MEAN(9), using
25 pairs.
DEBUG 2: Computing Categorical Statistics.
DEBUG 2: Computing Continuous Statistics.
DEBUG 2: Computing Scalar Partial Sums.
DEBUG 2:
DEBUG 2:
--------------------------------------------------------------------------------
DEBUG 2:
DEBUG 1: Output file: /point_stat_300000L_20080526_060000V.stat
DEBUG 1: Output file: /point_stat_300000L_20080526_060000V_fho.txt
DEBUG 1: Output file: /point_stat_300000L_20080526_060000V_ctc.txt
DEBUG 1: Output file: /point_stat_300000L_20080526_060000V_cts.txt
DEBUG 1: Output file: /point_stat_300000L_20080526_060000V_cnt.txt
DEBUG 1: Output file: /point_stat_300000L_20080526_060000V_sl1l2.txt
DEBUG 1: Output file: /point_stat_300000L_20080526_060000V_sal1l2.txt
DEBUG 1: Output file: /point_stat_300000L_20080526_060000V_vl1l2.txt
DEBUG 1: Output file: /point_stat_300000L_20080526_060000V_val1l2.txt
DEBUG 1: Output file: /point_stat_300000L_20080526_060000V_mpr.txt
DEBUG 1: Default Config File: PointStatConfig_default
DEBUG 1: User Config File: PointStatConfig
GSL_RNG_TYPE=mt19937
GSL_RNG_SEED=1
DEBUG 1: Forecast File: wrfout_d01_2008-05-26_07:00:00_PLEV
DEBUG 1: Climatology File: none
DEBUG 1: Observation File: gdas.2008052606_new.nc
DEBUG 2:
DEBUG 2:
--------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Reading data for T2(0,*,*).
DEBUG 2: For T2(0,*,*) found 1 forecast levels and 0 climatology
levels.
DEBUG 2:
DEBUG 2:
--------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Reading data for Q2(0,*,*).
DEBUG 2: For Q2(0,*,*) found 1 forecast levels and 0 climatology
levels.
DEBUG 2:
DEBUG 2:
--------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Searching 478 observations from 57 messages.
DEBUG 2:
DEBUG 2:
--------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Processing T2(0,*,*) versus TMP/Z2, for observation type
ANYSFC, over region ARMSGP, for interpolation method UW_MEAN(1), using
0 pairs.
DEBUG 2: Processing T2(0,*,*) versus TMP/Z2, for observation type
ANYSFC, over region ARMSGP, for interpolation method MEDIAN(9), using
0 pairs.
DEBUG 2: Processing T2(0,*,*) versus TMP/Z2, for observation type
ANYSFC, over region ARMSGP, for interpolation method DW_MEAN(9), using
0 pairs.
DEBUG 2:
DEBUG 2:
--------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Processing Q2(0,*,*) versus SPFH/Z2, for observation type
ANYSFC, over region ARMSGP, for interpolation method UW_MEAN(1), using
0 pairs.
DEBUG 2: Processing Q2(0,*,*) versus SPFH/Z2, for observation type
ANYSFC, over region ARMSGP, for interpolation method MEDIAN(9), using
0 pairs.
DEBUG 2: Processing Q2(0,*,*) versus SPFH/Z2, for observation type
ANYSFC, over region ARMSGP, for interpolation method DW_MEAN(9), using
0 pairs.
DEBUG 2:
DEBUG 2:
--------------------------------------------------------------------------------
DEBUG 2:
DEBUG 1: Output file: /point_stat_310000L_20080526_070000V.stat
DEBUG 1: Output file: /point_stat_310000L_20080526_070000V_fho.txt
DEBUG 1: Output file: /point_stat_310000L_20080526_070000V_ctc.txt
DEBUG 1: Output file: /point_stat_310000L_20080526_070000V_cts.txt
DEBUG 1: Output file: /point_stat_310000L_20080526_070000V_cnt.txt
DEBUG 1: Output file: /point_stat_310000L_20080526_070000V_sl1l2.txt
DEBUG 1: Output file: /point_stat_310000L_20080526_070000V_sal1l2.txt
DEBUG 1: Output file: /point_stat_310000L_20080526_070000V_vl1l2.txt
DEBUG 1: Output file: /point_stat_310000L_20080526_070000V_val1l2.txt
DEBUG 1: Output file: /point_stat_310000L_20080526_070000V_mpr.txt
________________________________________
From: John Halley Gotway via RT [met_help at ucar.edu]
Sent: Monday, July 29, 2013 3:51 PM
To: Lawston, Patricia Marie. (GSFC-617.0)[GSFC INTERNS]
Subject: [rt.rap.ucar.edu #62407] Resolved: MET with GDAS
According to our records, your request has been resolved. If you have
any
further questions or concerns, please respond to this message.
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #62407] Resolved: MET with GDAS
From: John Halley Gotway
Time: Thu Aug 01 10:31:53 2013
Tricia,
This is likely more of a problem with the availability of observation
data than anything eles. I would suggest increasing the verbosity
level to 3 (-v 3). That'll give you summary output for each
verification task similar to the following:
DEBUG 3: Number of matched pairs = 2955
DEBUG 3: Observations processed = 89893
DEBUG 3: Rejected: GRIB code = 77294
DEBUG 3: Rejected: valid time = 0
DEBUG 3: Rejected: bad obs value = 0
DEBUG 3: Rejected: off the grid = 6
DEBUG 3: Rejected: level mismatch = 8077
DEBUG 3: Rejected: quality marker = 0
DEBUG 3: Rejected: message type = 332
DEBUG 3: Rejected: masking region = 1229
DEBUG 3: Rejected: bad fcst value = 0
DEBUG 3: Rejected: duplicates = 0
This tells you the number of matched pairs used for that verification
task followed by counts of reasons for why the other observations were
not used. For the off-hour verification tasks, I suspect
you'll see that many of them are thrown out due to the valid time of
the observation. Make sense?
Thanks,
John
On 08/01/2013 10:25 AM, Lawston, Patricia Marie.[GSFC INTERNS] via RT
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62407 >
>
> Hi,
>
> Thank you for your previous help. I have a new question/problem
related to the GDAS data and point stats.
>
> Point stats runs successfully with 20 to 25 matched pairs for 00,
06, 12, 18 UTC. However, any other hour results in zero matched
pairs. For now, I'm not running point stats for hours where the GDAS
observations are split between two input files (3, 9, 15, 21z), but I
would like the point stats output for all other hours. I ran PB2NC
with quality_mark_thresh = 9. I set the duplicate_flag = SINGLE in
the PointStatsConfig and my window is +/- 300 seconds. My message
type is "ANYSFC".
>
> Below is an excerpt of the terminal messages, showing a successful
hour and a subsequent hour where there are no matched pairs.
>
> Please let me know if you have any suggestions or if you need any
additional information from me.
>
> Thank you!
>
> -Tricia
>
>
> DEBUG 1: Forecast File: wrfout_d01_2008-05-26_06:00:00_PLEV
> DEBUG 1: Climatology File: none
> DEBUG 1: Observation File: gdas.2008052606_new.nc
> DEBUG 2:
> DEBUG 2:
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Reading data for T2(0,*,*).
> DEBUG 2: For T2(0,*,*) found 1 forecast levels and 0 climatology
levels.
> DEBUG 2:
> DEBUG 2:
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Reading data for Q2(0,*,*).
> DEBUG 2: For Q2(0,*,*) found 1 forecast levels and 0 climatology
levels.
> DEBUG 2:
> DEBUG 2:
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Searching 478 observations from 57 messages.
> DEBUG 2:
> DEBUG 2:
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Processing T2(0,*,*) versus TMP/Z2, for observation type
ANYSFC, over region ARMSGP, for interpolation method UW_MEAN(1), using
25 pairs.
> DEBUG 2: Computing Categorical Statistics.
> DEBUG 2: Computing Continuous Statistics.
> DEBUG 2: Computing Scalar Partial Sums.
> DEBUG 2: Processing T2(0,*,*) versus TMP/Z2, for observation type
ANYSFC, over region ARMSGP, for interpolation method MEDIAN(9), using
25 pairs.
> DEBUG 2: Computing Categorical Statistics.
> DEBUG 2: Computing Continuous Statistics.
> DEBUG 2: Computing Scalar Partial Sums.
> DEBUG 2: Processing T2(0,*,*) versus TMP/Z2, for observation type
ANYSFC, over region ARMSGP, for interpolation method DW_MEAN(9), using
25 pairs.
> DEBUG 2: Computing Categorical Statistics.
> DEBUG 2: Computing Continuous Statistics.
> DEBUG 2: Computing Scalar Partial Sums.
> DEBUG 2:
> DEBUG 2:
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 1: Output file: /point_stat_300000L_20080526_060000V.stat
> DEBUG 1: Output file: /point_stat_300000L_20080526_060000V_fho.txt
> DEBUG 1: Output file: /point_stat_300000L_20080526_060000V_ctc.txt
> DEBUG 1: Output file: /point_stat_300000L_20080526_060000V_cts.txt
> DEBUG 1: Output file: /point_stat_300000L_20080526_060000V_cnt.txt
> DEBUG 1: Output file: /point_stat_300000L_20080526_060000V_sl1l2.txt
> DEBUG 1: Output file:
/point_stat_300000L_20080526_060000V_sal1l2.txt
> DEBUG 1: Output file: /point_stat_300000L_20080526_060000V_vl1l2.txt
> DEBUG 1: Output file:
/point_stat_300000L_20080526_060000V_val1l2.txt
> DEBUG 1: Output file: /point_stat_300000L_20080526_060000V_mpr.txt
> DEBUG 1: Default Config File: PointStatConfig_default
> DEBUG 1: User Config File: PointStatConfig
> GSL_RNG_TYPE=mt19937
> GSL_RNG_SEED=1
> DEBUG 1: Forecast File: wrfout_d01_2008-05-26_07:00:00_PLEV
> DEBUG 1: Climatology File: none
> DEBUG 1: Observation File: gdas.2008052606_new.nc
> DEBUG 2:
> DEBUG 2:
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Reading data for T2(0,*,*).
> DEBUG 2: For T2(0,*,*) found 1 forecast levels and 0 climatology
levels.
> DEBUG 2:
> DEBUG 2:
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Reading data for Q2(0,*,*).
> DEBUG 2: For Q2(0,*,*) found 1 forecast levels and 0 climatology
levels.
> DEBUG 2:
> DEBUG 2:
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Searching 478 observations from 57 messages.
> DEBUG 2:
> DEBUG 2:
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Processing T2(0,*,*) versus TMP/Z2, for observation type
ANYSFC, over region ARMSGP, for interpolation method UW_MEAN(1), using
0 pairs.
> DEBUG 2: Processing T2(0,*,*) versus TMP/Z2, for observation type
ANYSFC, over region ARMSGP, for interpolation method MEDIAN(9), using
0 pairs.
> DEBUG 2: Processing T2(0,*,*) versus TMP/Z2, for observation type
ANYSFC, over region ARMSGP, for interpolation method DW_MEAN(9), using
0 pairs.
> DEBUG 2:
> DEBUG 2:
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Processing Q2(0,*,*) versus SPFH/Z2, for observation type
ANYSFC, over region ARMSGP, for interpolation method UW_MEAN(1), using
0 pairs.
> DEBUG 2: Processing Q2(0,*,*) versus SPFH/Z2, for observation type
ANYSFC, over region ARMSGP, for interpolation method MEDIAN(9), using
0 pairs.
> DEBUG 2: Processing Q2(0,*,*) versus SPFH/Z2, for observation type
ANYSFC, over region ARMSGP, for interpolation method DW_MEAN(9), using
0 pairs.
> DEBUG 2:
> DEBUG 2:
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 1: Output file: /point_stat_310000L_20080526_070000V.stat
> DEBUG 1: Output file: /point_stat_310000L_20080526_070000V_fho.txt
> DEBUG 1: Output file: /point_stat_310000L_20080526_070000V_ctc.txt
> DEBUG 1: Output file: /point_stat_310000L_20080526_070000V_cts.txt
> DEBUG 1: Output file: /point_stat_310000L_20080526_070000V_cnt.txt
> DEBUG 1: Output file: /point_stat_310000L_20080526_070000V_sl1l2.txt
> DEBUG 1: Output file:
/point_stat_310000L_20080526_070000V_sal1l2.txt
> DEBUG 1: Output file: /point_stat_310000L_20080526_070000V_vl1l2.txt
> DEBUG 1: Output file:
/point_stat_310000L_20080526_070000V_val1l2.txt
> DEBUG 1: Output file: /point_stat_310000L_20080526_070000V_mpr.txt
>
>
>
>
>
>
> ________________________________________
> From: John Halley Gotway via RT [met_help at ucar.edu]
> Sent: Monday, July 29, 2013 3:51 PM
> To: Lawston, Patricia Marie. (GSFC-617.0)[GSFC INTERNS]
> Subject: [rt.rap.ucar.edu #62407] Resolved: MET with GDAS
>
> According to our records, your request has been resolved. If you
have any
> further questions or concerns, please respond to this message.
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #62407] Resolved: MET with GDAS
From: Lawston, Patricia Marie.[GSFC INTERNS]
Time: Thu Aug 01 10:56:54 2013
Hi John,
Thanks for your suggestion. You were right; all of the points were
being rejected for one reason or another:
DEBUG 3: Number of matched pairs = 0
DEBUG 3: Observations processed = 1322
DEBUG 3: Rejected: GRIB code = 1170
DEBUG 3: Rejected: valid time = 152
DEBUG 3: Rejected: bad obs value = 0
DEBUG 3: Rejected: off the grid = 0
DEBUG 3: Rejected: level mismatch = 0
DEBUG 3: Rejected: message type = 0
DEBUG 3: Rejected: masking region = 0
DEBUG 3: Rejected: bad fcst value = 0
I appreciate the help!
Thanks,
Tricia
________________________________________
From: John Halley Gotway via RT [met_help at ucar.edu]
Sent: Thursday, August 01, 2013 11:31 AM
To: Lawston, Patricia Marie. (GSFC-617.0)[GSFC INTERNS]
Subject: Re: [rt.rap.ucar.edu #62407] Resolved: MET with GDAS
Tricia,
This is likely more of a problem with the availability of observation
data than anything eles. I would suggest increasing the verbosity
level to 3 (-v 3). That'll give you summary output for each
verification task similar to the following:
DEBUG 3: Number of matched pairs = 2955
DEBUG 3: Observations processed = 89893
DEBUG 3: Rejected: GRIB code = 77294
DEBUG 3: Rejected: valid time = 0
DEBUG 3: Rejected: bad obs value = 0
DEBUG 3: Rejected: off the grid = 6
DEBUG 3: Rejected: level mismatch = 8077
DEBUG 3: Rejected: quality marker = 0
DEBUG 3: Rejected: message type = 332
DEBUG 3: Rejected: masking region = 1229
DEBUG 3: Rejected: bad fcst value = 0
DEBUG 3: Rejected: duplicates = 0
This tells you the number of matched pairs used for that verification
task followed by counts of reasons for why the other observations were
not used. For the off-hour verification tasks, I suspect
you'll see that many of them are thrown out due to the valid time of
the observation. Make sense?
Thanks,
John
On 08/01/2013 10:25 AM, Lawston, Patricia Marie.[GSFC INTERNS] via RT
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=62407 >
>
> Hi,
>
> Thank you for your previous help. I have a new question/problem
related to the GDAS data and point stats.
>
> Point stats runs successfully with 20 to 25 matched pairs for 00,
06, 12, 18 UTC. However, any other hour results in zero matched
pairs. For now, I'm not running point stats for hours where the GDAS
observations are split between two input files (3, 9, 15, 21z), but I
would like the point stats output for all other hours. I ran PB2NC
with quality_mark_thresh = 9. I set the duplicate_flag = SINGLE in
the PointStatsConfig and my window is +/- 300 seconds. My message
type is "ANYSFC".
>
> Below is an excerpt of the terminal messages, showing a successful
hour and a subsequent hour where there are no matched pairs.
>
> Please let me know if you have any suggestions or if you need any
additional information from me.
>
> Thank you!
>
> -Tricia
>
>
> DEBUG 1: Forecast File: wrfout_d01_2008-05-26_06:00:00_PLEV
> DEBUG 1: Climatology File: none
> DEBUG 1: Observation File: gdas.2008052606_new.nc
> DEBUG 2:
> DEBUG 2:
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Reading data for T2(0,*,*).
> DEBUG 2: For T2(0,*,*) found 1 forecast levels and 0 climatology
levels.
> DEBUG 2:
> DEBUG 2:
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Reading data for Q2(0,*,*).
> DEBUG 2: For Q2(0,*,*) found 1 forecast levels and 0 climatology
levels.
> DEBUG 2:
> DEBUG 2:
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Searching 478 observations from 57 messages.
> DEBUG 2:
> DEBUG 2:
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Processing T2(0,*,*) versus TMP/Z2, for observation type
ANYSFC, over region ARMSGP, for interpolation method UW_MEAN(1), using
25 pairs.
> DEBUG 2: Computing Categorical Statistics.
> DEBUG 2: Computing Continuous Statistics.
> DEBUG 2: Computing Scalar Partial Sums.
> DEBUG 2: Processing T2(0,*,*) versus TMP/Z2, for observation type
ANYSFC, over region ARMSGP, for interpolation method MEDIAN(9), using
25 pairs.
> DEBUG 2: Computing Categorical Statistics.
> DEBUG 2: Computing Continuous Statistics.
> DEBUG 2: Computing Scalar Partial Sums.
> DEBUG 2: Processing T2(0,*,*) versus TMP/Z2, for observation type
ANYSFC, over region ARMSGP, for interpolation method DW_MEAN(9), using
25 pairs.
> DEBUG 2: Computing Categorical Statistics.
> DEBUG 2: Computing Continuous Statistics.
> DEBUG 2: Computing Scalar Partial Sums.
> DEBUG 2:
> DEBUG 2:
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 1: Output file: /point_stat_300000L_20080526_060000V.stat
> DEBUG 1: Output file: /point_stat_300000L_20080526_060000V_fho.txt
> DEBUG 1: Output file: /point_stat_300000L_20080526_060000V_ctc.txt
> DEBUG 1: Output file: /point_stat_300000L_20080526_060000V_cts.txt
> DEBUG 1: Output file: /point_stat_300000L_20080526_060000V_cnt.txt
> DEBUG 1: Output file: /point_stat_300000L_20080526_060000V_sl1l2.txt
> DEBUG 1: Output file:
/point_stat_300000L_20080526_060000V_sal1l2.txt
> DEBUG 1: Output file: /point_stat_300000L_20080526_060000V_vl1l2.txt
> DEBUG 1: Output file:
/point_stat_300000L_20080526_060000V_val1l2.txt
> DEBUG 1: Output file: /point_stat_300000L_20080526_060000V_mpr.txt
> DEBUG 1: Default Config File: PointStatConfig_default
> DEBUG 1: User Config File: PointStatConfig
> GSL_RNG_TYPE=mt19937
> GSL_RNG_SEED=1
> DEBUG 1: Forecast File: wrfout_d01_2008-05-26_07:00:00_PLEV
> DEBUG 1: Climatology File: none
> DEBUG 1: Observation File: gdas.2008052606_new.nc
> DEBUG 2:
> DEBUG 2:
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Reading data for T2(0,*,*).
> DEBUG 2: For T2(0,*,*) found 1 forecast levels and 0 climatology
levels.
> DEBUG 2:
> DEBUG 2:
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Reading data for Q2(0,*,*).
> DEBUG 2: For Q2(0,*,*) found 1 forecast levels and 0 climatology
levels.
> DEBUG 2:
> DEBUG 2:
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Searching 478 observations from 57 messages.
> DEBUG 2:
> DEBUG 2:
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Processing T2(0,*,*) versus TMP/Z2, for observation type
ANYSFC, over region ARMSGP, for interpolation method UW_MEAN(1), using
0 pairs.
> DEBUG 2: Processing T2(0,*,*) versus TMP/Z2, for observation type
ANYSFC, over region ARMSGP, for interpolation method MEDIAN(9), using
0 pairs.
> DEBUG 2: Processing T2(0,*,*) versus TMP/Z2, for observation type
ANYSFC, over region ARMSGP, for interpolation method DW_MEAN(9), using
0 pairs.
> DEBUG 2:
> DEBUG 2:
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 2: Processing Q2(0,*,*) versus SPFH/Z2, for observation type
ANYSFC, over region ARMSGP, for interpolation method UW_MEAN(1), using
0 pairs.
> DEBUG 2: Processing Q2(0,*,*) versus SPFH/Z2, for observation type
ANYSFC, over region ARMSGP, for interpolation method MEDIAN(9), using
0 pairs.
> DEBUG 2: Processing Q2(0,*,*) versus SPFH/Z2, for observation type
ANYSFC, over region ARMSGP, for interpolation method DW_MEAN(9), using
0 pairs.
> DEBUG 2:
> DEBUG 2:
--------------------------------------------------------------------------------
> DEBUG 2:
> DEBUG 1: Output file: /point_stat_310000L_20080526_070000V.stat
> DEBUG 1: Output file: /point_stat_310000L_20080526_070000V_fho.txt
> DEBUG 1: Output file: /point_stat_310000L_20080526_070000V_ctc.txt
> DEBUG 1: Output file: /point_stat_310000L_20080526_070000V_cts.txt
> DEBUG 1: Output file: /point_stat_310000L_20080526_070000V_cnt.txt
> DEBUG 1: Output file: /point_stat_310000L_20080526_070000V_sl1l2.txt
> DEBUG 1: Output file:
/point_stat_310000L_20080526_070000V_sal1l2.txt
> DEBUG 1: Output file: /point_stat_310000L_20080526_070000V_vl1l2.txt
> DEBUG 1: Output file:
/point_stat_310000L_20080526_070000V_val1l2.txt
> DEBUG 1: Output file: /point_stat_310000L_20080526_070000V_mpr.txt
>
>
>
>
>
>
> ________________________________________
> From: John Halley Gotway via RT [met_help at ucar.edu]
> Sent: Monday, July 29, 2013 3:51 PM
> To: Lawston, Patricia Marie. (GSFC-617.0)[GSFC INTERNS]
> Subject: [rt.rap.ucar.edu #62407] Resolved: MET with GDAS
>
> According to our records, your request has been resolved. If you
have any
> further questions or concerns, please respond to this message.
>
------------------------------------------------
More information about the Met_help
mailing list