[Met_help] [rt.rap.ucar.edu #61157] History for No matching pairs
John Halley Gotway via RT
met_help at ucar.edu
Thu May 2 10:34:31 MDT 2013
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Dear all,
I am trying to use the point stat tool of METv4.0 in order to compare surface observations from meteorological stations around Greece to WRF-ARW model output.
After doing the steps required (I think), I get no matching pairs. Do you have any idea what happens?
The steps I take are:
1. Making the ascii file in the format required by ascii2nc (observation_for_met.txt)
2. Running ascii2nc and outputing file observation.nc
3. Running run_unipost (UPP) for converting the wrf output (wrfout_d01_2006-01-01_23) to grib unstaggered grid (WRFPRS_d01.19)
4. Running point_stat with inputs the UPP output file (WRFPRS_d01.19), the observation file (observation.nc), the configuration file (PointStatConfig_test) and the output folder with the -v 2 option.
The output files generated are empty (only headers) and I see the message that 0 pairs were used (log file).
Please tell me where to download the refered files, if you need it.
Thanks a lot,
Dr Emmanouil George
PhD in Atmospheric Physics/Meteorology
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #61157] No matching pairs
From: John Halley Gotway
Time: Mon Apr 29 12:08:18 2013
Dr Emmanouil George,
Those steps all sound correct. Please try rerunning the Point-Stat
tool at verbosity level 3 (-v 3). For each verification task, that'll
give you a list of reasons why the observations were
rejected, which will give us idea what's going on. For example:
DEBUG 2: Processing VGRD/Z10 versus VGRD/Z10, for observation type
ADPSFC, over region LMV, for interpolation method DW_MEAN(9), using
364 pairs.
DEBUG 3: Number of matched pairs = 364
DEBUG 3: Observations processed = 89892
DEBUG 3: Rejected: GRIB code = 77293
DEBUG 3: Rejected: valid time = 0
DEBUG 3: Rejected: bad obs value = 0
DEBUG 3: Rejected: off the grid = 6
DEBUG 3: Rejected: level mismatch = 8077
DEBUG 3: Rejected: quality marker = 0
DEBUG 3: Rejected: message type = 332
DEBUG 3: Rejected: masking region = 3820
DEBUG 3: Rejected: bad fcst value = 0
DEBUG 3: Rejected: duplicates = 0
DEBUG 2: Computing Categorical Statistics.
DEBUG 2: Computing Continuous Statistics.
DEBUG 2: Computing Vector Partial Sums, for forecast wind speed NA,
for observation wind speed NA, using 364 pairs.
This output tells us that when verifying 10-meter V-component of wind,
364 out of 89892 observations were used. The rest were rejected
because of a GRIB code, vertical level, message type, and
masking region mismatch.
Please give that a try and let me know what you find.
You can post data to our anonymous ftp site by following these
instructions:
http://www.dtcenter.org/met/users/support/met_help.php#ftp
If you'd like to send me data for this case, please include:
- ascii observation file
- GRIB1 output of UPP
- Point-Stat config file
- how you're calling Point-Stat on the command line
Thanks,
John Halley Gotway
met_help at ucar.edu
On 04/28/2013 02:29 AM, George Emmanouil via RT wrote:
>
> Sun Apr 28 02:29:36 2013: Request 61157 was acted upon.
> Transaction: Ticket created by george.emmanouil at yahoo.gr
> Queue: met_help
> Subject: No matching pairs
> Owner: Nobody
> Requestors: george.emmanouil at yahoo.gr
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=61157 >
>
>
> Dear all,
>
> I am trying to use the point stat tool of METv4.0 in order to
compare surface observations from meteorological stations around
Greece to WRF-ARW model output.
> After doing the steps required (I think), I get no matching pairs.
Do you have any idea what happens?
>
> The steps I take are:
> 1. Making the ascii file in the format required by ascii2nc
(observation_for_met.txt)
> 2. Running ascii2nc and outputing file observation.nc
> 3. Running run_unipost (UPP) for converting the wrf output
(wrfout_d01_2006-01-01_23) to grib unstaggered grid (WRFPRS_d01.19)
> 4. Running point_stat with inputs the UPP output file
(WRFPRS_d01.19), the observation file (observation.nc), the
configuration file (PointStatConfig_test) and the output folder with
the -v 2 option.
>
> The output files generated are empty (only headers) and I see the
message that 0 pairs were used (log file).
>
> Please tell me where to download the refered files, if you need it.
>
>
> Thanks a lot,
>
> Dr Emmanouil George
> PhD in Atmospheric Physics/Meteorology
>
------------------------------------------------
Subject: No matching pairs
From: George Emmanouil
Time: Wed May 01 04:10:34 2013
Dear John,
I hope you are doing well.
I want to thank you for your help.
I have put in your ftp-site the following files:
- point_stat.log: includes the command line for point_stat and the log
file when running with -v 3 option
I see that the 3 observations that I want to have statistics
are "off the grid".
But I think that my grid, as I see it in my wrfout file, is
around Greece and the 3 observations are also. So?
- the wrfout file that I used for the stats
- the observation file (both in txt and nc format)
- the output of point stat
- the config file
- the grib outputs of UPP and the run_unipost script
All these files with their names are described in my previous e-
mail.
Thank you again,
Dr Emmanouil George
PhD in Atmospheric Physics/Meteorology
________________________________
Απο: John Halley Gotway via RT <met_help at ucar.edu>
Προς: george.emmanouil at yahoo.gr
Στάλθηκε: 9:08 μ.μ. Δευτέρα, 29 Απριλίου 2013
Θέμα: Re: [rt.rap.ucar.edu #61157] No matching pairs
Dr Emmanouil George,
Those steps all sound correct. Please try rerunning the Point-Stat
tool at verbosity level 3 (-v 3). For each verification task, that'll
give you a list of reasons why the observations were
rejected, which will give us idea what's going on. For example:
DEBUG 2: Processing VGRD/Z10 versus VGRD/Z10, for observation type
ADPSFC, over region LMV, for interpolation method DW_MEAN(9), using
364 pairs.
DEBUG 3: Number of matched pairs = 364
DEBUG 3: Observations processed = 89892
DEBUG 3: Rejected: GRIB code = 77293
DEBUG 3: Rejected: valid time = 0
DEBUG 3: Rejected: bad obs value = 0
DEBUG 3: Rejected: off the grid = 6
DEBUG 3: Rejected: level mismatch = 8077
DEBUG 3: Rejected: quality marker = 0
DEBUG 3: Rejected: message type = 332
DEBUG 3: Rejected: masking region = 3820
DEBUG 3: Rejected: bad fcst value = 0
DEBUG 3: Rejected: duplicates = 0
DEBUG 2: Computing Categorical Statistics.
DEBUG 2: Computing Continuous Statistics.
DEBUG 2: Computing Vector Partial Sums, for forecast wind speed NA,
for observation wind speed NA, using 364 pairs.
This output tells us that when verifying 10-meter V-component of wind,
364 out of 89892 observations were used. The rest were rejected
because of a GRIB code, vertical level, message type, and
masking region mismatch.
Please give that a try and let me know what you find.
You can post data to our anonymous ftp site by following these
instructions:
http://www.dtcenter.org/met/users/support/met_help.php#ftp
If you'd like to send me data for this case, please include:
- ascii observation file
- GRIB1 output of UPP
- Point-Stat config file
- how you're calling Point-Stat on the command line
Thanks,
John Halley Gotway
met_help at ucar.edu
On 04/28/2013 02:29 AM, George Emmanouil via RT wrote:
>
> Sun Apr 28 02:29:36 2013: Request 61157 was acted upon.
> Transaction: Ticket created by george.emmanouil at yahoo.gr
> Queue: met_help
> Subject: No matching pairs
> Owner: Nobody
> Requestors: george.emmanouil at yahoo.gr
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=61157>
>
>
> Dear all,
>
> I am trying to use the point stat tool of METv4.0 in order to
compare surface observations from meteorological stations around
Greece to WRF-ARW model output.
> After doing the steps required (I think), I get no matching pairs.
Do you have any idea what happens?
>
> The steps I take are:
> 1. Making the ascii file in the format required by ascii2nc
(observation_for_met.txt)
> 2. Running ascii2nc and outputing file observation.nc
> 3. Running run_unipost (UPP) for converting the wrf output
(wrfout_d01_2006-01-01_23) to grib unstaggered grid (WRFPRS_d01.19)
> 4. Running point_stat with inputs the UPP output file
(WRFPRS_d01.19), the observation file (observation.nc), the
configuration file (PointStatConfig_test) and the output folder with
the -v 2 option.
>
> The output files generated are empty (only headers) and I see the
message that 0 pairs were used (log file).
>
> Please tell me where to download the refered files, if you need it.
>
>
> Thanks a lot,
>
> Dr Emmanouil George
> PhD in Atmospheric Physics/Meteorology
>
------------------------------------------------
Subject: No matching pairs
From: John Halley Gotway
Time: Wed May 01 10:57:45 2013
Dr Emmanouil George,
Thanks for sending the data. I ran Point-Stat on the data you sent
and saw the following log messages:
DEBUG 2: Processing TMP/Z2 versus TMP/Z2, for observation type ADPSFC,
over region FULL, for interpolation method UW_MEAN(25), using 0 pairs.
DEBUG 3: Number of matched pairs = 0
DEBUG 3: Observations processed = 3142
DEBUG 3: Rejected: GRIB code = 0
DEBUG 3: Rejected: valid time = 3139
DEBUG 3: Rejected: bad obs value = 0
DEBUG 3: Rejected: off the grid = 3
DEBUG 3: Rejected: level mismatch = 0
DEBUG 3: Rejected: message type = 0
DEBUG 3: Rejected: masking region = 0
DEBUG 3: Rejected: bad fcst value = 0
This is saying that Point-Stat read 3142 observations from the point
observation file. 3139 were not close enough to the valid time of the
forecast to be used. And the other 3 that were close enough
in time did not fall within your model domain.
My next step was to plot the location of your model data and your
observations to figure out the mismatch:
# Plot 2-m temperature
METv4.0/bin/plot_data_plane WRFPRS_d01.67 WRFPRS_d01.67_TMP_Z2.ps
'name="TMP"; level="Z2";'
# Plot temperature point observations
METv4.0/bin/plot_point_obs observation.nc observation.ps
I converted the output PostScript images to png and have attached them
to this message. It's pretty tough to see, but in "observation.png"
there are 3 red dots over Saudi-Arabia.
That's where your point observations are located.
So that's the source of the problem - your model data is over Greece
while your point observations are over Saudi-Arabia.
Here are the unique combinations of Lat/Lon values in the file
"observation_for_met.txt":
LAT LON
21.42 38.15
23.75 38.05
25.88 40.85
I next thought that perhaps you have your lat/lon values switched
around. So I switched them, reran the data through ASCII2NC, and
replotted the data. They now fall in Greece. That image is
attached in (observation_switch.png). I reran that data through
Point-Stat and got 3 matched pairs.
So that's the source of your problem. You listed "LON LAT" in your
ASCII point observations, when it should have been "LAT LON".
Hope that helps.
Thanks,
John
On 05/01/2013 04:10 AM, George Emmanouil via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=61157 >
>
> Dear John,
>
> I hope you are doing well.
> I want to thank you for your help.
>
> I have put in your ftp-site the following files:
> - point_stat.log: includes the command line for point_stat and the
log file when running with -v 3 option
> I see that the 3 observations that I want to have
statistics are "off the grid".
> But I think that my grid, as I see it in my wrfout file,
is around Greece and the 3 observations are also. So?
> - the wrfout file that I used for the stats
> - the observation file (both in txt and nc format)
> - the output of point stat
> - the config file
> - the grib outputs of UPP and the run_unipost script
>
> All these files with their names are described in my previous e-
mail.
>
> Thank you again,
>
> Dr Emmanouil George
> PhD in Atmospheric Physics/Meteorology
>
>
> ________________________________
> Απο: John Halley Gotway via RT <met_help at ucar.edu>
> Προς: george.emmanouil at yahoo.gr
> Στάλθηκε: 9:08 μ.μ. Δευτέρα, 29 Απριλίου 2013
> Θέμα: Re: [rt.rap.ucar.edu #61157] No matching pairs
>
>
> Dr Emmanouil George,
>
> Those steps all sound correct. Please try rerunning the Point-Stat
tool at verbosity level 3 (-v 3). For each verification task, that'll
give you a list of reasons why the observations were
> rejected, which will give us idea what's going on. For example:
>
> DEBUG 2: Processing VGRD/Z10 versus VGRD/Z10, for observation type
ADPSFC, over region LMV, for interpolation method DW_MEAN(9), using
364 pairs.
> DEBUG 3: Number of matched pairs = 364
> DEBUG 3: Observations processed = 89892
> DEBUG 3: Rejected: GRIB code = 77293
> DEBUG 3: Rejected: valid time = 0
> DEBUG 3: Rejected: bad obs value = 0
> DEBUG 3: Rejected: off the grid = 6
> DEBUG 3: Rejected: level mismatch = 8077
> DEBUG 3: Rejected: quality marker = 0
> DEBUG 3: Rejected: message type = 332
> DEBUG 3: Rejected: masking region = 3820
> DEBUG 3: Rejected: bad fcst value = 0
> DEBUG 3: Rejected: duplicates = 0
> DEBUG 2: Computing Categorical Statistics.
> DEBUG 2: Computing Continuous Statistics.
> DEBUG 2: Computing Vector Partial Sums, for forecast wind speed NA,
for observation wind speed NA, using 364 pairs.
>
> This output tells us that when verifying 10-meter V-component of
wind, 364 out of 89892 observations were used. The rest were rejected
because of a GRIB code, vertical level, message type, and
> masking region mismatch.
>
> Please give that a try and let me know what you find.
>
> You can post data to our anonymous ftp site by following these
instructions:
> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>
> If you'd like to send me data for this case, please include:
> - ascii observation file
> - GRIB1 output of UPP
> - Point-Stat config file
> - how you're calling Point-Stat on the command line
>
> Thanks,
> John Halley Gotway
> met_help at ucar.edu
>
>
> On 04/28/2013 02:29 AM, George Emmanouil via RT wrote:
>>
>> Sun Apr 28 02:29:36 2013: Request 61157 was acted upon.
>> Transaction: Ticket created by george.emmanouil at yahoo.gr
>> Queue: met_help
>> Subject: No matching pairs
>> Owner: Nobody
>> Requestors: george.emmanouil at yahoo.gr
>> Status: new
>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=61157>
>>
>>
>> Dear all,
>>
>> I am trying to use the point stat tool of METv4.0 in order to
compare surface observations from meteorological stations around
Greece to WRF-ARW model output.
>> After doing the steps required (I think), I get no matching pairs.
Do you have any idea what happens?
>>
>> The steps I take are:
>> 1. Making the ascii file in the format required by ascii2nc
(observation_for_met.txt)
>> 2. Running ascii2nc and outputing file observation.nc
>> 3. Running run_unipost (UPP) for converting the wrf output
(wrfout_d01_2006-01-01_23) to grib unstaggered grid (WRFPRS_d01.19)
>> 4. Running point_stat with inputs the UPP output file
(WRFPRS_d01.19), the observation file (observation.nc), the
configuration file (PointStatConfig_test) and the output folder with
the -v 2 option.
>>
>> The output files generated are empty (only headers) and I see the
message that 0 pairs were used (log file).
>>
>> Please tell me where to download the refered files, if you need it.
>>
>>
>> Thanks a lot,
>>
>> Dr Emmanouil George
>> PhD in Atmospheric Physics/Meteorology
>>
------------------------------------------------
Subject: No matching pairs
From: George Emmanouil
Time: Thu May 02 02:09:22 2013
Dr John,
I want to thank you for your help. My mind did not go to this point.
My fault was in the include file of the observations.
You were really helpfull to me.
I want to thank you again,
Emmanouil George
PhD in Atmospheric Physics/Meteorology
________________________________
Απο: John Halley Gotway via RT <met_help at ucar.edu>
Προς: george.emmanouil at yahoo.gr
Στάλθηκε: 7:57 μ.μ. Τετάρτη, 1 Μαΐου 2013
Θέμα: Re: Σχετ: [rt.rap.ucar.edu #61157] No matching pairs
Dr Emmanouil George,
Thanks for sending the data. I ran Point-Stat on the data you sent
and saw the following log messages:
DEBUG 2: Processing TMP/Z2 versus TMP/Z2, for observation type ADPSFC,
over region FULL, for interpolation method UW_MEAN(25), using 0 pairs.
DEBUG 3: Number of matched pairs = 0
DEBUG 3: Observations processed = 3142
DEBUG 3: Rejected: GRIB code = 0
DEBUG 3: Rejected: valid time = 3139
DEBUG 3: Rejected: bad obs value = 0
DEBUG 3: Rejected: off the grid = 3
DEBUG 3: Rejected: level mismatch = 0
DEBUG 3: Rejected: message type = 0
DEBUG 3: Rejected: masking region = 0
DEBUG 3: Rejected: bad fcst value = 0
This is saying that Point-Stat read 3142 observations from the point
observation file. 3139 were not close enough to the valid time of the
forecast to be used. And the other 3 that were close enough
in time did not fall within your model domain.
My next step was to plot the location of your model data and your
observations to figure out the mismatch:
# Plot 2-m temperature
METv4.0/bin/plot_data_plane WRFPRS_d01.67 WRFPRS_d01.67_TMP_Z2.ps
'name="TMP"; level="Z2";'
# Plot temperature point observations
METv4.0/bin/plot_point_obs observation.nc observation.ps
I converted the output PostScript images to png and have attached them
to this message. It's pretty tough to see, but in "observation.png"
there are 3 red dots over Saudi-Arabia.
That's where your point observations are located.
So that's the source of the problem - your model data is over Greece
while your point observations are over Saudi-Arabia.
Here are the unique combinations of Lat/Lon values in the file
"observation_for_met.txt":
LAT LON
21.42 38.15
23.75 38.05
25.88 40.85
I next thought that perhaps you have your lat/lon values switched
around. So I switched them, reran the data through ASCII2NC, and
replotted the data. They now fall in Greece. That image is
attached in (observation_switch.png). I reran that data through
Point-Stat and got 3 matched pairs.
So that's the source of your problem. You listed "LON LAT" in your
ASCII point observations, when it should have been "LAT LON".
Hope that helps.
Thanks,
John
On 05/01/2013 04:10 AM, George Emmanouil via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=61157>
>
> Dear John,
>
> I hope you are doing well.
> I want to thank you for your help.
>
> I have put in your ftp-site the following files:
> - point_stat.log: includes the command line for point_stat and the
log file when running with -v 3 option
> I see that the 3 observations that I want to have
statistics are "off the grid".
> But I think that my grid, as I see it in my wrfout file,
is around Greece and the 3 observations are also. So?
> - the wrfout file that I used for the stats
> - the observation file (both in txt and nc format)
> - the output of point stat
> - the config file
> - the grib outputs of UPP and the run_unipost script
>
> All these files with their names are described in my previous e-
mail.
>
> Thank you again,
>
> Dr Emmanouil George
> PhD in Atmospheric Physics/Meteorology
>
>
> ________________________________
> Απο: John Halley Gotway via RT <met_help at ucar.edu>
> Προς: george.emmanouil at yahoo.gr
> Στάλθηκε: 9:08 μ.μ. Δευτέρα, 29 Απριλίου 2013
> Θέμα: Re: [rt.rap.ucar.edu #61157] No matching pairs
>
>
> Dr Emmanouil George,
>
> Those steps all sound correct. Please try rerunning the Point-Stat
tool at verbosity level 3 (-v 3). For each verification task, that'll
give you a list of reasons why the observations were
> rejected, which will give us idea what's going on. For example:
>
> DEBUG 2: Processing VGRD/Z10 versus VGRD/Z10, for observation type
ADPSFC, over region LMV, for interpolation method DW_MEAN(9), using
364 pairs.
> DEBUG 3: Number of matched pairs = 364
> DEBUG 3: Observations processed = 89892
> DEBUG 3: Rejected: GRIB code = 77293
> DEBUG 3: Rejected: valid time = 0
> DEBUG 3: Rejected: bad obs value = 0
> DEBUG 3: Rejected: off the grid = 6
> DEBUG 3: Rejected: level mismatch = 8077
> DEBUG 3: Rejected: quality marker = 0
> DEBUG 3: Rejected: message type = 332
> DEBUG 3: Rejected: masking region = 3820
> DEBUG 3: Rejected: bad fcst value = 0
> DEBUG 3: Rejected: duplicates = 0
> DEBUG 2: Computing Categorical Statistics.
> DEBUG 2: Computing Continuous Statistics.
> DEBUG 2: Computing Vector Partial Sums, for forecast wind speed NA,
for observation wind speed NA, using 364 pairs.
>
> This output tells us that when verifying 10-meter V-component of
wind, 364 out of 89892 observations were used. The rest were rejected
because of a GRIB code, vertical level, message type, and
> masking region mismatch.
>
> Please give that a try and let me know what you find.
>
> You can post data to our anonymous ftp site by following these
instructions:
> http://www.dtcenter.org/met/users/support/met_help.php#ftp
>
> If you'd like to send me data for this case, please include:
> - ascii observation file
> - GRIB1 output of UPP
> - Point-Stat config file
> - how you're calling Point-Stat on the command line
>
> Thanks,
> John Halley Gotway
> met_help at ucar.edu
>
>
> On 04/28/2013 02:29 AM, George Emmanouil via RT wrote:
>>
>> Sun Apr 28 02:29:36 2013: Request 61157 was acted upon.
>> Transaction: Ticket created by george.emmanouil at yahoo.gr
>> Queue: met_help
>> Subject: No matching pairs
>> Owner: Nobody
>> Requestors: george.emmanouil at yahoo.gr
>> Status: new
>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=61157>
>>
>>
>> Dear all,
>>
>> I am trying to use the point stat tool of METv4.0 in order to
compare surface observations from meteorological stations around
Greece to WRF-ARW model output.
>> After doing the steps required (I think), I get no matching pairs.
Do you have any idea what happens?
>>
>> The steps I take are:
>> 1. Making the ascii file in the format required by ascii2nc
(observation_for_met.txt)
>> 2. Running ascii2nc and outputing file observation.nc
>> 3. Running run_unipost (UPP) for converting the wrf output
(wrfout_d01_2006-01-01_23) to grib unstaggered grid (WRFPRS_d01.19)
>> 4. Running point_stat with inputs the UPP output file
(WRFPRS_d01.19), the observation file (observation.nc), the
configuration file (PointStatConfig_test) and the output folder with
the -v 2 option.
>>
>> The output files generated are empty (only headers) and I see the
message that 0 pairs were used (log file).
>>
>> Please tell me where to download the refered files, if you need it.
>>
>>
>> Thanks a lot,
>>
>> Dr Emmanouil George
>> PhD in Atmospheric Physics/Meteorology
>>
------------------------------------------------
More information about the Met_help
mailing list