[Met_help] [rt.rap.ucar.edu #52210] History for about the ADPUPA verification
John Halley Gotway via RT
met_help at ucar.edu
Thu Jan 5 13:36:42 MST 2012
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Dear Sir:
Here is the output from one of "*mpr.txt". In the text below, we can see the forecast vaule is not varying. I'm not sure why. One reason i can guess from the text is that most of observation points level is missing "NA". This unchanged forecast value will lead large RMSE in that level. The level i posted is between 150-10hPa.
Here are also the common session of this section"
V3.0 WRF 120000 20100515_120000 20100515_120000 000000 20100515_120000 20100515_120000 TMP P150-10 TMP P150-10 ADPUPA FULL DW_MEAN 4 NA NA NA NA MPR 2367"
I have checked *cnt.txt files. The "TOTAL" in that file is 2367 which means all the pairs below is counted in the verification. I'm just wondering how the forecast value calculated when the observation level is missing? And can default program get rid of these points in the calculation of RMSE or other verification scores?
Many thanks!
INDEX OBS_SID OBS_LAT OBS_LON OBS_LVL OBS_ELV FCST OBS CLIMO
439 72201 24.55000 -81.79000 50.00000 20731.58008 214.66865 209.85001 NA
440 72201 24.55000 -81.79000 47.00000 NA 214.66865 211.25000 NA
441 72201 24.55000 -81.79000 45.10000 NA 214.66865 210.25000 NA
442 72201 24.55000 -81.79000 39.70000 NA 214.66865 215.85001 NA
443 72201 24.55000 -81.79000 36.10000 NA 214.66865 217.05000 NA
444 72201 24.55000 -81.79000 30.00000 23934.91602 214.66865 214.05000 NA
445 72201 24.55000 -81.79000 29.00000 NA 214.66865 214.05000 NA
446 72201 24.55000 -81.79000 27.30000 NA 214.66865 217.85001 NA
448 72201 24.55000 -81.79000 25.60000 NA 214.66865 218.64999 NA
449 72201 24.55000 -81.79000 24.90000 NA 214.66865 220.64999 NA
450 72201 24.55000 -81.79000 24.00000 NA 214.66865 219.85001 NA
451 72201 24.55000 -81.79000 22.30000 NA 214.66865 221.05000 NA
452 72201 24.55000 -81.79000 20.90000 NA 214.66865 224.45000 NA
453 72201 24.55000 -81.79000 20.00000 26547.63477 214.66865 224.85001 NA
454 72201 24.55000 -81.79000 16.60000 27765.90430 214.66865 227.05000 NA
455 72201 24.55000 -81.79000 13.50000 NA 214.66865 232.45000 NA
456 72201 24.55000 -81.79000 12.80000 NA 214.66865 231.45000 NA
457 72201 24.55000 -81.79000 10.00000 31212.49219 214.66865 232.25000 NA
kefeng
2011-12-21
kefeng
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #52210] about the ADPUPA verification
From: John Halley Gotway
Time: Thu Dec 22 09:45:12 2011
Kefeng,
Hmmm, interesting. This a good question.
The NA values in the OBS_ELV column do not impact the computation of
the statistics, and it doesn't make any sense to throw them away. And
here's why...
In the header columns, I see that you're verifying temperature against
upper-air observations between 150mb and 10mb. In the sample MPR
output you sent, I see that this is basically sounding data -
the station ID and lat/lon remain constant while the contents of the
"OBS_LVL" column changes. That "OBS_LVL" column indicates the
pressure level at which the observation occurred. Sine you're
verifying a range of pressure levels (150 to 10mb), these observations
are used because they fall in that range. The "OBS_ELV" column is the
meters above sea level at which the observations occurred.
Since you're verifying on pressure levels in this case, that
elevation information is not used at all by Point-Stat.
The real question is, why does the forecast value remain constant for
all of these observations. If I were you, I would look at your model
output to see at what levels do you have temperature
records? My guess is that you don't have any model output higher than
50 mb. So all of the observations higher than 50 mb are getting
compared to the forecast field at 50mb. That would explain why
the forecast value doesn't change.
Ultimately, it's your responsibility to know your data and make
sensible choices about what and how to do the verification. If you
don't have any model output higher than 50mb, it doesn't make sense
to verify there.
Hope that helps. If you have more questions about your particular
dataset, it would probably be easiest to just have you send us a
sample of data along with your question.
We'd need your Point-Stat forecast file, observation file,
configuration file, and a the command line you use to call Point-Stat.
You can follow the directions for sending us data listed here:
http://www.dtcenter.org/met/users/support/met_help.php#ftp
By the way, I thought I'd mention that there is in fact a way to use
the STAT-Analysis tool to filter out data and recompute statistics.
That isn't really the point of the issue you're having, but I
thought I'd mention it. The following type of job would read the data
MPR lines, filter out only those lines where OBS_ELV > 0, and then
recompute CNT statistics. That would basically throw out the
lines where OBS_ELV = NA:
stat_analysis -lookin /path/to/stat/file/with/MPR/lines -job
aggregate_stat -line_type MPR -out_line_type CNT -column_min OBS_ELV 0
Hope that helps.
Thanks,
John
On 12/21/2011 10:39 AM, kefeng via RT wrote:
>
> Wed Dec 21 10:39:23 2011: Request 52210 was acted upon.
> Transaction: Ticket created by kefeng at ou.edu
> Queue: met_help
> Subject: about the ADPUPA verification
> Owner: Nobody
> Requestors: kefeng at ou.edu
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=52210 >
>
>
> Dear Sir:
>
> Here is the output from one of "*mpr.txt". In the text below, we
can see the forecast vaule is not varying. I'm not sure why. One
reason i can guess from the text is that most of observation points
level is missing "NA". This unchanged forecast value will lead large
RMSE in that level. The level i posted is between 150-10hPa.
>
> Here are also the common session of this section"
> V3.0 WRF 120000 20100515_120000 20100515_120000 000000
20100515_120000 20100515_120000 TMP P150-10 TMP P150-10
ADPUPA FULL DW_MEAN 4 NA NA NA
NA MPR 2367"
>
> I have checked *cnt.txt files. The "TOTAL" in that file is 2367
which means all the pairs below is counted in the verification. I'm
just wondering how the forecast value calculated when the observation
level is missing? And can default program get rid of these points in
the calculation of RMSE or other verification scores?
>
> Many thanks!
>
>
>
> INDEX OBS_SID OBS_LAT OBS_LON OBS_LVL OBS_ELV FCST
OBS CLIMO
>
> 439 72201 24.55000 -81.79000 50.00000 20731.58008 214.66865
209.85001 NA
> 440 72201 24.55000 -81.79000 47.00000 NA 214.66865
211.25000 NA
> 441 72201 24.55000 -81.79000 45.10000 NA 214.66865
210.25000 NA
> 442 72201 24.55000 -81.79000 39.70000 NA 214.66865
215.85001 NA
> 443 72201 24.55000 -81.79000 36.10000 NA 214.66865
217.05000 NA
> 444 72201 24.55000 -81.79000 30.00000 23934.91602 214.66865
214.05000 NA
> 445 72201 24.55000 -81.79000 29.00000 NA 214.66865
214.05000 NA
> 446 72201 24.55000 -81.79000 27.30000 NA 214.66865
217.85001 NA
> 448 72201 24.55000 -81.79000 25.60000 NA 214.66865
218.64999 NA
> 449 72201 24.55000 -81.79000 24.90000 NA 214.66865
220.64999 NA
> 450 72201 24.55000 -81.79000 24.00000 NA 214.66865
219.85001 NA
> 451 72201 24.55000 -81.79000 22.30000 NA 214.66865
221.05000 NA
> 452 72201 24.55000 -81.79000 20.90000 NA 214.66865
224.45000 NA
> 453 72201 24.55000 -81.79000 20.00000 26547.63477 214.66865
224.85001 NA
> 454 72201 24.55000 -81.79000 16.60000 27765.90430 214.66865
227.05000 NA
> 455 72201 24.55000 -81.79000 13.50000 NA 214.66865
232.45000 NA
> 456 72201 24.55000 -81.79000 12.80000 NA 214.66865
231.45000 NA
> 457 72201 24.55000 -81.79000 10.00000 31212.49219 214.66865
232.25000 NA
>
>
>
> kefeng
>
> 2011-12-21
>
>
>
> kefeng
------------------------------------------------
Subject: about the ADPUPA verification
From: kefeng
Time: Thu Jan 05 10:27:06 2012
Thanks, John.
I was on vaction these days and just saw
the confirmation email.
The problem have been solved. It was
due to the WPP process. In that procedure, my default configuration
turned off the level above 50hPa. Therefore, the level above 50mb
always have the same value. I suggest to add a warning in the log in
the furture version when the observation level exceed model topest
level after WPP.
Kefeng
2012-01-05
kefeng
发件人： John Halley Gotway via RT
发送时间： 2011-12-22
10:45:13
收件人： kefeng at ou.edu
抄送：
主题： Re: [rt.rap.ucar.edu #52210]
about the ADPUPA verification
Kefeng,
Hmmm, interesting. This a
good question.
The NA values in the OBS_ELV column do not impact the
computation of the statistics, and it doesn't make any sense to throw
them away. And here's why...
In the header columns, I see that
you're verifying temperature against upper-air observations between
150mb and 10mb. In the sample MPR output you sent, I see that this is
basically sounding data -
the station ID and lat/lon remain constant
while the contents of the "OBS_LVL" column changes. That "OBS_LVL"
column indicates the pressure level at which the observation occurred.
Sine you're
verifying a range of pressure levels (150 to 10mb), these
observations are used because they fall in that range. The "OBS_ELV"
column is the meters above sea level at which the observations
occurred.
Since you're verifying on pressure levels in this case,
that elevation information is not used at all by Point-Stat.
The real
question is, why does the forecast value remain constant for all of
these observations. If I were you, I would look at your model output
to see at what levels do you have temperature
records? My guess is
that you don't have any model output higher than 50 mb. So all of the
observations higher than 50 mb are getting compared to the forecast
field at 50mb. That would explain why
the forecast value doesn't
change.
Ultimately, it's your responsibility to know your data and
make sensible choices about what and how to do the verification. If
you don't have any model output higher than 50mb, it doesn't make
sense
to verify there.
Hope that helps. If you have more questions
about your particular dataset, it would probably be easiest to just
have you send us a sample of data along with your question.
We'd need
your Point-Stat forecast file, observation file, configuration file,
and a the command line you use to call Point-Stat. You can follow the
directions for sending us data listed here:
http://www.dtcenter.org/met/users/support/met_help.php#ftp
By the
way, I thought I'd mention that there is in fact a way to use the
STAT-Analysis tool to filter out data and recompute statistics. That
isn't really the point of the issue you're having, but I
thought I'd
mention it. The following type of job would read the data MPR lines,
filter out only those lines where OBS_ELV > 0, and then recompute CNT
statistics. That would basically throw out the
lines where OBS_ELV =
NA:
stat_analysis -lookin /path/to/stat/file/with/MPR/lines -job
aggregate_stat -line_type MPR -out_line_type CNT -column_min OBS_ELV 0
Hope that helps.
Thanks,
John
On 12/21/2011 10:39 AM, kefeng via RT
wrote:
>
> Wed Dec 21 10:39:23 2011: Request 52210 was acted upon.
> Transaction: Ticket created by kefeng at ou.edu
> Queue:
met_help
> Subject: about the ADPUPA verification
>
Owner: Nobody
> Requestors: kefeng at ou.edu
> Status: new
>
Ticket <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=52210 >
>
>
> Dear Sir:
>
> Here is the output from one of "*mpr.txt".
In the text below, we can see the forecast vaule is not varying. I'm
not sure why. One reason i can guess from the text is that most of
observation points level is missing "NA". This unchanged forecast
value will lead large RMSE in that level. The level i posted is
between 150-10hPa.
>
> Here are also the common session of this
section"
> V3.0 WRF 120000 20100515_120000 20100515_120000
000000 20100515_120000 20100515_120000 TMP P150-10 TMP
P150-10 ADPUPA FULL DW_MEAN 4 NA NA
NA NA MPR 2367"
>
> I have checked *cnt.txt
files. The "TOTAL" in that file is 2367 which means all the pairs
below is counted in the verification. I'm just wondering how the
forecast value calculated when the observation level is missing? And
can default program get rid of these points in the calculation of RMSE
or other verification scores?
>
> Many thanks!
>
>
>
>
INDEX OBS_SID OBS_LAT OBS_LON OBS_LVL OBS_ELV FCST
OBS CLIMO
>
> 439 72201 24.55000 -81.79000
50.00000 20731.58008 214.66865 209.85001 NA
> 440 72201
24.55000 -81.79000 47.00000 NA 214.66865 211.25000
NA
> 441 72201 24.55000 -81.79000 45.10000 NA
214.66865 210.25000 NA
> 442 72201 24.55000 -81.79000
39.70000 NA 214.66865 215.85001 NA
> 443 72201
24.55000 -81.79000 36.10000 NA 214.66865 217.05000
NA
> 444 72201 24.55000 -81.79000 30.00000 23934.91602
214.66865 214.05000 NA
> 445 72201 24.55000 -81.79000
29.00000 NA 214.66865 214.05000 NA
> 446 72201
24.55000 -81.79000 27.30000 NA 214.66865 217.85001
NA
> 448 72201 24.55000 -81.79000 25.60000 NA
214.66865 218.64999 NA
> 449 72201 24.55000 -81.79000
24.90000 NA 214.66865 220.64999 NA
> 450 72201
24.55000 -81.79000 24.00000 NA 214.66865 219.85001
NA
> 451 72201 24.55000 -81.79000 22.30000 NA
214.66865 221.05000 NA
> 452 72201 24.55000 -81.79000
20.90000 NA 214.66865 224.45000 NA
> 453 72201
24.55000 -81.79000 20.00000 26547.63477 214.66865 224.85001
NA
> 454 72201 24.55000 -81.79000 16.60000 27765.90430
214.66865 227.05000 NA
> 455 72201 24.55000 -81.79000
13.50000 NA 214.66865 232.45000 NA
> 456 72201
24.55000 -81.79000 12.80000 NA 214.66865 231.45000
NA
> 457 72201 24.55000 -81.79000 10.00000 31212.49219
214.66865 232.25000 NA
>
>
>
> kefeng
>
> 2011-12-21
>
>
>
> kefeng
------------------------------------------------
More information about the Met_help
mailing list