[Met_help] [rt.rap.ucar.edu #52210] History for about the ADPUPA verification
John Halley Gotway via RT
met_help at ucar.edu
Thu Jan 5 09:36:53 MST 2012
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Dear Sir:
Here is the output from one of "*mpr.txt". In the text below, we can see the forecast vaule is not varying. I'm not sure why. One reason i can guess from the text is that most of observation points level is missing "NA". This unchanged forecast value will lead large RMSE in that level. The level i posted is between 150-10hPa.
Here are also the common session of this section"
V3.0 WRF 120000 20100515_120000 20100515_120000 000000 20100515_120000 20100515_120000 TMP P150-10 TMP P150-10 ADPUPA FULL DW_MEAN 4 NA NA NA NA MPR 2367"
I have checked *cnt.txt files. The "TOTAL" in that file is 2367 which means all the pairs below is counted in the verification. I'm just wondering how the forecast value calculated when the observation level is missing? And can default program get rid of these points in the calculation of RMSE or other verification scores?
Many thanks!
INDEX OBS_SID OBS_LAT OBS_LON OBS_LVL OBS_ELV FCST OBS CLIMO
439 72201 24.55000 -81.79000 50.00000 20731.58008 214.66865 209.85001 NA
440 72201 24.55000 -81.79000 47.00000 NA 214.66865 211.25000 NA
441 72201 24.55000 -81.79000 45.10000 NA 214.66865 210.25000 NA
442 72201 24.55000 -81.79000 39.70000 NA 214.66865 215.85001 NA
443 72201 24.55000 -81.79000 36.10000 NA 214.66865 217.05000 NA
444 72201 24.55000 -81.79000 30.00000 23934.91602 214.66865 214.05000 NA
445 72201 24.55000 -81.79000 29.00000 NA 214.66865 214.05000 NA
446 72201 24.55000 -81.79000 27.30000 NA 214.66865 217.85001 NA
448 72201 24.55000 -81.79000 25.60000 NA 214.66865 218.64999 NA
449 72201 24.55000 -81.79000 24.90000 NA 214.66865 220.64999 NA
450 72201 24.55000 -81.79000 24.00000 NA 214.66865 219.85001 NA
451 72201 24.55000 -81.79000 22.30000 NA 214.66865 221.05000 NA
452 72201 24.55000 -81.79000 20.90000 NA 214.66865 224.45000 NA
453 72201 24.55000 -81.79000 20.00000 26547.63477 214.66865 224.85001 NA
454 72201 24.55000 -81.79000 16.60000 27765.90430 214.66865 227.05000 NA
455 72201 24.55000 -81.79000 13.50000 NA 214.66865 232.45000 NA
456 72201 24.55000 -81.79000 12.80000 NA 214.66865 231.45000 NA
457 72201 24.55000 -81.79000 10.00000 31212.49219 214.66865 232.25000 NA
kefeng
2011-12-21
kefeng
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #52210] about the ADPUPA verification
From: John Halley Gotway
Time: Thu Dec 22 09:45:12 2011
Kefeng,
Hmmm, interesting. This a good question.
The NA values in the OBS_ELV column do not impact the computation of
the statistics, and it doesn't make any sense to throw them away. And
here's why...
In the header columns, I see that you're verifying temperature against
upper-air observations between 150mb and 10mb. In the sample MPR
output you sent, I see that this is basically sounding data -
the station ID and lat/lon remain constant while the contents of the
"OBS_LVL" column changes. That "OBS_LVL" column indicates the
pressure level at which the observation occurred. Sine you're
verifying a range of pressure levels (150 to 10mb), these observations
are used because they fall in that range. The "OBS_ELV" column is the
meters above sea level at which the observations occurred.
Since you're verifying on pressure levels in this case, that
elevation information is not used at all by Point-Stat.
The real question is, why does the forecast value remain constant for
all of these observations. If I were you, I would look at your model
output to see at what levels do you have temperature
records? My guess is that you don't have any model output higher than
50 mb. So all of the observations higher than 50 mb are getting
compared to the forecast field at 50mb. That would explain why
the forecast value doesn't change.
Ultimately, it's your responsibility to know your data and make
sensible choices about what and how to do the verification. If you
don't have any model output higher than 50mb, it doesn't make sense
to verify there.
Hope that helps. If you have more questions about your particular
dataset, it would probably be easiest to just have you send us a
sample of data along with your question.
We'd need your Point-Stat forecast file, observation file,
configuration file, and a the command line you use to call Point-Stat.
You can follow the directions for sending us data listed here:
http://www.dtcenter.org/met/users/support/met_help.php#ftp
By the way, I thought I'd mention that there is in fact a way to use
the STAT-Analysis tool to filter out data and recompute statistics.
That isn't really the point of the issue you're having, but I
thought I'd mention it. The following type of job would read the data
MPR lines, filter out only those lines where OBS_ELV > 0, and then
recompute CNT statistics. That would basically throw out the
lines where OBS_ELV = NA:
stat_analysis -lookin /path/to/stat/file/with/MPR/lines -job
aggregate_stat -line_type MPR -out_line_type CNT -column_min OBS_ELV 0
Hope that helps.
Thanks,
John
On 12/21/2011 10:39 AM, kefeng via RT wrote:
>
> Wed Dec 21 10:39:23 2011: Request 52210 was acted upon.
> Transaction: Ticket created by kefeng at ou.edu
> Queue: met_help
> Subject: about the ADPUPA verification
> Owner: Nobody
> Requestors: kefeng at ou.edu
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=52210 >
>
>
> Dear Sir:
>
> Here is the output from one of "*mpr.txt". In the text below, we
can see the forecast vaule is not varying. I'm not sure why. One
reason i can guess from the text is that most of observation points
level is missing "NA". This unchanged forecast value will lead large
RMSE in that level. The level i posted is between 150-10hPa.
>
> Here are also the common session of this section"
> V3.0 WRF 120000 20100515_120000 20100515_120000 000000
20100515_120000 20100515_120000 TMP P150-10 TMP P150-10
ADPUPA FULL DW_MEAN 4 NA NA NA
NA MPR 2367"
>
> I have checked *cnt.txt files. The "TOTAL" in that file is 2367
which means all the pairs below is counted in the verification. I'm
just wondering how the forecast value calculated when the observation
level is missing? And can default program get rid of these points in
the calculation of RMSE or other verification scores?
>
> Many thanks!
>
>
>
> INDEX OBS_SID OBS_LAT OBS_LON OBS_LVL OBS_ELV FCST
OBS CLIMO
>
> 439 72201 24.55000 -81.79000 50.00000 20731.58008 214.66865
209.85001 NA
> 440 72201 24.55000 -81.79000 47.00000 NA 214.66865
211.25000 NA
> 441 72201 24.55000 -81.79000 45.10000 NA 214.66865
210.25000 NA
> 442 72201 24.55000 -81.79000 39.70000 NA 214.66865
215.85001 NA
> 443 72201 24.55000 -81.79000 36.10000 NA 214.66865
217.05000 NA
> 444 72201 24.55000 -81.79000 30.00000 23934.91602 214.66865
214.05000 NA
> 445 72201 24.55000 -81.79000 29.00000 NA 214.66865
214.05000 NA
> 446 72201 24.55000 -81.79000 27.30000 NA 214.66865
217.85001 NA
> 448 72201 24.55000 -81.79000 25.60000 NA 214.66865
218.64999 NA
> 449 72201 24.55000 -81.79000 24.90000 NA 214.66865
220.64999 NA
> 450 72201 24.55000 -81.79000 24.00000 NA 214.66865
219.85001 NA
> 451 72201 24.55000 -81.79000 22.30000 NA 214.66865
221.05000 NA
> 452 72201 24.55000 -81.79000 20.90000 NA 214.66865
224.45000 NA
> 453 72201 24.55000 -81.79000 20.00000 26547.63477 214.66865
224.85001 NA
> 454 72201 24.55000 -81.79000 16.60000 27765.90430 214.66865
227.05000 NA
> 455 72201 24.55000 -81.79000 13.50000 NA 214.66865
232.45000 NA
> 456 72201 24.55000 -81.79000 12.80000 NA 214.66865
231.45000 NA
> 457 72201 24.55000 -81.79000 10.00000 31212.49219 214.66865
232.25000 NA
>
>
>
> kefeng
>
> 2011-12-21
>
>
>
> kefeng
------------------------------------------------
More information about the Met_help
mailing list