[Met_help] [rt.rap.ucar.edu #41926] History for about season data verifcation
RAL HelpDesk {for John Halley Gotway}
met_help at ucar.edu
Thu Nov 11 14:28:25 MST 2010
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Dear Sir or Madam,
Hi!
I am a MET user. I am writing to ask if MET can treat data from several different files as a whole sample space to make verifications, as I want to make a season test including data from several different cases. I haven't find any instruction for this in the manual. Is it my neglect, or the MET does not have such function at present.
Is it feasible if I put all the data from different cases into a single file to form one sample space as the input for MET, and corresponding obs into another input file, both with a common pesudo grids information?
Thanks a lot!
Jiangshan
2010-11-04
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #41926] about season data verifcation
From: John Halley Gotway
Time: Thu Nov 04 09:31:28 2010
Jiangshan,
I believe you will be able to use MET to do an analysis of multiple
cases,
however it cannot be done in a single step. Simply put, the MET
statistics tools are designed to be run once for each valid time. And
then you can use the analysis tools (Stat-Analysis and MODE-Analysis)
to
aggregate your results through time and/or space.
For example, suppose you want to verify your forecast of 24-hour
accumulated precipitation over the course of 1 month. And suppose
you're
comparing your gridded NWP forecast to a gridded analysis of
accumulated
precipitation. Here's what you'd do:
(1) Configure Grid-Stat to define exactly what thresholds/verification
methods/verification areas you'd like to use.
(2) Write a script to run Grid-Stat once for each of the days in the
month.
(3) Use the STAT-Analysis tool to aggregate together the results for
each
day's output, and recompute statistics over your entire month.
You can also use the STAT-Analysis tools to slice and dice the data in
a
variety of other ways.
Hope that helps clarify.
John
>
> Thu Nov 04 01:53:29 2010: Request 41926 was acted upon.
> Transaction: Ticket created by zhujiangshan at mail.iap.ac.cn
> Queue: met_help
> Subject: about season data verifcation
> Owner: Nobody
> Requestors: zhujiangshan at mail.iap.ac.cn
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=41926 >
>
>
> Dear Sir or Madam,
> Hi!
> I am a MET user. I am writing to ask if MET can treat data from
several
> different files as a whole sample space to make verifications, as I
want
> to make a season test including data from several different cases. I
> haven't find any instruction for this in the manual. Is it my
neglect, or
> the MET does not have such function at present.
> Is it feasible if I put all the data from different cases into a
single
> file to form one sample space as the input for MET, and
corresponding obs
> into another input file, both with a common pesudo grids
information?
>
> Thanks a lot!
>
> Jiangshan
>
> 2010-11-04
>
------------------------------------------------
Subject: about season data verifcation
From: zhujiangshan
Time: Fri Nov 05 01:34:45 2010
John,
Hi! Thanks for help.
There is one thing I'm not clear.
If I want to verify a score for 30 cases, and each case has 100
samples. I want a sample space of 30*100. After step(2) Write a script
to run Grid-Stat once for each of the days in the month, is all the
information of the 30*100 samples are recorded, or just 30 score
values. And does the STAT-Analysis sample from these 30*100 samples,
or some ways to simulate that procedure, or just sample from those 30
scores, especially when use bootstripe to get confidence intervals?
Thanks!
Jiangshan
2010-11-05
发件人： RAL HelpDesk {for
John Halley Gotway}
发送时间： 2010-11-04 23:31:40
收件人： zhujiangshan
抄送：
主题： Re: [rt.rap.ucar.edu #41926] about season data verifcation
Jiangshan,
I believe you will be able to use MET to do an analysis of
multiple cases,
however it cannot be done in a single step. Simply
put, the MET
statistics tools are designed to be run once for each
valid time. And
then you can use the analysis tools (Stat-Analysis
and MODE-Analysis) to
aggregate your results through time and/or
space.
For example, suppose you want to verify your forecast of 24-
hour
accumulated precipitation over the course of 1 month. And
suppose you're
comparing your gridded NWP forecast to a gridded
analysis of accumulated
precipitation. Here's what you'd do:
(1)
Configure Grid-Stat to define exactly what thresholds/verification
methods/verification areas you'd like to use.
(2) Write a script to
run Grid-Stat once for each of the days in the month.
(3) Use the
STAT-Analysis tool to aggregate together the results for each
day's
output, and recompute statistics over your entire month.
You can also
use the STAT-Analysis tools to slice and dice the data in a
variety
of other ways.
Hope that helps clarify.
John
>
> Thu Nov 04
01:53:29 2010: Request 41926 was acted upon.
> Transaction: Ticket
created by zhujiangshan at mail.iap.ac.cn
> Queue: met_help
>
Subject: about season data verifcation
> Owner: Nobody
>
Requestors: zhujiangshan at mail.iap.ac.cn
> Status: new
>
Ticket <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=41926 >
>
>
> Dear Sir or Madam,
> Hi!
> I am a MET user. I am writing to
ask if MET can treat data from several
> different files as a whole
sample space to make verifications, as I want
> to make a season test
including data from several different cases. I
> haven't find any
instruction for this in the manual. Is it my neglect, or
> the MET
does not have such function at present.
> Is it feasible if I put all
the data from different cases into a single
> file to form one sample
space as the input for MET, and corresponding obs
> into another
input file, both with a common pesudo grids information?
>
> Thanks
a lot!
>
> Jiangshan
>
> 2010-11-04
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #41926] about season data verifcation
From: John Halley Gotway
Time: Fri Nov 05 11:55:39 2010
Jiangshan,
Each time your run Grid-Stat, you can output several line types
including:
(1) Contingency Table Counts (CTC)
(2) Contingency Table Statistics (CTS)
(3) Scalar Partial Sums (SL1L2)
(3) Continuous Statistics (CNT)
After you've run Grid-Stat the 30 times, you can run a variety of jobs
using STAT-Analysis:
(1) You could run a "summary" job over 30 statistics values
contained in the CNT or CTS line types. For example, you could look
at the 30 RMSE (root-mean squared error) or 30 GSS (Gilbert Skill
Score) values and look at the distribution of those scores
(min/max/mean/median/percentiles).
(2) Also, you could run an "aggregate_stat" job to sum of the 30
contingency table counts or scalar partial sums and recompute the
statistics over the entire 30 cases. For example, add up 30
contingency tables and recompute the GSS for the 30 cases combined.
Or sum up the partial sums and recompute the RMSE for the 30 cases
combined. When there are parametric confidence intervals
available, STAT-Analysis will compute them.
*** BUT *** STAT-Analysis is NOT currently performing bootstrapping
when aggregating together data through time like this. We realize
this is a deficiency and intend to add support for it in future
releases. If it becomes necessary for you to do this, I could send
you a sample R-script that can read MET output and perform this type
of bootstrapping. But in the long run it'll end up in the
STAT-Analysis tool itself.
Hope that helps clarify.
Thanks,
John
On 11/05/2010 01:34 AM, RAL HelpDesk {for zhujiangshan} wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=41926 >
>
> John,
> Hi! Thanks for help.
>
> There is one thing I'm not clear.
> If I want to verify a score for 30 cases, and each case has 100
samples. I want a sample space of 30*100. After step(2) Write a script
to run Grid-Stat once for each of the days in the month, is all the
information of the 30*100 samples are recorded, or just 30 score
values. And does the STAT-Analysis sample from these 30*100 samples,
or some ways to simulate that procedure, or just sample from those 30
scores, especially when use bootstripe to get confidence intervals?
>
> Thanks!
>
> Jiangshan
>
> 2010-11-05
>
>
>
>
> 发件人： RAL HelpDesk {for John Halley Gotway}
> 发送时间： 2010-11-04 23:31:40
> 收件人： zhujiangshan
> 抄送：
> 主题： Re: [rt.rap.ucar.edu #41926] about season data verifcation
>
> Jiangshan,
> I believe you will be able to use MET to do an analysis of multiple
cases,
> however it cannot be done in a single step. Simply put, the MET
> statistics tools are designed to be run once for each valid time.
And
> then you can use the analysis tools (Stat-Analysis and MODE-
Analysis) to
> aggregate your results through time and/or space.
> For example, suppose you want to verify your forecast of 24-hour
> accumulated precipitation over the course of 1 month. And suppose
you're
> comparing your gridded NWP forecast to a gridded analysis of
accumulated
> precipitation. Here's what you'd do:
> (1) Configure Grid-Stat to define exactly what
thresholds/verification
> methods/verification areas you'd like to use.
> (2) Write a script to run Grid-Stat once for each of the days in the
month.
> (3) Use the STAT-Analysis tool to aggregate together the results for
each
> day's output, and recompute statistics over your entire month.
> You can also use the STAT-Analysis tools to slice and dice the data
in a
> variety of other ways.
> Hope that helps clarify.
> John
>>
>> Thu Nov 04 01:53:29 2010: Request 41926 was acted upon.
>> Transaction: Ticket created by zhujiangshan at mail.iap.ac.cn
>> Queue: met_help
>> Subject: about season data verifcation
>> Owner: Nobody
>> Requestors: zhujiangshan at mail.iap.ac.cn
>> Status: new
>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=41926 >
>>
>>
>> Dear Sir or Madam,
>> Hi!
>> I am a MET user. I am writing to ask if MET can treat data from
several
>> different files as a whole sample space to make verifications, as I
want
>> to make a season test including data from several different cases.
I
>> haven't find any instruction for this in the manual. Is it my
neglect, or
>> the MET does not have such function at present.
>> Is it feasible if I put all the data from different cases into a
single
>> file to form one sample space as the input for MET, and
corresponding obs
>> into another input file, both with a common pesudo grids
information?
>>
>> Thanks a lot!
>>
>> Jiangshan
>>
>> 2010-11-04
>>
------------------------------------------------
More information about the Met_help
mailing list