[Met_help] [rt.rap.ucar.edu #59751] History for Query Reg the differences between summay and aggregate jobs.

[John Halley Gotway via RT]

----------------------------------------------------------------
  Initial Request
----------------------------------------------------------------

Dear met_help, This is a query reg the stat_analysis tool. I wish to know the differences between the summay, aggregate and agggregate_stat Jobs. What different information they convey?  I am using point data for Rf verification from WRF.  geeta 		 	   		  

----------------------------------------------------------------
  Complete Ticket History
----------------------------------------------------------------

Subject: Query Reg the differences between summay and aggregate jobs.
From: Geeta Geeta
Time: Fri Jan 04 01:15:50 2013

Dear Met_helpThis is a query regarding the Stat-analysis tool. I wish
to know the difference between the summary, aggregate and
Aggregate_stat Jobs. I conmputed the ME/MAE etc from summary and
aggregate and aggegate_stat, they are different. In what way the
aggregate and aggregate_stat are the weighted means?????.  Thanks.
geeta
 > Subject: [rt.rap.ucar.edu #59751] AutoReply: Query Reg the
differences between summay and aggregate jobs.
> From: met_help at ucar.edu
> To: geeta124 at hotmail.com
> Date: Thu, 3 Jan 2013 01:41:01 -0700
>
> Greetings,
>
> This message has been automatically generated in response to the
> creation of a trouble ticket regarding:
> 	"Query Reg the differences between summay and aggregate jobs.",
> a summary of which appears below.
>
> There is no need to reply to this message right now.  Your ticket
has been
> assigned an ID of [rt.rap.ucar.edu #59751].
>
> Please include the string:
>
>          [rt.rap.ucar.edu #59751]
>
> in the subject line of all future correspondence about this issue.
To do so,
> you may reply to this message.
>
>                         Thank you,
>                         met_help at ucar.edu
>
>
-------------------------------------------------------------------------
> Dear met_help, This is a query reg the stat_analysis tool. I wish to
know the differences between the summay, aggregate and agggregate_stat
Jobs. What different information they convey?  I am using point data
for Rf verification from WRF.  geeta

------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #59751] AutoReply: Query Reg the differences between summay and aggregate jobs.
From: John Halley Gotway
Time: Fri Jan 04 11:36:21 2013

Geeta,

I apologize for the delay in getting back with you.  The differences
between summary, aggregate, and aggregate_stat jobs is pretty
straight-forward.  They is described in section 8.3.3 of the MET
User's Guide.

Here's a brief overview:

A "summary" job simply selects a column of output from the .stat files
and prints summary information about it, including min, max, mean,
standard deviation, and so on.  To run a summary job, you
specify a line type and a column name.  For example, "-line_type cnt
-column rmse" would select the root-mean-squared error column from the
continuous statistics lines.  Then, you'd see summary
information for that distribution of numbers.

An "aggregate" job combines the counts or sums from multiple lines
into a single line.  The output line type for an aggregate job is, by
definition, the same as the input line type.  So you only need
to specify the line type to be used using "-line_type".  For example,
suppose you have a bunch of contingency table counts stored in
multiple CTC lines in the MET output.  You can run an aggregate job
to sum up all those counts into a single output CTC line.  Take a look
in the METv4.0/data/config/README file for a list of line types that
can be aggregated in this way.

An "aggregate_stat" job does the same thing as the aggregate job, but
instead of writing out the same line type, it's writes out a different
line type.  So it first aggregates counts or sums, second
derives statistics, and third writes them out.  So you must specify an
input line type type and an output line type, such as, "-line_type CTC
-out_line_type CTS".  Continuing the previous example,
you've aggregated a bunch of CTC lines together, then derive
categorical statistics from those counts, and write out those
statistics in a CTS line.  Again, the list of valid conversions can be
found
in the METv4.0/data/config/README file.

You asked the question, why do I get different results from a
"summary" and an "aggregate_stat" job.  That behavior is expected
because you're doing two different processes.  Suppose you have two
SL1L2 and CNT lines.  Suppose the first is computed over 100 matched
pairs and the second is computed over 200 matched pairs.  Both CNT
lines contain an RMSE value.  The summary job will give you an
unweighted average of those two RMSE values.  The aggregate_stat job
will aggregate the SL1L2 partial sums together, weighted by the count
listed in the total column.  So the one with 200 matched
pairs will count twice as much as the one with 100 matched pairs.
It'll use those aggregated partial sums and derive a single RMSE value
from them.

To put it simple, the summary job performs unweighted averages while
the aggregate_stat job weighted by value in the TOTAL column.

Hope that helps clarify.

John



On 01/04/2013 01:15 AM, Geeta Geeta via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=59751 >
>
> Dear Met_helpThis is a query regarding the Stat-analysis tool. I
wish to know the difference between the summary, aggregate and
Aggregate_stat Jobs. I conmputed the ME/MAE etc from summary and
aggregate and aggegate_stat, they are different. In what way the
aggregate and aggregate_stat are the weighted means?????.  Thanks.
> geeta
>   > Subject: [rt.rap.ucar.edu #59751] AutoReply: Query Reg the
differences between summay and aggregate jobs.
>> From: met_help at ucar.edu
>> To: geeta124 at hotmail.com
>> Date: Thu, 3 Jan 2013 01:41:01 -0700
>>
>> Greetings,
>>
>> This message has been automatically generated in response to the
>> creation of a trouble ticket regarding:
>> 	"Query Reg the differences between summay and aggregate jobs.",
>> a summary of which appears below.
>>
>> There is no need to reply to this message right now.  Your ticket
has been
>> assigned an ID of [rt.rap.ucar.edu #59751].
>>
>> Please include the string:
>>
>>           [rt.rap.ucar.edu #59751]
>>
>> in the subject line of all future correspondence about this issue.
To do so,
>> you may reply to this message.
>>
>>                          Thank you,
>>                          met_help at ucar.edu
>>
>>
-------------------------------------------------------------------------
>> Dear met_help, This is a query reg the stat_analysis tool. I wish
to know the differences between the summay, aggregate and
agggregate_stat Jobs. What different information they convey?  I am
using point data for Rf verification from WRF.  geeta
>
>

------------------------------------------------