[Met_help] [rt.rap.ucar.edu #45775] History for Stat-Analysis Tool Clarification

Tue Apr 12 09:24:32 MDT 2011

----------------------------------------------------------------
  Initial Request
----------------------------------------------------------------

Hello,

When I aggregate the wavelet data (i.e., ISC, etc.) using stat_analysis, how is the data aggregated?  Is it simply just averaged over all the files based on the different parameters (i.e., forecast lead times, etc.)?  Or, is it computed a different way?  The reason I ask is because I want to aggregate the NBRCNT files for FBS and FSS, but (I think) I am only able to do this if using NBRCTC (and NBRCTS using line_type_out), which I'm not interested in.  Is this correct?  If I were to manually aggregate the FSS and FBS, would I just average them if that's how the aggregated ISC are computed?  Hopefully, you can see the point I'm trying to make.  If not, I'll try to explain myself in more depth if needed.

Thanks,
Derek Stratman

----------------------------------------------------------------
  Complete Ticket History
----------------------------------------------------------------

Subject: Re: [rt.rap.ucar.edu #45775] Stat-Analysis Tool Clarification
From: John Halley Gotway
Time: Thu Apr 07 12:19:40 2011

Derek,

I think I understand the question you're asking.  There is only one
output line type for the Wavelet-Stat tool, namely the ISC (for
Intensity-Scale) line type.  And you can aggregate those lines in
STAT-Analysis using the option "-job aggregate -line_type ISC".

The data values that occur in that line are: TOTAL, TILE_DIM,
TILE_XLL, TILE_YLL, NSCALE, ISCALE, MSE, ISC, FENERGY2, OENERGY2,
BASER, and FBIAS

- The TOTAL value for the output of this stat job is just the sum of
the TOTAL values in each input line.
- For TILE_DIM, TILE_XLL, and TILE_YLL, STAT-Analysis just checks to
make sure they remain constant.  If they don't, STAT-Analysis writes
out NA for that column.
- For NSCALE, STAT-Analysis checks to make sure it remains constant.
If not, STAT-Analysis errors out.
- For ISCALE, STAT-Analysis writes out one line for each value of
ISCALE.  Meaning, it only aggregates together lines with the same
value for ISCALE.
- For MSE, FENERGY2, and OENERGY2, STAT-Analysis computes a weighted
average of these values where the value of TOTAL determines the
weight.
- For FBIAS and BASER, these are aggregated and recomputed only for
the first scale, the binary image.
- For ISC, these are truly aggregated in the correct way.  I remember
working with Barbara Casati figuring out how this aggregation should
be handled.  For the actual equations used, please take a
look in the file "METv3.0/src/stat_analysis/aggr_stat_line.cc".  The
aggregated ISC is computed from the aggregated fbias, baser, and mse
values.  Look in the routine "aggr_isc_lines", approximately
lines 1479-1495.

Hope this helps.

John

On 04/04/2011 01:35 PM, RAL HelpDesk {for Derek.Stratman} wrote:
>
> Mon Apr 04 13:35:16 2011: Request 45775 was acted upon.
> Transaction: Ticket created by Derek.Stratman at noaa.gov
>        Queue: met_help
>      Subject: Stat-Analysis Tool Clarification
>        Owner: Nobody
>   Requestors: Derek.Stratman at noaa.gov
>       Status: new
>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=45775 >
>
>
> Hello,
>
> When I aggregate the wavelet data (i.e., ISC, etc.) using
stat_analysis, how is the data aggregated?  Is it simply just averaged
over all the files based on the different parameters (i.e., forecast
lead times, etc.)?  Or, is it computed a different way?  The reason I
ask is because I want to aggregate the NBRCNT files for FBS and FSS,
but (I think) I am only able to do this if using NBRCTC (and NBRCTS
using line_type_out), which I'm not interested in.  Is this correct?
If I were to manually aggregate the FSS and FBS, would I just average
them if that's how the aggregated ISC are computed?  Hopefully, you
can see the point I'm trying to make.  If not, I'll try to explain
myself in more depth if needed.
>
> Thanks,
> Derek Stratman

------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #45775] Stat-Analysis Tool Clarification
From: Derek.Stratman
Time: Tue Apr 12 09:11:47 2011

Thanks, John.  That was pretty much what I was looking for.   Also,
I'm guessing there isn't a way for stat_analysis to aggregate the
NBRCNT scores (i.e., FSS and FBS) across several days(??).  I didn't
see a way to do this, so I just aggregated the squared errors (Ebert
2009) in FBS and FSS by backing out the errors using the number of
neighborhoods, FSS, and FBS.  If this isn't already in stat_analysis,
I would suggest adding it in for future users if possible.

Thanks for your help and useful info!

Derek

----- Original Message -----
From: RAL HelpDesk {for John Halley Gotway} <met_help at ucar.edu>
Date: Thursday, April 7, 2011 1:19 pm
Subject: Re: [rt.rap.ucar.edu #45775] Stat-Analysis Tool Clarification

> Derek,
>
> I think I understand the question you're asking.  There is only one
> output line type for the Wavelet-Stat tool, namely the ISC (for
> Intensity-Scale) line type.  And you can aggregate those lines in
> STAT-Analysis using the option "-job aggregate -line_type ISC".
>
> The data values that occur in that line are: TOTAL, TILE_DIM,
> TILE_XLL, TILE_YLL, NSCALE, ISCALE, MSE, ISC, FENERGY2, OENERGY2,
> BASER, and FBIAS
>
> - The TOTAL value for the output of this stat job is just the sum
> of the TOTAL values in each input line.
> - For TILE_DIM, TILE_XLL, and TILE_YLL, STAT-Analysis just checks
> to make sure they remain constant.  If they don't, STAT-Analysis
> writes out NA for that column.
> - For NSCALE, STAT-Analysis checks to make sure it remains
> constant.  If not, STAT-Analysis errors out.
> - For ISCALE, STAT-Analysis writes out one line for each value of
> ISCALE.  Meaning, it only aggregates together lines with the same
> value for ISCALE.
> - For MSE, FENERGY2, and OENERGY2, STAT-Analysis computes a
> weighted average of these values where the value of TOTAL
> determines the weight.
> - For FBIAS and BASER, these are aggregated and recomputed only for
> the first scale, the binary image.
> - For ISC, these are truly aggregated in the correct way.  I
> remember working with Barbara Casati figuring out how this
> aggregation should be handled.  For the actual equations used,
> please take a
> look in the file "METv3.0/src/stat_analysis/aggr_stat_line.cc".
> The aggregated ISC is computed from the aggregated fbias, baser,
> and mse values.  Look in the routine "aggr_isc_lines", approximately
> lines 1479-1495.
>
> Hope this helps.
>
> John
>
> On 04/04/2011 01:35 PM, RAL HelpDesk {for Derek.Stratman} wrote:
> >
> > Mon Apr 04 13:35:16 2011: Request 45775 was acted upon.
> > Transaction: Ticket created by Derek.Stratman at noaa.gov
> >        Queue: met_help
> >      Subject: Stat-Analysis Tool Clarification
> >        Owner: Nobody
> >   Requestors: Derek.Stratman at noaa.gov
> >       Status: new
> >  Ticket <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=45775 >
> >
> >
> > Hello,
> >
> > When I aggregate the wavelet data (i.e., ISC, etc.) using
> stat_analysis, how is the data aggregated?  Is it simply just
> averaged over all the files based on the different parameters
> (i.e., forecast lead times, etc.)?  Or, is it computed a different
> way?  The reason I ask is because I want to aggregate the NBRCNT
> files for FBS and FSS, but (I think) I am only able to do this if
> using NBRCTC (and NBRCTS using line_type_out), which I'm not
> interested in.  Is this correct?  If I were to manually aggregate
> the FSS and FBS, would I just average them if that's how the
> aggregated ISC are computed?  Hopefully, you can see the point I'm
> trying to make.  If not, I'll try to explain myself in more depth
> if needed.
> >
> > Thanks,
> > Derek Stratman
>
>

------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #45775] Stat-Analysis Tool Clarification
From: John Halley Gotway
Time: Tue Apr 12 09:24:12 2011

Derek,

You're right, there currently is no way for STAT-Analysis to aggregate
FSS and FBS across multiple cases.  But you're suggestion is a very
good one.  I think we'd need to dump out an additional line
type from Grid-Stat - probably NBRSL1L2 that would contain the partial
sums for the neighborhood method.  Then we could easily aggregate
those together in STAT-Analysis and recompute FBS and FSS.
I've added that suggestion to our development list for consideration.

I'll go ahead an resolve this current ticket now.

Thanks,
John

On 04/12/2011 09:11 AM, RAL HelpDesk {for Derek.Stratman} wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=45775 >
>
> Thanks, John.  That was pretty much what I was looking for.   Also,
I'm guessing there isn't a way for stat_analysis to aggregate the
NBRCNT scores (i.e., FSS and FBS) across several days(??).  I didn't
see a way to do this, so I just aggregated the squared errors (Ebert
2009) in FBS and FSS by backing out the errors using the number of
neighborhoods, FSS, and FBS.  If this isn't already in stat_analysis,
I would suggest adding it in for future users if possible.
>
> Thanks for your help and useful info!
>
> Derek
>
>
> ----- Original Message -----
> From: RAL HelpDesk {for John Halley Gotway} <met_help at ucar.edu>
> Date: Thursday, April 7, 2011 1:19 pm
> Subject: Re: [rt.rap.ucar.edu #45775] Stat-Analysis Tool
Clarification
>
>> Derek,
>>
>> I think I understand the question you're asking.  There is only one
>> output line type for the Wavelet-Stat tool, namely the ISC (for
>> Intensity-Scale) line type.  And you can aggregate those lines in
>> STAT-Analysis using the option "-job aggregate -line_type ISC".
>>
>> The data values that occur in that line are: TOTAL, TILE_DIM,
>> TILE_XLL, TILE_YLL, NSCALE, ISCALE, MSE, ISC, FENERGY2, OENERGY2,
>> BASER, and FBIAS
>>
>> - The TOTAL value for the output of this stat job is just the sum
>> of the TOTAL values in each input line.
>> - For TILE_DIM, TILE_XLL, and TILE_YLL, STAT-Analysis just checks
>> to make sure they remain constant.  If they don't, STAT-Analysis
>> writes out NA for that column.
>> - For NSCALE, STAT-Analysis checks to make sure it remains
>> constant.  If not, STAT-Analysis errors out.
>> - For ISCALE, STAT-Analysis writes out one line for each value of
>> ISCALE.  Meaning, it only aggregates together lines with the same
>> value for ISCALE.
>> - For MSE, FENERGY2, and OENERGY2, STAT-Analysis computes a
>> weighted average of these values where the value of TOTAL
>> determines the weight.
>> - For FBIAS and BASER, these are aggregated and recomputed only for
>> the first scale, the binary image.
>> - For ISC, these are truly aggregated in the correct way.  I
>> remember working with Barbara Casati figuring out how this
>> aggregation should be handled.  For the actual equations used,
>> please take a
>> look in the file "METv3.0/src/stat_analysis/aggr_stat_line.cc".
>> The aggregated ISC is computed from the aggregated fbias, baser,
>> and mse values.  Look in the routine "aggr_isc_lines",
approximately
>> lines 1479-1495.
>>
>> Hope this helps.
>>
>> John
>>
>> On 04/04/2011 01:35 PM, RAL HelpDesk {for Derek.Stratman} wrote:
>>>
>>> Mon Apr 04 13:35:16 2011: Request 45775 was acted upon.
>>> Transaction: Ticket created by Derek.Stratman at noaa.gov
>>>        Queue: met_help
>>>      Subject: Stat-Analysis Tool Clarification
>>>        Owner: Nobody
>>>   Requestors: Derek.Stratman at noaa.gov
>>>       Status: new
>>>  Ticket <URL:
>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=45775 >
>>>
>>>
>>> Hello,
>>>
>>> When I aggregate the wavelet data (i.e., ISC, etc.) using
>> stat_analysis, how is the data aggregated?  Is it simply just
>> averaged over all the files based on the different parameters
>> (i.e., forecast lead times, etc.)?  Or, is it computed a different
>> way?  The reason I ask is because I want to aggregate the NBRCNT
>> files for FBS and FSS, but (I think) I am only able to do this if
>> using NBRCTC (and NBRCTS using line_type_out), which I'm not
>> interested in.  Is this correct?  If I were to manually aggregate
>> the FSS and FBS, would I just average them if that's how the
>> aggregated ISC are computed?  Hopefully, you can see the point I'm
>> trying to make.  If not, I'll try to explain myself in more depth
>> if needed.
>>>
>>> Thanks,
>>> Derek Stratman
>>
>>

------------------------------------------------