[Met_help] [rt.rap.ucar.edu #40186] History for Using Base Rate

[RAL HelpDesk {for John Halley Gotway}]

----------------------------------------------------------------
  Initial Request
----------------------------------------------------------------

Hi,

I'm using the Stat-Analysis tool and was wondering if I could use base rate as a filter.  For 
example, I want keep the forecast hours that have a base rate higher than 0.00152 and 
ignore those less than that value.  

Here is my example code (which works fine) for reference:

stat_analysis \
-lookin /home/input \
-out /home/output/model_job.txt \
-v 2 \
-job aggregate_stat \
-line_type CTC -out_line_type CTS \
-fcst_lead 060000 -fcst_thres >=20.000 \
-dump_row /home/output/model_job.stat

Thanks for your future help.

Derek Stratman


----------------------------------------------------------------
  Complete Ticket History
----------------------------------------------------------------

Subject: Re: [rt.rap.ucar.edu #40186] Using Base Rate
From: John Halley Gotway
Time: Thu Aug 19 14:00:42 2010

Derek,

Very good question.  Unfortunately the simple answer to your question,
is no, there's no easy way to do this for the specific job you sent
me.

However, you do have a few options...

The first thing to understand is that STAT-Analysis takes a very
simple approach to filtering data.  You can tell it column names and
values to use, but those columns must already exist in the output
you're filtering.  In the job you sent, you're adding up contingency
table counts and writing out contingency table statistics.  So STAT-
Analysis is reading CTC lines.  You'd be able to filter on any
of the CTC columns of data "TOTAL, FY_OY, FY_ON, FN_OY, FN_ON", but
none of those would give you the base rate.

However, another output line type from MET is the FHO line type -
forecast, hit, and observation rate line.  This data is actually
redundant with the CTC data - just presented slightly differently.
Fortunately though, one of these columns DOES give you the base rate.
The observation rate (O_RATE) column is the same as the base rate.

So if you have FHO lines in your output, the following job should do
the trick:

stat_analysis \
-lookin /home/input \
-out /home/output/model_job.txt \
-v 2 \
-job aggregate_stat \
-line_type FHO -out_line_type CTS \
-column_min O_RATE 0.00152 \
-fcst_lead 060000 -fcst_thresh >=20.000 \
-dump_row /home/output/model_job.stat

That'll throw out any lines with an O_RATE value less than the
threshold.

Another possible job you could consider is something like this:

stat_analysis \
-lookin /home/input \
-out /home/output/model_job.txt \
-v 2 \
-job summary -line_type CTS -column CSI -column GSS \
-column_min BASER 0.00152 \
-fcst_lead 060000 -fcst_thresh >=20.000 \
-dump_row /home/output/model_job.stat

That'll look through all the input CTS lines, throw out any lines with
a base rate (BASER in the CTS line) value less than the threshold, and
give you summary information for the CSI and GSS columns
of data.

Hope that does the trick for you.

Thanks,
John Halley Gotway
met_help at ucar.edu

RAL HelpDesk {for Derek.Stratman} wrote:
> Thu Aug 19 10:22:12 2010: Request 40186 was acted upon.
> Transaction: Ticket created by Derek.Stratman at noaa.gov
>        Queue: met_help
>      Subject: Using Base Rate
>        Owner: Nobody
>   Requestors: Derek.Stratman at noaa.gov
>       Status: new
>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=40186 >
>
>
> Hi,
>
> I'm using the Stat-Analysis tool and was wondering if I could use
base rate as a filter.  For
> example, I want keep the forecast hours that have a base rate higher
than 0.00152 and
> ignore those less than that value.
>
> Here is my example code (which works fine) for reference:
>
> stat_analysis \
> -lookin /home/input \
> -out /home/output/model_job.txt \
> -v 2 \
> -job aggregate_stat \
> -line_type CTC -out_line_type CTS \
> -fcst_lead 060000 -fcst_thres >=20.000 \
> -dump_row /home/output/model_job.stat
>
> Thanks for your future help.
>
> Derek Stratman

------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #40186] Using Base Rate
From: Derek.Stratman
Time: Fri Aug 20 09:42:26 2010

Thanks, John!

I ended up using the first method you suggested, and now it does what
I've been wanting it
to do.  I have no further questions at the moment.

Thanks again,
Derek Stratman

----- Original Message -----
From: RAL HelpDesk {for John Halley Gotway} <met_help at ucar.edu>
Date: Thursday, August 19, 2010 3:00 pm
Subject: Re: [rt.rap.ucar.edu #40186] Using Base Rate
> Derek,
>
> Very good question.  Unfortunately the simple answer to your
> question, is no, there's no easy way to do this for the specific
> job you sent me.
>
> However, you do have a few options...
>
> The first thing to understand is that STAT-Analysis takes a very
> simple approach to filtering data.  You can tell it column names
> and values to use, but those columns must already exist in the
output
> you're filtering.  In the job you sent, you're adding up
> contingency table counts and writing out contingency table
> statistics.  So STAT-Analysis is reading CTC lines.  You'd be able
> to filter on any
> of the CTC columns of data "TOTAL, FY_OY, FY_ON, FN_OY, FN_ON", but
> none of those would give you the base rate.
>
> However, another output line type from MET is the FHO line type -
> forecast, hit, and observation rate line.  This data is actually
> redundant with the CTC data - just presented slightly differently.
> Fortunately though, one of these columns DOES give you the base
> rate.  The observation rate (O_RATE) column is the same as the base
> rate.
> So if you have FHO lines in your output, the following job should
> do the trick:
>
> stat_analysis \
> -lookin /home/input \
> -out /home/output/model_job.txt \
> -v 2 \
> -job aggregate_stat \
> -line_type FHO -out_line_type CTS \
> -column_min O_RATE 0.00152 \
> -fcst_lead 060000 -fcst_thresh >=20.000 \
> -dump_row /home/output/model_job.stat
>
> That'll throw out any lines with an O_RATE value less than the
> threshold.
> Another possible job you could consider is something like this:
>
> stat_analysis \
> -lookin /home/input \
> -out /home/output/model_job.txt \
> -v 2 \
> -job summary -line_type CTS -column CSI -column GSS \
> -column_min BASER 0.00152 \
> -fcst_lead 060000 -fcst_thresh >=20.000 \
> -dump_row /home/output/model_job.stat
>
> That'll look through all the input CTS lines, throw out any lines
> with a base rate (BASER in the CTS line) value less than the
> threshold, and give you summary information for the CSI and GSS
> columnsof data.
>
> Hope that does the trick for you.
>
> Thanks,
> John Halley Gotway
> met_help at ucar.edu
>
> RAL HelpDesk {for Derek.Stratman} wrote:
> > Thu Aug 19 10:22:12 2010: Request 40186 was acted upon.
> > Transaction: Ticket created by Derek.Stratman at noaa.gov
> >        Queue: met_help
> >      Subject: Using Base Rate
> >        Owner: Nobody
> >   Requestors: Derek.Stratman at noaa.gov
> >       Status: new
> >  Ticket <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=40186 >
> >
> >
> > Hi,
> >
> > I'm using the Stat-Analysis tool and was wondering if I could use
> base rate as a filter.  For
> > example, I want keep the forecast hours that have a base rate
> higher than 0.00152 and
> > ignore those less than that value.
> >
> > Here is my example code (which works fine) for reference:
> >
> > stat_analysis \
> > -lookin /home/input \
> > -out /home/output/model_job.txt \
> > -v 2 \
> > -job aggregate_stat \
> > -line_type CTC -out_line_type CTS \
> > -fcst_lead 060000 -fcst_thres >=20.000 \
> > -dump_row /home/output/model_job.stat
> >
> > Thanks for your future help.
> >
> > Derek Stratman
>
>

------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #40186] Using Base Rate
From: John Halley Gotway
Time: Fri Aug 20 09:59:14 2010

Derek,

Great, glad that did the trick.  I'll go ahead and resolve this MET-
Help ticket.

Thanks,
John

RAL HelpDesk {for Derek.Stratman} wrote:
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=40186 >
>
> Thanks, John!
>
> I ended up using the first method you suggested, and now it does
what I've been wanting it
> to do.  I have no further questions at the moment.
>
> Thanks again,
> Derek Stratman
>
> ----- Original Message -----
> From: RAL HelpDesk {for John Halley Gotway} <met_help at ucar.edu>
> Date: Thursday, August 19, 2010 3:00 pm
> Subject: Re: [rt.rap.ucar.edu #40186] Using Base Rate
>> Derek,
>>
>> Very good question.  Unfortunately the simple answer to your
>> question, is no, there's no easy way to do this for the specific
>> job you sent me.
>>
>> However, you do have a few options...
>>
>> The first thing to understand is that STAT-Analysis takes a very
>> simple approach to filtering data.  You can tell it column names
>> and values to use, but those columns must already exist in the
output
>> you're filtering.  In the job you sent, you're adding up
>> contingency table counts and writing out contingency table
>> statistics.  So STAT-Analysis is reading CTC lines.  You'd be able
>> to filter on any
>> of the CTC columns of data "TOTAL, FY_OY, FY_ON, FN_OY, FN_ON", but
>> none of those would give you the base rate.
>>
>> However, another output line type from MET is the FHO line type -
>> forecast, hit, and observation rate line.  This data is actually
>> redundant with the CTC data - just presented slightly differently.
>> Fortunately though, one of these columns DOES give you the base
>> rate.  The observation rate (O_RATE) column is the same as the base
>> rate.
>> So if you have FHO lines in your output, the following job should
>> do the trick:
>>
>> stat_analysis \
>> -lookin /home/input \
>> -out /home/output/model_job.txt \
>> -v 2 \
>> -job aggregate_stat \
>> -line_type FHO -out_line_type CTS \
>> -column_min O_RATE 0.00152 \
>> -fcst_lead 060000 -fcst_thresh >=20.000 \
>> -dump_row /home/output/model_job.stat
>>
>> That'll throw out any lines with an O_RATE value less than the
>> threshold.
>> Another possible job you could consider is something like this:
>>
>> stat_analysis \
>> -lookin /home/input \
>> -out /home/output/model_job.txt \
>> -v 2 \
>> -job summary -line_type CTS -column CSI -column GSS \
>> -column_min BASER 0.00152 \
>> -fcst_lead 060000 -fcst_thresh >=20.000 \
>> -dump_row /home/output/model_job.stat
>>
>> That'll look through all the input CTS lines, throw out any lines
>> with a base rate (BASER in the CTS line) value less than the
>> threshold, and give you summary information for the CSI and GSS
>> columnsof data.
>>
>> Hope that does the trick for you.
>>
>> Thanks,
>> John Halley Gotway
>> met_help at ucar.edu
>>
>> RAL HelpDesk {for Derek.Stratman} wrote:
>>> Thu Aug 19 10:22:12 2010: Request 40186 was acted upon.
>>> Transaction: Ticket created by Derek.Stratman at noaa.gov
>>>        Queue: met_help
>>>      Subject: Using Base Rate
>>>        Owner: Nobody
>>>   Requestors: Derek.Stratman at noaa.gov
>>>       Status: new
>>>  Ticket <URL:
>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=40186 >
>>>
>>> Hi,
>>>
>>> I'm using the Stat-Analysis tool and was wondering if I could use
>> base rate as a filter.  For
>>> example, I want keep the forecast hours that have a base rate
>> higher than 0.00152 and
>>> ignore those less than that value.
>>>
>>> Here is my example code (which works fine) for reference:
>>>
>>> stat_analysis \
>>> -lookin /home/input \
>>> -out /home/output/model_job.txt \
>>> -v 2 \
>>> -job aggregate_stat \
>>> -line_type CTC -out_line_type CTS \
>>> -fcst_lead 060000 -fcst_thres >=20.000 \
>>> -dump_row /home/output/model_job.stat
>>>
>>> Thanks for your future help.
>>>
>>> Derek Stratman
>>

------------------------------------------------