[Met_help] [rt.rap.ucar.edu #56371] History for Verification of Rainfall within a Range.
Tressa Fowler via RT
met_help at ucar.edu
Mon Jul 9 08:57:39 MDT 2012
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Dear met_help,
I am verifying Rainfall output from WRF with the station data. For this I am using Pointstat tool.
I wish to obtain the statistcs for the rainfall with a range say, RF<=0.1 AND RF >=2.4
"ge 0.1 le 2.4".
1. This is the syntax given in the Configuration file.
fcst_thresh[] = [ "ge0.1 le2.4"];
obs_thresh[] = [ "ge0.1 le2.4"];
I am able to get the output, but AND condition is not statisfoed, I am getting the results individually. Eg.
This is output file.
-bash-3.2$ cat point_stat_1to24_270000L_20110612_030000V_ctc.txt
VERSION MODEL FCST_LEAD FCST_VALID_BEG FCST_VALID_END OBS_LEAD OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_LEV OBS_VAR OBS_LEV OBTYPE VX_MASK INTERP_MTHD INTERP_PNTS FCST_THRESH OBS_THRESH COV_THRESH ALPHA LINE_TYPE TOTAL FY_OY FY_ON FN_OY FN_ON
V3.0 WRF 270000 20110612_030000 20110612_030000 000000 20110611_210000 20110612_090000 APCP_24 A24 APCP_24 A24 ADPSFC FULL UW_MEAN 1 >=0.100 >=0.100 NA NA CTC 290 145 72 21 52
V3.0 WRF 270000 20110612_030000 20110612_030000 000000 20110611_210000 20110612_090000 APCP_24 A24 APCP_24 A24 ADPSFC FULL UW_MEAN 1 <=2.400 <=2.400 NA NA CTC 290 118 35 30 107
-bash-3.2$
2. I have also tried the fiollowing syntax in the configuration file.
fcst_field[] = [ "APCP_24(*,*)","APCP_24(*,*)"];
obs_field[] = ["61/A24","61/A24"];
fcst_thresh[] = [ "ge0.1","le2.4"];
obs_thresh[] = [ "ge0.1","le2.4"];
-bash-3.2$ cat point_stat_1to24_270000L_20110612_030000V_ctc.txt
VERSION MODEL FCST_LEAD FCST_VALID_BEG FCST_VALID_END OBS_LEAD OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_LEV OBS_VAR OBS_LEV OBTYPE VX_MASK INTERP_MTHD INTERP_PNTS FCST_THRESH OBS_THRESH COV_THRESH ALPHA LINE_TYPE TOTAL FY_OY FY_ON FN_OY FN_ON
V3.0 WRF 270000 20110612_030000 20110612_030000 000000 20110611_210000 20110612_090000 APCP_24 A24 APCP_24 A24 ADPSFC FULL UW_MEAN 1 >=0.100 >=0.100 NA NA CTC 290 145 72 21 52
V3.0 WRF 270000 20110612_030000 20110612_030000 000000 20110611_210000 20110612_090000 APCP_24 A24 APCP_24 A24 ADPSFC FULL UW_MEAN 1 <=2.400 <=2.400 NA NA CTC 290 118 35 30 107
-bash-3.2$
I obtain the same result. Pls help. I am attaching the files.
geeta
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #56371] Verification of Rainfall within a Range.
From: Paul Oldenburg
Time: Thu May 10 09:24:36 2012
Geeta,
Unfortunately, MET does not support a single event defined by more
than one threshold. There is support for a similar
type of analysis, called multi-category statistics. In order to
activate that functionality in MET, you should use the
following config file settings:
fcst_thresh[] = [ "ge0.1 ge2.4"]; // all inequalities must have the
same sign
output_flag[] = [ 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2 ]; //
activate flags 4 & 5 - MCTC & MCTS
If you are interested in creating a 2x2 contingency table using the
multiple threshold event, then I suggest using the
MET point_stat tool to generate the matched pair data (MPR - output
flag 15). Then, the MPR lines, which are the
individual fcst/obs values at each point can be read into a
statistical computing software package like R or matlab and
analyzed using the multi-threshold event. I'm sorry there is not more
support in MET at this time for multi-threshold
events. We are considering adding it to a future release.
Please let me know if you have any questions.
Thanks,
Paul
On 05/10/2012 05:07 AM, Geeta Geeta via RT wrote:
>
> Thu May 10 05:07:29 2012: Request 56371 was acted upon.
> Transaction: Ticket created by geeta124 at hotmail.com
> Queue: met_help
> Subject: Verification of Rainfall within a Range.
> Owner: Nobody
> Requestors: geeta124 at hotmail.com
> Status: new
> Ticket<URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=56371>
>
>
>
> Dear met_help,
> I am verifying Rainfall output from WRF with the station data. For
this I am using Pointstat tool.
>
> I wish to obtain the statistcs for the rainfall with a range say,
RF<=0.1 AND RF>=2.4
> "ge 0.1 le 2.4".
> 1. This is the syntax given in the Configuration file.
> fcst_thresh[] = [ "ge0.1 le2.4"];
> obs_thresh[] = [ "ge0.1 le2.4"];
>
> I am able to get the output, but AND condition is not statisfoed, I
am getting the results individually. Eg.
> This is output file.
>
> -bash-3.2$ cat point_stat_1to24_270000L_20110612_030000V_ctc.txt
> VERSION MODEL FCST_LEAD FCST_VALID_BEG FCST_VALID_END OBS_LEAD
OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_LEV OBS_VAR OBS_LEV
OBTYPE VX_MASK INTERP_MTHD INTERP_PNTS FCST_THRESH OBS_THRESH
COV_THRESH ALPHA LINE_TYPE TOTAL FY_OY FY_ON FN_OY FN_ON
> V3.0 WRF 270000 20110612_030000 20110612_030000 000000
20110611_210000 20110612_090000 APCP_24 A24 APCP_24 A24
ADPSFC FULL UW_MEAN 1>=0.100>=0.100 NA NA CTC
290 145 72 21 52
> V3.0 WRF 270000 20110612_030000 20110612_030000 000000
20110611_210000 20110612_090000 APCP_24 A24 APCP_24 A24
ADPSFC FULL UW_MEAN 1<=2.400<=2.400 NA NA CTC
290 118 35 30 107
> -bash-3.2$
>
> 2. I have also tried the fiollowing syntax in the configuration
file.
>
> fcst_field[] = [ "APCP_24(*,*)","APCP_24(*,*)"];
> obs_field[] = ["61/A24","61/A24"];
>
>
> fcst_thresh[] = [ "ge0.1","le2.4"];
> obs_thresh[] = [ "ge0.1","le2.4"];
>
> -bash-3.2$ cat point_stat_1to24_270000L_20110612_030000V_ctc.txt
> VERSION MODEL FCST_LEAD FCST_VALID_BEG FCST_VALID_END OBS_LEAD
OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_LEV OBS_VAR OBS_LEV
OBTYPE VX_MASK INTERP_MTHD INTERP_PNTS FCST_THRESH OBS_THRESH
COV_THRESH ALPHA LINE_TYPE TOTAL FY_OY FY_ON FN_OY FN_ON
> V3.0 WRF 270000 20110612_030000 20110612_030000 000000
20110611_210000 20110612_090000 APCP_24 A24 APCP_24 A24
ADPSFC FULL UW_MEAN 1>=0.100>=0.100 NA NA CTC
290 145 72 21 52
> V3.0 WRF 270000 20110612_030000 20110612_030000 000000
20110611_210000 20110612_090000 APCP_24 A24 APCP_24 A24
ADPSFC FULL UW_MEAN 1<=2.400<=2.400 NA NA CTC
290 118 35 30 107
> -bash-3.2$
>
> I obtain the same result. Pls help. I am attaching the files.
>
>
> geeta
------------------------------------------------
Subject: Verification of Rainfall within a Range.
From: Tressa Fowler
Time: Thu May 10 10:00:33 2012
Hi Geeta,
I am assuming that the analysis you truly want to undertake is via the
multiple category verification statistics, as mentioned by Paul. This
type of conditional analysis can give you much information about the
performance of your forecast for both common and extreme events,
though the number of skill scores ,etc available are fewer than for a
binary event.
The problem with using the two threshold to produce a binary event is
that you combine both low and high values into one category even
though they represent completely different conditions and usually have
very different frequencies. The skill scores were not designed (and
are not statistically defensible) for this situation. There is little
if anything to be learned this way. It is better to analyze these as
two separate categories.
I recommend reading more about categorical statistics and skill scores
in Dan Wilks' book, _Statistical Methods in the Atmospheric Sciences_.
Please let me know if you need any further assistance.
Tressa
On Thu May 10 05:07:29 2012, geeta124 at hotmail.com wrote:
>
> Dear met_help,
> I am verifying Rainfall output from WRF with the station data. For
> this I am using Pointstat tool.
>
> I wish to obtain the statistcs for the rainfall with a range say,
> RF<=0.1 AND RF >=2.4
> "ge 0.1 le 2.4".
> 1. This is the syntax given in the Configuration file.
> fcst_thresh[] = [ "ge0.1 le2.4"];
> obs_thresh[] = [ "ge0.1 le2.4"];
>
> I am able to get the output, but AND condition is not statisfoed, I
am
> getting the results individually. Eg.
> This is output file.
>
> -bash-3.2$ cat point_stat_1to24_270000L_20110612_030000V_ctc.txt
> VERSION MODEL FCST_LEAD FCST_VALID_BEG FCST_VALID_END OBS_LEAD
> OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_LEV OBS_VAR OBS_LEV
> OBTYPE VX_MASK INTERP_MTHD INTERP_PNTS FCST_THRESH OBS_THRESH
> COV_THRESH ALPHA LINE_TYPE TOTAL FY_OY FY_ON FN_OY FN_ON
> V3.0 WRF 270000 20110612_030000 20110612_030000 000000
> 20110611_210000 20110612_090000 APCP_24 A24 APCP_24 A24
> ADPSFC FULL UW_MEAN 1 >=0.100 >=0.100 NA
> NA CTC 290 145 72 21 52
> V3.0 WRF 270000 20110612_030000 20110612_030000 000000
> 20110611_210000 20110612_090000 APCP_24 A24 APCP_24 A24
> ADPSFC FULL UW_MEAN 1 <=2.400 <=2.400 NA
> NA CTC 290 118 35 30 107
> -bash-3.2$
>
> 2. I have also tried the fiollowing syntax in the configuration
file.
>
> fcst_field[] = [ "APCP_24(*,*)","APCP_24(*,*)"];
> obs_field[] = ["61/A24","61/A24"];
>
>
> fcst_thresh[] = [ "ge0.1","le2.4"];
> obs_thresh[] = [ "ge0.1","le2.4"];
>
> -bash-3.2$ cat point_stat_1to24_270000L_20110612_030000V_ctc.txt
> VERSION MODEL FCST_LEAD FCST_VALID_BEG FCST_VALID_END OBS_LEAD
> OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_LEV OBS_VAR OBS_LEV
> OBTYPE VX_MASK INTERP_MTHD INTERP_PNTS FCST_THRESH OBS_THRESH
> COV_THRESH ALPHA LINE_TYPE TOTAL FY_OY FY_ON FN_OY FN_ON
> V3.0 WRF 270000 20110612_030000 20110612_030000 000000
> 20110611_210000 20110612_090000 APCP_24 A24 APCP_24 A24
> ADPSFC FULL UW_MEAN 1 >=0.100 >=0.100 NA
> NA CTC 290 145 72 21 52
> V3.0 WRF 270000 20110612_030000 20110612_030000 000000
> 20110611_210000 20110612_090000 APCP_24 A24 APCP_24 A24
> ADPSFC FULL UW_MEAN 1 <=2.400 <=2.400 NA
> NA CTC 290 118 35 30 107
> -bash-3.2$
>
> I obtain the same result. Pls help. I am attaching the files.
>
>
> geeta
------------------------------------------------
Subject: Verification of Rainfall within a Range.
From: Geeta Geeta
Time: Fri May 11 01:38:59 2012
Hi Tressa,
Thanks for your mail.
I am not very clear when U say > The problem with using the two
threshold to produce a binary event
is that you combine both low and high values into one category even
though they represent completely different conditions and usually have
very different frequencies.
Kindly clarify when U say they represent two different conditions and
have diff frequencies!!!!
Actually, In our scheme of things in India Met dept, We define the
very light Rainfall if it is within 0.1-2.4mm,
light
Rainfall if it is 2.5-7.5 mm
moderate
if 7.6- 35.5 and so on.
so I was trying to do the same, i.e checking how the model behaves in
each category. in other words, in which category, its results are more
relaiable.
Finally, from this discussion i understand that I have to test model's
performance like this ge0.1, ge2.5, ge 7.6 etc and so on.
geeta
> Subject: [rt.rap.ucar.edu #56371] Verification of Rainfall within a
Range.
> From: met_help at ucar.edu
> To: geeta124 at hotmail.com
> CC: met_help-owner at mailman.ucar.edu; met_help at mailman.ucar.edu
> Date: Thu, 10 May 2012 10:00:34 -0600
>
> Hi Geeta,
>
> I am assuming that the analysis you truly want to undertake is via
the multiple category verification statistics, as mentioned by Paul.
This type of conditional analysis can give you much information about
the performance of your forecast for both common and extreme events,
though the number of skill scores ,etc available are fewer than for a
binary event.
>
> The problem with using the two threshold to produce a binary event
is that you combine both low and high values into one category even
though they represent completely different conditions and usually have
very different frequencies. The skill scores were not designed (and
are not statistically defensible) for this situation. There is little
if anything to be learned this way. It is better to analyze these as
two separate categories.
>
> I recommend reading more about categorical statistics and skill
scores in Dan Wilks' book, _Statistical Methods in the Atmospheric
Sciences_.
>
> Please let me know if you need any further assistance.
>
> Tressa
>
> On Thu May 10 05:07:29 2012, geeta124 at hotmail.com wrote:
> >
> > Dear met_help,
> > I am verifying Rainfall output from WRF with the station data. For
> > this I am using Pointstat tool.
> >
> > I wish to obtain the statistcs for the rainfall with a range say,
> > RF<=0.1 AND RF >=2.4
> > "ge 0.1 le 2.4".
> > 1. This is the syntax given in the Configuration file.
> > fcst_thresh[] = [ "ge0.1 le2.4"];
> > obs_thresh[] = [ "ge0.1 le2.4"];
> >
> > I am able to get the output, but AND condition is not statisfoed,
I am
> > getting the results individually. Eg.
> > This is output file.
> >
> > -bash-3.2$ cat point_stat_1to24_270000L_20110612_030000V_ctc.txt
> > VERSION MODEL FCST_LEAD FCST_VALID_BEG FCST_VALID_END OBS_LEAD
> > OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_LEV OBS_VAR
OBS_LEV
> > OBTYPE VX_MASK INTERP_MTHD INTERP_PNTS FCST_THRESH OBS_THRESH
> > COV_THRESH ALPHA LINE_TYPE TOTAL FY_OY FY_ON FN_OY FN_ON
> > V3.0 WRF 270000 20110612_030000 20110612_030000 000000
> > 20110611_210000 20110612_090000 APCP_24 A24 APCP_24 A24
> > ADPSFC FULL UW_MEAN 1 >=0.100 >=0.100
NA
> > NA CTC 290 145 72 21 52
> > V3.0 WRF 270000 20110612_030000 20110612_030000 000000
> > 20110611_210000 20110612_090000 APCP_24 A24 APCP_24 A24
> > ADPSFC FULL UW_MEAN 1 <=2.400 <=2.400
NA
> > NA CTC 290 118 35 30 107
> > -bash-3.2$
> >
> > 2. I have also tried the fiollowing syntax in the configuration
file.
> >
> > fcst_field[] = [ "APCP_24(*,*)","APCP_24(*,*)"];
> > obs_field[] = ["61/A24","61/A24"];
> >
> >
> > fcst_thresh[] = [ "ge0.1","le2.4"];
> > obs_thresh[] = [ "ge0.1","le2.4"];
> >
> > -bash-3.2$ cat point_stat_1to24_270000L_20110612_030000V_ctc.txt
> > VERSION MODEL FCST_LEAD FCST_VALID_BEG FCST_VALID_END OBS_LEAD
> > OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_LEV OBS_VAR
OBS_LEV
> > OBTYPE VX_MASK INTERP_MTHD INTERP_PNTS FCST_THRESH OBS_THRESH
> > COV_THRESH ALPHA LINE_TYPE TOTAL FY_OY FY_ON FN_OY FN_ON
> > V3.0 WRF 270000 20110612_030000 20110612_030000 000000
> > 20110611_210000 20110612_090000 APCP_24 A24 APCP_24 A24
> > ADPSFC FULL UW_MEAN 1 >=0.100 >=0.100
NA
> > NA CTC 290 145 72 21 52
> > V3.0 WRF 270000 20110612_030000 20110612_030000 000000
> > 20110611_210000 20110612_090000 APCP_24 A24 APCP_24 A24
> > ADPSFC FULL UW_MEAN 1 <=2.400 <=2.400
NA
> > NA CTC 290 118 35 30 107
> > -bash-3.2$
> >
> > I obtain the same result. Pls help. I am attaching the files.
> >
> >
> > geeta
>
>
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #56371] Verification of Rainfall within a Range.
From: Tressa Fowler
Time: Fri May 11 07:06:26 2012
Hi Geeta,
It sounds like you want to use our multi category analysis
In your original query, if our software would have applied your
thresholds with the 'and' criteria, then all rainfall events below 0.1
and above 2.4 would have been combined into the 'no event' counts.
This is what I was cautioning you about. It does not sound like that
was your intent however, just a misunderstanding about what exactly
use of thresholds produces.
Good luck with your analysis.
Tressa
On May 11, 2012, at 1:39 AM, Geeta Geeta via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=56371 >
>
>
> Hi Tressa,
> Thanks for your mail.
> I am not very clear when U say > The problem with using the two
threshold to produce a binary event
> is that you combine both low and high values into one category even
> though they represent completely different conditions and usually
have
> very different frequencies.
> Kindly clarify when U say they represent two different conditions
and have diff frequencies!!!!
>
> Actually, In our scheme of things in India Met dept, We define the
very light Rainfall if it is within 0.1-2.4mm,
>
light Rainfall if it is 2.5-7.5 mm
>
moderate if 7.6- 35.5 and so on.
> so I was trying to do the same, i.e checking how the model behaves
in each category. in other words, in which category, its results are
more relaiable.
> Finally, from this discussion i understand that I have to test
model's performance like this ge0.1, ge2.5, ge 7.6 etc and so on.
>
> geeta
>
>> Subject: [rt.rap.ucar.edu #56371] Verification of Rainfall within a
Range.
>> From: met_help at ucar.edu
>> To: geeta124 at hotmail.com
>> CC: met_help-owner at mailman.ucar.edu; met_help at mailman.ucar.edu
>> Date: Thu, 10 May 2012 10:00:34 -0600
>>
>> Hi Geeta,
>>
>> I am assuming that the analysis you truly want to undertake is via
the multiple category verification statistics, as mentioned by Paul.
This type of conditional analysis can give you much information about
the performance of your forecast for both common and extreme events,
though the number of skill scores ,etc available are fewer than for a
binary event.
>>
>> The problem with using the two threshold to produce a binary event
is that you combine both low and high values into one category even
though they represent completely different conditions and usually have
very different frequencies. The skill scores were not designed (and
are not statistically defensible) for this situation. There is little
if anything to be learned this way. It is better to analyze these as
two separate categories.
>>
>> I recommend reading more about categorical statistics and skill
scores in Dan Wilks' book, _Statistical Methods in the Atmospheric
Sciences_.
>>
>> Please let me know if you need any further assistance.
>>
>> Tressa
>>
>> On Thu May 10 05:07:29 2012, geeta124 at hotmail.com wrote:
>>>
>>> Dear met_help,
>>> I am verifying Rainfall output from WRF with the station data. For
>>> this I am using Pointstat tool.
>>>
>>> I wish to obtain the statistcs for the rainfall with a range say,
>>> RF<=0.1 AND RF >=2.4
>>> "ge 0.1 le 2.4".
>>> 1. This is the syntax given in the Configuration file.
>>> fcst_thresh[] = [ "ge0.1 le2.4"];
>>> obs_thresh[] = [ "ge0.1 le2.4"];
>>>
>>> I am able to get the output, but AND condition is not statisfoed,
I am
>>> getting the results individually. Eg.
>>> This is output file.
>>>
>>> -bash-3.2$ cat point_stat_1to24_270000L_20110612_030000V_ctc.txt
>>> VERSION MODEL FCST_LEAD FCST_VALID_BEG FCST_VALID_END OBS_LEAD
>>> OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_LEV OBS_VAR
OBS_LEV
>>> OBTYPE VX_MASK INTERP_MTHD INTERP_PNTS FCST_THRESH OBS_THRESH
>>> COV_THRESH ALPHA LINE_TYPE TOTAL FY_OY FY_ON FN_OY FN_ON
>>> V3.0 WRF 270000 20110612_030000 20110612_030000 000000
>>> 20110611_210000 20110612_090000 APCP_24 A24 APCP_24 A24
>>> ADPSFC FULL UW_MEAN 1 >=0.100 >=0.100 NA
>>> NA CTC 290 145 72 21 52
>>> V3.0 WRF 270000 20110612_030000 20110612_030000 000000
>>> 20110611_210000 20110612_090000 APCP_24 A24 APCP_24 A24
>>> ADPSFC FULL UW_MEAN 1 <=2.400 <=2.400 NA
>>> NA CTC 290 118 35 30 107
>>> -bash-3.2$
>>>
>>> 2. I have also tried the fiollowing syntax in the configuration
file.
>>>
>>> fcst_field[] = [ "APCP_24(*,*)","APCP_24(*,*)"];
>>> obs_field[] = ["61/A24","61/A24"];
>>>
>>>
>>> fcst_thresh[] = [ "ge0.1","le2.4"];
>>> obs_thresh[] = [ "ge0.1","le2.4"];
>>>
>>> -bash-3.2$ cat point_stat_1to24_270000L_20110612_030000V_ctc.txt
>>> VERSION MODEL FCST_LEAD FCST_VALID_BEG FCST_VALID_END OBS_LEAD
>>> OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_LEV OBS_VAR
OBS_LEV
>>> OBTYPE VX_MASK INTERP_MTHD INTERP_PNTS FCST_THRESH OBS_THRESH
>>> COV_THRESH ALPHA LINE_TYPE TOTAL FY_OY FY_ON FN_OY FN_ON
>>> V3.0 WRF 270000 20110612_030000 20110612_030000 000000
>>> 20110611_210000 20110612_090000 APCP_24 A24 APCP_24 A24
>>> ADPSFC FULL UW_MEAN 1 >=0.100 >=0.100 NA
>>> NA CTC 290 145 72 21 52
>>> V3.0 WRF 270000 20110612_030000 20110612_030000 000000
>>> 20110611_210000 20110612_090000 APCP_24 A24 APCP_24 A24
>>> ADPSFC FULL UW_MEAN 1 <=2.400 <=2.400 NA
>>> NA CTC 290 118 35 30 107
>>> -bash-3.2$
>>>
>>> I obtain the same result. Pls help. I am attaching the files.
>>>
>>>
>>> geeta
>>
>>
>>
>
------------------------------------------------
Subject: Verification of Rainfall within a Range.
From: Geeta Geeta
Time: Sat May 12 09:37:35 2012
hi tressa,
I am not clear again and a bit confused. Yes I want to use
muticategory analysis, but somehow not able to get the results.
My query was if MET software will give me those conditions when RF is
ge 0.1 AND le 2.4. In this category,there will be some events
definitely, Say, 2.2mm etc.
which is NOT "no event count".
As Paul has said that it is not possible at this time (using METv3.0),
and they are thinking of this for later releases. That is why I wrote
to you that I will test the model's performance by taking categories
like ge0.1, ge 2.4, ge7.5 INDIVIDUALLY (not with AND condition).
Hope I been able to express my problem clearly.
Thanks for support
geeta
> Subject: Re: [rt.rap.ucar.edu #56371] Verification of Rainfall
within a Range.
> From: met_help at ucar.edu
> To: geeta124 at hotmail.com
> CC: met_help-owner at mailman.ucar.edu; met_help at mailman.ucar.edu
> Date: Fri, 11 May 2012 07:06:26 -0600
>
> Hi Geeta,
>
> It sounds like you want to use our multi category analysis
>
> In your original query, if our software would have applied your
thresholds with the 'and' criteria, then all rainfall events below 0.1
and above 2.4 would have been combined into the 'no event' counts.
This is what I was cautioning you about. It does not sound like that
was your intent however, just a misunderstanding about what exactly
use of thresholds produces.
>
> Good luck with your analysis.
>
> Tressa
>
>
> On May 11, 2012, at 1:39 AM, Geeta Geeta via RT wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=56371 >
> >
> >
> > Hi Tressa,
> > Thanks for your mail.
> > I am not very clear when U say > The problem with using the two
threshold to produce a binary event
> > is that you combine both low and high values into one category
even
> > though they represent completely different conditions and usually
have
> > very different frequencies.
> > Kindly clarify when U say they represent two different conditions
and have diff frequencies!!!!
> >
> > Actually, In our scheme of things in India Met dept, We define the
very light Rainfall if it is within 0.1-2.4mm,
> >
light Rainfall if it is 2.5-7.5 mm
> >
moderate if 7.6- 35.5 and so on.
> > so I was trying to do the same, i.e checking how the model behaves
in each category. in other words, in which category, its results are
more relaiable.
> > Finally, from this discussion i understand that I have to test
model's performance like this ge0.1, ge2.5, ge 7.6 etc and so on.
> >
> > geeta
> >
> >> Subject: [rt.rap.ucar.edu #56371] Verification of Rainfall within
a Range.
> >> From: met_help at ucar.edu
> >> To: geeta124 at hotmail.com
> >> CC: met_help-owner at mailman.ucar.edu; met_help at mailman.ucar.edu
> >> Date: Thu, 10 May 2012 10:00:34 -0600
> >>
> >> Hi Geeta,
> >>
> >> I am assuming that the analysis you truly want to undertake is
via the multiple category verification statistics, as mentioned by
Paul. This type of conditional analysis can give you much information
about the performance of your forecast for both common and extreme
events, though the number of skill scores ,etc available are fewer
than for a binary event.
> >>
> >> The problem with using the two threshold to produce a binary
event is that you combine both low and high values into one category
even though they represent completely different conditions and usually
have very different frequencies. The skill scores were not designed
(and are not statistically defensible) for this situation. There is
little if anything to be learned this way. It is better to analyze
these as two separate categories.
> >>
> >> I recommend reading more about categorical statistics and skill
scores in Dan Wilks' book, _Statistical Methods in the Atmospheric
Sciences_.
> >>
> >> Please let me know if you need any further assistance.
> >>
> >> Tressa
> >>
> >> On Thu May 10 05:07:29 2012, geeta124 at hotmail.com wrote:
> >>>
> >>> Dear met_help,
> >>> I am verifying Rainfall output from WRF with the station data.
For
> >>> this I am using Pointstat tool.
> >>>
> >>> I wish to obtain the statistcs for the rainfall with a range
say,
> >>> RF<=0.1 AND RF >=2.4
> >>> "ge 0.1 le 2.4".
> >>> 1. This is the syntax given in the Configuration file.
> >>> fcst_thresh[] = [ "ge0.1 le2.4"];
> >>> obs_thresh[] = [ "ge0.1 le2.4"];
> >>>
> >>> I am able to get the output, but AND condition is not
statisfoed, I am
> >>> getting the results individually. Eg.
> >>> This is output file.
> >>>
> >>> -bash-3.2$ cat point_stat_1to24_270000L_20110612_030000V_ctc.txt
> >>> VERSION MODEL FCST_LEAD FCST_VALID_BEG FCST_VALID_END OBS_LEAD
> >>> OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_LEV OBS_VAR
OBS_LEV
> >>> OBTYPE VX_MASK INTERP_MTHD INTERP_PNTS FCST_THRESH OBS_THRESH
> >>> COV_THRESH ALPHA LINE_TYPE TOTAL FY_OY FY_ON FN_OY FN_ON
> >>> V3.0 WRF 270000 20110612_030000 20110612_030000 000000
> >>> 20110611_210000 20110612_090000 APCP_24 A24 APCP_24 A24
> >>> ADPSFC FULL UW_MEAN 1 >=0.100 >=0.100
NA
> >>> NA CTC 290 145 72 21 52
> >>> V3.0 WRF 270000 20110612_030000 20110612_030000 000000
> >>> 20110611_210000 20110612_090000 APCP_24 A24 APCP_24 A24
> >>> ADPSFC FULL UW_MEAN 1 <=2.400 <=2.400
NA
> >>> NA CTC 290 118 35 30 107
> >>> -bash-3.2$
> >>>
> >>> 2. I have also tried the fiollowing syntax in the configuration
file.
> >>>
> >>> fcst_field[] = [ "APCP_24(*,*)","APCP_24(*,*)"];
> >>> obs_field[] = ["61/A24","61/A24"];
> >>>
> >>>
> >>> fcst_thresh[] = [ "ge0.1","le2.4"];
> >>> obs_thresh[] = [ "ge0.1","le2.4"];
> >>>
> >>> -bash-3.2$ cat point_stat_1to24_270000L_20110612_030000V_ctc.txt
> >>> VERSION MODEL FCST_LEAD FCST_VALID_BEG FCST_VALID_END OBS_LEAD
> >>> OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_LEV OBS_VAR
OBS_LEV
> >>> OBTYPE VX_MASK INTERP_MTHD INTERP_PNTS FCST_THRESH OBS_THRESH
> >>> COV_THRESH ALPHA LINE_TYPE TOTAL FY_OY FY_ON FN_OY FN_ON
> >>> V3.0 WRF 270000 20110612_030000 20110612_030000 000000
> >>> 20110611_210000 20110612_090000 APCP_24 A24 APCP_24 A24
> >>> ADPSFC FULL UW_MEAN 1 >=0.100 >=0.100
NA
> >>> NA CTC 290 145 72 21 52
> >>> V3.0 WRF 270000 20110612_030000 20110612_030000 000000
> >>> 20110611_210000 20110612_090000 APCP_24 A24 APCP_24 A24
> >>> ADPSFC FULL UW_MEAN 1 <=2.400 <=2.400
NA
> >>> NA CTC 290 118 35 30 107
> >>> -bash-3.2$
> >>>
> >>> I obtain the same result. Pls help. I am attaching the files.
> >>>
> >>>
> >>> geeta
> >>
> >>
> >>
> >
>
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #56371] Verification of Rainfall within a Range.
From: Tressa Fowler
Time: Mon May 14 09:50:35 2012
Hi Geeta,
Which statistics exactly are you trying to produce using our software?
The categorical analysis requires at least 2 categories, usually event
and non-event, to produce skill scores like the Equitable Threat
score. It sounds like you are only interested in 1 category, from
which no skill scores can be derived, so maybe what you really want
are continuous statistics (like mean squared error) on a subset of
your data? This can be accomplished by filtering your observations
yourself first so that they are all between 0.1 and 2.4 mm of rain,
then only using those for verification. I would caution you again with
this type of analysis, since any false alarm forecasts would be
excluded.
I am sorry we are not more help right now, but I think we are not
clear about what you are trying to accomplish. If you write back,
perhaps you could clarify?
Tressa
On May 12, 2012, at 9:37 AM, Geeta Geeta via RT wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=56371 >
>
>
> hi tressa,
> I am not clear again and a bit confused. Yes I want to use
muticategory analysis, but somehow not able to get the results.
>
> My query was if MET software will give me those conditions when RF
is ge 0.1 AND le 2.4. In this category,there will be some events
definitely, Say, 2.2mm etc.
> which is NOT "no event count".
>
> As Paul has said that it is not possible at this time (using
METv3.0), and they are thinking of this for later releases. That is
why I wrote to you that I will test the model's performance by taking
categories like ge0.1, ge 2.4, ge7.5 INDIVIDUALLY (not with AND
condition).
>
> Hope I been able to express my problem clearly.
> Thanks for support
> geeta
>
>> Subject: Re: [rt.rap.ucar.edu #56371] Verification of Rainfall
within a Range.
>> From: met_help at ucar.edu
>> To: geeta124 at hotmail.com
>> CC: met_help-owner at mailman.ucar.edu; met_help at mailman.ucar.edu
>> Date: Fri, 11 May 2012 07:06:26 -0600
>>
>> Hi Geeta,
>>
>> It sounds like you want to use our multi category analysis
>>
>> In your original query, if our software would have applied your
thresholds with the 'and' criteria, then all rainfall events below 0.1
and above 2.4 would have been combined into the 'no event' counts.
This is what I was cautioning you about. It does not sound like that
was your intent however, just a misunderstanding about what exactly
use of thresholds produces.
>>
>> Good luck with your analysis.
>>
>> Tressa
>>
>>
>> On May 11, 2012, at 1:39 AM, Geeta Geeta via RT wrote:
>>
>>>
>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=56371 >
>>>
>>>
>>> Hi Tressa,
>>> Thanks for your mail.
>>> I am not very clear when U say > The problem with using the two
threshold to produce a binary event
>>> is that you combine both low and high values into one category
even
>>> though they represent completely different conditions and usually
have
>>> very different frequencies.
>>> Kindly clarify when U say they represent two different conditions
and have diff frequencies!!!!
>>>
>>> Actually, In our scheme of things in India Met dept, We define the
very light Rainfall if it is within 0.1-2.4mm,
>>>
light Rainfall if it is 2.5-7.5 mm
>>>
moderate if 7.6- 35.5 and so on.
>>> so I was trying to do the same, i.e checking how the model behaves
in each category. in other words, in which category, its results are
more relaiable.
>>> Finally, from this discussion i understand that I have to test
model's performance like this ge0.1, ge2.5, ge 7.6 etc and so on.
>>>
>>> geeta
>>>
>>>> Subject: [rt.rap.ucar.edu #56371] Verification of Rainfall within
a Range.
>>>> From: met_help at ucar.edu
>>>> To: geeta124 at hotmail.com
>>>> CC: met_help-owner at mailman.ucar.edu; met_help at mailman.ucar.edu
>>>> Date: Thu, 10 May 2012 10:00:34 -0600
>>>>
>>>> Hi Geeta,
>>>>
>>>> I am assuming that the analysis you truly want to undertake is
via the multiple category verification statistics, as mentioned by
Paul. This type of conditional analysis can give you much information
about the performance of your forecast for both common and extreme
events, though the number of skill scores ,etc available are fewer
than for a binary event.
>>>>
>>>> The problem with using the two threshold to produce a binary
event is that you combine both low and high values into one category
even though they represent completely different conditions and usually
have very different frequencies. The skill scores were not designed
(and are not statistically defensible) for this situation. There is
little if anything to be learned this way. It is better to analyze
these as two separate categories.
>>>>
>>>> I recommend reading more about categorical statistics and skill
scores in Dan Wilks' book, _Statistical Methods in the Atmospheric
Sciences_.
>>>>
>>>> Please let me know if you need any further assistance.
>>>>
>>>> Tressa
>>>>
>>>> On Thu May 10 05:07:29 2012, geeta124 at hotmail.com wrote:
>>>>>
>>>>> Dear met_help,
>>>>> I am verifying Rainfall output from WRF with the station data.
For
>>>>> this I am using Pointstat tool.
>>>>>
>>>>> I wish to obtain the statistcs for the rainfall with a range
say,
>>>>> RF<=0.1 AND RF >=2.4
>>>>> "ge 0.1 le 2.4".
>>>>> 1. This is the syntax given in the Configuration file.
>>>>> fcst_thresh[] = [ "ge0.1 le2.4"];
>>>>> obs_thresh[] = [ "ge0.1 le2.4"];
>>>>>
>>>>> I am able to get the output, but AND condition is not
statisfoed, I am
>>>>> getting the results individually. Eg.
>>>>> This is output file.
>>>>>
>>>>> -bash-3.2$ cat point_stat_1to24_270000L_20110612_030000V_ctc.txt
>>>>> VERSION MODEL FCST_LEAD FCST_VALID_BEG FCST_VALID_END OBS_LEAD
>>>>> OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_LEV OBS_VAR
OBS_LEV
>>>>> OBTYPE VX_MASK INTERP_MTHD INTERP_PNTS FCST_THRESH OBS_THRESH
>>>>> COV_THRESH ALPHA LINE_TYPE TOTAL FY_OY FY_ON FN_OY FN_ON
>>>>> V3.0 WRF 270000 20110612_030000 20110612_030000 000000
>>>>> 20110611_210000 20110612_090000 APCP_24 A24 APCP_24 A24
>>>>> ADPSFC FULL UW_MEAN 1 >=0.100 >=0.100
NA
>>>>> NA CTC 290 145 72 21 52
>>>>> V3.0 WRF 270000 20110612_030000 20110612_030000 000000
>>>>> 20110611_210000 20110612_090000 APCP_24 A24 APCP_24 A24
>>>>> ADPSFC FULL UW_MEAN 1 <=2.400 <=2.400
NA
>>>>> NA CTC 290 118 35 30 107
>>>>> -bash-3.2$
>>>>>
>>>>> 2. I have also tried the fiollowing syntax in the configuration
file.
>>>>>
>>>>> fcst_field[] = [ "APCP_24(*,*)","APCP_24(*,*)"];
>>>>> obs_field[] = ["61/A24","61/A24"];
>>>>>
>>>>>
>>>>> fcst_thresh[] = [ "ge0.1","le2.4"];
>>>>> obs_thresh[] = [ "ge0.1","le2.4"];
>>>>>
>>>>> -bash-3.2$ cat point_stat_1to24_270000L_20110612_030000V_ctc.txt
>>>>> VERSION MODEL FCST_LEAD FCST_VALID_BEG FCST_VALID_END OBS_LEAD
>>>>> OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_LEV OBS_VAR
OBS_LEV
>>>>> OBTYPE VX_MASK INTERP_MTHD INTERP_PNTS FCST_THRESH OBS_THRESH
>>>>> COV_THRESH ALPHA LINE_TYPE TOTAL FY_OY FY_ON FN_OY FN_ON
>>>>> V3.0 WRF 270000 20110612_030000 20110612_030000 000000
>>>>> 20110611_210000 20110612_090000 APCP_24 A24 APCP_24 A24
>>>>> ADPSFC FULL UW_MEAN 1 >=0.100 >=0.100
NA
>>>>> NA CTC 290 145 72 21 52
>>>>> V3.0 WRF 270000 20110612_030000 20110612_030000 000000
>>>>> 20110611_210000 20110612_090000 APCP_24 A24 APCP_24 A24
>>>>> ADPSFC FULL UW_MEAN 1 <=2.400 <=2.400
NA
>>>>> NA CTC 290 118 35 30 107
>>>>> -bash-3.2$
>>>>>
>>>>> I obtain the same result. Pls help. I am attaching the files.
>>>>>
>>>>>
>>>>> geeta
>>>>
>>>>
>>>>
>>>
>>
>>
>
------------------------------------------------
More information about the Met_help
mailing list