[Met_help] [rt.rap.ucar.edu #70133] History for Query Reg MCTC/MCTS
John Halley Gotway via RT
met_help at ucar.edu
Wed Jan 7 14:12:54 MST 2015
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
hi John.
this is again reg the MCTC/MCTS output. I want to know how can I get the O/p. as of now I m just getting header thing in my o/p files.
I mean I want to know the procedure and the concept behind this.
Regards
Geeta
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Query Reg MCTC/MCTS
From: John Halley Gotway
Time: Wed Dec 17 10:08:02 2014
Geeta,
MCTC stands for multi-category contingency table counts and MCTS
stands for
multi-category contingency table statistics. The contents of these
lines
are described in tables 4-7 and 4-8 of the MET User's Guide:
http://www.dtcenter.org/met/users/docs/users_guide/MET_Users_Guide_v5.0.pdf
Let's suppose that you're evaluating precipitation in millimeters, and
you
define a single threshold in the "cat_thresh" configuration entry:
cat_thresh = [ >25.4 ];
MET uses this single threshold to define a 2x2 contingency table (CTC)
and
corresponding statistics (CTS).
Now let's suppose you add a second threshold, as shown below:
cat_thresh = [ >25.4, >50.8 ];
You see 2 CTC lines and 2 CTS lines in the output, 1 for each
threshold
you've chosen.
But if you have MCTC and MCTS output turned on in the config file,
you'll
also see 1 MCTC and 1 MCTS output line. MET has used these 2
thresholds to
define a 3 x 3 contingency table. Since the thresholds are both of
the
same type (strictly-greater-than), they partition the space. So each
forecast and observed value can be placed into 1 of the 3 categories.
If
you pass in thresholds of different types, e.g. mixing > with <=, they
don't partition the space, and MET will not compute MCTC/MCTS output.
You MCTC and MCTS output files are empty, probably because you
either...
(1) did not specify at least two thresholds
(2) or you mixed threshold types, e.g. > and <=.
While multi-category contingency tables are nice, there are only 4
statistics that can be computed from them: accuracy, hanssen and
kuipers
discriminant, heidke skill score, and the gerrity score, as you'll see
in
table 4-8 of the user's guide.
Hope that helps.
John
On Wed, Dec 17, 2014 at 9:03 AM, Geeta Geeta via RT
<met_help at ucar.edu>
wrote:
>
>
> Wed Dec 17 09:03:37 2014: Request 70133 was acted upon.
> Transaction: Ticket created by geeta124 at hotmail.com
> Queue: met_help
> Subject: Query Reg MCTC/MCTS
> Owner: Nobody
> Requestors: geeta124 at hotmail.com
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=70133 >
>
>
> hi John.
> this is again reg the MCTC/MCTS output. I want to know how can I get
the
> O/p. as of now I m just getting header thing in my o/p files.
>
> I mean I want to know the procedure and the concept behind this.
>
> Regards
> Geeta
>
>
------------------------------------------------
Subject: Query Reg MCTC/MCTS
From: Geeta Geeta
Time: Thu Dec 18 08:57:28 2014
thanks John.
I have one small query reg this.
If I select two threholds, say >25mm and >35mm, So how will I get 3
Contingency Table???.
Say For example I have following datasets.
OBS FCST
5 50
10 20
1 3
3 2
7 10 etc.
100 20
150 67
in this case, when I choose, > 25mm as my threshold,
then A= 1, B=1, C=1 and D=4
the same CTC will be for > 35mm threshold too. .
Kindly explain how will I get 3x3 contingency TABLE.
Geeta
> Subject: Re: [rt.rap.ucar.edu #70133] Query Reg MCTC/MCTS
> From: met_help at ucar.edu
> To: geeta124 at hotmail.com
> Date: Wed, 17 Dec 2014 10:08:03 -0700
>
> Geeta,
>
> MCTC stands for multi-category contingency table counts and MCTS
stands for
> multi-category contingency table statistics. The contents of these
lines
> are described in tables 4-7 and 4-8 of the MET User's Guide:
>
>
http://www.dtcenter.org/met/users/docs/users_guide/MET_Users_Guide_v5.0.pdf
>
> Let's suppose that you're evaluating precipitation in millimeters,
and you
> define a single threshold in the "cat_thresh" configuration entry:
> cat_thresh = [ >25.4 ];
> MET uses this single threshold to define a 2x2 contingency table
(CTC) and
> corresponding statistics (CTS).
>
> Now let's suppose you add a second threshold, as shown below:
> cat_thresh = [ >25.4, >50.8 ];
> You see 2 CTC lines and 2 CTS lines in the output, 1 for each
threshold
> you've chosen.
>
> But if you have MCTC and MCTS output turned on in the config file,
you'll
> also see 1 MCTC and 1 MCTS output line. MET has used these 2
thresholds to
> define a 3 x 3 contingency table. Since the thresholds are both of
the
> same type (strictly-greater-than), they partition the space. So
each
> forecast and observed value can be placed into 1 of the 3
categories. If
> you pass in thresholds of different types, e.g. mixing > with <=,
they
> don't partition the space, and MET will not compute MCTC/MCTS
output.
>
> You MCTC and MCTS output files are empty, probably because you
either...
> (1) did not specify at least two thresholds
> (2) or you mixed threshold types, e.g. > and <=.
>
> While multi-category contingency tables are nice, there are only 4
> statistics that can be computed from them: accuracy, hanssen and
kuipers
> discriminant, heidke skill score, and the gerrity score, as you'll
see in
> table 4-8 of the user's guide.
>
> Hope that helps.
>
> John
>
> On Wed, Dec 17, 2014 at 9:03 AM, Geeta Geeta via RT
<met_help at ucar.edu>
> wrote:
> >
> >
> > Wed Dec 17 09:03:37 2014: Request 70133 was acted upon.
> > Transaction: Ticket created by geeta124 at hotmail.com
> > Queue: met_help
> > Subject: Query Reg MCTC/MCTS
> > Owner: Nobody
> > Requestors: geeta124 at hotmail.com
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=70133 >
> >
> >
> > hi John.
> > this is again reg the MCTC/MCTS output. I want to know how can I
get the
> > O/p. as of now I m just getting header thing in my o/p files.
> >
> > I mean I want to know the procedure and the concept behind this.
> >
> > Regards
> > Geeta
> >
> >
>
------------------------------------------------
Subject: Query Reg MCTC/MCTS
From: John Halley Gotway
Time: Thu Dec 18 09:37:43 2014
Geeta,
Let's use the following two thresholds as an example: <25 mm and <35
mm
They are used to define 3 possible bins:
(1) Less than 25mm
(2) Between 25mm and 35mm
(3) Greater than 35mm
I modified your list of pair values a little bit, and after each value
I
indicated what bin it would fall in: (1), (2), or (3):
OBS FCST
5 (1) 50 (3)
30 (2) 20 (1)
1 (1) 3 (1)
3 (1) 2 (1)
7 (1) 10 (1)
100 (3) 30 (2)
150 (3) 67 (3)
And then you'd put these pairs into a 3x3 by just counting up the
number of
cases for each cell. For example, cell(1,1) would have 3 in it, and
cell(3,3) would have 1 in it.
Make sense?
Thanks,
John
On Thu, Dec 18, 2014 at 8:57 AM, Geeta Geeta via RT
<met_help at ucar.edu>
wrote:
>
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=70133 >
>
> thanks John.
> I have one small query reg this.
> If I select two threholds, say >25mm and >35mm, So how will I get 3
> Contingency Table???.
> Say For example I have following datasets.
> OBS FCST
> 5 50
> 10 20
> 1 3
> 3 2
> 7 10 etc.
> 100 20
> 150 67
> in this case, when I choose, > 25mm as my threshold,
> then A= 1, B=1, C=1 and D=4
> the same CTC will be for > 35mm threshold too. .
>
> Kindly explain how will I get 3x3 contingency TABLE.
>
> Geeta
>
> > Subject: Re: [rt.rap.ucar.edu #70133] Query Reg MCTC/MCTS
> > From: met_help at ucar.edu
> > To: geeta124 at hotmail.com
> > Date: Wed, 17 Dec 2014 10:08:03 -0700
> >
> > Geeta,
> >
> > MCTC stands for multi-category contingency table counts and MCTS
stands
> for
> > multi-category contingency table statistics. The contents of
these lines
> > are described in tables 4-7 and 4-8 of the MET User's Guide:
> >
> >
>
http://www.dtcenter.org/met/users/docs/users_guide/MET_Users_Guide_v5.0.pdf
> >
> > Let's suppose that you're evaluating precipitation in millimeters,
and
> you
> > define a single threshold in the "cat_thresh" configuration entry:
> > cat_thresh = [ >25.4 ];
> > MET uses this single threshold to define a 2x2 contingency table
(CTC)
> and
> > corresponding statistics (CTS).
> >
> > Now let's suppose you add a second threshold, as shown below:
> > cat_thresh = [ >25.4, >50.8 ];
> > You see 2 CTC lines and 2 CTS lines in the output, 1 for each
threshold
> > you've chosen.
> >
> > But if you have MCTC and MCTS output turned on in the config file,
you'll
> > also see 1 MCTC and 1 MCTS output line. MET has used these 2
thresholds
> to
> > define a 3 x 3 contingency table. Since the thresholds are both
of the
> > same type (strictly-greater-than), they partition the space. So
each
> > forecast and observed value can be placed into 1 of the 3
categories. If
> > you pass in thresholds of different types, e.g. mixing > with <=,
they
> > don't partition the space, and MET will not compute MCTC/MCTS
output.
> >
> > You MCTC and MCTS output files are empty, probably because you
either...
> > (1) did not specify at least two thresholds
> > (2) or you mixed threshold types, e.g. > and <=.
> >
> > While multi-category contingency tables are nice, there are only 4
> > statistics that can be computed from them: accuracy, hanssen and
kuipers
> > discriminant, heidke skill score, and the gerrity score, as you'll
see in
> > table 4-8 of the user's guide.
> >
> > Hope that helps.
> >
> > John
> >
> > On Wed, Dec 17, 2014 at 9:03 AM, Geeta Geeta via RT
<met_help at ucar.edu>
> > wrote:
> > >
> > >
> > > Wed Dec 17 09:03:37 2014: Request 70133 was acted upon.
> > > Transaction: Ticket created by geeta124 at hotmail.com
> > > Queue: met_help
> > > Subject: Query Reg MCTC/MCTS
> > > Owner: Nobody
> > > Requestors: geeta124 at hotmail.com
> > > Status: new
> > > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=70133
> >
> > >
> > >
> > > hi John.
> > > this is again reg the MCTC/MCTS output. I want to know how can I
get
> the
> > > O/p. as of now I m just getting header thing in my o/p files.
> > >
> > > I mean I want to know the procedure and the concept behind this.
> > >
> > > Regards
> > > Geeta
> > >
> > >
> >
>
>
------------------------------------------------
Subject: Query Reg MCTC/MCTS
From: Geeta Geeta
Time: Thu Dec 18 10:17:39 2014
thanks John.
So if there are 4 categories, then I will have how many contingency
Tables???
Suppose I have 4 thresholds,
>25, >35,>45 and >55. then, the bins will be
1. >25
2. B/w 25 and 35
3. >35
4. B/w 35 n 45
5. >45
6. B/w 45 n 55.
7. >55
so in this case , there will be Overlapping???
Geeta
> Subject: Re: [rt.rap.ucar.edu #70133] Query Reg MCTC/MCTS
> From: met_help at ucar.edu
> To: geeta124 at hotmail.com
> Date: Thu, 18 Dec 2014 09:37:44 -0700
>
> Geeta,
>
> Let's use the following two thresholds as an example: <25 mm and <35
mm
> They are used to define 3 possible bins:
> (1) Less than 25mm
> (2) Between 25mm and 35mm
> (3) Greater than 35mm
>
> I modified your list of pair values a little bit, and after each
value I
> indicated what bin it would fall in: (1), (2), or (3):
> OBS FCST
> 5 (1) 50 (3)
> 30 (2) 20 (1)
> 1 (1) 3 (1)
> 3 (1) 2 (1)
> 7 (1) 10 (1)
> 100 (3) 30 (2)
> 150 (3) 67 (3)
>
> And then you'd put these pairs into a 3x3 by just counting up the
number of
> cases for each cell. For example, cell(1,1) would have 3 in it, and
> cell(3,3) would have 1 in it.
>
> Make sense?
>
> Thanks,
> John
>
> On Thu, Dec 18, 2014 at 8:57 AM, Geeta Geeta via RT
<met_help at ucar.edu>
> wrote:
> >
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=70133 >
> >
> > thanks John.
> > I have one small query reg this.
> > If I select two threholds, say >25mm and >35mm, So how will I get
3
> > Contingency Table???.
> > Say For example I have following datasets.
> > OBS FCST
> > 5 50
> > 10 20
> > 1 3
> > 3 2
> > 7 10 etc.
> > 100 20
> > 150 67
> > in this case, when I choose, > 25mm as my threshold,
> > then A= 1, B=1, C=1 and D=4
> > the same CTC will be for > 35mm threshold too. .
> >
> > Kindly explain how will I get 3x3 contingency TABLE.
> >
> > Geeta
> >
> > > Subject: Re: [rt.rap.ucar.edu #70133] Query Reg MCTC/MCTS
> > > From: met_help at ucar.edu
> > > To: geeta124 at hotmail.com
> > > Date: Wed, 17 Dec 2014 10:08:03 -0700
> > >
> > > Geeta,
> > >
> > > MCTC stands for multi-category contingency table counts and MCTS
stands
> > for
> > > multi-category contingency table statistics. The contents of
these lines
> > > are described in tables 4-7 and 4-8 of the MET User's Guide:
> > >
> > >
> >
http://www.dtcenter.org/met/users/docs/users_guide/MET_Users_Guide_v5.0.pdf
> > >
> > > Let's suppose that you're evaluating precipitation in
millimeters, and
> > you
> > > define a single threshold in the "cat_thresh" configuration
entry:
> > > cat_thresh = [ >25.4 ];
> > > MET uses this single threshold to define a 2x2 contingency table
(CTC)
> > and
> > > corresponding statistics (CTS).
> > >
> > > Now let's suppose you add a second threshold, as shown below:
> > > cat_thresh = [ >25.4, >50.8 ];
> > > You see 2 CTC lines and 2 CTS lines in the output, 1 for each
threshold
> > > you've chosen.
> > >
> > > But if you have MCTC and MCTS output turned on in the config
file, you'll
> > > also see 1 MCTC and 1 MCTS output line. MET has used these 2
thresholds
> > to
> > > define a 3 x 3 contingency table. Since the thresholds are both
of the
> > > same type (strictly-greater-than), they partition the space. So
each
> > > forecast and observed value can be placed into 1 of the 3
categories. If
> > > you pass in thresholds of different types, e.g. mixing > with
<=, they
> > > don't partition the space, and MET will not compute MCTC/MCTS
output.
> > >
> > > You MCTC and MCTS output files are empty, probably because you
either...
> > > (1) did not specify at least two thresholds
> > > (2) or you mixed threshold types, e.g. > and <=.
> > >
> > > While multi-category contingency tables are nice, there are only
4
> > > statistics that can be computed from them: accuracy, hanssen and
kuipers
> > > discriminant, heidke skill score, and the gerrity score, as
you'll see in
> > > table 4-8 of the user's guide.
> > >
> > > Hope that helps.
> > >
> > > John
> > >
> > > On Wed, Dec 17, 2014 at 9:03 AM, Geeta Geeta via RT
<met_help at ucar.edu>
> > > wrote:
> > > >
> > > >
> > > > Wed Dec 17 09:03:37 2014: Request 70133 was acted upon.
> > > > Transaction: Ticket created by geeta124 at hotmail.com
> > > > Queue: met_help
> > > > Subject: Query Reg MCTC/MCTS
> > > > Owner: Nobody
> > > > Requestors: geeta124 at hotmail.com
> > > > Status: new
> > > > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=70133
> > >
> > > >
> > > >
> > > > hi John.
> > > > this is again reg the MCTC/MCTS output. I want to know how can
I get
> > the
> > > > O/p. as of now I m just getting header thing in my o/p files.
> > > >
> > > > I mean I want to know the procedure and the concept behind
this.
> > > >
> > > > Regards
> > > > Geeta
> > > >
> > > >
> > >
> >
> >
>
------------------------------------------------
Subject: Query Reg MCTC/MCTS
From: John Halley Gotway
Time: Thu Dec 18 10:47:15 2014
Geeta,
Yes, 4 thresholds will result in a 5 x 5 multi-category contingency
table
being defined.
The MET tools check the thresholds in the order you define them and
put the
value into the first category that matches. For MCTC and MCTS output,
it
also requires that the thresholds be monotonically increasing.
Therefore,
I'd suggest using:
<25, <35, <45, and <55
For >25, >35, >45, and >55, a value of 100 would end up in the first
bin
since it meets that threshold criteria.
Since we're requiring the threshold values to be increasing, perhaps
we
should further restrict it to only allow for < and <= type thresholds.
Thanks,
John
On Thu, Dec 18, 2014 at 10:17 AM, Geeta Geeta via RT
<met_help at ucar.edu>
wrote:
>
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=70133 >
>
> thanks John.
> So if there are 4 categories, then I will have how many contingency
> Tables???
> Suppose I have 4 thresholds,
> >25, >35,>45 and >55. then, the bins will be
> 1. >25
> 2. B/w 25 and 35
> 3. >35
> 4. B/w 35 n 45
> 5. >45
> 6. B/w 45 n 55.
> 7. >55
> so in this case , there will be Overlapping???
>
> Geeta
>
>
>
>
>
> > Subject: Re: [rt.rap.ucar.edu #70133] Query Reg MCTC/MCTS
> > From: met_help at ucar.edu
> > To: geeta124 at hotmail.com
> > Date: Thu, 18 Dec 2014 09:37:44 -0700
> >
> > Geeta,
> >
> > Let's use the following two thresholds as an example: <25 mm and
<35 mm
> > They are used to define 3 possible bins:
> > (1) Less than 25mm
> > (2) Between 25mm and 35mm
> > (3) Greater than 35mm
> >
> > I modified your list of pair values a little bit, and after each
value I
> > indicated what bin it would fall in: (1), (2), or (3):
> > OBS FCST
> > 5 (1) 50 (3)
> > 30 (2) 20 (1)
> > 1 (1) 3 (1)
> > 3 (1) 2 (1)
> > 7 (1) 10 (1)
> > 100 (3) 30 (2)
> > 150 (3) 67 (3)
> >
> > And then you'd put these pairs into a 3x3 by just counting up the
number
> of
> > cases for each cell. For example, cell(1,1) would have 3 in it,
and
> > cell(3,3) would have 1 in it.
> >
> > Make sense?
> >
> > Thanks,
> > John
> >
> > On Thu, Dec 18, 2014 at 8:57 AM, Geeta Geeta via RT
<met_help at ucar.edu>
> > wrote:
> > >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=70133 >
> > >
> > > thanks John.
> > > I have one small query reg this.
> > > If I select two threholds, say >25mm and >35mm, So how will I
get 3
> > > Contingency Table???.
> > > Say For example I have following datasets.
> > > OBS FCST
> > > 5 50
> > > 10 20
> > > 1 3
> > > 3 2
> > > 7 10 etc.
> > > 100 20
> > > 150 67
> > > in this case, when I choose, > 25mm as my threshold,
> > > then A= 1, B=1, C=1 and D=4
> > > the same CTC will be for > 35mm threshold too. .
> > >
> > > Kindly explain how will I get 3x3 contingency TABLE.
> > >
> > > Geeta
> > >
> > > > Subject: Re: [rt.rap.ucar.edu #70133] Query Reg MCTC/MCTS
> > > > From: met_help at ucar.edu
> > > > To: geeta124 at hotmail.com
> > > > Date: Wed, 17 Dec 2014 10:08:03 -0700
> > > >
> > > > Geeta,
> > > >
> > > > MCTC stands for multi-category contingency table counts and
MCTS
> stands
> > > for
> > > > multi-category contingency table statistics. The contents of
these
> lines
> > > > are described in tables 4-7 and 4-8 of the MET User's Guide:
> > > >
> > > >
> > >
>
http://www.dtcenter.org/met/users/docs/users_guide/MET_Users_Guide_v5.0.pdf
> > > >
> > > > Let's suppose that you're evaluating precipitation in
millimeters,
> and
> > > you
> > > > define a single threshold in the "cat_thresh" configuration
entry:
> > > > cat_thresh = [ >25.4 ];
> > > > MET uses this single threshold to define a 2x2 contingency
table
> (CTC)
> > > and
> > > > corresponding statistics (CTS).
> > > >
> > > > Now let's suppose you add a second threshold, as shown below:
> > > > cat_thresh = [ >25.4, >50.8 ];
> > > > You see 2 CTC lines and 2 CTS lines in the output, 1 for each
> threshold
> > > > you've chosen.
> > > >
> > > > But if you have MCTC and MCTS output turned on in the config
file,
> you'll
> > > > also see 1 MCTC and 1 MCTS output line. MET has used these 2
> thresholds
> > > to
> > > > define a 3 x 3 contingency table. Since the thresholds are
both of
> the
> > > > same type (strictly-greater-than), they partition the space.
So each
> > > > forecast and observed value can be placed into 1 of the 3
> categories. If
> > > > you pass in thresholds of different types, e.g. mixing > with
<=,
> they
> > > > don't partition the space, and MET will not compute MCTC/MCTS
output.
> > > >
> > > > You MCTC and MCTS output files are empty, probably because you
> either...
> > > > (1) did not specify at least two thresholds
> > > > (2) or you mixed threshold types, e.g. > and <=.
> > > >
> > > > While multi-category contingency tables are nice, there are
only 4
> > > > statistics that can be computed from them: accuracy, hanssen
and
> kuipers
> > > > discriminant, heidke skill score, and the gerrity score, as
you'll
> see in
> > > > table 4-8 of the user's guide.
> > > >
> > > > Hope that helps.
> > > >
> > > > John
> > > >
> > > > On Wed, Dec 17, 2014 at 9:03 AM, Geeta Geeta via RT <
> met_help at ucar.edu>
> > > > wrote:
> > > > >
> > > > >
> > > > > Wed Dec 17 09:03:37 2014: Request 70133 was acted upon.
> > > > > Transaction: Ticket created by geeta124 at hotmail.com
> > > > > Queue: met_help
> > > > > Subject: Query Reg MCTC/MCTS
> > > > > Owner: Nobody
> > > > > Requestors: geeta124 at hotmail.com
> > > > > Status: new
> > > > > Ticket <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=70133
> > > >
> > > > >
> > > > >
> > > > > hi John.
> > > > > this is again reg the MCTC/MCTS output. I want to know how
can I
> get
> > > the
> > > > > O/p. as of now I m just getting header thing in my o/p
files.
> > > > >
> > > > > I mean I want to know the procedure and the concept behind
this.
> > > > >
> > > > > Regards
> > > > > Geeta
> > > > >
> > > > >
> > > >
> > >
> > >
> >
>
>
------------------------------------------------
Subject: Query Reg MCTC/MCTS
From: John Halley Gotway
Time: Thu Dec 18 12:01:40 2014
Geeta,
I checked the code more closely and see that we're doing the "right
thing"
for multi-category contingency tables. Here's an example of running
two
jobs through STAT-Analysis using the output of the MET test scripts:
> cd met-5.0
> bin/stat_analysis -lookin
out/point_stat/point_stat_360000L_20070331_120000V.stat \
-job aggregate_stat -line_type MPR -out_line_type MCTC -fcst_var
VGRD
-fcst_lev Z10 -obtype ADPSFC -vx_mask LMV -interp_mthd DW_MEAN \
-out_fcst_thresh le3 -out_fcst_thresh le4 -out_obs_thresh le3
-out_obs_thresh le4
> bin/stat_analysis -lookin
out/point_stat/point_stat_360000L_20070331_120000V.stat \
-job aggregate_stat -line_type MPR -out_line_type MCTC -fcst_var
VGRD
-fcst_lev Z10 -obtype ADPSFC -vx_mask LMV -interp_mthd DW_MEAN \
-out_fcst_thresh gt3 -out_fcst_thresh gt4 -out_obs_thresh gt3
-out_obs_thresh gt4
The first job uses <=3 and <=4 while the second uses >3 and >4. Both
result in the same 3x3 contingency table:
COL_NAME: TOTAL N_CAT F1_O1 F1_O2 F1_O3 F2_O1 F2_O2 F2_O3 F3_O1 F3_O2
F3_O3
MCTC: 364 3 54 2 1
27 7 4 127 62 80
Note, I did find a bug in the writing of the MCTC header column. I
fixed
that in the development version of MET, and it'll be included in the
next
release.
When checking thresholds, for thresholds of type < or <=, we check the
thresholds from left to right. For types > or >=, we check thresholds
right to left. That results in the same contingency table counts.
Thanks,
John
On Thu, Dec 18, 2014 at 10:46 AM, John Halley Gotway <johnhg at ucar.edu>
wrote:
>
> Geeta,
>
> Yes, 4 thresholds will result in a 5 x 5 multi-category contingency
table
> being defined.
>
> The MET tools check the thresholds in the order you define them and
put
> the value into the first category that matches. For MCTC and MCTS
output,
> it also requires that the thresholds be monotonically increasing.
> Therefore, I'd suggest using:
> <25, <35, <45, and <55
>
> For >25, >35, >45, and >55, a value of 100 would end up in the first
bin
> since it meets that threshold criteria.
>
> Since we're requiring the threshold values to be increasing, perhaps
we
> should further restrict it to only allow for < and <= type
thresholds.
>
> Thanks,
> John
>
>
> On Thu, Dec 18, 2014 at 10:17 AM, Geeta Geeta via RT
<met_help at ucar.edu>
> wrote:
>>
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=70133 >
>>
>> thanks John.
>> So if there are 4 categories, then I will have how many contingency
>> Tables???
>> Suppose I have 4 thresholds,
>> >25, >35,>45 and >55. then, the bins will be
>> 1. >25
>> 2. B/w 25 and 35
>> 3. >35
>> 4. B/w 35 n 45
>> 5. >45
>> 6. B/w 45 n 55.
>> 7. >55
>> so in this case , there will be Overlapping???
>>
>> Geeta
>>
>>
>>
>>
>>
>> > Subject: Re: [rt.rap.ucar.edu #70133] Query Reg MCTC/MCTS
>> > From: met_help at ucar.edu
>> > To: geeta124 at hotmail.com
>> > Date: Thu, 18 Dec 2014 09:37:44 -0700
>> >
>> > Geeta,
>> >
>> > Let's use the following two thresholds as an example: <25 mm and
<35 mm
>> > They are used to define 3 possible bins:
>> > (1) Less than 25mm
>> > (2) Between 25mm and 35mm
>> > (3) Greater than 35mm
>> >
>> > I modified your list of pair values a little bit, and after each
value I
>> > indicated what bin it would fall in: (1), (2), or (3):
>> > OBS FCST
>> > 5 (1) 50 (3)
>> > 30 (2) 20 (1)
>> > 1 (1) 3 (1)
>> > 3 (1) 2 (1)
>> > 7 (1) 10 (1)
>> > 100 (3) 30 (2)
>> > 150 (3) 67 (3)
>> >
>> > And then you'd put these pairs into a 3x3 by just counting up the
>> number of
>> > cases for each cell. For example, cell(1,1) would have 3 in it,
and
>> > cell(3,3) would have 1 in it.
>> >
>> > Make sense?
>> >
>> > Thanks,
>> > John
>> >
>> > On Thu, Dec 18, 2014 at 8:57 AM, Geeta Geeta via RT
<met_help at ucar.edu>
>> > wrote:
>> > >
>> > >
>> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=70133 >
>> > >
>> > > thanks John.
>> > > I have one small query reg this.
>> > > If I select two threholds, say >25mm and >35mm, So how will I
get 3
>> > > Contingency Table???.
>> > > Say For example I have following datasets.
>> > > OBS FCST
>> > > 5 50
>> > > 10 20
>> > > 1 3
>> > > 3 2
>> > > 7 10 etc.
>> > > 100 20
>> > > 150 67
>> > > in this case, when I choose, > 25mm as my threshold,
>> > > then A= 1, B=1, C=1 and D=4
>> > > the same CTC will be for > 35mm threshold too. .
>> > >
>> > > Kindly explain how will I get 3x3 contingency TABLE.
>> > >
>> > > Geeta
>> > >
>> > > > Subject: Re: [rt.rap.ucar.edu #70133] Query Reg MCTC/MCTS
>> > > > From: met_help at ucar.edu
>> > > > To: geeta124 at hotmail.com
>> > > > Date: Wed, 17 Dec 2014 10:08:03 -0700
>> > > >
>> > > > Geeta,
>> > > >
>> > > > MCTC stands for multi-category contingency table counts and
MCTS
>> stands
>> > > for
>> > > > multi-category contingency table statistics. The contents of
these
>> lines
>> > > > are described in tables 4-7 and 4-8 of the MET User's Guide:
>> > > >
>> > > >
>> > >
>>
http://www.dtcenter.org/met/users/docs/users_guide/MET_Users_Guide_v5.0.pdf
>> > > >
>> > > > Let's suppose that you're evaluating precipitation in
millimeters,
>> and
>> > > you
>> > > > define a single threshold in the "cat_thresh" configuration
entry:
>> > > > cat_thresh = [ >25.4 ];
>> > > > MET uses this single threshold to define a 2x2 contingency
table
>> (CTC)
>> > > and
>> > > > corresponding statistics (CTS).
>> > > >
>> > > > Now let's suppose you add a second threshold, as shown below:
>> > > > cat_thresh = [ >25.4, >50.8 ];
>> > > > You see 2 CTC lines and 2 CTS lines in the output, 1 for each
>> threshold
>> > > > you've chosen.
>> > > >
>> > > > But if you have MCTC and MCTS output turned on in the config
file,
>> you'll
>> > > > also see 1 MCTC and 1 MCTS output line. MET has used these 2
>> thresholds
>> > > to
>> > > > define a 3 x 3 contingency table. Since the thresholds are
both of
>> the
>> > > > same type (strictly-greater-than), they partition the space.
So
>> each
>> > > > forecast and observed value can be placed into 1 of the 3
>> categories. If
>> > > > you pass in thresholds of different types, e.g. mixing > with
<=,
>> they
>> > > > don't partition the space, and MET will not compute MCTC/MCTS
>> output.
>> > > >
>> > > > You MCTC and MCTS output files are empty, probably because
you
>> either...
>> > > > (1) did not specify at least two thresholds
>> > > > (2) or you mixed threshold types, e.g. > and <=.
>> > > >
>> > > > While multi-category contingency tables are nice, there are
only 4
>> > > > statistics that can be computed from them: accuracy, hanssen
and
>> kuipers
>> > > > discriminant, heidke skill score, and the gerrity score, as
you'll
>> see in
>> > > > table 4-8 of the user's guide.
>> > > >
>> > > > Hope that helps.
>> > > >
>> > > > John
>> > > >
>> > > > On Wed, Dec 17, 2014 at 9:03 AM, Geeta Geeta via RT <
>> met_help at ucar.edu>
>> > > > wrote:
>> > > > >
>> > > > >
>> > > > > Wed Dec 17 09:03:37 2014: Request 70133 was acted upon.
>> > > > > Transaction: Ticket created by geeta124 at hotmail.com
>> > > > > Queue: met_help
>> > > > > Subject: Query Reg MCTC/MCTS
>> > > > > Owner: Nobody
>> > > > > Requestors: geeta124 at hotmail.com
>> > > > > Status: new
>> > > > > Ticket <URL:
>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=70133
>> > > >
>> > > > >
>> > > > >
>> > > > > hi John.
>> > > > > this is again reg the MCTC/MCTS output. I want to know how
can I
>> get
>> > > the
>> > > > > O/p. as of now I m just getting header thing in my o/p
files.
>> > > > >
>> > > > > I mean I want to know the procedure and the concept behind
this.
>> > > > >
>> > > > > Regards
>> > > > > Geeta
>> > > > >
>> > > > >
>> > > >
>> > >
>> > >
>> >
>>
>>
------------------------------------------------
More information about the Met_help
mailing list