[Met_help] [rt.rap.ucar.edu #43929] History for Mode Analysis Centroid Distance

RAL HelpDesk {for John Halley Gotway} met_help at ucar.edu
Tue Feb 1 10:38:42 MST 2011


----------------------------------------------------------------
  Initial Request
----------------------------------------------------------------

John,

I'm working on Dr. Bill Gallus's project and having the issue with getting
centroid distance statistics.  I tried using all three flags (-matched -pair
-cluster) and it doesn't "keep" any lines.  However, when I run it with the
same three except -unmatched I get the same results as if I ran it without a
-matched/unmatched flag.  This seems to imply that there are no matched
pairs in our dataset, however, upon looking at the postscript files I know
for a fact that object pairs are matching above the .7 threshold we have
set.

Thanks for your help!
Casey Zoellick


----------------------------------------------------------------
  Complete Ticket History
----------------------------------------------------------------

Subject: Re: [rt.rap.ucar.edu #43929] Mode Analysis Centroid Distance
From: John Halley Gotway
Time: Sun Jan 30 20:51:43 2011

Casey,

Sorry you guys are having so much trouble with this.  Please send me
some
MODE output files you're trying to run through the MODE-Analysis tool
-
along with the exact command line you're using to run it.

You could just create a tar file with a bunch of the MODE output files
(*_obj.txt).  I'll run it through MODE-Analysis here and try to figure
out
what's going on.  You can post those files to our anonymous ftp site
using
the following instructions:
   http://www.dtcenter.org/met/users/support/met_help.php#ftp

Also, please remind me what version of MET you're running, 2.0 or 3.0.
I'd always recommend updating to the latest set of patches which are
available here:
   http://www.dtcenter.org/met/users/support/known_issues/index.php
Just click on the version you're using.

Lastly, I want to point out an alternative to the MODE-Analysis tool
that
you may (or may not) find helpful.  We have a sample R script posted
on
the website to parses a bunch of MODE info and dumps out summary stats
about it:
   http://www.dtcenter.org/met/users/downloads/analysis_scripts.php
Look for the file "mode_summary.R".  If you're familiar with R, you
may
find it easy to use and modify this script.  To run it, you'd type:
   Rscript mode_summary.R mode1_obj.txt mode2_obj.txt ...
moden_obj.txt,
where those "mode*_obj.txt" are the MODE output files from individual
runs.

Please write me back when you have posted that data to our website.
I'll
grab it and take a look.

Thanks,
John Halley Gotway




>
> Fri Jan 28 12:33:38 2011: Request 43929 was acted upon.
> Transaction: Ticket created by czoellic at iastate.edu
>        Queue: met_help
>      Subject: Mode Analysis Centroid Distance
>        Owner: Nobody
>   Requestors: czoellic at iastate.edu
>       Status: new
>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=43929 >
>
>
> John,
>
> I'm working on Dr. Bill Gallus's project and having the issue with
getting
> centroid distance statistics.  I tried using all three flags (-
matched
> -pair
> -cluster) and it doesn't "keep" any lines.  However, when I run it
with
> the
> same three except -unmatched I get the same results as if I ran it
without
> a
> -matched/unmatched flag.  This seems to imply that there are no
matched
> pairs in our dataset, however, upon looking at the postscript files
I know
> for a fact that object pairs are matching above the .7 threshold we
have
> set.
>
> Thanks for your help!
> Casey Zoellick
>



------------------------------------------------
Subject: Mode Analysis Centroid Distance
From: Casey Zoellick
Time: Mon Jan 31 13:07:55 2011

John,

I put the files on your site.  We're using METv3.0.  The command I'm
using
is mode_analysis -summary -lookin . -pair -matched -cluster -column
CENTROID_DIST

Thanks,
Casey


On Sun, Jan 30, 2011 at 9:51 PM, RAL HelpDesk {for John Halley Gotway}
<
met_help at ucar.edu> wrote:

> Casey,
>
> Sorry you guys are having so much trouble with this.  Please send me
some
> MODE output files you're trying to run through the MODE-Analysis
tool -
> along with the exact command line you're using to run it.
>
> You could just create a tar file with a bunch of the MODE output
files
> (*_obj.txt).  I'll run it through MODE-Analysis here and try to
figure out
> what's going on.  You can post those files to our anonymous ftp site
using
> the following instructions:
>   http://www.dtcenter.org/met/users/support/met_help.php#ftp
>
> Also, please remind me what version of MET you're running, 2.0 or
3.0.
> I'd always recommend updating to the latest set of patches which are
> available here:
>   http://www.dtcenter.org/met/users/support/known_issues/index.php
> Just click on the version you're using.
>
> Lastly, I want to point out an alternative to the MODE-Analysis tool
that
> you may (or may not) find helpful.  We have a sample R script posted
on
> the website to parses a bunch of MODE info and dumps out summary
stats
> about it:
>   http://www.dtcenter.org/met/users/downloads/analysis_scripts.php
> Look for the file "mode_summary.R".  If you're familiar with R, you
may
> find it easy to use and modify this script.  To run it, you'd type:
>   Rscript mode_summary.R mode1_obj.txt mode2_obj.txt ...
moden_obj.txt,
> where those "mode*_obj.txt" are the MODE output files from
individual runs.
>
> Please write me back when you have posted that data to our website.
I'll
> grab it and take a look.
>
> Thanks,
> John Halley Gotway
>
>
>
>
> >
> > Fri Jan 28 12:33:38 2011: Request 43929 was acted upon.
> > Transaction: Ticket created by czoellic at iastate.edu
> >        Queue: met_help
> >      Subject: Mode Analysis Centroid Distance
> >        Owner: Nobody
> >   Requestors: czoellic at iastate.edu
> >       Status: new
> >  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=43929 >
> >
> >
> > John,
> >
> > I'm working on Dr. Bill Gallus's project and having the issue with
> getting
> > centroid distance statistics.  I tried using all three flags (-
matched
> > -pair
> > -cluster) and it doesn't "keep" any lines.  However, when I run it
with
> > the
> > same three except -unmatched I get the same results as if I ran it
> without
> > a
> > -matched/unmatched flag.  This seems to imply that there are no
matched
> > pairs in our dataset, however, upon looking at the postscript
files I
> know
> > for a fact that object pairs are matching above the .7 threshold
we have
> > set.
> >
> > Thanks for your help!
> > Casey Zoellick
> >
>
>
>
>


--
CASEY L. ZOELLICK, C/Col, AFROTC
Wing Commander
Detachment 250
630-890-2911
czoellic at iastate.edu

------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #43929] Mode Analysis Centroid Distance
From: John Halley Gotway
Time: Mon Jan 31 13:40:27 2011

Casey,

OK, try running without the "-matched" option.  All "cluster" objects
are by definition matched.  I tried running using just the data you
sent in "mode_obj_inch.tar".  Here's the command and output I see:

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

/d1/johnhg/MET/MET_releases/METv3.0/bin/mode_analysis -lookin .
-summary -column CENTROID_DIST -cluster -pair

Total mode lines read =  1,278
Total mode lines kept =     26

        Field    N    Min     Max    Mean   StdDev    P10    P25
P50     P75     P90      Sum
-------------   --   ----   -----   -----   ------   ----   ----
-----   -----   -----   ------
centroid_dist   26   1.30   50.24   16.66    11.59   5.90   8.72
14.91   20.10   26.14   433.20

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

I did ran a combination of grep/sed/cut commands on the command line
and verified that 26 lines is the correct number of lines it should
find.  Also, I ran a second time using the "-dump_row" option:

/d1/johnhg/MET/MET_releases/METv3.0/bin/mode_analysis -lookin .
-summary -column CENTROID_DIST -cluster -pair -dump_row mode_lines.out

That created a file called "mode_lines.out" that listed out the MODE
lines the job used.  It's always a good idea to use the "-dump_row"
option when setting up a MODE-Analysis job - just to make sure
that the tool is doing what you expect.

Sorry this is so confusing, but I hope it helps.

John

On 01/31/2011 01:07 PM, RAL HelpDesk {for Casey Zoellick} wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=43929 >
>
> John,
>
> I put the files on your site.  We're using METv3.0.  The command I'm
using
> is mode_analysis -summary -lookin . -pair -matched -cluster -column
> CENTROID_DIST
>
> Thanks,
> Casey
>
>
> On Sun, Jan 30, 2011 at 9:51 PM, RAL HelpDesk {for John Halley
Gotway} <
> met_help at ucar.edu> wrote:
>
>> Casey,
>>
>> Sorry you guys are having so much trouble with this.  Please send
me some
>> MODE output files you're trying to run through the MODE-Analysis
tool -
>> along with the exact command line you're using to run it.
>>
>> You could just create a tar file with a bunch of the MODE output
files
>> (*_obj.txt).  I'll run it through MODE-Analysis here and try to
figure out
>> what's going on.  You can post those files to our anonymous ftp
site using
>> the following instructions:
>>   http://www.dtcenter.org/met/users/support/met_help.php#ftp
>>
>> Also, please remind me what version of MET you're running, 2.0 or
3.0.
>> I'd always recommend updating to the latest set of patches which
are
>> available here:
>>   http://www.dtcenter.org/met/users/support/known_issues/index.php
>> Just click on the version you're using.
>>
>> Lastly, I want to point out an alternative to the MODE-Analysis
tool that
>> you may (or may not) find helpful.  We have a sample R script
posted on
>> the website to parses a bunch of MODE info and dumps out summary
stats
>> about it:
>>   http://www.dtcenter.org/met/users/downloads/analysis_scripts.php
>> Look for the file "mode_summary.R".  If you're familiar with R, you
may
>> find it easy to use and modify this script.  To run it, you'd type:
>>   Rscript mode_summary.R mode1_obj.txt mode2_obj.txt ...
moden_obj.txt,
>> where those "mode*_obj.txt" are the MODE output files from
individual runs.
>>
>> Please write me back when you have posted that data to our website.
I'll
>> grab it and take a look.
>>
>> Thanks,
>> John Halley Gotway
>>
>>
>>
>>
>>>
>>> Fri Jan 28 12:33:38 2011: Request 43929 was acted upon.
>>> Transaction: Ticket created by czoellic at iastate.edu
>>>        Queue: met_help
>>>      Subject: Mode Analysis Centroid Distance
>>>        Owner: Nobody
>>>   Requestors: czoellic at iastate.edu
>>>       Status: new
>>>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=43929 >
>>>
>>>
>>> John,
>>>
>>> I'm working on Dr. Bill Gallus's project and having the issue with
>> getting
>>> centroid distance statistics.  I tried using all three flags (-
matched
>>> -pair
>>> -cluster) and it doesn't "keep" any lines.  However, when I run it
with
>>> the
>>> same three except -unmatched I get the same results as if I ran it
>> without
>>> a
>>> -matched/unmatched flag.  This seems to imply that there are no
matched
>>> pairs in our dataset, however, upon looking at the postscript
files I
>> know
>>> for a fact that object pairs are matching above the .7 threshold
we have
>>> set.
>>>
>>> Thanks for your help!
>>> Casey Zoellick
>>>
>>
>>
>>
>>
>
>

------------------------------------------------
Subject: Mode Analysis Centroid Distance
From: John Halley Gotway
Time: Mon Jan 31 13:59:46 2011

Casey,

I also ran the "inch" data through the "mode_summary.R" script.  Your
data actually pointed out a bug in that script when there are either 0
forecast or 0 observation objects.  So I updated it and
also enhanced it to dump out summary information about the centroid
distance.  I've attached the updated script as well as a file showing
what the output looks like for the "inch" data.  It was
actually a good way of double-checking the MODE-Analysis output to
make sure it's the same as the mode_summary.R output... fortunately,
they are :)

Here's how I ran that script:
   Rscript mode_summary.R mode*_obj.txt

Thanks,
John

On 01/31/2011 01:40 PM, RAL HelpDesk {for John Halley Gotway} wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=43929 >
>
> Casey,
>
> OK, try running without the "-matched" option.  All "cluster"
objects are by definition matched.  I tried running using just the
data you sent in "mode_obj_inch.tar".  Here's the command and output I
see:
>
> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
> /d1/johnhg/MET/MET_releases/METv3.0/bin/mode_analysis -lookin .
-summary -column CENTROID_DIST -cluster -pair
>
> Total mode lines read =  1,278
> Total mode lines kept =     26
>
>         Field    N    Min     Max    Mean   StdDev    P10    P25
P50     P75     P90      Sum
> -------------   --   ----   -----   -----   ------   ----   ----
-----   -----   -----   ------
> centroid_dist   26   1.30   50.24   16.66    11.59   5.90   8.72
14.91   20.10   26.14   433.20
>
> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>
> I did ran a combination of grep/sed/cut commands on the command line
and verified that 26 lines is the correct number of lines it should
find.  Also, I ran a second time using the "-dump_row" option:
>
> /d1/johnhg/MET/MET_releases/METv3.0/bin/mode_analysis -lookin .
-summary -column CENTROID_DIST -cluster -pair -dump_row mode_lines.out
>
> That created a file called "mode_lines.out" that listed out the MODE
lines the job used.  It's always a good idea to use the "-dump_row"
option when setting up a MODE-Analysis job - just to make sure
> that the tool is doing what you expect.
>
> Sorry this is so confusing, but I hope it helps.
>
> John
>
> On 01/31/2011 01:07 PM, RAL HelpDesk {for Casey Zoellick} wrote:
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=43929 >
>>
>> John,
>>
>> I put the files on your site.  We're using METv3.0.  The command
I'm using
>> is mode_analysis -summary -lookin . -pair -matched -cluster -column
>> CENTROID_DIST
>>
>> Thanks,
>> Casey
>>
>>
>> On Sun, Jan 30, 2011 at 9:51 PM, RAL HelpDesk {for John Halley
Gotway} <
>> met_help at ucar.edu> wrote:
>>
>>> Casey,
>>>
>>> Sorry you guys are having so much trouble with this.  Please send
me some
>>> MODE output files you're trying to run through the MODE-Analysis
tool -
>>> along with the exact command line you're using to run it.
>>>
>>> You could just create a tar file with a bunch of the MODE output
files
>>> (*_obj.txt).  I'll run it through MODE-Analysis here and try to
figure out
>>> what's going on.  You can post those files to our anonymous ftp
site using
>>> the following instructions:
>>>   http://www.dtcenter.org/met/users/support/met_help.php#ftp
>>>
>>> Also, please remind me what version of MET you're running, 2.0 or
3.0.
>>> I'd always recommend updating to the latest set of patches which
are
>>> available here:
>>>   http://www.dtcenter.org/met/users/support/known_issues/index.php
>>> Just click on the version you're using.
>>>
>>> Lastly, I want to point out an alternative to the MODE-Analysis
tool that
>>> you may (or may not) find helpful.  We have a sample R script
posted on
>>> the website to parses a bunch of MODE info and dumps out summary
stats
>>> about it:
>>>   http://www.dtcenter.org/met/users/downloads/analysis_scripts.php
>>> Look for the file "mode_summary.R".  If you're familiar with R,
you may
>>> find it easy to use and modify this script.  To run it, you'd
type:
>>>   Rscript mode_summary.R mode1_obj.txt mode2_obj.txt ...
moden_obj.txt,
>>> where those "mode*_obj.txt" are the MODE output files from
individual runs.
>>>
>>> Please write me back when you have posted that data to our
website.  I'll
>>> grab it and take a look.
>>>
>>> Thanks,
>>> John Halley Gotway
>>>
>>>
>>>
>>>
>>>>
>>>> Fri Jan 28 12:33:38 2011: Request 43929 was acted upon.
>>>> Transaction: Ticket created by czoellic at iastate.edu
>>>>        Queue: met_help
>>>>      Subject: Mode Analysis Centroid Distance
>>>>        Owner: Nobody
>>>>   Requestors: czoellic at iastate.edu
>>>>       Status: new
>>>>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=43929 >
>>>>
>>>>
>>>> John,
>>>>
>>>> I'm working on Dr. Bill Gallus's project and having the issue
with
>>> getting
>>>> centroid distance statistics.  I tried using all three flags (-
matched
>>>> -pair
>>>> -cluster) and it doesn't "keep" any lines.  However, when I run
it with
>>>> the
>>>> same three except -unmatched I get the same results as if I ran
it
>>> without
>>>> a
>>>> -matched/unmatched flag.  This seems to imply that there are no
matched
>>>> pairs in our dataset, however, upon looking at the postscript
files I
>>> know
>>>> for a fact that object pairs are matching above the .7 threshold
we have
>>>> set.
>>>>
>>>> Thanks for your help!
>>>> Casey Zoellick
>>>>
>>>
>>>
>>>
>>>
>>
>>

------------------------------------------------
Subject: Mode Analysis Centroid Distance
From: John Halley Gotway
Time: Mon Jan 31 13:59:46 2011

Read 0 lines from MODE file:
mode_300000L_20100512_060000V_060000A_obj.txt
Read 1 lines from MODE file:
mode_300000L_20100601_060000V_060000A_obj.txt
Read 3 lines from MODE file:
mode_300000L_20100605_060000V_060000A_obj.txt
Read 2 lines from MODE file:
mode_300000L_20100612_060000V_060000A_obj.txt
Read 119 lines from MODE file:
mode_300000L_20100618_060000V_060000A_obj.txt
Read 65 lines from MODE file:
mode_300000L_20100622_060000V_060000A_obj.txt
Read 30 lines from MODE file:
mode_300000L_20100623_060000V_060000A_obj.txt
Read 20 lines from MODE file:
mode_300000L_20100627_060000V_060000A_obj.txt
Read 0 lines from MODE file:
mode_300000L_20100704_060000V_060000A_obj.txt
Read 2 lines from MODE file:
mode_300000L_20100719_060000V_060000A_obj.txt
Read 14 lines from MODE file:
mode_300000L_20100722_060000V_060000A_obj.txt
Read 100 lines from MODE file:
mode_300000L_20100724_060000V_060000A_obj.txt
Read 0 lines from MODE file:
mode_300000L_20100808_060000V_060000A_obj.txt
Read 72 lines from MODE file:
mode_300000L_20100809_060000V_060000A_obj.txt
Read 39 lines from MODE file:
mode_300000L_20100810_060000V_060000A_obj.txt
Read 6 lines from MODE file:
mode_300000L_20100831_060000V_060000A_obj.txt
Read 0 lines from MODE file:
mode_360000L_20100512_120000V_060000A_obj.txt
Read 0 lines from MODE file:
mode_360000L_20100601_120000V_060000A_obj.txt
Read 16 lines from MODE file:
mode_360000L_20100605_120000V_060000A_obj.txt
Read 19 lines from MODE file:
mode_360000L_20100612_120000V_060000A_obj.txt
Read 5 lines from MODE file:
mode_360000L_20100618_120000V_060000A_obj.txt
Read 47 lines from MODE file:
mode_360000L_20100622_120000V_060000A_obj.txt
Read 41 lines from MODE file:
mode_360000L_20100623_120000V_060000A_obj.txt
Read 19 lines from MODE file:
mode_360000L_20100627_120000V_060000A_obj.txt
Read 2 lines from MODE file:
mode_360000L_20100704_120000V_060000A_obj.txt
Read 2 lines from MODE file:
mode_360000L_20100719_120000V_060000A_obj.txt
Read 10 lines from MODE file:
mode_360000L_20100722_120000V_060000A_obj.txt
Read 102 lines from MODE file:
mode_360000L_20100724_120000V_060000A_obj.txt
Read 8 lines from MODE file:
mode_360000L_20100808_120000V_060000A_obj.txt
Read 89 lines from MODE file:
mode_360000L_20100809_120000V_060000A_obj.txt
Read 10 lines from MODE file:
mode_360000L_20100810_120000V_060000A_obj.txt
Read 6 lines from MODE file:
mode_360000L_20100831_120000V_060000A_obj.txt
Read 7 lines from MODE file:
mode_420000L_20100512_180000V_060000A_obj.txt
Read 1 lines from MODE file:
mode_420000L_20100601_180000V_060000A_obj.txt
Read 5 lines from MODE file:
mode_420000L_20100605_180000V_060000A_obj.txt
Read 7 lines from MODE file:
mode_420000L_20100612_180000V_060000A_obj.txt
Read 1 lines from MODE file:
mode_420000L_20100618_180000V_060000A_obj.txt
Read 5 lines from MODE file:
mode_420000L_20100622_180000V_060000A_obj.txt
Read 14 lines from MODE file:
mode_420000L_20100623_180000V_060000A_obj.txt
Read 7 lines from MODE file:
mode_420000L_20100627_180000V_060000A_obj.txt
Read 32 lines from MODE file:
mode_420000L_20100704_180000V_060000A_obj.txt
Read 3 lines from MODE file:
mode_420000L_20100719_180000V_060000A_obj.txt
Read 14 lines from MODE file:
mode_420000L_20100722_180000V_060000A_obj.txt
Read 3 lines from MODE file:
mode_420000L_20100724_180000V_060000A_obj.txt
Read 1 lines from MODE file:
mode_420000L_20100808_180000V_060000A_obj.txt
Read 5 lines from MODE file:
mode_420000L_20100809_180000V_060000A_obj.txt
Read 9 lines from MODE file:
mode_420000L_20100810_180000V_060000A_obj.txt
Read 5 lines from MODE file:
mode_420000L_20100831_180000V_060000A_obj.txt
Read 4 lines from MODE file:
mode_480000L_20100513_000000V_060000A_obj.txt
Read 23 lines from MODE file:
mode_480000L_20100602_000000V_060000A_obj.txt
Read 3 lines from MODE file:
mode_480000L_20100606_000000V_060000A_obj.txt
Read 49 lines from MODE file:
mode_480000L_20100613_000000V_060000A_obj.txt
Read 5 lines from MODE file:
mode_480000L_20100619_000000V_060000A_obj.txt
Read 0 lines from MODE file:
mode_480000L_20100623_000000V_060000A_obj.txt
Read 26 lines from MODE file:
mode_480000L_20100624_000000V_060000A_obj.txt
Read 5 lines from MODE file:
mode_480000L_20100628_000000V_060000A_obj.txt
Read 50 lines from MODE file:
mode_480000L_20100705_000000V_060000A_obj.txt
Read 1 lines from MODE file:
mode_480000L_20100720_000000V_060000A_obj.txt
Read 14 lines from MODE file:
mode_480000L_20100723_000000V_060000A_obj.txt
Read 7 lines from MODE file:
mode_480000L_20100725_000000V_060000A_obj.txt
Read 1 lines from MODE file:
mode_480000L_20100809_000000V_060000A_obj.txt
Read 4 lines from MODE file:
mode_480000L_20100810_000000V_060000A_obj.txt
Read 29 lines from MODE file:
mode_480000L_20100811_000000V_060000A_obj.txt
Read 89 lines from MODE file:
mode_480000L_20100901_000000V_060000A_obj.txt
Total Number of Files Processed = 64
Total Number of Single Objects = 397
Number of Single Fcst Objects = 128
Number of Matched Single Fcst Objects = 26
Number of Unmatched Single Fcst Objects = 102
Number of Single Obs Objects = 269
Number of Matched Single Obs Objects = 26
Number of Unmatched Single Obs Objects = 243
Total Number of Cluster Objects = 52
Total Number of Fcst Cluster Objects = 26
Total Number of Obs Cluster Objects = 26
Total Area of Objects = 95971
Area of Single Fcst Objects = 37448
Area of Matched Single Fcst Objects = 24839
Area of Unmatched Single Fcst Objects = 12609
Area of Single Obs Objects = 58523
Area of Matched Single Obs Objects = 21697
Area of Unmatched Single Obs Objects = 36826
Median of Max Interest by Fcst Object = 0.57435
Median of Max Interest by Obs Object = 0.53199
Median of Max Interest by Fcst+Obs Objects = 0.556635
Cluster Pair Centroid Distance:
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
  1.297   8.723  14.910  16.660  20.100  50.240
Cluster Pair Centroid X-Offsets:
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
-26.400 -10.430   3.969   1.131   7.860  42.920
Cluster Pair Centroid Y-Offsets:
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
-26.120  -4.555   1.109   1.735   5.766  47.990
Cluster Pair Centroid X,Y Offsets and Distance [26]:
(-0.8987, 0.9349) 1.2968
(-5.5570, 2.7714) 6.2098
(-11.1843, -0.8654) 11.2178
(6.2597, -11.3853) 12.9926
(20.2623, 5.8179) 21.0810
(8.0000, 1.0000) 8.0623
(14.4987, 4.1437) 15.0792
(4.7882, -2.1812) 5.2616
(-10.6737, 10.1552) 14.7328
(16.0097, 1.2176) 16.0560
(42.9162, -26.1246) 50.2424
(7.0060, 1.8475) 7.2455
(-9.6863, -5.5404) 11.1589
(-22.6638, 0.4568) 22.6684
(0.9783, 18.0960) 18.1224
(4.6992, 18.1052) 18.7051
(3.2387, 11.5994) 12.0430
(1.6144, -5.3458) 5.5843
(-13.4248, 47.9927) 49.8350
(7.4386, 3.6403) 8.2816
(-21.6371, 5.6097) 22.3525
(-26.3975, -13.3997) 29.6038
(8.1428, -18.4546) 20.1712
(5.6305, 8.3191) 10.0454
(15.2442, -0.4567) 15.2510
(-15.1959, -12.8454) 19.8977

------------------------------------------------
Subject: Mode Analysis Centroid Distance
From: John Halley Gotway
Time: Mon Jan 31 13:59:46 2011

########################################################################
##
##   Name: mode_summary.R
##
##   Description:
##      This R script summarizes the contents of one or more MODE
##      object statistics file.
##
##   Usage:
##      Rscript mode_summary.R mode_obj_file_list
##
##   Arguments:
##      "mode_obj_file_list" is a space-seperated list of the MODE
##      object statistics file names to be used.
##
##   Details:
##      This R script summarizes the contents of one or more MODE
##      object statistics files.  It reads in the MODE file(s) passed
##      on the command line, counts the numbers/areas of the
##      matched/unmatched forecast/observation objects and prints them
##      to the screen.  It also computes the maximum interest value
for
##      each object and prints out the median of them for the forecast
##      objects, observation objects, and both.
##
##      This R script is meant as an example.  Users are welcome and
##      encouraged to adapt it to perform the type of analysis they
##      need.
##
##   Examples:
##      Rscript mode_summary.R
mode_120000L_20050807_120000V_120000A_obj.txt
##
##   Author:
##      John Halley Gotway, MET Development Team, 10/22/2008
##      johnhg at ucar.edu
##
########################################################################

########################################################################
#
# Handle the arguments.
#
########################################################################

# Retreive the arguments
args <- commandArgs(TRUE)

# Check the number of arguments
if(length(args) < 1) {
   cat("Usage: mode_summary.R mode_obj_file_list\n")
   cat("   where mode_obj_file_list is a space-separated list of MODE
object statistic files.\n")
   quit()
}

# Constants
i_start<-1

# Variables to store the object counts
sum_n_fcst<-0
sum_n_fcst_match<-0
sum_n_fcst_unmatch<-0
sum_n_obs<-0
sum_n_obs_match<-0
sum_n_obs_unmatch<-0
sum_n_fcst_clus<-0
sum_n_obs_clus<-0

# Variables to store the object areas
sum_a_fcst<-0
sum_a_fcst_match<-0
sum_a_fcst_unmatch<-0
sum_a_obs<-0
sum_a_obs_match<-0
sum_a_obs_unmatch<-0
sum_a_fcst_clus<-0
sum_a_obs_clus<-0

# List of maximum interest values by forecast and observation objects
list_max_interest_fcst <-c()
list_max_interest_obs  <-c()

# List of cluster pair centroid distances and x,y offsets
list_centroid_distance <-c()
list_centroid_x_offset <-c()
list_centroid_y_offset <-c()

# Handle each of the MODE files one at a time
for(i in i_start:length(args)) {

   # Store the current MODE file
   mode_file<-args[i]

   # Read the MODE file passed as an argument
   mode<-read.table(mode_file, header=TRUE)

   cat(paste("Read", dim(mode)[1], "lines from MODE file:", mode_file,
"\n"))

   # Check for an empty file containing only the header row
   if(dim(mode)[1] > 0) {

      # Replace instances of -9999 with NA
      mode[mode==-9999]=NA

      ########################################################################
      #
      # Compute indicators for each of the line types.
      #
      ########################################################################

      ind_simp_fcst <-regexpr("^F[0-9][0-9][0-9]$", mode$OBJECT_ID) ==
1
      ind_simp_obs  <-regexpr("^O[0-9][0-9][0-9]$", mode$OBJECT_ID) ==
1
      ind_simp_pair <-regexpr("^F[0-9][0-9][0-9]_O[0-9][0-9][0-9]$",
mode$OBJECT_ID) == 1

      ind_clus_fcst <-regexpr("^CF[0-9][0-9][0-9]$", mode$OBJECT_ID)
== 1
      ind_clus_obs  <-regexpr("^CO[0-9][0-9][0-9]$", mode$OBJECT_ID)
== 1
      ind_clus_pair <-regexpr("^CF[0-9][0-9][0-9]_CO[0-9][0-9][0-9]$",
mode$OBJECT_ID) == 1

      ind_match     <-(regexpr("^C[F,O]000$", mode$OBJECT_CAT) == -1)
*
                      (regexpr("^C[F,O][0-9][0-9][0-9]$",
mode$OBJECT_CAT) == 1)
      ind_unmatch   <-regexpr("^C[F,O]000$", mode$OBJECT_CAT) == 1

      ########################################################################
      #
      # Keep a running sum of object counts and areas.
      #
      ########################################################################

      # Keep a running sum of the object counts
      sum_n_fcst         <-sum_n_fcst+sum(ind_simp_fcst)
      sum_n_fcst_match   <-
sum_n_fcst_match+sum(ind_simp_fcst*ind_match > 0)
      sum_n_fcst_unmatch <-
sum_n_fcst_unmatch+sum(ind_simp_fcst*ind_unmatch > 0)

      sum_n_obs          <-sum_n_obs+sum(ind_simp_obs)
      sum_n_obs_match    <-sum_n_obs_match+sum(ind_simp_obs*ind_match
> 0)
      sum_n_obs_unmatch  <-
sum_n_obs_unmatch+sum(ind_simp_obs*ind_unmatch > 0)

      sum_n_fcst_clus    <-sum_n_fcst_clus+sum(ind_clus_fcst)
      sum_n_obs_clus     <-sum_n_obs_clus+sum(ind_clus_obs)

      # Keep a running sum of the object areas
      sum_a_fcst         <-sum_a_fcst+sum(mode$AREA[ind_simp_fcst])
      sum_a_fcst_match   <-
sum_a_fcst_match+sum(mode$AREA[ind_simp_fcst*ind_match > 0])
      sum_a_fcst_unmatch <-
sum_a_fcst_unmatch+sum(mode$AREA[ind_simp_fcst*ind_unmatch > 0])

      sum_a_obs          <-sum_a_obs+sum(mode$AREA[ind_simp_obs])
      sum_a_obs_match    <-
sum_a_obs_match+sum(mode$AREA[ind_simp_obs*ind_match > 0])
      sum_a_obs_unmatch  <-
sum_a_obs_unmatch+sum(mode$AREA[ind_simp_obs*ind_unmatch > 0])

      sum_a_fcst_clus    <-
sum_a_fcst_clus+sum(mode$AREA[ind_clus_fcst])
      sum_a_obs_clus     <-sum_a_obs_clus+sum(mode$AREA[ind_clus_obs])

      # Check for zero forecast or zero observation objects
      if(sum(ind_simp_fcst) == 0 || sum(ind_simp_obs) == 0) next

      ########################################################################
      #
      # Keep track of maximum interest values for each object.
      #
      ########################################################################

      # Use only the simple pair lines.
      ind<-ind_simp_pair

      # Find the maximum interest for each forecast object.
      max_interest_fcst <-aggregate(mode$INTEREST[ind],
by=list(fcst=substr(mode$OBJECT_ID[ind], 1, 4)), FUN=max)

      # Find the maximum interest for each observation object.
      max_interest_obs <-aggregate(mode$INTEREST[ind],
by=list(fcst=substr(mode$OBJECT_ID[ind], 6, 9)), FUN=max)

      # Append this array to the previous ones
      list_max_interest_fcst <-c(list_max_interest_fcst,
max_interest_fcst$x)
      list_max_interest_obs  <-c(list_max_interest_obs,
max_interest_obs$x)

      ########################################################################
      #
      # Keep track of centroid X,Y offsets.
      #
      ########################################################################

      # For each cluster pair, compute the centroid X/Y offset.
      for(str in mode$OBJECT_ID[ind_clus_pair]) {

         # Pull out the fcst/obs object id's
         fcst = substr(str, 1, 5)
         obs  = substr(str, 7, 11)

         # Compute the X/Y offsets.
         distance = mode$CENTROID_DIST[mode$OBJECT_ID==str]
         x_offset = mode$CENTROID_X[mode$OBJECT_ID==fcst] -
mode$CENTROID_X[mode$OBJECT_ID==obs]
         y_offset = mode$CENTROID_Y[mode$OBJECT_ID==fcst] -
mode$CENTROID_Y[mode$OBJECT_ID==obs]

         # Append to the list of distances and X/Y offsets.
         list_centroid_distance <-c(list_centroid_distance, distance)
         list_centroid_x_offset <-c(list_centroid_x_offset, x_offset)
         list_centroid_y_offset <-c(list_centroid_y_offset, y_offset)

      } # end for loop

   } # end if statement

} # end for loop

########################################################################
#
# Print the results to the screen.
#
########################################################################

# Print out the counts and areas of the single objects
cat(paste("Total Number of Files Processed =", length(args)), "\n")
cat(paste("Total Number of Single Objects =", sum_n_fcst+sum_n_obs),
"\n")

cat(paste("Number of Single Fcst Objects =", sum_n_fcst), "\n")
cat(paste("Number of Matched Single Fcst Objects =",
sum_n_fcst_match), "\n")
cat(paste("Number of Unmatched Single Fcst Objects =",
sum_n_fcst_unmatch), "\n")

cat(paste("Number of Single Obs Objects =", sum_n_obs), "\n")
cat(paste("Number of Matched Single Obs Objects =", sum_n_obs_match),
"\n")
cat(paste("Number of Unmatched Single Obs Objects =",
sum_n_obs_unmatch), "\n")

cat(paste("Total Number of Cluster Objects =",
sum_n_fcst_clus+sum_n_obs_clus), "\n")
cat(paste("Total Number of Fcst Cluster Objects =", sum_n_fcst_clus),
"\n")
cat(paste("Total Number of Obs Cluster Objects =", sum_n_obs_clus),
"\n")

cat(paste("Total Area of Objects =", sum_a_fcst+sum_a_obs), "\n")

cat(paste("Area of Single Fcst Objects =", sum_a_fcst), "\n")
cat(paste("Area of Matched Single Fcst Objects =", sum_a_fcst_match),
"\n")
cat(paste("Area of Unmatched Single Fcst Objects =",
sum_a_fcst_unmatch), "\n")

cat(paste("Area of Single Obs Objects =", sum_a_obs), "\n")
cat(paste("Area of Matched Single Obs Objects =", sum_a_obs_match),
"\n")
cat(paste("Area of Unmatched Single Obs Objects =",
sum_a_obs_unmatch), "\n")

# Print out the median of the maximums
cat(paste("Median of Max Interest by Fcst Object =",
median(list_max_interest_fcst)), "\n")
cat(paste("Median of Max Interest by Obs Object =",
median(list_max_interest_obs)), "\n")
cat(paste("Median of Max Interest by Fcst+Obs Objects =",
median(c(list_max_interest_fcst, list_max_interest_obs))), "\n")

#cat(paste("Dump Max Interest by Fcst Object ="),
list_max_interest_fcst, "\n")
#cat(paste("Dump Max Interest by Obs Object ="),
list_max_interest_obs, "\n")
#cat(paste("Dump Max Interest by Fcst+Obs Object ="),
list_max_interest_fcst, list_max_interest_obs, "\n")

# Print out information about the cluster centroid distances and
offsets
cat("Cluster Pair Centroid Distance:\n")
summary(list_centroid_distance)
cat("Cluster Pair Centroid X-Offsets:\n")
summary(list_centroid_x_offset)
cat("Cluster Pair Centroid Y-Offsets:\n")
summary(list_centroid_y_offset)

cat(paste("Cluster Pair Centroid X,Y Offsets and Distance [",
length(list_centroid_x_offset), "]:\n", sep=""))
cat(sprintf("(%.4f, %.4f) %.4f\n", list_centroid_x_offset,
list_centroid_y_offset, list_centroid_distance), sep="")

save.image()

------------------------------------------------
Subject: Mode Analysis Centroid Distance
From: Casey Zoellick
Time: Mon Jan 31 22:54:55 2011

John,

Using -cluster and -pair yields the same results for me so I think I'm
all
set!  I'll have to play around with the Rscript and see what it can do
for
us.  Otherwise, thanks again for your help!

-Casey


On Mon, Jan 31, 2011 at 2:59 PM, RAL HelpDesk {for John Halley Gotway}
<
met_help at ucar.edu> wrote:

> Casey,
>
> I also ran the "inch" data through the "mode_summary.R" script.
Your data
> actually pointed out a bug in that script when there are either 0
forecast
> or 0 observation objects.  So I updated it and
> also enhanced it to dump out summary information about the centroid
> distance.  I've attached the updated script as well as a file
showing what
> the output looks like for the "inch" data.  It was
> actually a good way of double-checking the MODE-Analysis output to
make
> sure it's the same as the mode_summary.R output... fortunately, they
are :)
>
> Here's how I ran that script:
>   Rscript mode_summary.R mode*_obj.txt
>
> Thanks,
> John
>
> On 01/31/2011 01:40 PM, RAL HelpDesk {for John Halley Gotway} wrote:
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=43929 >
> >
> > Casey,
> >
> > OK, try running without the "-matched" option.  All "cluster"
objects are
> by definition matched.  I tried running using just the data you sent
in
> "mode_obj_inch.tar".  Here's the command and output I see:
> >
> > +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >
> > /d1/johnhg/MET/MET_releases/METv3.0/bin/mode_analysis -lookin .
-summary
> -column CENTROID_DIST -cluster -pair
> >
> > Total mode lines read =  1,278
> > Total mode lines kept =     26
> >
> >         Field    N    Min     Max    Mean   StdDev    P10    P25
P50
>     P75     P90      Sum
> > -------------   --   ----   -----   -----   ------   ----   ----
-----
>   -----   -----   ------
> > centroid_dist   26   1.30   50.24   16.66    11.59   5.90   8.72
14.91
>   20.10   26.14   433.20
> >
> > +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >
> > I did ran a combination of grep/sed/cut commands on the command
line and
> verified that 26 lines is the correct number of lines it should
find.  Also,
> I ran a second time using the "-dump_row" option:
> >
> > /d1/johnhg/MET/MET_releases/METv3.0/bin/mode_analysis -lookin .
-summary
> -column CENTROID_DIST -cluster -pair -dump_row mode_lines.out
> >
> > That created a file called "mode_lines.out" that listed out the
MODE
> lines the job used.  It's always a good idea to use the "-dump_row"
option
> when setting up a MODE-Analysis job - just to make sure
> > that the tool is doing what you expect.
> >
> > Sorry this is so confusing, but I hope it helps.
> >
> > John
> >
> > On 01/31/2011 01:07 PM, RAL HelpDesk {for Casey Zoellick} wrote:
> >>
> >> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=43929 >
> >>
> >> John,
> >>
> >> I put the files on your site.  We're using METv3.0.  The command
I'm
> using
> >> is mode_analysis -summary -lookin . -pair -matched -cluster
-column
> >> CENTROID_DIST
> >>
> >> Thanks,
> >> Casey
> >>
> >>
> >> On Sun, Jan 30, 2011 at 9:51 PM, RAL HelpDesk {for John Halley
Gotway} <
> >> met_help at ucar.edu> wrote:
> >>
> >>> Casey,
> >>>
> >>> Sorry you guys are having so much trouble with this.  Please
send me
> some
> >>> MODE output files you're trying to run through the MODE-Analysis
tool -
> >>> along with the exact command line you're using to run it.
> >>>
> >>> You could just create a tar file with a bunch of the MODE output
files
> >>> (*_obj.txt).  I'll run it through MODE-Analysis here and try to
figure
> out
> >>> what's going on.  You can post those files to our anonymous ftp
site
> using
> >>> the following instructions:
> >>>   http://www.dtcenter.org/met/users/support/met_help.php#ftp
> >>>
> >>> Also, please remind me what version of MET you're running, 2.0
or 3.0.
> >>> I'd always recommend updating to the latest set of patches which
are
> >>> available here:
> >>>
http://www.dtcenter.org/met/users/support/known_issues/index.php
> >>> Just click on the version you're using.
> >>>
> >>> Lastly, I want to point out an alternative to the MODE-Analysis
tool
> that
> >>> you may (or may not) find helpful.  We have a sample R script
posted on
> >>> the website to parses a bunch of MODE info and dumps out summary
stats
> >>> about it:
> >>>
http://www.dtcenter.org/met/users/downloads/analysis_scripts.php
> >>> Look for the file "mode_summary.R".  If you're familiar with R,
you may
> >>> find it easy to use and modify this script.  To run it, you'd
type:
> >>>   Rscript mode_summary.R mode1_obj.txt mode2_obj.txt ...
moden_obj.txt,
> >>> where those "mode*_obj.txt" are the MODE output files from
individual
> runs.
> >>>
> >>> Please write me back when you have posted that data to our
website.
>  I'll
> >>> grab it and take a look.
> >>>
> >>> Thanks,
> >>> John Halley Gotway
> >>>
> >>>
> >>>
> >>>
> >>>>
> >>>> Fri Jan 28 12:33:38 2011: Request 43929 was acted upon.
> >>>> Transaction: Ticket created by czoellic at iastate.edu
> >>>>        Queue: met_help
> >>>>      Subject: Mode Analysis Centroid Distance
> >>>>        Owner: Nobody
> >>>>   Requestors: czoellic at iastate.edu
> >>>>       Status: new
> >>>>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=43929>
> >>>>
> >>>>
> >>>> John,
> >>>>
> >>>> I'm working on Dr. Bill Gallus's project and having the issue
with
> >>> getting
> >>>> centroid distance statistics.  I tried using all three flags (-
matched
> >>>> -pair
> >>>> -cluster) and it doesn't "keep" any lines.  However, when I run
it
> with
> >>>> the
> >>>> same three except -unmatched I get the same results as if I ran
it
> >>> without
> >>>> a
> >>>> -matched/unmatched flag.  This seems to imply that there are no
> matched
> >>>> pairs in our dataset, however, upon looking at the postscript
files I
> >>> know
> >>>> for a fact that object pairs are matching above the .7
threshold we
> have
> >>>> set.
> >>>>
> >>>> Thanks for your help!
> >>>> Casey Zoellick
> >>>>
> >>>
> >>>
> >>>
> >>>
> >>
> >>
>
>
> Read 0 lines from MODE file:
mode_300000L_20100512_060000V_060000A_obj.txt
> Read 1 lines from MODE file:
mode_300000L_20100601_060000V_060000A_obj.txt
> Read 3 lines from MODE file:
mode_300000L_20100605_060000V_060000A_obj.txt
> Read 2 lines from MODE file:
mode_300000L_20100612_060000V_060000A_obj.txt
> Read 119 lines from MODE file:
> mode_300000L_20100618_060000V_060000A_obj.txt
> Read 65 lines from MODE file:
mode_300000L_20100622_060000V_060000A_obj.txt
> Read 30 lines from MODE file:
mode_300000L_20100623_060000V_060000A_obj.txt
> Read 20 lines from MODE file:
mode_300000L_20100627_060000V_060000A_obj.txt
> Read 0 lines from MODE file:
mode_300000L_20100704_060000V_060000A_obj.txt
> Read 2 lines from MODE file:
mode_300000L_20100719_060000V_060000A_obj.txt
> Read 14 lines from MODE file:
mode_300000L_20100722_060000V_060000A_obj.txt
> Read 100 lines from MODE file:
> mode_300000L_20100724_060000V_060000A_obj.txt
> Read 0 lines from MODE file:
mode_300000L_20100808_060000V_060000A_obj.txt
> Read 72 lines from MODE file:
mode_300000L_20100809_060000V_060000A_obj.txt
> Read 39 lines from MODE file:
mode_300000L_20100810_060000V_060000A_obj.txt
> Read 6 lines from MODE file:
mode_300000L_20100831_060000V_060000A_obj.txt
> Read 0 lines from MODE file:
mode_360000L_20100512_120000V_060000A_obj.txt
> Read 0 lines from MODE file:
mode_360000L_20100601_120000V_060000A_obj.txt
> Read 16 lines from MODE file:
mode_360000L_20100605_120000V_060000A_obj.txt
> Read 19 lines from MODE file:
mode_360000L_20100612_120000V_060000A_obj.txt
> Read 5 lines from MODE file:
mode_360000L_20100618_120000V_060000A_obj.txt
> Read 47 lines from MODE file:
mode_360000L_20100622_120000V_060000A_obj.txt
> Read 41 lines from MODE file:
mode_360000L_20100623_120000V_060000A_obj.txt
> Read 19 lines from MODE file:
mode_360000L_20100627_120000V_060000A_obj.txt
> Read 2 lines from MODE file:
mode_360000L_20100704_120000V_060000A_obj.txt
> Read 2 lines from MODE file:
mode_360000L_20100719_120000V_060000A_obj.txt
> Read 10 lines from MODE file:
mode_360000L_20100722_120000V_060000A_obj.txt
> Read 102 lines from MODE file:
> mode_360000L_20100724_120000V_060000A_obj.txt
> Read 8 lines from MODE file:
mode_360000L_20100808_120000V_060000A_obj.txt
> Read 89 lines from MODE file:
mode_360000L_20100809_120000V_060000A_obj.txt
> Read 10 lines from MODE file:
mode_360000L_20100810_120000V_060000A_obj.txt
> Read 6 lines from MODE file:
mode_360000L_20100831_120000V_060000A_obj.txt
> Read 7 lines from MODE file:
mode_420000L_20100512_180000V_060000A_obj.txt
> Read 1 lines from MODE file:
mode_420000L_20100601_180000V_060000A_obj.txt
> Read 5 lines from MODE file:
mode_420000L_20100605_180000V_060000A_obj.txt
> Read 7 lines from MODE file:
mode_420000L_20100612_180000V_060000A_obj.txt
> Read 1 lines from MODE file:
mode_420000L_20100618_180000V_060000A_obj.txt
> Read 5 lines from MODE file:
mode_420000L_20100622_180000V_060000A_obj.txt
> Read 14 lines from MODE file:
mode_420000L_20100623_180000V_060000A_obj.txt
> Read 7 lines from MODE file:
mode_420000L_20100627_180000V_060000A_obj.txt
> Read 32 lines from MODE file:
mode_420000L_20100704_180000V_060000A_obj.txt
> Read 3 lines from MODE file:
mode_420000L_20100719_180000V_060000A_obj.txt
> Read 14 lines from MODE file:
mode_420000L_20100722_180000V_060000A_obj.txt
> Read 3 lines from MODE file:
mode_420000L_20100724_180000V_060000A_obj.txt
> Read 1 lines from MODE file:
mode_420000L_20100808_180000V_060000A_obj.txt
> Read 5 lines from MODE file:
mode_420000L_20100809_180000V_060000A_obj.txt
> Read 9 lines from MODE file:
mode_420000L_20100810_180000V_060000A_obj.txt
> Read 5 lines from MODE file:
mode_420000L_20100831_180000V_060000A_obj.txt
> Read 4 lines from MODE file:
mode_480000L_20100513_000000V_060000A_obj.txt
> Read 23 lines from MODE file:
mode_480000L_20100602_000000V_060000A_obj.txt
> Read 3 lines from MODE file:
mode_480000L_20100606_000000V_060000A_obj.txt
> Read 49 lines from MODE file:
mode_480000L_20100613_000000V_060000A_obj.txt
> Read 5 lines from MODE file:
mode_480000L_20100619_000000V_060000A_obj.txt
> Read 0 lines from MODE file:
mode_480000L_20100623_000000V_060000A_obj.txt
> Read 26 lines from MODE file:
mode_480000L_20100624_000000V_060000A_obj.txt
> Read 5 lines from MODE file:
mode_480000L_20100628_000000V_060000A_obj.txt
> Read 50 lines from MODE file:
mode_480000L_20100705_000000V_060000A_obj.txt
> Read 1 lines from MODE file:
mode_480000L_20100720_000000V_060000A_obj.txt
> Read 14 lines from MODE file:
mode_480000L_20100723_000000V_060000A_obj.txt
> Read 7 lines from MODE file:
mode_480000L_20100725_000000V_060000A_obj.txt
> Read 1 lines from MODE file:
mode_480000L_20100809_000000V_060000A_obj.txt
> Read 4 lines from MODE file:
mode_480000L_20100810_000000V_060000A_obj.txt
> Read 29 lines from MODE file:
mode_480000L_20100811_000000V_060000A_obj.txt
> Read 89 lines from MODE file:
mode_480000L_20100901_000000V_060000A_obj.txt
> Total Number of Files Processed = 64
> Total Number of Single Objects = 397
> Number of Single Fcst Objects = 128
> Number of Matched Single Fcst Objects = 26
> Number of Unmatched Single Fcst Objects = 102
> Number of Single Obs Objects = 269
> Number of Matched Single Obs Objects = 26
> Number of Unmatched Single Obs Objects = 243
> Total Number of Cluster Objects = 52
> Total Number of Fcst Cluster Objects = 26
> Total Number of Obs Cluster Objects = 26
> Total Area of Objects = 95971
> Area of Single Fcst Objects = 37448
> Area of Matched Single Fcst Objects = 24839
> Area of Unmatched Single Fcst Objects = 12609
> Area of Single Obs Objects = 58523
> Area of Matched Single Obs Objects = 21697
> Area of Unmatched Single Obs Objects = 36826
> Median of Max Interest by Fcst Object = 0.57435
> Median of Max Interest by Obs Object = 0.53199
> Median of Max Interest by Fcst+Obs Objects = 0.556635
> Cluster Pair Centroid Distance:
>   Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
>  1.297   8.723  14.910  16.660  20.100  50.240
> Cluster Pair Centroid X-Offsets:
>   Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
> -26.400 -10.430   3.969   1.131   7.860  42.920
> Cluster Pair Centroid Y-Offsets:
>   Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
> -26.120  -4.555   1.109   1.735   5.766  47.990
> Cluster Pair Centroid X,Y Offsets and Distance [26]:
> (-0.8987, 0.9349) 1.2968
> (-5.5570, 2.7714) 6.2098
> (-11.1843, -0.8654) 11.2178
> (6.2597, -11.3853) 12.9926
> (20.2623, 5.8179) 21.0810
> (8.0000, 1.0000) 8.0623
> (14.4987, 4.1437) 15.0792
> (4.7882, -2.1812) 5.2616
> (-10.6737, 10.1552) 14.7328
> (16.0097, 1.2176) 16.0560
> (42.9162, -26.1246) 50.2424
> (7.0060, 1.8475) 7.2455
> (-9.6863, -5.5404) 11.1589
> (-22.6638, 0.4568) 22.6684
> (0.9783, 18.0960) 18.1224
> (4.6992, 18.1052) 18.7051
> (3.2387, 11.5994) 12.0430
> (1.6144, -5.3458) 5.5843
> (-13.4248, 47.9927) 49.8350
> (7.4386, 3.6403) 8.2816
> (-21.6371, 5.6097) 22.3525
> (-26.3975, -13.3997) 29.6038
> (8.1428, -18.4546) 20.1712
> (5.6305, 8.3191) 10.0454
> (15.2442, -0.4567) 15.2510
> (-15.1959, -12.8454) 19.8977
>
>
########################################################################
> ##
> ##   Name: mode_summary.R
> ##
> ##   Description:
> ##      This R script summarizes the contents of one or more MODE
> ##      object statistics file.
> ##
> ##   Usage:
> ##      Rscript mode_summary.R mode_obj_file_list
> ##
> ##   Arguments:
> ##      "mode_obj_file_list" is a space-seperated list of the MODE
> ##      object statistics file names to be used.
> ##
> ##   Details:
> ##      This R script summarizes the contents of one or more MODE
> ##      object statistics files.  It reads in the MODE file(s)
passed
> ##      on the command line, counts the numbers/areas of the
> ##      matched/unmatched forecast/observation objects and prints
them
> ##      to the screen.  It also computes the maximum interest value
for
> ##      each object and prints out the median of them for the
forecast
> ##      objects, observation objects, and both.
> ##
> ##      This R script is meant as an example.  Users are welcome and
> ##      encouraged to adapt it to perform the type of analysis they
> ##      need.
> ##
> ##   Examples:
> ##      Rscript mode_summary.R
> mode_120000L_20050807_120000V_120000A_obj.txt
> ##
> ##   Author:
> ##      John Halley Gotway, MET Development Team, 10/22/2008
> ##      johnhg at ucar.edu
> ##
>
########################################################################
>
>
########################################################################
> #
> # Handle the arguments.
> #
>
########################################################################
>
> # Retreive the arguments
> args <- commandArgs(TRUE)
>
> # Check the number of arguments
> if(length(args) < 1) {
>   cat("Usage: mode_summary.R mode_obj_file_list\n")
>   cat("   where mode_obj_file_list is a space-separated list of MODE
object
> statistic files.\n")
>   quit()
> }
>
> # Constants
> i_start<-1
>
> # Variables to store the object counts
> sum_n_fcst<-0
> sum_n_fcst_match<-0
> sum_n_fcst_unmatch<-0
> sum_n_obs<-0
> sum_n_obs_match<-0
> sum_n_obs_unmatch<-0
> sum_n_fcst_clus<-0
> sum_n_obs_clus<-0
>
> # Variables to store the object areas
> sum_a_fcst<-0
> sum_a_fcst_match<-0
> sum_a_fcst_unmatch<-0
> sum_a_obs<-0
> sum_a_obs_match<-0
> sum_a_obs_unmatch<-0
> sum_a_fcst_clus<-0
> sum_a_obs_clus<-0
>
> # List of maximum interest values by forecast and observation
objects
> list_max_interest_fcst <-c()
> list_max_interest_obs  <-c()
>
> # List of cluster pair centroid distances and x,y offsets
> list_centroid_distance <-c()
> list_centroid_x_offset <-c()
> list_centroid_y_offset <-c()
>
> # Handle each of the MODE files one at a time
> for(i in i_start:length(args)) {
>
>   # Store the current MODE file
>   mode_file<-args[i]
>
>   # Read the MODE file passed as an argument
>   mode<-read.table(mode_file, header=TRUE)
>
>   cat(paste("Read", dim(mode)[1], "lines from MODE file:",
mode_file,
> "\n"))
>
>   # Check for an empty file containing only the header row
>   if(dim(mode)[1] > 0) {
>
>      # Replace instances of -9999 with NA
>      mode[mode==-9999]=NA
>
>
>
########################################################################
>      #
>      # Compute indicators for each of the line types.
>      #
>
>
########################################################################
>
>      ind_simp_fcst <-regexpr("^F[0-9][0-9][0-9]$", mode$OBJECT_ID)
== 1
>      ind_simp_obs  <-regexpr("^O[0-9][0-9][0-9]$", mode$OBJECT_ID)
== 1
>      ind_simp_pair <-regexpr("^F[0-9][0-9][0-9]_O[0-9][0-9][0-9]$",
> mode$OBJECT_ID) == 1
>
>      ind_clus_fcst <-regexpr("^CF[0-9][0-9][0-9]$", mode$OBJECT_ID)
== 1
>      ind_clus_obs  <-regexpr("^CO[0-9][0-9][0-9]$", mode$OBJECT_ID)
== 1
>      ind_clus_pair <-regexpr("^CF[0-9][0-9][0-9]_CO[0-9][0-9][0-
9]$",
> mode$OBJECT_ID) == 1
>
>      ind_match     <-(regexpr("^C[F,O]000$", mode$OBJECT_CAT) == -1)
*
>                      (regexpr("^C[F,O][0-9][0-9][0-9]$",
mode$OBJECT_CAT)
> == 1)
>      ind_unmatch   <-regexpr("^C[F,O]000$", mode$OBJECT_CAT) == 1
>
>
>
########################################################################
>      #
>      # Keep a running sum of object counts and areas.
>      #
>
>
########################################################################
>
>      # Keep a running sum of the object counts
>      sum_n_fcst         <-sum_n_fcst+sum(ind_simp_fcst)
>      sum_n_fcst_match   <-
sum_n_fcst_match+sum(ind_simp_fcst*ind_match > 0)
>      sum_n_fcst_unmatch <-
sum_n_fcst_unmatch+sum(ind_simp_fcst*ind_unmatch
> > 0)
>
>      sum_n_obs          <-sum_n_obs+sum(ind_simp_obs)
>      sum_n_obs_match    <-sum_n_obs_match+sum(ind_simp_obs*ind_match
> 0)
>      sum_n_obs_unmatch  <-
sum_n_obs_unmatch+sum(ind_simp_obs*ind_unmatch >
> 0)
>
>      sum_n_fcst_clus    <-sum_n_fcst_clus+sum(ind_clus_fcst)
>      sum_n_obs_clus     <-sum_n_obs_clus+sum(ind_clus_obs)
>
>      # Keep a running sum of the object areas
>      sum_a_fcst         <-sum_a_fcst+sum(mode$AREA[ind_simp_fcst])
>      sum_a_fcst_match
> <-sum_a_fcst_match+sum(mode$AREA[ind_simp_fcst*ind_match > 0])
>      sum_a_fcst_unmatch
> <-sum_a_fcst_unmatch+sum(mode$AREA[ind_simp_fcst*ind_unmatch > 0])
>
>      sum_a_obs          <-sum_a_obs+sum(mode$AREA[ind_simp_obs])
>      sum_a_obs_match
>  <-sum_a_obs_match+sum(mode$AREA[ind_simp_obs*ind_match > 0])
>      sum_a_obs_unmatch
>  <-sum_a_obs_unmatch+sum(mode$AREA[ind_simp_obs*ind_unmatch > 0])
>
>      sum_a_fcst_clus    <-
sum_a_fcst_clus+sum(mode$AREA[ind_clus_fcst])
>      sum_a_obs_clus     <-
sum_a_obs_clus+sum(mode$AREA[ind_clus_obs])
>
>      # Check for zero forecast or zero observation objects
>      if(sum(ind_simp_fcst) == 0 || sum(ind_simp_obs) == 0) next
>
>
>
########################################################################
>      #
>      # Keep track of maximum interest values for each object.
>      #
>
>
########################################################################
>
>      # Use only the simple pair lines.
>      ind<-ind_simp_pair
>
>      # Find the maximum interest for each forecast object.
>      max_interest_fcst <-aggregate(mode$INTEREST[ind],
> by=list(fcst=substr(mode$OBJECT_ID[ind], 1, 4)), FUN=max)
>
>      # Find the maximum interest for each observation object.
>      max_interest_obs <-aggregate(mode$INTEREST[ind],
> by=list(fcst=substr(mode$OBJECT_ID[ind], 6, 9)), FUN=max)
>
>      # Append this array to the previous ones
>      list_max_interest_fcst <-c(list_max_interest_fcst,
> max_interest_fcst$x)
>      list_max_interest_obs  <-c(list_max_interest_obs,
max_interest_obs$x)
>
>
>
########################################################################
>      #
>      # Keep track of centroid X,Y offsets.
>      #
>
>
########################################################################
>
>      # For each cluster pair, compute the centroid X/Y offset.
>      for(str in mode$OBJECT_ID[ind_clus_pair]) {
>
>         # Pull out the fcst/obs object id's
>         fcst = substr(str, 1, 5)
>         obs  = substr(str, 7, 11)
>
>         # Compute the X/Y offsets.
>         distance = mode$CENTROID_DIST[mode$OBJECT_ID==str]
>         x_offset = mode$CENTROID_X[mode$OBJECT_ID==fcst] -
> mode$CENTROID_X[mode$OBJECT_ID==obs]
>         y_offset = mode$CENTROID_Y[mode$OBJECT_ID==fcst] -
> mode$CENTROID_Y[mode$OBJECT_ID==obs]
>
>         # Append to the list of distances and X/Y offsets.
>         list_centroid_distance <-c(list_centroid_distance, distance)
>         list_centroid_x_offset <-c(list_centroid_x_offset, x_offset)
>         list_centroid_y_offset <-c(list_centroid_y_offset, y_offset)
>
>      } # end for loop
>
>   } # end if statement
>
> } # end for loop
>
>
########################################################################
> #
> # Print the results to the screen.
> #
>
########################################################################
>
> # Print out the counts and areas of the single objects
> cat(paste("Total Number of Files Processed =", length(args)), "\n")
> cat(paste("Total Number of Single Objects =", sum_n_fcst+sum_n_obs),
"\n")
>
> cat(paste("Number of Single Fcst Objects =", sum_n_fcst), "\n")
> cat(paste("Number of Matched Single Fcst Objects =",
sum_n_fcst_match),
> "\n")
> cat(paste("Number of Unmatched Single Fcst Objects =",
sum_n_fcst_unmatch),
> "\n")
>
> cat(paste("Number of Single Obs Objects =", sum_n_obs), "\n")
> cat(paste("Number of Matched Single Obs Objects =",
sum_n_obs_match), "\n")
> cat(paste("Number of Unmatched Single Obs Objects =",
sum_n_obs_unmatch),
> "\n")
>
> cat(paste("Total Number of Cluster Objects =",
> sum_n_fcst_clus+sum_n_obs_clus), "\n")
> cat(paste("Total Number of Fcst Cluster Objects =",
sum_n_fcst_clus), "\n")
> cat(paste("Total Number of Obs Cluster Objects =", sum_n_obs_clus),
"\n")
>
> cat(paste("Total Area of Objects =", sum_a_fcst+sum_a_obs), "\n")
>
> cat(paste("Area of Single Fcst Objects =", sum_a_fcst), "\n")
> cat(paste("Area of Matched Single Fcst Objects =",
sum_a_fcst_match), "\n")
> cat(paste("Area of Unmatched Single Fcst Objects =",
sum_a_fcst_unmatch),
> "\n")
>
> cat(paste("Area of Single Obs Objects =", sum_a_obs), "\n")
> cat(paste("Area of Matched Single Obs Objects =", sum_a_obs_match),
"\n")
> cat(paste("Area of Unmatched Single Obs Objects =",
sum_a_obs_unmatch),
> "\n")
>
> # Print out the median of the maximums
> cat(paste("Median of Max Interest by Fcst Object =",
> median(list_max_interest_fcst)), "\n")
> cat(paste("Median of Max Interest by Obs Object =",
> median(list_max_interest_obs)), "\n")
> cat(paste("Median of Max Interest by Fcst+Obs Objects =",
> median(c(list_max_interest_fcst, list_max_interest_obs))), "\n")
>
> #cat(paste("Dump Max Interest by Fcst Object ="),
list_max_interest_fcst,
> "\n")
> #cat(paste("Dump Max Interest by Obs Object ="),
list_max_interest_obs,
> "\n")
> #cat(paste("Dump Max Interest by Fcst+Obs Object ="),
> list_max_interest_fcst, list_max_interest_obs, "\n")
>
> # Print out information about the cluster centroid distances and
offsets
> cat("Cluster Pair Centroid Distance:\n")
> summary(list_centroid_distance)
> cat("Cluster Pair Centroid X-Offsets:\n")
> summary(list_centroid_x_offset)
> cat("Cluster Pair Centroid Y-Offsets:\n")
> summary(list_centroid_y_offset)
>
> cat(paste("Cluster Pair Centroid X,Y Offsets and Distance [",
> length(list_centroid_x_offset), "]:\n", sep=""))
> cat(sprintf("(%.4f, %.4f) %.4f\n", list_centroid_x_offset,
> list_centroid_y_offset, list_centroid_distance), sep="")
>
> save.image()
>
>


--
CASEY L. ZOELLICK, C/Col, AFROTC
Wing Commander
Detachment 250
630-890-2911
czoellic at iastate.edu

------------------------------------------------


More information about the Met_help mailing list