[Met_help] [rt.rap.ucar.edu #44212] History for MODE Tool
RAL HelpDesk {for John Halley Gotway}
met_help at ucar.edu
Wed Feb 23 14:42:34 MST 2011
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
MET team,
I have a question regarding the MODE tool for precipitation verification:
I noticed that the input must be GRIB forecast files and netCDF observation
files (from PCP-combine). My question is this: is it possible to utilize
the netCDF output from the ASCII2NC tool as the observation file for the
MODE tool?
Thanks.
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #44212] MODE Tool
From: John Halley Gotway
Time: Wed Feb 09 09:09:06 2011
James,
The short answer is no.
MODE is a tool within MET that performs an object-based verification
approach. Based on the configuration you set up, it defines "objects"
in the forecast and observed fields and compares those
objects to one another. It can only define objects over a spatial
area - meaning that it can only be running using two input gridded
fields. The ASCII2NC tool is used to reformat point observations
for use by the Point-Stat tool to verify a gridded forecast. The
output of ASCII2NC is just point observations - and they cannot be
used by the MODE tool.
Some users do take dense networks of point observations and convert
them into gridded fields. If you were to do that, presumably, you
could run that gridded data through MODE. However, the MET
package itself doesn't provide tools for converting point observations
into a gridded field since that's not strictly verification and there
are often a lot of problems doing so.
Hope that helps clarify.
John
On 02/09/2011 08:13 AM, RAL HelpDesk {for James P Cipriani} wrote:
>
> Wed Feb 09 08:13:05 2011: Request 44212 was acted upon.
> Transaction: Ticket created by jpcipria at us.ibm.com
> Queue: met_help
> Subject: MODE Tool
> Owner: Nobody
> Requestors: jpcipria at us.ibm.com
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=44212 >
>
>
>
>
> MET team,
>
> I have a question regarding the MODE tool for precipitation
verification:
>
> I noticed that the input must be GRIB forecast files and netCDF
observation
> files (from PCP-combine). My question is this: is it possible to
utilize
> the netCDF output from the ASCII2NC tool as the observation file for
the
> MODE tool?
>
>
> Thanks.
------------------------------------------------
Subject: MODE Tool
From: James P Cipriani
Time: Wed Feb 09 09:55:47 2011
Thanks for the quick response, John. That was helpful.
Basically, I would like to obtain contingency table stats for
categorical
variables (ie precipitation). I know that I can use point-stat and
then
stat-analysis to aggregate the stats - is this the most efficient way
to do
so when using point observations?
Thanks.
From: "RAL HelpDesk {for John Halley Gotway}" <met_help at ucar.edu>
To: James P Cipriani/Watson/IBM at IBMUS
Cc: met_help at ucar.edu
Date: 02/09/2011 11:09 AM
Subject: Re: [rt.rap.ucar.edu #44212] MODE Tool
James,
The short answer is no.
MODE is a tool within MET that performs an object-based verification
approach. Based on the configuration you set up, it defines "objects"
in
the forecast and observed fields and compares those
objects to one another. It can only define objects over a spatial
area -
meaning that it can only be running using two input gridded fields.
The
ASCII2NC tool is used to reformat point observations
for use by the Point-Stat tool to verify a gridded forecast. The
output of
ASCII2NC is just point observations - and they cannot be used by the
MODE
tool.
Some users do take dense networks of point observations and convert
them
into gridded fields. If you were to do that, presumably, you could
run
that gridded data through MODE. However, the MET
package itself doesn't provide tools for converting point observations
into
a gridded field since that's not strictly verification and there are
often
a lot of problems doing so.
Hope that helps clarify.
John
On 02/09/2011 08:13 AM, RAL HelpDesk {for James P Cipriani} wrote:
>
> Wed Feb 09 08:13:05 2011: Request 44212 was acted upon.
> Transaction: Ticket created by jpcipria at us.ibm.com
> Queue: met_help
> Subject: MODE Tool
> Owner: Nobody
> Requestors: jpcipria at us.ibm.com
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=44212 >
>
>
>
>
> MET team,
>
> I have a question regarding the MODE tool for precipitation
verification:
>
> I noticed that the input must be GRIB forecast files and netCDF
observation
> files (from PCP-combine). My question is this: is it possible to
utilize
> the netCDF output from the ASCII2NC tool as the observation file for
the
> MODE tool?
>
>
> Thanks.
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #44212] MODE Tool
From: John Halley Gotway
Time: Wed Feb 09 10:21:41 2011
James,
Yep, that's the way I'd do it. Basically, run Point-Stat once for
each forecast valid time. When you configure Point-Stat, choose the
precipitation thresholds that are of interest to you. Also,
since it sounds like you're using ASCII observations, be sure that the
units match up. Precipitation is stored in GRIB files in mm. Just be
sure your ASCII observations are consistent. Also be sure
that the forecast and observation accumulation intervals are
identical.
When you run Point-Stat, make sure you're writing out the CTC and CTS
line types - this is set in the "output_flag" parameter of the config
file. Once you're run Point-Stat for all the forecast
times, you can use the STAT-Analysis tools aggregate your results
through time. For example:
stat_analysis -lookin out/point_stat -job aggregate_stat -line_type
CTC -out_line_type CTS -fcst_var APCP_12 -fcst_lead 12 -dump_row
ctc_to_cts_job.stat
The job listed above will start at the directory "out/point_stat" and
then recursively search through its tree looking for files ending in
".stat". It'll read each ".stat" file, looking for
contingency table count (-line_type CTC) lines for 12-hour accumulated
precipitation (-fcst_var APCP_12) with a forecast lead time of 12
hours (-fcst_lead 12). It'll read all those contingency
tables, sum them up into a single one, and then compute the
corresponding contingency table statistics (-out_line_type CTS).
Also, it'll write the lines used by this job out to a file named
"ctc_to_cts_job.stat". When setting up a job, it's always a good idea
to use the -dump_row option. Then, look at that file to make sure the
aggregation performed uses the lines you intended. For
example, perhaps you've used multiple interpolation methods or
multiple verification regions, and you didn't intend to aggregate
those results together. You can check to see if you've included lines
you didn't intend. And then you can add more filtering criteria to
the job to fix it.
You should of course set up the STAT-Analysis job to do the type of
aggregation you want. In my example, I aggregated all the 12-hour
lead times together. But you can do whatever aggregation you'd
like by setting the filter criteria yourself.
Hope that helps get you started.
John
On 02/09/2011 09:55 AM, RAL HelpDesk {for James P Cipriani} wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=44212 >
>
>
> Thanks for the quick response, John. That was helpful.
>
> Basically, I would like to obtain contingency table stats for
categorical
> variables (ie precipitation). I know that I can use point-stat and
then
> stat-analysis to aggregate the stats - is this the most efficient
way to do
> so when using point observations?
>
> Thanks.
>
>
>
> From: "RAL HelpDesk {for John Halley Gotway}" <met_help at ucar.edu>
> To: James P Cipriani/Watson/IBM at IBMUS
> Cc: met_help at ucar.edu
> Date: 02/09/2011 11:09 AM
> Subject: Re: [rt.rap.ucar.edu #44212] MODE Tool
>
>
>
> James,
>
> The short answer is no.
>
> MODE is a tool within MET that performs an object-based verification
> approach. Based on the configuration you set up, it defines
"objects" in
> the forecast and observed fields and compares those
> objects to one another. It can only define objects over a spatial
area -
> meaning that it can only be running using two input gridded fields.
The
> ASCII2NC tool is used to reformat point observations
> for use by the Point-Stat tool to verify a gridded forecast. The
output of
> ASCII2NC is just point observations - and they cannot be used by the
MODE
> tool.
>
> Some users do take dense networks of point observations and convert
them
> into gridded fields. If you were to do that, presumably, you could
run
> that gridded data through MODE. However, the MET
> package itself doesn't provide tools for converting point
observations into
> a gridded field since that's not strictly verification and there are
often
> a lot of problems doing so.
>
> Hope that helps clarify.
>
> John
>
> On 02/09/2011 08:13 AM, RAL HelpDesk {for James P Cipriani} wrote:
>>
>> Wed Feb 09 08:13:05 2011: Request 44212 was acted upon.
>> Transaction: Ticket created by jpcipria at us.ibm.com
>> Queue: met_help
>> Subject: MODE Tool
>> Owner: Nobody
>> Requestors: jpcipria at us.ibm.com
>> Status: new
>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=44212 >
>>
>>
>>
>>
>> MET team,
>>
>> I have a question regarding the MODE tool for precipitation
verification:
>>
>> I noticed that the input must be GRIB forecast files and netCDF
> observation
>> files (from PCP-combine). My question is this: is it possible to
utilize
>> the netCDF output from the ASCII2NC tool as the observation file
for the
>> MODE tool?
>>
>>
>> Thanks.
>
>
------------------------------------------------
Subject: MODE Tool
From: James P Cipriani
Time: Wed Feb 09 15:05:12 2011
Thanks again, John. This has been most helpful.
One more question, and I'll illustrate with an example:
Let's say I'm verifying accumulated 12-hour accumulated precipitation
but I
want to use an observation window of +/- 2 hours --> does this mean
that
the observations (formatted before ASCII2NC) need to have the
accumulation
interval labelled as "12" for the 2 hours behind or 2 hours ahead? Or
is
MET tools smart enough to know that if I want to look back 2 hours
(ahead 2
hours), it means that I wish to utilize the 10-hour (14-hour)
accumulation
amounts in my verification of 12-hour accumulated precipitation?
Hope that wasn't too confusing.
Thank you.
From: "RAL HelpDesk {for John Halley Gotway}" <met_help at ucar.edu>
To: James P Cipriani/Watson/IBM at IBMUS
Cc: met_help at ucar.edu
Date: 02/09/2011 12:21 PM
Subject: Re: [rt.rap.ucar.edu #44212] MODE Tool
James,
Yep, that's the way I'd do it. Basically, run Point-Stat once for
each
forecast valid time. When you configure Point-Stat, choose the
precipitation thresholds that are of interest to you. Also,
since it sounds like you're using ASCII observations, be sure that the
units match up. Precipitation is stored in GRIB files in mm. Just be
sure
your ASCII observations are consistent. Also be sure
that the forecast and observation accumulation intervals are
identical.
When you run Point-Stat, make sure you're writing out the CTC and CTS
line
types - this is set in the "output_flag" parameter of the config file.
Once you're run Point-Stat for all the forecast
times, you can use the STAT-Analysis tools aggregate your results
through
time. For example:
stat_analysis -lookin out/point_stat -job aggregate_stat -line_type
CTC
-out_line_type CTS -fcst_var APCP_12 -fcst_lead 12 -dump_row
ctc_to_cts_job.stat
The job listed above will start at the directory "out/point_stat" and
then
recursively search through its tree looking for files ending in
".stat".
It'll read each ".stat" file, looking for
contingency table count (-line_type CTC) lines for 12-hour accumulated
precipitation (-fcst_var APCP_12) with a forecast lead time of 12
hours
(-fcst_lead 12). It'll read all those contingency
tables, sum them up into a single one, and then compute the
corresponding
contingency table statistics (-out_line_type CTS). Also, it'll write
the
lines used by this job out to a file named
"ctc_to_cts_job.stat". When setting up a job, it's always a good idea
to
use the -dump_row option. Then, look at that file to make sure the
aggregation performed uses the lines you intended. For
example, perhaps you've used multiple interpolation methods or
multiple
verification regions, and you didn't intend to aggregate those results
together. You can check to see if you've included lines
you didn't intend. And then you can add more filtering criteria to
the job
to fix it.
You should of course set up the STAT-Analysis job to do the type of
aggregation you want. In my example, I aggregated all the 12-hour
lead
times together. But you can do whatever aggregation you'd
like by setting the filter criteria yourself.
Hope that helps get you started.
John
On 02/09/2011 09:55 AM, RAL HelpDesk {for James P Cipriani} wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=44212 >
>
>
> Thanks for the quick response, John. That was helpful.
>
> Basically, I would like to obtain contingency table stats for
categorical
> variables (ie precipitation). I know that I can use point-stat and
then
> stat-analysis to aggregate the stats - is this the most efficient
way to
do
> so when using point observations?
>
> Thanks.
>
>
>
> From: "RAL HelpDesk {for John Halley Gotway}"
<met_help at ucar.edu>
> To: James P Cipriani/Watson/IBM at IBMUS
> Cc: met_help at ucar.edu
> Date: 02/09/2011 11:09 AM
> Subject: Re: [rt.rap.ucar.edu #44212] MODE Tool
>
>
>
> James,
>
> The short answer is no.
>
> MODE is a tool within MET that performs an object-based verification
> approach. Based on the configuration you set up, it defines
"objects" in
> the forecast and observed fields and compares those
> objects to one another. It can only define objects over a spatial
area -
> meaning that it can only be running using two input gridded fields.
The
> ASCII2NC tool is used to reformat point observations
> for use by the Point-Stat tool to verify a gridded forecast. The
output
of
> ASCII2NC is just point observations - and they cannot be used by the
MODE
> tool.
>
> Some users do take dense networks of point observations and convert
them
> into gridded fields. If you were to do that, presumably, you could
run
> that gridded data through MODE. However, the MET
> package itself doesn't provide tools for converting point
observations
into
> a gridded field since that's not strictly verification and there are
often
> a lot of problems doing so.
>
> Hope that helps clarify.
>
> John
>
> On 02/09/2011 08:13 AM, RAL HelpDesk {for James P Cipriani} wrote:
>>
>> Wed Feb 09 08:13:05 2011: Request 44212 was acted upon.
>> Transaction: Ticket created by jpcipria at us.ibm.com
>> Queue: met_help
>> Subject: MODE Tool
>> Owner: Nobody
>> Requestors: jpcipria at us.ibm.com
>> Status: new
>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=44212 >
>>
>>
>>
>>
>> MET team,
>>
>> I have a question regarding the MODE tool for precipitation
verification:
>>
>> I noticed that the input must be GRIB forecast files and netCDF
> observation
>> files (from PCP-combine). My question is this: is it possible to
utilize
>> the netCDF output from the ASCII2NC tool as the observation file
for the
>> MODE tool?
>>
>>
>> Thanks.
>
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #44212] MODE Tool
From: John Halley Gotway
Time: Wed Feb 09 15:51:57 2011
James,
Without playing around with some actual data, I can't be positive on
this. But here's my take on it:
Since you're using the ASCII2NC tool, you're starting with
observations that are in the expected MET-ASCII format, containing 10
columns:
Message_Type Station_ID Valid_Time(YYYYMMDD_HHMMSS)
Lat(Deg North) Lon(Deg East) Elevation(msl)
Grib_Code Level Height(msl or agl) Observation_Value
In point-stat, you set the matching time window relative to the
forecast valid time. If you want to look +/- 2 hours, and for
example, you forecast is valid at 12Z, point-stat will use
observations
who's valid time falls between 10Z and 14Z. So it'll only use the
ASCII observations where the "Valid_Time" column contains a time
falling in that range.
Next, the "Level" column is used for precip variables to indicate the
accumulation interval. Since you're verifying 12-hour accumulated
precip, point-stat will only use the observations who's "Level"
value is exactly 12-hours.
So I believe the answer to your question is NO, point-stat isn't smart
enough to do what you're asking.
In the past, I've never really run into variable precip accumulation
intervals. Typically, they're very well defined as being hourly, 6-
hourly, 12-hourly, or 24-hourly. But if that's how your
observations are defined, you'll need to do whatever rounding is
necessary to get them to the exact accumulation interval you're
verifying.
Clear as mud?
John
On 02/09/2011 03:05 PM, RAL HelpDesk {for James P Cipriani} wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=44212 >
>
>
> Thanks again, John. This has been most helpful.
>
> One more question, and I'll illustrate with an example:
>
> Let's say I'm verifying accumulated 12-hour accumulated
precipitation but I
> want to use an observation window of +/- 2 hours --> does this mean
that
> the observations (formatted before ASCII2NC) need to have the
accumulation
> interval labelled as "12" for the 2 hours behind or 2 hours ahead?
Or is
> MET tools smart enough to know that if I want to look back 2 hours
(ahead 2
> hours), it means that I wish to utilize the 10-hour (14-hour)
accumulation
> amounts in my verification of 12-hour accumulated precipitation?
>
> Hope that wasn't too confusing.
>
> Thank you.
>
>
>
> From: "RAL HelpDesk {for John Halley Gotway}" <met_help at ucar.edu>
> To: James P Cipriani/Watson/IBM at IBMUS
> Cc: met_help at ucar.edu
> Date: 02/09/2011 12:21 PM
> Subject: Re: [rt.rap.ucar.edu #44212] MODE Tool
>
>
>
> James,
>
> Yep, that's the way I'd do it. Basically, run Point-Stat once for
each
> forecast valid time. When you configure Point-Stat, choose the
> precipitation thresholds that are of interest to you. Also,
> since it sounds like you're using ASCII observations, be sure that
the
> units match up. Precipitation is stored in GRIB files in mm. Just
be sure
> your ASCII observations are consistent. Also be sure
> that the forecast and observation accumulation intervals are
identical.
>
> When you run Point-Stat, make sure you're writing out the CTC and
CTS line
> types - this is set in the "output_flag" parameter of the config
file.
> Once you're run Point-Stat for all the forecast
> times, you can use the STAT-Analysis tools aggregate your results
through
> time. For example:
>
> stat_analysis -lookin out/point_stat -job aggregate_stat -line_type
CTC
> -out_line_type CTS -fcst_var APCP_12 -fcst_lead 12 -dump_row
> ctc_to_cts_job.stat
>
> The job listed above will start at the directory "out/point_stat"
and then
> recursively search through its tree looking for files ending in
".stat".
> It'll read each ".stat" file, looking for
> contingency table count (-line_type CTC) lines for 12-hour
accumulated
> precipitation (-fcst_var APCP_12) with a forecast lead time of 12
hours
> (-fcst_lead 12). It'll read all those contingency
> tables, sum them up into a single one, and then compute the
corresponding
> contingency table statistics (-out_line_type CTS). Also, it'll
write the
> lines used by this job out to a file named
> "ctc_to_cts_job.stat". When setting up a job, it's always a good
idea to
> use the -dump_row option. Then, look at that file to make sure the
> aggregation performed uses the lines you intended. For
> example, perhaps you've used multiple interpolation methods or
multiple
> verification regions, and you didn't intend to aggregate those
results
> together. You can check to see if you've included lines
> you didn't intend. And then you can add more filtering criteria to
the job
> to fix it.
>
> You should of course set up the STAT-Analysis job to do the type of
> aggregation you want. In my example, I aggregated all the 12-hour
lead
> times together. But you can do whatever aggregation you'd
> like by setting the filter criteria yourself.
>
> Hope that helps get you started.
>
> John
>
> On 02/09/2011 09:55 AM, RAL HelpDesk {for James P Cipriani} wrote:
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=44212 >
>>
>>
>> Thanks for the quick response, John. That was helpful.
>>
>> Basically, I would like to obtain contingency table stats for
categorical
>> variables (ie precipitation). I know that I can use point-stat and
then
>> stat-analysis to aggregate the stats - is this the most efficient
way to
> do
>> so when using point observations?
>>
>> Thanks.
>>
>>
>>
>> From: "RAL HelpDesk {for John Halley Gotway}"
> <met_help at ucar.edu>
>> To: James P Cipriani/Watson/IBM at IBMUS
>> Cc: met_help at ucar.edu
>> Date: 02/09/2011 11:09 AM
>> Subject: Re: [rt.rap.ucar.edu #44212] MODE Tool
>>
>>
>>
>> James,
>>
>> The short answer is no.
>>
>> MODE is a tool within MET that performs an object-based
verification
>> approach. Based on the configuration you set up, it defines
"objects" in
>> the forecast and observed fields and compares those
>> objects to one another. It can only define objects over a spatial
area -
>> meaning that it can only be running using two input gridded fields.
The
>> ASCII2NC tool is used to reformat point observations
>> for use by the Point-Stat tool to verify a gridded forecast. The
output
> of
>> ASCII2NC is just point observations - and they cannot be used by
the MODE
>> tool.
>>
>> Some users do take dense networks of point observations and convert
them
>> into gridded fields. If you were to do that, presumably, you could
run
>> that gridded data through MODE. However, the MET
>> package itself doesn't provide tools for converting point
observations
> into
>> a gridded field since that's not strictly verification and there
are
> often
>> a lot of problems doing so.
>>
>> Hope that helps clarify.
>>
>> John
>>
>> On 02/09/2011 08:13 AM, RAL HelpDesk {for James P Cipriani} wrote:
>>>
>>> Wed Feb 09 08:13:05 2011: Request 44212 was acted upon.
>>> Transaction: Ticket created by jpcipria at us.ibm.com
>>> Queue: met_help
>>> Subject: MODE Tool
>>> Owner: Nobody
>>> Requestors: jpcipria at us.ibm.com
>>> Status: new
>>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=44212 >
>>>
>>>
>>>
>>>
>>> MET team,
>>>
>>> I have a question regarding the MODE tool for precipitation
> verification:
>>>
>>> I noticed that the input must be GRIB forecast files and netCDF
>> observation
>>> files (from PCP-combine). My question is this: is it possible to
> utilize
>>> the netCDF output from the ASCII2NC tool as the observation file
for the
>>> MODE tool?
>>>
>>>
>>> Thanks.
>>
>>
>
>
------------------------------------------------
More information about the Met_help
mailing list