[Met_help] [rt.rap.ucar.edu #88013] History for Matching forecasts among many models in MET-TC
John Halley Gotway via RT
met_help at ucar.edu
Wed Dec 5 15:11:12 MST 2018
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hello,
I am trying to use MET-TC to compare the skill of several models over 3
years. I have noticed that in my tc_stat output there are different
numbers of cases (forecasts) used in the verification for each model. I
believe it is standard to only compare forecasts that are present in all
model being compared, to be fair. Is there a way in tc_pairs or tc_stat to
make sure that I am only using tracks present for all models?
Thank you,
Shannon
--
Shannon Rees
UCAR Visiting Scientist
Geophysical Fluid Dynamics Lab
Princeton University Forrestal Campus
201 Forrestal Rd Princeton, NJ
609-452-5384
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Matching forecasts among many models in MET-TC
From: Shannon Rees - NOAA Affiliate
Time: Tue Dec 04 16:32:28 2018
I saw that in the MET 7.0 manual there is a figure showing
"homogeneous" TC
intensity comparisons, however no mention of how to achieve that in
the
MET-TC tools section. Is there an off-line script that can be used to
homogenize the sample size among models or does every user just make
their
own script to do this?
On Mon, Dec 3, 2018 at 12:01 PM met_help at ucar.edu via RT
<met_help at ucar.edu>
wrote:
> Greetings,
>
> This message has been automatically generated in response to the
> creation of a trouble ticket regarding:
> "Matching forecasts among many models in MET-TC",
> a summary of which appears below.
>
> There is no need to reply to this message right now. Your ticket
has been
> assigned an ID of [rt.rap.ucar.edu #88013].
>
> Please include the string:
>
> [rt.rap.ucar.edu #88013]
>
> in the subject line of all future correspondence about this issue.
To do
> so,
> you may reply to this message.
>
> For more information, please see:
>
> MET Online Tutorial:
>
https://www.dtcenter.org/met/users/support/online_tutorial/index.php
>
> MET Users Guide:
> https://www.dtcenter.org/met/users/docs/overview.php
>
> MET FAQs:
> https://www.dtcenter.org/met/users/support/faqs/index.php
>
> MET-Help Email Archive:
> http://mailman.ucar.edu/pipermail/met_help
>
> Thank you,
> met_help at ucar.edu
>
>
-------------------------------------------------------------------------
> Hello,
>
> I am trying to use MET-TC to compare the skill of several models
over 3
> years. I have noticed that in my tc_stat output there are different
> numbers of cases (forecasts) used in the verification for each
model. I
> believe it is standard to only compare forecasts that are present in
all
> model being compared, to be fair. Is there a way in tc_pairs or
tc_stat to
> make sure that I am only using tracks present for all models?
>
> Thank you,
> Shannon
> --
> Shannon Rees
> UCAR Visiting Scientist
> Geophysical Fluid Dynamics Lab
> Princeton University Forrestal Campus
> 201 Forrestal Rd Princeton, NJ
> 609-452-5384
>
>
--
Shannon Rees
UCAR Visiting Scientist
Geophysical Fluid Dynamics Lab
Princeton University Forrestal Campus
201 Forrestal Rd Princeton, NJ
609-452-5384
------------------------------------------------
Subject: Matching forecasts among many models in MET-TC
From: John Halley Gotway
Time: Wed Dec 05 09:27:00 2018
Shannon,
I see that you have a question about how to produce a homogeneous
sample of
data for the MET-TC data. This is done using the "event_equal" flag
in the
TC-Stat configuration file. Here's a selection from the file
met-7.0/share/met/config/README_TC, which is also included in the
user's
guide:
//
// Specify whether only those cases common to all models in the
dataset
should
// be retained. May modify using the "-event_equal" job command
option.
//
// e.g. event_equal = FALSE;
//
event_equal = FALSE;
We have also included a sample R plotting script in
met-7.0/share/met/Rscripts/plot_tcmpr.R. This R-script does 2 main
things...
- take the user-specified command line filtering options to call the
TC-Stat tool to filter the input data
- create one or more plots of the resulting filtered data
If you're running that R-script then "event_equal = TRUE" by default.
But
the "-no_ee" command line option can be used to switch it back to
false.
Hope that helps clarify.
Thanks,
John Halley Gotway
On Tue, Dec 4, 2018 at 4:33 PM Shannon Rees - NOAA Affiliate via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=88013 >
>
> I saw that in the MET 7.0 manual there is a figure showing
"homogeneous" TC
> intensity comparisons, however no mention of how to achieve that in
the
> MET-TC tools section. Is there an off-line script that can be used
to
> homogenize the sample size among models or does every user just make
their
> own script to do this?
>
> On Mon, Dec 3, 2018 at 12:01 PM met_help at ucar.edu via RT <
> met_help at ucar.edu>
> wrote:
>
> > Greetings,
> >
> > This message has been automatically generated in response to the
> > creation of a trouble ticket regarding:
> > "Matching forecasts among many models in MET-TC",
> > a summary of which appears below.
> >
> > There is no need to reply to this message right now. Your ticket
has
> been
> > assigned an ID of [rt.rap.ucar.edu #88013].
> >
> > Please include the string:
> >
> > [rt.rap.ucar.edu #88013]
> >
> > in the subject line of all future correspondence about this issue.
To do
> > so,
> > you may reply to this message.
> >
> > For more information, please see:
> >
> > MET Online Tutorial:
> >
https://www.dtcenter.org/met/users/support/online_tutorial/index.php
> >
> > MET Users Guide:
> > https://www.dtcenter.org/met/users/docs/overview.php
> >
> > MET FAQs:
> > https://www.dtcenter.org/met/users/support/faqs/index.php
> >
> > MET-Help Email Archive:
> > http://mailman.ucar.edu/pipermail/met_help
> >
> > Thank you,
> > met_help at ucar.edu
> >
> >
-------------------------------------------------------------------------
> > Hello,
> >
> > I am trying to use MET-TC to compare the skill of several models
over 3
> > years. I have noticed that in my tc_stat output there are
different
> > numbers of cases (forecasts) used in the verification for each
model. I
> > believe it is standard to only compare forecasts that are present
in all
> > model being compared, to be fair. Is there a way in tc_pairs or
tc_stat
> to
> > make sure that I am only using tracks present for all models?
> >
> > Thank you,
> > Shannon
> > --
> > Shannon Rees
> > UCAR Visiting Scientist
> > Geophysical Fluid Dynamics Lab
> > Princeton University Forrestal Campus
> > 201 Forrestal Rd Princeton, NJ
> > 609-452-5384
> >
> >
>
> --
> Shannon Rees
> UCAR Visiting Scientist
> Geophysical Fluid Dynamics Lab
> Princeton University Forrestal Campus
> 201 Forrestal Rd Princeton, NJ
> 609-452-5384
>
>
------------------------------------------------
Subject: Matching forecasts among many models in MET-TC
From: Shannon Rees - NOAA Affiliate
Time: Wed Dec 05 14:49:00 2018
John,
Thanks for the explanation. That makes it easy! I guess I was only
looking for options in the tc_pairs tool.
On Wed, Dec 5, 2018 at 11:27 AM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Shannon,
>
> I see that you have a question about how to produce a homogeneous
sample of
> data for the MET-TC data. This is done using the "event_equal" flag
in the
> TC-Stat configuration file. Here's a selection from the file
> met-7.0/share/met/config/README_TC, which is also included in the
user's
> guide:
>
> //
> // Specify whether only those cases common to all models in the
dataset
> should
> // be retained. May modify using the "-event_equal" job command
option.
> //
> // e.g. event_equal = FALSE;
> //
> event_equal = FALSE;
>
> We have also included a sample R plotting script in
> met-7.0/share/met/Rscripts/plot_tcmpr.R. This R-script does 2 main
> things...
> - take the user-specified command line filtering options to call the
> TC-Stat tool to filter the input data
> - create one or more plots of the resulting filtered data
>
> If you're running that R-script then "event_equal = TRUE" by
default. But
> the "-no_ee" command line option can be used to switch it back to
false.
>
> Hope that helps clarify.
>
> Thanks,
> John Halley Gotway
>
>
> On Tue, Dec 4, 2018 at 4:33 PM Shannon Rees - NOAA Affiliate via RT
<
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=88013 >
> >
> > I saw that in the MET 7.0 manual there is a figure showing
"homogeneous"
> TC
> > intensity comparisons, however no mention of how to achieve that
in the
> > MET-TC tools section. Is there an off-line script that can be
used to
> > homogenize the sample size among models or does every user just
make
> their
> > own script to do this?
> >
> > On Mon, Dec 3, 2018 at 12:01 PM met_help at ucar.edu via RT <
> > met_help at ucar.edu>
> > wrote:
> >
> > > Greetings,
> > >
> > > This message has been automatically generated in response to the
> > > creation of a trouble ticket regarding:
> > > "Matching forecasts among many models in MET-TC",
> > > a summary of which appears below.
> > >
> > > There is no need to reply to this message right now. Your
ticket has
> > been
> > > assigned an ID of [rt.rap.ucar.edu #88013].
> > >
> > > Please include the string:
> > >
> > > [rt.rap.ucar.edu #88013]
> > >
> > > in the subject line of all future correspondence about this
issue. To
> do
> > > so,
> > > you may reply to this message.
> > >
> > > For more information, please see:
> > >
> > > MET Online Tutorial:
> > >
> https://www.dtcenter.org/met/users/support/online_tutorial/index.php
> > >
> > > MET Users Guide:
> > > https://www.dtcenter.org/met/users/docs/overview.php
> > >
> > > MET FAQs:
> > > https://www.dtcenter.org/met/users/support/faqs/index.php
> > >
> > > MET-Help Email Archive:
> > > http://mailman.ucar.edu/pipermail/met_help
> > >
> > > Thank you,
> > > met_help at ucar.edu
> > >
> > >
>
-------------------------------------------------------------------------
> > > Hello,
> > >
> > > I am trying to use MET-TC to compare the skill of several models
over 3
> > > years. I have noticed that in my tc_stat output there are
different
> > > numbers of cases (forecasts) used in the verification for each
model.
> I
> > > believe it is standard to only compare forecasts that are
present in
> all
> > > model being compared, to be fair. Is there a way in tc_pairs or
> tc_stat
> > to
> > > make sure that I am only using tracks present for all models?
> > >
> > > Thank you,
> > > Shannon
> > > --
> > > Shannon Rees
> > > UCAR Visiting Scientist
> > > Geophysical Fluid Dynamics Lab
> > > Princeton University Forrestal Campus
> > > 201 Forrestal Rd Princeton, NJ
> > > 609-452-5384
> > >
> > >
> >
> > --
> > Shannon Rees
> > UCAR Visiting Scientist
> > Geophysical Fluid Dynamics Lab
> > Princeton University Forrestal Campus
> > 201 Forrestal Rd Princeton, NJ
> > 609-452-5384
> >
> >
>
>
--
Shannon Rees
UCAR Visiting Scientist
Geophysical Fluid Dynamics Lab
Princeton University Forrestal Campus
201 Forrestal Rd Princeton, NJ
609-452-5384
------------------------------------------------
Subject: Matching forecasts among many models in MET-TC
From: John Halley Gotway
Time: Wed Dec 05 15:11:00 2018
Sure thing. I'll go ahead and resolve the ticket. But just let us
know
what other questions or issues arise.
Thanks,
John
On Wed, Dec 5, 2018 at 2:49 PM Shannon Rees - NOAA Affiliate via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=88013 >
>
> John,
>
> Thanks for the explanation. That makes it easy! I guess I was only
> looking for options in the tc_pairs tool.
>
>
>
>
> On Wed, Dec 5, 2018 at 11:27 AM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > Shannon,
> >
> > I see that you have a question about how to produce a homogeneous
sample
> of
> > data for the MET-TC data. This is done using the "event_equal"
flag in
> the
> > TC-Stat configuration file. Here's a selection from the file
> > met-7.0/share/met/config/README_TC, which is also included in the
user's
> > guide:
> >
> > //
> > // Specify whether only those cases common to all models in the
dataset
> > should
> > // be retained. May modify using the "-event_equal" job command
option.
> > //
> > // e.g. event_equal = FALSE;
> > //
> > event_equal = FALSE;
> >
> > We have also included a sample R plotting script in
> > met-7.0/share/met/Rscripts/plot_tcmpr.R. This R-script does 2
main
> > things...
> > - take the user-specified command line filtering options to call
the
> > TC-Stat tool to filter the input data
> > - create one or more plots of the resulting filtered data
> >
> > If you're running that R-script then "event_equal = TRUE" by
default.
> But
> > the "-no_ee" command line option can be used to switch it back to
false.
> >
> > Hope that helps clarify.
> >
> > Thanks,
> > John Halley Gotway
> >
> >
> > On Tue, Dec 4, 2018 at 4:33 PM Shannon Rees - NOAA Affiliate via
RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=88013 >
> > >
> > > I saw that in the MET 7.0 manual there is a figure showing
> "homogeneous"
> > TC
> > > intensity comparisons, however no mention of how to achieve that
in the
> > > MET-TC tools section. Is there an off-line script that can be
used to
> > > homogenize the sample size among models or does every user just
make
> > their
> > > own script to do this?
> > >
> > > On Mon, Dec 3, 2018 at 12:01 PM met_help at ucar.edu via RT <
> > > met_help at ucar.edu>
> > > wrote:
> > >
> > > > Greetings,
> > > >
> > > > This message has been automatically generated in response to
the
> > > > creation of a trouble ticket regarding:
> > > > "Matching forecasts among many models in MET-TC",
> > > > a summary of which appears below.
> > > >
> > > > There is no need to reply to this message right now. Your
ticket has
> > > been
> > > > assigned an ID of [rt.rap.ucar.edu #88013].
> > > >
> > > > Please include the string:
> > > >
> > > > [rt.rap.ucar.edu #88013]
> > > >
> > > > in the subject line of all future correspondence about this
issue. To
> > do
> > > > so,
> > > > you may reply to this message.
> > > >
> > > > For more information, please see:
> > > >
> > > > MET Online Tutorial:
> > > >
> >
https://www.dtcenter.org/met/users/support/online_tutorial/index.php
> > > >
> > > > MET Users Guide:
> > > > https://www.dtcenter.org/met/users/docs/overview.php
> > > >
> > > > MET FAQs:
> > > > https://www.dtcenter.org/met/users/support/faqs/index.php
> > > >
> > > > MET-Help Email Archive:
> > > > http://mailman.ucar.edu/pipermail/met_help
> > > >
> > > > Thank you,
> > > > met_help at ucar.edu
> > > >
> > > >
> >
-------------------------------------------------------------------------
> > > > Hello,
> > > >
> > > > I am trying to use MET-TC to compare the skill of several
models
> over 3
> > > > years. I have noticed that in my tc_stat output there are
different
> > > > numbers of cases (forecasts) used in the verification for each
model.
> > I
> > > > believe it is standard to only compare forecasts that are
present in
> > all
> > > > model being compared, to be fair. Is there a way in tc_pairs
or
> > tc_stat
> > > to
> > > > make sure that I am only using tracks present for all models?
> > > >
> > > > Thank you,
> > > > Shannon
> > > > --
> > > > Shannon Rees
> > > > UCAR Visiting Scientist
> > > > Geophysical Fluid Dynamics Lab
> > > > Princeton University Forrestal Campus
> > > > 201 Forrestal Rd Princeton, NJ
> > > > 609-452-5384
> > > >
> > > >
> > >
> > > --
> > > Shannon Rees
> > > UCAR Visiting Scientist
> > > Geophysical Fluid Dynamics Lab
> > > Princeton University Forrestal Campus
> > > 201 Forrestal Rd Princeton, NJ
> > > 609-452-5384
> > >
> > >
> >
> >
>
> --
> Shannon Rees
> UCAR Visiting Scientist
> Geophysical Fluid Dynamics Lab
> Princeton University Forrestal Campus
> 201 Forrestal Rd Princeton, NJ
> 609-452-5384
>
>
------------------------------------------------
More information about the Met_help
mailing list