[Met_help] [rt.rap.ucar.edu #96797] History for taylor diagram with obs threshold option
John Halley Gotway via RT
met_help at ucar.edu
Tue Sep 22 08:20:52 MDT 2020
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
To Whom It May Concern:
I created a successful Taylor diagram, 20200921_184324.
I also want to add additional fix conditions to see how the diagram changes
according to different thresholds "OBS THRESHOLD". But it failed
20200921_184803
The purpose is to see model performance in the condition of forecasting low
PM concentration (normal day) and high PM concentration (such as wildfire).
Ho-Chun Huang
IMSG at NOAA/NWS/NCEP/EMC
5830 University Research Ct., Rm. 2792
College Park, MD 20740
Ho-Chun.Huang at noaa.gov <Joe.Smith at noaa.gov>
301-683-3958
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: taylor diagram with obs threshold option
From: John Halley Gotway
Time: Mon Sep 21 16:28:27 2020
Ho-Chun,
I see you have a question about the Taylor Diagram in METviewer. I
haven't
created Taylor Diagrams much myself, but some googling reveals that
they
plot continuous statistics like correlations and standard deviations.
They
DO NOT contain categorical statistics.
Defining an OBS_THRESH setting = >15 would only work if you configured
Point-Stat or Grid-Stat to apply a "cnt_thresh" filtering threshold.
It
would not work if ">15" was used to define a categorical threshold
with
which CTC and CTS output line types were generated.
I did some testing with your database and strongly suspect the latter
is
the case. You applied thresholds of >5, >10, >12, >15, >20, >25, and
>35 to
do categorical verification, not as continuous filtering thresholds.
With categorical data, you can create a "Performance Diagram" instead
of a
"Taylor Diagram". Please see that attached example. Note that in
plot_hochun_perf_example, I did not carefully select init and lead
times as
you'd done in the Taylor Diagram example you sent.
In the Performance Diagram, the closer the points are to the upper-
right
corner, the better.
Thanks,
John
On Mon, Sep 21, 2020 at 12:50 PM Ho-Chun Huang - NOAA Affiliate via RT
<
met_help at ucar.edu> wrote:
>
> Mon Sep 21 12:50:26 2020: Request 96797 was acted upon.
> Transaction: Ticket created by ho-chun.huang at noaa.gov
> Queue: met_help
> Subject: taylor diagram with obs threshold option
> Owner: Nobody
> Requestors: ho-chun.huang at noaa.gov
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96797 >
>
>
> To Whom It May Concern:
>
> I created a successful Taylor diagram, 20200921_184324.
>
> I also want to add additional fix conditions to see how the diagram
changes
> according to different thresholds "OBS THRESHOLD". But it failed
> 20200921_184803
>
> The purpose is to see model performance in the condition of
forecasting low
> PM concentration (normal day) and high PM concentration (such as
wildfire).
>
> Ho-Chun Huang
>
> IMSG at NOAA/NWS/NCEP/EMC
>
> 5830 University Research Ct., Rm. 2792
>
> College Park, MD 20740
>
> Ho-Chun.Huang at noaa.gov <Joe.Smith at noaa.gov>
>
> 301-683-3958
>
>
------------------------------------------------
Subject: taylor diagram with obs threshold option
From: Ho-Chun Huang - NOAA Affiliate
Time: Tue Sep 22 06:26:21 2020
Hi, John:
Thanks for the description of the Taylor's and Performance diagram.
The
reason I want to check is that I saw 6 exp models show different
comparison
results in CSI plot for low and high observed PM concentration areas.
I will research using cnt_thresh to filter out data at the very
beginning
in MET.
Please close the ticket.
Ho-Chun Huang
IMSG at NOAA/NWS/NCEP/EMC
5830 University Research Ct., Rm. 2792
College Park, MD 20740
Ho-Chun.Huang at noaa.gov <Joe.Smith at noaa.gov>
301-683-3958
On Mon, Sep 21, 2020 at 6:29 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Ho-Chun,
>
> I see you have a question about the Taylor Diagram in METviewer. I
haven't
> created Taylor Diagrams much myself, but some googling reveals that
they
> plot continuous statistics like correlations and standard
deviations. They
> DO NOT contain categorical statistics.
>
> Defining an OBS_THRESH setting = >15 would only work if you
configured
> Point-Stat or Grid-Stat to apply a "cnt_thresh" filtering threshold.
It
> would not work if ">15" was used to define a categorical threshold
with
> which CTC and CTS output line types were generated.
>
> I did some testing with your database and strongly suspect the
latter is
> the case. You applied thresholds of >5, >10, >12, >15, >20, >25, and
>35 to
> do categorical verification, not as continuous filtering thresholds.
>
> With categorical data, you can create a "Performance Diagram"
instead of a
> "Taylor Diagram". Please see that attached example. Note that in
> plot_hochun_perf_example, I did not carefully select init and lead
times as
> you'd done in the Taylor Diagram example you sent.
>
> In the Performance Diagram, the closer the points are to the upper-
right
> corner, the better.
>
> Thanks,
> John
>
> On Mon, Sep 21, 2020 at 12:50 PM Ho-Chun Huang - NOAA Affiliate via
RT <
> met_help at ucar.edu> wrote:
>
> >
> > Mon Sep 21 12:50:26 2020: Request 96797 was acted upon.
> > Transaction: Ticket created by ho-chun.huang at noaa.gov
> > Queue: met_help
> > Subject: taylor diagram with obs threshold option
> > Owner: Nobody
> > Requestors: ho-chun.huang at noaa.gov
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96797 >
> >
> >
> > To Whom It May Concern:
> >
> > I created a successful Taylor diagram, 20200921_184324.
> >
> > I also want to add additional fix conditions to see how the
diagram
> changes
> > according to different thresholds "OBS THRESHOLD". But it failed
> > 20200921_184803
> >
> > The purpose is to see model performance in the condition of
forecasting
> low
> > PM concentration (normal day) and high PM concentration (such as
> wildfire).
> >
> > Ho-Chun Huang
> >
> > IMSG at NOAA/NWS/NCEP/EMC
> >
> > 5830 University Research Ct., Rm. 2792
> >
> > College Park, MD 20740
> >
> > Ho-Chun.Huang at noaa.gov <Joe.Smith at noaa.gov>
> >
> > 301-683-3958
> >
> >
>
>
------------------------------------------------
Subject: taylor diagram with obs threshold option
From: John Halley Gotway
Time: Tue Sep 22 08:20:40 2020
Ho-Chun,
Will do. If you search for "cnt_thresh" on this page, you'll find a
bit
more information about it.
https://dtcenter.github.io/MET/Users_Guide/data_io.html
It's used to FILTER down the matched pairs to only those which meet
your
criteria. The filter can be defined for the "fcst", the "obs", or in
both
places. The "cnt_logic" option defines how they should be combined.
For
example, do you want cases of forecast temperature > 273, observed
temperature > 273, or only cases where both are true. You still get
out the
same continuous statistics in the CNT line type... but only using
pairs
where those filtering conditions are satisfied.
Thanks,
John
On Tue, Sep 22, 2020 at 6:26 AM Ho-Chun Huang - NOAA Affiliate via RT
<
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96797 >
>
> Hi, John:
>
> Thanks for the description of the Taylor's and Performance diagram.
The
> reason I want to check is that I saw 6 exp models show different
comparison
> results in CSI plot for low and high observed PM concentration
areas.
>
> I will research using cnt_thresh to filter out data at the very
beginning
> in MET.
>
> Please close the ticket.
>
> Ho-Chun Huang
>
> IMSG at NOAA/NWS/NCEP/EMC
>
> 5830 University Research Ct., Rm. 2792
>
> College Park, MD 20740
>
> Ho-Chun.Huang at noaa.gov <Joe.Smith at noaa.gov>
>
> 301-683-3958
>
>
> On Mon, Sep 21, 2020 at 6:29 PM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > Ho-Chun,
> >
> > I see you have a question about the Taylor Diagram in METviewer. I
> haven't
> > created Taylor Diagrams much myself, but some googling reveals
that they
> > plot continuous statistics like correlations and standard
deviations.
> They
> > DO NOT contain categorical statistics.
> >
> > Defining an OBS_THRESH setting = >15 would only work if you
configured
> > Point-Stat or Grid-Stat to apply a "cnt_thresh" filtering
threshold. It
> > would not work if ">15" was used to define a categorical threshold
with
> > which CTC and CTS output line types were generated.
> >
> > I did some testing with your database and strongly suspect the
latter is
> > the case. You applied thresholds of >5, >10, >12, >15, >20, >25,
and >35
> to
> > do categorical verification, not as continuous filtering
thresholds.
> >
> > With categorical data, you can create a "Performance Diagram"
instead of
> a
> > "Taylor Diagram". Please see that attached example. Note that in
> > plot_hochun_perf_example, I did not carefully select init and lead
times
> as
> > you'd done in the Taylor Diagram example you sent.
> >
> > In the Performance Diagram, the closer the points are to the
upper-right
> > corner, the better.
> >
> > Thanks,
> > John
> >
> > On Mon, Sep 21, 2020 at 12:50 PM Ho-Chun Huang - NOAA Affiliate
via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > Mon Sep 21 12:50:26 2020: Request 96797 was acted upon.
> > > Transaction: Ticket created by ho-chun.huang at noaa.gov
> > > Queue: met_help
> > > Subject: taylor diagram with obs threshold option
> > > Owner: Nobody
> > > Requestors: ho-chun.huang at noaa.gov
> > > Status: new
> > > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96797
> >
> > >
> > >
> > > To Whom It May Concern:
> > >
> > > I created a successful Taylor diagram, 20200921_184324.
> > >
> > > I also want to add additional fix conditions to see how the
diagram
> > changes
> > > according to different thresholds "OBS THRESHOLD". But it
failed
> > > 20200921_184803
> > >
> > > The purpose is to see model performance in the condition of
forecasting
> > low
> > > PM concentration (normal day) and high PM concentration (such as
> > wildfire).
> > >
> > > Ho-Chun Huang
> > >
> > > IMSG at NOAA/NWS/NCEP/EMC
> > >
> > > 5830 University Research Ct., Rm. 2792
> > >
> > > College Park, MD 20740
> > >
> > > Ho-Chun.Huang at noaa.gov <Joe.Smith at noaa.gov>
> > >
> > > 301-683-3958
> > >
> > >
> >
> >
>
>
------------------------------------------------
More information about the Met_help
mailing list