[Met_help] [rt.rap.ucar.edu #95347] History for Reliability diagram
John Halley Gotway via RT
met_help at ucar.edu
Thu May 28 16:43:29 MDT 2020
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Folks - I get from MET tools OY_TP [observed YES total proportion] and ON_TP [observed NO total proportion] and I'm plotting OY, [.1, ..., .9] as the reliability diagrams. Is this correct?
Efren A. Serra (Contractor)
Physicist
DeVine Consulting, Inc.
Naval Research Laboratory
Marine Meteorology Division
7 Grace Hopper Ave., STOP 2
Monterey, CA 93943
Code 7542
Mobile: 408-425-5027
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Reliability diagram
From: John Halley Gotway
Time: Sun May 24 15:06:47 2020
Efren,
Not quite. Here's a useful link about reliability diagrams from the
WMO:
http://www.bom.gov.au/wmo/lrfvs/reliability.shtml
The points along the X-axis are the forecast probability thresholds..
Since you're using probability bins of 0 to 0.1, 0.1 to 0.2, and so
on, I'd
recommend plotting the center-point of each bin. So along the X-axis,
you'd
plot values at 0.5, 0.15, 0.25, 0.35, 0.45, 0.55, 0.65, 0.75, 0.85,
and
0.95.
The points along the Y-axis are the observed relative frequency
values. And
that's the OY_TP value from the MET output.
I also worked up an example using METviewer on some sample
probabilistic output data:
http://www.dtcenter.org/met/metviewer/metviewer1.jsp
See the attached XML and resulting plot. This shows the reliability
diagrams for probabilities from 3 different ensembles for 1-hour
precip >
0.10". I've also attached the corresponding data file and R-script
used to
generate this plot (also output from METviewer). The templatized R-
script
is rather difficult to parse through, but it might be useful.
Thanks,
John Halley Gotway
On Fri, May 22, 2020 at 6:17 PM efren.serra.ctr at nrlmry.navy.mil via RT
<
met_help at ucar.edu> wrote:
>
> Fri May 22 18:17:18 2020: Request 95347 was acted upon.
> Transaction: Ticket created by efren.serra.ctr at nrlmry.navy.mil
> Queue: met_help
> Subject: Reliability diagram
> Owner: Nobody
> Requestors: efren.serra.ctr at nrlmry.navy.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95347 >
>
>
> Folks - I get from MET tools OY_TP [observed YES total proportion]
and
> ON_TP [observed NO total proportion] and I'm plotting OY, [.1, ...,
.9] as
> the reliability diagrams. Is this correct?
>
> Efren A. Serra (Contractor)
> Physicist
>
> DeVine Consulting, Inc.
> Naval Research Laboratory
> Marine Meteorology Division
> 7 Grace Hopper Ave., STOP 2
> Monterey, CA 93943
> Code 7542
> Mobile: 408-425-5027
>
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #95347] Reliability diagram
From: efren.serra.ctr at nrlmry.navy.mil
Time: Mon May 25 15:05:39 2020
John - Then, according to WMO link below, the plot of
OY_TP[i]/(OY_TP[i]+ON_TP[i]) versus the .5, .15, ..., .95 would be the
reliability diagram, isn't it?
-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Sunday, May 24, 2020 2:07 PM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #95347] Reliability diagram
Efren,
Not quite. Here's a useful link about reliability diagrams from the
WMO:
http://www.bom.gov.au/wmo/lrfvs/reliability.shtml
The points along the X-axis are the forecast probability thresholds..
Since you're using probability bins of 0 to 0.1, 0.1 to 0.2, and so
on, I'd recommend plotting the center-point of each bin. So along the
X-axis, you'd plot values at 0.5, 0.15, 0.25, 0.35, 0.45, 0.55, 0.65,
0.75, 0.85, and 0.95.
The points along the Y-axis are the observed relative frequency
values. And that's the OY_TP value from the MET output.
I also worked up an example using METviewer on some sample
probabilistic output data:
http://www.dtcenter.org/met/metviewer/metviewer1.jsp
See the attached XML and resulting plot. This shows the reliability
diagrams for probabilities from 3 different ensembles for 1-hour
precip > 0.10". I've also attached the corresponding data file and R-
script used to generate this plot (also output from METviewer). The
templatized R-script is rather difficult to parse through, but it
might be useful.
Thanks,
John Halley Gotway
On Fri, May 22, 2020 at 6:17 PM efren.serra.ctr at nrlmry.navy.mil via RT
< met_help at ucar.edu> wrote:
>
> Fri May 22 18:17:18 2020: Request 95347 was acted upon.
> Transaction: Ticket created by efren.serra.ctr at nrlmry.navy.mil
> Queue: met_help
> Subject: Reliability diagram
> Owner: Nobody
> Requestors: efren.serra.ctr at nrlmry.navy.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95347
> >
>
>
> Folks - I get from MET tools OY_TP [observed YES total proportion]
and
> ON_TP [observed NO total proportion] and I'm plotting OY, [.1, ...,
> .9] as the reliability diagrams. Is this correct?
>
> Efren A. Serra (Contractor)
> Physicist
>
> DeVine Consulting, Inc.
> Naval Research Laboratory
> Marine Meteorology Division
> 7 Grace Hopper Ave., STOP 2
> Monterey, CA 93943
> Code 7542
> Mobile: 408-425-5027
>
>
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #95347] Reliability diagram
From: efren.serra.ctr at nrlmry.navy.mil
Time: Mon May 25 23:41:24 2020
John - I should point out that I'm using v8.1 of MET tools and that
I'm parsing *pjc.txt* output files.
-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Sunday, May 24, 2020 2:07 PM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #95347] Reliability diagram
Efren,
Not quite. Here's a useful link about reliability diagrams from the
WMO:
http://www.bom.gov.au/wmo/lrfvs/reliability.shtml
The points along the X-axis are the forecast probability thresholds..
Since you're using probability bins of 0 to 0.1, 0.1 to 0.2, and so
on, I'd recommend plotting the center-point of each bin. So along the
X-axis, you'd plot values at 0.5, 0.15, 0.25, 0.35, 0.45, 0.55, 0.65,
0.75, 0.85, and 0.95.
The points along the Y-axis are the observed relative frequency
values. And that's the OY_TP value from the MET output.
I also worked up an example using METviewer on some sample
probabilistic output data:
http://www.dtcenter.org/met/metviewer/metviewer1.jsp
See the attached XML and resulting plot. This shows the reliability
diagrams for probabilities from 3 different ensembles for 1-hour
precip > 0.10". I've also attached the corresponding data file and R-
script used to generate this plot (also output from METviewer). The
templatized R-script is rather difficult to parse through, but it
might be useful.
Thanks,
John Halley Gotway
On Fri, May 22, 2020 at 6:17 PM efren.serra.ctr at nrlmry.navy.mil via RT
< met_help at ucar.edu> wrote:
>
> Fri May 22 18:17:18 2020: Request 95347 was acted upon.
> Transaction: Ticket created by efren.serra.ctr at nrlmry.navy.mil
> Queue: met_help
> Subject: Reliability diagram
> Owner: Nobody
> Requestors: efren.serra.ctr at nrlmry.navy.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95347
> >
>
>
> Folks - I get from MET tools OY_TP [observed YES total proportion]
and
> ON_TP [observed NO total proportion] and I'm plotting OY, [.1, ...,
> .9] as the reliability diagrams. Is this correct?
>
> Efren A. Serra (Contractor)
> Physicist
>
> DeVine Consulting, Inc.
> Naval Research Laboratory
> Marine Meteorology Division
> 7 Grace Hopper Ave., STOP 2
> Monterey, CA 93943
> Code 7542
> Mobile: 408-425-5027
>
>
>
------------------------------------------------
Subject: Reliability diagram
From: Julie Prestopnik
Time: Tue May 26 13:09:44 2020
Hi Efren. I just wanted to let you know that John is out of the
office
until later this week. Unfortunately, I am unable to answer your
question. I'm sure you will hear from John shortly after he returns
to
work.
Julie
On Mon, May 25, 2020 at 11:42 PM efren.serra.ctr at nrlmry.navy.mil via
RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95347 >
>
> John - I should point out that I'm using v8.1 of MET tools and that
I'm
> parsing *pjc.txt* output files.
>
> -----Original Message-----
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Sunday, May 24, 2020 2:07 PM
> To: Serra, Mr. Efren, Contractor, Code 7531 <
> efren.serra.ctr at nrlmry.navy.mil>
> Subject: Re: [rt.rap.ucar.edu #95347] Reliability diagram
>
> Efren,
>
> Not quite. Here's a useful link about reliability diagrams from the
WMO:
> http://www.bom.gov.au/wmo/lrfvs/reliability.shtml
>
> The points along the X-axis are the forecast probability
thresholds..
> Since you're using probability bins of 0 to 0.1, 0.1 to 0.2, and so
on,
> I'd recommend plotting the center-point of each bin. So along the X-
axis,
> you'd plot values at 0.5, 0.15, 0.25, 0.35, 0.45, 0.55, 0.65, 0.75,
0.85,
> and 0.95.
>
> The points along the Y-axis are the observed relative frequency
values.
> And that's the OY_TP value from the MET output.
>
> I also worked up an example using METviewer on some sample
probabilistic
> output data:
> http://www.dtcenter.org/met/metviewer/metviewer1.jsp
>
> See the attached XML and resulting plot. This shows the reliability
> diagrams for probabilities from 3 different ensembles for 1-hour
precip >
> 0.10". I've also attached the corresponding data file and R-script
used to
> generate this plot (also output from METviewer). The templatized R-
script
> is rather difficult to parse through, but it might be useful.
>
> Thanks,
> John Halley Gotway
>
> On Fri, May 22, 2020 at 6:17 PM efren.serra.ctr at nrlmry.navy.mil via
RT <
> met_help at ucar.edu> wrote:
>
> >
> > Fri May 22 18:17:18 2020: Request 95347 was acted upon.
> > Transaction: Ticket created by efren.serra.ctr at nrlmry.navy.mil
> > Queue: met_help
> > Subject: Reliability diagram
> > Owner: Nobody
> > Requestors: efren.serra.ctr at nrlmry.navy.mil
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95347
> > >
> >
> >
> > Folks - I get from MET tools OY_TP [observed YES total proportion]
and
> > ON_TP [observed NO total proportion] and I'm plotting OY, [.1,
...,
> > .9] as the reliability diagrams. Is this correct?
> >
> > Efren A. Serra (Contractor)
> > Physicist
> >
> > DeVine Consulting, Inc.
> > Naval Research Laboratory
> > Marine Meteorology Division
> > 7 Grace Hopper Ave., STOP 2
> > Monterey, CA 93943
> > Code 7542
> > Mobile: 408-425-5027
> >
> >
> >
>
>
>
>
--
Julie Prestopnik
Software Engineer
National Center for Atmospheric Research
Research Applications Laboratory
Phone: 303.497.8399
Email: jpresto at ucar.edu
My working day may not be your working day. Please do not feel
obliged to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #95347] Reliability diagram
From: efren.serra.ctr at nrlmry.navy.mil
Time: Tue May 26 13:22:27 2020
Thanks!
-----Original Message-----
From: Julie Prestopnik via RT <met_help at ucar.edu>
Sent: Tuesday, May 26, 2020 12:10 PM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #95347] Reliability diagram
Hi Efren. I just wanted to let you know that John is out of the
office until later this week. Unfortunately, I am unable to answer
your question. I'm sure you will hear from John shortly after he
returns to work.
Julie
On Mon, May 25, 2020 at 11:42 PM efren.serra.ctr at nrlmry.navy.mil via
RT < met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95347 >
>
> John - I should point out that I'm using v8.1 of MET tools and that
> I'm parsing *pjc.txt* output files.
>
> -----Original Message-----
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Sunday, May 24, 2020 2:07 PM
> To: Serra, Mr. Efren, Contractor, Code 7531 <
> efren.serra.ctr at nrlmry.navy.mil>
> Subject: Re: [rt.rap.ucar.edu #95347] Reliability diagram
>
> Efren,
>
> Not quite. Here's a useful link about reliability diagrams from the
WMO:
> http://www.bom.gov.au/wmo/lrfvs/reliability.shtml
>
> The points along the X-axis are the forecast probability
thresholds..
> Since you're using probability bins of 0 to 0.1, 0.1 to 0.2, and so
> on, I'd recommend plotting the center-point of each bin. So along
the
> X-axis, you'd plot values at 0.5, 0.15, 0.25, 0.35, 0.45, 0.55,
0.65,
> 0.75, 0.85, and 0.95.
>
> The points along the Y-axis are the observed relative frequency
values.
> And that's the OY_TP value from the MET output.
>
> I also worked up an example using METviewer on some sample
> probabilistic output data:
> http://www.dtcenter.org/met/metviewer/metviewer1.jsp
>
> See the attached XML and resulting plot. This shows the reliability
> diagrams for probabilities from 3 different ensembles for 1-hour
> precip > 0.10". I've also attached the corresponding data file and
> R-script used to generate this plot (also output from METviewer).
The
> templatized R-script is rather difficult to parse through, but it
might be useful.
>
> Thanks,
> John Halley Gotway
>
> On Fri, May 22, 2020 at 6:17 PM efren.serra.ctr at nrlmry.navy.mil via
RT
> < met_help at ucar.edu> wrote:
>
> >
> > Fri May 22 18:17:18 2020: Request 95347 was acted upon.
> > Transaction: Ticket created by efren.serra.ctr at nrlmry.navy.mil
> > Queue: met_help
> > Subject: Reliability diagram
> > Owner: Nobody
> > Requestors: efren.serra.ctr at nrlmry.navy.mil
> > Status: new
> > Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95347
> > >
> >
> >
> > Folks - I get from MET tools OY_TP [observed YES total proportion]
> > and ON_TP [observed NO total proportion] and I'm plotting OY, [.1,
> > ..., .9] as the reliability diagrams. Is this correct?
> >
> > Efren A. Serra (Contractor)
> > Physicist
> >
> > DeVine Consulting, Inc.
> > Naval Research Laboratory
> > Marine Meteorology Division
> > 7 Grace Hopper Ave., STOP 2
> > Monterey, CA 93943
> > Code 7542
> > Mobile: 408-425-5027
> >
> >
> >
>
>
>
>
--
Julie Prestopnik
Software Engineer
National Center for Atmospheric Research Research Applications
Laboratory
Phone: 303.497.8399
Email: jpresto at ucar.edu
My working day may not be your working day. Please do not feel
obliged to reply to this email outside of your normal working hours.
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #95347] Reliability diagram
From: efren.serra.ctr at nrlmry.navy.mil
Time: Wed May 27 17:35:44 2020
Julie - In some R code that John sent me, there's this file I need:
setwd("/opt/vxwww/tomcat/webapps/metviewer/R_work/");
source("include/util_plot.R"); *<<< This one. >>>*
I was wondering if you could send it to me. Thank you.
-----Original Message-----
From: Julie Prestopnik via RT <met_help at ucar.edu>
Sent: Tuesday, May 26, 2020 12:10 PM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #95347] Reliability diagram
Hi Efren. I just wanted to let you know that John is out of the
office until later this week. Unfortunately, I am unable to answer
your question. I'm sure you will hear from John shortly after he
returns to work.
Julie
On Mon, May 25, 2020 at 11:42 PM efren.serra.ctr at nrlmry.navy.mil via
RT < met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95347 >
>
> John - I should point out that I'm using v8.1 of MET tools and that
> I'm parsing *pjc.txt* output files.
>
> -----Original Message-----
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Sunday, May 24, 2020 2:07 PM
> To: Serra, Mr. Efren, Contractor, Code 7531 <
> efren.serra.ctr at nrlmry.navy.mil>
> Subject: Re: [rt.rap.ucar.edu #95347] Reliability diagram
>
> Efren,
>
> Not quite. Here's a useful link about reliability diagrams from the
WMO:
> http://www.bom.gov.au/wmo/lrfvs/reliability.shtml
>
> The points along the X-axis are the forecast probability
thresholds..
> Since you're using probability bins of 0 to 0.1, 0.1 to 0.2, and so
> on, I'd recommend plotting the center-point of each bin. So along
the
> X-axis, you'd plot values at 0.5, 0.15, 0.25, 0.35, 0.45, 0.55,
0.65,
> 0.75, 0.85, and 0.95.
>
> The points along the Y-axis are the observed relative frequency
values.
> And that's the OY_TP value from the MET output.
>
> I also worked up an example using METviewer on some sample
> probabilistic output data:
> http://www.dtcenter.org/met/metviewer/metviewer1.jsp
>
> See the attached XML and resulting plot. This shows the reliability
> diagrams for probabilities from 3 different ensembles for 1-hour
> precip > 0.10". I've also attached the corresponding data file and
> R-script used to generate this plot (also output from METviewer).
The
> templatized R-script is rather difficult to parse through, but it
might be useful.
>
> Thanks,
> John Halley Gotway
>
> On Fri, May 22, 2020 at 6:17 PM efren.serra.ctr at nrlmry.navy.mil via
RT
> < met_help at ucar.edu> wrote:
>
> >
> > Fri May 22 18:17:18 2020: Request 95347 was acted upon.
> > Transaction: Ticket created by efren.serra.ctr at nrlmry.navy.mil
> > Queue: met_help
> > Subject: Reliability diagram
> > Owner: Nobody
> > Requestors: efren.serra.ctr at nrlmry.navy.mil
> > Status: new
> > Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95347
> > >
> >
> >
> > Folks - I get from MET tools OY_TP [observed YES total proportion]
> > and ON_TP [observed NO total proportion] and I'm plotting OY, [.1,
> > ..., .9] as the reliability diagrams. Is this correct?
> >
> > Efren A. Serra (Contractor)
> > Physicist
> >
> > DeVine Consulting, Inc.
> > Naval Research Laboratory
> > Marine Meteorology Division
> > 7 Grace Hopper Ave., STOP 2
> > Monterey, CA 93943
> > Code 7542
> > Mobile: 408-425-5027
> >
> >
> >
>
>
>
>
--
Julie Prestopnik
Software Engineer
National Center for Atmospheric Research Research Applications
Laboratory
Phone: 303.497.8399
Email: jpresto at ucar.edu
My working day may not be your working day. Please do not feel
obliged to reply to this email outside of your normal working hours.
------------------------------------------------
Subject: Reliability diagram
From: George McCabe
Time: Wed May 27 18:30:03 2020
Hi,
John and Julie are both out of the office today, but I can do my best
to
help.
The util_plot.R script can be found in the METViewer repository:
https://github.com/NCAR/METviewer/blob/master/R_work/include/util_plot.R
The command before the source line that sets the working directory to
/opt/vxwww/tomcat/webapps/metviewer/R_work/ should allow the script to
find
the file in the include directory. I am not sure why that file would
not be
in that directory. I may not be understanding the issue correctly. Let
me
know if that doesn't help. We may have to wait until John gets back.
Thanks,
George
On Wed, May 27, 2020 at 5:35 PM efren.serra.ctr at nrlmry.navy.mil via RT
<
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95347 >
>
> Julie - In some R code that John sent me, there's this file I need:
>
> setwd("/opt/vxwww/tomcat/webapps/metviewer/R_work/");
> source("include/util_plot.R"); *<<< This one. >>>*
>
> I was wondering if you could send it to me. Thank you.
>
>
> -----Original Message-----
> From: Julie Prestopnik via RT <met_help at ucar.edu>
> Sent: Tuesday, May 26, 2020 12:10 PM
> To: Serra, Mr. Efren, Contractor, Code 7531 <
> efren.serra.ctr at nrlmry.navy.mil>
> Subject: Re: [rt.rap.ucar.edu #95347] Reliability diagram
>
> Hi Efren. I just wanted to let you know that John is out of the
office
> until later this week. Unfortunately, I am unable to answer your
> question. I'm sure you will hear from John shortly after he returns
to
> work.
>
> Julie
>
> On Mon, May 25, 2020 at 11:42 PM efren.serra.ctr at nrlmry.navy.mil via
RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95347 >
> >
> > John - I should point out that I'm using v8.1 of MET tools and
that
> > I'm parsing *pjc.txt* output files.
> >
> > -----Original Message-----
> > From: John Halley Gotway via RT <met_help at ucar.edu>
> > Sent: Sunday, May 24, 2020 2:07 PM
> > To: Serra, Mr. Efren, Contractor, Code 7531 <
> > efren.serra.ctr at nrlmry.navy.mil>
> > Subject: Re: [rt.rap.ucar.edu #95347] Reliability diagram
> >
> > Efren,
> >
> > Not quite. Here's a useful link about reliability diagrams from
the WMO:
> > http://www.bom.gov.au/wmo/lrfvs/reliability.shtml
> >
> > The points along the X-axis are the forecast probability
thresholds..
> > Since you're using probability bins of 0 to 0.1, 0.1 to 0.2, and
so
> > on, I'd recommend plotting the center-point of each bin. So along
the
> > X-axis, you'd plot values at 0.5, 0.15, 0.25, 0.35, 0.45, 0.55,
0.65,
> > 0.75, 0.85, and 0.95.
> >
> > The points along the Y-axis are the observed relative frequency
values.
> > And that's the OY_TP value from the MET output.
> >
> > I also worked up an example using METviewer on some sample
> > probabilistic output data:
> > http://www.dtcenter.org/met/metviewer/metviewer1.jsp
> >
> > See the attached XML and resulting plot. This shows the
reliability
> > diagrams for probabilities from 3 different ensembles for 1-hour
> > precip > 0.10". I've also attached the corresponding data file and
> > R-script used to generate this plot (also output from METviewer).
The
> > templatized R-script is rather difficult to parse through, but it
might
> be useful.
> >
> > Thanks,
> > John Halley Gotway
> >
> > On Fri, May 22, 2020 at 6:17 PM efren.serra.ctr at nrlmry.navy.mil
via RT
> > < met_help at ucar.edu> wrote:
> >
> > >
> > > Fri May 22 18:17:18 2020: Request 95347 was acted upon.
> > > Transaction: Ticket created by efren.serra.ctr at nrlmry.navy.mil
> > > Queue: met_help
> > > Subject: Reliability diagram
> > > Owner: Nobody
> > > Requestors: efren.serra.ctr at nrlmry.navy.mil
> > > Status: new
> > > Ticket <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95347
> > > >
> > >
> > >
> > > Folks - I get from MET tools OY_TP [observed YES total
proportion]
> > > and ON_TP [observed NO total proportion] and I'm plotting OY,
[.1,
> > > ..., .9] as the reliability diagrams. Is this correct?
> > >
> > > Efren A. Serra (Contractor)
> > > Physicist
> > >
> > > DeVine Consulting, Inc.
> > > Naval Research Laboratory
> > > Marine Meteorology Division
> > > 7 Grace Hopper Ave., STOP 2
> > > Monterey, CA 93943
> > > Code 7542
> > > Mobile: 408-425-5027
> > >
> > >
> > >
> >
> >
> >
> >
>
> --
> Julie Prestopnik
> Software Engineer
> National Center for Atmospheric Research Research Applications
Laboratory
> Phone: 303.497.8399
> Email: jpresto at ucar.edu
>
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #95347] Reliability diagram
From: efren.serra.ctr at nrlmry.navy.mil
Time: Wed May 27 18:31:59 2020
George - I'm running the code from my desktop using R studio in
Anaconda. Do you know how one can plot a reliability diagram from the
*pjc.txt, pstd.txt, prc.txt* files that MET v8 grid_stat tools
generate? Thanks.
-----Original Message-----
From: George McCabe via RT <met_help at ucar.edu>
Sent: Wednesday, May 27, 2020 5:30 PM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #95347] Reliability diagram
Hi,
John and Julie are both out of the office today, but I can do my best
to help.
The util_plot.R script can be found in the METViewer repository:
https://github.com/NCAR/METviewer/blob/master/R_work/include/util_plot.R
The command before the source line that sets the working directory to
/opt/vxwww/tomcat/webapps/metviewer/R_work/ should allow the script to
find the file in the include directory. I am not sure why that file
would not be in that directory. I may not be understanding the issue
correctly. Let me know if that doesn't help. We may have to wait until
John gets back.
Thanks,
George
On Wed, May 27, 2020 at 5:35 PM efren.serra.ctr at nrlmry.navy.mil via RT
< met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95347 >
>
> Julie - In some R code that John sent me, there's this file I need:
>
> setwd("/opt/vxwww/tomcat/webapps/metviewer/R_work/");
> source("include/util_plot.R"); *<<< This one. >>>*
>
> I was wondering if you could send it to me. Thank you.
>
>
> -----Original Message-----
> From: Julie Prestopnik via RT <met_help at ucar.edu>
> Sent: Tuesday, May 26, 2020 12:10 PM
> To: Serra, Mr. Efren, Contractor, Code 7531 <
> efren.serra.ctr at nrlmry.navy.mil>
> Subject: Re: [rt.rap.ucar.edu #95347] Reliability diagram
>
> Hi Efren. I just wanted to let you know that John is out of the
> office until later this week. Unfortunately, I am unable to answer
> your question. I'm sure you will hear from John shortly after he
> returns to work.
>
> Julie
>
> On Mon, May 25, 2020 at 11:42 PM efren.serra.ctr at nrlmry.navy.mil via
> RT < met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95347 >
> >
> > John - I should point out that I'm using v8.1 of MET tools and
that
> > I'm parsing *pjc.txt* output files.
> >
> > -----Original Message-----
> > From: John Halley Gotway via RT <met_help at ucar.edu>
> > Sent: Sunday, May 24, 2020 2:07 PM
> > To: Serra, Mr. Efren, Contractor, Code 7531 <
> > efren.serra.ctr at nrlmry.navy.mil>
> > Subject: Re: [rt.rap.ucar.edu #95347] Reliability diagram
> >
> > Efren,
> >
> > Not quite. Here's a useful link about reliability diagrams from
the WMO:
> > http://www.bom.gov.au/wmo/lrfvs/reliability.shtml
> >
> > The points along the X-axis are the forecast probability
thresholds..
> > Since you're using probability bins of 0 to 0.1, 0.1 to 0.2, and
so
> > on, I'd recommend plotting the center-point of each bin. So along
> > the X-axis, you'd plot values at 0.5, 0.15, 0.25, 0.35, 0.45,
0.55,
> > 0.65, 0.75, 0.85, and 0.95.
> >
> > The points along the Y-axis are the observed relative frequency
values.
> > And that's the OY_TP value from the MET output.
> >
> > I also worked up an example using METviewer on some sample
> > probabilistic output data:
> > http://www.dtcenter.org/met/metviewer/metviewer1.jsp
> >
> > See the attached XML and resulting plot. This shows the
reliability
> > diagrams for probabilities from 3 different ensembles for 1-hour
> > precip > 0.10". I've also attached the corresponding data file and
> > R-script used to generate this plot (also output from METviewer).
> > The templatized R-script is rather difficult to parse through, but
> > it might
> be useful.
> >
> > Thanks,
> > John Halley Gotway
> >
> > On Fri, May 22, 2020 at 6:17 PM efren.serra.ctr at nrlmry.navy.mil
via
> > RT < met_help at ucar.edu> wrote:
> >
> > >
> > > Fri May 22 18:17:18 2020: Request 95347 was acted upon.
> > > Transaction: Ticket created by efren.serra.ctr at nrlmry.navy.mil
> > > Queue: met_help
> > > Subject: Reliability diagram
> > > Owner: Nobody
> > > Requestors: efren.serra.ctr at nrlmry.navy.mil
> > > Status: new
> > > Ticket <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95347
> > > >
> > >
> > >
> > > Folks - I get from MET tools OY_TP [observed YES total
proportion]
> > > and ON_TP [observed NO total proportion] and I'm plotting OY,
[.1,
> > > ..., .9] as the reliability diagrams. Is this correct?
> > >
> > > Efren A. Serra (Contractor)
> > > Physicist
> > >
> > > DeVine Consulting, Inc.
> > > Naval Research Laboratory
> > > Marine Meteorology Division
> > > 7 Grace Hopper Ave., STOP 2
> > > Monterey, CA 93943
> > > Code 7542
> > > Mobile: 408-425-5027
> > >
> > >
> > >
> >
> >
> >
> >
>
> --
> Julie Prestopnik
> Software Engineer
> National Center for Atmospheric Research Research Applications
> Laboratory
> Phone: 303.497.8399
> Email: jpresto at ucar.edu
>
> My working day may not be your working day. Please do not feel
> obliged to reply to this email outside of your normal working hours.
>
>
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research Research Applications
Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to reply to this email outside of your normal working hours.
------------------------------------------------
Subject: Reliability diagram
From: John Halley Gotway
Time: Thu May 28 13:05:08 2020
Efren,
I see you've had some correspondence with George and Julie on this
issue. I
just wanted to check in to see what questions remain.
You had asked, "according to WMO link below, the plot of
OY_TP[i]/(OY_TP[i]+ON_TP[i]) versus the .5, .15, ..., .95 would be the
reliability diagram, isn't it?"
It's actually even easier than that. The x-value is midpoint of the
forecast probability bin and the y-value is the contents of the
BASER_i
column. I've attached sample PJC and PCT output files.
The following R commands make a reliability diagram for the 4-th line
of
the PJC output:
*pjc=read.table("grid_stat_PROB_GFS_CLIMO_1.0DEG_150000L_20120409_120000V_pjc.txt",
header=TRUE)row=4x=c()y=c()ni=pjc[row,"N_THRESH"]for(i in 0:(ni-1))
{ x=c(x, (pjc[row,paste("THRESH_", i, sep='')] +
pjc[row,paste("THRESH_",
i+1, sep='')])/2) y=c(y, pjc[row,paste("BASER_", i,
sep='')])}png("reli_pjc.png")plot(x, y, type="b", xlim=c(0,1),
ylim=c(0,1),main=pjc[row,"FCST_VAR"])dev.off()*
Or you can process the raw PCT counts instead of the rates using this
sample R code:
*pct=read.table("grid_stat_PROB_GFS_CLIMO_1.0DEG_150000L_20120409_120000V_pct.txt",
header=TRUE)row=4x=c()oy=c()oy_plus_on=c()ni=pct[row,"N_THRESH"]for(i
in
0:(ni-1)) { x=c(x, (pct[row,paste("THRESH_", i, sep='')] +
pct[row,paste("THRESH_", i+1, sep='')])/2) oy=c(oy,
pct[row,paste("OY_", i,
sep='')]) oy_plus_on=c(oy_plus_on, pct[row,paste("OY_", i, sep='')] +
pct[row,paste("ON_", i, sep='')])}png("reli.png)plot(x, oy/oy_plus_on,
type="b", xlim=c(0,1), ylim=c(0,1),main=pct[row,"FCST_VAR"])dev.off()*
Both produce the same reliability curve.
John
On Wed, May 27, 2020 at 6:32 PM efren.serra.ctr at nrlmry.navy.mil via RT
<
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95347 >
>
> George - I'm running the code from my desktop using R studio in
Anaconda.
> Do you know how one can plot a reliability diagram from the
*pjc.txt,
> pstd.txt, prc.txt* files that MET v8 grid_stat tools generate?
Thanks.
>
> -----Original Message-----
> From: George McCabe via RT <met_help at ucar.edu>
> Sent: Wednesday, May 27, 2020 5:30 PM
> To: Serra, Mr. Efren, Contractor, Code 7531 <
> efren.serra.ctr at nrlmry.navy.mil>
> Subject: Re: [rt.rap.ucar.edu #95347] Reliability diagram
>
> Hi,
>
> John and Julie are both out of the office today, but I can do my
best to
> help.
>
> The util_plot.R script can be found in the METViewer repository:
>
>
https://github.com/NCAR/METviewer/blob/master/R_work/include/util_plot.R
>
> The command before the source line that sets the working directory
to
> /opt/vxwww/tomcat/webapps/metviewer/R_work/ should allow the script
to find
> the file in the include directory. I am not sure why that file would
not be
> in that directory. I may not be understanding the issue correctly.
Let me
> know if that doesn't help. We may have to wait until John gets back.
>
> Thanks,
> George
>
>
> On Wed, May 27, 2020 at 5:35 PM efren.serra.ctr at nrlmry.navy.mil via
RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95347 >
> >
> > Julie - In some R code that John sent me, there's this file I
need:
> >
> > setwd("/opt/vxwww/tomcat/webapps/metviewer/R_work/");
> > source("include/util_plot.R"); *<<< This one. >>>*
> >
> > I was wondering if you could send it to me. Thank you.
> >
> >
> > -----Original Message-----
> > From: Julie Prestopnik via RT <met_help at ucar.edu>
> > Sent: Tuesday, May 26, 2020 12:10 PM
> > To: Serra, Mr. Efren, Contractor, Code 7531 <
> > efren.serra.ctr at nrlmry.navy.mil>
> > Subject: Re: [rt.rap.ucar.edu #95347] Reliability diagram
> >
> > Hi Efren. I just wanted to let you know that John is out of the
> > office until later this week. Unfortunately, I am unable to
answer
> > your question. I'm sure you will hear from John shortly after he
> > returns to work.
> >
> > Julie
> >
> > On Mon, May 25, 2020 at 11:42 PM efren.serra.ctr at nrlmry.navy.mil
via
> > RT < met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95347 >
> > >
> > > John - I should point out that I'm using v8.1 of MET tools and
that
> > > I'm parsing *pjc.txt* output files.
> > >
> > > -----Original Message-----
> > > From: John Halley Gotway via RT <met_help at ucar.edu>
> > > Sent: Sunday, May 24, 2020 2:07 PM
> > > To: Serra, Mr. Efren, Contractor, Code 7531 <
> > > efren.serra.ctr at nrlmry.navy.mil>
> > > Subject: Re: [rt.rap.ucar.edu #95347] Reliability diagram
> > >
> > > Efren,
> > >
> > > Not quite. Here's a useful link about reliability diagrams from
the
> WMO:
> > > http://www.bom.gov.au/wmo/lrfvs/reliability.shtml
> > >
> > > The points along the X-axis are the forecast probability
thresholds..
> > > Since you're using probability bins of 0 to 0.1, 0.1 to 0.2, and
so
> > > on, I'd recommend plotting the center-point of each bin. So
along
> > > the X-axis, you'd plot values at 0.5, 0.15, 0.25, 0.35, 0.45,
0.55,
> > > 0.65, 0.75, 0.85, and 0.95.
> > >
> > > The points along the Y-axis are the observed relative frequency
values.
> > > And that's the OY_TP value from the MET output.
> > >
> > > I also worked up an example using METviewer on some sample
> > > probabilistic output data:
> > > http://www.dtcenter.org/met/metviewer/metviewer1.jsp
> > >
> > > See the attached XML and resulting plot. This shows the
reliability
> > > diagrams for probabilities from 3 different ensembles for 1-hour
> > > precip > 0.10". I've also attached the corresponding data file
and
> > > R-script used to generate this plot (also output from
METviewer).
> > > The templatized R-script is rather difficult to parse through,
but
> > > it might
> > be useful.
> > >
> > > Thanks,
> > > John Halley Gotway
> > >
> > > On Fri, May 22, 2020 at 6:17 PM efren.serra.ctr at nrlmry.navy.mil
via
> > > RT < met_help at ucar.edu> wrote:
> > >
> > > >
> > > > Fri May 22 18:17:18 2020: Request 95347 was acted upon.
> > > > Transaction: Ticket created by efren.serra.ctr at nrlmry.navy.mil
> > > > Queue: met_help
> > > > Subject: Reliability diagram
> > > > Owner: Nobody
> > > > Requestors: efren.serra.ctr at nrlmry.navy.mil
> > > > Status: new
> > > > Ticket <URL:
> > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95347
> > > > >
> > > >
> > > >
> > > > Folks - I get from MET tools OY_TP [observed YES total
proportion]
> > > > and ON_TP [observed NO total proportion] and I'm plotting OY,
[.1,
> > > > ..., .9] as the reliability diagrams. Is this correct?
> > > >
> > > > Efren A. Serra (Contractor)
> > > > Physicist
> > > >
> > > > DeVine Consulting, Inc.
> > > > Naval Research Laboratory
> > > > Marine Meteorology Division
> > > > 7 Grace Hopper Ave., STOP 2
> > > > Monterey, CA 93943
> > > > Code 7542
> > > > Mobile: 408-425-5027
> > > >
> > > >
> > > >
> > >
> > >
> > >
> > >
> >
> > --
> > Julie Prestopnik
> > Software Engineer
> > National Center for Atmospheric Research Research Applications
> > Laboratory
> > Phone: 303.497.8399
> > Email: jpresto at ucar.edu
> >
> > My working day may not be your working day. Please do not feel
> > obliged to reply to this email outside of your normal working
hours.
> >
> >
> >
> >
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research Research Applications
Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>
>
>
------------------------------------------------
Subject: Reliability diagram
From: John Halley Gotway
Time: Thu May 28 13:05:08 2020
VERSION MODEL DESC FCST_LEAD FCST_VALID_BEG FCST_VALID_END
OBS_LEAD OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_UNITS
FCST_LEV OBS_VAR OBS_UNITS OBS_LEV OBTYPE VX_MASK INTERP_MTHD
INTERP_PNTS FCST_THRESH OBS_THRESH COV_THRESH ALPHA LINE_TYPE TOTAL
N_THRESH THRESH_1 OY_TP_1 ON_TP_1 CALIBRATION_1 REFINEMENT_1
LIKELIHOOD_1 BASER_1 THRESH_2 OY_TP_2 ON_TP_2 CALIBRATION_2
REFINEMENT_2 LIKELIHOOD_2 BASER_2 THRESH_3 OY_TP_3 ON_TP_3
CALIBRATION_3 REFINEMENT_3 LIKELIHOOD_3 BASER_3 THRESH_4 OY_TP_4
ON_TP_4 CALIBRATION_4 REFINEMENT_4 LIKELIHOOD_4 BASER_4 THRESH_5
OY_TP_5 ON_TP_5 CALIBRATION_5 REFINEMENT_5 LIKELIHOOD_5 BASER_5
THRESH_6 OY_TP_6 ON_TP_6 CALIBRATION_6 REFINEMENT_6 LIKELIHOOD_6
BASER_6 THRESH_7 OY_TP_7 ON_TP_7 CALIBRATION_7 REFINEMENT_7
LIKELIHOOD_7 BASER_7 THRESH_8 OY_TP_8 ON_TP_8 CALIBRATION_8
REFINEMENT_8 LIKELIHOOD_8 BASER_8 THRESH_9 OY_TP_9 ON_TP_9
CALIBRATION_9 REFINEMENT_9 LIKELIHOOD_9 BASER_9 THRESH_10 OY_TP_10
ON_TP_10 CALIBRATION_10 REFINEMENT_10 LIKELIHOOD_10 BASER_10
THRESH_11
V9.1 SREF CLIMO_BINS 150000 20120409_120000 20120409_120000
000000 20120409_120000 20120409_120000 PROB(WIND>12.890) %
Z10 WIND m/s Z10 GFSANL FULL_BIN_1 NEAREST 1
==0.10000 >12.89 NA NA PJC 2635 11
0 0 1 0 1 NA 0
0.1 0 0 NA 0 NA NA
0.2 0 0 NA 0 NA NA
0.3 0 0 NA 0 NA NA
0.4 0 0 NA 0 NA NA
0.5 0 0 NA 0 NA NA
0.6 0 0 NA 0 NA NA
0.7 0 0 NA 0 NA NA
0.8 0 0 NA 0 NA NA
0.9 0 0 NA 0 NA NA
1
V9.1 SREF CLIMO_BINS 150000 20120409_120000 20120409_120000
000000 20120409_120000 20120409_120000 PROB(WIND>12.890) %
Z10 WIND m/s Z10 GFSANL FULL_BIN_2 NEAREST 1
==0.10000 >12.89 NA NA PJC 5907 11
0 0 1 0 1 NA 0
0.1 0 0 NA 0 NA NA
0.2 0 0 NA 0 NA NA
0.3 0 0 NA 0 NA NA
0.4 0 0 NA 0 NA NA
0.5 0 0 NA 0 NA NA
0.6 0 0 NA 0 NA NA
0.7 0 0 NA 0 NA NA
0.8 0 0 NA 0 NA NA
0.9 0 0 NA 0 NA NA
1
V9.1 SREF CLIMO_BINS 150000 20120409_120000 20120409_120000
000000 20120409_120000 20120409_120000 PROB(WIND>12.890) %
Z10 WIND m/s Z10 GFSANL FULL_BIN_3 NEAREST 1
==0.10000 >12.89 NA NA PJC 15323 11
0 0.00065261 0.95647 0.00068185 0.95712 0.037453
0.00068185 0.1 0.00039157 0.010442 0.036145 0.010833
0.022472 0.036145 0.2 0.00071788 0.0088103 0.075342
0.0095282 0.041199 0.075342 0.3 0.00097892 0.0017621
0.35714 0.002741 0.05618 0.35714 0.4 0.0008484
0.0016968 0.33333 0.0025452 0.048689 0.33333 0.5
0.0011094 0.0013052 0.45946 0.0024147 0.06367 0.45946
0.6 0.001501 0.0015663 0.48936 0.0030673 0.086142
0.48936 0.7 0.0017621 0.00045683 0.79412 0.0022189
0.10112 0.79412 0.8 0.0021536 6.5261e-05 0.97059
0.0022189 0.1236 0.97059 0.9 0.0073093 0
1 0.0073093 0.41948 1 1
V9.1 SREF CLIMO_BINS 150000 20120409_120000 20120409_120000
000000 20120409_120000 20120409_120000 PROB(TMP<273.000) %
Z2 TMP K Z2 GFSANL FULL_BIN_1 NEAREST 1
==0.10000 <273.0 NA NA PJC 14571 11
0 0.0025393 0.76673 0.0033009 0.76927 0.012724 0.0033009
0.1 0.00075492 0.0083042 0.083333 0.0090591 0.0037827
0.083333 0.2 0.003706 0.010157 0.26733 0.013863
0.018569 0.26733 0.3 0.0019216 0.0025393 0.43077
0.0044609 0.0096286 0.43077 0.4 0.0027452 0.0043237
0.38835 0.0070688 0.013755 0.38835 0.5 0.012628
0.0027452 0.82143 0.015373 0.063274 0.82143 0.6
0.0046668 0.0021275 0.68687 0.0067943 0.023384 0.68687
0.7 0.0099513 0.0017844 0.84795 0.011736 0.049862
0.84795 0.8 0.0040491 0.00075492 0.84286 0.0048041
0.020289 0.84286 0.9 0.15661 0.00096081 0.9939
0.15757 0.78473 0.9939 1
V9.1 SREF CLIMO_BINS 150000 20120409_120000 20120409_120000
000000 20120409_120000 20120409_120000 PROB(TMP<273.000) %
Z2 TMP K Z2 GFSANL FULL_BIN_2 NEAREST 1
==0.10000 <273.0 NA NA PJC 4160 11
0 0.00024038 0.78293 0.00030694 0.78317 0.0013812
0.00030694 0.1 0.00096154 0.010577 0.083333 0.011538
0.0055249 0.083333 0.2 0.0038462 0.014183 0.21333
0.018029 0.022099 0.21333 0.3 0.0019231 0.0024038
0.44444 0.0043269 0.01105 0.44444 0.4 0.0021635
0.0028846 0.42857 0.0050481 0.012431 0.42857 0.5
0.011058 0.0057692 0.65714 0.016827 0.063536 0.65714
0.6 0.010096 0.0026442 0.79245 0.01274 0.058011
0.79245 0.7 0.016827 0.0040865 0.8046 0.020913
0.096685 0.8046 0.8 0.0064904 0.00024038 0.96429
0.0067308 0.037293 0.96429 0.9 0.12043 0.00024038
0.99801 0.12067 0.69199 0.99801 1
V9.1 SREF CLIMO_BINS 150000 20120409_120000 20120409_120000
000000 20120409_120000 20120409_120000 PROB(TMP<273.000) %
Z2 TMP K Z2 GFSANL FULL_BIN_3 NEAREST 1
==0.10000 <273.0 NA NA PJC 5134 11
0 0.00038956 0.5857 0.00066467 0.58609 0.001113
0.00066467 0.1 0.0013635 0.0070121 0.16279 0.0083755
0.0038954 0.16279 0.2 0.0021426 0.009739 0.18033
0.011882 0.0061213 0.18033 0.3 0.006233 0.0054538
0.53333 0.011687 0.017807 0.53333 0.4 0.0042852
0.0048695 0.46809 0.0091547 0.012243 0.46809 0.5
0.0070121 0.0083755 0.4557 0.015388 0.020033 0.4557
0.6 0.0068173 0.0058434 0.53846 0.012661 0.019477
0.53846 0.7 0.007986 0.006233 0.56164 0.014219
0.022816 0.56164 0.8 0.0083755 0.0044799 0.65152
0.012855 0.023929 0.65152 0.9 0.30541 0.012271
0.96137 0.31769 0.87257 0.96137 1
V9.1 SREF CLIMO_BINS 150000 20120409_120000 20120409_120000
000000 20120409_120000 20120409_120000 PROB(TMP>298.800) %
Z2 TMP K Z2 GFSANL FULL_BIN_1 NEAREST 1
==0.10000 >298.8 NA NA PJC 14571 11
0 0 0.97715 0 0.97715 0 0
0.1 0 0.0021275 0 0.0021275 0 0
0.2 0.00020589 0.0068629 0.029126 0.0070688 0.034884
0.029126 0.3 6.8629e-05 0.0015785 0.041667 0.0016471
0.011628 0.041667 0.4 0 0.0010981 0
0.0010981 0 0 0.5 0.00020589 0.00041178
0.33333 0.00061767 0.034884 0.33333 0.6 6.8629e-05
0.00061767 0.1 0.00068629 0.011628 0.1 0.7
0.00054904 0.0012353 0.30769 0.0017844 0.093023 0.30769
0.8 0.00054904 0.00089218 0.38095 0.0014412 0.093023
0.38095 0.9 0.004255 0.0021275 0.66667 0.0063825
0.72093 0.66667 1
V9.1 SREF CLIMO_BINS 150000 20120409_120000 20120409_120000
000000 20120409_120000 20120409_120000 PROB(TMP>298.800) %
Z2 TMP K Z2 GFSANL FULL_BIN_2 NEAREST 1
==0.10000 >298.8 NA NA PJC 4160 11
0 0 0.95144 0 0.95144 0 0
0.1 0 0.0033654 0 0.0033654 0 0
0.2 0 0.0043269 0 0.0043269 0 0
0.3 0 0.0012019 0 0.0012019 0 0
0.4 0 0.00096154 0 0.00096154 0 0
0.5 0.00096154 0.0012019 0.44444 0.0021635 0.029851
0.44444 0.6 0.0012019 0.00096154 0.55556 0.0021635
0.037313 0.55556 0.7 0.00072115 0.0012019 0.375
0.0019231 0.022388 0.375 0.8 0.00625 0.0019231
0.76471 0.0081731 0.19403 0.76471 0.9 0.023077
0.0012019 0.9505 0.024279 0.71642 0.9505
1
V9.1 SREF CLIMO_BINS 150000 20120409_120000 20120409_120000
000000 20120409_120000 20120409_120000 PROB(TMP>298.800) %
Z2 TMP K Z2 GFSANL FULL_BIN_3 NEAREST 1
==0.10000 >298.8 NA NA PJC 5134 11
0 0.00077912 0.94079 0.00082747 0.94157 0.014035
0.00082747 0.1 0.00077912 0.0021426 0.26667 0.0029217
0.014035 0.26667 0.2 0.0019478 0.00077912 0.71429
0.0027269 0.035088 0.71429 0.3 0.0015582 0.00019478
0.88889 0.001753 0.02807 0.88889 0.4 0.0013635
0.00038956 0.77778 0.001753 0.024561 0.77778 0.5
0.00038956 0.00019478 0.66667 0.00058434 0.0070175 0.66667
0.6 0.001753 0 1 0.001753 0.031579 1
0.7 0.006233 0 1 0.006233 0.11228 1
0.8 0.011297 0 1 0.011297 0.20351 1
0.9 0.029412 0 1 0.029412 0.52982 1
1
V9.1 SREF CLIMO_BINS 150000 20120409_120000 20120409_120000
000000 20120409_120000 20120409_120000 PROB(WIND>12.890) %
Z10 WIND m/s Z10 GFSANL FULL_BIN_1 NEAREST 1
==0.10000 >12.89 NA NA PJC 2635 11
0 0 1 0 1 NA 0
0.1 0 0 NA 0 NA NA
0.2 0 0 NA 0 NA NA
0.3 0 0 NA 0 NA NA
0.4 0 0 NA 0 NA NA
0.5 0 0 NA 0 NA NA
0.6 0 0 NA 0 NA NA
0.7 0 0 NA 0 NA NA
0.8 0 0 NA 0 NA NA
0.9 0 0 NA 0 NA NA
1
V9.1 SREF CLIMO_BINS 150000 20120409_120000 20120409_120000
000000 20120409_120000 20120409_120000 PROB(WIND>12.890) %
Z10 WIND m/s Z10 GFSANL FULL_BIN_2 NEAREST 1
==0.10000 >12.89 NA NA PJC 5907 11
0 0 1 0 1 NA 0
0.1 0 0 NA 0 NA NA
0.2 0 0 NA 0 NA NA
0.3 0 0 NA 0 NA NA
0.4 0 0 NA 0 NA NA
0.5 0 0 NA 0 NA NA
0.6 0 0 NA 0 NA NA
0.7 0 0 NA 0 NA NA
0.8 0 0 NA 0 NA NA
0.9 0 0 NA 0 NA NA
1
V9.1 SREF CLIMO_BINS 150000 20120409_120000 20120409_120000
000000 20120409_120000 20120409_120000 PROB(WIND>12.890) %
Z10 WIND m/s Z10 GFSANL FULL_BIN_3 NEAREST 1
==0.10000 >12.89 NA NA PJC 15323 11
0 0.00065261 0.95647 0.00068185 0.95712 0.037453
0.00068185 0.1 0.00039157 0.010442 0.036145 0.010833
0.022472 0.036145 0.2 0.00071788 0.0088103 0.075342
0.0095282 0.041199 0.075342 0.3 0.00097892 0.0017621
0.35714 0.002741 0.05618 0.35714 0.4 0.0008484
0.0016968 0.33333 0.0025452 0.048689 0.33333 0.5
0.0011094 0.0013052 0.45946 0.0024147 0.06367 0.45946
0.6 0.001501 0.0015663 0.48936 0.0030673 0.086142
0.48936 0.7 0.0017621 0.00045683 0.79412 0.0022189
0.10112 0.79412 0.8 0.0021536 6.5261e-05 0.97059
0.0022189 0.1236 0.97059 0.9 0.0073093 0
1 0.0073093 0.41948 1 1
------------------------------------------------
Subject: Reliability diagram
From: John Halley Gotway
Time: Thu May 28 13:05:08 2020
VERSION MODEL DESC FCST_LEAD FCST_VALID_BEG FCST_VALID_END
OBS_LEAD OBS_VALID_BEG OBS_VALID_END FCST_VAR FCST_UNITS
FCST_LEV OBS_VAR OBS_UNITS OBS_LEV OBTYPE VX_MASK INTERP_MTHD
INTERP_PNTS FCST_THRESH OBS_THRESH COV_THRESH ALPHA LINE_TYPE TOTAL
N_THRESH THRESH_1 OY_1 ON_1 THRESH_2 OY_2 ON_2 THRESH_3 OY_3 ON_3
THRESH_4 OY_4 ON_4 THRESH_5 OY_5 ON_5 THRESH_6 OY_6 ON_6 THRESH_7 OY_7
ON_7 THRESH_8 OY_8 ON_8 THRESH_9 OY_9 ON_9 THRESH_10 OY_10 ON_10
THRESH_11
V9.1 SREF CLIMO_BINS 150000 20120409_120000 20120409_120000
000000 20120409_120000 20120409_120000 PROB(WIND>12.890) %
Z10 WIND m/s Z10 GFSANL FULL_BIN_1 NEAREST 1
==0.10000 >12.89 NA NA PCT 2635 11
0 0 2635 0.1 0 0 0.2 0 0 0.3 0 0
0.4 0 0 0.5 0 0 0.6 0 0 0.7 0 0
0.8 0 0 0.9 0 0 1
V9.1 SREF CLIMO_BINS 150000 20120409_120000 20120409_120000
000000 20120409_120000 20120409_120000 PROB(WIND>12.890) %
Z10 WIND m/s Z10 GFSANL FULL_BIN_2 NEAREST 1
==0.10000 >12.89 NA NA PCT 5907 11
0 0 5907 0.1 0 0 0.2 0 0 0.3 0 0
0.4 0 0 0.5 0 0 0.6 0 0 0.7 0 0
0.8 0 0 0.9 0 0 1
V9.1 SREF CLIMO_BINS 150000 20120409_120000 20120409_120000
000000 20120409_120000 20120409_120000 PROB(WIND>12.890) %
Z10 WIND m/s Z10 GFSANL FULL_BIN_3 NEAREST 1
==0.10000 >12.89 NA NA PCT 15323 11
0 10 14656 0.1 6 160 0.2 11 135 0.3 15 27
0.4 13 26 0.5 17 20 0.6 23 24 0.7 27 7
0.8 33 1 0.9 112 0 1
V9.1 SREF CLIMO_BINS 150000 20120409_120000 20120409_120000
000000 20120409_120000 20120409_120000 PROB(TMP<273.000) %
Z2 TMP K Z2 GFSANL FULL_BIN_1 NEAREST 1
==0.10000 <273.0 NA NA PCT 14571 11
0 37 11172 0.1 11 121 0.2 54 148 0.3 28 37
0.4 40 63 0.5 184 40 0.6 68 31 0.7 145 26
0.8 59 11 0.9 2282 14 1
V9.1 SREF CLIMO_BINS 150000 20120409_120000 20120409_120000
000000 20120409_120000 20120409_120000 PROB(TMP<273.000) %
Z2 TMP K Z2 GFSANL FULL_BIN_2 NEAREST 1
==0.10000 <273.0 NA NA PCT 4160 11
0 1 3257 0.1 4 44 0.2 16 59 0.3 8 10
0.4 9 12 0.5 46 24 0.6 42 11 0.7 70 17
0.8 27 1 0.9 501 1 1
V9.1 SREF CLIMO_BINS 150000 20120409_120000 20120409_120000
000000 20120409_120000 20120409_120000 PROB(TMP<273.000) %
Z2 TMP K Z2 GFSANL FULL_BIN_3 NEAREST 1
==0.10000 <273.0 NA NA PCT 5134 11
0 2 3007 0.1 7 36 0.2 11 50 0.3 32 28
0.4 22 25 0.5 36 43 0.6 35 30 0.7 41 32
0.8 43 23 0.9 1568 63 1
V9.1 SREF CLIMO_BINS 150000 20120409_120000 20120409_120000
000000 20120409_120000 20120409_120000 PROB(TMP>298.800) %
Z2 TMP K Z2 GFSANL FULL_BIN_1 NEAREST 1
==0.10000 >298.8 NA NA PCT 14571 11
0 0 14238 0.1 0 31 0.2 3 100 0.3 1 23
0.4 0 16 0.5 3 6 0.6 1 9 0.7 8 18
0.8 8 13 0.9 62 31 1
V9.1 SREF CLIMO_BINS 150000 20120409_120000 20120409_120000
000000 20120409_120000 20120409_120000 PROB(TMP>298.800) %
Z2 TMP K Z2 GFSANL FULL_BIN_2 NEAREST 1
==0.10000 >298.8 NA NA PCT 4160 11
0 0 3958 0.1 0 14 0.2 0 18 0.3 0 5
0.4 0 4 0.5 4 5 0.6 5 4 0.7 3 5
0.8 26 8 0.9 96 5 1
V9.1 SREF CLIMO_BINS 150000 20120409_120000 20120409_120000
000000 20120409_120000 20120409_120000 PROB(TMP>298.800) %
Z2 TMP K Z2 GFSANL FULL_BIN_3 NEAREST 1
==0.10000 >298.8 NA NA PCT 5134 11
0 4 4830 0.1 4 11 0.2 10 4 0.3 8 1
0.4 7 2 0.5 2 1 0.6 9 0 0.7 32 0
0.8 58 0 0.9 151 0 1
V9.1 SREF CLIMO_BINS 150000 20120409_120000 20120409_120000
000000 20120409_120000 20120409_120000 PROB(WIND>12.890) %
Z10 WIND m/s Z10 GFSANL FULL_BIN_1 NEAREST 1
==0.10000 >12.89 NA NA PCT 2635 11
0 0 2635 0.1 0 0 0.2 0 0 0.3 0 0
0.4 0 0 0.5 0 0 0.6 0 0 0.7 0 0
0.8 0 0 0.9 0 0 1
V9.1 SREF CLIMO_BINS 150000 20120409_120000 20120409_120000
000000 20120409_120000 20120409_120000 PROB(WIND>12.890) %
Z10 WIND m/s Z10 GFSANL FULL_BIN_2 NEAREST 1
==0.10000 >12.89 NA NA PCT 5907 11
0 0 5907 0.1 0 0 0.2 0 0 0.3 0 0
0.4 0 0 0.5 0 0 0.6 0 0 0.7 0 0
0.8 0 0 0.9 0 0 1
V9.1 SREF CLIMO_BINS 150000 20120409_120000 20120409_120000
000000 20120409_120000 20120409_120000 PROB(WIND>12.890) %
Z10 WIND m/s Z10 GFSANL FULL_BIN_3 NEAREST 1
==0.10000 >12.89 NA NA PCT 15323 11
0 10 14656 0.1 6 160 0.2 11 135 0.3 15 27
0.4 13 26 0.5 17 20 0.6 23 24 0.7 27 7
0.8 33 1 0.9 112 0 1
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #95347] Reliability diagram
From: efren.serra.ctr at nrlmry.navy.mil
Time: Thu May 28 16:42:14 2020
Beatiful! Thanks
-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Thursday, May 28, 2020 12:05 PM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #95347] Reliability diagram
Efren,
I see you've had some correspondence with George and Julie on this
issue. I just wanted to check in to see what questions remain.
You had asked, "according to WMO link below, the plot of
OY_TP[i]/(OY_TP[i]+ON_TP[i]) versus the .5, .15, ..., .95 would be the
reliability diagram, isn't it?"
It's actually even easier than that. The x-value is midpoint of the
forecast probability bin and the y-value is the contents of the
BASER_i column. I've attached sample PJC and PCT output files.
The following R commands make a reliability diagram for the 4-th line
of the PJC output:
*pjc=read.table("grid_stat_PROB_GFS_CLIMO_1.0DEG_150000L_20120409_120000V_pjc.txt",
header=TRUE)row=4x=c()y=c()ni=pjc[row,"N_THRESH"]for(i in 0:(ni-1)) {
x=c(x, (pjc[row,paste("THRESH_", i, sep='')] +
pjc[row,paste("THRESH_",
i+1, sep='')])/2) y=c(y, pjc[row,paste("BASER_", i,
sep='')])}png("reli_pjc.png")plot(x, y, type="b", xlim=c(0,1),
ylim=c(0,1),main=pjc[row,"FCST_VAR"])dev.off()*
Or you can process the raw PCT counts instead of the rates using this
sample R code:
*pct=read.table("grid_stat_PROB_GFS_CLIMO_1.0DEG_150000L_20120409_120000V_pct.txt",
header=TRUE)row=4x=c()oy=c()oy_plus_on=c()ni=pct[row,"N_THRESH"]for(i
in
0:(ni-1)) { x=c(x, (pct[row,paste("THRESH_", i, sep='')] +
pct[row,paste("THRESH_", i+1, sep='')])/2) oy=c(oy,
pct[row,paste("OY_", i,
sep='')]) oy_plus_on=c(oy_plus_on, pct[row,paste("OY_", i, sep='')] +
pct[row,paste("ON_", i, sep='')])}png("reli.png)plot(x, oy/oy_plus_on,
type="b", xlim=c(0,1), ylim=c(0,1),main=pct[row,"FCST_VAR"])dev.off()*
Both produce the same reliability curve.
John
On Wed, May 27, 2020 at 6:32 PM efren.serra.ctr at nrlmry.navy.mil via RT
< met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95347 >
>
> George - I'm running the code from my desktop using R studio in
Anaconda.
> Do you know how one can plot a reliability diagram from the
*pjc.txt,
> pstd.txt, prc.txt* files that MET v8 grid_stat tools generate?
Thanks.
>
> -----Original Message-----
> From: George McCabe via RT <met_help at ucar.edu>
> Sent: Wednesday, May 27, 2020 5:30 PM
> To: Serra, Mr. Efren, Contractor, Code 7531 <
> efren.serra.ctr at nrlmry.navy.mil>
> Subject: Re: [rt.rap.ucar.edu #95347] Reliability diagram
>
> Hi,
>
> John and Julie are both out of the office today, but I can do my
best
> to help.
>
> The util_plot.R script can be found in the METViewer repository:
>
>
https://github.com/NCAR/METviewer/blob/master/R_work/include/util_plot
> .R
>
> The command before the source line that sets the working directory
to
> /opt/vxwww/tomcat/webapps/metviewer/R_work/ should allow the script
to
> find the file in the include directory. I am not sure why that file
> would not be in that directory. I may not be understanding the issue
> correctly. Let me know if that doesn't help. We may have to wait
until John gets back.
>
> Thanks,
> George
>
>
> On Wed, May 27, 2020 at 5:35 PM efren.serra.ctr at nrlmry.navy.mil via
RT
> < met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95347 >
> >
> > Julie - In some R code that John sent me, there's this file I
need:
> >
> > setwd("/opt/vxwww/tomcat/webapps/metviewer/R_work/");
> > source("include/util_plot.R"); *<<< This one. >>>*
> >
> > I was wondering if you could send it to me. Thank you.
> >
> >
> > -----Original Message-----
> > From: Julie Prestopnik via RT <met_help at ucar.edu>
> > Sent: Tuesday, May 26, 2020 12:10 PM
> > To: Serra, Mr. Efren, Contractor, Code 7531 <
> > efren.serra.ctr at nrlmry.navy.mil>
> > Subject: Re: [rt.rap.ucar.edu #95347] Reliability diagram
> >
> > Hi Efren. I just wanted to let you know that John is out of the
> > office until later this week. Unfortunately, I am unable to
answer
> > your question. I'm sure you will hear from John shortly after he
> > returns to work.
> >
> > Julie
> >
> > On Mon, May 25, 2020 at 11:42 PM efren.serra.ctr at nrlmry.navy.mil
via
> > RT < met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95347 >
> > >
> > > John - I should point out that I'm using v8.1 of MET tools and
> > > that I'm parsing *pjc.txt* output files.
> > >
> > > -----Original Message-----
> > > From: John Halley Gotway via RT <met_help at ucar.edu>
> > > Sent: Sunday, May 24, 2020 2:07 PM
> > > To: Serra, Mr. Efren, Contractor, Code 7531 <
> > > efren.serra.ctr at nrlmry.navy.mil>
> > > Subject: Re: [rt.rap.ucar.edu #95347] Reliability diagram
> > >
> > > Efren,
> > >
> > > Not quite. Here's a useful link about reliability diagrams from
> > > the
> WMO:
> > > http://www.bom.gov.au/wmo/lrfvs/reliability.shtml
> > >
> > > The points along the X-axis are the forecast probability
thresholds..
> > > Since you're using probability bins of 0 to 0.1, 0.1 to 0.2, and
> > > so on, I'd recommend plotting the center-point of each bin. So
> > > along the X-axis, you'd plot values at 0.5, 0.15, 0.25, 0.35,
> > > 0.45, 0.55, 0.65, 0.75, 0.85, and 0.95.
> > >
> > > The points along the Y-axis are the observed relative frequency
values.
> > > And that's the OY_TP value from the MET output.
> > >
> > > I also worked up an example using METviewer on some sample
> > > probabilistic output data:
> > > http://www.dtcenter.org/met/metviewer/metviewer1.jsp
> > >
> > > See the attached XML and resulting plot. This shows the
> > > reliability diagrams for probabilities from 3 different
ensembles
> > > for 1-hour precip > 0.10". I've also attached the corresponding
> > > data file and R-script used to generate this plot (also output
from METviewer).
> > > The templatized R-script is rather difficult to parse through,
but
> > > it might
> > be useful.
> > >
> > > Thanks,
> > > John Halley Gotway
> > >
> > > On Fri, May 22, 2020 at 6:17 PM efren.serra.ctr at nrlmry.navy.mil
> > > via RT < met_help at ucar.edu> wrote:
> > >
> > > >
> > > > Fri May 22 18:17:18 2020: Request 95347 was acted upon.
> > > > Transaction: Ticket created by efren.serra.ctr at nrlmry.navy.mil
> > > > Queue: met_help
> > > > Subject: Reliability diagram
> > > > Owner: Nobody
> > > > Requestors: efren.serra.ctr at nrlmry.navy.mil
> > > > Status: new
> > > > Ticket <URL:
> > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95347
> > > > >
> > > >
> > > >
> > > > Folks - I get from MET tools OY_TP [observed YES total
> > > > proportion] and ON_TP [observed NO total proportion] and I'm
> > > > plotting OY, [.1, ..., .9] as the reliability diagrams. Is
this correct?
> > > >
> > > > Efren A. Serra (Contractor)
> > > > Physicist
> > > >
> > > > DeVine Consulting, Inc.
> > > > Naval Research Laboratory
> > > > Marine Meteorology Division
> > > > 7 Grace Hopper Ave., STOP 2
> > > > Monterey, CA 93943
> > > > Code 7542
> > > > Mobile: 408-425-5027
> > > >
> > > >
> > > >
> > >
> > >
> > >
> > >
> >
> > --
> > Julie Prestopnik
> > Software Engineer
> > National Center for Atmospheric Research Research Applications
> > Laboratory
> > Phone: 303.497.8399
> > Email: jpresto at ucar.edu
> >
> > My working day may not be your working day. Please do not feel
> > obliged to reply to this email outside of your normal working
hours.
> >
> >
> >
> >
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research Research Applications
> Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged
> to reply to this email outside of your normal working hours.
>
>
>
>
------------------------------------------------
More information about the Met_help
mailing list