[Met_help] [rt.rap.ucar.edu #97513] History for ROC diagrams
John Halley Gotway via RT
met_help at ucar.edu
Fri Nov 20 12:37:05 MST 2020
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Folks - I'm creating plots of POFD_i vs PODY_i and connecting these points to generate a ROC as indicated on p. 414 or MET User's Guide v9. I am a little concerned that my "ROC" diagrams don't look anything like your Figure C.2: Example of ROC Curve on page 415. Attached is my sample ROC.
Efren A. Serra (Contractor)
Physicist
DeVine Consulting, Inc.
Naval Research Laboratory
Marine Meteorology Division
7 Grace Hopper Ave., STOP 2
Monterey, CA 93943
Code 7542
Mobile: 408-425-5027
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: ROC diagrams
From: John Halley Gotway
Time: Wed Nov 18 14:05:11 2020
Efren,
When making a ROC diagram, you usually include both points (0,0) and
(1,1).
That should result in a plot more similar this the one in Appendix C
of the
MET User's Guide:
https://dtcenter.github.io/MET/latest/Users_Guide/appendixC.html?highlight=roc#receiver-
operating-characteristic
John
On Wed, Nov 18, 2020 at 1:34 PM George McCabe via RT
<met_help at ucar.edu>
wrote:
>
> Wed Nov 18 13:34:31 2020: Request 97513 was acted upon.
> Transaction: Given to johnhg (John Halley Gotway) by mccabe
> Queue: met_help
> Subject: ROC diagrams
> Owner: johnhg
> Requestors: efren.serra.ctr at nrlmry.navy.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97513 >
>
>
> This transaction appears to have no content
>
------------------------------------------------
Subject: ROC diagrams
From: Sampson, Mr. Buck
Time: Wed Nov 18 14:06:56 2020
Thanks. As we thought.
Buck
-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Wednesday, November 18, 2020 1:05 PM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #97513] ROC diagrams
Efren,
When making a ROC diagram, you usually include both points (0,0) and
(1,1).
That should result in a plot more similar this the one in Appendix C
of the
MET User's Guide:
https://dtcenter.github.io/MET/latest/Users_Guide/appendixC.html?highlight=roc#receiver-
operating-characteristic
John
On Wed, Nov 18, 2020 at 1:34 PM George McCabe via RT
<met_help at ucar.edu>
wrote:
>
> Wed Nov 18 13:34:31 2020: Request 97513 was acted upon.
> Transaction: Given to johnhg (John Halley Gotway) by mccabe
> Queue: met_help
> Subject: ROC diagrams
> Owner: johnhg
> Requestors: efren.serra.ctr at nrlmry.navy.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97513
> >
>
>
> This transaction appears to have no content
>
------------------------------------------------
Subject: ROC diagrams
From: efren.serra.ctr at nrlmry.navy.mil
Time: Thu Nov 19 11:07:09 2020
Thanks John!
-----Original Message-----
From: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
Sent: Wednesday, November 18, 2020 1:07 PM
To: met_help at ucar.edu; Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Subject: RE: [rt.rap.ucar.edu #97513] ROC diagrams
Thanks. As we thought.
Buck
-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Wednesday, November 18, 2020 1:05 PM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #97513] ROC diagrams
Efren,
When making a ROC diagram, you usually include both points (0,0) and
(1,1).
That should result in a plot more similar this the one in Appendix C
of the
MET User's Guide:
https://dtcenter.github.io/MET/latest/Users_Guide/appendixC.html?highlight=roc#receiver-
operating-characteristic
John
On Wed, Nov 18, 2020 at 1:34 PM George McCabe via RT
<met_help at ucar.edu>
wrote:
>
> Wed Nov 18 13:34:31 2020: Request 97513 was acted upon.
> Transaction: Given to johnhg (John Halley Gotway) by mccabe
> Queue: met_help
> Subject: ROC diagrams
> Owner: johnhg
> Requestors: efren.serra.ctr at nrlmry.navy.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97513
> >
>
>
> This transaction appears to have no content
>
------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #97513] ROC diagrams
From: efren.serra.ctr at nrlmry.navy.mil
Time: Thu Nov 19 12:47:22 2020
John - For the ROC plots, we think that ROC value at (POFD=1, PODY=1)
should be probability threshold 0.0, even though there are (in our
case) 10 POFD_i/PODY_i and 11 THRESH_i. Is this correct? How does your
METViewer does it?
-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Wednesday, November 18, 2020 1:05 PM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #97513] ROC diagrams
Efren,
When making a ROC diagram, you usually include both points (0,0) and
(1,1).
That should result in a plot more similar this the one in Appendix C
of the MET User's Guide:
https://dtcenter.github.io/MET/latest/Users_Guide/appendixC.html?highlight=roc#receiver-
operating-characteristic
John
On Wed, Nov 18, 2020 at 1:34 PM George McCabe via RT
<met_help at ucar.edu>
wrote:
>
> Wed Nov 18 13:34:31 2020: Request 97513 was acted upon.
> Transaction: Given to johnhg (John Halley Gotway) by mccabe
> Queue: met_help
> Subject: ROC diagrams
> Owner: johnhg
> Requestors: efren.serra.ctr at nrlmry.navy.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97513
> >
>
>
> This transaction appears to have no content
>
------------------------------------------------
Subject: ROC diagrams
From: John Halley Gotway
Time: Fri Nov 20 11:18:58 2020
Efren,
I just used METviewer to create a sample ROC plot to test. Please see
attached image, xml file, and Rscript.
Here are lines 52-54 of that Rscript:
52 # initialize the plot data, appending a beginning and end
point
53 listPOFD = c(1, dfROC$pofd, 0);
54 listPODY = c(1, dfROC$pody, 0);
Notice that 1 and 0 are tacked on the ends of the points to be
plotted.
FYI, you can replicate this yourself in METviewer, by...
(1) Saving the XML file to your machine.
(2) Go to http://dtcenter.ucar.edu/met/metviewer/metviewer1.jsp
(3) Click "Load XML" button in the top-right corner.
(4) Navigate to and select the XML file you downloaded.
(5) Click "Generate Plot" button in the top middle of the METviewer
GUI.
Thanks,
John
On Thu, Nov 19, 2020 at 12:47 PM efren.serra.ctr at nrlmry.navy.mil via
RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97513 >
>
> John - For the ROC plots, we think that ROC value at (POFD=1,
PODY=1)
> should be probability threshold 0.0, even though there are (in our
case) 10
> POFD_i/PODY_i and 11 THRESH_i. Is this correct? How does your
METViewer
> does it?
>
> -----Original Message-----
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Wednesday, November 18, 2020 1:05 PM
> To: Serra, Mr. Efren, Contractor, Code 7531 <
> efren.serra.ctr at nrlmry.navy.mil>
> Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> Subject: Re: [rt.rap.ucar.edu #97513] ROC diagrams
>
> Efren,
>
> When making a ROC diagram, you usually include both points (0,0) and
(1,1).
>
> That should result in a plot more similar this the one in Appendix C
of
> the MET User's Guide:
>
>
https://dtcenter.github.io/MET/latest/Users_Guide/appendixC.html?highlight=roc#receiver-
operating-characteristic
>
> John
>
> On Wed, Nov 18, 2020 at 1:34 PM George McCabe via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > Wed Nov 18 13:34:31 2020: Request 97513 was acted upon.
> > Transaction: Given to johnhg (John Halley Gotway) by mccabe
> > Queue: met_help
> > Subject: ROC diagrams
> > Owner: johnhg
> > Requestors: efren.serra.ctr at nrlmry.navy.mil
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97513
> > >
> >
> >
> > This transaction appears to have no content
> >
>
>
>
>
------------------------------------------------
Subject: ROC diagrams
From: efren.serra.ctr at nrlmry.navy.mil
Time: Fri Nov 20 11:49:30 2020
John - Thanks a lot! This is precisely how we are currently doing
this. Here's a latest plot.
-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Friday, November 20, 2020 10:19 AM
To: Serra, Mr. Efren, Contractor, Code 7531
<efren.serra.ctr at nrlmry.navy.mil>
Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
Subject: Re: [rt.rap.ucar.edu #97513] ROC diagrams
Efren,
I just used METviewer to create a sample ROC plot to test. Please see
attached image, xml file, and Rscript.
Here are lines 52-54 of that Rscript:
52 # initialize the plot data, appending a beginning and end
point
53 listPOFD = c(1, dfROC$pofd, 0);
54 listPODY = c(1, dfROC$pody, 0);
Notice that 1 and 0 are tacked on the ends of the points to be
plotted.
FYI, you can replicate this yourself in METviewer, by...
(1) Saving the XML file to your machine.
(2) Go to http://dtcenter.ucar.edu/met/metviewer/metviewer1.jsp
(3) Click "Load XML" button in the top-right corner.
(4) Navigate to and select the XML file you downloaded.
(5) Click "Generate Plot" button in the top middle of the METviewer
GUI.
Thanks,
John
On Thu, Nov 19, 2020 at 12:47 PM efren.serra.ctr at nrlmry.navy.mil via
RT < met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97513 >
>
> John - For the ROC plots, we think that ROC value at (POFD=1,
PODY=1)
> should be probability threshold 0.0, even though there are (in our
> case) 10 POFD_i/PODY_i and 11 THRESH_i. Is this correct? How does
your
> METViewer does it?
>
> -----Original Message-----
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Wednesday, November 18, 2020 1:05 PM
> To: Serra, Mr. Efren, Contractor, Code 7531 <
> efren.serra.ctr at nrlmry.navy.mil>
> Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> Subject: Re: [rt.rap.ucar.edu #97513] ROC diagrams
>
> Efren,
>
> When making a ROC diagram, you usually include both points (0,0) and
(1,1).
>
> That should result in a plot more similar this the one in Appendix C
> of the MET User's Guide:
>
>
https://dtcenter.github.io/MET/latest/Users_Guide/appendixC.html?highl
> ight=roc#receiver-operating-characteristic
>
> John
>
> On Wed, Nov 18, 2020 at 1:34 PM George McCabe via RT
> <met_help at ucar.edu>
> wrote:
>
> >
> > Wed Nov 18 13:34:31 2020: Request 97513 was acted upon.
> > Transaction: Given to johnhg (John Halley Gotway) by mccabe
> > Queue: met_help
> > Subject: ROC diagrams
> > Owner: johnhg
> > Requestors: efren.serra.ctr at nrlmry.navy.mil
> > Status: new
> > Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97513
> > >
> >
> >
> > This transaction appears to have no content
> >
>
>
>
>
------------------------------------------------
Subject: ROC diagrams
From: John Halley Gotway
Time: Fri Nov 20 12:35:26 2020
Looks great, thanks. I'll go ahead and resolve this ticket.
John
On Fri, Nov 20, 2020 at 11:50 AM efren.serra.ctr at nrlmry.navy.mil via
RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97513 >
>
> John - Thanks a lot! This is precisely how we are currently doing
this.
> Here's a latest plot.
>
> -----Original Message-----
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Friday, November 20, 2020 10:19 AM
> To: Serra, Mr. Efren, Contractor, Code 7531 <
> efren.serra.ctr at nrlmry.navy.mil>
> Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> Subject: Re: [rt.rap.ucar.edu #97513] ROC diagrams
>
> Efren,
>
> I just used METviewer to create a sample ROC plot to test. Please
see
> attached image, xml file, and Rscript.
>
> Here are lines 52-54 of that Rscript:
> 52 # initialize the plot data, appending a beginning and end
point
> 53 listPOFD = c(1, dfROC$pofd, 0);
> 54 listPODY = c(1, dfROC$pody, 0);
>
> Notice that 1 and 0 are tacked on the ends of the points to be
plotted.
>
> FYI, you can replicate this yourself in METviewer, by...
> (1) Saving the XML file to your machine.
> (2) Go to http://dtcenter.ucar.edu/met/metviewer/metviewer1.jsp
> (3) Click "Load XML" button in the top-right corner.
> (4) Navigate to and select the XML file you downloaded.
> (5) Click "Generate Plot" button in the top middle of the METviewer
GUI.
>
> Thanks,
> John
>
> On Thu, Nov 19, 2020 at 12:47 PM efren.serra.ctr at nrlmry.navy.mil via
RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97513 >
> >
> > John - For the ROC plots, we think that ROC value at (POFD=1,
PODY=1)
> > should be probability threshold 0.0, even though there are (in our
> > case) 10 POFD_i/PODY_i and 11 THRESH_i. Is this correct? How does
your
> > METViewer does it?
> >
> > -----Original Message-----
> > From: John Halley Gotway via RT <met_help at ucar.edu>
> > Sent: Wednesday, November 18, 2020 1:05 PM
> > To: Serra, Mr. Efren, Contractor, Code 7531 <
> > efren.serra.ctr at nrlmry.navy.mil>
> > Cc: Sampson, Mr. Buck <Buck.Sampson at nrlmry.navy.mil>
> > Subject: Re: [rt.rap.ucar.edu #97513] ROC diagrams
> >
> > Efren,
> >
> > When making a ROC diagram, you usually include both points (0,0)
and
> (1,1).
> >
> > That should result in a plot more similar this the one in Appendix
C
> > of the MET User's Guide:
> >
> >
https://dtcenter.github.io/MET/latest/Users_Guide/appendixC.html?highl
> > ight=roc#receiver-operating-characteristic
> >
> > John
> >
> > On Wed, Nov 18, 2020 at 1:34 PM George McCabe via RT
> > <met_help at ucar.edu>
> > wrote:
> >
> > >
> > > Wed Nov 18 13:34:31 2020: Request 97513 was acted upon.
> > > Transaction: Given to johnhg (John Halley Gotway) by mccabe
> > > Queue: met_help
> > > Subject: ROC diagrams
> > > Owner: johnhg
> > > Requestors: efren.serra.ctr at nrlmry.navy.mil
> > > Status: new
> > > Ticket <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97513
> > > >
> > >
> > >
> > > This transaction appears to have no content
> > >
> >
> >
> >
> >
>
>
>
------------------------------------------------
More information about the Met_help
mailing list