[Met_help] [rt.rap.ucar.edu #97167] History for Bootstrapping FSS in METviewer

Thu Nov 5 09:35:05 MST 2020

----------------------------------------------------------------
  Initial Request
----------------------------------------------------------------

Hello!

I would like to add bootstrap confidence intervals to my aggregate
Fractions Skill Scores (FSS) calculated with MET.
Although Stat-Analysis doesn't have this ability, MET Viewer apparently
does, according to this page from MET-help archive:
https://mailman.ucar.edu/pipermail/met_help/2019-July/003502.html
Do you still recommend we use the MET Viewer? I briefly poked around the
web application, but didn't find any database examples with FSS.
If yes, should I start with the containerized version of MET Viewer, or is
there a dedicated module on NCAR HPCs like cheyenne or casper? I'm
sorry--I'm having trouble finding information on this via the usual
googling.

Dave

----------------------------------------------------------------
  Complete Ticket History
----------------------------------------------------------------

Subject: Bootstrapping FSS in METviewer
From: John Halley Gotway
Time: Wed Oct 21 12:01:30 2020

David,

I used your latest question to create a new met-help ticket instead of
continuing the older one.

As of right now, there is no instance of METviewer running on
cheyenne, and I don't expect there to be one because of the need for a
database. However, our team is in the process of refactoring/breaking
METviewer down into smaller pieces, including METplotpy and
METcalcpy... for plotting and computations in python, respectively.
The goal is for those packages to be able to run standalone reading
from flat files instead of requiring a database. But that doesn't
actually exist yet. So I don't have an example to share.

So yes, I would recommend running METviewer on a laptop using a Docker
container. In fact, just last week I posted a training video
describing how to do that:

https://dtcenter.github.io/METplus-
Training/modules/METviewer/docker.html

Please LMK if you have any feedback on that training content.

As for an example using existing data to compute bootstrap CI's for
FSS, please go to this instance of METviewer:
http://dtcenter.ucar.edu/met/metviewer/metviewer1.jsp

You can use the attached .png/.xml to upload there and play around
with the example. I created FSS output when comparing CCPA and StageIV
precip datasets to quantify how similar they are.

Thanks,
John

------------------------------------------------
Subject: Bootstrapping FSS in METviewer
From: David Ahijevych
Time: Wed Oct 21 12:06:00 2020

Very nice! This looks like exactly what I need.

On Wed, Oct 21, 2020 at 12:02 PM John Halley Gotway via RT <
met_help at ucar.edu> wrote:

> David,
>
> I used your latest question to create a new met-help ticket instead
of
> continuing the older one.
>
> As of right now, there is no instance of METviewer running on
cheyenne,
> and I don't expect there to be one because of the need for a
database.
> However, our team is in the process of refactoring/breaking
METviewer down
> into smaller pieces, including METplotpy and METcalcpy... for
plotting and
> computations in python, respectively. The goal is for those packages
to be
> able to run standalone reading from flat files instead of requiring
a
> database. But that doesn't actually exist yet. So I don't have an
example
> to share.
>
> So yes, I would recommend running METviewer on a laptop using a
Docker
> container. In fact, just last week I posted a training video
describing how
> to do that:
>
> https://dtcenter.github.io/METplus-
Training/modules/METviewer/docker.html
>
> Please LMK if you have any feedback on that training content.
>
> As for an example using existing data to compute bootstrap CI's for
FSS,
> please go to this instance of METviewer:
> http://dtcenter.ucar.edu/met/metviewer/metviewer1.jsp
>
> You can use the attached .png/.xml to upload there and play around
with
> the example. I created FSS output when comparing CCPA and StageIV
precip
> datasets to quantify how similar they are.
>
> Thanks,
> John
>
>
>
> mohawk mv_hmt_ccpa_st4 ****** ****** mariadb
/usr/local/R/bin/Rscript
> /opt/vxwww/tomcat/webapps/metviewer//R_tmpl
> /opt/vxwww/tomcat/webapps/metviewer//R_work
> /d2/www/dtcenter/met/metviewer_output//plots
> /d2/www/dtcenter/met/metviewer_output//data
> /d2/www/dtcenter/met/metviewer_output//scripts series_plot.R_tmpl
NBR_FSS
> CCPA_vs_ST4 >=2.540 >=6.350 >=12.700 >=25.400 >=50.800 NBRHD 9 25 49
true
> 1000 perc false false median plot_bootstrap_fss_example.data
> plot_bootstrap_fss_example.png plot_bootstrap_fss_example.R
Neighborhood
> Size FSS
> bootstrap_fss_example false list() list() Rscript false false false
false
> true false true false false false false false true png16m 8.5 11 72
in
> c(8,4,5,4) c(1,1,0) 1 2 1.4 -2 0.5 1 -0.75 0.5 0 1 1 1 2 0.5 1 0.5
0.5 1 1
> 1 -2 0.5 3 #cccccc 1 listX 1 1 0.5 0.8 0.8 -0.5 0.5 1 0.5 0.5 1 1 1
0.5 o c(0,
> -.25) 3 0.8 1 #333333 0.8 3 0 0.05
c("boot","boot","boot","boot","boot")
> c(FALSE,FALSE,FALSE,FALSE,FALSE) c(TRUE,TRUE,TRUE,TRUE,TRUE)
> c("#ff0000FF","#8000ffFF","#00ff7fFF","#55ff00FF","#bfff00FF")
> c(20,20,20,20,20) c("b","b","b","b","b") c(1,1,1,1,1) c(1,1,1,1,1)
> c(1,1,1,1,1) c(1,2,3,4,5) c("","","","","") c() c() 0.04 c()

------------------------------------------------
Subject: Bootstrapping FSS in METviewer
From: David Ahijevych
Time: Fri Oct 23 12:24:40 2020

Thank you for your assistance. Nice tutorial! I set up the docker on
my
work laptop.

On Fri, Oct 23, 2020 at 10:50 AM John Halley Gotway via RT <
met_help at ucar.edu> wrote:

> According to our records, your request has been resolved. If you
have any
> further questions or concerns, please respond to this message.
>

------------------------------------------------
Subject: Bootstrapping FSS in METviewer
From: David Ahijevych
Time: Wed Nov 04 18:58:46 2020

This may be a dumb question, but was there supposed to be another
attachment? Or do I copy and paste the text at the end of your email
into
an uploadable .xml file?

On Wed, Oct 21, 2020 at 12:05 PM David Ahijevych <ahijevyc at ucar.edu>
wrote:

> Very nice! This looks like exactly what I need.
>
>
> On Wed, Oct 21, 2020 at 12:02 PM John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
>> David,
>>
>> I used your latest question to create a new met-help ticket instead
of
>> continuing the older one.
>>
>> As of right now, there is no instance of METviewer running on
cheyenne,
>> and I don't expect there to be one because of the need for a
database.
>> However, our team is in the process of refactoring/breaking
METviewer down
>> into smaller pieces, including METplotpy and METcalcpy... for
plotting and
>> computations in python, respectively. The goal is for those
packages to be
>> able to run standalone reading from flat files instead of requiring
a
>> database. But that doesn't actually exist yet. So I don't have an
example
>> to share.
>>
>> So yes, I would recommend running METviewer on a laptop using a
Docker
>> container. In fact, just last week I posted a training video
describing how
>> to do that:
>>
>> https://dtcenter.github.io/METplus-
Training/modules/METviewer/docker.html
>>
>> Please LMK if you have any feedback on that training content.
>>
>> As for an example using existing data to compute bootstrap CI's for
FSS,
>> please go to this instance of METviewer:
>> http://dtcenter.ucar.edu/met/metviewer/metviewer1.jsp
>>
>> You can use the attached .png/.xml to upload there and play around
with
>> the example. I created FSS output when comparing CCPA and StageIV
precip
>> datasets to quantify how similar they are.
>>
>> Thanks,
>> John
>>
>>
>>
>> mohawk mv_hmt_ccpa_st4 ****** ****** mariadb
/usr/local/R/bin/Rscript
>> /opt/vxwww/tomcat/webapps/metviewer//R_tmpl
>> /opt/vxwww/tomcat/webapps/metviewer//R_work
>> /d2/www/dtcenter/met/metviewer_output//plots
>> /d2/www/dtcenter/met/metviewer_output//data
>> /d2/www/dtcenter/met/metviewer_output//scripts series_plot.R_tmpl
NBR_FSS
>> CCPA_vs_ST4 >=2.540 >=6.350 >=12.700 >=25.400 >=50.800 NBRHD 9 25
49 true
>> 1000 perc false false median plot_bootstrap_fss_example.data
>> plot_bootstrap_fss_example.png plot_bootstrap_fss_example.R
Neighborhood
>> Size FSS
>> bootstrap_fss_example false list() list() Rscript false false false
false
>> true false true false false false false false true png16m 8.5 11 72
in
>> c(8,4,5,4) c(1,1,0) 1 2 1.4 -2 0.5 1 -0.75 0.5 0 1 1 1 2 0.5 1 0.5
0.5 1
>> 1 1 -2 0.5 3 #cccccc 1 listX 1 1 0.5 0.8 0.8 -0.5 0.5 1 0.5 0.5 1 1
1 0.5
>> o c(0, -.25) 3 0.8 1 #333333 0.8 3 0 0.05
>> c("boot","boot","boot","boot","boot")
c(FALSE,FALSE,FALSE,FALSE,FALSE)
>> c(TRUE,TRUE,TRUE,TRUE,TRUE)
>> c("#ff0000FF","#8000ffFF","#00ff7fFF","#55ff00FF","#bfff00FF")
>> c(20,20,20,20,20) c("b","b","b","b","b") c(1,1,1,1,1) c(1,1,1,1,1)
>> c(1,1,1,1,1) c(1,2,3,4,5) c("","","","","") c() c() 0.04 c()
>
>

------------------------------------------------
Subject: Bootstrapping FSS in METviewer
From: John Halley Gotway
Time: Thu Nov 05 09:13:38 2020

David,

Yes, sorry, looks like I forgot the attachment. Good news, I was able
to
find it on the METviewer server before it was aged off. Please see
attached.

Thanks,
John

On Wed, Nov 4, 2020 at 6:59 PM David Ahijevych via RT
<met_help at ucar.edu>
wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97167 >
>
> This may be a dumb question, but was there supposed to be another
> attachment? Or do I copy and paste the text at the end of your email
into
> an uploadable .xml file?
>
>
> On Wed, Oct 21, 2020 at 12:05 PM David Ahijevych <ahijevyc at ucar.edu>
> wrote:
>
> > Very nice! This looks like exactly what I need.
> >
> >
> > On Wed, Oct 21, 2020 at 12:02 PM John Halley Gotway via RT <
> > met_help at ucar.edu> wrote:
> >
> >> David,
> >>
> >> I used your latest question to create a new met-help ticket
instead of
> >> continuing the older one.
> >>
> >> As of right now, there is no instance of METviewer running on
cheyenne,
> >> and I don't expect there to be one because of the need for a
database.
> >> However, our team is in the process of refactoring/breaking
METviewer
> down
> >> into smaller pieces, including METplotpy and METcalcpy... for
plotting
> and
> >> computations in python, respectively. The goal is for those
packages to
> be
> >> able to run standalone reading from flat files instead of
requiring a
> >> database. But that doesn't actually exist yet. So I don't have an
> example
> >> to share.
> >>
> >> So yes, I would recommend running METviewer on a laptop using a
Docker
> >> container. In fact, just last week I posted a training video
describing
> how
> >> to do that:
> >>
> >>
> https://dtcenter.github.io/METplus-
Training/modules/METviewer/docker.html
> >>
> >> Please LMK if you have any feedback on that training content.
> >>
> >> As for an example using existing data to compute bootstrap CI's
for FSS,
> >> please go to this instance of METviewer:
> >> http://dtcenter.ucar.edu/met/metviewer/metviewer1.jsp
> >>
> >> You can use the attached .png/.xml to upload there and play
around with
> >> the example. I created FSS output when comparing CCPA and StageIV
precip
> >> datasets to quantify how similar they are.
> >>
> >> Thanks,
> >> John
> >>
> >>
> >>
> >> mohawk mv_hmt_ccpa_st4 ****** ****** mariadb
/usr/local/R/bin/Rscript
> >> /opt/vxwww/tomcat/webapps/metviewer//R_tmpl
> >> /opt/vxwww/tomcat/webapps/metviewer//R_work
> >> /d2/www/dtcenter/met/metviewer_output//plots
> >> /d2/www/dtcenter/met/metviewer_output//data
> >> /d2/www/dtcenter/met/metviewer_output//scripts series_plot.R_tmpl
> NBR_FSS
> >> CCPA_vs_ST4 >=2.540 >=6.350 >=12.700 >=25.400 >=50.800 NBRHD 9 25
49
> true
> >> 1000 perc false false median plot_bootstrap_fss_example.data
> >> plot_bootstrap_fss_example.png plot_bootstrap_fss_example.R
Neighborhood
> >> Size FSS
> >> bootstrap_fss_example false list() list() Rscript false false
false
> false
> >> true false true false false false false false true png16m 8.5 11
72 in
> >> c(8,4,5,4) c(1,1,0) 1 2 1.4 -2 0.5 1 -0.75 0.5 0 1 1 1 2 0.5 1
0.5 0.5 1
> >> 1 1 -2 0.5 3 #cccccc 1 listX 1 1 0.5 0.8 0.8 -0.5 0.5 1 0.5 0.5 1
1 1
> 0.5
> >> o c(0, -.25) 3 0.8 1 #333333 0.8 3 0 0.05
> >> c("boot","boot","boot","boot","boot")
c(FALSE,FALSE,FALSE,FALSE,FALSE)
> >> c(TRUE,TRUE,TRUE,TRUE,TRUE)
> >> c("#ff0000FF","#8000ffFF","#00ff7fFF","#55ff00FF","#bfff00FF")
> >> c(20,20,20,20,20) c("b","b","b","b","b") c(1,1,1,1,1)
c(1,1,1,1,1)
> >> c(1,1,1,1,1) c(1,2,3,4,5) c("","","","","") c() c() 0.04 c()
> >
> >
>
>

------------------------------------------------
Subject: Bootstrapping FSS in METviewer
From: David Ahijevych
Time: Thu Nov 05 09:20:08 2020

Great, John, thank you. Now I understand how to upload xml.
Good thing is: I replicated the plot by poking around METviewer on my
own.
Not having the xml file forced me to learn.

On Thu, Nov 5, 2020 at 9:14 AM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:

> David,
>
> Yes, sorry, looks like I forgot the attachment. Good news, I was
able to
> find it on the METviewer server before it was aged off. Please see
> attached.
>
> Thanks,
> John
>
> On Wed, Nov 4, 2020 at 6:59 PM David Ahijevych via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97167 >
> >
> > This may be a dumb question, but was there supposed to be another
> > attachment? Or do I copy and paste the text at the end of your
email into
> > an uploadable .xml file?
> >
> >
> > On Wed, Oct 21, 2020 at 12:05 PM David Ahijevych
<ahijevyc at ucar.edu>
> > wrote:
> >
> > > Very nice! This looks like exactly what I need.
> > >
> > >
> > > On Wed, Oct 21, 2020 at 12:02 PM John Halley Gotway via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > >> David,
> > >>
> > >> I used your latest question to create a new met-help ticket
instead of
> > >> continuing the older one.
> > >>
> > >> As of right now, there is no instance of METviewer running on
> cheyenne,
> > >> and I don't expect there to be one because of the need for a
database.
> > >> However, our team is in the process of refactoring/breaking
METviewer
> > down
> > >> into smaller pieces, including METplotpy and METcalcpy... for
plotting
> > and
> > >> computations in python, respectively. The goal is for those
packages
> to
> > be
> > >> able to run standalone reading from flat files instead of
requiring a
> > >> database. But that doesn't actually exist yet. So I don't have
an
> > example
> > >> to share.
> > >>
> > >> So yes, I would recommend running METviewer on a laptop using a
Docker
> > >> container. In fact, just last week I posted a training video
> describing
> > how
> > >> to do that:
> > >>
> > >>
> >
> https://dtcenter.github.io/METplus-
Training/modules/METviewer/docker.html
> > >>
> > >> Please LMK if you have any feedback on that training content.
> > >>
> > >> As for an example using existing data to compute bootstrap CI's
for
> FSS,
> > >> please go to this instance of METviewer:
> > >> http://dtcenter.ucar.edu/met/metviewer/metviewer1.jsp
> > >>
> > >> You can use the attached .png/.xml to upload there and play
around
> with
> > >> the example. I created FSS output when comparing CCPA and
StageIV
> precip
> > >> datasets to quantify how similar they are.
> > >>
> > >> Thanks,
> > >> John
> > >>
> > >>
> > >>
> > >> mohawk mv_hmt_ccpa_st4 ****** ****** mariadb
/usr/local/R/bin/Rscript
> > >> /opt/vxwww/tomcat/webapps/metviewer//R_tmpl
> > >> /opt/vxwww/tomcat/webapps/metviewer//R_work
> > >> /d2/www/dtcenter/met/metviewer_output//plots
> > >> /d2/www/dtcenter/met/metviewer_output//data
> > >> /d2/www/dtcenter/met/metviewer_output//scripts
series_plot.R_tmpl
> > NBR_FSS
> > >> CCPA_vs_ST4 >=2.540 >=6.350 >=12.700 >=25.400 >=50.800 NBRHD 9
25 49
> > true
> > >> 1000 perc false false median plot_bootstrap_fss_example.data
> > >> plot_bootstrap_fss_example.png plot_bootstrap_fss_example.R
> Neighborhood
> > >> Size FSS
> > >> bootstrap_fss_example false list() list() Rscript false false
false
> > false
> > >> true false true false false false false false true png16m 8.5
11 72 in
> > >> c(8,4,5,4) c(1,1,0) 1 2 1.4 -2 0.5 1 -0.75 0.5 0 1 1 1 2 0.5 1
0.5
> 0.5 1
> > >> 1 1 -2 0.5 3 #cccccc 1 listX 1 1 0.5 0.8 0.8 -0.5 0.5 1 0.5 0.5
1 1 1
> > 0.5
> > >> o c(0, -.25) 3 0.8 1 #333333 0.8 3 0 0.05
> > >> c("boot","boot","boot","boot","boot")
c(FALSE,FALSE,FALSE,FALSE,FALSE)
> > >> c(TRUE,TRUE,TRUE,TRUE,TRUE)
> > >> c("#ff0000FF","#8000ffFF","#00ff7fFF","#55ff00FF","#bfff00FF")
> > >> c(20,20,20,20,20) c("b","b","b","b","b") c(1,1,1,1,1)
c(1,1,1,1,1)
> > >> c(1,1,1,1,1) c(1,2,3,4,5) c("","","","","") c() c() 0.04 c()
> > >
> > >
> >
> >
>
>

------------------------------------------------