[Met_help] [rt.rap.ucar.edu #95896] History for Stat Anal Confidence Intervals
John Halley Gotway via RT
met_help at ucar.edu
Tue Jul 14 17:18:04 MDT 2020
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
John, I asked this question before but didn't get back to you on your answer
- you suggested something I had already done. In stat_analysis, I am
running aggregate_stat over a month's worth of data. In the configuration
file, I do have the n_rep set to 1000 but I am getting NAs for RMSE_BCL and
RMSE_BCU. My config file is attached. The command line is
/h/MET/bin/stat_analysis -lookin
/h/data/global/WXQC/data/met/ptstat/stats/gfs/TT -out
/h/data/global/WXQC/data/met/ptstat/stats/monthly/stat_anal_gfs_FULL_TT_Z2_2
0200601_20200630.txt -tmp_dir /h/data/global/WXQC/data/met/tmp -job
aggregate_stat -line_type SL1L2 -out_line_type CNT -fcst_init_beg
20200601_000000 -fcst_init_end 20200630_180000 -by fcst_lead -fcst_lev Z2
-by model -vx_mask FULL -by vx_mask -by fcst_init_hour -config
/h/data/global/WXQC/data/met/config/met_config/STATAnalysisConfig_bootstrap_
aggstat -v 4.0. I sent the data to you on AMRSAFE is you need it.
The listing when I run the job is below:
[1241 qcteam hpc10-lsn-02p
/h/data/global/WXQC/data/met/ptstat/stats/galwem/TT]
ig/STATAnalysisConfig_bootstrap_aggstat -v 4.0
<
DEBUG 1: Reading user-defined grib1 MET_GRIB_TABLES file:
/h/data/global/WXQC/data/met/config/grib2/grib1_af_131_57.txt
DEBUG 1: Reading user-defined grib2 MET_GRIB_TABLES file:
/h/data/global/WXQC/data/met/config/grib2/grib2_geps.txt
DEBUG 1: Creating STAT-Analysis output file
"/h/data/global/WXQC/data/met/ptstat/stats/monthly/stat_anal_gfs_FULL_TT_Z2_
20200601_20200630.txt"
DEBUG 1: Default Config File:
/h/MET/share/met/config/STATAnalysisConfig_default
DEBUG 1: User Config File:
/h/data/global/WXQC/data/met/config/met_config/STATAnalysisConfig_bootstrap_
aggstat
DEBUG 4: Default Job from the config file: "(nul)"
DEBUG 4: Amending default job with command line options: "-job
aggregate_stat -line_type SL1L2 -out_line_type CNT -fcst_init_beg
20200601_000000 -fcst_init_end 20200630_180000 -by fcst_lead -fcst_lev Z2
-by model -vx_mask FULL -by vx_mask -by fcst_init_hour"
DEBUG 4: New default jobstring: "-job aggregate_stat -fcst_init_beg
20200601_000000 -fcst_init_end 20200630_180000 -fcst_lev Z2 -vx_mask FULL
-line_type SL1L2 -by FCST_LEAD -by MODEL -by VX_MASK -by FCST_INIT_HOUR
-out_line_type CNT -out_alpha 0.05000 -rank_corr_flag 0 "
DEBUG 2: Processing 202 STAT files.
DEBUG 3: Processing STAT file
"/h/data/global/WXQC/data/met/ptstat/stats/galwem/TT/temp/point_stat_galwem_
all_SFC_240000L_20200610_000000V.stat" .... 1 of 202
What am I missing?
Thanks
Bob
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Stat Anal Confidence Intervals
From: Tara Jensen
Time: Mon Jul 13 13:54:40 2020
Hi Bob,
John is not in the office right now. Can you send me the data via
AMRSAFE
so I can run the case and see if I can help you?
Thanks, Tara
On Mon, Jul 13, 2020 at 1:02 PM robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> Mon Jul 13 13:02:02 2020: Request 95896 was acted upon.
> Transaction: Ticket created by robert.craig.2 at us.af.mil
> Queue: met_help
> Subject: Stat Anal Confidence Intervals
> Owner: Nobody
> Requestors: robert.craig.2 at us.af.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95896 >
>
>
> John, I asked this question before but didn't get back to you on
your
> answer
> - you suggested something I had already done. In stat_analysis, I
am
> running aggregate_stat over a month's worth of data. In the
configuration
> file, I do have the n_rep set to 1000 but I am getting NAs for
RMSE_BCL and
> RMSE_BCU. My config file is attached. The command line is
> /h/MET/bin/stat_analysis -lookin
> /h/data/global/WXQC/data/met/ptstat/stats/gfs/TT -out
>
>
/h/data/global/WXQC/data/met/ptstat/stats/monthly/stat_anal_gfs_FULL_TT_Z2_2
> 0200601_20200630.txt -tmp_dir /h/data/global/WXQC/data/met/tmp -job
> aggregate_stat -line_type SL1L2 -out_line_type CNT -fcst_init_beg
> 20200601_000000 -fcst_init_end 20200630_180000 -by fcst_lead
-fcst_lev Z2
> -by model -vx_mask FULL -by vx_mask -by fcst_init_hour -config
>
>
/h/data/global/WXQC/data/met/config/met_config/STATAnalysisConfig_bootstrap_
> aggstat -v 4.0. I sent the data to you on AMRSAFE is you need it.
>
>
>
> The listing when I run the job is below:
>
> [1241 qcteam hpc10-lsn-02p
> /h/data/global/WXQC/data/met/ptstat/stats/galwem/TT]
> ig/STATAnalysisConfig_bootstrap_aggstat -v 4.0
> <
>
> DEBUG 1: Reading user-defined grib1 MET_GRIB_TABLES file:
> /h/data/global/WXQC/data/met/config/grib2/grib1_af_131_57.txt
>
> DEBUG 1: Reading user-defined grib2 MET_GRIB_TABLES file:
> /h/data/global/WXQC/data/met/config/grib2/grib2_geps.txt
>
> DEBUG 1: Creating STAT-Analysis output file
>
>
"/h/data/global/WXQC/data/met/ptstat/stats/monthly/stat_anal_gfs_FULL_TT_Z2_
> 20200601_20200630.txt"
>
> DEBUG 1: Default Config File:
> /h/MET/share/met/config/STATAnalysisConfig_default
>
> DEBUG 1: User Config File:
>
>
/h/data/global/WXQC/data/met/config/met_config/STATAnalysisConfig_bootstrap_
> aggstat
>
> DEBUG 4: Default Job from the config file: "(nul)"
>
> DEBUG 4: Amending default job with command line options: "-job
> aggregate_stat -line_type SL1L2 -out_line_type CNT -fcst_init_beg
> 20200601_000000 -fcst_init_end 20200630_180000 -by fcst_lead
-fcst_lev Z2
> -by model -vx_mask FULL -by vx_mask -by fcst_init_hour"
>
> DEBUG 4: New default jobstring: "-job aggregate_stat -fcst_init_beg
> 20200601_000000 -fcst_init_end 20200630_180000 -fcst_lev Z2 -vx_mask
FULL
> -line_type SL1L2 -by FCST_LEAD -by MODEL -by VX_MASK -by
FCST_INIT_HOUR
> -out_line_type CNT -out_alpha 0.05000 -rank_corr_flag 0 "
>
> DEBUG 2: Processing 202 STAT files.
>
> DEBUG 3: Processing STAT file
>
>
"/h/data/global/WXQC/data/met/ptstat/stats/galwem/TT/temp/point_stat_galwem_
> all_SFC_240000L_20200610_000000V.stat" .... 1 of 202
>
>
>
>
>
> What am I missing?
>
>
>
> Thanks
>
> Bob
>
>
>
--
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Tara Jensen
Project Manager II
NCAR RAL and DTC
PO Box 3000, Boulder, Colorado 80307 USA
+1 303-497-8479 jensen at ucar.edu
------------------------------------------------
Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95896] Stat Anal Confidence Intervals
From: robert.craig.2 at us.af.mil
Time: Mon Jul 13 13:58:04 2020
Okay, what is your email? AMRSAFE won't work to met_help.
-----Original Message-----
From: Tara Jensen via RT <met_help at ucar.edu>
Sent: Monday, July 13, 2020 2:55 PM
To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
<robert.craig.2 at us.af.mil>
Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #95896] Stat Anal
Confidence Intervals
Hi Bob,
John is not in the office right now. Can you send me the data via
AMRSAFE so I can run the case and see if I can help you?
Thanks, Tara
On Mon, Jul 13, 2020 at 1:02 PM robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> Mon Jul 13 13:02:02 2020: Request 95896 was acted upon.
> Transaction: Ticket created by robert.craig.2 at us.af.mil
> Queue: met_help
> Subject: Stat Anal Confidence Intervals
> Owner: Nobody
> Requestors: robert.craig.2 at us.af.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95896
> >
>
>
> John, I asked this question before but didn't get back to you on
your
> answer
> - you suggested something I had already done. In stat_analysis, I
am
> running aggregate_stat over a month's worth of data. In the
> configuration file, I do have the n_rep set to 1000 but I am getting
NAs for RMSE_BCL and
> RMSE_BCU. My config file is attached. The command line is
> /h/MET/bin/stat_analysis -lookin
> /h/data/global/WXQC/data/met/ptstat/stats/gfs/TT -out
>
>
/h/data/global/WXQC/data/met/ptstat/stats/monthly/stat_anal_gfs_FULL_T
> T_Z2_2 0200601_20200630.txt -tmp_dir
/h/data/global/WXQC/data/met/tmp
> -job aggregate_stat -line_type SL1L2 -out_line_type CNT
-fcst_init_beg
> 20200601_000000 -fcst_init_end 20200630_180000 -by fcst_lead
-fcst_lev
> Z2 -by model -vx_mask FULL -by vx_mask -by fcst_init_hour -config
>
>
/h/data/global/WXQC/data/met/config/met_config/STATAnalysisConfig_boot
> strap_ aggstat -v 4.0. I sent the data to you on AMRSAFE is you
need
> it.
>
>
>
> The listing when I run the job is below:
>
> [1241 qcteam hpc10-lsn-02p
> /h/data/global/WXQC/data/met/ptstat/stats/galwem/TT]
> ig/STATAnalysisConfig_bootstrap_aggstat -v 4.0 <
>
> DEBUG 1: Reading user-defined grib1 MET_GRIB_TABLES file:
> /h/data/global/WXQC/data/met/config/grib2/grib1_af_131_57.txt
>
> DEBUG 1: Reading user-defined grib2 MET_GRIB_TABLES file:
> /h/data/global/WXQC/data/met/config/grib2/grib2_geps.txt
>
> DEBUG 1: Creating STAT-Analysis output file
>
>
"/h/data/global/WXQC/data/met/ptstat/stats/monthly/stat_anal_gfs_FULL_
> TT_Z2_
> 20200601_20200630.txt"
>
> DEBUG 1: Default Config File:
> /h/MET/share/met/config/STATAnalysisConfig_default
>
> DEBUG 1: User Config File:
>
>
/h/data/global/WXQC/data/met/config/met_config/STATAnalysisConfig_boot
> strap_
> aggstat
>
> DEBUG 4: Default Job from the config file: "(nul)"
>
> DEBUG 4: Amending default job with command line options: "-job
> aggregate_stat -line_type SL1L2 -out_line_type CNT -fcst_init_beg
> 20200601_000000 -fcst_init_end 20200630_180000 -by fcst_lead
-fcst_lev
> Z2 -by model -vx_mask FULL -by vx_mask -by fcst_init_hour"
>
> DEBUG 4: New default jobstring: "-job aggregate_stat -fcst_init_beg
> 20200601_000000 -fcst_init_end 20200630_180000 -fcst_lev Z2 -vx_mask
> FULL -line_type SL1L2 -by FCST_LEAD -by MODEL -by VX_MASK -by
> FCST_INIT_HOUR -out_line_type CNT -out_alpha 0.05000 -rank_corr_flag
0 "
>
> DEBUG 2: Processing 202 STAT files.
>
> DEBUG 3: Processing STAT file
>
>
"/h/data/global/WXQC/data/met/ptstat/stats/galwem/TT/temp/point_stat_g
> alwem_ all_SFC_240000L_20200610_000000V.stat" .... 1 of 202
>
>
>
>
>
> What am I missing?
>
>
>
> Thanks
>
> Bob
>
>
>
--
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Tara Jensen
Project Manager II
NCAR RAL and DTC
PO Box 3000, Boulder, Colorado 80307 USA
+1 303-497-8479 jensen at ucar.edu
------------------------------------------------
Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95896] Stat Anal Confidence Intervals
From: robert.craig.2 at us.af.mil
Time: Mon Jul 13 14:01:29 2020
I sent it to jensen at ucar.edu.
Bob
-----Original Message-----
From: Tara Jensen via RT <met_help at ucar.edu>
Sent: Monday, July 13, 2020 2:55 PM
To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
<robert.craig.2 at us.af.mil>
Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #95896] Stat Anal
Confidence Intervals
Hi Bob,
John is not in the office right now. Can you send me the data via
AMRSAFE so I can run the case and see if I can help you?
Thanks, Tara
On Mon, Jul 13, 2020 at 1:02 PM robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> Mon Jul 13 13:02:02 2020: Request 95896 was acted upon.
> Transaction: Ticket created by robert.craig.2 at us.af.mil
> Queue: met_help
> Subject: Stat Anal Confidence Intervals
> Owner: Nobody
> Requestors: robert.craig.2 at us.af.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95896
> >
>
>
> John, I asked this question before but didn't get back to you on
your
> answer
> - you suggested something I had already done. In stat_analysis, I
am
> running aggregate_stat over a month's worth of data. In the
> configuration file, I do have the n_rep set to 1000 but I am getting
NAs for RMSE_BCL and
> RMSE_BCU. My config file is attached. The command line is
> /h/MET/bin/stat_analysis -lookin
> /h/data/global/WXQC/data/met/ptstat/stats/gfs/TT -out
>
>
/h/data/global/WXQC/data/met/ptstat/stats/monthly/stat_anal_gfs_FULL_T
> T_Z2_2 0200601_20200630.txt -tmp_dir
/h/data/global/WXQC/data/met/tmp
> -job aggregate_stat -line_type SL1L2 -out_line_type CNT
-fcst_init_beg
> 20200601_000000 -fcst_init_end 20200630_180000 -by fcst_lead
-fcst_lev
> Z2 -by model -vx_mask FULL -by vx_mask -by fcst_init_hour -config
>
>
/h/data/global/WXQC/data/met/config/met_config/STATAnalysisConfig_boot
> strap_ aggstat -v 4.0. I sent the data to you on AMRSAFE is you
need
> it.
>
>
>
> The listing when I run the job is below:
>
> [1241 qcteam hpc10-lsn-02p
> /h/data/global/WXQC/data/met/ptstat/stats/galwem/TT]
> ig/STATAnalysisConfig_bootstrap_aggstat -v 4.0 <
>
> DEBUG 1: Reading user-defined grib1 MET_GRIB_TABLES file:
> /h/data/global/WXQC/data/met/config/grib2/grib1_af_131_57.txt
>
> DEBUG 1: Reading user-defined grib2 MET_GRIB_TABLES file:
> /h/data/global/WXQC/data/met/config/grib2/grib2_geps.txt
>
> DEBUG 1: Creating STAT-Analysis output file
>
>
"/h/data/global/WXQC/data/met/ptstat/stats/monthly/stat_anal_gfs_FULL_
> TT_Z2_
> 20200601_20200630.txt"
>
> DEBUG 1: Default Config File:
> /h/MET/share/met/config/STATAnalysisConfig_default
>
> DEBUG 1: User Config File:
>
>
/h/data/global/WXQC/data/met/config/met_config/STATAnalysisConfig_boot
> strap_
> aggstat
>
> DEBUG 4: Default Job from the config file: "(nul)"
>
> DEBUG 4: Amending default job with command line options: "-job
> aggregate_stat -line_type SL1L2 -out_line_type CNT -fcst_init_beg
> 20200601_000000 -fcst_init_end 20200630_180000 -by fcst_lead
-fcst_lev
> Z2 -by model -vx_mask FULL -by vx_mask -by fcst_init_hour"
>
> DEBUG 4: New default jobstring: "-job aggregate_stat -fcst_init_beg
> 20200601_000000 -fcst_init_end 20200630_180000 -fcst_lev Z2 -vx_mask
> FULL -line_type SL1L2 -by FCST_LEAD -by MODEL -by VX_MASK -by
> FCST_INIT_HOUR -out_line_type CNT -out_alpha 0.05000 -rank_corr_flag
0 "
>
> DEBUG 2: Processing 202 STAT files.
>
> DEBUG 3: Processing STAT file
>
>
"/h/data/global/WXQC/data/met/ptstat/stats/galwem/TT/temp/point_stat_g
> alwem_ all_SFC_240000L_20200610_000000V.stat" .... 1 of 202
>
>
>
>
>
> What am I missing?
>
>
>
> Thanks
>
> Bob
>
>
>
--
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Tara Jensen
Project Manager II
NCAR RAL and DTC
PO Box 3000, Boulder, Colorado 80307 USA
+1 303-497-8479 jensen at ucar.edu
------------------------------------------------
Subject: Stat Anal Confidence Intervals
From: Tara Jensen
Time: Mon Jul 13 14:08:14 2020
Bob,
Thanks. I got the email and have downloaded. I'll take a look later
today.
Cheers, Tara
On Mon, Jul 13, 2020 at 2:02 PM robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95896 >
>
> I sent it to jensen at ucar.edu.
>
> Bob
>
> -----Original Message-----
> From: Tara Jensen via RT <met_help at ucar.edu>
> Sent: Monday, July 13, 2020 2:55 PM
> To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
<robert.craig.2 at us.af.mil>
> Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #95896] Stat Anal
> Confidence Intervals
>
> Hi Bob,
>
> John is not in the office right now. Can you send me the data via
AMRSAFE
> so I can run the case and see if I can help you?
>
> Thanks, Tara
>
> On Mon, Jul 13, 2020 at 1:02 PM robert.craig.2 at us.af.mil via RT <
> met_help at ucar.edu> wrote:
>
> >
> > Mon Jul 13 13:02:02 2020: Request 95896 was acted upon.
> > Transaction: Ticket created by robert.craig.2 at us.af.mil
> > Queue: met_help
> > Subject: Stat Anal Confidence Intervals
> > Owner: Nobody
> > Requestors: robert.craig.2 at us.af.mil
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95896
> > >
> >
> >
> > John, I asked this question before but didn't get back to you on
your
> > answer
> > - you suggested something I had already done. In stat_analysis, I
am
> > running aggregate_stat over a month's worth of data. In the
> > configuration file, I do have the n_rep set to 1000 but I am
getting NAs
> for RMSE_BCL and
> > RMSE_BCU. My config file is attached. The command line is
> > /h/MET/bin/stat_analysis -lookin
> > /h/data/global/WXQC/data/met/ptstat/stats/gfs/TT -out
> >
> >
/h/data/global/WXQC/data/met/ptstat/stats/monthly/stat_anal_gfs_FULL_T
> > T_Z2_2 0200601_20200630.txt -tmp_dir
/h/data/global/WXQC/data/met/tmp
> > -job aggregate_stat -line_type SL1L2 -out_line_type CNT
-fcst_init_beg
> > 20200601_000000 -fcst_init_end 20200630_180000 -by fcst_lead
-fcst_lev
> > Z2 -by model -vx_mask FULL -by vx_mask -by fcst_init_hour -config
> >
> >
/h/data/global/WXQC/data/met/config/met_config/STATAnalysisConfig_boot
> > strap_ aggstat -v 4.0. I sent the data to you on AMRSAFE is you
need
> > it.
> >
> >
> >
> > The listing when I run the job is below:
> >
> > [1241 qcteam hpc10-lsn-02p
> > /h/data/global/WXQC/data/met/ptstat/stats/galwem/TT]
> > ig/STATAnalysisConfig_bootstrap_aggstat -v 4.0 <
> >
> > DEBUG 1: Reading user-defined grib1 MET_GRIB_TABLES file:
> > /h/data/global/WXQC/data/met/config/grib2/grib1_af_131_57.txt
> >
> > DEBUG 1: Reading user-defined grib2 MET_GRIB_TABLES file:
> > /h/data/global/WXQC/data/met/config/grib2/grib2_geps.txt
> >
> > DEBUG 1: Creating STAT-Analysis output file
> >
> >
"/h/data/global/WXQC/data/met/ptstat/stats/monthly/stat_anal_gfs_FULL_
> > TT_Z2_
> > 20200601_20200630.txt"
> >
> > DEBUG 1: Default Config File:
> > /h/MET/share/met/config/STATAnalysisConfig_default
> >
> > DEBUG 1: User Config File:
> >
> >
/h/data/global/WXQC/data/met/config/met_config/STATAnalysisConfig_boot
> > strap_
> > aggstat
> >
> > DEBUG 4: Default Job from the config file: "(nul)"
> >
> > DEBUG 4: Amending default job with command line options: "-job
> > aggregate_stat -line_type SL1L2 -out_line_type CNT -fcst_init_beg
> > 20200601_000000 -fcst_init_end 20200630_180000 -by fcst_lead
-fcst_lev
> > Z2 -by model -vx_mask FULL -by vx_mask -by fcst_init_hour"
> >
> > DEBUG 4: New default jobstring: "-job aggregate_stat
-fcst_init_beg
> > 20200601_000000 -fcst_init_end 20200630_180000 -fcst_lev Z2
-vx_mask
> > FULL -line_type SL1L2 -by FCST_LEAD -by MODEL -by VX_MASK -by
> > FCST_INIT_HOUR -out_line_type CNT -out_alpha 0.05000
-rank_corr_flag 0 "
> >
> > DEBUG 2: Processing 202 STAT files.
> >
> > DEBUG 3: Processing STAT file
> >
> >
"/h/data/global/WXQC/data/met/ptstat/stats/galwem/TT/temp/point_stat_g
> > alwem_ all_SFC_240000L_20200610_000000V.stat" .... 1 of 202
> >
> >
> >
> >
> >
> > What am I missing?
> >
> >
> >
> > Thanks
> >
> > Bob
> >
> >
> >
>
> --
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> Tara Jensen
> Project Manager II
> NCAR RAL and DTC
> PO Box 3000, Boulder, Colorado 80307 USA
> +1 303-497-8479 jensen at ucar.edu
>
>
>
>
--
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Tara Jensen
Project Manager II
NCAR RAL and DTC
PO Box 3000, Boulder, Colorado 80307 USA
+1 303-497-8479 jensen at ucar.edu
------------------------------------------------
Subject: Stat Anal Confidence Intervals
From: John Halley Gotway
Time: Mon Jul 13 21:29:53 2020
Bob (and Tara),
It looks like you're running STAT-Analysis to aggregate SL1L2 lines
together into a CNT line type and are wondering why you are getting no
bootstrap confidence intervals. STAT-Analysis is only designed to
compute
bootstrap CI's when processing individual matched pairs from the MPR
line
type, not SL1L2 partial sums.
And that's why you're getting NA's in the *_BCL and *_BCU output
columns.
Logically, STAT-Analysis could be enhanced to bootstrap the
aggregation of
SL1L2 partial sums (and several other line types as well). But it just
has
never been a priority since we developed and used METviewer to
bootstrap
those aggregations instead.
In fact, here's the issue from 2011 that describes this feature
request!
https://github.com/NCAR/MET/issues/59
Hope that helps clarify.
John
On Mon, Jul 13, 2020 at 2:08 PM Tara Jensen via RT <met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95896 >
>
> Bob,
>
> Thanks. I got the email and have downloaded. I'll take a look later
today.
>
> Cheers, Tara
>
> On Mon, Jul 13, 2020 at 2:02 PM robert.craig.2 at us.af.mil via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95896 >
> >
> > I sent it to jensen at ucar.edu.
> >
> > Bob
> >
> > -----Original Message-----
> > From: Tara Jensen via RT <met_help at ucar.edu>
> > Sent: Monday, July 13, 2020 2:55 PM
> > To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
<robert.craig.2 at us.af.mil>
> > Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #95896] Stat Anal
> > Confidence Intervals
> >
> > Hi Bob,
> >
> > John is not in the office right now. Can you send me the data via
> AMRSAFE
> > so I can run the case and see if I can help you?
> >
> > Thanks, Tara
> >
> > On Mon, Jul 13, 2020 at 1:02 PM robert.craig.2 at us.af.mil via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > Mon Jul 13 13:02:02 2020: Request 95896 was acted upon.
> > > Transaction: Ticket created by robert.craig.2 at us.af.mil
> > > Queue: met_help
> > > Subject: Stat Anal Confidence Intervals
> > > Owner: Nobody
> > > Requestors: robert.craig.2 at us.af.mil
> > > Status: new
> > > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95896
> > > >
> > >
> > >
> > > John, I asked this question before but didn't get back to you on
your
> > > answer
> > > - you suggested something I had already done. In stat_analysis,
I am
> > > running aggregate_stat over a month's worth of data. In the
> > > configuration file, I do have the n_rep set to 1000 but I am
getting
> NAs
> > for RMSE_BCL and
> > > RMSE_BCU. My config file is attached. The command line is
> > > /h/MET/bin/stat_analysis -lookin
> > > /h/data/global/WXQC/data/met/ptstat/stats/gfs/TT -out
> > >
> > >
/h/data/global/WXQC/data/met/ptstat/stats/monthly/stat_anal_gfs_FULL_T
> > > T_Z2_2 0200601_20200630.txt -tmp_dir
/h/data/global/WXQC/data/met/tmp
> > > -job aggregate_stat -line_type SL1L2 -out_line_type CNT
-fcst_init_beg
> > > 20200601_000000 -fcst_init_end 20200630_180000 -by fcst_lead
-fcst_lev
> > > Z2 -by model -vx_mask FULL -by vx_mask -by fcst_init_hour
-config
> > >
> > >
/h/data/global/WXQC/data/met/config/met_config/STATAnalysisConfig_boot
> > > strap_ aggstat -v 4.0. I sent the data to you on AMRSAFE is you
need
> > > it.
> > >
> > >
> > >
> > > The listing when I run the job is below:
> > >
> > > [1241 qcteam hpc10-lsn-02p
> > > /h/data/global/WXQC/data/met/ptstat/stats/galwem/TT]
> > > ig/STATAnalysisConfig_bootstrap_aggstat -v 4.0 <
> > >
> > > DEBUG 1: Reading user-defined grib1 MET_GRIB_TABLES file:
> > > /h/data/global/WXQC/data/met/config/grib2/grib1_af_131_57.txt
> > >
> > > DEBUG 1: Reading user-defined grib2 MET_GRIB_TABLES file:
> > > /h/data/global/WXQC/data/met/config/grib2/grib2_geps.txt
> > >
> > > DEBUG 1: Creating STAT-Analysis output file
> > >
> > >
"/h/data/global/WXQC/data/met/ptstat/stats/monthly/stat_anal_gfs_FULL_
> > > TT_Z2_
> > > 20200601_20200630.txt"
> > >
> > > DEBUG 1: Default Config File:
> > > /h/MET/share/met/config/STATAnalysisConfig_default
> > >
> > > DEBUG 1: User Config File:
> > >
> > >
/h/data/global/WXQC/data/met/config/met_config/STATAnalysisConfig_boot
> > > strap_
> > > aggstat
> > >
> > > DEBUG 4: Default Job from the config file: "(nul)"
> > >
> > > DEBUG 4: Amending default job with command line options: "-job
> > > aggregate_stat -line_type SL1L2 -out_line_type CNT
-fcst_init_beg
> > > 20200601_000000 -fcst_init_end 20200630_180000 -by fcst_lead
-fcst_lev
> > > Z2 -by model -vx_mask FULL -by vx_mask -by fcst_init_hour"
> > >
> > > DEBUG 4: New default jobstring: "-job aggregate_stat
-fcst_init_beg
> > > 20200601_000000 -fcst_init_end 20200630_180000 -fcst_lev Z2
-vx_mask
> > > FULL -line_type SL1L2 -by FCST_LEAD -by MODEL -by VX_MASK -by
> > > FCST_INIT_HOUR -out_line_type CNT -out_alpha 0.05000
-rank_corr_flag 0
> "
> > >
> > > DEBUG 2: Processing 202 STAT files.
> > >
> > > DEBUG 3: Processing STAT file
> > >
> > >
"/h/data/global/WXQC/data/met/ptstat/stats/galwem/TT/temp/point_stat_g
> > > alwem_ all_SFC_240000L_20200610_000000V.stat" .... 1 of 202
> > >
> > >
> > >
> > >
> > >
> > > What am I missing?
> > >
> > >
> > >
> > > Thanks
> > >
> > > Bob
> > >
> > >
> > >
> >
> > --
> > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > Tara Jensen
> > Project Manager II
> > NCAR RAL and DTC
> > PO Box 3000, Boulder, Colorado 80307 USA
> > +1 303-497-8479 jensen at ucar.edu
> >
> >
> >
> >
>
> --
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> Tara Jensen
> Project Manager II
> NCAR RAL and DTC
> PO Box 3000, Boulder, Colorado 80307 USA
> +1 303-497-8479 jensen at ucar.edu
>
>
------------------------------------------------
Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95896] Stat Anal Confidence Intervals
From: robert.craig.2 at us.af.mil
Time: Tue Jul 14 07:25:22 2020
I guess if MET Viewer is going to be part of the standard MET
distribution so every MET user can get it, that makes sense. We are
still several months away here. If not, then this capability is
needed in MET. When running CNT aggregate from MPR lines, that is a
very slow process for a global model - hence the reluctance to do it
routinely.
Thanks
Bob
-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Monday, July 13, 2020 10:30 PM
To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
<robert.craig.2 at us.af.mil>
Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95896] Stat Anal
Confidence Intervals
Bob (and Tara),
It looks like you're running STAT-Analysis to aggregate SL1L2 lines
together into a CNT line type and are wondering why you are getting no
bootstrap confidence intervals. STAT-Analysis is only designed to
compute bootstrap CI's when processing individual matched pairs from
the MPR line type, not SL1L2 partial sums.
And that's why you're getting NA's in the *_BCL and *_BCU output
columns.
Logically, STAT-Analysis could be enhanced to bootstrap the
aggregation of
SL1L2 partial sums (and several other line types as well). But it just
has never been a priority since we developed and used METviewer to
bootstrap those aggregations instead.
In fact, here's the issue from 2011 that describes this feature
request!
https://github.com/NCAR/MET/issues/59
Hope that helps clarify.
John
On Mon, Jul 13, 2020 at 2:08 PM Tara Jensen via RT <met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95896 >
>
> Bob,
>
> Thanks. I got the email and have downloaded. I'll take a look later
today.
>
> Cheers, Tara
>
> On Mon, Jul 13, 2020 at 2:02 PM robert.craig.2 at us.af.mil via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95896 >
> >
> > I sent it to jensen at ucar.edu.
> >
> > Bob
> >
> > -----Original Message-----
> > From: Tara Jensen via RT <met_help at ucar.edu>
> > Sent: Monday, July 13, 2020 2:55 PM
> > To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
> > <robert.craig.2 at us.af.mil>
> > Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #95896] Stat Anal
> > Confidence Intervals
> >
> > Hi Bob,
> >
> > John is not in the office right now. Can you send me the data via
> AMRSAFE
> > so I can run the case and see if I can help you?
> >
> > Thanks, Tara
> >
> > On Mon, Jul 13, 2020 at 1:02 PM robert.craig.2 at us.af.mil via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > Mon Jul 13 13:02:02 2020: Request 95896 was acted upon.
> > > Transaction: Ticket created by robert.craig.2 at us.af.mil
> > > Queue: met_help
> > > Subject: Stat Anal Confidence Intervals
> > > Owner: Nobody
> > > Requestors: robert.craig.2 at us.af.mil
> > > Status: new
> > > Ticket <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95896
> > > >
> > >
> > >
> > > John, I asked this question before but didn't get back to you on
> > > your answer
> > > - you suggested something I had already done. In stat_analysis,
I
> > > am running aggregate_stat over a month's worth of data. In the
> > > configuration file, I do have the n_rep set to 1000 but I am
> > > getting
> NAs
> > for RMSE_BCL and
> > > RMSE_BCU. My config file is attached. The command line is
> > > /h/MET/bin/stat_analysis -lookin
> > > /h/data/global/WXQC/data/met/ptstat/stats/gfs/TT -out
> > >
> > >
/h/data/global/WXQC/data/met/ptstat/stats/monthly/stat_anal_gfs_FU
> > > LL_T
> > > T_Z2_2 0200601_20200630.txt -tmp_dir
> > > /h/data/global/WXQC/data/met/tmp -job aggregate_stat -line_type
> > > SL1L2 -out_line_type CNT -fcst_init_beg
> > > 20200601_000000 -fcst_init_end 20200630_180000 -by fcst_lead
> > > -fcst_lev
> > > Z2 -by model -vx_mask FULL -by vx_mask -by fcst_init_hour
-config
> > >
> > >
/h/data/global/WXQC/data/met/config/met_config/STATAnalysisConfig_
> > > boot strap_ aggstat -v 4.0. I sent the data to you on AMRSAFE
is
> > > you need it.
> > >
> > >
> > >
> > > The listing when I run the job is below:
> > >
> > > [1241 qcteam hpc10-lsn-02p
> > > /h/data/global/WXQC/data/met/ptstat/stats/galwem/TT]
> > > ig/STATAnalysisConfig_bootstrap_aggstat -v 4.0 <
> > >
> > > DEBUG 1: Reading user-defined grib1 MET_GRIB_TABLES file:
> > > /h/data/global/WXQC/data/met/config/grib2/grib1_af_131_57.txt
> > >
> > > DEBUG 1: Reading user-defined grib2 MET_GRIB_TABLES file:
> > > /h/data/global/WXQC/data/met/config/grib2/grib2_geps.txt
> > >
> > > DEBUG 1: Creating STAT-Analysis output file
> > >
> > >
"/h/data/global/WXQC/data/met/ptstat/stats/monthly/stat_anal_gfs_F
> > > ULL_
> > > TT_Z2_
> > > 20200601_20200630.txt"
> > >
> > > DEBUG 1: Default Config File:
> > > /h/MET/share/met/config/STATAnalysisConfig_default
> > >
> > > DEBUG 1: User Config File:
> > >
> > >
/h/data/global/WXQC/data/met/config/met_config/STATAnalysisConfig_
> > > boot
> > > strap_
> > > aggstat
> > >
> > > DEBUG 4: Default Job from the config file: "(nul)"
> > >
> > > DEBUG 4: Amending default job with command line options: "-job
> > > aggregate_stat -line_type SL1L2 -out_line_type CNT
-fcst_init_beg
> > > 20200601_000000 -fcst_init_end 20200630_180000 -by fcst_lead
> > > -fcst_lev
> > > Z2 -by model -vx_mask FULL -by vx_mask -by fcst_init_hour"
> > >
> > > DEBUG 4: New default jobstring: "-job aggregate_stat
> > > -fcst_init_beg
> > > 20200601_000000 -fcst_init_end 20200630_180000 -fcst_lev Z2
> > > -vx_mask FULL -line_type SL1L2 -by FCST_LEAD -by MODEL -by
VX_MASK
> > > -by FCST_INIT_HOUR -out_line_type CNT -out_alpha 0.05000
> > > -rank_corr_flag 0
> "
> > >
> > > DEBUG 2: Processing 202 STAT files.
> > >
> > > DEBUG 3: Processing STAT file
> > >
> > >
"/h/data/global/WXQC/data/met/ptstat/stats/galwem/TT/temp/point_st
> > > at_g alwem_ all_SFC_240000L_20200610_000000V.stat" .... 1 of 202
> > >
> > >
> > >
> > >
> > >
> > > What am I missing?
> > >
> > >
> > >
> > > Thanks
> > >
> > > Bob
> > >
> > >
> > >
> >
> > --
> > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > Tara Jensen
> > Project Manager II
> > NCAR RAL and DTC
> > PO Box 3000, Boulder, Colorado 80307 USA
> > +1 303-497-8479 jensen at ucar.edu
> >
> >
> >
> >
>
> --
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> Tara Jensen
> Project Manager II
> NCAR RAL and DTC
> PO Box 3000, Boulder, Colorado 80307 USA
> +1 303-497-8479 jensen at ucar.edu
>
>
------------------------------------------------
Subject: Stat Anal Confidence Intervals
From: John Halley Gotway
Time: Tue Jul 14 16:03:58 2020
Tara,
Any thoughts or opinions on this issue? Do we have a funding mechanism
by
which to prioritize AF development requests?
This is certainly a doable development task, but I certainly wouldn't
call
it trivial. I'd very roughly estimate this as 1 week of work.
Thanks,
John
On Tue, Jul 14, 2020 at 7:25 AM robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95896 >
>
> I guess if MET Viewer is going to be part of the standard MET
distribution
> so every MET user can get it, that makes sense. We are still
several
> months away here. If not, then this capability is needed in MET.
When
> running CNT aggregate from MPR lines, that is a very slow process
for a
> global model - hence the reluctance to do it routinely.
>
> Thanks
> Bob
>
> -----Original Message-----
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Monday, July 13, 2020 10:30 PM
> To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
<robert.craig.2 at us.af.mil>
> Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95896] Stat Anal
> Confidence Intervals
>
> Bob (and Tara),
>
> It looks like you're running STAT-Analysis to aggregate SL1L2 lines
> together into a CNT line type and are wondering why you are getting
no
> bootstrap confidence intervals. STAT-Analysis is only designed to
compute
> bootstrap CI's when processing individual matched pairs from the MPR
line
> type, not SL1L2 partial sums.
>
> And that's why you're getting NA's in the *_BCL and *_BCU output
columns.
>
> Logically, STAT-Analysis could be enhanced to bootstrap the
aggregation of
> SL1L2 partial sums (and several other line types as well). But it
just has
> never been a priority since we developed and used METviewer to
bootstrap
> those aggregations instead.
>
> In fact, here's the issue from 2011 that describes this feature
request!
> https://github.com/NCAR/MET/issues/59
>
> Hope that helps clarify.
> John
>
> On Mon, Jul 13, 2020 at 2:08 PM Tara Jensen via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95896 >
> >
> > Bob,
> >
> > Thanks. I got the email and have downloaded. I'll take a look
later
> today.
> >
> > Cheers, Tara
> >
> > On Mon, Jul 13, 2020 at 2:02 PM robert.craig.2 at us.af.mil via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95896 >
> > >
> > > I sent it to jensen at ucar.edu.
> > >
> > > Bob
> > >
> > > -----Original Message-----
> > > From: Tara Jensen via RT <met_help at ucar.edu>
> > > Sent: Monday, July 13, 2020 2:55 PM
> > > To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
> > > <robert.craig.2 at us.af.mil>
> > > Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #95896] Stat Anal
> > > Confidence Intervals
> > >
> > > Hi Bob,
> > >
> > > John is not in the office right now. Can you send me the data
via
> > AMRSAFE
> > > so I can run the case and see if I can help you?
> > >
> > > Thanks, Tara
> > >
> > > On Mon, Jul 13, 2020 at 1:02 PM robert.craig.2 at us.af.mil via RT
<
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > Mon Jul 13 13:02:02 2020: Request 95896 was acted upon.
> > > > Transaction: Ticket created by robert.craig.2 at us.af.mil
> > > > Queue: met_help
> > > > Subject: Stat Anal Confidence Intervals
> > > > Owner: Nobody
> > > > Requestors: robert.craig.2 at us.af.mil
> > > > Status: new
> > > > Ticket <URL:
> > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95896
> > > > >
> > > >
> > > >
> > > > John, I asked this question before but didn't get back to you
on
> > > > your answer
> > > > - you suggested something I had already done. In
stat_analysis, I
> > > > am running aggregate_stat over a month's worth of data. In
the
> > > > configuration file, I do have the n_rep set to 1000 but I am
> > > > getting
> > NAs
> > > for RMSE_BCL and
> > > > RMSE_BCU. My config file is attached. The command line is
> > > > /h/MET/bin/stat_analysis -lookin
> > > > /h/data/global/WXQC/data/met/ptstat/stats/gfs/TT -out
> > > >
> > > >
/h/data/global/WXQC/data/met/ptstat/stats/monthly/stat_anal_gfs_FU
> > > > LL_T
> > > > T_Z2_2 0200601_20200630.txt -tmp_dir
> > > > /h/data/global/WXQC/data/met/tmp -job aggregate_stat
-line_type
> > > > SL1L2 -out_line_type CNT -fcst_init_beg
> > > > 20200601_000000 -fcst_init_end 20200630_180000 -by fcst_lead
> > > > -fcst_lev
> > > > Z2 -by model -vx_mask FULL -by vx_mask -by fcst_init_hour
-config
> > > >
> > > >
/h/data/global/WXQC/data/met/config/met_config/STATAnalysisConfig_
> > > > boot strap_ aggstat -v 4.0. I sent the data to you on AMRSAFE
is
> > > > you need it.
> > > >
> > > >
> > > >
> > > > The listing when I run the job is below:
> > > >
> > > > [1241 qcteam hpc10-lsn-02p
> > > > /h/data/global/WXQC/data/met/ptstat/stats/galwem/TT]
> > > > ig/STATAnalysisConfig_bootstrap_aggstat -v 4.0 <
> > > >
> > > > DEBUG 1: Reading user-defined grib1 MET_GRIB_TABLES file:
> > > > /h/data/global/WXQC/data/met/config/grib2/grib1_af_131_57.txt
> > > >
> > > > DEBUG 1: Reading user-defined grib2 MET_GRIB_TABLES file:
> > > > /h/data/global/WXQC/data/met/config/grib2/grib2_geps.txt
> > > >
> > > > DEBUG 1: Creating STAT-Analysis output file
> > > >
> > > >
"/h/data/global/WXQC/data/met/ptstat/stats/monthly/stat_anal_gfs_F
> > > > ULL_
> > > > TT_Z2_
> > > > 20200601_20200630.txt"
> > > >
> > > > DEBUG 1: Default Config File:
> > > > /h/MET/share/met/config/STATAnalysisConfig_default
> > > >
> > > > DEBUG 1: User Config File:
> > > >
> > > >
/h/data/global/WXQC/data/met/config/met_config/STATAnalysisConfig_
> > > > boot
> > > > strap_
> > > > aggstat
> > > >
> > > > DEBUG 4: Default Job from the config file: "(nul)"
> > > >
> > > > DEBUG 4: Amending default job with command line options: "-job
> > > > aggregate_stat -line_type SL1L2 -out_line_type CNT
-fcst_init_beg
> > > > 20200601_000000 -fcst_init_end 20200630_180000 -by fcst_lead
> > > > -fcst_lev
> > > > Z2 -by model -vx_mask FULL -by vx_mask -by fcst_init_hour"
> > > >
> > > > DEBUG 4: New default jobstring: "-job aggregate_stat
> > > > -fcst_init_beg
> > > > 20200601_000000 -fcst_init_end 20200630_180000 -fcst_lev Z2
> > > > -vx_mask FULL -line_type SL1L2 -by FCST_LEAD -by MODEL -by
VX_MASK
> > > > -by FCST_INIT_HOUR -out_line_type CNT -out_alpha 0.05000
> > > > -rank_corr_flag 0
> > "
> > > >
> > > > DEBUG 2: Processing 202 STAT files.
> > > >
> > > > DEBUG 3: Processing STAT file
> > > >
> > > >
"/h/data/global/WXQC/data/met/ptstat/stats/galwem/TT/temp/point_st
> > > > at_g alwem_ all_SFC_240000L_20200610_000000V.stat" .... 1 of
202
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > What am I missing?
> > > >
> > > >
> > > >
> > > > Thanks
> > > >
> > > > Bob
> > > >
> > > >
> > > >
> > >
> > > --
> > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > Tara Jensen
> > > Project Manager II
> > > NCAR RAL and DTC
> > > PO Box 3000, Boulder, Colorado 80307 USA
> > > +1 303-497-8479 jensen at ucar.edu
> > >
> > >
> > >
> > >
> >
> > --
> > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > Tara Jensen
> > Project Manager II
> > NCAR RAL and DTC
> > PO Box 3000, Boulder, Colorado 80307 USA
> > +1 303-497-8479 jensen at ucar.edu
> >
> >
>
>
>
>
------------------------------------------------
Subject: Stat Anal Confidence Intervals
From: Tara Jensen
Time: Tue Jul 14 16:28:48 2020
Taking the response to this request for discussion off-line. Look for
a
follow-up email outside of met_help.
On Tue, Jul 14, 2020 at 4:04 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95896 >
>
> Tara,
>
> Any thoughts or opinions on this issue? Do we have a funding
mechanism by
> which to prioritize AF development requests?
>
> This is certainly a doable development task, but I certainly
wouldn't call
> it trivial. I'd very roughly estimate this as 1 week of work.
>
> Thanks,
> John
>
> On Tue, Jul 14, 2020 at 7:25 AM robert.craig.2 at us.af.mil via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95896 >
> >
> > I guess if MET Viewer is going to be part of the standard MET
> distribution
> > so every MET user can get it, that makes sense. We are still
several
> > months away here. If not, then this capability is needed in MET.
When
> > running CNT aggregate from MPR lines, that is a very slow process
for a
> > global model - hence the reluctance to do it routinely.
> >
> > Thanks
> > Bob
> >
> > -----Original Message-----
> > From: John Halley Gotway via RT <met_help at ucar.edu>
> > Sent: Monday, July 13, 2020 10:30 PM
> > To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
<robert.craig.2 at us.af.mil>
> > Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #95896] Stat
Anal
> > Confidence Intervals
> >
> > Bob (and Tara),
> >
> > It looks like you're running STAT-Analysis to aggregate SL1L2
lines
> > together into a CNT line type and are wondering why you are
getting no
> > bootstrap confidence intervals. STAT-Analysis is only designed to
> compute
> > bootstrap CI's when processing individual matched pairs from the
MPR line
> > type, not SL1L2 partial sums.
> >
> > And that's why you're getting NA's in the *_BCL and *_BCU output
columns.
> >
> > Logically, STAT-Analysis could be enhanced to bootstrap the
aggregation
> of
> > SL1L2 partial sums (and several other line types as well). But it
just
> has
> > never been a priority since we developed and used METviewer to
bootstrap
> > those aggregations instead.
> >
> > In fact, here's the issue from 2011 that describes this feature
request!
> > https://github.com/NCAR/MET/issues/59
> >
> > Hope that helps clarify.
> > John
> >
> > On Mon, Jul 13, 2020 at 2:08 PM Tara Jensen via RT
<met_help at ucar.edu>
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95896 >
> > >
> > > Bob,
> > >
> > > Thanks. I got the email and have downloaded. I'll take a look
later
> > today.
> > >
> > > Cheers, Tara
> > >
> > > On Mon, Jul 13, 2020 at 2:02 PM robert.craig.2 at us.af.mil via RT
<
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95896
>
> > > >
> > > > I sent it to jensen at ucar.edu.
> > > >
> > > > Bob
> > > >
> > > > -----Original Message-----
> > > > From: Tara Jensen via RT <met_help at ucar.edu>
> > > > Sent: Monday, July 13, 2020 2:55 PM
> > > > To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
> > > > <robert.craig.2 at us.af.mil>
> > > > Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #95896] Stat
Anal
> > > > Confidence Intervals
> > > >
> > > > Hi Bob,
> > > >
> > > > John is not in the office right now. Can you send me the data
via
> > > AMRSAFE
> > > > so I can run the case and see if I can help you?
> > > >
> > > > Thanks, Tara
> > > >
> > > > On Mon, Jul 13, 2020 at 1:02 PM robert.craig.2 at us.af.mil via
RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > Mon Jul 13 13:02:02 2020: Request 95896 was acted upon.
> > > > > Transaction: Ticket created by robert.craig.2 at us.af.mil
> > > > > Queue: met_help
> > > > > Subject: Stat Anal Confidence Intervals
> > > > > Owner: Nobody
> > > > > Requestors: robert.craig.2 at us.af.mil
> > > > > Status: new
> > > > > Ticket <URL:
> > > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95896
> > > > > >
> > > > >
> > > > >
> > > > > John, I asked this question before but didn't get back to
you on
> > > > > your answer
> > > > > - you suggested something I had already done. In
stat_analysis, I
> > > > > am running aggregate_stat over a month's worth of data. In
the
> > > > > configuration file, I do have the n_rep set to 1000 but I am
> > > > > getting
> > > NAs
> > > > for RMSE_BCL and
> > > > > RMSE_BCU. My config file is attached. The command line is
> > > > > /h/MET/bin/stat_analysis -lookin
> > > > > /h/data/global/WXQC/data/met/ptstat/stats/gfs/TT -out
> > > > >
> > > > >
/h/data/global/WXQC/data/met/ptstat/stats/monthly/stat_anal_gfs_FU
> > > > > LL_T
> > > > > T_Z2_2 0200601_20200630.txt -tmp_dir
> > > > > /h/data/global/WXQC/data/met/tmp -job aggregate_stat
-line_type
> > > > > SL1L2 -out_line_type CNT -fcst_init_beg
> > > > > 20200601_000000 -fcst_init_end 20200630_180000 -by fcst_lead
> > > > > -fcst_lev
> > > > > Z2 -by model -vx_mask FULL -by vx_mask -by fcst_init_hour
-config
> > > > >
> > > > >
/h/data/global/WXQC/data/met/config/met_config/STATAnalysisConfig_
> > > > > boot strap_ aggstat -v 4.0. I sent the data to you on
AMRSAFE is
> > > > > you need it.
> > > > >
> > > > >
> > > > >
> > > > > The listing when I run the job is below:
> > > > >
> > > > > [1241 qcteam hpc10-lsn-02p
> > > > > /h/data/global/WXQC/data/met/ptstat/stats/galwem/TT]
> > > > > ig/STATAnalysisConfig_bootstrap_aggstat -v 4.0 <
> > > > >
> > > > > DEBUG 1: Reading user-defined grib1 MET_GRIB_TABLES file:
> > > > >
/h/data/global/WXQC/data/met/config/grib2/grib1_af_131_57.txt
> > > > >
> > > > > DEBUG 1: Reading user-defined grib2 MET_GRIB_TABLES file:
> > > > > /h/data/global/WXQC/data/met/config/grib2/grib2_geps.txt
> > > > >
> > > > > DEBUG 1: Creating STAT-Analysis output file
> > > > >
> > > > >
"/h/data/global/WXQC/data/met/ptstat/stats/monthly/stat_anal_gfs_F
> > > > > ULL_
> > > > > TT_Z2_
> > > > > 20200601_20200630.txt"
> > > > >
> > > > > DEBUG 1: Default Config File:
> > > > > /h/MET/share/met/config/STATAnalysisConfig_default
> > > > >
> > > > > DEBUG 1: User Config File:
> > > > >
> > > > >
/h/data/global/WXQC/data/met/config/met_config/STATAnalysisConfig_
> > > > > boot
> > > > > strap_
> > > > > aggstat
> > > > >
> > > > > DEBUG 4: Default Job from the config file: "(nul)"
> > > > >
> > > > > DEBUG 4: Amending default job with command line options: "-
job
> > > > > aggregate_stat -line_type SL1L2 -out_line_type CNT
-fcst_init_beg
> > > > > 20200601_000000 -fcst_init_end 20200630_180000 -by fcst_lead
> > > > > -fcst_lev
> > > > > Z2 -by model -vx_mask FULL -by vx_mask -by fcst_init_hour"
> > > > >
> > > > > DEBUG 4: New default jobstring: "-job aggregate_stat
> > > > > -fcst_init_beg
> > > > > 20200601_000000 -fcst_init_end 20200630_180000 -fcst_lev Z2
> > > > > -vx_mask FULL -line_type SL1L2 -by FCST_LEAD -by MODEL -by
VX_MASK
> > > > > -by FCST_INIT_HOUR -out_line_type CNT -out_alpha 0.05000
> > > > > -rank_corr_flag 0
> > > "
> > > > >
> > > > > DEBUG 2: Processing 202 STAT files.
> > > > >
> > > > > DEBUG 3: Processing STAT file
> > > > >
> > > > >
"/h/data/global/WXQC/data/met/ptstat/stats/galwem/TT/temp/point_st
> > > > > at_g alwem_ all_SFC_240000L_20200610_000000V.stat" .... 1 of
202
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > What am I missing?
> > > > >
> > > > >
> > > > >
> > > > > Thanks
> > > > >
> > > > > Bob
> > > > >
> > > > >
> > > > >
> > > >
> > > > --
> > > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > > Tara Jensen
> > > > Project Manager II
> > > > NCAR RAL and DTC
> > > > PO Box 3000, Boulder, Colorado 80307 USA
> > > > +1 303-497-8479 jensen at ucar.edu
> > > >
> > > >
> > > >
> > > >
> > >
> > > --
> > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> > > Tara Jensen
> > > Project Manager II
> > > NCAR RAL and DTC
> > > PO Box 3000, Boulder, Colorado 80307 USA
> > > +1 303-497-8479 jensen at ucar.edu
> > >
> > >
> >
> >
> >
> >
>
>
--
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Tara Jensen
Project Manager II
NCAR RAL and DTC
PO Box 3000, Boulder, Colorado 80307 USA
+1 303-497-8479 jensen at ucar.edu
------------------------------------------------
More information about the Met_help
mailing list