[Met_help] [rt.rap.ucar.edu #95588] History for Summary FIles Confidence Intervals are NA
John Halley Gotway via RT
met_help at ucar.edu
Fri Jun 19 12:43:44 MDT 2020
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
John, when running Stat Anal Summary option, a lot of times I get confidence intervals of NA. This happens if I set PCTILE or BCA and n_rep to 1000 or 0. What causes this to happen? A portion of my config file is:
////////////////////////////////////////////////////////////////////////////////
//
// Array of STAT-Analysis jobs to be performed on the filtered data
//
jobs = [
"-job summary"
];
////////////////////////////////////////////////////////////////////////////////
//
// Confidence interval settings
//
out_alpha = 0.05;
boot = {
interval = PCTILE;
rep_prop = 1.0;
n_rep = 0;
rng = "mt19937";
seed = "10";
}
////////////////////////////////////////////////////////////////////////////////
//
// WMO mean computation logic
//
wmo_sqrt_stats = [ "CNT:FSTDEV", "CNT:OSTDEV", "CNT:ESTDEV",
"CNT:RMSE", "CNT:RMSFA", "CNT:RMSOA",
"VCNT:FS_RMS", "VCNT:OS_RMS", "VCNT:RMSVE",
"VCNT:FSTDEV", "VCNT:OSTDEV" ];
wmo_fisher_stats = [ "CNT:PR_CORR", "CNT:SP_CORR",
"CNT:KT_CORR", "CNT:ANOM_CORR" ];
////////////////////////////////////////////////////////////////////////////////
rank_corr_flag = FALSE;
vif_flag = FALSE;
tmp_dir = "/tmp";
version = "V8.1";
The command line I am using is:
/h/MET/bin/stat_analysis -lookin /h/data/global/WXQC/data/met/ptstat/stats/galwem/TT/temp -out /h/data/global/WXQC/data/met/ptstat/stats/timeseries/stat_anal_galwem_full_TT_Z2_20200401_20200430.txt -tmp_dir /h/data/global/WXQC/data/met/tmp -job summary -column CNT:RMSE,CNT:ME -fcst_init_beg 20200401_000000 -fcst_init_end 20204030_180000 -fcst_lead 24 -fcst_lev Z2 -vx_mask FULL -by fcst_lead -by fcst_init_hour -by model -by fcst_valid_beg -config /h/data/global/WXQC/data/met/config/met_config/STATAnalysisConfig_bootstrap_summary -v 4.0
Thanks
Bob
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Summary FIles Confidence Intervals are NA
From: Minna Win
Time: Thu Jun 11 10:23:35 2020
Hi John,
It looks like you have a few questions about confidence intervals in
MET.
I'm assigning this ticket to John Halley Gotway.
Regards,
Minna
---------------
Minna Win
National Center for Atmospheric Research
Developmental Testbed Center
Phone: 303-497-8423
Fax: 303-497-8401
On Thu, Jun 11, 2020 at 10:16 AM robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> Thu Jun 11 10:15:32 2020: Request 95588 was acted upon.
> Transaction: Ticket created by robert.craig.2 at us.af.mil
> Queue: met_help
> Subject: Summary FIles Confidence Intervals are NA
> Owner: Nobody
> Requestors: robert.craig.2 at us.af.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95588 >
>
>
> John, when running Stat Anal Summary option, a lot of times I get
> confidence intervals of NA. This happens if I set PCTILE or BCA and
n_rep
> to 1000 or 0. What causes this to happen? A portion of my config
file is:
>
>
>
////////////////////////////////////////////////////////////////////////////////
>
> //
> // Array of STAT-Analysis jobs to be performed on the filtered data
> //
> jobs = [
> "-job summary"
> ];
>
>
>
////////////////////////////////////////////////////////////////////////////////
>
> //
> // Confidence interval settings
> //
> out_alpha = 0.05;
>
> boot = {
> interval = PCTILE;
> rep_prop = 1.0;
> n_rep = 0;
> rng = "mt19937";
> seed = "10";
> }
>
>
>
////////////////////////////////////////////////////////////////////////////////
>
> //
> // WMO mean computation logic
> //
> wmo_sqrt_stats = [ "CNT:FSTDEV", "CNT:OSTDEV", "CNT:ESTDEV",
> "CNT:RMSE", "CNT:RMSFA", "CNT:RMSOA",
> "VCNT:FS_RMS", "VCNT:OS_RMS", "VCNT:RMSVE",
> "VCNT:FSTDEV", "VCNT:OSTDEV" ];
>
> wmo_fisher_stats = [ "CNT:PR_CORR", "CNT:SP_CORR",
> "CNT:KT_CORR", "CNT:ANOM_CORR" ];
>
>
>
////////////////////////////////////////////////////////////////////////////////
>
> rank_corr_flag = FALSE;
> vif_flag = FALSE;
> tmp_dir = "/tmp";
> version = "V8.1";
>
> The command line I am using is:
> /h/MET/bin/stat_analysis -lookin
> /h/data/global/WXQC/data/met/ptstat/stats/galwem/TT/temp -out
>
/h/data/global/WXQC/data/met/ptstat/stats/timeseries/stat_anal_galwem_full_TT_Z2_20200401_20200430.txt
> -tmp_dir /h/data/global/WXQC/data/met/tmp -job summary -column
> CNT:RMSE,CNT:ME -fcst_init_beg 20200401_000000 -fcst_init_end
> 20204030_180000 -fcst_lead 24 -fcst_lev Z2 -vx_mask FULL -by
fcst_lead -by
> fcst_init_hour -by model -by fcst_valid_beg -config
>
/h/data/global/WXQC/data/met/config/met_config/STATAnalysisConfig_bootstrap_summary
> -v 4.0
>
> Thanks
> Bob
>
>
------------------------------------------------
Subject: RE: [Non-DoD Source] Re: [rt.rap.ucar.edu #95588] Summary FIles Confidence Intervals are NA
From: robert.craig.2 at us.af.mil
Time: Thu Jun 11 10:38:07 2020
I think I answered my own question. I was running the job to output
data daily - so not enough data to calculate a confidence Interval.
Bob
-----Original Message-----
From: Minna Win via RT <met_help at ucar.edu>
Sent: Thursday, June 11, 2020 11:24 AM
To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
<robert.craig.2 at us.af.mil>
Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #95588] Summary FIles
Confidence Intervals are NA
Hi John,
It looks like you have a few questions about confidence intervals in
MET.
I'm assigning this ticket to John Halley Gotway.
Regards,
Minna
---------------
Minna Win
National Center for Atmospheric Research Developmental Testbed Center
Phone: 303-497-8423
Fax: 303-497-8401
On Thu, Jun 11, 2020 at 10:16 AM robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> Thu Jun 11 10:15:32 2020: Request 95588 was acted upon.
> Transaction: Ticket created by robert.craig.2 at us.af.mil
> Queue: met_help
> Subject: Summary FIles Confidence Intervals are NA
> Owner: Nobody
> Requestors: robert.craig.2 at us.af.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95588
> >
>
>
> John, when running Stat Anal Summary option, a lot of times I get
> confidence intervals of NA. This happens if I set PCTILE or BCA and
> n_rep to 1000 or 0. What causes this to happen? A portion of my
config file is:
>
>
>
//////////////////////////////////////////////////////////////////////
> //////////
>
> //
> // Array of STAT-Analysis jobs to be performed on the filtered data
//
> jobs = [
> "-job summary"
> ];
>
>
>
//////////////////////////////////////////////////////////////////////
> //////////
>
> //
> // Confidence interval settings
> //
> out_alpha = 0.05;
>
> boot = {
> interval = PCTILE;
> rep_prop = 1.0;
> n_rep = 0;
> rng = "mt19937";
> seed = "10";
> }
>
>
>
//////////////////////////////////////////////////////////////////////
> //////////
>
> //
> // WMO mean computation logic
> //
> wmo_sqrt_stats = [ "CNT:FSTDEV", "CNT:OSTDEV", "CNT:ESTDEV",
> "CNT:RMSE", "CNT:RMSFA", "CNT:RMSOA",
> "VCNT:FS_RMS", "VCNT:OS_RMS", "VCNT:RMSVE",
> "VCNT:FSTDEV", "VCNT:OSTDEV" ];
>
> wmo_fisher_stats = [ "CNT:PR_CORR", "CNT:SP_CORR",
> "CNT:KT_CORR", "CNT:ANOM_CORR" ];
>
>
>
//////////////////////////////////////////////////////////////////////
> //////////
>
> rank_corr_flag = FALSE;
> vif_flag = FALSE;
> tmp_dir = "/tmp";
> version = "V8.1";
>
> The command line I am using is:
> /h/MET/bin/stat_analysis -lookin
> /h/data/global/WXQC/data/met/ptstat/stats/galwem/TT/temp -out
>
/h/data/global/WXQC/data/met/ptstat/stats/timeseries/stat_anal_galwem_
> full_TT_Z2_20200401_20200430.txt -tmp_dir
> /h/data/global/WXQC/data/met/tmp -job summary -column
CNT:RMSE,CNT:ME
> -fcst_init_beg 20200401_000000 -fcst_init_end
> 20204030_180000 -fcst_lead 24 -fcst_lev Z2 -vx_mask FULL -by
fcst_lead
> -by fcst_init_hour -by model -by fcst_valid_beg -config
>
/h/data/global/WXQC/data/met/config/met_config/STATAnalysisConfig_boot
> strap_summary
> -v 4.0
>
> Thanks
> Bob
>
>
------------------------------------------------
Subject: Summary FIles Confidence Intervals are NA
From: Minna Win
Time: Thu Jun 11 10:47:03 2020
Hi Bob,
Thanks for the update. I'll close this ticket.
Regards,
Minna
---------------
Minna Win
National Center for Atmospheric Research
Developmental Testbed Center
Phone: 303-497-8423
Fax: 303-497-8401
On Thu, Jun 11, 2020 at 10:38 AM robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95588 >
>
> I think I answered my own question. I was running the job to output
data
> daily - so not enough data to calculate a confidence Interval.
>
> Bob
>
> -----Original Message-----
> From: Minna Win via RT <met_help at ucar.edu>
> Sent: Thursday, June 11, 2020 11:24 AM
> To: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
<robert.craig.2 at us.af.mil>
> Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #95588] Summary FIles
> Confidence Intervals are NA
>
> Hi John,
>
> It looks like you have a few questions about confidence intervals in
MET.
> I'm assigning this ticket to John Halley Gotway.
>
> Regards,
> Minna
> ---------------
> Minna Win
> National Center for Atmospheric Research Developmental Testbed
Center
> Phone: 303-497-8423
> Fax: 303-497-8401
>
>
>
> On Thu, Jun 11, 2020 at 10:16 AM robert.craig.2 at us.af.mil via RT <
> met_help at ucar.edu> wrote:
>
> >
> > Thu Jun 11 10:15:32 2020: Request 95588 was acted upon.
> > Transaction: Ticket created by robert.craig.2 at us.af.mil
> > Queue: met_help
> > Subject: Summary FIles Confidence Intervals are NA
> > Owner: Nobody
> > Requestors: robert.craig.2 at us.af.mil
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95588
> > >
> >
> >
> > John, when running Stat Anal Summary option, a lot of times I get
> > confidence intervals of NA. This happens if I set PCTILE or BCA
and
> > n_rep to 1000 or 0. What causes this to happen? A portion of my
config
> file is:
> >
> >
> >
//////////////////////////////////////////////////////////////////////
> > //////////
> >
> > //
> > // Array of STAT-Analysis jobs to be performed on the filtered
data //
> > jobs = [
> > "-job summary"
> > ];
> >
> >
> >
//////////////////////////////////////////////////////////////////////
> > //////////
> >
> > //
> > // Confidence interval settings
> > //
> > out_alpha = 0.05;
> >
> > boot = {
> > interval = PCTILE;
> > rep_prop = 1.0;
> > n_rep = 0;
> > rng = "mt19937";
> > seed = "10";
> > }
> >
> >
> >
//////////////////////////////////////////////////////////////////////
> > //////////
> >
> > //
> > // WMO mean computation logic
> > //
> > wmo_sqrt_stats = [ "CNT:FSTDEV", "CNT:OSTDEV", "CNT:ESTDEV",
> > "CNT:RMSE", "CNT:RMSFA", "CNT:RMSOA",
> > "VCNT:FS_RMS", "VCNT:OS_RMS", "VCNT:RMSVE",
> > "VCNT:FSTDEV", "VCNT:OSTDEV" ];
> >
> > wmo_fisher_stats = [ "CNT:PR_CORR", "CNT:SP_CORR",
> > "CNT:KT_CORR", "CNT:ANOM_CORR" ];
> >
> >
> >
//////////////////////////////////////////////////////////////////////
> > //////////
> >
> > rank_corr_flag = FALSE;
> > vif_flag = FALSE;
> > tmp_dir = "/tmp";
> > version = "V8.1";
> >
> > The command line I am using is:
> > /h/MET/bin/stat_analysis -lookin
> > /h/data/global/WXQC/data/met/ptstat/stats/galwem/TT/temp -out
> >
/h/data/global/WXQC/data/met/ptstat/stats/timeseries/stat_anal_galwem_
> > full_TT_Z2_20200401_20200430.txt -tmp_dir
> > /h/data/global/WXQC/data/met/tmp -job summary -column
CNT:RMSE,CNT:ME
> > -fcst_init_beg 20200401_000000 -fcst_init_end
> > 20204030_180000 -fcst_lead 24 -fcst_lev Z2 -vx_mask FULL -by
fcst_lead
> > -by fcst_init_hour -by model -by fcst_valid_beg -config
> >
/h/data/global/WXQC/data/met/config/met_config/STATAnalysisConfig_boot
> > strap_summary
> > -v 4.0
> >
> > Thanks
> > Bob
> >
> >
>
>
>
>
------------------------------------------------
Subject: Summary FIles Confidence Intervals are NA
From: John Halley Gotway
Time: Thu Jun 11 10:47:05 2020
John,
I see you have a question about enabling the computation of bootstrap
CI's
for the summary job type in Stat-Analysis. In order to get bootstrap
CI's
you need to specify a number of replicates > 0. In the example below,
I
switched to 1000.
Search for "n_rep" in this README file for a more complete
description:
https://github.com/NCAR/MET/blob/master_v9.0/met/data/config/README
*boot = {*
* interval = PCTILE; rep_prop = 1.0; n_rep = 1000; rng
=
"mt19937"; seed = "10";}*
Thanks,
John
On Thu, Jun 11, 2020 at 10:24 AM Minna Win via RT <met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95588 >
>
> Hi John,
>
> It looks like you have a few questions about confidence intervals in
MET.
> I'm assigning this ticket to John Halley Gotway.
>
> Regards,
> Minna
> ---------------
> Minna Win
> National Center for Atmospheric Research
> Developmental Testbed Center
> Phone: 303-497-8423
> Fax: 303-497-8401
>
>
>
> On Thu, Jun 11, 2020 at 10:16 AM robert.craig.2 at us.af.mil via RT <
> met_help at ucar.edu> wrote:
>
> >
> > Thu Jun 11 10:15:32 2020: Request 95588 was acted upon.
> > Transaction: Ticket created by robert.craig.2 at us.af.mil
> > Queue: met_help
> > Subject: Summary FIles Confidence Intervals are NA
> > Owner: Nobody
> > Requestors: robert.craig.2 at us.af.mil
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95588 >
> >
> >
> > John, when running Stat Anal Summary option, a lot of times I get
> > confidence intervals of NA. This happens if I set PCTILE or BCA
and
> n_rep
> > to 1000 or 0. What causes this to happen? A portion of my config
file
> is:
> >
> >
> >
>
////////////////////////////////////////////////////////////////////////////////
> >
> > //
> > // Array of STAT-Analysis jobs to be performed on the filtered
data
> > //
> > jobs = [
> > "-job summary"
> > ];
> >
> >
> >
>
////////////////////////////////////////////////////////////////////////////////
> >
> > //
> > // Confidence interval settings
> > //
> > out_alpha = 0.05;
> >
> > boot = {
> > interval = PCTILE;
> > rep_prop = 1.0;
> > n_rep = 0;
> > rng = "mt19937";
> > seed = "10";
> > }
> >
> >
> >
>
////////////////////////////////////////////////////////////////////////////////
> >
> > //
> > // WMO mean computation logic
> > //
> > wmo_sqrt_stats = [ "CNT:FSTDEV", "CNT:OSTDEV", "CNT:ESTDEV",
> > "CNT:RMSE", "CNT:RMSFA", "CNT:RMSOA",
> > "VCNT:FS_RMS", "VCNT:OS_RMS", "VCNT:RMSVE",
> > "VCNT:FSTDEV", "VCNT:OSTDEV" ];
> >
> > wmo_fisher_stats = [ "CNT:PR_CORR", "CNT:SP_CORR",
> > "CNT:KT_CORR", "CNT:ANOM_CORR" ];
> >
> >
> >
>
////////////////////////////////////////////////////////////////////////////////
> >
> > rank_corr_flag = FALSE;
> > vif_flag = FALSE;
> > tmp_dir = "/tmp";
> > version = "V8.1";
> >
> > The command line I am using is:
> > /h/MET/bin/stat_analysis -lookin
> > /h/data/global/WXQC/data/met/ptstat/stats/galwem/TT/temp -out
> >
>
/h/data/global/WXQC/data/met/ptstat/stats/timeseries/stat_anal_galwem_full_TT_Z2_20200401_20200430.txt
> > -tmp_dir /h/data/global/WXQC/data/met/tmp -job summary -column
> > CNT:RMSE,CNT:ME -fcst_init_beg 20200401_000000 -fcst_init_end
> > 20204030_180000 -fcst_lead 24 -fcst_lev Z2 -vx_mask FULL -by
fcst_lead
> -by
> > fcst_init_hour -by model -by fcst_valid_beg -config
> >
>
/h/data/global/WXQC/data/met/config/met_config/STATAnalysisConfig_bootstrap_summary
> > -v 4.0
> >
> > Thanks
> > Bob
> >
> >
>
>
------------------------------------------------
More information about the Met_help
mailing list