[Met_help] [rt.rap.ucar.edu #90342] History for Ensemble ANOM_CORR Mean
Howard Soh via RT
met_help at ucar.edu
Wed May 29 08:47:12 MDT 2019
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
How can we out the Anomaly Correlation mean for an ensemble. ensemble_stat 20
outputs the stats for the 20 member ensemble. Our problem is that
ensemble_stat does not have an output for ANOM_CORR, or if it does we do not
know how to output it. We have found we can output the Anomaly Correlation
for grid_stat however only for each member.
Thanks,
Glen
--
Mr. Glen CARL
NRL Monterey Contractor SAIC/0cog, Bldg 704 Rm 133, Code 7532
831.325.6269
glen.carl.ctr at nrlmry.navy.mil
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Ensemble ANOM_CORR Mean
From: John Halley Gotway
Time: Fri May 24 11:49:40 2019
Glen,
You're correct. The ANOM_CORR statistic is included in the CNT output
line
type which is included in the output Point-Stat and Grid-Stat. I see
that
you're running a 20-member ensemble. It sounds like you'd like to see
the
mean of the ANOM_CORR value for those 20 ensemble members. There are
a few
options here:
(1) One of the outputs from ensemble_stat is a NetCDF file containing
which you can configure to include the ensemble mean field. Just turn
"mean" on in the "ensemble_flag" config file option:
*ensemble_flag = {... mean = TRUE;*
*...*
*}*
Then you can run the ensemble mean field through Grid-Stat or Point-
Stat
and compute the ANOM_CORR statistic. This isn't exactly what you
asked for
but is similar.
(2) A second option is computing the ANOM_CORR statistic for each of
the 20
ensemble members and using stat_analysis to compute the mean of those
2
stats. You can do this using the "summary" job type or the
"aggregate_stat" job type.
Run a stat_analysis "summary" job on the ANOM_CORR column of output.
I
used some sample data to demonstrate:
*met-8.1/bin/stat_analysis -lookin
point_stat_GFS_CLIMO_1.0DEG_120000L_20120409_120000V.stat -job summary
-line_type CNT -column ANOM_CORR -by FCST_VAR*
This reads data from .stat file listed and dumps out summary info for
the
ANOM_CORR column, but does so for each unique FCST_VAR value
encountered:
COL_NAME: LINE_TYPE COLUMN FCST_VAR TOTAL MEAN MEAN_NCL
MEAN_NCU
MEAN_BCL MEAN_BCU STDEV STDEV_BCL STDEV_BCU MIN P10 P25
P50 P75 P90 MAX IQR RANGE WMO_TYPE WMO_MEAN
WMO_WEIGHTED_MEAN
SUMMARY: CNT ANOM_CORR TMP 4 0.5472 0.090704
1.0037
NA NA 0.28688 NA NA 0.40376 0.40376 0.40376
0.40376
0.5472 0.8054 0.97753 0.14344 0.57377 FISHER 0.70681
0.41373
SUMMARY: CNT ANOM_CORR UGRD 1 0.71647 NA NA
NA NA NA NA NA 0.71647 0.71647 0.71647
0.71647
0.71647 0.71647 0.71647 0 0 FISHER 0.71647
0.71647
SUMMARY: CNT ANOM_CORR VGRD 1 0.46991 NA NA
NA NA NA NA NA 0.46991 0.46991 0.46991
0.46991
0.46991 0.46991 0.46991 0 0 FISHER 0.46991
0.46991
SUMMARY: CNT ANOM_CORR WIND 1 0.64622 NA NA
NA NA NA NA NA 0.64622 0.64622 0.64622
0.64622
0.64622 0.64622 0.64622 0 0 FISHER 0.64622
0.64622
Run a stat_analysis "aggregate_stat" job type to aggregate together
SAL1L2
partial sums and recompute the CNT output line type:
*met-8.1/bin/stat_analysis -lookin
point_stat_GFS_CLIMO_1.0DEG_120000L_20120409_120000V.stat -job
aggregate_stat -line_type SAL1L2 -out_line_type CNT -by FCST_VAR
-out_stat
cnt.stat*
That logic would recompute the CNT stats aggregated across all 20 of
your
members.
(3) The third option, and what we'd probably use in the DTC, is to
just use
METviewer to apply the stat_analysis aggregations shown above as we're
plotting the data.
The ensemble-stat tool does not include any logic to compute stats
separately for each member and then report the mean of those stats
across
the members.
Make sense?
Thanks,
John Halley Gotway
On Fri, May 24, 2019 at 8:47 AM Carl, Mr. Glen, Contractor, Code 7532
via
RT <met_help at ucar.edu> wrote:
>
> Fri May 24 08:47:25 2019: Request 90342 was acted upon.
> Transaction: Ticket created by glen.carl.ctr at nrlmry.navy.mil
> Queue: met_help
> Subject: Ensemble ANOM_CORR Mean
> Owner: Nobody
> Requestors: glen.carl.ctr at nrlmry.navy.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90342 >
>
>
> How can we out the Anomaly Correlation mean for an ensemble.
ensemble_stat
> 20
> outputs the stats for the 20 member ensemble. Our problem is that
> ensemble_stat does not have an output for ANOM_CORR, or if it does
we do
> not
> know how to output it. We have found we can output the Anomaly
> Correlation
> for grid_stat however only for each member.
>
> Thanks,
> Glen
>
> --
> Mr. Glen CARL
> NRL Monterey Contractor SAIC/0cog, Bldg 704 Rm 133, Code 7532
> 831.325.6269
> glen.carl.ctr at nrlmry.navy.mil
>
>
>
------------------------------------------------
Subject: Ensemble ANOM_CORR Mean
From: Carl, Mr. Glen, Contractor, Code 7532
Time: Fri May 24 13:03:48 2019
I will give it a go. At least for the first two options.
Thanks and have a fun weekend!
Glen
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Friday, May 24, 2019 10:50 AM
To: Carl, Mr. Glen, Contractor, Code 7532
Subject: Re: [rt.rap.ucar.edu #90342] Ensemble ANOM_CORR Mean
Glen,
You're correct. The ANOM_CORR statistic is included in the CNT output
line
type which is included in the output Point-Stat and Grid-Stat. I see
that
you're running a 20-member ensemble. It sounds like you'd like to see
the
mean of the ANOM_CORR value for those 20 ensemble members. There are
a few
options here:
(1) One of the outputs from ensemble_stat is a NetCDF file containing
which you can configure to include the ensemble mean field. Just turn
"mean" on in the "ensemble_flag" config file option:
*ensemble_flag = {... mean = TRUE;*
*...*
*}*
Then you can run the ensemble mean field through Grid-Stat or Point-
Stat
and compute the ANOM_CORR statistic. This isn't exactly what you
asked for
but is similar.
(2) A second option is computing the ANOM_CORR statistic for each of
the 20
ensemble members and using stat_analysis to compute the mean of those
2
stats. You can do this using the "summary" job type or the
"aggregate_stat" job type.
Run a stat_analysis "summary" job on the ANOM_CORR column of output.
I
used some sample data to demonstrate:
*met-8.1/bin/stat_analysis -lookin
point_stat_GFS_CLIMO_1.0DEG_120000L_20120409_120000V.stat -job summary
-line_type CNT -column ANOM_CORR -by FCST_VAR*
This reads data from .stat file listed and dumps out summary info for
the
ANOM_CORR column, but does so for each unique FCST_VAR value
encountered:
COL_NAME: LINE_TYPE COLUMN FCST_VAR TOTAL MEAN MEAN_NCL
MEAN_NCU
MEAN_BCL MEAN_BCU STDEV STDEV_BCL STDEV_BCU MIN P10 P25
P50 P75 P90 MAX IQR RANGE WMO_TYPE WMO_MEAN
WMO_WEIGHTED_MEAN
SUMMARY: CNT ANOM_CORR TMP 4 0.5472 0.090704
1.0037
NA NA 0.28688 NA NA 0.40376 0.40376 0.40376
0.40376
0.5472 0.8054 0.97753 0.14344 0.57377 FISHER 0.70681
0.41373
SUMMARY: CNT ANOM_CORR UGRD 1 0.71647 NA NA
NA NA NA NA NA 0.71647 0.71647 0.71647
0.71647
0.71647 0.71647 0.71647 0 0 FISHER 0.71647
0.71647
SUMMARY: CNT ANOM_CORR VGRD 1 0.46991 NA NA
NA NA NA NA NA 0.46991 0.46991 0.46991
0.46991
0.46991 0.46991 0.46991 0 0 FISHER 0.46991
0.46991
SUMMARY: CNT ANOM_CORR WIND 1 0.64622 NA NA
NA NA NA NA NA 0.64622 0.64622 0.64622
0.64622
0.64622 0.64622 0.64622 0 0 FISHER 0.64622
0.64622
Run a stat_analysis "aggregate_stat" job type to aggregate together
SAL1L2
partial sums and recompute the CNT output line type:
*met-8.1/bin/stat_analysis -lookin
point_stat_GFS_CLIMO_1.0DEG_120000L_20120409_120000V.stat -job
aggregate_stat -line_type SAL1L2 -out_line_type CNT -by FCST_VAR
-out_stat
cnt.stat*
That logic would recompute the CNT stats aggregated across all 20 of
your
members.
(3) The third option, and what we'd probably use in the DTC, is to
just use
METviewer to apply the stat_analysis aggregations shown above as we're
plotting the data.
The ensemble-stat tool does not include any logic to compute stats
separately for each member and then report the mean of those stats
across
the members.
Make sense?
Thanks,
John Halley Gotway
On Fri, May 24, 2019 at 8:47 AM Carl, Mr. Glen, Contractor, Code 7532
via
RT <met_help at ucar.edu> wrote:
>
> Fri May 24 08:47:25 2019: Request 90342 was acted upon.
> Transaction: Ticket created by glen.carl.ctr at nrlmry.navy.mil
> Queue: met_help
> Subject: Ensemble ANOM_CORR Mean
> Owner: Nobody
> Requestors: glen.carl.ctr at nrlmry.navy.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90342 >
>
>
> How can we out the Anomaly Correlation mean for an ensemble.
ensemble_stat
> 20
> outputs the stats for the 20 member ensemble. Our problem is that
> ensemble_stat does not have an output for ANOM_CORR, or if it does
we do
> not
> know how to output it. We have found we can output the Anomaly
> Correlation
> for grid_stat however only for each member.
>
> Thanks,
> Glen
>
> --
> Mr. Glen CARL
> NRL Monterey Contractor SAIC/0cog, Bldg 704 Rm 133, Code 7532
> 831.325.6269
> glen.carl.ctr at nrlmry.navy.mil
>
>
>
------------------------------------------------
Subject: Ensemble ANOM_CORR Mean
From: Carl, Mr. Glen, Contractor, Code 7532
Time: Fri May 24 13:10:39 2019
I was told to only do Opt #1 the other options will not work for us at
this
time.
Glen
-----Original Message-----
From: Carl, Mr. Glen, Contractor, Code 7532
Sent: Friday, May 24, 2019 12:04 PM
To: 'met_help at ucar.edu'
Subject: RE: [rt.rap.ucar.edu #90342] Ensemble ANOM_CORR Mean
I will give it a go. At least for the first two options.
Thanks and have a fun weekend!
Glen
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Friday, May 24, 2019 10:50 AM
To: Carl, Mr. Glen, Contractor, Code 7532
Subject: Re: [rt.rap.ucar.edu #90342] Ensemble ANOM_CORR Mean
Glen,
You're correct. The ANOM_CORR statistic is included in the CNT output
line
type which is included in the output Point-Stat and Grid-Stat. I see
that
you're running a 20-member ensemble. It sounds like you'd like to see
the
mean of the ANOM_CORR value for those 20 ensemble members. There are
a few
options here:
(1) One of the outputs from ensemble_stat is a NetCDF file containing
which you can configure to include the ensemble mean field. Just turn
"mean" on in the "ensemble_flag" config file option:
*ensemble_flag = {... mean = TRUE;*
*...*
*}*
Then you can run the ensemble mean field through Grid-Stat or Point-
Stat
and compute the ANOM_CORR statistic. This isn't exactly what you
asked for
but is similar.
(2) A second option is computing the ANOM_CORR statistic for each of
the 20
ensemble members and using stat_analysis to compute the mean of those
2
stats. You can do this using the "summary" job type or the
"aggregate_stat" job type.
Run a stat_analysis "summary" job on the ANOM_CORR column of output.
I
used some sample data to demonstrate:
*met-8.1/bin/stat_analysis -lookin
point_stat_GFS_CLIMO_1.0DEG_120000L_20120409_120000V.stat -job summary
-line_type CNT -column ANOM_CORR -by FCST_VAR*
This reads data from .stat file listed and dumps out summary info for
the
ANOM_CORR column, but does so for each unique FCST_VAR value
encountered:
COL_NAME: LINE_TYPE COLUMN FCST_VAR TOTAL MEAN MEAN_NCL
MEAN_NCU
MEAN_BCL MEAN_BCU STDEV STDEV_BCL STDEV_BCU MIN P10 P25
P50 P75 P90 MAX IQR RANGE WMO_TYPE WMO_MEAN
WMO_WEIGHTED_MEAN
SUMMARY: CNT ANOM_CORR TMP 4 0.5472 0.090704
1.0037
NA NA 0.28688 NA NA 0.40376 0.40376 0.40376
0.40376
0.5472 0.8054 0.97753 0.14344 0.57377 FISHER 0.70681
0.41373
SUMMARY: CNT ANOM_CORR UGRD 1 0.71647 NA NA
NA NA NA NA NA 0.71647 0.71647 0.71647
0.71647
0.71647 0.71647 0.71647 0 0 FISHER 0.71647
0.71647
SUMMARY: CNT ANOM_CORR VGRD 1 0.46991 NA NA
NA NA NA NA NA 0.46991 0.46991 0.46991
0.46991
0.46991 0.46991 0.46991 0 0 FISHER 0.46991
0.46991
SUMMARY: CNT ANOM_CORR WIND 1 0.64622 NA NA
NA NA NA NA NA 0.64622 0.64622 0.64622
0.64622
0.64622 0.64622 0.64622 0 0 FISHER 0.64622
0.64622
Run a stat_analysis "aggregate_stat" job type to aggregate together
SAL1L2
partial sums and recompute the CNT output line type:
*met-8.1/bin/stat_analysis -lookin
point_stat_GFS_CLIMO_1.0DEG_120000L_20120409_120000V.stat -job
aggregate_stat -line_type SAL1L2 -out_line_type CNT -by FCST_VAR
-out_stat
cnt.stat*
That logic would recompute the CNT stats aggregated across all 20 of
your
members.
(3) The third option, and what we'd probably use in the DTC, is to
just use
METviewer to apply the stat_analysis aggregations shown above as we're
plotting the data.
The ensemble-stat tool does not include any logic to compute stats
separately for each member and then report the mean of those stats
across
the members.
Make sense?
Thanks,
John Halley Gotway
On Fri, May 24, 2019 at 8:47 AM Carl, Mr. Glen, Contractor, Code 7532
via
RT <met_help at ucar.edu> wrote:
>
> Fri May 24 08:47:25 2019: Request 90342 was acted upon.
> Transaction: Ticket created by glen.carl.ctr at nrlmry.navy.mil
> Queue: met_help
> Subject: Ensemble ANOM_CORR Mean
> Owner: Nobody
> Requestors: glen.carl.ctr at nrlmry.navy.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90342 >
>
>
> How can we out the Anomaly Correlation mean for an ensemble.
ensemble_stat
> 20
> outputs the stats for the 20 member ensemble. Our problem is that
> ensemble_stat does not have an output for ANOM_CORR, or if it does
we do
> not
> know how to output it. We have found we can output the Anomaly
> Correlation
> for grid_stat however only for each member.
>
> Thanks,
> Glen
>
> --
> Mr. Glen CARL
> NRL Monterey Contractor SAIC/0cog, Bldg 704 Rm 133, Code 7532
> 831.325.6269
> glen.carl.ctr at nrlmry.navy.mil
>
>
>
------------------------------------------------
Subject: Ensemble ANOM_CORR Mean
From: Carl, Mr. Glen, Contractor, Code 7532
Time: Fri May 24 14:53:43 2019
having trouble with our variable geopht_pre_0500.0. Here is the config
section
that could be at issue, and some of my output with debugging and
prints.
//
// Ensemble product fields to be processed
//
ens = {
ens_thresh = 1.0;
vld_thresh = 1.0;
field = [
{
name = "geopht_pre_0500.0";
level = "(*,*)";
cat_thresh = [ >5400., >=5500. ];
}
];
}
////////////////////////////////////////////////////////////////////////////////
//
// Forecast and observation fields to be verified
//
fcst = {
field = [
{
name = "geopht_pre_0500.0";
level = [ "(*,*)" ];
cat_thresh = [ >5400., >=5500. ];
}
];
}
obs = {
field = [
{
name = "geopht_pre_0500.0";
level = [ "(*,*)" ];
cat_thresh = [ >5400., >=5500. ];
}
];
}
ensemble_stat 20 navgem_ens*2016121500*.nc
nsembleStatConfigRegions -grid_obs verifecmwf*2016122000*.nc
DEBUG 1: Default Config File:
/software/depot/met-8.1/met-
8.1/share/met/config/GridStatConfig_default
DEBUG 1: User Config File: EnsembleStatConfigRegions
GSL_RNG_TYPE=mt19937
GSL_RNG_SEED=18446744072876451932
DEBUG 1: Forecast File: ensemble_stat_20161220_000000V_ens.nc
DEBUG 1: Observation File: verifecmwf_geopht_pre_0500_2016122000.nc
WARNING:
WARNING: read_data_plane() -> geopht_pre_0500.0(*,*) not found in
file:
ensemble_stat_20161220_000000V_ens.nc
WARNING:
ERROR :
ERROR : process_scores() -> no requested data found! Exiting...
ERROR :
grid_stat -outdir ANOM_CORR/geopht_pre_0500_2016121500
ensemble_stat_20161220_000000V_ens.nc
verifecmwf_geopht_pre_0500_2016122000.nc
EnsembleStatConfigRegions
-----Original Message-----
From: Carl, Mr. Glen, Contractor, Code 7532
Sent: Friday, May 24, 2019 12:10 PM
To: 'met_help at ucar.edu'
Subject: RE: [rt.rap.ucar.edu #90342] Ensemble ANOM_CORR Mean
I was told to only do Opt #1 the other options will not work for us at
this
time.
Glen
-----Original Message-----
From: Carl, Mr. Glen, Contractor, Code 7532
Sent: Friday, May 24, 2019 12:04 PM
To: 'met_help at ucar.edu'
Subject: RE: [rt.rap.ucar.edu #90342] Ensemble ANOM_CORR Mean
I will give it a go. At least for the first two options.
Thanks and have a fun weekend!
Glen
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Friday, May 24, 2019 10:50 AM
To: Carl, Mr. Glen, Contractor, Code 7532
Subject: Re: [rt.rap.ucar.edu #90342] Ensemble ANOM_CORR Mean
Glen,
You're correct. The ANOM_CORR statistic is included in the CNT output
line
type which is included in the output Point-Stat and Grid-Stat. I see
that
you're running a 20-member ensemble. It sounds like you'd like to see
the
mean of the ANOM_CORR value for those 20 ensemble members. There are
a few
options here:
(1) One of the outputs from ensemble_stat is a NetCDF file containing
which you can configure to include the ensemble mean field. Just turn
"mean" on in the "ensemble_flag" config file option:
*ensemble_flag = {... mean = TRUE;*
*...*
*}*
Then you can run the ensemble mean field through Grid-Stat or Point-
Stat
and compute the ANOM_CORR statistic. This isn't exactly what you
asked for
but is similar.
(2) A second option is computing the ANOM_CORR statistic for each of
the 20
ensemble members and using stat_analysis to compute the mean of those
2
stats. You can do this using the "summary" job type or the
"aggregate_stat" job type.
Run a stat_analysis "summary" job on the ANOM_CORR column of output.
I
used some sample data to demonstrate:
*met-8.1/bin/stat_analysis -lookin
point_stat_GFS_CLIMO_1.0DEG_120000L_20120409_120000V.stat -job summary
-line_type CNT -column ANOM_CORR -by FCST_VAR*
This reads data from .stat file listed and dumps out summary info for
the
ANOM_CORR column, but does so for each unique FCST_VAR value
encountered:
COL_NAME: LINE_TYPE COLUMN FCST_VAR TOTAL MEAN MEAN_NCL
MEAN_NCU
MEAN_BCL MEAN_BCU STDEV STDEV_BCL STDEV_BCU MIN P10 P25
P50 P75 P90 MAX IQR RANGE WMO_TYPE WMO_MEAN
WMO_WEIGHTED_MEAN
SUMMARY: CNT ANOM_CORR TMP 4 0.5472 0.090704
1.0037
NA NA 0.28688 NA NA 0.40376 0.40376 0.40376
0.40376
0.5472 0.8054 0.97753 0.14344 0.57377 FISHER 0.70681
0.41373
SUMMARY: CNT ANOM_CORR UGRD 1 0.71647 NA NA
NA NA NA NA NA 0.71647 0.71647 0.71647
0.71647
0.71647 0.71647 0.71647 0 0 FISHER 0.71647
0.71647
SUMMARY: CNT ANOM_CORR VGRD 1 0.46991 NA NA
NA NA NA NA NA 0.46991 0.46991 0.46991
0.46991
0.46991 0.46991 0.46991 0 0 FISHER 0.46991
0.46991
SUMMARY: CNT ANOM_CORR WIND 1 0.64622 NA NA
NA NA NA NA NA 0.64622 0.64622 0.64622
0.64622
0.64622 0.64622 0.64622 0 0 FISHER 0.64622
0.64622
Run a stat_analysis "aggregate_stat" job type to aggregate together
SAL1L2
partial sums and recompute the CNT output line type:
*met-8.1/bin/stat_analysis -lookin
point_stat_GFS_CLIMO_1.0DEG_120000L_20120409_120000V.stat -job
aggregate_stat -line_type SAL1L2 -out_line_type CNT -by FCST_VAR
-out_stat
cnt.stat*
That logic would recompute the CNT stats aggregated across all 20 of
your
members.
(3) The third option, and what we'd probably use in the DTC, is to
just use
METviewer to apply the stat_analysis aggregations shown above as we're
plotting the data.
The ensemble-stat tool does not include any logic to compute stats
separately for each member and then report the mean of those stats
across
the members.
Make sense?
Thanks,
John Halley Gotway
On Fri, May 24, 2019 at 8:47 AM Carl, Mr. Glen, Contractor, Code 7532
via
RT <met_help at ucar.edu> wrote:
>
> Fri May 24 08:47:25 2019: Request 90342 was acted upon.
> Transaction: Ticket created by glen.carl.ctr at nrlmry.navy.mil
> Queue: met_help
> Subject: Ensemble ANOM_CORR Mean
> Owner: Nobody
> Requestors: glen.carl.ctr at nrlmry.navy.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90342 >
>
>
> How can we out the Anomaly Correlation mean for an ensemble.
ensemble_stat
> 20
> outputs the stats for the 20 member ensemble. Our problem is that
> ensemble_stat does not have an output for ANOM_CORR, or if it does
we do
> not
> know how to output it. We have found we can output the Anomaly
> Correlation
> for grid_stat however only for each member.
>
> Thanks,
> Glen
>
> --
> Mr. Glen CARL
> NRL Monterey Contractor SAIC/0cog, Bldg 704 Rm 133, Code 7532
> 831.325.6269
> glen.carl.ctr at nrlmry.navy.mil
>
>
>
------------------------------------------------
Subject: Ensemble ANOM_CORR Mean
From: Carl, Mr. Glen, Contractor, Code 7532
Time: Fri May 24 15:10:27 2019
Here is the NetCDF header of the file we are trying to process.
[carl at predict netcdf]$ ncdump -h ensemble_stat_20161220_000000V_ens.nc
netcdf ensemble_stat_20161220_000000V_ens {
dimensions:
lat = 181 ;
lon = 360 ;
variables:
float geopht_pre_0500.0_pre_0500.0_ENS_MEAN(lat, lon) ;
geopht_pre_0500.0_pre_0500.0_ENS_MEAN:name =
"geopht_pre_0500.0" ;
geopht_pre_0500.0_pre_0500.0_ENS_MEAN:long_name =
"geopht_pre_0500.0 at pre_0500.0 Ensemble Mean" ;
geopht_pre_0500.0_pre_0500.0_ENS_MEAN:level =
"pre_0500.0" ;
geopht_pre_0500.0_pre_0500.0_ENS_MEAN:units =
"UNKNOWN" ;
geopht_pre_0500.0_pre_0500.0_ENS_MEAN:_FillValue =
-9999.f ;
geopht_pre_0500.0_pre_0500.0_ENS_MEAN:init_time =
"20161215_000000" ;
geopht_pre_0500.0_pre_0500.0_ENS_MEAN:init_time_ut =
"1481760000" ;
geopht_pre_0500.0_pre_0500.0_ENS_MEAN:valid_time =
"20161220_000000" ;
geopht_pre_0500.0_pre_0500.0_ENS_MEAN:valid_time_ut =
"1482192000" ;
geopht_pre_0500.0_pre_0500.0_ENS_MEAN:accum_time =
"NA" ;
geopht_pre_0500.0_pre_0500.0_ENS_MEAN:accum_time_sec =
-9999 ;
float geopht_pre_0500.0_pre_0500.0_ENS_STDEV(lat, lon) ;
geopht_pre_0500.0_pre_0500.0_ENS_STDEV:name =
"geopht_pre_0500.0_ENS_STDEV" ;
geopht_pre_0500.0_pre_0500.0_ENS_STDEV:long_name =
"geopht_pre_0500.0 at pre_0500.0 Ensemble Standard Deviation" ;
geopht_pre_0500.0_pre_0500.0_ENS_STDEV:level =
"pre_0500.0" ;
geopht_pre_0500.0_pre_0500.0_ENS_STDEV:units =
"UNKNOWN" ;
geopht_pre_0500.0_pre_0500.0_ENS_STDEV:_FillValue =
-9999.f ;
geopht_pre_0500.0_pre_0500.0_ENS_STDEV:init_time =
"20161215_000000" ;
geopht_pre_0500.0_pre_0500.0_ENS_STDEV:init_time_ut =
"1481760000" ;
geopht_pre_0500.0_pre_0500.0_ENS_STDEV:valid_time =
"20161220_000000" ;
geopht_pre_0500.0_pre_0500.0_ENS_STDEV:valid_time_ut =
"1482192000" ;
geopht_pre_0500.0_pre_0500.0_ENS_STDEV:accum_time =
"NA" ;
geopht_pre_0500.0_pre_0500.0_ENS_STDEV:accum_time_sec
= -9999
;
// global attributes:
:FileOrigins = "File
./ensemble_stat_20161220_000000V_ens.nc
generated 20190524_200824 UTC on host predict.nrlmry.navy.mil by the
MET
ensemble_stat tool" ;
:MET_version = "V8.1" ;
:MET_tool = "ensemble_stat" ;
:model = "NAVGEM" ;
:obtype = "ECANL" ;
:Projection = "LatLon" ;
:lat_ll = "-90.000000 degrees_north" ;
:lon_ll = "0.000000 degrees_east" ;
:delta_lat = "1.000000 degrees" ;
:delta_lon = "1.000000 degrees" ;
:Nlat = "181 grid_points" ;
:Nlon = "360 grid_points" ;
-----Original Message-----
From: Carl, Mr. Glen, Contractor, Code 7532
Sent: Friday, May 24, 2019 1:54 PM
To: met_help at ucar.edu
Subject: RE: [rt.rap.ucar.edu #90342] Ensemble ANOM_CORR Mean
having trouble with our variable geopht_pre_0500.0. Here is the config
section
that could be at issue, and some of my output with debugging and
prints.
//
// Ensemble product fields to be processed
//
ens = {
ens_thresh = 1.0;
vld_thresh = 1.0;
field = [
{
name = "geopht_pre_0500.0";
level = "(*,*)";
cat_thresh = [ >5400., >=5500. ];
}
];
}
////////////////////////////////////////////////////////////////////////////////
//
// Forecast and observation fields to be verified
//
fcst = {
field = [
{
name = "geopht_pre_0500.0";
level = [ "(*,*)" ];
cat_thresh = [ >5400., >=5500. ];
}
];
}
obs = {
field = [
{
name = "geopht_pre_0500.0";
level = [ "(*,*)" ];
cat_thresh = [ >5400., >=5500. ];
}
];
}
ensemble_stat 20 navgem_ens*2016121500*.nc
nsembleStatConfigRegions -grid_obs verifecmwf*2016122000*.nc
DEBUG 1: Default Config File:
/software/depot/met-8.1/met-
8.1/share/met/config/GridStatConfig_default
DEBUG 1: User Config File: EnsembleStatConfigRegions
GSL_RNG_TYPE=mt19937
GSL_RNG_SEED=18446744072876451932
DEBUG 1: Forecast File: ensemble_stat_20161220_000000V_ens.nc
DEBUG 1: Observation File: verifecmwf_geopht_pre_0500_2016122000.nc
WARNING:
WARNING: read_data_plane() -> geopht_pre_0500.0(*,*) not found in
file:
ensemble_stat_20161220_000000V_ens.nc
WARNING:
ERROR :
ERROR : process_scores() -> no requested data found! Exiting...
ERROR :
grid_stat -outdir ANOM_CORR/geopht_pre_0500_2016121500
ensemble_stat_20161220_000000V_ens.nc
verifecmwf_geopht_pre_0500_2016122000.nc
EnsembleStatConfigRegions
-----Original Message-----
From: Carl, Mr. Glen, Contractor, Code 7532
Sent: Friday, May 24, 2019 12:10 PM
To: 'met_help at ucar.edu'
Subject: RE: [rt.rap.ucar.edu #90342] Ensemble ANOM_CORR Mean
I was told to only do Opt #1 the other options will not work for us at
this
time.
Glen
-----Original Message-----
From: Carl, Mr. Glen, Contractor, Code 7532
Sent: Friday, May 24, 2019 12:04 PM
To: 'met_help at ucar.edu'
Subject: RE: [rt.rap.ucar.edu #90342] Ensemble ANOM_CORR Mean
I will give it a go. At least for the first two options.
Thanks and have a fun weekend!
Glen
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Friday, May 24, 2019 10:50 AM
To: Carl, Mr. Glen, Contractor, Code 7532
Subject: Re: [rt.rap.ucar.edu #90342] Ensemble ANOM_CORR Mean
Glen,
You're correct. The ANOM_CORR statistic is included in the CNT output
line
type which is included in the output Point-Stat and Grid-Stat. I see
that
you're running a 20-member ensemble. It sounds like you'd like to see
the
mean of the ANOM_CORR value for those 20 ensemble members. There are
a few
options here:
(1) One of the outputs from ensemble_stat is a NetCDF file containing
which you can configure to include the ensemble mean field. Just turn
"mean" on in the "ensemble_flag" config file option:
*ensemble_flag = {... mean = TRUE;*
*...*
*}*
Then you can run the ensemble mean field through Grid-Stat or Point-
Stat
and compute the ANOM_CORR statistic. This isn't exactly what you
asked for
but is similar.
(2) A second option is computing the ANOM_CORR statistic for each of
the 20
ensemble members and using stat_analysis to compute the mean of those
2
stats. You can do this using the "summary" job type or the
"aggregate_stat" job type.
Run a stat_analysis "summary" job on the ANOM_CORR column of output.
I
used some sample data to demonstrate:
*met-8.1/bin/stat_analysis -lookin
point_stat_GFS_CLIMO_1.0DEG_120000L_20120409_120000V.stat -job summary
-line_type CNT -column ANOM_CORR -by FCST_VAR*
This reads data from .stat file listed and dumps out summary info for
the
ANOM_CORR column, but does so for each unique FCST_VAR value
encountered:
COL_NAME: LINE_TYPE COLUMN FCST_VAR TOTAL MEAN MEAN_NCL
MEAN_NCU
MEAN_BCL MEAN_BCU STDEV STDEV_BCL STDEV_BCU MIN P10 P25
P50 P75 P90 MAX IQR RANGE WMO_TYPE WMO_MEAN
WMO_WEIGHTED_MEAN
SUMMARY: CNT ANOM_CORR TMP 4 0.5472 0.090704
1.0037
NA NA 0.28688 NA NA 0.40376 0.40376 0.40376
0.40376
0.5472 0.8054 0.97753 0.14344 0.57377 FISHER 0.70681
0.41373
SUMMARY: CNT ANOM_CORR UGRD 1 0.71647 NA NA
NA NA NA NA NA 0.71647 0.71647 0.71647
0.71647
0.71647 0.71647 0.71647 0 0 FISHER 0.71647
0.71647
SUMMARY: CNT ANOM_CORR VGRD 1 0.46991 NA NA
NA NA NA NA NA 0.46991 0.46991 0.46991
0.46991
0.46991 0.46991 0.46991 0 0 FISHER 0.46991
0.46991
SUMMARY: CNT ANOM_CORR WIND 1 0.64622 NA NA
NA NA NA NA NA 0.64622 0.64622 0.64622
0.64622
0.64622 0.64622 0.64622 0 0 FISHER 0.64622
0.64622
Run a stat_analysis "aggregate_stat" job type to aggregate together
SAL1L2
partial sums and recompute the CNT output line type:
*met-8.1/bin/stat_analysis -lookin
point_stat_GFS_CLIMO_1.0DEG_120000L_20120409_120000V.stat -job
aggregate_stat -line_type SAL1L2 -out_line_type CNT -by FCST_VAR
-out_stat
cnt.stat*
That logic would recompute the CNT stats aggregated across all 20 of
your
members.
(3) The third option, and what we'd probably use in the DTC, is to
just use
METviewer to apply the stat_analysis aggregations shown above as we're
plotting the data.
The ensemble-stat tool does not include any logic to compute stats
separately for each member and then report the mean of those stats
across
the members.
Make sense?
Thanks,
John Halley Gotway
On Fri, May 24, 2019 at 8:47 AM Carl, Mr. Glen, Contractor, Code 7532
via
RT <met_help at ucar.edu> wrote:
>
> Fri May 24 08:47:25 2019: Request 90342 was acted upon.
> Transaction: Ticket created by glen.carl.ctr at nrlmry.navy.mil
> Queue: met_help
> Subject: Ensemble ANOM_CORR Mean
> Owner: Nobody
> Requestors: glen.carl.ctr at nrlmry.navy.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90342 >
>
>
> How can we out the Anomaly Correlation mean for an ensemble.
ensemble_stat
> 20
> outputs the stats for the 20 member ensemble. Our problem is that
> ensemble_stat does not have an output for ANOM_CORR, or if it does
we do
> not
> know how to output it. We have found we can output the Anomaly
> Correlation
> for grid_stat however only for each member.
>
> Thanks,
> Glen
>
> --
> Mr. Glen CARL
> NRL Monterey Contractor SAIC/0cog, Bldg 704 Rm 133, Code 7532
> 831.325.6269
> glen.carl.ctr at nrlmry.navy.mil
>
>
>
------------------------------------------------
Subject: Ensemble ANOM_CORR Mean
From: Carl, Mr. Glen, Contractor, Code 7532
Time: Tue May 28 16:55:22 2019
I am having trouble with the read_data_plane() for processing the
anomaly
correlation for the ensemble per your option 1 suggestion. I dumped
some
debug, and config options that had different output. Is there
something I need
to check within the ensemble_stat....nc file, or in the
EnsembleStatConfig.
Thanks,
Glen
--
Mr. Glen CARL
NRL Monterey Contractor SAIC/0cog, Bldg 704 Rm 133, Code 7532
831.325.6269
glen.carl.ctr at nrlmry.navy.mil
I set the configs with
ensemble_flag = {
...
mean = TRUE;
..
and run
grid_stat -v 5 -outdir ANOM_CORR ensemble_stat_20161220_000000V_ens.nc
verifecmwf_geopht_pre_0500_2016122000.nc EnsembleStatConfigACORR
I have been trying to get my config to process the ensemble_stat_ens.n
and
verify.nc
here are the configs and results I am getting
fcst = {
field = [
{
name = "geopht_pre_0500.0";
level = ["(,)"];
cat_thresh = [ >5400., >=5500. ];
}
];
}
WARNING:
WARNING: read_data_plane() -> geopht_pre_0500.0(,) not found in file:
ensemble_stat_20161220_000000V_ens.nc
WARNING:
or when I set fcst field
fcst = {
field = [
{
// name = "geopht_pre_0500.0_pre_0500.0_ENS_MEAN";
level = ["(,)"];
cat_thresh = [ >5400., >=5500. ];
}
];
}
VarInfoNcMet::dump():
Dimension:
Nelements = 0
Nalloc = 0
AllocInc = 10
DEBUG 5: Parsed observation field:
VarInfo::dump():
MagicStr = geopht_pre_0500.0(,)
ReqName = geopht_pre_0500.0
Name = geopht_pre_0500.0
LongName = geopht_pre_0500.0(,)
Units = NA
PFlag = 0
PName = (nul)
PUnits = (nul)
PAsScalar = 0
UVIndex = -1
Init = 19700101_000000 (0)
Valid = 19700101_000000 (0)
Ensemble = (nul)
Lead = NA (-9999)
ConvertFx = (nul)
CensorThresh = NA
CensorVal =
Regrid = NEAREST
LevelInfo::dump():
Type = LevelType_None
TypeNum = -9999
ReqName = ,
Name = ,
Units = (nul)
Upper = 0
Increment = 0
VarInfoNcMet::dump():
Dimension:
Nelements = 0
Nalloc = 0
AllocInc = 10
DEBUG 5: Parsed thresholds:
DEBUG 5: Forecast categorical thresholds: >5400.,>=5500.
DEBUG 5: Observed categorical thresholds: >5400.,>=5500.
DEBUG 5: Forecast continuous thresholds: NA
DEBUG 5: Observed continuous thresholds: NA
DEBUG 5: Continuous threshold logic: UNION
DEBUG 5: Forecast wind speed thresholds: NA
DEBUG 5: Observed wind speed thresholds: NA
DEBUG 5: Wind speed threshold logic: UNION
DEBUG 3: Use the grid named "G003".
DEBUG 3: Grid Definition: Projection: Lat/Lon Nx: 360 Ny: 181 lat_ll:
-90.000
lon_ll: -0.000 delta_lat: 1.000 delta_lon: 1.000
GSL_RNG_TYPE=mt19937
GSL_RNG_SEED=549055836
DEBUG 1: Forecast File: ensemble_stat_20161220_000000V_ens.nc
DEBUG 1: Observation File: verifecmwf_geopht_pre_0500_2016122000.nc
ERROR :
ERROR : MetNcFile::data(NcVar *, const LongArray &, DataPlane &) ->
needed 2
arguments for variable geopht_pre_0500.0_pre_0500.0_ENS_MEAN, got 0
ERROR :
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Friday, May 24, 2019 10:50 AM
To: Carl, Mr. Glen, Contractor, Code 7532
Subject: Re: [rt.rap.ucar.edu #90342] Ensemble ANOM_CORR Mean
Glen,
You're correct. The ANOM_CORR statistic is included in the CNT output
line
type which is included in the output Point-Stat and Grid-Stat. I see
that
you're running a 20-member ensemble. It sounds like you'd like to see
the
mean of the ANOM_CORR value for those 20 ensemble members. There are
a few
options here:
(1) One of the outputs from ensemble_stat is a NetCDF file containing
which you can configure to include the ensemble mean field. Just turn
"mean" on in the "ensemble_flag" config file option:
*ensemble_flag = {... mean = TRUE;*
*...*
*}*
Then you can run the ensemble mean field through Grid-Stat or Point-
Stat
and compute the ANOM_CORR statistic. This isn't exactly what you
asked for
but is similar.
(2) A second option is computing the ANOM_CORR statistic for each of
the 20
ensemble members and using stat_analysis to compute the mean of those
2
stats. You can do this using the "summary" job type or the
"aggregate_stat" job type.
Run a stat_analysis "summary" job on the ANOM_CORR column of output.
I
used some sample data to demonstrate:
*met-8.1/bin/stat_analysis -lookin
point_stat_GFS_CLIMO_1.0DEG_120000L_20120409_120000V.stat -job summary
-line_type CNT -column ANOM_CORR -by FCST_VAR*
This reads data from .stat file listed and dumps out summary info for
the
ANOM_CORR column, but does so for each unique FCST_VAR value
encountered:
COL_NAME: LINE_TYPE COLUMN FCST_VAR TOTAL MEAN MEAN_NCL
MEAN_NCU
MEAN_BCL MEAN_BCU STDEV STDEV_BCL STDEV_BCU MIN P10 P25
P50 P75 P90 MAX IQR RANGE WMO_TYPE WMO_MEAN
WMO_WEIGHTED_MEAN
SUMMARY: CNT ANOM_CORR TMP 4 0.5472 0.090704
1.0037
NA NA 0.28688 NA NA 0.40376 0.40376 0.40376
0.40376
0.5472 0.8054 0.97753 0.14344 0.57377 FISHER 0.70681
0.41373
SUMMARY: CNT ANOM_CORR UGRD 1 0.71647 NA NA
NA NA NA NA NA 0.71647 0.71647 0.71647
0.71647
0.71647 0.71647 0.71647 0 0 FISHER 0.71647
0.71647
SUMMARY: CNT ANOM_CORR VGRD 1 0.46991 NA NA
NA NA NA NA NA 0.46991 0.46991 0.46991
0.46991
0.46991 0.46991 0.46991 0 0 FISHER 0.46991
0.46991
SUMMARY: CNT ANOM_CORR WIND 1 0.64622 NA NA
NA NA NA NA NA 0.64622 0.64622 0.64622
0.64622
0.64622 0.64622 0.64622 0 0 FISHER 0.64622
0.64622
Run a stat_analysis "aggregate_stat" job type to aggregate together
SAL1L2
partial sums and recompute the CNT output line type:
*met-8.1/bin/stat_analysis -lookin
point_stat_GFS_CLIMO_1.0DEG_120000L_20120409_120000V.stat -job
aggregate_stat -line_type SAL1L2 -out_line_type CNT -by FCST_VAR
-out_stat
cnt.stat*
That logic would recompute the CNT stats aggregated across all 20 of
your
members.
(3) The third option, and what we'd probably use in the DTC, is to
just use
METviewer to apply the stat_analysis aggregations shown above as we're
plotting the data.
The ensemble-stat tool does not include any logic to compute stats
separately for each member and then report the mean of those stats
across
the members.
Make sense?
Thanks,
John Halley Gotway
On Fri, May 24, 2019 at 8:47 AM Carl, Mr. Glen, Contractor, Code 7532
via
RT <met_help at ucar.edu> wrote:
>
> Fri May 24 08:47:25 2019: Request 90342 was acted upon.
> Transaction: Ticket created by glen.carl.ctr at nrlmry.navy.mil
> Queue: met_help
> Subject: Ensemble ANOM_CORR Mean
> Owner: Nobody
> Requestors: glen.carl.ctr at nrlmry.navy.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90342 >
>
>
> How can we out the Anomaly Correlation mean for an ensemble.
ensemble_stat
> 20
> outputs the stats for the 20 member ensemble. Our problem is that
> ensemble_stat does not have an output for ANOM_CORR, or if it does
we do
> not
> know how to output it. We have found we can output the Anomaly
> Correlation
> for grid_stat however only for each member.
>
> Thanks,
> Glen
>
> --
> Mr. Glen CARL
> NRL Monterey Contractor SAIC/0cog, Bldg 704 Rm 133, Code 7532
> 831.325.6269
> glen.carl.ctr at nrlmry.navy.mil
>
>
>
------------------------------------------------
Subject: Ensemble ANOM_CORR Mean
From: Howard Soh
Time: Wed May 29 08:24:29 2019
John is out of office this week.
I'm not an expert on grid_stat/ensemble_stat. I saw a syntax which MET
does not support. Would you change the level configuration: "(,)" to
"(*,*)" ?
fcst = {
field = [
{
name = "geopht_pre_0500.0";
level = ["(,)"];
cat_thresh = [ >5400., >=5500. ];
}
];
}
==>
fcst = {
field = [
{
name = "geopht_pre_0500.0";
level = ["(*,*)"];
cat_thresh = [ >5400., >=5500. ];
}
];
}
Cheers,
Howard
On Tue May 28 16:55:22 2019, glen.carl.ctr at nrlmry.navy.mil wrote:
> I am having trouble with the read_data_plane() for processing the
> anomaly
> correlation for the ensemble per your option 1 suggestion. I dumped
> some
> debug, and config options that had different output. Is there
> something I need
> to check within the ensemble_stat....nc file, or in the
> EnsembleStatConfig.
> Thanks,
> Glen
> --
> Mr. Glen CARL
> NRL Monterey Contractor SAIC/0cog, Bldg 704 Rm 133, Code 7532
> 831.325.6269
> glen.carl.ctr at nrlmry.navy.mil
>
>
>
>
>
> I set the configs with
> ensemble_flag = {
> ...
> mean = TRUE;
> ..
>
> and run
> grid_stat -v 5 -outdir ANOM_CORR
> ensemble_stat_20161220_000000V_ens.nc
> verifecmwf_geopht_pre_0500_2016122000.nc EnsembleStatConfigACORR
>
>
> I have been trying to get my config to process the
ensemble_stat_ens.n
> and
> verify.nc
>
> here are the configs and results I am getting
>
> fcst = {
> field = [
> {
> name = "geopht_pre_0500.0";
> level = ["(,)"];
> cat_thresh = [ >5400., >=5500. ];
> }
> ];
> }
>
> WARNING:
> WARNING: read_data_plane() -> geopht_pre_0500.0(,) not found in
file:
> ensemble_stat_20161220_000000V_ens.nc
> WARNING:
>
> or when I set fcst field
> fcst = {
> field = [
> {
> // name = "geopht_pre_0500.0_pre_0500.0_ENS_MEAN";
> level = ["(,)"];
> cat_thresh = [ >5400., >=5500. ];
> }
> ];
> }
>
>
> VarInfoNcMet::dump():
> Dimension:
> Nelements = 0
> Nalloc = 0
> AllocInc = 10
> DEBUG 5: Parsed observation field:
> VarInfo::dump():
> MagicStr = geopht_pre_0500.0(,)
> ReqName = geopht_pre_0500.0
> Name = geopht_pre_0500.0
> LongName = geopht_pre_0500.0(,)
> Units = NA
> PFlag = 0
> PName = (nul)
> PUnits = (nul)
> PAsScalar = 0
> UVIndex = -1
> Init = 19700101_000000 (0)
> Valid = 19700101_000000 (0)
> Ensemble = (nul)
> Lead = NA (-9999)
> ConvertFx = (nul)
> CensorThresh = NA
> CensorVal =
> Regrid = NEAREST
> LevelInfo::dump():
> Type = LevelType_None
> TypeNum = -9999
> ReqName = ,
> Name = ,
> Units = (nul)
> Upper = 0
> Increment = 0
> VarInfoNcMet::dump():
> Dimension:
> Nelements = 0
> Nalloc = 0
> AllocInc = 10
> DEBUG 5: Parsed thresholds:
> DEBUG 5: Forecast categorical thresholds: >5400.,>=5500.
> DEBUG 5: Observed categorical thresholds: >5400.,>=5500.
> DEBUG 5: Forecast continuous thresholds: NA
> DEBUG 5: Observed continuous thresholds: NA
> DEBUG 5: Continuous threshold logic: UNION
> DEBUG 5: Forecast wind speed thresholds: NA
> DEBUG 5: Observed wind speed thresholds: NA
> DEBUG 5: Wind speed threshold logic: UNION
> DEBUG 3: Use the grid named "G003".
> DEBUG 3: Grid Definition: Projection: Lat/Lon Nx: 360 Ny: 181
lat_ll:
> -90.000
> lon_ll: -0.000 delta_lat: 1.000 delta_lon: 1.000
> GSL_RNG_TYPE=mt19937
> GSL_RNG_SEED=549055836
> DEBUG 1: Forecast File: ensemble_stat_20161220_000000V_ens.nc
> DEBUG 1: Observation File: verifecmwf_geopht_pre_0500_2016122000.nc
> ERROR :
> ERROR : MetNcFile::data(NcVar *, const LongArray &, DataPlane &)
->
> needed 2
> arguments for variable geopht_pre_0500.0_pre_0500.0_ENS_MEAN, got 0
> ERROR :
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Friday, May 24, 2019 10:50 AM
> To: Carl, Mr. Glen, Contractor, Code 7532
> Subject: Re: [rt.rap.ucar.edu #90342] Ensemble ANOM_CORR Mean
>
> Glen,
>
> You're correct. The ANOM_CORR statistic is included in the CNT
output
> line
> type which is included in the output Point-Stat and Grid-Stat. I
see
> that
> you're running a 20-member ensemble. It sounds like you'd like to
see
> the
> mean of the ANOM_CORR value for those 20 ensemble members. There
are
> a few
> options here:
>
> (1) One of the outputs from ensemble_stat is a NetCDF file
containing
> which you can configure to include the ensemble mean field. Just
turn
> "mean" on in the "ensemble_flag" config file option:
>
>
> *ensemble_flag = {... mean = TRUE;*
> *...*
> *}*
> Then you can run the ensemble mean field through Grid-Stat or Point-
> Stat
> and compute the ANOM_CORR statistic. This isn't exactly what you
> asked for
> but is similar.
>
> (2) A second option is computing the ANOM_CORR statistic for each of
> the 20
> ensemble members and using stat_analysis to compute the mean of
those
> 2
> stats. You can do this using the "summary" job type or the
> "aggregate_stat" job type.
>
> Run a stat_analysis "summary" job on the ANOM_CORR column of output.
> I
> used some sample data to demonstrate:
> *met-8.1/bin/stat_analysis -lookin
> point_stat_GFS_CLIMO_1.0DEG_120000L_20120409_120000V.stat -job
summary
> -line_type CNT -column ANOM_CORR -by FCST_VAR*
>
> This reads data from .stat file listed and dumps out summary info
for
> the
> ANOM_CORR column, but does so for each unique FCST_VAR value
> encountered:
> COL_NAME: LINE_TYPE COLUMN FCST_VAR TOTAL MEAN MEAN_NCL
> MEAN_NCU
> MEAN_BCL MEAN_BCU STDEV STDEV_BCL STDEV_BCU MIN P10
P25
> P50 P75 P90 MAX IQR RANGE WMO_TYPE WMO_MEAN
> WMO_WEIGHTED_MEAN
> SUMMARY: CNT ANOM_CORR TMP 4 0.5472 0.090704
> 1.0037
> NA NA 0.28688 NA NA 0.40376 0.40376 0.40376
> 0.40376
> 0.5472 0.8054 0.97753 0.14344 0.57377 FISHER 0.70681
> 0.41373
> SUMMARY: CNT ANOM_CORR UGRD 1 0.71647 NA NA
> NA NA NA NA NA 0.71647 0.71647 0.71647
> 0.71647
> 0.71647 0.71647 0.71647 0 0 FISHER 0.71647
> 0.71647
> SUMMARY: CNT ANOM_CORR VGRD 1 0.46991 NA NA
> NA NA NA NA NA 0.46991 0.46991 0.46991
> 0.46991
> 0.46991 0.46991 0.46991 0 0 FISHER 0.46991
> 0.46991
> SUMMARY: CNT ANOM_CORR WIND 1 0.64622 NA NA
> NA NA NA NA NA 0.64622 0.64622 0.64622
> 0.64622
> 0.64622 0.64622 0.64622 0 0 FISHER 0.64622
> 0.64622
>
> Run a stat_analysis "aggregate_stat" job type to aggregate together
> SAL1L2
> partial sums and recompute the CNT output line type:
> *met-8.1/bin/stat_analysis -lookin
> point_stat_GFS_CLIMO_1.0DEG_120000L_20120409_120000V.stat -job
> aggregate_stat -line_type SAL1L2 -out_line_type CNT -by FCST_VAR
> -out_stat
> cnt.stat*
>
> That logic would recompute the CNT stats aggregated across all 20 of
> your
> members.
>
> (3) The third option, and what we'd probably use in the DTC, is to
> just use
> METviewer to apply the stat_analysis aggregations shown above as
we're
> plotting the data.
>
> The ensemble-stat tool does not include any logic to compute stats
> separately for each member and then report the mean of those stats
> across
> the members.
>
> Make sense?
>
> Thanks,
> John Halley Gotway
>
>
> On Fri, May 24, 2019 at 8:47 AM Carl, Mr. Glen, Contractor, Code
7532
> via
> RT <met_help at ucar.edu> wrote:
>
> >
> > Fri May 24 08:47:25 2019: Request 90342 was acted upon.
> > Transaction: Ticket created by glen.carl.ctr at nrlmry.navy.mil
> > Queue: met_help
> > Subject: Ensemble ANOM_CORR Mean
> > Owner: Nobody
> > Requestors: glen.carl.ctr at nrlmry.navy.mil
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90342
> > >
> >
> >
> > How can we out the Anomaly Correlation mean for an ensemble.
> > ensemble_stat
> > 20
> > outputs the stats for the 20 member ensemble. Our problem is that
> > ensemble_stat does not have an output for ANOM_CORR, or if it does
we
> > do
> > not
> > know how to output it. We have found we can output the Anomaly
> > Correlation
> > for grid_stat however only for each member.
> >
> > Thanks,
> > Glen
> >
> > --
> > Mr. Glen CARL
> > NRL Monterey Contractor SAIC/0cog, Bldg 704 Rm 133, Code 7532
> > 831.325.6269
> > glen.carl.ctr at nrlmry.navy.mil
> >
> >
> >
------------------------------------------------
Subject: Ensemble ANOM_CORR Mean
From: Howard Soh
Time: Wed May 29 08:34:46 2019
There is another warning.
WARNING: read_data_plane() -> geopht_pre_0500.0(*,*) not found in
file:
ensemble_stat_20161220_000000V_ens.nc
WARNING:
Would you try this command?
./plot_data_plane <input_netcdf_name> <output_image_name>
<field_configuration>
It will be
./plot_data_plane ensemble_stat_20161220_000000V_ens.nc
tmp_data_plane_output.ps 'name="geopht_pre_0500.0"; level="(*,*)";' -v
5
Cheers,
Howard
On Wed May 29 08:24:29 2019, hsoh wrote:
> John is out of office this week.
>
> I'm not an expert on grid_stat/ensemble_stat. I saw a syntax which
MET
> does not support. Would you change the level configuration: "(,)" to
> "(*,*)" ?
>
> fcst = {
> field = [
> {
> name = "geopht_pre_0500.0";
> level = ["(,)"];
> cat_thresh = [ >5400., >=5500. ];
> }
> ];
> }
>
> ==>
>
> fcst = {
> field = [
> {
> name = "geopht_pre_0500.0";
> level = ["(*,*)"];
> cat_thresh = [ >5400., >=5500. ];
> }
> ];
> }
>
> Cheers,
> Howard
>
> On Tue May 28 16:55:22 2019, glen.carl.ctr at nrlmry.navy.mil wrote:
> > I am having trouble with the read_data_plane() for processing the
> > anomaly
> > correlation for the ensemble per your option 1 suggestion. I
dumped
> > some
> > debug, and config options that had different output. Is there
> > something I need
> > to check within the ensemble_stat....nc file, or in the
> > EnsembleStatConfig.
> > Thanks,
> > Glen
> > --
> > Mr. Glen CARL
> > NRL Monterey Contractor SAIC/0cog, Bldg 704 Rm 133, Code 7532
> > 831.325.6269
> > glen.carl.ctr at nrlmry.navy.mil
> >
> >
> >
> >
> >
> > I set the configs with
> > ensemble_flag = {
> > ...
> > mean = TRUE;
> > ..
> >
> > and run
> > grid_stat -v 5 -outdir ANOM_CORR
> > ensemble_stat_20161220_000000V_ens.nc
> > verifecmwf_geopht_pre_0500_2016122000.nc EnsembleStatConfigACORR
> >
> >
> > I have been trying to get my config to process the
> > ensemble_stat_ens.n
> > and
> > verify.nc
> >
> > here are the configs and results I am getting
> >
> > fcst = {
> > field = [
> > {
> > name = "geopht_pre_0500.0";
> > level = ["(,)"];
> > cat_thresh = [ >5400., >=5500. ];
> > }
> > ];
> > }
> >
> > WARNING:
> > WARNING: read_data_plane() -> geopht_pre_0500.0(,) not found in
> > file:
> > ensemble_stat_20161220_000000V_ens.nc
> > WARNING:
> >
> > or when I set fcst field
> > fcst = {
> > field = [
> > {
> > // name = "geopht_pre_0500.0_pre_0500.0_ENS_MEAN";
> > level = ["(,)"];
> > cat_thresh = [ >5400., >=5500. ];
> > }
> > ];
> > }
> >
> >
> > VarInfoNcMet::dump():
> > Dimension:
> > Nelements = 0
> > Nalloc = 0
> > AllocInc = 10
> > DEBUG 5: Parsed observation field:
> > VarInfo::dump():
> > MagicStr = geopht_pre_0500.0(,)
> > ReqName = geopht_pre_0500.0
> > Name = geopht_pre_0500.0
> > LongName = geopht_pre_0500.0(,)
> > Units = NA
> > PFlag = 0
> > PName = (nul)
> > PUnits = (nul)
> > PAsScalar = 0
> > UVIndex = -1
> > Init = 19700101_000000 (0)
> > Valid = 19700101_000000 (0)
> > Ensemble = (nul)
> > Lead = NA (-9999)
> > ConvertFx = (nul)
> > CensorThresh = NA
> > CensorVal =
> > Regrid = NEAREST
> > LevelInfo::dump():
> > Type = LevelType_None
> > TypeNum = -9999
> > ReqName = ,
> > Name = ,
> > Units = (nul)
> > Upper = 0
> > Increment = 0
> > VarInfoNcMet::dump():
> > Dimension:
> > Nelements = 0
> > Nalloc = 0
> > AllocInc = 10
> > DEBUG 5: Parsed thresholds:
> > DEBUG 5: Forecast categorical thresholds: >5400.,>=5500.
> > DEBUG 5: Observed categorical thresholds: >5400.,>=5500.
> > DEBUG 5: Forecast continuous thresholds: NA
> > DEBUG 5: Observed continuous thresholds: NA
> > DEBUG 5: Continuous threshold logic: UNION
> > DEBUG 5: Forecast wind speed thresholds: NA
> > DEBUG 5: Observed wind speed thresholds: NA
> > DEBUG 5: Wind speed threshold logic: UNION
> > DEBUG 3: Use the grid named "G003".
> > DEBUG 3: Grid Definition: Projection: Lat/Lon Nx: 360 Ny: 181
> > lat_ll:
> > -90.000
> > lon_ll: -0.000 delta_lat: 1.000 delta_lon: 1.000
> > GSL_RNG_TYPE=mt19937
> > GSL_RNG_SEED=549055836
> > DEBUG 1: Forecast File: ensemble_stat_20161220_000000V_ens.nc
> > DEBUG 1: Observation File:
verifecmwf_geopht_pre_0500_2016122000.nc
> > ERROR :
> > ERROR : MetNcFile::data(NcVar *, const LongArray &, DataPlane &)
->
> > needed 2
> > arguments for variable geopht_pre_0500.0_pre_0500.0_ENS_MEAN, got
0
> > ERROR :
> >
> > -----Original Message-----
> > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > Sent: Friday, May 24, 2019 10:50 AM
> > To: Carl, Mr. Glen, Contractor, Code 7532
> > Subject: Re: [rt.rap.ucar.edu #90342] Ensemble ANOM_CORR Mean
> >
> > Glen,
> >
> > You're correct. The ANOM_CORR statistic is included in the CNT
> > output
> > line
> > type which is included in the output Point-Stat and Grid-Stat. I
see
> > that
> > you're running a 20-member ensemble. It sounds like you'd like to
> > see
> > the
> > mean of the ANOM_CORR value for those 20 ensemble members. There
are
> > a few
> > options here:
> >
> > (1) One of the outputs from ensemble_stat is a NetCDF file
containing
> > which you can configure to include the ensemble mean field. Just
> > turn
> > "mean" on in the "ensemble_flag" config file option:
> >
> >
> > *ensemble_flag = {... mean = TRUE;*
> > *...*
> > *}*
> > Then you can run the ensemble mean field through Grid-Stat or
Point-
> > Stat
> > and compute the ANOM_CORR statistic. This isn't exactly what you
> > asked for
> > but is similar.
> >
> > (2) A second option is computing the ANOM_CORR statistic for each
of
> > the 20
> > ensemble members and using stat_analysis to compute the mean of
those
> > 2
> > stats. You can do this using the "summary" job type or the
> > "aggregate_stat" job type.
> >
> > Run a stat_analysis "summary" job on the ANOM_CORR column of
output.
> > I
> > used some sample data to demonstrate:
> > *met-8.1/bin/stat_analysis -lookin
> > point_stat_GFS_CLIMO_1.0DEG_120000L_20120409_120000V.stat -job
> > summary
> > -line_type CNT -column ANOM_CORR -by FCST_VAR*
> >
> > This reads data from .stat file listed and dumps out summary info
for
> > the
> > ANOM_CORR column, but does so for each unique FCST_VAR value
> > encountered:
> > COL_NAME: LINE_TYPE COLUMN FCST_VAR TOTAL MEAN MEAN_NCL
> > MEAN_NCU
> > MEAN_BCL MEAN_BCU STDEV STDEV_BCL STDEV_BCU MIN P10
> > P25
> > P50 P75 P90 MAX IQR RANGE WMO_TYPE WMO_MEAN
> > WMO_WEIGHTED_MEAN
> > SUMMARY: CNT ANOM_CORR TMP 4 0.5472 0.090704
> > 1.0037
> > NA NA 0.28688 NA NA 0.40376 0.40376 0.40376
> > 0.40376
> > 0.5472 0.8054 0.97753 0.14344 0.57377 FISHER 0.70681
> > 0.41373
> > SUMMARY: CNT ANOM_CORR UGRD 1 0.71647 NA NA
> > NA NA NA NA NA 0.71647 0.71647
0.71647
> > 0.71647
> > 0.71647 0.71647 0.71647 0 0 FISHER 0.71647
> > 0.71647
> > SUMMARY: CNT ANOM_CORR VGRD 1 0.46991 NA NA
> > NA NA NA NA NA 0.46991 0.46991
0.46991
> > 0.46991
> > 0.46991 0.46991 0.46991 0 0 FISHER 0.46991
> > 0.46991
> > SUMMARY: CNT ANOM_CORR WIND 1 0.64622 NA NA
> > NA NA NA NA NA 0.64622 0.64622
0.64622
> > 0.64622
> > 0.64622 0.64622 0.64622 0 0 FISHER 0.64622
> > 0.64622
> >
> > Run a stat_analysis "aggregate_stat" job type to aggregate
together
> > SAL1L2
> > partial sums and recompute the CNT output line type:
> > *met-8.1/bin/stat_analysis -lookin
> > point_stat_GFS_CLIMO_1.0DEG_120000L_20120409_120000V.stat -job
> > aggregate_stat -line_type SAL1L2 -out_line_type CNT -by FCST_VAR
> > -out_stat
> > cnt.stat*
> >
> > That logic would recompute the CNT stats aggregated across all 20
of
> > your
> > members.
> >
> > (3) The third option, and what we'd probably use in the DTC, is to
> > just use
> > METviewer to apply the stat_analysis aggregations shown above as
> > we're
> > plotting the data.
> >
> > The ensemble-stat tool does not include any logic to compute stats
> > separately for each member and then report the mean of those stats
> > across
> > the members.
> >
> > Make sense?
> >
> > Thanks,
> > John Halley Gotway
> >
> >
> > On Fri, May 24, 2019 at 8:47 AM Carl, Mr. Glen, Contractor, Code
7532
> > via
> > RT <met_help at ucar.edu> wrote:
> >
> > >
> > > Fri May 24 08:47:25 2019: Request 90342 was acted upon.
> > > Transaction: Ticket created by glen.carl.ctr at nrlmry.navy.mil
> > > Queue: met_help
> > > Subject: Ensemble ANOM_CORR Mean
> > > Owner: Nobody
> > > Requestors: glen.carl.ctr at nrlmry.navy.mil
> > > Status: new
> > > Ticket <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90342
> > > >
> > >
> > >
> > > How can we out the Anomaly Correlation mean for an ensemble.
> > > ensemble_stat
> > > 20
> > > outputs the stats for the 20 member ensemble. Our problem is
that
> > > ensemble_stat does not have an output for ANOM_CORR, or if it
does
> > > we
> > > do
> > > not
> > > know how to output it. We have found we can output the Anomaly
> > > Correlation
> > > for grid_stat however only for each member.
> > >
> > > Thanks,
> > > Glen
> > >
> > > --
> > > Mr. Glen CARL
> > > NRL Monterey Contractor SAIC/0cog, Bldg 704 Rm 133, Code 7532
> > > 831.325.6269
> > > glen.carl.ctr at nrlmry.navy.mil
> > >
> > >
> > >
------------------------------------------------
Subject: Ensemble ANOM_CORR Mean
From: Carl, Mr. Glen, Contractor, Code 7532
Time: Wed May 29 08:39:57 2019
Hi Howard,
Thanks for the catch on
Changed the level's to level = ["(*,*)"];
I changed to
fcst = {
field = [
{
name = "geopht_pre_0500.0_pre_0500.0_ENS_MEAN";
level = ["(*,*)"];
cat_thresh = [ >5400., >=5500. ];
}
];
}
And was able to generate the AC in the expected file
grid_stat_1200000L_20161220_000000V_cnt.txt
I was focused on trying different settings of level, that when I
changed focus
to name, I forgot to correct the level. I was so close.
Anyways, thanks for the help!
You can close the issue.
Glen
-----Original Message-----
From: Howard Soh via RT [mailto:met_help at ucar.edu]
Sent: Wednesday, May 29, 2019 7:24 AM
To: Carl, Mr. Glen, Contractor, Code 7532
Subject: [rt.rap.ucar.edu #90342] Ensemble ANOM_CORR Mean
John is out of office this week.
I'm not an expert on grid_stat/ensemble_stat. I saw a syntax which MET
does
not support. Would you change the level configuration: "(,)" to
"(*,*)" ?
fcst = {
field = [
{
name = "geopht_pre_0500.0";
level = ["(,)"];
cat_thresh = [ >5400., >=5500. ];
}
];
}
==>
fcst = {
field = [
{
name = "geopht_pre_0500.0";
level = ["(*,*)"];
cat_thresh = [ >5400., >=5500. ];
}
];
}
Cheers,
Howard
On Tue May 28 16:55:22 2019, glen.carl.ctr at nrlmry.navy.mil wrote:
> I am having trouble with the read_data_plane() for processing the
> anomaly
> correlation for the ensemble per your option 1 suggestion. I dumped
> some
> debug, and config options that had different output. Is there
> something I need
> to check within the ensemble_stat....nc file, or in the
> EnsembleStatConfig.
> Thanks,
> Glen
> --
> Mr. Glen CARL
> NRL Monterey Contractor SAIC/0cog, Bldg 704 Rm 133, Code 7532
> 831.325.6269
> glen.carl.ctr at nrlmry.navy.mil
>
>
>
>
>
> I set the configs with
> ensemble_flag = {
> ...
> mean = TRUE;
> ..
>
> and run
> grid_stat -v 5 -outdir ANOM_CORR
> ensemble_stat_20161220_000000V_ens.nc
> verifecmwf_geopht_pre_0500_2016122000.nc EnsembleStatConfigACORR
>
>
> I have been trying to get my config to process the
ensemble_stat_ens.n
> and
> verify.nc
>
> here are the configs and results I am getting
>
> fcst = {
> field = [
> {
> name = "geopht_pre_0500.0";
> level = ["(,)"];
> cat_thresh = [ >5400., >=5500. ];
> }
> ];
> }
>
> WARNING:
> WARNING: read_data_plane() -> geopht_pre_0500.0(,) not found in
file:
> ensemble_stat_20161220_000000V_ens.nc
> WARNING:
>
> or when I set fcst field
> fcst = {
> field = [
> {
> // name = "geopht_pre_0500.0_pre_0500.0_ENS_MEAN";
> level = ["(,)"];
> cat_thresh = [ >5400., >=5500. ];
> }
> ];
> }
>
>
> VarInfoNcMet::dump():
> Dimension:
> Nelements = 0
> Nalloc = 0
> AllocInc = 10
> DEBUG 5: Parsed observation field:
> VarInfo::dump():
> MagicStr = geopht_pre_0500.0(,)
> ReqName = geopht_pre_0500.0
> Name = geopht_pre_0500.0
> LongName = geopht_pre_0500.0(,)
> Units = NA
> PFlag = 0
> PName = (nul)
> PUnits = (nul)
> PAsScalar = 0
> UVIndex = -1
> Init = 19700101_000000 (0)
> Valid = 19700101_000000 (0)
> Ensemble = (nul)
> Lead = NA (-9999)
> ConvertFx = (nul)
> CensorThresh = NA
> CensorVal =
> Regrid = NEAREST
> LevelInfo::dump():
> Type = LevelType_None
> TypeNum = -9999
> ReqName = ,
> Name = ,
> Units = (nul)
> Upper = 0
> Increment = 0
> VarInfoNcMet::dump():
> Dimension:
> Nelements = 0
> Nalloc = 0
> AllocInc = 10
> DEBUG 5: Parsed thresholds:
> DEBUG 5: Forecast categorical thresholds: >5400.,>=5500.
> DEBUG 5: Observed categorical thresholds: >5400.,>=5500.
> DEBUG 5: Forecast continuous thresholds: NA
> DEBUG 5: Observed continuous thresholds: NA
> DEBUG 5: Continuous threshold logic: UNION
> DEBUG 5: Forecast wind speed thresholds: NA
> DEBUG 5: Observed wind speed thresholds: NA
> DEBUG 5: Wind speed threshold logic: UNION
> DEBUG 3: Use the grid named "G003".
> DEBUG 3: Grid Definition: Projection: Lat/Lon Nx: 360 Ny: 181
lat_ll:
> -90.000
> lon_ll: -0.000 delta_lat: 1.000 delta_lon: 1.000
> GSL_RNG_TYPE=mt19937
> GSL_RNG_SEED=549055836
> DEBUG 1: Forecast File: ensemble_stat_20161220_000000V_ens.nc
> DEBUG 1: Observation File: verifecmwf_geopht_pre_0500_2016122000.nc
> ERROR :
> ERROR : MetNcFile::data(NcVar *, const LongArray &, DataPlane &)
->
> needed 2
> arguments for variable geopht_pre_0500.0_pre_0500.0_ENS_MEAN, got 0
> ERROR :
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Friday, May 24, 2019 10:50 AM
> To: Carl, Mr. Glen, Contractor, Code 7532
> Subject: Re: [rt.rap.ucar.edu #90342] Ensemble ANOM_CORR Mean
>
> Glen,
>
> You're correct. The ANOM_CORR statistic is included in the CNT
output
> line
> type which is included in the output Point-Stat and Grid-Stat. I
see
> that
> you're running a 20-member ensemble. It sounds like you'd like to
see
> the
> mean of the ANOM_CORR value for those 20 ensemble members. There
are
> a few
> options here:
>
> (1) One of the outputs from ensemble_stat is a NetCDF file
containing
> which you can configure to include the ensemble mean field. Just
turn
> "mean" on in the "ensemble_flag" config file option:
>
>
> *ensemble_flag = {... mean = TRUE;*
> *...*
> *}*
> Then you can run the ensemble mean field through Grid-Stat or Point-
> Stat
> and compute the ANOM_CORR statistic. This isn't exactly what you
> asked for
> but is similar.
>
> (2) A second option is computing the ANOM_CORR statistic for each of
> the 20
> ensemble members and using stat_analysis to compute the mean of
those
> 2
> stats. You can do this using the "summary" job type or the
> "aggregate_stat" job type.
>
> Run a stat_analysis "summary" job on the ANOM_CORR column of output.
> I
> used some sample data to demonstrate:
> *met-8.1/bin/stat_analysis -lookin
> point_stat_GFS_CLIMO_1.0DEG_120000L_20120409_120000V.stat -job
summary
> -line_type CNT -column ANOM_CORR -by FCST_VAR*
>
> This reads data from .stat file listed and dumps out summary info
for
> the
> ANOM_CORR column, but does so for each unique FCST_VAR value
> encountered:
> COL_NAME: LINE_TYPE COLUMN FCST_VAR TOTAL MEAN MEAN_NCL
> MEAN_NCU
> MEAN_BCL MEAN_BCU STDEV STDEV_BCL STDEV_BCU MIN P10
P25
> P50 P75 P90 MAX IQR RANGE WMO_TYPE WMO_MEAN
> WMO_WEIGHTED_MEAN
> SUMMARY: CNT ANOM_CORR TMP 4 0.5472 0.090704
> 1.0037
> NA NA 0.28688 NA NA 0.40376 0.40376 0.40376
> 0.40376
> 0.5472 0.8054 0.97753 0.14344 0.57377 FISHER 0.70681
> 0.41373
> SUMMARY: CNT ANOM_CORR UGRD 1 0.71647 NA NA
> NA NA NA NA NA 0.71647 0.71647 0.71647
> 0.71647
> 0.71647 0.71647 0.71647 0 0 FISHER 0.71647
> 0.71647
> SUMMARY: CNT ANOM_CORR VGRD 1 0.46991 NA NA
> NA NA NA NA NA 0.46991 0.46991 0.46991
> 0.46991
> 0.46991 0.46991 0.46991 0 0 FISHER 0.46991
> 0.46991
> SUMMARY: CNT ANOM_CORR WIND 1 0.64622 NA NA
> NA NA NA NA NA 0.64622 0.64622 0.64622
> 0.64622
> 0.64622 0.64622 0.64622 0 0 FISHER 0.64622
> 0.64622
>
> Run a stat_analysis "aggregate_stat" job type to aggregate together
> SAL1L2
> partial sums and recompute the CNT output line type:
> *met-8.1/bin/stat_analysis -lookin
> point_stat_GFS_CLIMO_1.0DEG_120000L_20120409_120000V.stat -job
> aggregate_stat -line_type SAL1L2 -out_line_type CNT -by FCST_VAR
> -out_stat
> cnt.stat*
>
> That logic would recompute the CNT stats aggregated across all 20 of
> your
> members.
>
> (3) The third option, and what we'd probably use in the DTC, is to
> just use
> METviewer to apply the stat_analysis aggregations shown above as
we're
> plotting the data.
>
> The ensemble-stat tool does not include any logic to compute stats
> separately for each member and then report the mean of those stats
> across
> the members.
>
> Make sense?
>
> Thanks,
> John Halley Gotway
>
>
> On Fri, May 24, 2019 at 8:47 AM Carl, Mr. Glen, Contractor, Code
7532
> via
> RT <met_help at ucar.edu> wrote:
>
> >
> > Fri May 24 08:47:25 2019: Request 90342 was acted upon.
> > Transaction: Ticket created by glen.carl.ctr at nrlmry.navy.mil
> > Queue: met_help
> > Subject: Ensemble ANOM_CORR Mean
> > Owner: Nobody
> > Requestors: glen.carl.ctr at nrlmry.navy.mil
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90342
> > >
> >
> >
> > How can we out the Anomaly Correlation mean for an ensemble.
> > ensemble_stat
> > 20
> > outputs the stats for the 20 member ensemble. Our problem is that
> > ensemble_stat does not have an output for ANOM_CORR, or if it does
we
> > do
> > not
> > know how to output it. We have found we can output the Anomaly
> > Correlation
> > for grid_stat however only for each member.
> >
> > Thanks,
> > Glen
> >
> > --
> > Mr. Glen CARL
> > NRL Monterey Contractor SAIC/0cog, Bldg 704 Rm 133, Code 7532
> > 831.325.6269
> > glen.carl.ctr at nrlmry.navy.mil
> >
> >
> >
------------------------------------------------
Subject: Ensemble ANOM_CORR Mean
From: Howard Soh
Time: Wed May 29 08:46:42 2019
I'm glad to hear that.
I will close the ticket.
Cheers,
Howard
On Wed May 29 08:39:57 2019, glen.carl.ctr at nrlmry.navy.mil wrote:
> Hi Howard,
> Thanks for the catch on
> Changed the level's to level = ["(*,*)"];
>
> I changed to
> fcst = {
> field = [
> {
> name = "geopht_pre_0500.0_pre_0500.0_ENS_MEAN";
> level = ["(*,*)"];
> cat_thresh = [ >5400., >=5500. ];
> }
> ];
> }
>
> And was able to generate the AC in the expected file
> grid_stat_1200000L_20161220_000000V_cnt.txt
>
> I was focused on trying different settings of level, that when I
> changed focus
> to name, I forgot to correct the level. I was so close.
>
> Anyways, thanks for the help!
>
> You can close the issue.
> Glen
>
>
> -----Original Message-----
> From: Howard Soh via RT [mailto:met_help at ucar.edu]
> Sent: Wednesday, May 29, 2019 7:24 AM
> To: Carl, Mr. Glen, Contractor, Code 7532
> Subject: [rt.rap.ucar.edu #90342] Ensemble ANOM_CORR Mean
>
> John is out of office this week.
>
> I'm not an expert on grid_stat/ensemble_stat. I saw a syntax which
MET
> does
> not support. Would you change the level configuration: "(,)" to
> "(*,*)" ?
>
> fcst = {
> field = [
> {
> name = "geopht_pre_0500.0";
> level = ["(,)"];
> cat_thresh = [ >5400., >=5500. ];
> }
> ];
> }
>
> ==>
>
> fcst = {
> field = [
> {
> name = "geopht_pre_0500.0";
> level = ["(*,*)"];
> cat_thresh = [ >5400., >=5500. ];
> }
> ];
> }
>
> Cheers,
> Howard
>
> On Tue May 28 16:55:22 2019, glen.carl.ctr at nrlmry.navy.mil wrote:
> > I am having trouble with the read_data_plane() for processing the
> > anomaly
> > correlation for the ensemble per your option 1 suggestion. I
dumped
> > some
> > debug, and config options that had different output. Is there
> > something I need
> > to check within the ensemble_stat....nc file, or in the
> > EnsembleStatConfig.
> > Thanks,
> > Glen
> > --
> > Mr. Glen CARL
> > NRL Monterey Contractor SAIC/0cog, Bldg 704 Rm 133, Code 7532
> > 831.325.6269
> > glen.carl.ctr at nrlmry.navy.mil
> >
> >
> >
> >
> >
> > I set the configs with
> > ensemble_flag = {
> > ...
> > mean = TRUE;
> > ..
> >
> > and run
> > grid_stat -v 5 -outdir ANOM_CORR
> > ensemble_stat_20161220_000000V_ens.nc
> > verifecmwf_geopht_pre_0500_2016122000.nc EnsembleStatConfigACORR
> >
> >
> > I have been trying to get my config to process the
> > ensemble_stat_ens.n
> > and
> > verify.nc
> >
> > here are the configs and results I am getting
> >
> > fcst = {
> > field = [
> > {
> > name = "geopht_pre_0500.0";
> > level = ["(,)"];
> > cat_thresh = [ >5400., >=5500. ];
> > }
> > ];
> > }
> >
> > WARNING:
> > WARNING: read_data_plane() -> geopht_pre_0500.0(,) not found in
> > file:
> > ensemble_stat_20161220_000000V_ens.nc
> > WARNING:
> >
> > or when I set fcst field
> > fcst = {
> > field = [
> > {
> > // name = "geopht_pre_0500.0_pre_0500.0_ENS_MEAN";
> > level = ["(,)"];
> > cat_thresh = [ >5400., >=5500. ];
> > }
> > ];
> > }
> >
> >
> > VarInfoNcMet::dump():
> > Dimension:
> > Nelements = 0
> > Nalloc = 0
> > AllocInc = 10
> > DEBUG 5: Parsed observation field:
> > VarInfo::dump():
> > MagicStr = geopht_pre_0500.0(,)
> > ReqName = geopht_pre_0500.0
> > Name = geopht_pre_0500.0
> > LongName = geopht_pre_0500.0(,)
> > Units = NA
> > PFlag = 0
> > PName = (nul)
> > PUnits = (nul)
> > PAsScalar = 0
> > UVIndex = -1
> > Init = 19700101_000000 (0)
> > Valid = 19700101_000000 (0)
> > Ensemble = (nul)
> > Lead = NA (-9999)
> > ConvertFx = (nul)
> > CensorThresh = NA
> > CensorVal =
> > Regrid = NEAREST
> > LevelInfo::dump():
> > Type = LevelType_None
> > TypeNum = -9999
> > ReqName = ,
> > Name = ,
> > Units = (nul)
> > Upper = 0
> > Increment = 0
> > VarInfoNcMet::dump():
> > Dimension:
> > Nelements = 0
> > Nalloc = 0
> > AllocInc = 10
> > DEBUG 5: Parsed thresholds:
> > DEBUG 5: Forecast categorical thresholds: >5400.,>=5500.
> > DEBUG 5: Observed categorical thresholds: >5400.,>=5500.
> > DEBUG 5: Forecast continuous thresholds: NA
> > DEBUG 5: Observed continuous thresholds: NA
> > DEBUG 5: Continuous threshold logic: UNION
> > DEBUG 5: Forecast wind speed thresholds: NA
> > DEBUG 5: Observed wind speed thresholds: NA
> > DEBUG 5: Wind speed threshold logic: UNION
> > DEBUG 3: Use the grid named "G003".
> > DEBUG 3: Grid Definition: Projection: Lat/Lon Nx: 360 Ny: 181
> > lat_ll:
> > -90.000
> > lon_ll: -0.000 delta_lat: 1.000 delta_lon: 1.000
> > GSL_RNG_TYPE=mt19937
> > GSL_RNG_SEED=549055836
> > DEBUG 1: Forecast File: ensemble_stat_20161220_000000V_ens.nc
> > DEBUG 1: Observation File:
verifecmwf_geopht_pre_0500_2016122000.nc
> > ERROR :
> > ERROR : MetNcFile::data(NcVar *, const LongArray &, DataPlane &)
->
> > needed 2
> > arguments for variable geopht_pre_0500.0_pre_0500.0_ENS_MEAN, got
0
> > ERROR :
> >
> > -----Original Message-----
> > From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> > Sent: Friday, May 24, 2019 10:50 AM
> > To: Carl, Mr. Glen, Contractor, Code 7532
> > Subject: Re: [rt.rap.ucar.edu #90342] Ensemble ANOM_CORR Mean
> >
> > Glen,
> >
> > You're correct. The ANOM_CORR statistic is included in the CNT
> > output
> > line
> > type which is included in the output Point-Stat and Grid-Stat. I
see
> > that
> > you're running a 20-member ensemble. It sounds like you'd like to
> > see
> > the
> > mean of the ANOM_CORR value for those 20 ensemble members. There
are
> > a few
> > options here:
> >
> > (1) One of the outputs from ensemble_stat is a NetCDF file
containing
> > which you can configure to include the ensemble mean field. Just
> > turn
> > "mean" on in the "ensemble_flag" config file option:
> >
> >
> > *ensemble_flag = {... mean = TRUE;*
> > *...*
> > *}*
> > Then you can run the ensemble mean field through Grid-Stat or
Point-
> > Stat
> > and compute the ANOM_CORR statistic. This isn't exactly what you
> > asked for
> > but is similar.
> >
> > (2) A second option is computing the ANOM_CORR statistic for each
of
> > the 20
> > ensemble members and using stat_analysis to compute the mean of
those
> > 2
> > stats. You can do this using the "summary" job type or the
> > "aggregate_stat" job type.
> >
> > Run a stat_analysis "summary" job on the ANOM_CORR column of
output.
> > I
> > used some sample data to demonstrate:
> > *met-8.1/bin/stat_analysis -lookin
> > point_stat_GFS_CLIMO_1.0DEG_120000L_20120409_120000V.stat -job
> > summary
> > -line_type CNT -column ANOM_CORR -by FCST_VAR*
> >
> > This reads data from .stat file listed and dumps out summary info
for
> > the
> > ANOM_CORR column, but does so for each unique FCST_VAR value
> > encountered:
> > COL_NAME: LINE_TYPE COLUMN FCST_VAR TOTAL MEAN MEAN_NCL
> > MEAN_NCU
> > MEAN_BCL MEAN_BCU STDEV STDEV_BCL STDEV_BCU MIN P10
> > P25
> > P50 P75 P90 MAX IQR RANGE WMO_TYPE WMO_MEAN
> > WMO_WEIGHTED_MEAN
> > SUMMARY: CNT ANOM_CORR TMP 4 0.5472 0.090704
> > 1.0037
> > NA NA 0.28688 NA NA 0.40376 0.40376 0.40376
> > 0.40376
> > 0.5472 0.8054 0.97753 0.14344 0.57377 FISHER 0.70681
> > 0.41373
> > SUMMARY: CNT ANOM_CORR UGRD 1 0.71647 NA NA
> > NA NA NA NA NA 0.71647 0.71647
0.71647
> > 0.71647
> > 0.71647 0.71647 0.71647 0 0 FISHER 0.71647
> > 0.71647
> > SUMMARY: CNT ANOM_CORR VGRD 1 0.46991 NA NA
> > NA NA NA NA NA 0.46991 0.46991
0.46991
> > 0.46991
> > 0.46991 0.46991 0.46991 0 0 FISHER 0.46991
> > 0.46991
> > SUMMARY: CNT ANOM_CORR WIND 1 0.64622 NA NA
> > NA NA NA NA NA 0.64622 0.64622
0.64622
> > 0.64622
> > 0.64622 0.64622 0.64622 0 0 FISHER 0.64622
> > 0.64622
> >
> > Run a stat_analysis "aggregate_stat" job type to aggregate
together
> > SAL1L2
> > partial sums and recompute the CNT output line type:
> > *met-8.1/bin/stat_analysis -lookin
> > point_stat_GFS_CLIMO_1.0DEG_120000L_20120409_120000V.stat -job
> > aggregate_stat -line_type SAL1L2 -out_line_type CNT -by FCST_VAR
> > -out_stat
> > cnt.stat*
> >
> > That logic would recompute the CNT stats aggregated across all 20
of
> > your
> > members.
> >
> > (3) The third option, and what we'd probably use in the DTC, is to
> > just use
> > METviewer to apply the stat_analysis aggregations shown above as
> > we're
> > plotting the data.
> >
> > The ensemble-stat tool does not include any logic to compute stats
> > separately for each member and then report the mean of those stats
> > across
> > the members.
> >
> > Make sense?
> >
> > Thanks,
> > John Halley Gotway
> >
> >
> > On Fri, May 24, 2019 at 8:47 AM Carl, Mr. Glen, Contractor, Code
7532
> > via
> > RT <met_help at ucar.edu> wrote:
> >
> > >
> > > Fri May 24 08:47:25 2019: Request 90342 was acted upon.
> > > Transaction: Ticket created by glen.carl.ctr at nrlmry.navy.mil
> > > Queue: met_help
> > > Subject: Ensemble ANOM_CORR Mean
> > > Owner: Nobody
> > > Requestors: glen.carl.ctr at nrlmry.navy.mil
> > > Status: new
> > > Ticket <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90342
> > > >
> > >
> > >
> > > How can we out the Anomaly Correlation mean for an ensemble.
> > > ensemble_stat
> > > 20
> > > outputs the stats for the 20 member ensemble. Our problem is
that
> > > ensemble_stat does not have an output for ANOM_CORR, or if it
does
> > > we
> > > do
> > > not
> > > know how to output it. We have found we can output the Anomaly
> > > Correlation
> > > for grid_stat however only for each member.
> > >
> > > Thanks,
> > > Glen
> > >
> > > --
> > > Mr. Glen CARL
> > > NRL Monterey Contractor SAIC/0cog, Bldg 704 Rm 133, Code 7532
> > > 831.325.6269
> > > glen.carl.ctr at nrlmry.navy.mil
> > >
> > >
> > >
------------------------------------------------
More information about the Met_help
mailing list