[Met_help] [rt.rap.ucar.edu #82033] History for question about MET V4.1 Ensemble-Stat (UNCLASSIFIED)
John Halley Gotway via RT
met_help at ucar.edu
Thu Nov 9 11:22:13 MST 2017
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
CLASSIFICATION: UNCLASSIFIED
I ran with and without a masking region and compared the output. I noted
differences in the statistics given in the ascii files which were as
expected, but not in the NETCDF output of the various fields such as mean,
stdev, range, min, max. It seems that the graphics show the same area with
or without the masking region. I had a case where the forecast model output
had some strange edge effects which were confined to only the outer 2-3 grid
points along the boundaries of the domain and I wanted to see the graphics
generated without including the edge effects. Is there a way to generate the
graphics for the reduced area of the masking region?
Thanks.
R/
John
Mr John W. Raby, Meteorologist
U.S. Army Research Laboratory
White Sands Missile Range, NM 88002
(575) 678-2004 DSN 258-2004
FAX (575) 678-1230 DSN 258-1230
Email: john.w.raby2.civ at mail.mil
CLASSIFICATION: UNCLASSIFIED
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: question about MET V4.1 Ensemble-Stat (UNCLASSIFIED)
From: John Halley Gotway
Time: Tue Sep 19 10:21:14 2017
John,
Based on the list of NetCDF variables you mentioned, I assume you're
talking about running the Ensemble-Stat tool.
The Ensemble-Stat tool really does 2 different types of processing:
(1) Process the fields defined in the "ens" config file section and
derive
fields like min, max, mean, stdev, and so on. This isn't
verification...
it's just very simplistic ensemble post-processing.
(2) Process the fields defined in the "fcst" and "obs" config file
sections
and do ensemble verification by computing things like ranked
histograms
(RHIST), PIT histograms (PHIST), and so on.
The masking regions defined in the "mask" config file section are only
applied to the logic of step (2), not step (1).
However, I understand that you'd like to apply a masking region to the
NetCDF output of step (1).
There is no way to do this using the Ensemble-Stat tool. The logic
just
wasn't defined to support that.
However, there are 2 ways you could do it.
First, you could run grid_stat to process the output of Ensemble-Stat
and
write a NetCDF file. That NetCDF file *will* have the masking regions
applied in the output.
The second option is by running the gen_vx_mask tool in a tricky way.
I'll
list an example for met-6.0 using the NetCDF output of the
ensemble_stat
tool generated when you run "make test":
met-6.0/bin/gen_vx_mask \
met-6.0/out/ensemble_stat/ensemble_stat_20100101_120000V_ens.nc \
met-6.0/data/poly/NWC.poly ens_mean_NWC.nc \
-input_field 'name="REFC_L0_ENS_MEAN"; level="(*,*)"; \
-complement -value -9999
This is a little complicated, but here's what's going on:
(1) Get the grid information from the ensemble stat NetCDF file
listed.
(2) Read the lat/lon polyline from the file specified.
(3) Write output to the third file listed.
(4) Rather than initializing the grid with 0's, instead read data
values
for the field named "REFC_L0_ENS_MEAN" from the input file.
(5) Rather than processing the inside of the polyline, process the
outside
using -complement.
(6) Rather than writing the default mask value of 1, write a bad data
value
of -9999.
All these steps combine to read a gridded data field and set
everything
outside the polyline region to a bad data value.
Hope that helps.
John
On Mon, Sep 18, 2017 at 11:04 AM, Raby, John W USA CIV via RT <
met_help at ucar.edu> wrote:
>
> Mon Sep 18 11:04:07 2017: Request 82033 was acted upon.
> Transaction: Ticket created by john.w.raby2.civ at mail.mil
> Queue: met_help
> Subject: question about MET V4.1 Ensemble-Stat (UNCLASSIFIED)
> Owner: Nobody
> Requestors: john.w.raby2.civ at mail.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=82033 >
>
>
> CLASSIFICATION: UNCLASSIFIED
>
> I ran with and without a masking region and compared the output. I
noted
> differences in the statistics given in the ascii files which were as
> expected, but not in the NETCDF output of the various fields such as
mean,
> stdev, range, min, max. It seems that the graphics show the same
area with
> or without the masking region. I had a case where the forecast model
output
> had some strange edge effects which were confined to only the outer
2-3
> grid
> points along the boundaries of the domain and I wanted to see the
graphics
> generated without including the edge effects. Is there a way to
generate
> the
> graphics for the reduced area of the masking region?
>
> Thanks.
>
> R/
> John
>
> Mr John W. Raby, Meteorologist
> U.S. Army Research Laboratory
> White Sands Missile Range, NM 88002
> (575) 678-2004 DSN 258-2004
> FAX (575) 678-1230 DSN 258-1230
> Email: john.w.raby2.civ at mail.mil
>
>
>
> CLASSIFICATION: UNCLASSIFIED
>
>
------------------------------------------------
Subject: question about MET V4.1 Ensemble-Stat (UNCLASSIFIED)
From: Raby, John W USA CIV
Time: Tue Sep 19 12:11:15 2017
CLASSIFICATION: UNCLASSIFIED
John -
Thanks.
This confirms that what I am seeing is expected behavior, which helps.
Very
resourceful workarounds. So to run grid-stat, would I use the
Ensemble-Stat
NetCDF output fields as the input?
R/
John
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Tuesday, September 19, 2017 10:21 AM
To: Raby, John W CIV USARMY RDECOM ARL (US)
<john.w.raby2.civ at mail.mil>
Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #82033] question about
MET V4.1
Ensemble-Stat (UNCLASSIFIED)
All active links contained in this email were disabled. Please verify
the
identity of the sender, and confirm the authenticity of all links
contained
within the message prior to copying and pasting the address to a Web
browser.
----
John,
Based on the list of NetCDF variables you mentioned, I assume you're
talking
about running the Ensemble-Stat tool.
The Ensemble-Stat tool really does 2 different types of processing:
(1) Process the fields defined in the "ens" config file section and
derive
fields like min, max, mean, stdev, and so on. This isn't
verification...
it's just very simplistic ensemble post-processing.
(2) Process the fields defined in the "fcst" and "obs" config file
sections
and do ensemble verification by computing things like ranked
histograms
(RHIST), PIT histograms (PHIST), and so on.
The masking regions defined in the "mask" config file section are only
applied
to the logic of step (2), not step (1).
However, I understand that you'd like to apply a masking region to the
NetCDF
output of step (1).
There is no way to do this using the Ensemble-Stat tool. The logic
just
wasn't defined to support that.
However, there are 2 ways you could do it.
First, you could run grid_stat to process the output of Ensemble-Stat
and
write a NetCDF file. That NetCDF file *will* have the masking regions
applied
in the output.
The second option is by running the gen_vx_mask tool in a tricky way.
I'll
list an example for met-6.0 using the NetCDF output of the
ensemble_stat tool
generated when you run "make test":
met-6.0/bin/gen_vx_mask \
met-6.0/out/ensemble_stat/ensemble_stat_20100101_120000V_ens.nc \
met-6.0/data/poly/NWC.poly ens_mean_NWC.nc \
-input_field 'name="REFC_L0_ENS_MEAN"; level="(*,*)"; \
-complement -value -9999
This is a little complicated, but here's what's going on:
(1) Get the grid information from the ensemble stat NetCDF file
listed.
(2) Read the lat/lon polyline from the file specified.
(3) Write output to the third file listed.
(4) Rather than initializing the grid with 0's, instead read data
values for
the field named "REFC_L0_ENS_MEAN" from the input file.
(5) Rather than processing the inside of the polyline, process the
outside
using -complement.
(6) Rather than writing the default mask value of 1, write a bad data
value
of -9999.
All these steps combine to read a gridded data field and set
everything
outside the polyline region to a bad data value.
Hope that helps.
John
On Mon, Sep 18, 2017 at 11:04 AM, Raby, John W USA CIV via RT <
met_help at ucar.edu> wrote:
>
> Mon Sep 18 11:04:07 2017: Request 82033 was acted upon.
> Transaction: Ticket created by john.w.raby2.civ at mail.mil
> Queue: met_help
> Subject: question about MET V4.1 Ensemble-Stat (UNCLASSIFIED)
> Owner: Nobody
> Requestors: john.w.raby2.civ at mail.mil
> Status: new
> Ticket <Caution-url:
> Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=82033 >
>
>
> CLASSIFICATION: UNCLASSIFIED
>
> I ran with and without a masking region and compared the output. I
> noted differences in the statistics given in the ascii files which
> were as expected, but not in the NETCDF output of the various fields
> such as mean, stdev, range, min, max. It seems that the graphics
show
> the same area with or without the masking region. I had a case where
> the forecast model output had some strange edge effects which were
> confined to only the outer 2-3 grid points along the boundaries of
the
> domain and I wanted to see the graphics generated without including
> the edge effects. Is there a way to generate the graphics for the
> reduced area of the masking region?
>
> Thanks.
>
> R/
> John
>
> Mr John W. Raby, Meteorologist
> U.S. Army Research Laboratory
> White Sands Missile Range, NM 88002
> (575) 678-2004 DSN 258-2004
> FAX (575) 678-1230 DSN 258-1230
> Email: john.w.raby2.civ at mail.mil
>
>
>
> CLASSIFICATION: UNCLASSIFIED
>
>
CLASSIFICATION: UNCLASSIFIED
------------------------------------------------
Subject: question about MET V4.1 Ensemble-Stat (UNCLASSIFIED)
From: John Halley Gotway
Time: Tue Sep 19 12:16:11 2017
John,
Yes, you'd pass the NetCDF output of Ensemble-Stat as input to Grid-
Stat.
If you don't really care about the stats and all you really want is
the
masked out NetCDF, just pass the same Ensemble-Stat NetCDF file as
both the
forecast and observation files to Grid-Stat.
John
On Tue, Sep 19, 2017 at 12:11 PM, Raby, John W USA CIV via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=82033 >
>
> CLASSIFICATION: UNCLASSIFIED
>
> John -
>
> Thanks.
>
> This confirms that what I am seeing is expected behavior, which
helps. Very
> resourceful workarounds. So to run grid-stat, would I use the
> Ensemble-Stat
> NetCDF output fields as the input?
>
> R/
> John
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Tuesday, September 19, 2017 10:21 AM
> To: Raby, John W CIV USARMY RDECOM ARL (US)
<john.w.raby2.civ at mail.mil>
> Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #82033] question
about MET
> V4.1
> Ensemble-Stat (UNCLASSIFIED)
>
> All active links contained in this email were disabled. Please
verify the
> identity of the sender, and confirm the authenticity of all links
contained
> within the message prior to copying and pasting the address to a Web
> browser.
>
>
>
>
> ----
>
> John,
>
> Based on the list of NetCDF variables you mentioned, I assume you're
> talking
> about running the Ensemble-Stat tool.
>
> The Ensemble-Stat tool really does 2 different types of processing:
>
> (1) Process the fields defined in the "ens" config file section and
derive
> fields like min, max, mean, stdev, and so on. This isn't
verification...
> it's just very simplistic ensemble post-processing.
>
> (2) Process the fields defined in the "fcst" and "obs" config file
sections
> and do ensemble verification by computing things like ranked
histograms
> (RHIST), PIT histograms (PHIST), and so on.
>
> The masking regions defined in the "mask" config file section are
only
> applied
> to the logic of step (2), not step (1).
>
> However, I understand that you'd like to apply a masking region to
the
> NetCDF
> output of step (1).
>
> There is no way to do this using the Ensemble-Stat tool. The logic
just
> wasn't defined to support that.
>
> However, there are 2 ways you could do it.
>
> First, you could run grid_stat to process the output of Ensemble-
Stat and
> write a NetCDF file. That NetCDF file *will* have the masking
regions
> applied
> in the output.
>
> The second option is by running the gen_vx_mask tool in a tricky
way. I'll
> list an example for met-6.0 using the NetCDF output of the
ensemble_stat
> tool
> generated when you run "make test":
>
> met-6.0/bin/gen_vx_mask \
>
> met-6.0/out/ensemble_stat/ensemble_stat_20100101_120000V_ens.nc \
>
> met-6.0/data/poly/NWC.poly ens_mean_NWC.nc \
>
> -input_field 'name="REFC_L0_ENS_MEAN"; level="(*,*)"; \
>
> -complement -value -9999
>
>
> This is a little complicated, but here's what's going on:
> (1) Get the grid information from the ensemble stat NetCDF file
listed.
> (2) Read the lat/lon polyline from the file specified.
> (3) Write output to the third file listed.
> (4) Rather than initializing the grid with 0's, instead read data
values
> for
> the field named "REFC_L0_ENS_MEAN" from the input file.
> (5) Rather than processing the inside of the polyline, process the
outside
> using -complement.
> (6) Rather than writing the default mask value of 1, write a bad
data value
> of -9999.
>
> All these steps combine to read a gridded data field and set
everything
> outside the polyline region to a bad data value.
>
> Hope that helps.
>
> John
>
> On Mon, Sep 18, 2017 at 11:04 AM, Raby, John W USA CIV via RT <
> met_help at ucar.edu> wrote:
>
> >
> > Mon Sep 18 11:04:07 2017: Request 82033 was acted upon.
> > Transaction: Ticket created by john.w.raby2.civ at mail.mil
> > Queue: met_help
> > Subject: question about MET V4.1 Ensemble-Stat (UNCLASSIFIED)
> > Owner: Nobody
> > Requestors: john.w.raby2.civ at mail.mil
> > Status: new
> > Ticket <Caution-url:
> > Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=82033 >
> >
> >
> > CLASSIFICATION: UNCLASSIFIED
> >
> > I ran with and without a masking region and compared the output. I
> > noted differences in the statistics given in the ascii files which
> > were as expected, but not in the NETCDF output of the various
fields
> > such as mean, stdev, range, min, max. It seems that the graphics
show
> > the same area with or without the masking region. I had a case
where
> > the forecast model output had some strange edge effects which were
> > confined to only the outer 2-3 grid points along the boundaries of
the
> > domain and I wanted to see the graphics generated without
including
> > the edge effects. Is there a way to generate the graphics for the
> > reduced area of the masking region?
> >
> > Thanks.
> >
> > R/
> > John
> >
> > Mr John W. Raby, Meteorologist
> > U.S. Army Research Laboratory
> > White Sands Missile Range, NM 88002
> > (575) 678-2004 DSN 258-2004
> > FAX (575) 678-1230 DSN 258-1230
> > Email: john.w.raby2.civ at mail.mil
> >
> >
> >
> > CLASSIFICATION: UNCLASSIFIED
> >
> >
>
>
> CLASSIFICATION: UNCLASSIFIED
>
>
------------------------------------------------
Subject: question about MET V4.1 Ensemble-Stat (UNCLASSIFIED)
From: Raby, John W USA CIV
Time: Tue Sep 19 12:22:50 2017
CLASSIFICATION: UNCLASSIFIED
John -
Thanks for the quick follow-up. Interesting, I had not thought about
pre-processing input to grid-stat using ensemble-stat. Hmmm. now, let
me think
about the output stats from grid-stat and what they would mean.
R/
John
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Tuesday, September 19, 2017 12:16 PM
To: Raby, John W CIV USARMY RDECOM ARL (US)
<john.w.raby2.civ at mail.mil>
Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #82033] question
about MET
V4.1 Ensemble-Stat (UNCLASSIFIED)
All active links contained in this email were disabled. Please verify
the
identity of the sender, and confirm the authenticity of all links
contained
within the message prior to copying and pasting the address to a Web
browser.
----
John,
Yes, you'd pass the NetCDF output of Ensemble-Stat as input to Grid-
Stat.
If you don't really care about the stats and all you really want is
the masked
out NetCDF, just pass the same Ensemble-Stat NetCDF file as both the
forecast
and observation files to Grid-Stat.
John
On Tue, Sep 19, 2017 at 12:11 PM, Raby, John W USA CIV via RT <
met_help at ucar.edu> wrote:
>
> <Caution-url:
> Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=82033 >
>
> CLASSIFICATION: UNCLASSIFIED
>
> John -
>
> Thanks.
>
> This confirms that what I am seeing is expected behavior, which
helps.
> Very resourceful workarounds. So to run grid-stat, would I use the
> Ensemble-Stat NetCDF output fields as the input?
>
> R/
> John
> -----Original Message-----
> From: John Halley Gotway via RT [Caution-mailto:met_help at ucar.edu]
> Sent: Tuesday, September 19, 2017 10:21 AM
> To: Raby, John W CIV USARMY RDECOM ARL (US)
> <john.w.raby2.civ at mail.mil>
> Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #82033] question
about
> MET
> V4.1
> Ensemble-Stat (UNCLASSIFIED)
>
> All active links contained in this email were disabled. Please
verify
> the identity of the sender, and confirm the authenticity of all
links
> contained within the message prior to copying and pasting the
address
> to a Web browser.
>
>
>
>
> ----
>
> John,
>
> Based on the list of NetCDF variables you mentioned, I assume you're
> talking about running the Ensemble-Stat tool.
>
> The Ensemble-Stat tool really does 2 different types of processing:
>
> (1) Process the fields defined in the "ens" config file section and
> derive fields like min, max, mean, stdev, and so on. This isn't
> verification...
> it's just very simplistic ensemble post-processing.
>
> (2) Process the fields defined in the "fcst" and "obs" config file
> sections and do ensemble verification by computing things like
ranked
> histograms (RHIST), PIT histograms (PHIST), and so on.
>
> The masking regions defined in the "mask" config file section are
only
> applied to the logic of step (2), not step (1).
>
> However, I understand that you'd like to apply a masking region to
the
> NetCDF output of step (1).
>
> There is no way to do this using the Ensemble-Stat tool. The logic
> just wasn't defined to support that.
>
> However, there are 2 ways you could do it.
>
> First, you could run grid_stat to process the output of Ensemble-
Stat
> and write a NetCDF file. That NetCDF file *will* have the masking
> regions applied in the output.
>
> The second option is by running the gen_vx_mask tool in a tricky
way.
> I'll list an example for met-6.0 using the NetCDF output of the
> ensemble_stat tool generated when you run "make test":
>
> met-6.0/bin/gen_vx_mask \
>
> met-6.0/out/ensemble_stat/ensemble_stat_20100101_120000V_ens.nc \
>
> met-6.0/data/poly/NWC.poly ens_mean_NWC.nc \
>
> -input_field 'name="REFC_L0_ENS_MEAN"; level="(*,*)"; \
>
> -complement -value -9999
>
>
> This is a little complicated, but here's what's going on:
> (1) Get the grid information from the ensemble stat NetCDF file
listed.
> (2) Read the lat/lon polyline from the file specified.
> (3) Write output to the third file listed.
> (4) Rather than initializing the grid with 0's, instead read data
> values for the field named "REFC_L0_ENS_MEAN" from the input file.
> (5) Rather than processing the inside of the polyline, process the
> outside using -complement.
> (6) Rather than writing the default mask value of 1, write a bad
data
> value of -9999.
>
> All these steps combine to read a gridded data field and set
> everything outside the polyline region to a bad data value.
>
> Hope that helps.
>
> John
>
> On Mon, Sep 18, 2017 at 11:04 AM, Raby, John W USA CIV via RT <
> met_help at ucar.edu> wrote:
>
> >
> > Mon Sep 18 11:04:07 2017: Request 82033 was acted upon.
> > Transaction: Ticket created by john.w.raby2.civ at mail.mil
> > Queue: met_help
> > Subject: question about MET V4.1 Ensemble-Stat (UNCLASSIFIED)
> > Owner: Nobody
> > Requestors: john.w.raby2.civ at mail.mil
> > Status: new
> > Ticket <Caution-Caution-url:
> > Caution-Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=82
> > 033 >
> >
> >
> > CLASSIFICATION: UNCLASSIFIED
> >
> > I ran with and without a masking region and compared the output. I
> > noted differences in the statistics given in the ascii files which
> > were as expected, but not in the NETCDF output of the various
fields
> > such as mean, stdev, range, min, max. It seems that the graphics
> > show the same area with or without the masking region. I had a
case
> > where the forecast model output had some strange edge effects
which
> > were confined to only the outer 2-3 grid points along the
boundaries
> > of the domain and I wanted to see the graphics generated without
> > including the edge effects. Is there a way to generate the
graphics
> > for the reduced area of the masking region?
> >
> > Thanks.
> >
> > R/
> > John
> >
> > Mr John W. Raby, Meteorologist
> > U.S. Army Research Laboratory
> > White Sands Missile Range, NM 88002
> > (575) 678-2004 DSN 258-2004
> > FAX (575) 678-1230 DSN 258-1230
> > Email: john.w.raby2.civ at mail.mil
> >
> >
> >
> > CLASSIFICATION: UNCLASSIFIED
> >
> >
>
>
> CLASSIFICATION: UNCLASSIFIED
>
>
CLASSIFICATION: UNCLASSIFIED
------------------------------------------------
Subject: question about MET V4.1 Ensemble-Stat (UNCLASSIFIED)
From: John Halley Gotway
Time: Tue Sep 19 12:28:14 2017
John,
We use Ensemble-Stat like this in two ways...
(1) to generate a very simplistic ensemble mean field.
(2) to generate very simplistic probability forecasts... and since
they
aren't calibrated, we call them relative frequencies instead of
probabilities.
Then we pass the ensemble_stat NetCDF output file to grid_stat (and/or
point_stat and mode) to verify the ensemble mean and probability
fields.
John
On Tue, Sep 19, 2017 at 12:22 PM, Raby, John W USA CIV via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=82033 >
>
> CLASSIFICATION: UNCLASSIFIED
>
> John -
>
> Thanks for the quick follow-up. Interesting, I had not thought about
> pre-processing input to grid-stat using ensemble-stat. Hmmm. now,
let me
> think
> about the output stats from grid-stat and what they would mean.
>
> R/
> John
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Tuesday, September 19, 2017 12:16 PM
> To: Raby, John W CIV USARMY RDECOM ARL (US)
<john.w.raby2.civ at mail.mil>
> Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #82033] question
about
> MET
> V4.1 Ensemble-Stat (UNCLASSIFIED)
>
> All active links contained in this email were disabled. Please
verify the
> identity of the sender, and confirm the authenticity of all links
contained
> within the message prior to copying and pasting the address to a Web
> browser.
>
>
>
>
> ----
>
> John,
>
> Yes, you'd pass the NetCDF output of Ensemble-Stat as input to Grid-
Stat.
>
> If you don't really care about the stats and all you really want is
the
> masked
> out NetCDF, just pass the same Ensemble-Stat NetCDF file as both the
> forecast
> and observation files to Grid-Stat.
>
> John
>
> On Tue, Sep 19, 2017 at 12:11 PM, Raby, John W USA CIV via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <Caution-url:
> > Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=82033 >
> >
> > CLASSIFICATION: UNCLASSIFIED
> >
> > John -
> >
> > Thanks.
> >
> > This confirms that what I am seeing is expected behavior, which
helps.
> > Very resourceful workarounds. So to run grid-stat, would I use
the
> > Ensemble-Stat NetCDF output fields as the input?
> >
> > R/
> > John
> > -----Original Message-----
> > From: John Halley Gotway via RT [Caution-mailto:met_help at ucar.edu]
> > Sent: Tuesday, September 19, 2017 10:21 AM
> > To: Raby, John W CIV USARMY RDECOM ARL (US)
> > <john.w.raby2.civ at mail.mil>
> > Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #82033] question
about
> > MET
> > V4.1
> > Ensemble-Stat (UNCLASSIFIED)
> >
> > All active links contained in this email were disabled. Please
verify
> > the identity of the sender, and confirm the authenticity of all
links
> > contained within the message prior to copying and pasting the
address
> > to a Web browser.
> >
> >
> >
> >
> > ----
> >
> > John,
> >
> > Based on the list of NetCDF variables you mentioned, I assume
you're
> > talking about running the Ensemble-Stat tool.
> >
> > The Ensemble-Stat tool really does 2 different types of
processing:
> >
> > (1) Process the fields defined in the "ens" config file section
and
> > derive fields like min, max, mean, stdev, and so on. This isn't
> > verification...
> > it's just very simplistic ensemble post-processing.
> >
> > (2) Process the fields defined in the "fcst" and "obs" config file
> > sections and do ensemble verification by computing things like
ranked
> > histograms (RHIST), PIT histograms (PHIST), and so on.
> >
> > The masking regions defined in the "mask" config file section are
only
> > applied to the logic of step (2), not step (1).
> >
> > However, I understand that you'd like to apply a masking region to
the
> > NetCDF output of step (1).
> >
> > There is no way to do this using the Ensemble-Stat tool. The
logic
> > just wasn't defined to support that.
> >
> > However, there are 2 ways you could do it.
> >
> > First, you could run grid_stat to process the output of Ensemble-
Stat
> > and write a NetCDF file. That NetCDF file *will* have the masking
> > regions applied in the output.
> >
> > The second option is by running the gen_vx_mask tool in a tricky
way.
> > I'll list an example for met-6.0 using the NetCDF output of the
> > ensemble_stat tool generated when you run "make test":
> >
> > met-6.0/bin/gen_vx_mask \
> >
> > met-6.0/out/ensemble_stat/ensemble_stat_20100101_120000V_ens.nc \
> >
> > met-6.0/data/poly/NWC.poly ens_mean_NWC.nc \
> >
> > -input_field 'name="REFC_L0_ENS_MEAN"; level="(*,*)"; \
> >
> > -complement -value -9999
> >
> >
> > This is a little complicated, but here's what's going on:
> > (1) Get the grid information from the ensemble stat NetCDF file
listed.
> > (2) Read the lat/lon polyline from the file specified.
> > (3) Write output to the third file listed.
> > (4) Rather than initializing the grid with 0's, instead read data
> > values for the field named "REFC_L0_ENS_MEAN" from the input file.
> > (5) Rather than processing the inside of the polyline, process the
> > outside using -complement.
> > (6) Rather than writing the default mask value of 1, write a bad
data
> > value of -9999.
> >
> > All these steps combine to read a gridded data field and set
> > everything outside the polyline region to a bad data value.
> >
> > Hope that helps.
> >
> > John
> >
> > On Mon, Sep 18, 2017 at 11:04 AM, Raby, John W USA CIV via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > Mon Sep 18 11:04:07 2017: Request 82033 was acted upon.
> > > Transaction: Ticket created by john.w.raby2.civ at mail.mil
> > > Queue: met_help
> > > Subject: question about MET V4.1 Ensemble-Stat
(UNCLASSIFIED)
> > > Owner: Nobody
> > > Requestors: john.w.raby2.civ at mail.mil
> > > Status: new
> > > Ticket <Caution-Caution-url:
> > > Caution-Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=82
> > > 033 >
> > >
> > >
> > > CLASSIFICATION: UNCLASSIFIED
> > >
> > > I ran with and without a masking region and compared the output.
I
> > > noted differences in the statistics given in the ascii files
which
> > > were as expected, but not in the NETCDF output of the various
fields
> > > such as mean, stdev, range, min, max. It seems that the graphics
> > > show the same area with or without the masking region. I had a
case
> > > where the forecast model output had some strange edge effects
which
> > > were confined to only the outer 2-3 grid points along the
boundaries
> > > of the domain and I wanted to see the graphics generated without
> > > including the edge effects. Is there a way to generate the
graphics
> > > for the reduced area of the masking region?
> > >
> > > Thanks.
> > >
> > > R/
> > > John
> > >
> > > Mr John W. Raby, Meteorologist
> > > U.S. Army Research Laboratory
> > > White Sands Missile Range, NM 88002
> > > (575) 678-2004 DSN 258-2004
> > > FAX (575) 678-1230 DSN 258-1230
> > > Email: john.w.raby2.civ at mail.mil
> > >
> > >
> > >
> > > CLASSIFICATION: UNCLASSIFIED
> > >
> > >
> >
> >
> > CLASSIFICATION: UNCLASSIFIED
> >
> >
>
>
>
> CLASSIFICATION: UNCLASSIFIED
>
>
------------------------------------------------
Subject: question about MET V4.1 Ensemble-Stat (UNCLASSIFIED)
From: Raby, John W USA CIV
Time: Tue Sep 19 15:47:17 2017
CLASSIFICATION: UNCLASSIFIED
John -
Thanks for those examples of what you are doing with grid-stat
operating on
ensemble-stat output. I'd like to take your suggestion about verifying
the ens
mean fields. My problem is that I only have point observations and no
gridded
obs for the case study I'm working with now. Maybe I could use Point-
Stat to
verify the ens mean against point obs. Your thoughts? Maybe Ensemble-
Stat is
already doing this?
R/
John
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Tuesday, September 19, 2017 12:28 PM
To: Raby, John W CIV USARMY RDECOM ARL (US)
<john.w.raby2.civ at mail.mil>
Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #82033] question
about MET
V4.1 Ensemble-Stat (UNCLASSIFIED)
All active links contained in this email were disabled. Please verify
the
identity of the sender, and confirm the authenticity of all links
contained
within the message prior to copying and pasting the address to a Web
browser.
----
John,
We use Ensemble-Stat like this in two ways...
(1) to generate a very simplistic ensemble mean field.
(2) to generate very simplistic probability forecasts... and since
they aren't
calibrated, we call them relative frequencies instead of
probabilities.
Then we pass the ensemble_stat NetCDF output file to grid_stat (and/or
point_stat and mode) to verify the ensemble mean and probability
fields.
John
On Tue, Sep 19, 2017 at 12:22 PM, Raby, John W USA CIV via RT <
met_help at ucar.edu> wrote:
>
> <Caution-url:
> Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=82033 >
>
> CLASSIFICATION: UNCLASSIFIED
>
> John -
>
> Thanks for the quick follow-up. Interesting, I had not thought about
> pre-processing input to grid-stat using ensemble-stat. Hmmm. now,
let
> me think about the output stats from grid-stat and what they would
> mean.
>
> R/
> John
>
> -----Original Message-----
> From: John Halley Gotway via RT [Caution-mailto:met_help at ucar.edu]
> Sent: Tuesday, September 19, 2017 12:16 PM
> To: Raby, John W CIV USARMY RDECOM ARL (US)
> <john.w.raby2.civ at mail.mil>
> Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #82033] question
> about MET
> V4.1 Ensemble-Stat (UNCLASSIFIED)
>
> All active links contained in this email were disabled. Please
verify
> the identity of the sender, and confirm the authenticity of all
links
> contained within the message prior to copying and pasting the
address
> to a Web browser.
>
>
>
>
> ----
>
> John,
>
> Yes, you'd pass the NetCDF output of Ensemble-Stat as input to Grid-
Stat.
>
> If you don't really care about the stats and all you really want is
> the masked out NetCDF, just pass the same Ensemble-Stat NetCDF file
as
> both the forecast and observation files to Grid-Stat.
>
> John
>
> On Tue, Sep 19, 2017 at 12:11 PM, Raby, John W USA CIV via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <Caution-Caution-url:
> > Caution-Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=82
> > 033 >
> >
> > CLASSIFICATION: UNCLASSIFIED
> >
> > John -
> >
> > Thanks.
> >
> > This confirms that what I am seeing is expected behavior, which
helps.
> > Very resourceful workarounds. So to run grid-stat, would I use
the
> > Ensemble-Stat NetCDF output fields as the input?
> >
> > R/
> > John
> > -----Original Message-----
> > From: John Halley Gotway via RT
> > [Caution-Caution-mailto:met_help at ucar.edu]
> > Sent: Tuesday, September 19, 2017 10:21 AM
> > To: Raby, John W CIV USARMY RDECOM ARL (US)
> > <john.w.raby2.civ at mail.mil>
> > Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #82033] question
> > about MET
> > V4.1
> > Ensemble-Stat (UNCLASSIFIED)
> >
> > All active links contained in this email were disabled. Please
> > verify the identity of the sender, and confirm the authenticity of
> > all links contained within the message prior to copying and
pasting
> > the address to a Web browser.
> >
> >
> >
> >
> > ----
> >
> > John,
> >
> > Based on the list of NetCDF variables you mentioned, I assume
you're
> > talking about running the Ensemble-Stat tool.
> >
> > The Ensemble-Stat tool really does 2 different types of
processing:
> >
> > (1) Process the fields defined in the "ens" config file section
and
> > derive fields like min, max, mean, stdev, and so on. This isn't
> > verification...
> > it's just very simplistic ensemble post-processing.
> >
> > (2) Process the fields defined in the "fcst" and "obs" config file
> > sections and do ensemble verification by computing things like
> > ranked histograms (RHIST), PIT histograms (PHIST), and so on.
> >
> > The masking regions defined in the "mask" config file section are
> > only applied to the logic of step (2), not step (1).
> >
> > However, I understand that you'd like to apply a masking region to
> > the NetCDF output of step (1).
> >
> > There is no way to do this using the Ensemble-Stat tool. The
logic
> > just wasn't defined to support that.
> >
> > However, there are 2 ways you could do it.
> >
> > First, you could run grid_stat to process the output of
> > Ensemble-Stat and write a NetCDF file. That NetCDF file *will*
have
> > the masking regions applied in the output.
> >
> > The second option is by running the gen_vx_mask tool in a tricky
way.
> > I'll list an example for met-6.0 using the NetCDF output of the
> > ensemble_stat tool generated when you run "make test":
> >
> > met-6.0/bin/gen_vx_mask \
> >
> > met-6.0/out/ensemble_stat/ensemble_stat_20100101_120000V_ens.nc \
> >
> > met-6.0/data/poly/NWC.poly ens_mean_NWC.nc \
> >
> > -input_field 'name="REFC_L0_ENS_MEAN"; level="(*,*)"; \
> >
> > -complement -value -9999
> >
> >
> > This is a little complicated, but here's what's going on:
> > (1) Get the grid information from the ensemble stat NetCDF file
listed.
> > (2) Read the lat/lon polyline from the file specified.
> > (3) Write output to the third file listed.
> > (4) Rather than initializing the grid with 0's, instead read data
> > values for the field named "REFC_L0_ENS_MEAN" from the input file.
> > (5) Rather than processing the inside of the polyline, process the
> > outside using -complement.
> > (6) Rather than writing the default mask value of 1, write a bad
> > data value of -9999.
> >
> > All these steps combine to read a gridded data field and set
> > everything outside the polyline region to a bad data value.
> >
> > Hope that helps.
> >
> > John
> >
> > On Mon, Sep 18, 2017 at 11:04 AM, Raby, John W USA CIV via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > Mon Sep 18 11:04:07 2017: Request 82033 was acted upon.
> > > Transaction: Ticket created by john.w.raby2.civ at mail.mil
> > > Queue: met_help
> > > Subject: question about MET V4.1 Ensemble-Stat
(UNCLASSIFIED)
> > > Owner: Nobody
> > > Requestors: john.w.raby2.civ at mail.mil
> > > Status: new
> > > Ticket <Caution-Caution-Caution-url:
> > > Caution-Caution-Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.
> > > html?id=82
> > > 033 >
> > >
> > >
> > > CLASSIFICATION: UNCLASSIFIED
> > >
> > > I ran with and without a masking region and compared the output.
I
> > > noted differences in the statistics given in the ascii files
which
> > > were as expected, but not in the NETCDF output of the various
> > > fields such as mean, stdev, range, min, max. It seems that the
> > > graphics show the same area with or without the masking region.
I
> > > had a case where the forecast model output had some strange edge
> > > effects which were confined to only the outer 2-3 grid points
> > > along the boundaries of the domain and I wanted to see the
> > > graphics generated without including the edge effects. Is there
a
> > > way to generate the graphics for the reduced area of the masking
region?
> > >
> > > Thanks.
> > >
> > > R/
> > > John
> > >
> > > Mr John W. Raby, Meteorologist
> > > U.S. Army Research Laboratory
> > > White Sands Missile Range, NM 88002
> > > (575) 678-2004 DSN 258-2004
> > > FAX (575) 678-1230 DSN 258-1230
> > > Email: john.w.raby2.civ at mail.mil
> > >
> > >
> > >
> > > CLASSIFICATION: UNCLASSIFIED
> > >
> > >
> >
> >
> > CLASSIFICATION: UNCLASSIFIED
> >
> >
>
>
>
> CLASSIFICATION: UNCLASSIFIED
>
>
CLASSIFICATION: UNCLASSIFIED
------------------------------------------------
Subject: question about MET V4.1 Ensemble-Stat (UNCLASSIFIED)
From: John Halley Gotway
Time: Tue Sep 19 15:56:53 2017
John,
Yes, if you have point observations, you'd use Point-Stat to compare
the
ensemble mean to those point observations.
You're right, Ensemble-Stat does already process point observations.
But
it uses them in a different way.
Ensemble-Stat uses point observations to compute ensemble stats, like
ranked histograms, spread-skill variance, and so on.
Point-Stat uses point observations to verify the ensemble mean as a
deterministic forecast, computing continuous and/or categorical stats.
Point-Stat can also verify the relative frequencies from ensemble_stat
like
a probability forecast.
I realize that there's lot's of little details to think about.
We have been working on MET+, a suite of python scripts to make it
easier
for folks to apply MET to their data. The goal is to work up several
different use cases that implement various types of verification
logic.
Ultimately, our goal is to make getting up and running with MET a lot
easier.
Thanks,
John
On Tue, Sep 19, 2017 at 3:47 PM, Raby, John W USA CIV via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=82033 >
>
> CLASSIFICATION: UNCLASSIFIED
>
> John -
>
> Thanks for those examples of what you are doing with grid-stat
operating on
> ensemble-stat output. I'd like to take your suggestion about
verifying the
> ens
> mean fields. My problem is that I only have point observations and
no
> gridded
> obs for the case study I'm working with now. Maybe I could use
Point-Stat
> to
> verify the ens mean against point obs. Your thoughts? Maybe
Ensemble-Stat
> is
> already doing this?
>
> R/
> John
>
> -----Original Message-----
> From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
> Sent: Tuesday, September 19, 2017 12:28 PM
> To: Raby, John W CIV USARMY RDECOM ARL (US)
<john.w.raby2.civ at mail.mil>
> Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #82033] question
about
> MET
> V4.1 Ensemble-Stat (UNCLASSIFIED)
>
> All active links contained in this email were disabled. Please
verify the
> identity of the sender, and confirm the authenticity of all links
contained
> within the message prior to copying and pasting the address to a Web
> browser.
>
>
>
>
> ----
>
> John,
>
> We use Ensemble-Stat like this in two ways...
>
> (1) to generate a very simplistic ensemble mean field.
>
> (2) to generate very simplistic probability forecasts... and since
they
> aren't
> calibrated, we call them relative frequencies instead of
probabilities.
>
> Then we pass the ensemble_stat NetCDF output file to grid_stat
(and/or
> point_stat and mode) to verify the ensemble mean and probability
fields.
>
> John
>
> On Tue, Sep 19, 2017 at 12:22 PM, Raby, John W USA CIV via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <Caution-url:
> > Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=82033 >
> >
> > CLASSIFICATION: UNCLASSIFIED
> >
> > John -
> >
> > Thanks for the quick follow-up. Interesting, I had not thought
about
> > pre-processing input to grid-stat using ensemble-stat. Hmmm. now,
let
> > me think about the output stats from grid-stat and what they would
> > mean.
> >
> > R/
> > John
> >
> > -----Original Message-----
> > From: John Halley Gotway via RT [Caution-mailto:met_help at ucar.edu]
> > Sent: Tuesday, September 19, 2017 12:16 PM
> > To: Raby, John W CIV USARMY RDECOM ARL (US)
> > <john.w.raby2.civ at mail.mil>
> > Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #82033]
question
> > about MET
> > V4.1 Ensemble-Stat (UNCLASSIFIED)
> >
> > All active links contained in this email were disabled. Please
verify
> > the identity of the sender, and confirm the authenticity of all
links
> > contained within the message prior to copying and pasting the
address
> > to a Web browser.
> >
> >
> >
> >
> > ----
> >
> > John,
> >
> > Yes, you'd pass the NetCDF output of Ensemble-Stat as input to
Grid-Stat.
> >
> > If you don't really care about the stats and all you really want
is
> > the masked out NetCDF, just pass the same Ensemble-Stat NetCDF
file as
> > both the forecast and observation files to Grid-Stat.
> >
> > John
> >
> > On Tue, Sep 19, 2017 at 12:11 PM, Raby, John W USA CIV via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <Caution-Caution-url:
> > > Caution-Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=82
> > > 033 >
> > >
> > > CLASSIFICATION: UNCLASSIFIED
> > >
> > > John -
> > >
> > > Thanks.
> > >
> > > This confirms that what I am seeing is expected behavior, which
helps.
> > > Very resourceful workarounds. So to run grid-stat, would I use
the
> > > Ensemble-Stat NetCDF output fields as the input?
> > >
> > > R/
> > > John
> > > -----Original Message-----
> > > From: John Halley Gotway via RT
> > > [Caution-Caution-mailto:met_help at ucar.edu]
> > > Sent: Tuesday, September 19, 2017 10:21 AM
> > > To: Raby, John W CIV USARMY RDECOM ARL (US)
> > > <john.w.raby2.civ at mail.mil>
> > > Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #82033] question
> > > about MET
> > > V4.1
> > > Ensemble-Stat (UNCLASSIFIED)
> > >
> > > All active links contained in this email were disabled. Please
> > > verify the identity of the sender, and confirm the authenticity
of
> > > all links contained within the message prior to copying and
pasting
> > > the address to a Web browser.
> > >
> > >
> > >
> > >
> > > ----
> > >
> > > John,
> > >
> > > Based on the list of NetCDF variables you mentioned, I assume
you're
> > > talking about running the Ensemble-Stat tool.
> > >
> > > The Ensemble-Stat tool really does 2 different types of
processing:
> > >
> > > (1) Process the fields defined in the "ens" config file section
and
> > > derive fields like min, max, mean, stdev, and so on. This isn't
> > > verification...
> > > it's just very simplistic ensemble post-processing.
> > >
> > > (2) Process the fields defined in the "fcst" and "obs" config
file
> > > sections and do ensemble verification by computing things like
> > > ranked histograms (RHIST), PIT histograms (PHIST), and so on.
> > >
> > > The masking regions defined in the "mask" config file section
are
> > > only applied to the logic of step (2), not step (1).
> > >
> > > However, I understand that you'd like to apply a masking region
to
> > > the NetCDF output of step (1).
> > >
> > > There is no way to do this using the Ensemble-Stat tool. The
logic
> > > just wasn't defined to support that.
> > >
> > > However, there are 2 ways you could do it.
> > >
> > > First, you could run grid_stat to process the output of
> > > Ensemble-Stat and write a NetCDF file. That NetCDF file *will*
have
> > > the masking regions applied in the output.
> > >
> > > The second option is by running the gen_vx_mask tool in a tricky
way.
> > > I'll list an example for met-6.0 using the NetCDF output of the
> > > ensemble_stat tool generated when you run "make test":
> > >
> > > met-6.0/bin/gen_vx_mask \
> > >
> > > met-6.0/out/ensemble_stat/ensemble_stat_20100101_120000V_ens.nc
\
> > >
> > > met-6.0/data/poly/NWC.poly ens_mean_NWC.nc \
> > >
> > > -input_field 'name="REFC_L0_ENS_MEAN"; level="(*,*)"; \
> > >
> > > -complement -value -9999
> > >
> > >
> > > This is a little complicated, but here's what's going on:
> > > (1) Get the grid information from the ensemble stat NetCDF file
listed.
> > > (2) Read the lat/lon polyline from the file specified.
> > > (3) Write output to the third file listed.
> > > (4) Rather than initializing the grid with 0's, instead read
data
> > > values for the field named "REFC_L0_ENS_MEAN" from the input
file.
> > > (5) Rather than processing the inside of the polyline, process
the
> > > outside using -complement.
> > > (6) Rather than writing the default mask value of 1, write a bad
> > > data value of -9999.
> > >
> > > All these steps combine to read a gridded data field and set
> > > everything outside the polyline region to a bad data value.
> > >
> > > Hope that helps.
> > >
> > > John
> > >
> > > On Mon, Sep 18, 2017 at 11:04 AM, Raby, John W USA CIV via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > Mon Sep 18 11:04:07 2017: Request 82033 was acted upon.
> > > > Transaction: Ticket created by john.w.raby2.civ at mail.mil
> > > > Queue: met_help
> > > > Subject: question about MET V4.1 Ensemble-Stat
(UNCLASSIFIED)
> > > > Owner: Nobody
> > > > Requestors: john.w.raby2.civ at mail.mil
> > > > Status: new
> > > > Ticket <Caution-Caution-Caution-url:
> > > > Caution-Caution-Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.
> > > > html?id=82
> > > > 033 >
> > > >
> > > >
> > > > CLASSIFICATION: UNCLASSIFIED
> > > >
> > > > I ran with and without a masking region and compared the
output. I
> > > > noted differences in the statistics given in the ascii files
which
> > > > were as expected, but not in the NETCDF output of the various
> > > > fields such as mean, stdev, range, min, max. It seems that the
> > > > graphics show the same area with or without the masking
region. I
> > > > had a case where the forecast model output had some strange
edge
> > > > effects which were confined to only the outer 2-3 grid points
> > > > along the boundaries of the domain and I wanted to see the
> > > > graphics generated without including the edge effects. Is
there a
> > > > way to generate the graphics for the reduced area of the
masking
> region?
> > > >
> > > > Thanks.
> > > >
> > > > R/
> > > > John
> > > >
> > > > Mr John W. Raby, Meteorologist
> > > > U.S. Army Research Laboratory
> > > > White Sands Missile Range, NM 88002
> > > > (575) 678-2004 DSN 258-2004
> > > > FAX (575) 678-1230 DSN 258-1230
> > > > Email: john.w.raby2.civ at mail.mil
> > > >
> > > >
> > > >
> > > > CLASSIFICATION: UNCLASSIFIED
> > > >
> > > >
> > >
> > >
> > > CLASSIFICATION: UNCLASSIFIED
> > >
> > >
> >
> >
> >
> > CLASSIFICATION: UNCLASSIFIED
> >
> >
>
>
>
> CLASSIFICATION: UNCLASSIFIED
>
>
------------------------------------------------
Subject: question about MET V4.1 Ensemble-Stat (UNCLASSIFIED)
From: Raby, John W USA CIV
Time: Tue Sep 19 16:37:54 2017
CLASSIFICATION: UNCLASSIFIED
Thanks for your responses which are very informative and with some
nuances. Appreciate noting the distinction between what Point-Stat
would do compared to Ensemble-Stat. Interesting news about the Met+
scripts. I think it’s a good direction to go for simplification. Good
info.
R/
John
-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Tuesday, September 19, 2017 3:57 PM
To: Raby, John W CIV USARMY RDECOM ARL (US)
<john.w.raby2.civ at mail.mil>
Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #82033] question
about MET V4.1 Ensemble-Stat (UNCLASSIFIED)
All active links contained in this email were disabled. Please verify
the identity of the sender, and confirm the authenticity of all links
contained within the message prior to copying and pasting the address
to a Web browser.
----
John,
Yes, if you have point observations, you'd use Point-Stat to compare
the ensemble mean to those point observations.
You're right, Ensemble-Stat does already process point observations.
But it uses them in a different way.
Ensemble-Stat uses point observations to compute ensemble stats, like
ranked histograms, spread-skill variance, and so on.
Point-Stat uses point observations to verify the ensemble mean as a
deterministic forecast, computing continuous and/or categorical stats.
Point-Stat can also verify the relative frequencies from ensemble_stat
like a probability forecast.
I realize that there's lot's of little details to think about.
We have been working on MET+, a suite of python scripts to make it
easier for folks to apply MET to their data. The goal is to work up
several different use cases that implement various types of
verification logic.
Ultimately, our goal is to make getting up and running with MET a lot
easier.
Thanks,
John
On Tue, Sep 19, 2017 at 3:47 PM, Raby, John W USA CIV via RT <
met_help at ucar.edu> wrote:
>
> <Caution-url:
> Caution-https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=82033 >
>
> CLASSIFICATION: UNCLASSIFIED
>
> John -
>
> Thanks for those examples of what you are doing with grid-stat
> operating on ensemble-stat output. I'd like to take your suggestion
> about verifying the ens mean fields. My problem is that I only have
> point observations and no gridded obs for the case study I'm working
> with now. Maybe I could use Point-Stat to verify the ens mean
against
> point obs. Your thoughts? Maybe Ensemble-Stat is already doing this?
>
> R/
> John
>
> -----Original Message-----
> From: John Halley Gotway via RT [Caution-mailto:met_help at ucar.edu]
> Sent: Tuesday, September 19, 2017 12:28 PM
> To: Raby, John W CIV USARMY RDECOM ARL (US)
> <john.w.raby2.civ at mail.mil>
> Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #82033] question
> about MET
> V4.1 Ensemble-Stat (UNCLASSIFIED)
>
> All active links contained in this email were disabled. Please
verify
> the identity of the sender, and confirm the authenticity of all
links
> contained within the message prior to copying and pasting the
address
> to a Web browser.
>
>
>
>
> ----
>
> John,
>
> We use Ensemble-Stat like this in two ways...
>
> (1) to generate a very simplistic ensemble mean field.
>
> (2) to generate very simplistic probability forecasts... and since
> they aren't calibrated, we call them relative frequencies instead of
> probabilities.
>
> Then we pass the ensemble_stat NetCDF output file to grid_stat
(and/or
> point_stat and mode) to verify the ensemble mean and probability
fields.
>
> John
>
> On Tue, Sep 19, 2017 at 12:22 PM, Raby, John W USA CIV via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <Caution-Caution-url:
> > Caution-Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=82
> > 033 >
> >
> > CLASSIFICATION: UNCLASSIFIED
> >
> > John -
> >
> > Thanks for the quick follow-up. Interesting, I had not thought
about
> > pre-processing input to grid-stat using ensemble-stat. Hmmm. now,
> > let me think about the output stats from grid-stat and what they
> > would mean.
> >
> > R/
> > John
> >
> > -----Original Message-----
> > From: John Halley Gotway via RT
> > [Caution-Caution-mailto:met_help at ucar.edu]
> > Sent: Tuesday, September 19, 2017 12:16 PM
> > To: Raby, John W CIV USARMY RDECOM ARL (US)
> > <john.w.raby2.civ at mail.mil>
> > Subject: Re: [Non-DoD Source] Re: [rt.rap.ucar.edu #82033]
question
> > about MET
> > V4.1 Ensemble-Stat (UNCLASSIFIED)
> >
> > All active links contained in this email were disabled. Please
> > verify the identity of the sender, and confirm the authenticity of
> > all links contained within the message prior to copying and
pasting
> > the address to a Web browser.
> >
> >
> >
> >
> > ----
> >
> > John,
> >
> > Yes, you'd pass the NetCDF output of Ensemble-Stat as input to
Grid-Stat.
> >
> > If you don't really care about the stats and all you really want
is
> > the masked out NetCDF, just pass the same Ensemble-Stat NetCDF
file
> > as both the forecast and observation files to Grid-Stat.
> >
> > John
> >
> > On Tue, Sep 19, 2017 at 12:11 PM, Raby, John W USA CIV via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <Caution-Caution-Caution-url:
> > > Caution-Caution-Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.
> > > html?id=82
> > > 033 >
> > >
> > > CLASSIFICATION: UNCLASSIFIED
> > >
> > > John -
> > >
> > > Thanks.
> > >
> > > This confirms that what I am seeing is expected behavior, which
helps.
> > > Very resourceful workarounds. So to run grid-stat, would I use
> > > the Ensemble-Stat NetCDF output fields as the input?
> > >
> > > R/
> > > John
> > > -----Original Message-----
> > > From: John Halley Gotway via RT
> > > [Caution-Caution-Caution-mailto:met_help at ucar.edu]
> > > Sent: Tuesday, September 19, 2017 10:21 AM
> > > To: Raby, John W CIV USARMY RDECOM ARL (US)
> > > <john.w.raby2.civ at mail.mil>
> > > Subject: [Non-DoD Source] Re: [rt.rap.ucar.edu #82033] question
> > > about MET
> > > V4.1
> > > Ensemble-Stat (UNCLASSIFIED)
> > >
> > > All active links contained in this email were disabled. Please
> > > verify the identity of the sender, and confirm the authenticity
of
> > > all links contained within the message prior to copying and
> > > pasting the address to a Web browser.
> > >
> > >
> > >
> > >
> > > ----
> > >
> > > John,
> > >
> > > Based on the list of NetCDF variables you mentioned, I assume
> > > you're talking about running the Ensemble-Stat tool.
> > >
> > > The Ensemble-Stat tool really does 2 different types of
processing:
> > >
> > > (1) Process the fields defined in the "ens" config file section
> > > and derive fields like min, max, mean, stdev, and so on. This
> > > isn't verification...
> > > it's just very simplistic ensemble post-processing.
> > >
> > > (2) Process the fields defined in the "fcst" and "obs" config
file
> > > sections and do ensemble verification by computing things like
> > > ranked histograms (RHIST), PIT histograms (PHIST), and so on.
> > >
> > > The masking regions defined in the "mask" config file section
are
> > > only applied to the logic of step (2), not step (1).
> > >
> > > However, I understand that you'd like to apply a masking region
to
> > > the NetCDF output of step (1).
> > >
> > > There is no way to do this using the Ensemble-Stat tool. The
> > > logic just wasn't defined to support that.
> > >
> > > However, there are 2 ways you could do it.
> > >
> > > First, you could run grid_stat to process the output of
> > > Ensemble-Stat and write a NetCDF file. That NetCDF file *will*
> > > have the masking regions applied in the output.
> > >
> > > The second option is by running the gen_vx_mask tool in a tricky
way.
> > > I'll list an example for met-6.0 using the NetCDF output of the
> > > ensemble_stat tool generated when you run "make test":
> > >
> > > met-6.0/bin/gen_vx_mask \
> > >
> > > met-6.0/out/ensemble_stat/ensemble_stat_20100101_120000V_ens.nc
\
> > >
> > > met-6.0/data/poly/NWC.poly ens_mean_NWC.nc \
> > >
> > > -input_field 'name="REFC_L0_ENS_MEAN"; level="(*,*)"; \
> > >
> > > -complement -value -9999
> > >
> > >
> > > This is a little complicated, but here's what's going on:
> > > (1) Get the grid information from the ensemble stat NetCDF file
listed.
> > > (2) Read the lat/lon polyline from the file specified.
> > > (3) Write output to the third file listed.
> > > (4) Rather than initializing the grid with 0's, instead read
data
> > > values for the field named "REFC_L0_ENS_MEAN" from the input
file.
> > > (5) Rather than processing the inside of the polyline, process
the
> > > outside using -complement.
> > > (6) Rather than writing the default mask value of 1, write a bad
> > > data value of -9999.
> > >
> > > All these steps combine to read a gridded data field and set
> > > everything outside the polyline region to a bad data value.
> > >
> > > Hope that helps.
> > >
> > > John
> > >
> > > On Mon, Sep 18, 2017 at 11:04 AM, Raby, John W USA CIV via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > Mon Sep 18 11:04:07 2017: Request 82033 was acted upon.
> > > > Transaction: Ticket created by john.w.raby2.civ at mail.mil
> > > > Queue: met_help
> > > > Subject: question about MET V4.1 Ensemble-Stat
(UNCLASSIFIED)
> > > > Owner: Nobody
> > > > Requestors: john.w.raby2.civ at mail.mil
> > > > Status: new
> > > > Ticket <Caution-Caution-Caution-Caution-url:
> > > > Caution-Caution-Caution-Caution-
https://rt.rap.ucar.edu/rt/Ticket/Display.
> > > > html?id=82
> > > > 033 >
> > > >
> > > >
> > > > CLASSIFICATION: UNCLASSIFIED
> > > >
> > > > I ran with and without a masking region and compared the
output.
> > > > I noted differences in the statistics given in the ascii files
> > > > which were as expected, but not in the NETCDF output of the
> > > > various fields such as mean, stdev, range, min, max. It seems
> > > > that the graphics show the same area with or without the
masking
> > > > region. I had a case where the forecast model output had some
> > > > strange edge effects which were confined to only the outer 2-3
> > > > grid points along the boundaries of the domain and I wanted to
> > > > see the graphics generated without including the edge effects.
> > > > Is there a way to generate the graphics for the reduced area
of
> > > > the masking
> region?
> > > >
> > > > Thanks.
> > > >
> > > > R/
> > > > John
> > > >
> > > > Mr John W. Raby, Meteorologist
> > > > U.S. Army Research Laboratory
> > > > White Sands Missile Range, NM 88002
> > > > (575) 678-2004 DSN 258-2004
> > > > FAX (575) 678-1230 DSN 258-1230
> > > > Email: john.w.raby2.civ at mail.mil
> > > >
> > > >
> > > >
> > > > CLASSIFICATION: UNCLASSIFIED
> > > >
> > > >
> > >
> > >
> > > CLASSIFICATION: UNCLASSIFIED
> > >
> > >
> >
> >
> >
> > CLASSIFICATION: UNCLASSIFIED
> >
> >
>
>
>
> CLASSIFICATION: UNCLASSIFIED
>
>
CLASSIFICATION: UNCLASSIFIED
------------------------------------------------
More information about the Met_help
mailing list