[Met_help] [rt.rap.ucar.edu #70338] History for Setting a range of pressure levels to verify in ensemble_stat

[John Halley Gotway via RT]

----------------------------------------------------------------
  Initial Request
----------------------------------------------------------------

Hello,

Regarding the ensemble stat tool, I was wondering if there is a way to imply processing of variables at multiple pressure levels in one name call. For example, having P200-P925 in the ens and obs=fcst sections will only give verification of variables at 200 hPa.
name       = "TMP";
         level      = [ "P200-P925" ];
         cat_thresh = [];

I saw that this was used in the point-stat tool documentation example so was wondering if it could extend to ensemble stat? I could just declare each variable at each pressure level in that range, but was wondering for the sake of simplifying config scripts if I could set a range like above?

Thanks,
Brock

Brock Burghardt
Atmospheric Science PhD Student
Dept. of Geosciences
Texas Tech University



----------------------------------------------------------------
  Complete Ticket History
----------------------------------------------------------------

Subject: Setting a range of pressure levels to verify in ensemble_stat
From: John Halley Gotway
Time: Mon Jan 12 12:17:49 2015

Brock,

Here's how the logic works in Point-Stat...

For a range of pressure levels like P200-925, point_stat first looks
to see
if there's a single GRIB record defined for that range.  If so, it
reads
it.  If not, it reads all single pressure levels falling into that
range.
If you have output every 100mb, it'd read records for 200, 300, 400,
500,
600, 700, 800, and 900.  Next, for each point observation falling in
the
defined range, it'll look at the forecast levels above/below the
observation.  For an observation at 575mb, it'll look at the forecast
levels of 500 and 600 mb.  It'll interpolate to the observations
lat/lon
horizontally on the 500 and 600 levels.  Then it's interpolate
vertically
between those levels, linear in the log of pressure.  Since 575 is
closer
to 600 than 500, the value at 600 will be weighted more that the value
at
500.

Point-Stat groups together all matched forecast and observation pairs
for
the observations falling into that range and uses them to compute
output
statistics.  So the verification statistics are computed over all
those
levels.

Rather than aggregating results together in the vertical, you may
instead
want to evaluate performance level-by-level.  So define multiple
levels,
like this: level = [ "P200", "P300", ..., "P900" ];
That generate 8 different sets of output statistics, one for each
level.

You need to decide how you want to slice/dice/aggregate the data when
you
set up the configuration file, and that will determine the levels or
range
of levels you use.

Now on to your question about ensemble_stat...
There are two different sections where you specify the "fields" to be
evaluated: ens.field and fcst.field.

In the "ens" section, the field defines the 2D fields for which you
want to
compute an ensemble-mean.  This cannot be a range of levels - you need
to
list them explicitly.  However, there are some short cuts you could
use.
(1) If you list multiple entries for the "level", it'll treat them
separately.  The example below will read temperature from each of the
levels listed:

   field = [
      {
         name  = "TMP";
         level = [ "P900", "P800", "P700", "P600", "P500", "P400",
"P300",
"P200" ];
         cat_thresh = [ >300 ];
      }
   ];

(2) You can take advantage of the scope in the config file.  Suppose
you
want to evaluate 3 different variables on the same set of 8 levels.
The
example will do that and you only need to specify those levels once.
MET
will try to read "level" from each array entry of "field", but if it
can't
find it there, it'll step up to the next scope, find it, and read it
there:

   level = [ "P900", "P800", "P700", "P600", "P500", "P400", "P300",
"P200"
];
   field = [
      { name  = "TMP";  cat_thresh = [ >300 ]; },
      { name  = "UGRD"; cat_thresh = [ >2 ]; },
      { name  = "VGRD"; cat_thresh = [ >2 ]; }
   ];

Those shortcuts may make your Ensemble-Stat config file easier to
write and
maintain.

"field" also shows up in the "fcst" section which is used to define to
what
statistics should be computed.  Here's you can define a range of
pressure
levels like this:
   field = [
      { name  = "TMP"; level = [ "P900-200" ]; }
   ];

That'll compute stats over a range of pressure levels, just like
Point-Stat
does.

However, you may see an error message like this one:
ERROR  : process_point_ens() -> no fields matching
TMP_P900-200_ENS_MEAN(*,*) found in file:
out/ensemble_stat_20140820_000000V_ens.nc

To get rid of it, just disable the output flag for spread-skill
variance:
   ssvar = NONE;

In order to compute spread-skill variance, Ensemble-Stat needs an
ensemble
mean.  But no ensemble-mean exists for that field in the NetCDF output
that
Ensemble-Stat computed.

Give that a shot and let me know if you have more questions.

Thanks,
John


On Mon, Jan 12, 2015 at 10:57 AM, Burghardt, Brock via RT
<met_help at ucar.edu
> wrote:

>
> Mon Jan 12 10:57:25 2015: Request 70338 was acted upon.
> Transaction: Ticket created by brock.burghardt at ttu.edu
>        Queue: met_help
>      Subject: Setting a range of pressure levels to verify in
ensemble_stat
>        Owner: Nobody
>   Requestors: brock.burghardt at ttu.edu
>       Status: new
>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=70338 >
>
>
> Hello,
>
> Regarding the ensemble stat tool, I was wondering if there is a way
to
> imply processing of variables at multiple pressure levels in one
name call.
> For example, having P200-P925 in the ens and obs=fcst sections will
only
> give verification of variables at 200 hPa.
> name       = "TMP";
>          level      = [ "P200-P925" ];
>          cat_thresh = [];
>
> I saw that this was used in the point-stat tool documentation
example so
> was wondering if it could extend to ensemble stat? I could just
declare
> each variable at each pressure level in that range, but was
wondering for
> the sake of simplifying config scripts if I could set a range like
above?
>
> Thanks,
> Brock
>
> Brock Burghardt
> Atmospheric Science PhD Student
> Dept. of Geosciences
> Texas Tech University
>
>
>

------------------------------------------------
Subject: RE: [rt.rap.ucar.edu #70338] Setting a range of pressure levels to verify in ensemble_stat
From: Burghardt, Brock
Time: Mon Jan 12 12:35:24 2015

Excellent, thank you for explaining that so well, it makes much more
sense to me now. That also explains why I was seeing the SSVAR error,
but I'll likely want to look at the individual level statistics so
will need to break up the levels.

Thanks again!
Brock

-----Original Message-----
From: John Halley Gotway via RT [mailto:met_help at ucar.edu]
Sent: Monday, January 12, 2015 1:18 PM
To: Burghardt, Brock
Subject: Re: [rt.rap.ucar.edu #70338] Setting a range of pressure
levels to verify in ensemble_stat

Brock,

Here's how the logic works in Point-Stat...

For a range of pressure levels like P200-925, point_stat first looks
to see if there's a single GRIB record defined for that range.  If so,
it reads it.  If not, it reads all single pressure levels falling into
that range.
If you have output every 100mb, it'd read records for 200, 300, 400,
500, 600, 700, 800, and 900.  Next, for each point observation falling
in the defined range, it'll look at the forecast levels above/below
the observation.  For an observation at 575mb, it'll look at the
forecast levels of 500 and 600 mb.  It'll interpolate to the
observations lat/lon horizontally on the 500 and 600 levels.  Then
it's interpolate vertically between those levels, linear in the log of
pressure.  Since 575 is closer to 600 than 500, the value at 600 will
be weighted more that the value at 500.

Point-Stat groups together all matched forecast and observation pairs
for the observations falling into that range and uses them to compute
output statistics.  So the verification statistics are computed over
all those levels.

Rather than aggregating results together in the vertical, you may
instead want to evaluate performance level-by-level.  So define
multiple levels, like this: level = [ "P200", "P300", ..., "P900" ];
That generate 8 different sets of output statistics, one for each
level.

You need to decide how you want to slice/dice/aggregate the data when
you set up the configuration file, and that will determine the levels
or range of levels you use.

Now on to your question about ensemble_stat...
There are two different sections where you specify the "fields" to be
evaluated: ens.field and fcst.field.

In the "ens" section, the field defines the 2D fields for which you
want to compute an ensemble-mean.  This cannot be a range of levels -
you need to list them explicitly.  However, there are some short cuts
you could use.
(1) If you list multiple entries for the "level", it'll treat them
separately.  The example below will read temperature from each of the
levels listed:

   field = [
      {
         name  = "TMP";
         level = [ "P900", "P800", "P700", "P600", "P500", "P400",
"P300", "P200" ];
         cat_thresh = [ >300 ];
      }
   ];

(2) You can take advantage of the scope in the config file.  Suppose
you want to evaluate 3 different variables on the same set of 8
levels.  The example will do that and you only need to specify those
levels once.  MET will try to read "level" from each array entry of
"field", but if it can't find it there, it'll step up to the next
scope, find it, and read it there:

   level = [ "P900", "P800", "P700", "P600", "P500", "P400", "P300",
"P200"
];
   field = [
      { name  = "TMP";  cat_thresh = [ >300 ]; },
      { name  = "UGRD"; cat_thresh = [ >2 ]; },
      { name  = "VGRD"; cat_thresh = [ >2 ]; }
   ];

Those shortcuts may make your Ensemble-Stat config file easier to
write and maintain.

"field" also shows up in the "fcst" section which is used to define to
what statistics should be computed.  Here's you can define a range of
pressure levels like this:
   field = [
      { name  = "TMP"; level = [ "P900-200" ]; }
   ];

That'll compute stats over a range of pressure levels, just like
Point-Stat does.

However, you may see an error message like this one:
ERROR  : process_point_ens() -> no fields matching
TMP_P900-200_ENS_MEAN(*,*) found in file:
out/ensemble_stat_20140820_000000V_ens.nc

To get rid of it, just disable the output flag for spread-skill
variance:
   ssvar = NONE;

In order to compute spread-skill variance, Ensemble-Stat needs an
ensemble mean.  But no ensemble-mean exists for that field in the
NetCDF output that Ensemble-Stat computed.

Give that a shot and let me know if you have more questions.

Thanks,
John


On Mon, Jan 12, 2015 at 10:57 AM, Burghardt, Brock via RT
<met_help at ucar.edu
> wrote:

>
> Mon Jan 12 10:57:25 2015: Request 70338 was acted upon.
> Transaction: Ticket created by brock.burghardt at ttu.edu
>        Queue: met_help
>      Subject: Setting a range of pressure levels to verify in
ensemble_stat
>        Owner: Nobody
>   Requestors: brock.burghardt at ttu.edu
>       Status: new
>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=70338
> >
>
>
> Hello,
>
> Regarding the ensemble stat tool, I was wondering if there is a way
to
> imply processing of variables at multiple pressure levels in one
name call.
> For example, having P200-P925 in the ens and obs=fcst sections will
> only give verification of variables at 200 hPa.
> name       = "TMP";
>          level      = [ "P200-P925" ];
>          cat_thresh = [];
>
> I saw that this was used in the point-stat tool documentation
example
> so was wondering if it could extend to ensemble stat? I could just
> declare each variable at each pressure level in that range, but was
> wondering for the sake of simplifying config scripts if I could set
a range like above?
>
> Thanks,
> Brock
>
> Brock Burghardt
> Atmospheric Science PhD Student
> Dept. of Geosciences
> Texas Tech University
>
>
>



------------------------------------------------