[Met_help] [rt.rap.ucar.edu #98640] History for Summarizing point observations

Wed Feb 17 10:48:52 MST 2021

----------------------------------------------------------------
  Initial Request
----------------------------------------------------------------

Hi all,

One last question! I'm hoping to lean on your expertise and opinion to leverage my next step forward.

It's become clear to me that I'm going to have to run an analysis with point observations (via point_stat), but with maximum values over various time periods.  For example, the maximum forecasted wind speed (or gust) over a 12 hour period, run against the observed 12 hour wind speed maximum-at given point observation locations.

My issue at the moment is the creation of the 'max' observation file for the point observations, which are in netCDF format.  MET currently does not have the capability to do this, correct?  Basically, mimic the functionality of pcp_combine to comb through hourly observations and find the 'max' or 'sum' -- but for point observations.

Given the absence of a snazzy MET function to do this for me, do you happen to have a suggestion on the best way to do this? I would lean Python (via xarray)-but also know that NCO should allow this functionality (?).  Either way will require a bit of self-educating and code writing to create the 'max' file over customizable timeframe-and netCDF is still a bit foreign to me--so if someone in the community has approached this issue and has a suggestion, I'd love to hear it!

Also, has there been any discussion to add that functionality to MET in the future?  Just curious to hear if this functionality has ever been discussed before.

Sorry for the weird question. As always, thanks!
-Tom

[https://firstenergycorp.com/content/dam/opcologos/emailsig/FE-logo.png]

Thomas Workoff
Sr Scientist
office: 330-436-1475 (850-1475)
tworkoff at firstenergycorp.com
341 White Pond Drive, Akron, OH 44320 | mailstop: A-WAC-C1 / AK-West Akron Campus

------------------------------------------------------------------------------

The information contained in this message is intended only for the personal and confidential use of the recipient(s) named above. If the reader of this message is not the intended recipient or an agent responsible for delivering it to the intended recipient, you are hereby notified that you have received this document in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify us immediately, and delete the original message.

----------------------------------------------------------------
  Complete Ticket History
----------------------------------------------------------------

Subject: Summarizing point observations
From: John Halley Gotway
Time: Wed Feb 10 13:06:42 2021

Tom,

In fact, there are options that already exist that should be helpful.

First, the ascii2nc and pb2nc tools can be configured to derive a
maximum
value for each observation location. This is done using the
"time_summary"
config file option:
https://github.com/dtcenter/MET/blob/0fe2e42609120df236f74ae23c07f3fa4b930247/met/data/config/Ascii2NcConfig_default#L19

And you can find a description of that be searching for "time_summary"
here:
https://dtcenter.github.io/MET/latest/Users_Guide/reformat_point.html

Next, the obs_summary option in the Point-Stat confg file could also
be
useful:
https://github.com/dtcenter/MET/blob/0fe2e42609120df236f74ae23c07f3fa4b930247/met/data/config/PointStatConfig_default#L99

Search for "obs_summary" on this page:
https://dtcenter.github.io/MET/latest/Users_Guide/data_io.html#data-
io-met-configuration-file-options

So, you could either use ascii2nc or pb2nc to pre-compute the maximum
value... or you could pass all 12 hours of observations to point-stat
and
use the obs_summary option. By setting 'obs_summary = "MAX";', point-
stat
will derive the maximum of all observations falling within the time
window
prior to comparing to the forecast value.

Hopefully it all goes smoothly.

John

On Wed, Feb 10, 2021 at 12:50 PM Workoff, Thomas E via RT
<met_help at ucar.edu>
wrote:

>
> Wed Feb 10 12:49:55 2021: Request 98640 was acted upon.
> Transaction: Ticket created by tworkoff at firstenergycorp.com
>        Queue: met_help
>      Subject: Summarizing point observations
>        Owner: Nobody
>   Requestors: tworkoff at firstenergycorp.com
>       Status: new
>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98640 >
>
>
> Hi all,
>
> One last question! I'm hoping to lean on your expertise and opinion
to
> leverage my next step forward.
>
> It's become clear to me that I'm going to have to run an analysis
with
> point observations (via point_stat), but with maximum values over
various
> time periods.  For example, the maximum forecasted wind speed (or
gust)
> over a 12 hour period, run against the observed 12 hour wind speed
> maximum-at given point observation locations.
>
> My issue at the moment is the creation of the 'max' observation file
for
> the point observations, which are in netCDF format.  MET currently
does not
> have the capability to do this, correct?  Basically, mimic the
> functionality of pcp_combine to comb through hourly observations and
find
> the 'max' or 'sum' -- but for point observations.
>
> Given the absence of a snazzy MET function to do this for me, do you
> happen to have a suggestion on the best way to do this? I would lean
Python
> (via xarray)-but also know that NCO should allow this functionality
(?).
> Either way will require a bit of self-educating and code writing to
create
> the 'max' file over customizable timeframe-and netCDF is still a bit
> foreign to me--so if someone in the community has approached this
issue and
> has a suggestion, I'd love to hear it!
>
> Also, has there been any discussion to add that functionality to MET
in
> the future?  Just curious to hear if this functionality has ever
been
> discussed before.
>
> Sorry for the weird question. As always, thanks!
> -Tom
>
>
> [https://firstenergycorp.com/content/dam/opcologos/emailsig/FE-
logo.png]
>
> Thomas Workoff
> Sr Scientist
> office: 330-436-1475 (850-1475)
> tworkoff at firstenergycorp.com
> 341 White Pond Drive, Akron, OH 44320 | mailstop: A-WAC-C1 / AK-West
Akron
> Campus
>
>
>
------------------------------------------------------------------------------
>
> The information contained in this message is intended only for the
> personal and confidential use of the recipient(s) named above. If
the
> reader of this message is not the intended recipient or an agent
> responsible for delivering it to the intended recipient, you are
hereby
> notified that you have received this document in error and that any
review,
> dissemination, distribution, or copying of this message is strictly
> prohibited. If you have received this communication in error, please
notify
> us immediately, and delete the original message.
>
>

------------------------------------------------
Subject: RE: [EXTERNAL] Re: [rt.rap.ucar.edu #98640] Summarizing point observations
From: Workoff, Thomas E
Time: Wed Feb 10 13:15:33 2021

Hi John,

How did I miss that setting in point_stat? Talk about overthinking the
problem. That will definitely be the point of least resistance, given
our observations are all netCDF format and ready to go.

I'll script this up and give it a shot.  I should've known that I was
missing something...you have usually already thought of everything!

I'm glad I asked.  As always, thanks for your help!

-Tom

Thomas Workoff
Senior Scientist
Office: 330-436-1475 (850-1475)
tworkoff at firstenergycorp.com
341 White Pond Drive, Akron, OH 44320 | mailstop: A-WAC-C1 / AK-West
Akron Campus

-----Original Message-----
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Wednesday, February 10, 2021 3:07 PM
To: Workoff, Thomas E <tworkoff at firstenergycorp.com>
Subject: [EXTERNAL] Re: [rt.rap.ucar.edu #98640] Summarizing point
observations

Tom,

In fact, there are options that already exist that should be helpful.

First, the ascii2nc and pb2nc tools can be configured to derive a
maximum value for each observation location. This is done using the
"time_summary"
config file option:
https://github.com/dtcenter/MET/blob/0fe2e42609120df236f74ae23c07f3fa4b930247/met/data/config/Ascii2NcConfig_default#L19

And you can find a description of that be searching for "time_summary"
here:
https://secure-
web.cisco.com/1PwsXFEcnMM7rOXRdOBeegIYEUPqYBCMMw9DoSs90rhs081yGNBk7jI-
Xhu9TibMkRcocVP3SzNVPAPlORznQwmn7zleYdsOgXm3c4BVwG9-
vPrhgqIWs7T8RhGGW8o71fFD_QWsBsl541XfTFCAl9fR2J0kRQiY6bGbtfo5wP_YfYM2Pzx_FC9CxTcYu7UaegZfvEO7GEDFimSX7YJQVe8DFCKVdCOEOXXSRL6GNY-
0Vu4yrjOMLNendvjz4uiK31pE1Q1bjD689RHwLsgCc64nsvYO8R0auaSsYPlydabmZARQChqg5nIDyGh5wrFyJ6D7osc8onO3F2Ynl2QnamiIBFLJBIJWEvC-
DgLzydLyYU_NMgYymGARh1JaWmL_g/https%3A%2F%2Fdtcenter.github.io%2FMET%2Flatest%2FUsers_Guide%2Freformat_point.html

Next, the obs_summary option in the Point-Stat confg file could also
be
useful:
https://github.com/dtcenter/MET/blob/0fe2e42609120df236f74ae23c07f3fa4b930247/met/data/config/PointStatConfig_default#L99

Search for "obs_summary" on this page:
https://secure-
web.cisco.com/1gD4vkRYwy7fYB0fwx79WQZlMrHKluWMC12wCphCN_i9zsWX3TTdPCgwXD0lA9ZsKE7TYr5-
4ErPFPCs0HkOgcdakYM-
ALQYdkAauLP1vWc5ba4DvcY2jOPctbFtI5RtzcTrL94FaD8erVoxb8kLr-
1dNL7XKeaoCJ_AA_jkaua_nrhbbnCy-2ESUeUK0c39IZINXrrJGTgaMN1DV7RX-
esYwAP0B_xsjOHxpJgX5OsI9tl4gbRZsscB30pPeeB0MP7huEZq6HoBBF0FtEHCcWdDcCo7MS4oBws5yDScCsHbI27afHa9dRnkIoETGuh4t00ZR_NPjKjyAiNfHkCSvkoMSQqcZaTHDE2SKfS5qBootU0fnB5JX0f6GtyguVbT5/https%3A%2F%2Fdtcenter.github.io%2FMET%2Flatest%2FUsers_Guide%2Fdata_io.html%23data-
io-met-configuration-file-options

So, you could either use ascii2nc or pb2nc to pre-compute the maximum
value... or you could pass all 12 hours of observations to point-stat
and use the obs_summary option. By setting 'obs_summary = "MAX";',
point-stat will derive the maximum of all observations falling within
the time window prior to comparing to the forecast value.

Hopefully it all goes smoothly.

John

On Wed, Feb 10, 2021 at 12:50 PM Workoff, Thomas E via RT
<met_help at ucar.edu>
wrote:

>
> Wed Feb 10 12:49:55 2021: Request 98640 was acted upon.
> Transaction: Ticket created by tworkoff at firstenergycorp.com
>        Queue: met_help
>      Subject: Summarizing point observations
>        Owner: Nobody
>   Requestors: tworkoff at firstenergycorp.com
>       Status: new
>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=98640
> >
>
>
> Hi all,
>
> One last question! I'm hoping to lean on your expertise and opinion
to
> leverage my next step forward.
>
> It's become clear to me that I'm going to have to run an analysis
with
> point observations (via point_stat), but with maximum values over
> various time periods.  For example, the maximum forecasted wind
speed
> (or gust) over a 12 hour period, run against the observed 12 hour
wind
> speed maximum-at given point observation locations.
>
> My issue at the moment is the creation of the 'max' observation file
> for the point observations, which are in netCDF format.  MET
currently
> does not have the capability to do this, correct?  Basically, mimic
> the functionality of pcp_combine to comb through hourly observations
> and find the 'max' or 'sum' -- but for point observations.
>
> Given the absence of a snazzy MET function to do this for me, do you
> happen to have a suggestion on the best way to do this? I would lean
> Python (via xarray)-but also know that NCO should allow this
functionality (?).
> Either way will require a bit of self-educating and code writing to
> create the 'max' file over customizable timeframe-and netCDF is
still
> a bit foreign to me--so if someone in the community has approached
> this issue and has a suggestion, I'd love to hear it!
>
> Also, has there been any discussion to add that functionality to MET
> in the future?  Just curious to hear if this functionality has ever
> been discussed before.
>
> Sorry for the weird question. As always, thanks!
> -Tom
>
>
> [https://firstenergycorp.com/content/dam/opcologos/emailsig/FE-
logo.pn
> g]
>
> Thomas Workoff
> Sr Scientist
> office: 330-436-1475 (850-1475)
> tworkoff at firstenergycorp.com
> 341 White Pond Drive, Akron, OH 44320 | mailstop: A-WAC-C1 / AK-West
> Akron Campus
>
>
>
----------------------------------------------------------------------
> --------
>
> The information contained in this message is intended only for the
> personal and confidential use of the recipient(s) named above. If
the
> reader of this message is not the intended recipient or an agent
> responsible for delivering it to the intended recipient, you are
> hereby notified that you have received this document in error and
that
> any review, dissemination, distribution, or copying of this message
is
> strictly prohibited. If you have received this communication in
error,
> please notify us immediately, and delete the original message.
>
>

------------------------------------------------------------------------------

The information contained in this message is intended only for the
personal and confidential use of the recipient(s) named above. If the
reader of this message is not the intended recipient or an agent
responsible for delivering it to the intended recipient, you are
hereby notified that you have received this document in error and that
any review, dissemination, distribution, or copying of this message is
strictly prohibited. If you have received this communication in error,
please notify us immediately, and delete the original message.

------------------------------------------------