[Met_help] [rt.rap.ucar.edu #97272] History for STAT file output
Minna Win via RT
met_help at ucar.edu
Fri Oct 30 08:14:47 MDT 2020
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hi all,
Just have a quick question about how MET populates STAT files-as I'm writing a function to plot a timeseries of some metrics, but run into occasional gaps of missing data from the files.
As an example, I'm running point_stat for a variety of wind parameters (GUST, WIND, UGRD, VGRD) over several forecast 'thresholds' (e.g. >=4.47&&<8.94, >=8.94&&<13.41, >=13.41&&<17.88).
While working with the STAT file output, I've noticed that for some dates, the STAT file does not have entries for some parameters and/or forecast thresholds. For example: a forecast for October 1 would have 3 GUST entries (one for each threshold), October 2 would only have 1 (with, say, the >=4.47&&<8.94 threshold), and October 3 has no GUST entries in the STAT file output.
My assumption is, that if there are no forecasts and/or observations for a particular variable/threshold combination in the data, it is simply not written to the STAT file (versus writing missing or NA in place of the statistics). Is this correct? I just want to clarify this, before I go digging through my data to see if there are any errors.
Hope that makes sense!
Thanks, (again)!
-Tom
[https://firstenergycorp.com/content/dam/opcologos/emailsig/FE-logo.png]
Thomas Workoff
Sr Scientist
office: 330-436-1475 (850-1475)
tworkoff at firstenergycorp.com
341 White Pond Drive, Akron, OH 44320 | mailstop: A-WAC-C1 / AK-West Akron Campus
------------------------------------------------------------------------------
The information contained in this message is intended only for the personal and confidential use of the recipient(s) named above. If the reader of this message is not the intended recipient or an agent responsible for delivering it to the intended recipient, you are hereby notified that you have received this document in error and that any review, dissemination, distribution, or copying of this message is strictly prohibited. If you have received this communication in error, please notify us immediately, and delete the original message.
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: STAT file output
From: Minna Win
Time: Thu Oct 29 14:35:10 2020
Hi Tom,
It looks like you want to verify that when no results are available,
that
MET does not report these as NA in the output .stat files from
PointStat
and instead no entry exists. That is correct. As a sanity check, you
can
see how many matches were rejected by increasing your log level to a
high
number (eg 10). In your log file, you should see why observations were
rejected (eg by station id, valid time, bad obs value, etc.).
In the example you cited (for October 1,2, &3), are these results
consistent with the above information, (ie there are expected 'holes'
because the threshold values aren't met)?
I hope that answered your question. Feel free to reach out if you
still
have questions.
Regards,
Minna
---------------
Minna Win
National Center for Atmospheric Research
Developmental Testbed Center
Phone: 303-497-8423
Fax: 303-497-8401
---------------
Pronouns: she/her
On Thu, Oct 29, 2020 at 10:54 AM Workoff, Thomas E via RT
<met_help at ucar.edu>
wrote:
>
> Thu Oct 29 10:53:54 2020: Request 97272 was acted upon.
> Transaction: Ticket created by tworkoff at firstenergycorp.com
> Queue: met_help
> Subject: STAT file output
> Owner: Nobody
> Requestors: tworkoff at firstenergycorp.com
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97272 >
>
>
> Hi all,
>
> Just have a quick question about how MET populates STAT files-as I'm
> writing a function to plot a timeseries of some metrics, but run
into
> occasional gaps of missing data from the files.
>
> As an example, I'm running point_stat for a variety of wind
parameters
> (GUST, WIND, UGRD, VGRD) over several forecast 'thresholds' (e.g.
> >=4.47&&<8.94, >=8.94&&<13.41, >=13.41&&<17.88).
>
> While working with the STAT file output, I've noticed that for some
dates,
> the STAT file does not have entries for some parameters and/or
forecast
> thresholds. For example: a forecast for October 1 would have 3 GUST
entries
> (one for each threshold), October 2 would only have 1 (with, say,
the
> >=4.47&&<8.94 threshold), and October 3 has no GUST entries in the
STAT
> file output.
>
> My assumption is, that if there are no forecasts and/or observations
for a
> particular variable/threshold combination in the data, it is simply
not
> written to the STAT file (versus writing missing or NA in place of
the
> statistics). Is this correct? I just want to clarify this, before I
go
> digging through my data to see if there are any errors.
>
> Hope that makes sense!
>
> Thanks, (again)!
>
> -Tom
>
>
> [https://firstenergycorp.com/content/dam/opcologos/emailsig/FE-
logo.png]
>
> Thomas Workoff
> Sr Scientist
> office: 330-436-1475 (850-1475)
> tworkoff at firstenergycorp.com
> 341 White Pond Drive, Akron, OH 44320 | mailstop: A-WAC-C1 / AK-West
Akron
> Campus
>
>
>
------------------------------------------------------------------------------
>
> The information contained in this message is intended only for the
> personal and confidential use of the recipient(s) named above. If
the
> reader of this message is not the intended recipient or an agent
> responsible for delivering it to the intended recipient, you are
hereby
> notified that you have received this document in error and that any
review,
> dissemination, distribution, or copying of this message is strictly
> prohibited. If you have received this communication in error, please
notify
> us immediately, and delete the original message.
>
>
------------------------------------------------
Subject: RE: [EXTERNAL] Re: [rt.rap.ucar.edu #97272] STAT file output
From: Workoff, Thomas E
Time: Fri Oct 30 06:21:48 2020
Hi Minna,
That's exactly the information I needed. I suspected that, but wanted
to check with you before I went down the rabbit hole and starting re-
running data with higher logging.
Thanks for your time, as always!
-Tom
-----Original Message-----
From: Minna Win via RT <met_help at ucar.edu>
Sent: Thursday, October 29, 2020 4:35 PM
To: Workoff, Thomas E <tworkoff at firstenergycorp.com>
Subject: [EXTERNAL] Re: [rt.rap.ucar.edu #97272] STAT file output
Hi Tom,
It looks like you want to verify that when no results are available,
that MET does not report these as NA in the output .stat files from
PointStat and instead no entry exists. That is correct. As a sanity
check, you can see how many matches were rejected by increasing your
log level to a high number (eg 10). In your log file, you should see
why observations were rejected (eg by station id, valid time, bad obs
value, etc.).
In the example you cited (for October 1,2, &3), are these results
consistent with the above information, (ie there are expected 'holes'
because the threshold values aren't met)?
I hope that answered your question. Feel free to reach out if you
still have questions.
Regards,
Minna
---------------
Minna Win
National Center for Atmospheric Research Developmental Testbed Center
Phone: 303-497-8423
Fax: 303-497-8401
---------------
Pronouns: she/her
On Thu, Oct 29, 2020 at 10:54 AM Workoff, Thomas E via RT
<met_help at ucar.edu>
wrote:
>
> Thu Oct 29 10:53:54 2020: Request 97272 was acted upon.
> Transaction: Ticket created by tworkoff at firstenergycorp.com
> Queue: met_help
> Subject: STAT file output
> Owner: Nobody
> Requestors: tworkoff at firstenergycorp.com
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=97272
> >
>
>
> Hi all,
>
> Just have a quick question about how MET populates STAT files-as I'm
> writing a function to plot a timeseries of some metrics, but run
into
> occasional gaps of missing data from the files.
>
> As an example, I'm running point_stat for a variety of wind
parameters
> (GUST, WIND, UGRD, VGRD) over several forecast 'thresholds' (e.g.
> >=4.47&&<8.94, >=8.94&&<13.41, >=13.41&&<17.88).
>
> While working with the STAT file output, I've noticed that for some
> dates, the STAT file does not have entries for some parameters
and/or
> forecast thresholds. For example: a forecast for October 1 would
have
> 3 GUST entries (one for each threshold), October 2 would only have 1
> (with, say, the
> >=4.47&&<8.94 threshold), and October 3 has no GUST entries in the
> >STAT
> file output.
>
> My assumption is, that if there are no forecasts and/or observations
> for a particular variable/threshold combination in the data, it is
> simply not written to the STAT file (versus writing missing or NA in
> place of the statistics). Is this correct? I just want to clarify
> this, before I go digging through my data to see if there are any
errors.
>
> Hope that makes sense!
>
> Thanks, (again)!
>
> -Tom
>
>
> [https://firstenergycorp.com/content/dam/opcologos/emailsig/FE-
logo.pn
> g]
>
> Thomas Workoff
> Sr Scientist
> office: 330-436-1475 (850-1475)
> tworkoff at firstenergycorp.com
> 341 White Pond Drive, Akron, OH 44320 | mailstop: A-WAC-C1 / AK-West
> Akron Campus
>
>
>
----------------------------------------------------------------------
> --------
>
> The information contained in this message is intended only for the
> personal and confidential use of the recipient(s) named above. If
the
> reader of this message is not the intended recipient or an agent
> responsible for delivering it to the intended recipient, you are
> hereby notified that you have received this document in error and
that
> any review, dissemination, distribution, or copying of this message
is
> strictly prohibited. If you have received this communication in
error,
> please notify us immediately, and delete the original message.
>
>
------------------------------------------------------------------------------
The information contained in this message is intended only for the
personal and confidential use of the recipient(s) named above. If the
reader of this message is not the intended recipient or an agent
responsible for delivering it to the intended recipient, you are
hereby notified that you have received this document in error and that
any review, dissemination, distribution, or copying of this message is
strictly prohibited. If you have received this communication in error,
please notify us immediately, and delete the original message.
------------------------------------------------
More information about the Met_help
mailing list