[Met_help] [rt.rap.ucar.edu #95033] History for MET Stat Anal Summary Question
John Halley Gotway via RT
met_help at ucar.edu
Mon Apr 27 15:23:42 MDT 2020
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
John, working from home on python code to plot MET data - MET Viewer is still tied up in redtape. Say I run stat analysis summary job for two statistics RMSE and ME. In the output, the job line will have -column CNT:RMSE -column CNT:ME in that order. But the data will have the first line being ME and then the second line being RMSE. See below:
JOB_LIST: -job summary -model galwem10_p25 -fcst_lead 000000 -fcst_lead 240000 -fcst_lead 480000 -fcst_lead 720000 -fcst_lead 1200000 -fcst_lead 2400000 -fcst_valid_beg 20191226_000000 -fcst_valid_end 20200305_180000 -fcst_var TMP -fcst_lev Z2 -vx_mask FULL -column FULL:RMSE -column FULL:ME -by FCST_THRESH -by FCST_LEAD -by FCST_LEAD -by FCST_INIT_BEG -out_alpha 0.05000
MODEL: VX_MASK COLUMN FCST_THRESH FCST_LEV FCST_LEAD FCST_INIT_BEG TOTAL MEAN MEAN_NCL MEAN_NCU MEAN_BCL MEAN_BCU STDEV STDEV_BCL STDEV_BCU MIN P10 P25 P50 P75 P90 MAX IQR RANGE WMO_TYPE WMO_MEAN WMO_WEIGHTED_MEAN
GALWEM10_P25: FULL ME >=-60|| Z2 000000 20191227_000000 1 -0.46691 NA NA NA NA NA NA NA -0.46691 -0.46691 -0.46691 -0.46691 -0.46691 -0.46691 -0.46691 0 0 MEAN -0.46691 -0.46691
GALWEM10_P25: FULL RMSE >=-60|| Z2 000000 20191227_000000 1 2.61733 NA NA NA NA NA NA NA 2.61733 2.61733 2.61733 2.61733 2.61733 2.61733 2.61733 0 0 SQRT 2.61733 2.61733
So the question I have, is the order of statistics in the COLUMN field arbitrary or something else like alphabetical? As I parse these lines, how can predict the order of the statistics?
Thanks
Bob
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: MET Stat Anal Summary Question
From: John Halley Gotway
Time: Fri Apr 24 16:31:54 2020
Bob,
I see you have a question about what order is used when writing the
summary
output for multiple columns in Stat-Analysis. That's a good question,
and
I didn't know the answer off hand. I played around with it myself
using
CNT columns FBAR, ME, OBAR, and RMSE. And no matter what order I
provided
them in the input, they always came out in the order I just listed.
And
that order sure looks like alphabetical order.
We store this data in the MET code in a data structure called a map.
Maps
are very convenient and powerful, but you generally don't control the
order
inside the map unless you make some intentional effort to do so. In
this
case we haven't and it looks like it's defaulting to an alphabetic
storage
order. I can see how maintaining the insertion order would be
somewhat
more intuitive. If you're reading this output into a plotting
package, it
would be more robust to just look at the COLUMN name of the output
instead
of assuming some order.
Here's a few more comments...
(1) I'm surprised to these command line options: -column FULL:RMSE
-column
FULL:ME
To specify those input columns, you can either do it like this:
-line_type CNT -column RMSE,ME
Or indicate the line type separately for each column:
-column CNT:RMSE,CNT:ME
But there is no "FULL" line type. I'm guessing that was just a typo
in the
command you sent.
(2) Remember that you can use the "-out" command line option to
redirect
this data to an output file. Perhaps it might be slightly easier to
parse
that output file instead of the Stat-Analysis log file.
(3) Remember that the "summary" job reports the mean of the daily
statistics. You can also run an aggregate stat job to aggregate
statistics
across multiple cases and use the "-out_stat" option to write the
result to
an output .stat file:
-job aggregate -line_type SL1L2 -out_line_type CNT -out_stat
agg_CNT.stat
Hope that helps.
If you really do need Stat-Analysis to report the summary output
columns
using the order that you provided on the command line, that's doable.
We'd
just need to document the request and add it to the list for met-9.1.
Thanks,
John
On Fri, Apr 24, 2020 at 10:17 AM robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> Fri Apr 24 10:16:51 2020: Request 95033 was acted upon.
> Transaction: Ticket created by robert.craig.2 at us.af.mil
> Queue: met_help
> Subject: MET Stat Anal Summary Question
> Owner: Nobody
> Requestors: robert.craig.2 at us.af.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95033 >
>
>
> John, working from home on python code to plot MET data - MET Viewer
is
> still tied up in redtape. Say I run stat analysis summary job for
two
> statistics RMSE and ME. In the output, the job line will have
-column
> CNT:RMSE -column CNT:ME in that order. But the data will have the
first
> line being ME and then the second line being RMSE. See below:
>
> JOB_LIST: -job summary -model galwem10_p25 -fcst_lead 000000
> -fcst_lead 240000 -fcst_lead 480000 -fcst_lead 720000 -fcst_lead
1200000
> -fcst_lead 2400000 -fcst_valid_beg 20191226_000000 -fcst_valid_end
> 20200305_180000 -fcst_var TMP -fcst_lev Z2 -vx_mask FULL -column
FULL:RMSE
> -column FULL:ME -by FCST_THRESH -by FCST_LEAD -by FCST_LEAD -by
> FCST_INIT_BEG -out_alpha 0.05000
> MODEL: VX_MASK COLUMN FCST_THRESH FCST_LEV FCST_LEAD FCST_INIT_BEG
> TOTAL MEAN MEAN_NCL MEAN_NCU MEAN_BCL MEAN_BCU STDEV
STDEV_BCL
> STDEV_BCU MIN P10 P25 P50
P75
> P90 MAX IQR RANGE WMO_TYPE WMO_MEAN WMO_WEIGHTED_MEAN
> GALWEM10_P25: FULL ME >=-60|| Z2 000000
> 20191227_000000 1 -0.46691 NA NA NA
NA
> NA NA NA -0.46691 -0.46691 -0.46691 -0.46691
> -0.46691 -0.46691 -0.46691 0 0 MEAN -0.46691
> -0.46691
> GALWEM10_P25: FULL RMSE >=-60|| Z2 000000
> 20191227_000000 1 2.61733 NA NA NA
NA
> NA NA NA 2.61733 2.61733 2.61733 2.61733
> 2.61733 2.61733 2.61733 0 0 SQRT 2.61733
> 2.61733
>
> So the question I have, is the order of statistics in the COLUMN
field
> arbitrary or something else like alphabetical? As I parse these
lines, how
> can predict the order of the statistics?
>
>
> Thanks
>
> Bob
>
>
------------------------------------------------
Subject: MET Stat Anal Summary Question
From: John Halley Gotway
Time: Mon Apr 27 15:22:46 2020
Bob,
Sounds good. Things are going alright here in Boulder. I've been
working
remotely with 3 kids at home for just about as long as I can remember
now
;) But checking the calendar, it's been about 6 weeks. But we're
able to
work from home pretty well.
I'll go ahead and resolve this ticket. Just let us know what other
questions or issues arise.
Thanks,
John
On Mon, Apr 27, 2020 at 9:24 AM robert.craig.2 at us.af.mil via RT <
met_help at ucar.edu> wrote:
>
> Mon Apr 27 09:24:11 2020: Request 95061 was acted upon.
> Transaction: Ticket created by robert.craig.2 at us.af.mil
> Queue: met_help
> Subject: Re: MET Stat Anal Summary Question
> Owner: Nobody
> Requestors: robert.craig.2 at us.af.mil
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95061 >
>
>
> Thanks, I am not sure where the FULL:RMSE and FULL:ME came from. In
my
> data file they are CNT:RMSE and CNT:ME like you would expect -
weird. I
> figured the order was probably alphabetical but wanted to make sure.
I was
> asking since I read the MET summary files into a python structure
arrays
> and parse the various statistics from the column field to their own
fields.
>
>
> We don't need any changes to the code. I am currently stuck at home
so it
> seemed like a good time to update my plotting code to be in python
(it was
> IDL) and to read MET output (instead of our legacy code output).
Luckily,
> I captured MET summary, CNT, CTS, CTC, and PSTD files before heading
home
> so I would have something to work with. I hope they will let us
back in
> the near future.
>
>
> Hope all is well in Boulder.
>
>
> Bob
>
>
> ________________________________
> From: CRAIG, ROBERT J GS-12 USAF ACC 16 WS/WXN
> Sent: Friday, April 24, 2020 11:16 AM
> To: John Halley Gotway via RT <met_help at ucar.edu>
> Subject: MET Stat Anal Summary Question
>
>
> John, working from home on python code to plot MET data - MET Viewer
is
> still tied up in redtape. Say I run stat analysis summary job for
two
> statistics RMSE and ME. In the output, the job line will have
-column
> CNT:RMSE -column CNT:ME in that order. But the data will have the
first
> line being ME and then the second line being RMSE. See below:
>
> JOB_LIST: -job summary -model galwem10_p25 -fcst_lead 000000
> -fcst_lead 240000 -fcst_lead 480000 -fcst_lead 720000 -fcst_lead
1200000
> -fcst_lead 2400000 -fcst_valid_beg 20191226_000000 -fcst_valid_end
> 20200305_180000 -fcst_var TMP -fcst_lev Z2 -vx_mask FULL -column
FULL:RMSE
> -column FULL:ME -by FCST_THRESH -by FCST_LEAD -by FCST_LEAD -by
> FCST_INIT_BEG -out_alpha 0.05000
> MODEL: VX_MASK COLUMN FCST_THRESH FCST_LEV FCST_LEAD FCST_INIT_BEG
> TOTAL MEAN MEAN_NCL MEAN_NCU MEAN_BCL MEAN_BCU STDEV
STDEV_BCL
> STDEV_BCU MIN P10 P25 P50
P75
> P90 MAX IQR RANGE WMO_TYPE WMO_MEAN WMO_WEIGHTED_MEAN
> GALWEM10_P25: FULL ME >=-60|| Z2 000000
> 20191227_000000 1 -0.46691 NA NA NA
NA
> NA NA NA -0.46691 -0.46691 -0.46691 -0.46691
> -0.46691 -0.46691 -0.46691 0 0 MEAN -0.46691
> -0.46691
> GALWEM10_P25: FULL RMSE >=-60|| Z2 000000
> 20191227_000000 1 2.61733 NA NA NA
NA
> NA NA NA 2.61733 2.61733 2.61733 2.61733
> 2.61733 2.61733 2.61733 0 0 SQRT 2.61733
> 2.61733
>
> So the question I have, is the order of statistics in the COLUMN
field
> arbitrary or something else like alphabetical? As I parse these
lines, how
> can predict the order of the statistics?
>
>
> Thanks
>
> Bob
>
>
------------------------------------------------
More information about the Met_help
mailing list