[Met_help] [rt.rap.ucar.edu #94238] History for Running into some issues with MET stat_analysis

Wed Apr 1 11:04:10 MDT 2020

----------------------------------------------------------------
  Initial Request
----------------------------------------------------------------

MET help staff,

I'm a new MET user and hoped you could point out any errors in my command lines.

Using version 8.1.2, I followed the MET tutorial online. My questions are in regard to using stat_analysis.

First,
I am using the stat_analysis tool as described in the Tutorial<https://dtcenter.org/met-online-tutorial-metv8-0/stat-analysis/run>. I made a simple change to perform a summary job instead of the aggregate job.

stat_analysis \
-lookin $MET_TUTORIAL_DATA/output/point_stat \
-out_stat $MET_TUTORIAL_DATA/output/stat_analysis/cnt_RMSE_set_hdr.stat \
-job summary -line_type CNT -column RMSE -fcst_var TMP -fcst_lev P850-750 -obtype ADPUPA -vx_mask EAST,WEST \
-by INTERP_PNTS -set_hdr VX_MASK CONUS \
-v 2

But the file created by -out_stat is empty and all the data I expected to be in that new file is written to the terminal. Is this a bug, or is there another setting I need to set to write the data to the file instead of the terminal screen?

Second,
I am trying to aggregate RMSE statistics between multiple model runs so I can compare different experiments. I thought the stat_analysis aggregate job would do this, but when I try to use the aggregate function on the CNT line type, it says I have to specify a different line type. Perhaps the method is to aggregate the SL1L2 line types and derive the average RMSE from the mean for all the model runs. Is that the best method?

Thanks for your time. I am impressed with MET's capability and wish I would have learned to use it a few years ago.

Dr. Brian K. Blaylock
NRC Postdoctoral Research Associate
Marine Meteorology Division
U.S. Naval Research Laboratory

----------------------------------------------------------------
  Complete Ticket History
----------------------------------------------------------------

Subject: Running into some issues with MET stat_analysis
From: Julie Prestopnik
Time: Wed Feb 26 10:52:15 2020

Hi Dr. Blaylock.

Thank you for writing to us with your questions. My apologies for the
very
delayed response.   Usually our response time is much faster.

We are so glad that you have been impressed with MET's capability.
We're
happy to help you with whatever questions arise.

I am assigning this ticket to John Halley Gotway who will likely be
best
able to answer your questions.

Julie

On Wed, Feb 19, 2020 at 2:47 PM Blaylock, Dr. Brian, Contractor, Code
7531
via RT <met_help at ucar.edu> wrote:

>
> Wed Feb 19 14:46:53 2020: Request 94238 was acted upon.
> Transaction: Ticket created by brian.blaylock.ctr at nrlmry.navy.mil
>        Queue: met_help
>      Subject: Running into some issues with MET stat_analysis
>        Owner: Nobody
>   Requestors: brian.blaylock.ctr at nrlmry.navy.mil
>       Status: new
>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94238 >
>
>
> MET help staff,
>
> I'm a new MET user and hoped you could point out any errors in my
command
> lines.
>
> Using version 8.1.2, I followed the MET tutorial online. My
questions are
> in regard to using stat_analysis.
>
> First,
> I am using the stat_analysis tool as described in the Tutorial<
> https://dtcenter.org/met-online-tutorial-metv8-0/stat-analysis/run>.
I
> made a simple change to perform a summary job instead of the
aggregate job.
>
> stat_analysis \
> -lookin $MET_TUTORIAL_DATA/output/point_stat \
> -out_stat
$MET_TUTORIAL_DATA/output/stat_analysis/cnt_RMSE_set_hdr.stat \
> -job summary -line_type CNT -column RMSE -fcst_var TMP -fcst_lev
P850-750
> -obtype ADPUPA -vx_mask EAST,WEST \
> -by INTERP_PNTS -set_hdr VX_MASK CONUS \
> -v 2
>
> But the file created by -out_stat is empty and all the data I
expected to
> be in that new file is written to the terminal. Is this a bug, or is
there
> another setting I need to set to write the data to the file instead
of the
> terminal screen?
>
>
> Second,
> I am trying to aggregate RMSE statistics between multiple model runs
so I
> can compare different experiments. I thought the stat_analysis
aggregate
> job would do this, but when I try to use the aggregate function on
the CNT
> line type, it says I have to specify a different line type. Perhaps
the
> method is to aggregate the SL1L2 line types and derive the average
RMSE
> from the mean for all the model runs. Is that the best method?
>
> Thanks for your time. I am impressed with MET's capability and wish
I
> would have learned to use it a few years ago.
>
> Dr. Brian K. Blaylock
> NRC Postdoctoral Research Associate
> Marine Meteorology Division
> U.S. Naval Research Laboratory
>
>
>

--
Julie Prestopnik
Software Engineer
National Center for Atmospheric Research
Research Applications Laboratory
Phone: 303.497.8399
Email: jpresto at ucar.edu

My working day may not be your working day.  Please do not feel
obliged to
reply to this email outside of your normal working hours.

------------------------------------------------
Subject: Running into some issues with MET stat_analysis
From: John Halley Gotway
Time: Wed Feb 26 11:29:12 2020

Dr. Blalock,

I see that you're using stat_analysis to run the "summary" job type.
You've used the "-out_stat" job command option to redirect output to a
.stat file, but that resulting file is empty.  There is a good reason
why
that is the case.  The short answer is that you should switch from
using
"-out_stat" to "-out" to capture the output of the summary job.  This
redirects the output that's printed to the screen to an output file
instead.

I could go into a long history here explaining how the stat_analysis
options have changed over time... but I'll refrain for now... unless
you'd
like more details.

The short explanation is that the "aggregate" and "aggregate_stat" job
types are the only jobs that write output to the "-out_stat" output
file (I
think).  The output of those jobs is one of the STAT output line types
that
are produced by the other MET tools (like point_stat and grid_stat).
The
"summary" job does not write a STAT output line type... it just writes
a
summary of the distribution of values from a column of data... not an
actual STAT line type.  So it's output does not go the the "-out_stat"
output file.

Sorry, I do realize that this is a bit confusing.

For your second question, you want to compute RMSE aggregated across
multiple runs.  You can do this by running the aggregate_stat job type
with
the SL1L2 line type as input and the CNT line type as output:
   -job aggregate_stat -line_type SL1L2 -out_line_type CNT -out_stat
aggr_sl1l2_to_cnt.stat

Since the output of this job is a CNT STAT line, you can write that to
the
.stat output file using the "-out_stat" job command option.  If you'd
like
to see a list of supported line types for the aggregate job or the
valid
combinations of input/output for the aggregate_stat job, please take a
look
starting at line 2695 of this README file:
   https://github.com/NCAR/MET/blob/master_v8.1/met/data/config/README
This info is also included in the MET User's Guide.

Hope this helps.  If you have additional specific questions, feel free
to
post some sample .stat data to our anonymous ftp site (
https://dtcenter.org/community-code/model-evaluation-tools-met/met-
help-desk#ftp)
with a description of how you'd like to process it.

Thanks,
John Halley Gotway

On Wed, Feb 26, 2020 at 11:01 AM Julie Prestopnik via RT
<met_help at ucar.edu>
wrote:

>
> Wed Feb 26 11:01:24 2020: Request 94238 was acted upon.
> Transaction: Given to johnhg (John Halley Gotway) by jpresto
>        Queue: met_help
>      Subject: Running into some issues with MET stat_analysis
>        Owner: johnhg
>   Requestors: brian.blaylock.ctr at nrlmry.navy.mil
>       Status: open
>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94238 >
>
>
> This transaction appears to have no content
>

------------------------------------------------