[Met_help] [rt.rap.ucar.edu #83912] History for How to calculate the time_mean statistic with MET

John Halley Gotway via RT met_help at ucar.edu
Tue Feb 6 12:01:41 MST 2018


----------------------------------------------------------------
  Initial Request
----------------------------------------------------------------

Zhang, Mandy has shared OneDrive for Business files with you. To view them, click the links below.


<https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-1_ou_edu/EQyLf-7Dh0lEqg5GMGrk8I4Ba-fB7yRBAtGEPz2d66qsyw>
[https://r1.res.office365.com/owa/prem/images/dc-generic_20.png]<https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-1_ou_edu/EQyLf-7Dh0lEqg5GMGrk8I4Ba-fB7yRBAtGEPz2d66qsyw>

filter_job.stat<https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-1_ou_edu/EQyLf-7Dh0lEqg5GMGrk8I4Ba-fB7yRBAtGEPz2d66qsyw>

<https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-1_ou_edu/EdqUHTs3orlHsYfgVU_lJCsB-Aq9TwBiu_jPJ_pI5fu3Gg>
[https://r1.res.office365.com/owa/prem/images/dc-generic_20.png]<https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-1_ou_edu/EdqUHTs3orlHsYfgVU_lJCsB-Aq9TwBiu_jPJ_pI5fu3Gg>

test_stat_analysis.sh<https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-1_ou_edu/EdqUHTs3orlHsYfgVU_lJCsB-Aq9TwBiu_jPJ_pI5fu3Gg>




Hi All,

     I use MET point_stat and ensemble_stat to generate forecast evaluation at certain forecast time and restore the results with multiple times at different directories. Now I want to get time_mean evaluation, from the user guide, I try the Stat_analysis tool, but my output results looks it just merge several  files into one file.

    Could anyone give me a guide?

    And I attach my results and running scripts.

Thank you very much!

Mandy


----------------------------------------------------------------
  Complete Ticket History
----------------------------------------------------------------

Subject: How to calculate the time_mean statistic with MET
From: John Halley Gotway
Time: Mon Feb 05 13:47:37 2018

Mandy,

The stat-analysis tool can be run either with or without a config
file.  If
it is run with a config file, then all of the "jobs" listed in that
config
file are executed... and any jobs listed on the command line are
ignored.
If it is run without a config file, then exactly one job must be
defined on
the command line.

In the script you sent, I see that you specified:
   -config STATAnalysisConfig_default

And looking in the STATAnalysisConfig_default distributed with MET, I
see
the following job listed:
jobs = [
   "-job filter -dump_row ./filter_job.stat"
];

Because you passed in that config file, STAT-Analysis ran that single
filter job.  So that's the source of the confusion!

Please try rerunning your test_stat_analysis.sh script but remove the
"-config STATAnalysisConfig_default" option.

I'll take a look at the code to see if I can easily add a warning
message
when the user specifies a job on the command line and uses a config
file.

Hope that helps clarify.

Thanks,
John

On Thu, Feb 1, 2018 at 11:08 AM, Zhang, Mandy via RT
<met_help at ucar.edu>
wrote:

>
> Thu Feb 01 11:08:51 2018: Request 83912 was acted upon.
> Transaction: Ticket created by Man.Zhang-1 at ou.edu
>        Queue: met_help
>      Subject: How to calculate the time_mean statistic with MET
>        Owner: Nobody
>   Requestors: Man.Zhang-1 at ou.edu
>       Status: new
>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83912 >
>
>
> Zhang, Mandy has shared OneDrive for Business files with you. To
view
> them, click the links below.
>
>
> <https://sooners-my.sharepoint.com/:u:/g/personal/
> man_zhang-1_ou_edu/EQyLf-7Dh0lEqg5GMGrk8I4Ba-fB7yRBAtGEPz2d66qsyw>
> [https://r1.res.office365.com/owa/prem/images/dc-generic_20.png]<
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EQyLf-
> 7Dh0lEqg5GMGrk8I4Ba-fB7yRBAtGEPz2d66qsyw>
>
> filter_job.stat<https://sooners-my.sharepoint.com/:u:/
> g/personal/man_zhang-1_ou_edu/EQyLf-7Dh0lEqg5GMGrk8I4Ba-
> fB7yRBAtGEPz2d66qsyw>
>
> <https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/
> EdqUHTs3orlHsYfgVU_lJCsB-Aq9TwBiu_jPJ_pI5fu3Gg>
> [https://r1.res.office365.com/owa/prem/images/dc-generic_20.png]<
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-1_ou_edu/
> EdqUHTs3orlHsYfgVU_lJCsB-Aq9TwBiu_jPJ_pI5fu3Gg>
>
> test_stat_analysis.sh<https://sooners-my.sharepoint.com/:u:/
> g/personal/man_zhang-1_ou_edu/EdqUHTs3orlHsYfgVU_lJCsB-
> Aq9TwBiu_jPJ_pI5fu3Gg>
>
>
>
>
> Hi All,
>
>      I use MET point_stat and ensemble_stat to generate forecast
> evaluation at certain forecast time and restore the results with
multiple
> times at different directories. Now I want to get time_mean
evaluation,
> from the user guide, I try the Stat_analysis tool, but my output
results
> looks it just merge several  files into one file.
>
>     Could anyone give me a guide?
>
>     And I attach my results and running scripts.
>
> Thank you very much!
>
> Mandy
>
>

------------------------------------------------
Subject: How to calculate the time_mean statistic with MET
From: Zhang, Mandy
Time: Tue Feb 06 10:07:15 2018

Hi John,

     It works by removing the default configure_file.

My question now is I restore every day met_point_stat run in different
directory as: metenm_test/ensm_stat_sfc/met/20180109/0600h and
metenm_test/ensm_stat_sfc/met/20180110/06h...

and the output file name is
:point_stat_sfcuvt_000000L_20180109_160000V.stat and
point_stat_sfcuvt_000000L_20180110_160000V.stat


and if I want to do monthly_mean summary, do I need to write 30 times
look_in in run_scripts like:

  -lookin /home-
gs/users/SZQXG152/metenm_test/ensm_stat_sfc/met/20180109/0600/point_stat/
\
   -lookin /home-
gs/users/SZQXG152/metenm_test/ensm_stat_sfc/met/20180109/0600/point_stat/
\
   -lookin /home-
gs/users/SZQXG152/metenm_test/ensm_stat_sfc/met/20180109/0600/point_stat
......

Thank you very much!
Mandy


________________________________
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Monday, February 5, 2018 2:47:37 PM
To: Zhang, Mandy
Subject: Re: [rt.rap.ucar.edu #83912] How to calculate the time_mean
statistic with MET

Mandy,

The stat-analysis tool can be run either with or without a config
file.  If
it is run with a config file, then all of the "jobs" listed in that
config
file are executed... and any jobs listed on the command line are
ignored.
If it is run without a config file, then exactly one job must be
defined on
the command line.

In the script you sent, I see that you specified:
   -config STATAnalysisConfig_default

And looking in the STATAnalysisConfig_default distributed with MET, I
see
the following job listed:
jobs = [
   "-job filter -dump_row ./filter_job.stat"
];

Because you passed in that config file, STAT-Analysis ran that single
filter job.  So that's the source of the confusion!

Please try rerunning your test_stat_analysis.sh script but remove the
"-config STATAnalysisConfig_default" option.

I'll take a look at the code to see if I can easily add a warning
message
when the user specifies a job on the command line and uses a config
file.

Hope that helps clarify.

Thanks,
John

On Thu, Feb 1, 2018 at 11:08 AM, Zhang, Mandy via RT
<met_help at ucar.edu>
wrote:

>
> Thu Feb 01 11:08:51 2018: Request 83912 was acted upon.
> Transaction: Ticket created by Man.Zhang-1 at ou.edu
>        Queue: met_help
>      Subject: How to calculate the time_mean statistic with MET
>        Owner: Nobody
>   Requestors: Man.Zhang-1 at ou.edu
>       Status: new
>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83912 >
>
>
> Zhang, Mandy has shared OneDrive for Business files with you. To
view
> them, click the links below.
>
>
> <https://sooners-my.sharepoint.com/:u:/g/personal/
> man_zhang-1_ou_edu/EQyLf-7Dh0lEqg5GMGrk8I4Ba-fB7yRBAtGEPz2d66qsyw>
> [https://r1.res.office365.com/owa/prem/images/dc-generic_20.png]<
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EQyLf-
> 7Dh0lEqg5GMGrk8I4Ba-fB7yRBAtGEPz2d66qsyw>
>
> filter_job.stat<https://sooners-my.sharepoint.com/:u:/
> g/personal/man_zhang-1_ou_edu/EQyLf-7Dh0lEqg5GMGrk8I4Ba-
> fB7yRBAtGEPz2d66qsyw>
>
> <https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/
> EdqUHTs3orlHsYfgVU_lJCsB-Aq9TwBiu_jPJ_pI5fu3Gg>
> [https://r1.res.office365.com/owa/prem/images/dc-generic_20.png]<
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-1_ou_edu/
> EdqUHTs3orlHsYfgVU_lJCsB-Aq9TwBiu_jPJ_pI5fu3Gg>
>
> test_stat_analysis.sh<https://sooners-my.sharepoint.com/:u:/
> g/personal/man_zhang-1_ou_edu/EdqUHTs3orlHsYfgVU_lJCsB-
> Aq9TwBiu_jPJ_pI5fu3Gg>
>
>
>
>
> Hi All,
>
>      I use MET point_stat and ensemble_stat to generate forecast
> evaluation at certain forecast time and restore the results with
multiple
> times at different directories. Now I want to get time_mean
evaluation,
> from the user guide, I try the Stat_analysis tool, but my output
results
> looks it just merge several  files into one file.
>
>     Could anyone give me a guide?
>
>     And I attach my results and running scripts.
>
> Thank you very much!
>
> Mandy
>
>


------------------------------------------------
Subject: How to calculate the time_mean statistic with MET
From: John Halley Gotway
Time: Tue Feb 06 10:17:08 2018

Mandy,

There are two ways you could do this.  You can decide which you
prefer.

(1) The -lookin option takes one OR MORE arguments.  Rather then using
-lookin 30 times to specify directories, you can just use it once with
wildcards:
     -lookin /home-gs/users/SZQXG152/metenm_test/ensm_stat_sfc/met/
201801*/0600/point_stat/

(2) As you know, if you set -lookin to a top-level directory, MET will
search recursively through that directory looking for files ending
with
.stat.  You could just pass in the top-level directory of your output
and
then use job command options to filter the data:
   -lookin /home-gs/users/SZQXG152/metenm_test/ensm_stat_sfc/met
-fcst_init_beg 20180101_00 -fcst_init_end 20180131_23

That'll read in ALL of the .stat files for the job but only use the
data
whose initialization time was between 20180101_00 and 201801321_23.

Since option (2) is reads a lot more data than option (1), my guess is
that
option (1) will run faster.

Hopefully that helps clarify.

Thanks,
John


On Tue, Feb 6, 2018 at 10:07 AM, Zhang, Mandy via RT
<met_help at ucar.edu>
wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83912 >
>
> Hi John,
>
>      It works by removing the default configure_file.
>
> My question now is I restore every day met_point_stat run in
different
> directory as: metenm_test/ensm_stat_sfc/met/20180109/0600h and
> metenm_test/ensm_stat_sfc/met/20180110/06h...
>
> and the output file name is
:point_stat_sfcuvt_000000L_20180109_160000V.stat
> and point_stat_sfcuvt_000000L_20180110_160000V.stat
>
>
> and if I want to do monthly_mean summary, do I need to write 30
times
> look_in in run_scripts like:
>
>   -lookin /home-
gs/users/SZQXG152/metenm_test/ensm_stat_sfc/met/20180109/0600/point_stat/
> \
>    -lookin /home-
gs/users/SZQXG152/metenm_test/ensm_stat_sfc/met/20180109/0600/point_stat/
> \
>    -lookin /home-
gs/users/SZQXG152/metenm_test/ensm_stat_sfc/met/20180109/0600/point_stat
> ......
>
> Thank you very much!
> Mandy
>
>
> ________________________________
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Monday, February 5, 2018 2:47:37 PM
> To: Zhang, Mandy
> Subject: Re: [rt.rap.ucar.edu #83912] How to calculate the time_mean
> statistic with MET
>
> Mandy,
>
> The stat-analysis tool can be run either with or without a config
file.  If
> it is run with a config file, then all of the "jobs" listed in that
config
> file are executed... and any jobs listed on the command line are
ignored.
> If it is run without a config file, then exactly one job must be
defined on
> the command line.
>
> In the script you sent, I see that you specified:
>    -config STATAnalysisConfig_default
>
> And looking in the STATAnalysisConfig_default distributed with MET,
I see
> the following job listed:
> jobs = [
>    "-job filter -dump_row ./filter_job.stat"
> ];
>
> Because you passed in that config file, STAT-Analysis ran that
single
> filter job.  So that's the source of the confusion!
>
> Please try rerunning your test_stat_analysis.sh script but remove
the
> "-config STATAnalysisConfig_default" option.
>
> I'll take a look at the code to see if I can easily add a warning
message
> when the user specifies a job on the command line and uses a config
file.
>
> Hope that helps clarify.
>
> Thanks,
> John
>
> On Thu, Feb 1, 2018 at 11:08 AM, Zhang, Mandy via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > Thu Feb 01 11:08:51 2018: Request 83912 was acted upon.
> > Transaction: Ticket created by Man.Zhang-1 at ou.edu
> >        Queue: met_help
> >      Subject: How to calculate the time_mean statistic with MET
> >        Owner: Nobody
> >   Requestors: Man.Zhang-1 at ou.edu
> >       Status: new
> >  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83912 >
> >
> >
> > Zhang, Mandy has shared OneDrive for Business files with you. To
view
> > them, click the links below.
> >
> >
> > <https://sooners-my.sharepoint.com/:u:/g/personal/
> > man_zhang-1_ou_edu/EQyLf-7Dh0lEqg5GMGrk8I4Ba-fB7yRBAtGEPz2d66qsyw>
> > [https://r1.res.office365.com/owa/prem/images/dc-generic_20.png]<
> > https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
> 1_ou_edu/EQyLf-
> > 7Dh0lEqg5GMGrk8I4Ba-fB7yRBAtGEPz2d66qsyw>
> >
> > filter_job.stat<https://sooners-my.sharepoint.com/:u:/
> > g/personal/man_zhang-1_ou_edu/EQyLf-7Dh0lEqg5GMGrk8I4Ba-
> > fB7yRBAtGEPz2d66qsyw>
> >
> > <https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/
> > EdqUHTs3orlHsYfgVU_lJCsB-Aq9TwBiu_jPJ_pI5fu3Gg>
> > [https://r1.res.office365.com/owa/prem/images/dc-generic_20.png]<
> > https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/
> > EdqUHTs3orlHsYfgVU_lJCsB-Aq9TwBiu_jPJ_pI5fu3Gg>
> >
> > test_stat_analysis.sh<https://sooners-my.sharepoint.com/:u:/
> > g/personal/man_zhang-1_ou_edu/EdqUHTs3orlHsYfgVU_lJCsB-
> > Aq9TwBiu_jPJ_pI5fu3Gg>
> >
> >
> >
> >
> > Hi All,
> >
> >      I use MET point_stat and ensemble_stat to generate forecast
> > evaluation at certain forecast time and restore the results with
multiple
> > times at different directories. Now I want to get time_mean
evaluation,
> > from the user guide, I try the Stat_analysis tool, but my output
results
> > looks it just merge several  files into one file.
> >
> >     Could anyone give me a guide?
> >
> >     And I attach my results and running scripts.
> >
> > Thank you very much!
> >
> > Mandy
> >
> >
>
>
>

------------------------------------------------
Subject: How to calculate the time_mean statistic with MET
From: Zhang, Mandy
Time: Tue Feb 06 10:36:49 2018

Hi John,

   It is very clear!

Thank you again!

Mandy

________________________________
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Tuesday, February 6, 2018 11:17:08 AM
To: Zhang, Mandy
Subject: Re: [rt.rap.ucar.edu #83912] How to calculate the time_mean
statistic with MET

Mandy,

There are two ways you could do this.  You can decide which you
prefer.

(1) The -lookin option takes one OR MORE arguments.  Rather then using
-lookin 30 times to specify directories, you can just use it once with
wildcards:
     -lookin /home-gs/users/SZQXG152/metenm_test/ensm_stat_sfc/met/
201801*/0600/point_stat/

(2) As you know, if you set -lookin to a top-level directory, MET will
search recursively through that directory looking for files ending
with
.stat.  You could just pass in the top-level directory of your output
and
then use job command options to filter the data:
   -lookin /home-gs/users/SZQXG152/metenm_test/ensm_stat_sfc/met
-fcst_init_beg 20180101_00 -fcst_init_end 20180131_23

That'll read in ALL of the .stat files for the job but only use the
data
whose initialization time was between 20180101_00 and 201801321_23.

Since option (2) is reads a lot more data than option (1), my guess is
that
option (1) will run faster.

Hopefully that helps clarify.

Thanks,
John


On Tue, Feb 6, 2018 at 10:07 AM, Zhang, Mandy via RT
<met_help at ucar.edu>
wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83912 >
>
> Hi John,
>
>      It works by removing the default configure_file.
>
> My question now is I restore every day met_point_stat run in
different
> directory as: metenm_test/ensm_stat_sfc/met/20180109/0600h and
> metenm_test/ensm_stat_sfc/met/20180110/06h...
>
> and the output file name is
:point_stat_sfcuvt_000000L_20180109_160000V.stat
> and point_stat_sfcuvt_000000L_20180110_160000V.stat
>
>
> and if I want to do monthly_mean summary, do I need to write 30
times
> look_in in run_scripts like:
>
>   -lookin /home-
gs/users/SZQXG152/metenm_test/ensm_stat_sfc/met/20180109/0600/point_stat/
> \
>    -lookin /home-
gs/users/SZQXG152/metenm_test/ensm_stat_sfc/met/20180109/0600/point_stat/
> \
>    -lookin /home-
gs/users/SZQXG152/metenm_test/ensm_stat_sfc/met/20180109/0600/point_stat
> ......
>
> Thank you very much!
> Mandy
>
>
> ________________________________
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Monday, February 5, 2018 2:47:37 PM
> To: Zhang, Mandy
> Subject: Re: [rt.rap.ucar.edu #83912] How to calculate the time_mean
> statistic with MET
>
> Mandy,
>
> The stat-analysis tool can be run either with or without a config
file.  If
> it is run with a config file, then all of the "jobs" listed in that
config
> file are executed... and any jobs listed on the command line are
ignored.
> If it is run without a config file, then exactly one job must be
defined on
> the command line.
>
> In the script you sent, I see that you specified:
>    -config STATAnalysisConfig_default
>
> And looking in the STATAnalysisConfig_default distributed with MET,
I see
> the following job listed:
> jobs = [
>    "-job filter -dump_row ./filter_job.stat"
> ];
>
> Because you passed in that config file, STAT-Analysis ran that
single
> filter job.  So that's the source of the confusion!
>
> Please try rerunning your test_stat_analysis.sh script but remove
the
> "-config STATAnalysisConfig_default" option.
>
> I'll take a look at the code to see if I can easily add a warning
message
> when the user specifies a job on the command line and uses a config
file.
>
> Hope that helps clarify.
>
> Thanks,
> John
>
> On Thu, Feb 1, 2018 at 11:08 AM, Zhang, Mandy via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > Thu Feb 01 11:08:51 2018: Request 83912 was acted upon.
> > Transaction: Ticket created by Man.Zhang-1 at ou.edu
> >        Queue: met_help
> >      Subject: How to calculate the time_mean statistic with MET
> >        Owner: Nobody
> >   Requestors: Man.Zhang-1 at ou.edu
> >       Status: new
> >  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83912 >
> >
> >
> > Zhang, Mandy has shared OneDrive for Business files with you. To
view
> > them, click the links below.
> >
> >
> > <https://sooners-my.sharepoint.com/:u:/g/personal/
> > man_zhang-1_ou_edu/EQyLf-7Dh0lEqg5GMGrk8I4Ba-fB7yRBAtGEPz2d66qsyw>
> > [https://r1.res.office365.com/owa/prem/images/dc-generic_20.png]<
> > https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
> 1_ou_edu/EQyLf-
> > 7Dh0lEqg5GMGrk8I4Ba-fB7yRBAtGEPz2d66qsyw>
> >
> > filter_job.stat<https://sooners-my.sharepoint.com/:u:/
> > g/personal/man_zhang-1_ou_edu/EQyLf-7Dh0lEqg5GMGrk8I4Ba-
> > fB7yRBAtGEPz2d66qsyw>
> >
> > <https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/
> > EdqUHTs3orlHsYfgVU_lJCsB-Aq9TwBiu_jPJ_pI5fu3Gg>
> > [https://r1.res.office365.com/owa/prem/images/dc-generic_20.png]<
> > https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/
> > EdqUHTs3orlHsYfgVU_lJCsB-Aq9TwBiu_jPJ_pI5fu3Gg>
> >
> > test_stat_analysis.sh<https://sooners-my.sharepoint.com/:u:/
> > g/personal/man_zhang-1_ou_edu/EdqUHTs3orlHsYfgVU_lJCsB-
> > Aq9TwBiu_jPJ_pI5fu3Gg>
> >
> >
> >
> >
> > Hi All,
> >
> >      I use MET point_stat and ensemble_stat to generate forecast
> > evaluation at certain forecast time and restore the results with
multiple
> > times at different directories. Now I want to get time_mean
evaluation,
> > from the user guide, I try the Stat_analysis tool, but my output
results
> > looks it just merge several  files into one file.
> >
> >     Could anyone give me a guide?
> >
> >     And I attach my results and running scripts.
> >
> > Thank you very much!
> >
> > Mandy
> >
> >
>
>
>


------------------------------------------------


More information about the Met_help mailing list