[Met_help] [rt.rap.ucar.edu #83912] History for How to calculate the time_mean statistic with MET
John Halley Gotway via RT
met_help at ucar.edu
Tue Feb 6 12:01:41 MST 2018
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Zhang, Mandy has shared OneDrive for Business files with you. To view them, click the links below.
<https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-1_ou_edu/EQyLf-7Dh0lEqg5GMGrk8I4Ba-fB7yRBAtGEPz2d66qsyw>
[https://r1.res.office365.com/owa/prem/images/dc-generic_20.png]<https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-1_ou_edu/EQyLf-7Dh0lEqg5GMGrk8I4Ba-fB7yRBAtGEPz2d66qsyw>
filter_job.stat<https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-1_ou_edu/EQyLf-7Dh0lEqg5GMGrk8I4Ba-fB7yRBAtGEPz2d66qsyw>
<https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-1_ou_edu/EdqUHTs3orlHsYfgVU_lJCsB-Aq9TwBiu_jPJ_pI5fu3Gg>
[https://r1.res.office365.com/owa/prem/images/dc-generic_20.png]<https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-1_ou_edu/EdqUHTs3orlHsYfgVU_lJCsB-Aq9TwBiu_jPJ_pI5fu3Gg>
test_stat_analysis.sh<https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-1_ou_edu/EdqUHTs3orlHsYfgVU_lJCsB-Aq9TwBiu_jPJ_pI5fu3Gg>
Hi All,
I use MET point_stat and ensemble_stat to generate forecast evaluation at certain forecast time and restore the results with multiple times at different directories. Now I want to get time_mean evaluation, from the user guide, I try the Stat_analysis tool, but my output results looks it just merge several files into one file.
Could anyone give me a guide?
And I attach my results and running scripts.
Thank you very much!
Mandy
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: How to calculate the time_mean statistic with MET
From: John Halley Gotway
Time: Mon Feb 05 13:47:37 2018
Mandy,
The stat-analysis tool can be run either with or without a config
file. If
it is run with a config file, then all of the "jobs" listed in that
config
file are executed... and any jobs listed on the command line are
ignored.
If it is run without a config file, then exactly one job must be
defined on
the command line.
In the script you sent, I see that you specified:
-config STATAnalysisConfig_default
And looking in the STATAnalysisConfig_default distributed with MET, I
see
the following job listed:
jobs = [
"-job filter -dump_row ./filter_job.stat"
];
Because you passed in that config file, STAT-Analysis ran that single
filter job. So that's the source of the confusion!
Please try rerunning your test_stat_analysis.sh script but remove the
"-config STATAnalysisConfig_default" option.
I'll take a look at the code to see if I can easily add a warning
message
when the user specifies a job on the command line and uses a config
file.
Hope that helps clarify.
Thanks,
John
On Thu, Feb 1, 2018 at 11:08 AM, Zhang, Mandy via RT
<met_help at ucar.edu>
wrote:
>
> Thu Feb 01 11:08:51 2018: Request 83912 was acted upon.
> Transaction: Ticket created by Man.Zhang-1 at ou.edu
> Queue: met_help
> Subject: How to calculate the time_mean statistic with MET
> Owner: Nobody
> Requestors: Man.Zhang-1 at ou.edu
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83912 >
>
>
> Zhang, Mandy has shared OneDrive for Business files with you. To
view
> them, click the links below.
>
>
> <https://sooners-my.sharepoint.com/:u:/g/personal/
> man_zhang-1_ou_edu/EQyLf-7Dh0lEqg5GMGrk8I4Ba-fB7yRBAtGEPz2d66qsyw>
> [https://r1.res.office365.com/owa/prem/images/dc-generic_20.png]<
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EQyLf-
> 7Dh0lEqg5GMGrk8I4Ba-fB7yRBAtGEPz2d66qsyw>
>
> filter_job.stat<https://sooners-my.sharepoint.com/:u:/
> g/personal/man_zhang-1_ou_edu/EQyLf-7Dh0lEqg5GMGrk8I4Ba-
> fB7yRBAtGEPz2d66qsyw>
>
> <https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/
> EdqUHTs3orlHsYfgVU_lJCsB-Aq9TwBiu_jPJ_pI5fu3Gg>
> [https://r1.res.office365.com/owa/prem/images/dc-generic_20.png]<
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-1_ou_edu/
> EdqUHTs3orlHsYfgVU_lJCsB-Aq9TwBiu_jPJ_pI5fu3Gg>
>
> test_stat_analysis.sh<https://sooners-my.sharepoint.com/:u:/
> g/personal/man_zhang-1_ou_edu/EdqUHTs3orlHsYfgVU_lJCsB-
> Aq9TwBiu_jPJ_pI5fu3Gg>
>
>
>
>
> Hi All,
>
> I use MET point_stat and ensemble_stat to generate forecast
> evaluation at certain forecast time and restore the results with
multiple
> times at different directories. Now I want to get time_mean
evaluation,
> from the user guide, I try the Stat_analysis tool, but my output
results
> looks it just merge several files into one file.
>
> Could anyone give me a guide?
>
> And I attach my results and running scripts.
>
> Thank you very much!
>
> Mandy
>
>
------------------------------------------------
Subject: How to calculate the time_mean statistic with MET
From: Zhang, Mandy
Time: Tue Feb 06 10:07:15 2018
Hi John,
It works by removing the default configure_file.
My question now is I restore every day met_point_stat run in different
directory as: metenm_test/ensm_stat_sfc/met/20180109/0600h and
metenm_test/ensm_stat_sfc/met/20180110/06h...
and the output file name is
:point_stat_sfcuvt_000000L_20180109_160000V.stat and
point_stat_sfcuvt_000000L_20180110_160000V.stat
and if I want to do monthly_mean summary, do I need to write 30 times
look_in in run_scripts like:
-lookin /home-
gs/users/SZQXG152/metenm_test/ensm_stat_sfc/met/20180109/0600/point_stat/
\
-lookin /home-
gs/users/SZQXG152/metenm_test/ensm_stat_sfc/met/20180109/0600/point_stat/
\
-lookin /home-
gs/users/SZQXG152/metenm_test/ensm_stat_sfc/met/20180109/0600/point_stat
......
Thank you very much!
Mandy
________________________________
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Monday, February 5, 2018 2:47:37 PM
To: Zhang, Mandy
Subject: Re: [rt.rap.ucar.edu #83912] How to calculate the time_mean
statistic with MET
Mandy,
The stat-analysis tool can be run either with or without a config
file. If
it is run with a config file, then all of the "jobs" listed in that
config
file are executed... and any jobs listed on the command line are
ignored.
If it is run without a config file, then exactly one job must be
defined on
the command line.
In the script you sent, I see that you specified:
-config STATAnalysisConfig_default
And looking in the STATAnalysisConfig_default distributed with MET, I
see
the following job listed:
jobs = [
"-job filter -dump_row ./filter_job.stat"
];
Because you passed in that config file, STAT-Analysis ran that single
filter job. So that's the source of the confusion!
Please try rerunning your test_stat_analysis.sh script but remove the
"-config STATAnalysisConfig_default" option.
I'll take a look at the code to see if I can easily add a warning
message
when the user specifies a job on the command line and uses a config
file.
Hope that helps clarify.
Thanks,
John
On Thu, Feb 1, 2018 at 11:08 AM, Zhang, Mandy via RT
<met_help at ucar.edu>
wrote:
>
> Thu Feb 01 11:08:51 2018: Request 83912 was acted upon.
> Transaction: Ticket created by Man.Zhang-1 at ou.edu
> Queue: met_help
> Subject: How to calculate the time_mean statistic with MET
> Owner: Nobody
> Requestors: Man.Zhang-1 at ou.edu
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83912 >
>
>
> Zhang, Mandy has shared OneDrive for Business files with you. To
view
> them, click the links below.
>
>
> <https://sooners-my.sharepoint.com/:u:/g/personal/
> man_zhang-1_ou_edu/EQyLf-7Dh0lEqg5GMGrk8I4Ba-fB7yRBAtGEPz2d66qsyw>
> [https://r1.res.office365.com/owa/prem/images/dc-generic_20.png]<
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/EQyLf-
> 7Dh0lEqg5GMGrk8I4Ba-fB7yRBAtGEPz2d66qsyw>
>
> filter_job.stat<https://sooners-my.sharepoint.com/:u:/
> g/personal/man_zhang-1_ou_edu/EQyLf-7Dh0lEqg5GMGrk8I4Ba-
> fB7yRBAtGEPz2d66qsyw>
>
> <https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/
> EdqUHTs3orlHsYfgVU_lJCsB-Aq9TwBiu_jPJ_pI5fu3Gg>
> [https://r1.res.office365.com/owa/prem/images/dc-generic_20.png]<
> https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-1_ou_edu/
> EdqUHTs3orlHsYfgVU_lJCsB-Aq9TwBiu_jPJ_pI5fu3Gg>
>
> test_stat_analysis.sh<https://sooners-my.sharepoint.com/:u:/
> g/personal/man_zhang-1_ou_edu/EdqUHTs3orlHsYfgVU_lJCsB-
> Aq9TwBiu_jPJ_pI5fu3Gg>
>
>
>
>
> Hi All,
>
> I use MET point_stat and ensemble_stat to generate forecast
> evaluation at certain forecast time and restore the results with
multiple
> times at different directories. Now I want to get time_mean
evaluation,
> from the user guide, I try the Stat_analysis tool, but my output
results
> looks it just merge several files into one file.
>
> Could anyone give me a guide?
>
> And I attach my results and running scripts.
>
> Thank you very much!
>
> Mandy
>
>
------------------------------------------------
Subject: How to calculate the time_mean statistic with MET
From: John Halley Gotway
Time: Tue Feb 06 10:17:08 2018
Mandy,
There are two ways you could do this. You can decide which you
prefer.
(1) The -lookin option takes one OR MORE arguments. Rather then using
-lookin 30 times to specify directories, you can just use it once with
wildcards:
-lookin /home-gs/users/SZQXG152/metenm_test/ensm_stat_sfc/met/
201801*/0600/point_stat/
(2) As you know, if you set -lookin to a top-level directory, MET will
search recursively through that directory looking for files ending
with
.stat. You could just pass in the top-level directory of your output
and
then use job command options to filter the data:
-lookin /home-gs/users/SZQXG152/metenm_test/ensm_stat_sfc/met
-fcst_init_beg 20180101_00 -fcst_init_end 20180131_23
That'll read in ALL of the .stat files for the job but only use the
data
whose initialization time was between 20180101_00 and 201801321_23.
Since option (2) is reads a lot more data than option (1), my guess is
that
option (1) will run faster.
Hopefully that helps clarify.
Thanks,
John
On Tue, Feb 6, 2018 at 10:07 AM, Zhang, Mandy via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83912 >
>
> Hi John,
>
> It works by removing the default configure_file.
>
> My question now is I restore every day met_point_stat run in
different
> directory as: metenm_test/ensm_stat_sfc/met/20180109/0600h and
> metenm_test/ensm_stat_sfc/met/20180110/06h...
>
> and the output file name is
:point_stat_sfcuvt_000000L_20180109_160000V.stat
> and point_stat_sfcuvt_000000L_20180110_160000V.stat
>
>
> and if I want to do monthly_mean summary, do I need to write 30
times
> look_in in run_scripts like:
>
> -lookin /home-
gs/users/SZQXG152/metenm_test/ensm_stat_sfc/met/20180109/0600/point_stat/
> \
> -lookin /home-
gs/users/SZQXG152/metenm_test/ensm_stat_sfc/met/20180109/0600/point_stat/
> \
> -lookin /home-
gs/users/SZQXG152/metenm_test/ensm_stat_sfc/met/20180109/0600/point_stat
> ......
>
> Thank you very much!
> Mandy
>
>
> ________________________________
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Monday, February 5, 2018 2:47:37 PM
> To: Zhang, Mandy
> Subject: Re: [rt.rap.ucar.edu #83912] How to calculate the time_mean
> statistic with MET
>
> Mandy,
>
> The stat-analysis tool can be run either with or without a config
file. If
> it is run with a config file, then all of the "jobs" listed in that
config
> file are executed... and any jobs listed on the command line are
ignored.
> If it is run without a config file, then exactly one job must be
defined on
> the command line.
>
> In the script you sent, I see that you specified:
> -config STATAnalysisConfig_default
>
> And looking in the STATAnalysisConfig_default distributed with MET,
I see
> the following job listed:
> jobs = [
> "-job filter -dump_row ./filter_job.stat"
> ];
>
> Because you passed in that config file, STAT-Analysis ran that
single
> filter job. So that's the source of the confusion!
>
> Please try rerunning your test_stat_analysis.sh script but remove
the
> "-config STATAnalysisConfig_default" option.
>
> I'll take a look at the code to see if I can easily add a warning
message
> when the user specifies a job on the command line and uses a config
file.
>
> Hope that helps clarify.
>
> Thanks,
> John
>
> On Thu, Feb 1, 2018 at 11:08 AM, Zhang, Mandy via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > Thu Feb 01 11:08:51 2018: Request 83912 was acted upon.
> > Transaction: Ticket created by Man.Zhang-1 at ou.edu
> > Queue: met_help
> > Subject: How to calculate the time_mean statistic with MET
> > Owner: Nobody
> > Requestors: Man.Zhang-1 at ou.edu
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83912 >
> >
> >
> > Zhang, Mandy has shared OneDrive for Business files with you. To
view
> > them, click the links below.
> >
> >
> > <https://sooners-my.sharepoint.com/:u:/g/personal/
> > man_zhang-1_ou_edu/EQyLf-7Dh0lEqg5GMGrk8I4Ba-fB7yRBAtGEPz2d66qsyw>
> > [https://r1.res.office365.com/owa/prem/images/dc-generic_20.png]<
> > https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
> 1_ou_edu/EQyLf-
> > 7Dh0lEqg5GMGrk8I4Ba-fB7yRBAtGEPz2d66qsyw>
> >
> > filter_job.stat<https://sooners-my.sharepoint.com/:u:/
> > g/personal/man_zhang-1_ou_edu/EQyLf-7Dh0lEqg5GMGrk8I4Ba-
> > fB7yRBAtGEPz2d66qsyw>
> >
> > <https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/
> > EdqUHTs3orlHsYfgVU_lJCsB-Aq9TwBiu_jPJ_pI5fu3Gg>
> > [https://r1.res.office365.com/owa/prem/images/dc-generic_20.png]<
> > https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/
> > EdqUHTs3orlHsYfgVU_lJCsB-Aq9TwBiu_jPJ_pI5fu3Gg>
> >
> > test_stat_analysis.sh<https://sooners-my.sharepoint.com/:u:/
> > g/personal/man_zhang-1_ou_edu/EdqUHTs3orlHsYfgVU_lJCsB-
> > Aq9TwBiu_jPJ_pI5fu3Gg>
> >
> >
> >
> >
> > Hi All,
> >
> > I use MET point_stat and ensemble_stat to generate forecast
> > evaluation at certain forecast time and restore the results with
multiple
> > times at different directories. Now I want to get time_mean
evaluation,
> > from the user guide, I try the Stat_analysis tool, but my output
results
> > looks it just merge several files into one file.
> >
> > Could anyone give me a guide?
> >
> > And I attach my results and running scripts.
> >
> > Thank you very much!
> >
> > Mandy
> >
> >
>
>
>
------------------------------------------------
Subject: How to calculate the time_mean statistic with MET
From: Zhang, Mandy
Time: Tue Feb 06 10:36:49 2018
Hi John,
It is very clear!
Thank you again!
Mandy
________________________________
From: John Halley Gotway via RT <met_help at ucar.edu>
Sent: Tuesday, February 6, 2018 11:17:08 AM
To: Zhang, Mandy
Subject: Re: [rt.rap.ucar.edu #83912] How to calculate the time_mean
statistic with MET
Mandy,
There are two ways you could do this. You can decide which you
prefer.
(1) The -lookin option takes one OR MORE arguments. Rather then using
-lookin 30 times to specify directories, you can just use it once with
wildcards:
-lookin /home-gs/users/SZQXG152/metenm_test/ensm_stat_sfc/met/
201801*/0600/point_stat/
(2) As you know, if you set -lookin to a top-level directory, MET will
search recursively through that directory looking for files ending
with
.stat. You could just pass in the top-level directory of your output
and
then use job command options to filter the data:
-lookin /home-gs/users/SZQXG152/metenm_test/ensm_stat_sfc/met
-fcst_init_beg 20180101_00 -fcst_init_end 20180131_23
That'll read in ALL of the .stat files for the job but only use the
data
whose initialization time was between 20180101_00 and 201801321_23.
Since option (2) is reads a lot more data than option (1), my guess is
that
option (1) will run faster.
Hopefully that helps clarify.
Thanks,
John
On Tue, Feb 6, 2018 at 10:07 AM, Zhang, Mandy via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83912 >
>
> Hi John,
>
> It works by removing the default configure_file.
>
> My question now is I restore every day met_point_stat run in
different
> directory as: metenm_test/ensm_stat_sfc/met/20180109/0600h and
> metenm_test/ensm_stat_sfc/met/20180110/06h...
>
> and the output file name is
:point_stat_sfcuvt_000000L_20180109_160000V.stat
> and point_stat_sfcuvt_000000L_20180110_160000V.stat
>
>
> and if I want to do monthly_mean summary, do I need to write 30
times
> look_in in run_scripts like:
>
> -lookin /home-
gs/users/SZQXG152/metenm_test/ensm_stat_sfc/met/20180109/0600/point_stat/
> \
> -lookin /home-
gs/users/SZQXG152/metenm_test/ensm_stat_sfc/met/20180109/0600/point_stat/
> \
> -lookin /home-
gs/users/SZQXG152/metenm_test/ensm_stat_sfc/met/20180109/0600/point_stat
> ......
>
> Thank you very much!
> Mandy
>
>
> ________________________________
> From: John Halley Gotway via RT <met_help at ucar.edu>
> Sent: Monday, February 5, 2018 2:47:37 PM
> To: Zhang, Mandy
> Subject: Re: [rt.rap.ucar.edu #83912] How to calculate the time_mean
> statistic with MET
>
> Mandy,
>
> The stat-analysis tool can be run either with or without a config
file. If
> it is run with a config file, then all of the "jobs" listed in that
config
> file are executed... and any jobs listed on the command line are
ignored.
> If it is run without a config file, then exactly one job must be
defined on
> the command line.
>
> In the script you sent, I see that you specified:
> -config STATAnalysisConfig_default
>
> And looking in the STATAnalysisConfig_default distributed with MET,
I see
> the following job listed:
> jobs = [
> "-job filter -dump_row ./filter_job.stat"
> ];
>
> Because you passed in that config file, STAT-Analysis ran that
single
> filter job. So that's the source of the confusion!
>
> Please try rerunning your test_stat_analysis.sh script but remove
the
> "-config STATAnalysisConfig_default" option.
>
> I'll take a look at the code to see if I can easily add a warning
message
> when the user specifies a job on the command line and uses a config
file.
>
> Hope that helps clarify.
>
> Thanks,
> John
>
> On Thu, Feb 1, 2018 at 11:08 AM, Zhang, Mandy via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > Thu Feb 01 11:08:51 2018: Request 83912 was acted upon.
> > Transaction: Ticket created by Man.Zhang-1 at ou.edu
> > Queue: met_help
> > Subject: How to calculate the time_mean statistic with MET
> > Owner: Nobody
> > Requestors: Man.Zhang-1 at ou.edu
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=83912 >
> >
> >
> > Zhang, Mandy has shared OneDrive for Business files with you. To
view
> > them, click the links below.
> >
> >
> > <https://sooners-my.sharepoint.com/:u:/g/personal/
> > man_zhang-1_ou_edu/EQyLf-7Dh0lEqg5GMGrk8I4Ba-fB7yRBAtGEPz2d66qsyw>
> > [https://r1.res.office365.com/owa/prem/images/dc-generic_20.png]<
> > https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
> 1_ou_edu/EQyLf-
> > 7Dh0lEqg5GMGrk8I4Ba-fB7yRBAtGEPz2d66qsyw>
> >
> > filter_job.stat<https://sooners-my.sharepoint.com/:u:/
> > g/personal/man_zhang-1_ou_edu/EQyLf-7Dh0lEqg5GMGrk8I4Ba-
> > fB7yRBAtGEPz2d66qsyw>
> >
> > <https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/
> > EdqUHTs3orlHsYfgVU_lJCsB-Aq9TwBiu_jPJ_pI5fu3Gg>
> > [https://r1.res.office365.com/owa/prem/images/dc-generic_20.png]<
> > https://sooners-my.sharepoint.com/:u:/g/personal/man_zhang-
1_ou_edu/
> > EdqUHTs3orlHsYfgVU_lJCsB-Aq9TwBiu_jPJ_pI5fu3Gg>
> >
> > test_stat_analysis.sh<https://sooners-my.sharepoint.com/:u:/
> > g/personal/man_zhang-1_ou_edu/EdqUHTs3orlHsYfgVU_lJCsB-
> > Aq9TwBiu_jPJ_pI5fu3Gg>
> >
> >
> >
> >
> > Hi All,
> >
> > I use MET point_stat and ensemble_stat to generate forecast
> > evaluation at certain forecast time and restore the results with
multiple
> > times at different directories. Now I want to get time_mean
evaluation,
> > from the user guide, I try the Stat_analysis tool, but my output
results
> > looks it just merge several files into one file.
> >
> > Could anyone give me a guide?
> >
> > And I attach my results and running scripts.
> >
> > Thank you very much!
> >
> > Mandy
> >
> >
>
>
>
------------------------------------------------
More information about the Met_help
mailing list