[Met_help] [rt.rap.ucar.edu #85457] History for ref: Parameters definitions found in config files (track_and_intensity.conf, TCPairsETCConfig, and metplus_runtime.coef)
John Halley Gotway via RT
met_help at ucar.edu
Wed Jul 10 16:39:15 MDT 2019
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Good afternoon,
I have several questions concerning the usages of the following parameters:
1. location --> use_cases/track_and_intensity/track_and_intensity.conf:
1.a lead =
My understanding of the above parameter is referring to lead time ie;
00 12 24 ...
IF lead = "blank" --> does this means that it takes "all" files
times
vs
lead = 00, 06, 12, 18, 24 --> meaning only the 00hr 06hr 12hr 18hr
and 24hr
files are examined. ( ie: 36hr 48hr .... files that may be
present are not consider.
Hence, if I want 00hr 12hr 24hr 36hr ....120hr, should I define "lead"
as
lead = 00 12 24 36 48 ...96 108 120
1.b plot_type =
IF plot_type = "blank" Default plot is boxplot, unless otherwise
indicated. If box plot
is needed in addition to other plots, this needs to be
indicated. So,
plot_type = boxplot, point --> would give me a single
value of the TK_ERR
for a specify hour instead of a series: ie;
108,GFSO,ML0302532015,20150327_120000,TK_ERR,609.49759
108,GFSO,ML0100872015,20150112_000000,TK_ERR,588.38804
108,GFSO,ML0101032015,20150114_120000,TK_ERR,575.01756
plot_type = boxplot, point
would give me something like 108,GFSO,TK_ERR,591
1.c PROCESS_LIST = --> indicates the ush "scripts" to run
Currently, I'm using PROCESS_LIST = TcPairs, TCMPRPlotter and
the results I'm
getting given the file name "TK_ERR_boxplot.log" has the following
format for
forecast hours (12, 24,36,48,60,72,84,96,108,120 hours)
LEAD_HR,AMODEL,STORM_ID,INIT,VARIABLE,OUTLIER
12,GFSO,ML0102532015,20150129_120000,TK_ERR,198.06748
12,GFSO,ML0302252015,20150325_060000,TK_ERR,191.37016
12,GFSO,ML0200452015,20150206_060000,TK_ERR,182.65776
12,GFSO,ML0101442015,20150117_120000,TK_ERR,171.52535
12,GFSO,ML0102702015,20150131_060000,TK_ERR,161.57036
12,GFSO,ML0102432015,20150127_120000,TK_ERR,159.13559
12,GFSO,ML0301412015,20150316_180000,TK_ERR,138.83312
.
.
.
108,GFSO,ML0302532015,20150327_120000,TK_ERR,609.49759
108,GFSO,ML0100872015,20150112_000000,TK_ERR,588.38804
108,GFSO,ML0101032015,20150114_120000,TK_ERR,575.01756
120,GFSO,ML0302532015,20150327_120000,TK_ERR,688.85228
120,GFSO,ML0101032015,20150114_120000,TK_ERR,614.56354
Would adding TcStat to the PROCESS_LIST effect the current output
(listed above) in file "TK_ERR_boxplot.log" ? My understanding is the
TcStat
provides a detail listing of the "*test" files. However, is there a
parameter I could
changed to give me the result I am looking for(ie single TK_ERR for a given
hour)?
And, if so, what would be the order of the ush scripts?
2. Location --> /parm/met_config/TCPairsETCConfig
2.a I am not sure what the following parameters does or need to be used:
//
// CLIPER/SHIFOR baseline forecasts to be derived from the BEST
// and operational (CARQ) tracks.
//
best_technique = [ "BEST" ];
best_baseline = [];
oper_technique = [ "CARQ" ];
oper_baseline = [];
Now, for best_baseline, you may use the following "options":
BCLP, BCS5, BCD5, BCLA --> how are these "options" used for
tracking cyclones...I read where one of them is a "best" 3-day track
(???) and
there is a 5-day track(???). How does the above "options" effect(if any)
the
outcome of the TK_ERR? Should I consider these "options" when generating
track errors?
NOTE: The same for the oper_baseline: ie OCLB,OCS5,OCD5, OCDT
IF best_baseline and oper_baseline = []; does that mean that the
best_technique /
oper_technique are not used?
3. Location --> /parm/metplus_config/metplus_runtime.conf
3.a
I am a little confused about how the following "parameter group" works?
## Options are: processes, times
## Looping by time- runs all items in the PROCESS_LIST for each
## initialization time and repeats until all times have been evaluated.
## Looping by processes- run each item in the PROCESS_LIST for all
## specified initialization times then repeat for the next item in the
## PROCESS_LIST.
LOOP_METHOD = processes
3.a1 From the above definitions, since I am doing a seasonal
evaluation,
the above parameter should be set to "times" (will this summarizes the
3 months period entirely)??
By setting LOOP_METHOD = processes, hence, each month is being summarized
individually(Right)?
NOTE: See init time below....
## Processes to run in master script (master_metplus.py)
PROCESS_LIST = Usage
3.a2 Is "Usage" an "option"?? If so, what does it mean?
Or, will it be better to replace "usage" with TcPairs and TCMPRPlotter
here, as well as in
track_and_intensity.conf( in which case "usage" is override???)
NOTE: LOOP_METHOD is also defined in both config files:
## NOTE: "TOTAL" is a REQUIRED cnt statistic used by the series analysis
scripts
STAT_LIST = TOTAL, FBAR, OBAR, ME, MAE, RMSE, BCMSE, E50, EIQR, MAD
3.a3 STAT_LIST --> Is this "list" consider to be the standard for all
cases???
Are there any other parameters which are not listed??? So, if you only
want to
note the mean (ME/AVG); can you shorten the STAT _LIST = TOTAL, ME (ie;
TOTAL MUST ALWAYS appears as the first parameters in order to compute the
other parameters..Right???
## Init time
INIT_TIME_FMT = %Y%m%d
INIT_BEG = 20150101
INIT_END = 20150331
INIT_INC = 21600
As always, I appreciate your responses which are helping me to better
understand how MET+ cyclone track and intensity works.
Have a Great Day,
Steve
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: ref: Parameters definitions found in config files (track_and_intensity.conf, TCPairsETCConfig, and metplus_runtime.coef)
From: Minna Win
Time: Thu Jun 07 15:22:36 2018
Hello Steve,
I hope I can clarify the track_and_intensity use case for you. I have
responses to those questions to which I have the answer. I will need
to do
further research/solicit assistance for the second parts of 1c, 2a,
and
3a1.
1a. lead time:
If you leave the lead value empty, you will indeed have all available
lead
times evaluated. If you wish to evaluate only some lead times,
you do so as a comma-separated or whitespace separated list of hours:
lead
= 00, 12, 24, 48
1b. plot type:
The plot_type, if undefined does by default generate a box plot. If
you
want a box plot and any other plots, then yes, you do need to
explicitly
add 'boxplot' to the list.
1c. PROCESS_LIST
The use case has TcPairs, TCMPRPlotter for the PROCESS_LIST and
changing
the order of the scripts is not recommended. TcPairs runs the wrapper
to
MET's tc_pairs tool, which creates the paired data for the times of
interest. The TCMPRPlotter is the Python wrapper to the plot_tcmpr.R
R-script, which generates the plots of interest. There is currently
no
support for invoking TcStat via the PROCESS_LIST, as the Python
wrapper for
MET tc_stat was not originally written to be run stand-alone. Adding
TcStat to the PROCCESS_LIST will result in an error message. Creating
a
stand alone TcStat wrapper is on our long list of tasks.
I will need to do further investigation/check with some MET experts as
to
how to get a single TK_ERR for a specific time.
2a Location:
TCPairsETCConfig best_technique, oper_technique, etc.
I will need to do further research and ask someone more familiar with
MET
to get the answer to this question.
3a., 3a1 metplus_runtime.conf
You do not need to (and should not) modify the value to the
LOOP_METHOD.
In developing the wrappers for the MET tools, some use cases require
the
wrappers to be run differently. In the metplus_runtime.conf file, it
is set
to 'processes' by default and can be overridden by later config files.
The
track_and_intensity.conf config file overrides whatever is set in the
metplus_runtime.conf file, where it is set to 'processes'. This
particular
use case was created to run the tc_pairs wrapper for all the
initialization
times before running the plot_tcmpr.R Python wrapper, TCMPRPlotter
(eg.for
all initialization times, run wrapper A, then for all initialization
times,
run wrapper B, and output from wrapper A is input to wrapper B.).
There
are other use cases which run each wrapper in the PROCCESS_LIST for
one
initialization time, then repeat for each successive initialization
time
(e.g 'time': for init time 2017010100, run wrapper A, then wrapper B.
Then, for init time 2017010106, run wrapper A, then run wrapper B.
Input
from wrapper A is input to wrapper B) So, in summary, this setting
has
nothing to do with how the data is summarized and you should not
modify
this value (in the track_and_intensity.conf file).
I cannot immediately think of how you could summarize your data into
3-month periods within the confines of this use case. I'll need to do
some
further investigating.
3a2. PROCESS_LIST = Usage
This is a default value, which prints the usage in the event that the
user
forgets to indicate legitimate wrappers in the PROCESS_LIST of the use
case
config file. Again, you do not need to modify the PROCESS_LIST for
your
use case. LOOP_METHOD is indeed defined in both config files:
metplus_runtime.conf and track_and_intensity.conf. The MET+
configuration
files were designed to be overridden to allow multiple users to share
a
"core" set of config files: metplus_runtime.conf, metplus_data.conf,
metplus_system.conf, and then be "customized" for individuals by
additional
configuration files (such as the use case config files). You do not
need to
specify the three core config files on the command line, these are
automatically included when running MET+.
You can customize things further with your own config file, which you
would
invoke after the track_and_intensity.conf file:
master_metplus.py -c
$METPLUS_HOME/parm/use_cases/track_and_intensity/track_and_intensity.conf
-c /path/to/your/custom.conf
your custom.conf file could, for example, contain different Plot_TCMPR
options and then you could still share the default
track_and_intensity.conf
file with a colleague. Your colleague could then use her/his own
custom.conf file with different Plot_TCMPR options.
TOTAL is indeed required in the STAT_LIST, if it isn't in the STAT
list,
there is an error from MET. It's on our long list of things to change
when
we finish implementing all the wrappers.
3a3. STAT_LIST
The user has total freedom to make any necessary changes (with the
exception of TOTAL) to this list. The use case was set up with these
statistics based on the statistics 'typical' MET users tend to
request. So
if you wanted the ME/AVG, then yes, you would assign STAT_LIST= TOTAL,
ME.
Thanks for your questions. Please don't hesitate to contact us with
any
further questions.
Regards,
Minna
---------------
Minna Win
NCAR
Research Applications Lab
Phone: 303-497-8423
Fax: 302-497-8401
On Thu, Jun 7, 2018 at 6:12 PM Steven Lilly - NOAA Federal via RT <
met_help at ucar.edu> wrote:
>
> Thu Jun 07 12:11:53 2018: Request 85457 was acted upon.
> Transaction: Ticket created by steven.lilly at noaa.gov
> Queue: met_help
> Subject: ref: Parameters definitions found in config files
> (track_and_intensity.conf, TCPairsETCConfig, and
metplus_runtime.coef)
> Owner: Nobody
> Requestors: steven.lilly at noaa.gov
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=85457 >
>
>
> Good afternoon,
>
> I have several questions concerning the usages of the following
parameters:
>
> 1. location -->
use_cases/track_and_intensity/track_and_intensity.conf:
>
> 1.a lead =
> My understanding of the above parameter is referring to lead
time ie;
> 00 12 24 ...
> IF lead = "blank" --> does this means that it takes "all"
files
> times
> vs
> lead = 00, 06, 12, 18, 24 --> meaning only the 00hr 06hr 12hr
18hr
> and 24hr
> files are examined. ( ie: 36hr 48hr .... files that may
be
> present are not consider.
>
> Hence, if I want 00hr 12hr 24hr 36hr ....120hr, should I define
"lead"
> as
> lead = 00 12 24 36 48 ...96 108 120
>
> 1.b plot_type =
> IF plot_type = "blank" Default plot is boxplot, unless
otherwise
> indicated. If box plot
> is needed in addition to other plots, this needs to be
> indicated. So,
> plot_type = boxplot, point --> would give me a
single
> value of the TK_ERR
> for a specify hour instead of a series: ie;
>
> 108,GFSO,ML0302532015,20150327_120000,TK_ERR,609.49759
> 108,GFSO,ML0100872015,20150112_000000,TK_ERR,588.38804
> 108,GFSO,ML0101032015,20150114_120000,TK_ERR,575.01756
>
> plot_type = boxplot, point
> would give me something like 108,GFSO,TK_ERR,591
>
>
> 1.c PROCESS_LIST = --> indicates the ush "scripts" to run
> Currently, I'm using PROCESS_LIST = TcPairs, TCMPRPlotter
and
> the results I'm
> getting given the file name "TK_ERR_boxplot.log" has the
following
> format for
> forecast hours (12, 24,36,48,60,72,84,96,108,120 hours)
>
> LEAD_HR,AMODEL,STORM_ID,INIT,VARIABLE,OUTLIER
> 12,GFSO,ML0102532015,20150129_120000,TK_ERR,198.06748
> 12,GFSO,ML0302252015,20150325_060000,TK_ERR,191.37016
> 12,GFSO,ML0200452015,20150206_060000,TK_ERR,182.65776
> 12,GFSO,ML0101442015,20150117_120000,TK_ERR,171.52535
> 12,GFSO,ML0102702015,20150131_060000,TK_ERR,161.57036
> 12,GFSO,ML0102432015,20150127_120000,TK_ERR,159.13559
> 12,GFSO,ML0301412015,20150316_180000,TK_ERR,138.83312
> .
> .
> .
> 108,GFSO,ML0302532015,20150327_120000,TK_ERR,609.49759
> 108,GFSO,ML0100872015,20150112_000000,TK_ERR,588.38804
> 108,GFSO,ML0101032015,20150114_120000,TK_ERR,575.01756
> 120,GFSO,ML0302532015,20150327_120000,TK_ERR,688.85228
> 120,GFSO,ML0101032015,20150114_120000,TK_ERR,614.56354
>
> Would adding TcStat to the PROCESS_LIST effect the current output
> (listed above) in file "TK_ERR_boxplot.log" ? My understanding is
the
> TcStat
> provides a detail listing of the "*test" files. However, is there a
> parameter I could
> changed to give me the result I am looking for(ie single TK_ERR for
a given
> hour)?
> And, if so, what would be the order of the ush scripts?
>
>
>
>
> 2. Location --> /parm/met_config/TCPairsETCConfig
>
> 2.a I am not sure what the following parameters does or need to be
used:
> //
> // CLIPER/SHIFOR baseline forecasts to be derived from the BEST
> // and operational (CARQ) tracks.
> //
> best_technique = [ "BEST" ];
> best_baseline = [];
> oper_technique = [ "CARQ" ];
> oper_baseline = [];
>
> Now, for best_baseline, you may use the following "options":
> BCLP, BCS5, BCD5, BCLA --> how are these "options" used for
> tracking cyclones...I read where one of them is a "best" 3-day
track
> (???) and
> there is a 5-day track(???). How does the above "options"
effect(if any)
> the
> outcome of the TK_ERR? Should I consider these "options" when
generating
> track errors?
>
> NOTE: The same for the oper_baseline: ie OCLB,OCS5,OCD5, OCDT
>
> IF best_baseline and oper_baseline = []; does that mean that the
> best_technique /
> oper_technique are not used?
>
>
>
> 3. Location --> /parm/metplus_config/metplus_runtime.conf
>
> 3.a
> I am a little confused about how the following "parameter group"
works?
>
> ## Options are: processes, times
> ## Looping by time- runs all items in the PROCESS_LIST for each
> ## initialization time and repeats until all times have been
evaluated.
> ## Looping by processes- run each item in the PROCESS_LIST for all
> ## specified initialization times then repeat for the next item in
the
> ## PROCESS_LIST.
>
> LOOP_METHOD = processes
>
> 3.a1 From the above definitions, since I am doing a seasonal
> evaluation,
> the above parameter should be set to "times" (will this summarizes
the
> 3 months period entirely)??
>
> By setting LOOP_METHOD = processes, hence, each month is being
> summarized
> individually(Right)?
>
> NOTE: See init time below....
>
> ## Processes to run in master script (master_metplus.py)
> PROCESS_LIST = Usage
>
> 3.a2 Is "Usage" an "option"?? If so, what does it mean?
> Or, will it be better to replace "usage" with TcPairs and
TCMPRPlotter
> here, as well as in
> track_and_intensity.conf( in which case "usage" is override???)
>
> NOTE: LOOP_METHOD is also defined in both config files:
>
> ## NOTE: "TOTAL" is a REQUIRED cnt statistic used by the series
analysis
> scripts
> STAT_LIST = TOTAL, FBAR, OBAR, ME, MAE, RMSE, BCMSE, E50, EIQR, MAD
>
> 3.a3 STAT_LIST --> Is this "list" consider to be the standard
for all
> cases???
>
> Are there any other parameters which are not listed??? So, if
you only
> want to
> note the mean (ME/AVG); can you shorten the STAT _LIST = TOTAL, ME
(ie;
> TOTAL MUST ALWAYS appears as the first parameters in order to
compute the
> other parameters..Right???
>
> ## Init time
> INIT_TIME_FMT = %Y%m%d
> INIT_BEG = 20150101
> INIT_END = 20150331
> INIT_INC = 21600
>
>
>
> As always, I appreciate your responses which are helping me to
better
> understand how MET+ cyclone track and intensity works.
>
> Have a Great Day,
>
> Steve
>
>
------------------------------------------------
Subject: ref: Parameters definitions found in config files (track_and_intensity.conf, TCPairsETCConfig, and metplus_runtime.coef)
From: Minna Win
Time: Mon Jun 18 16:15:55 2018
Hi Steve,
I wanted to follow up on the questions which I didn't have immediate
answers. For 1c, where you wanted to calculate the TK_ERR for one
hour, it looks like you won't be able to do that within the confines
of the use case config files. You could call the MET TC-stat tool
(unfortunately, you will need to invoke the tool from the command line
or write your own script, the wrapper cannot be run stand-alone).
Please refer to Chapter 20 of the User's Manual for more details.
2a. CLIPER/SHIFOR settings- the default values are sufficient for
most users. You will need to refer to the MET User's Guide, chapter
19, page 322 (for MET 6.1) to see whether these other settings apply
to your case.
3a1. Clustering results in 3-month 'chunks'- the use case does not
support this.
I hope that clarifies things.
Regards,
Minna
------------------------------------------------
More information about the Met_help
mailing list