[Met_help] [rt.rap.ucar.edu #96458] History for Tropical Cyclone verification: TCPairs and CyclonePlotter
John Halley Gotway via RT
met_help at ucar.edu
Wed Sep 2 17:05:55 MDT 2020
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Good afternoon,
I found it relatively straightforward to set-up some scripts to process
forecast output along with obs by using this resource:
https://dtcenter.github.io/METplus/generated/model_applications/tc_and_extra_tc/Plotter_fcstGFS_obsGFS_ExtraTC.html#sphx-glr-generated-model-applications-tc-and-extra-tc-plotter-fcstgfs-obsgfs-extratc-py.
I have been running the global model which generates track data for the
post processing step following this naming convention:
atcfunix.gfs.YYYYMMDDHH. The best track data is stored here on WCOSS:
/gpfs/hps3/emc/hwrf/noscrub/emc.hurpara/trak/abdeck/btk/.
I have a python script that determines which file to use based on basin
information, storm name, and date. That is piped into the
TC_PAIRS_BDECK_TEMPLATE while the forecast file above is piped into the
TC_PAIRS_ADECK_TEMPLATE this way: atcfunix.gfs.{date?fmt=%Y%m%d%H}.
Inside my main script
*(/gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/METplus/scripts/run_metplus_TCPlotter.sh*,
I have three configure files that being processed like so:*
${MET_PLUS}/ush/master_metplus.py -c
${MET_PLUS_CONF}/Plotter_fcstGFS_obsGFS_ExtraTC.conf -c
${MET_PLUS_TMP}/shared.conf_TC -c ${MET_PLUS_CONF}/system_TC.conf*
The system_TC.conf is of little concern because it is mostly unchanged
except for the OUTPUT_BASE, which is changed specifically to where I plan
on storing the TC output. The shared_TC.conf, on the other hand, is used
to replace the content inside my Plotter_fcstGFS_obsGFS_ExtraTC.conf file.
In here I define
the ADECK and BDECK input directories based on the where the template files
above are being stored. Since I have several runs encompassing the
tropical cyclone event - 20190924 through 20190929 - I have specified my
INIT_BEG as 20190924 and my INIT_END as 20190929. I have also set my
INIT_INCREMENT to 86400 since I'm only processing the 00z cycle for now. I
have a couple of lines of code before the configure file that joins these
dates in a comma separated list such that the CYCLONE_PLOTTER_INIT_DATE is
"20190924,20190925,...,20190929". I'm not sure if that is allowed though.
When running the script the log that is generated produces 0 errors.
However, I find *two issues*: *1) there is only one file being carried
over to generate results; that is the 20190924 date - there should be 6*,
and *2) The files that are created are empty and thus the plot that is
generated is simply a map with no data overlaid.*
In attempting to debug further, I found something that looks important
shown here:
DEBUG 3: Rejected for init mask = 0
DEBUG 3: Rejected for valid mask = 0
*DEBUG 2: Matching 0 ADECK tracks to 0 BDECK tracks.*
DEBUG 1: Watch/Warning file:
/gpfs/dell2/emc/verification/noscrub/Julie.Prestopnik/met/9.1_beta2/share/met/tc_data/wwpts_us.txt
DEBUG 5: TrackPairInfoArray: NPairs = 0, NAlloc = 0, Pairs:
DEBUG 5:
DEBUG 1: Output file:
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs/201909/AL_q2019092400.gfs.PRNA20.tcst
08/26 18:35:12.422 metplus.TCPairs (command_runner.py:191) DEBUG: Finished
running
/gpfs/dell2/emc/verification/noscrub/Julie.Prestopnik/met/9.1_beta2/bin/tc_pairs
in 0:00:00.607349
08/26 18:35:12.422 metplus.CyclonePlotter (cyclone_plotter_wrapper.py:77)
DEBUG: Begin retrieving data...
08/26 18:35:12.422 metplus.CyclonePlotter (cyclone_plotter_wrapper.py:83)
DEBUG: Generate plot for all files in the
directory/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs
08/26 18:35:12.423 metplus.CyclonePlotter (cyclone_plotter_wrapper.py:98)
DEBUG: Parsing file
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs/201909/AL_q2019092400.gfs.PRNA20.tcst
08/26 18:35:19.778 metplus (config_launcher.py:520) DEBUG: Setting [config]
SCRUB_STAGING_DIR to default value: False.
08/26 18:35:19.779 metplus INFO:
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/metplus_final.conf:
write metplus.conf here
08/26 18:35:19.783 metplus (met_util.py:213) DEBUG: METplus took
0:00:08.783240 to run.
08/26 18:35:19.783 metplus INFO: METplus has successfully finished running.
Specifically the line that says matching 0 ADECK tracks to 0 BDECK tracks
lends me to believe that this is the reason why nothing is being added to
my TC_PAIRS_OUTPUT_TEMPLATE file, and therefore no plots generated on a
map.
What's interesting is the file that is extracted for BDECK is for hurricane
Lorenzo and includes all the dates coinciding with the ADECK file - a case
that is run coinciding with Lorenzo. I've checked this.
One possible issue is the formatting of these files. I looked in the test
case that comes with the cloned met stuff. In there the header/field info
looks something like this:
*/gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/METplus/source_data/met_test/new/track_data/201412/amlq2014123118.gfso.0104*
ML, 0104, 2014121318_F000_478N_1780W_FOF, 2014121318, 03, GFSO, 000, 478N,
1780W, 38, 986, XX, 34, NEQ, 0283, 0640, 0645, 0000, 990, 133, -99,
69, 227, -99, -9999, -9999, 208, 315, 135, 156, 0063
This differs from the structure of both the ADECK and BDECK input files:
*BDECK:
/gpfs/hps3/emc/hwrf/noscrub/emc.hurpara/trak/abdeck/btk/bal132019.dat *
AL, 13, 2019092218, , BEST, 0, 109N, 188W, 25, 1008, LO, 0, ,
0, 0, 0, 0, 1012, 100, 90, 40, 0, L, 0, , 0, 0,
INVEST, S, 0, , 0, 0, 0, 0, genesis-num, 027,
*ADECK:
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400*
AL, 0912, 03, PRNA, 000, 157N, 655W, 27, 1008, XX, 34, NEQ, 0000, 0000,
0000, 0000, -9999, -9999, 85
My suspicion is that these formats are not compatible. There are major
differences between these formats in my opinion. The ADECK was generated
automatically when making a run while BDECK is done by someone else. Those
files are stored whenever a storm is identified.
I think what I'll need to do next is start by running TCRMW to produce the
ADECK files formatted by metplus. I have the grib files for that. The
question next, though, is the BDECK files okay for processing?
--
Edward Strobach
EMC/NCEP/NWS/
IMSG Contractor
Cubicle#: 2029
301-683-3717
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Tropical Cyclone verification: TCPairs and CyclonePlotter
From: George McCabe
Time: Wed Aug 26 16:50:53 2020
Hi Edward,
The BDECK data is compatible to be processed. The use case you used as
an
example uses extra tropical cyclone data from SBU that is reformatted
by
TCPairs wrapper before processing. You will see your config file has
the
following:
## tc-pairs filtering optionsTC_PAIRS_REFORMAT_DECK =
yesTC_PAIRS_REFORMAT_TYPE = SBU
Changing TC_PAIRS_REFORMAT_DECK to 'no' or 'false' will skip the
reformatting step. There is another use case in the repo that uses
data
that looks like yours (at least as far as I can tell).
https://dtcenter.github.io/METplus/generated/met_tool_wrapper/TCPairs/TCPairs_tropical.html#metplus-
configuration
This use case uses BDECK data in met_test/new/hwrf/bdeck. Here is part
of
the file called bal062018.dat:
AL, 06, 2018082906, , BEST, 0, 129N, 109W, 10, 1010, DB, 0,
,
0, 0, 0, 0, 0, 0, 0, 0, 0, , 0, , 0,
0,
GENESIS019, , 0, , 0, 0, 0, 0, genesis-num, 019,
Let me know if changing that setting doesn't work and I can take a
closer
look.
Thanks,
George
On Wed, Aug 26, 2020 at 1:17 PM Edward Strobach - NOAA Affiliate via
RT <
met_help at ucar.edu> wrote:
>
> Wed Aug 26 13:13:28 2020: Request 96458 was acted upon.
> Transaction: Ticket created by edward.strobach at noaa.gov
> Queue: met_help
> Subject: Tropical Cyclone verification: TCPairs and
CyclonePlotter
> Owner: Nobody
> Requestors: edward.strobach at noaa.gov
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458 >
>
>
> Good afternoon,
>
> I found it relatively straightforward to set-up some scripts to
process
> forecast output along with obs by using this resource:
>
>
https://dtcenter.github.io/METplus/generated/model_applications/tc_and_extra_tc/Plotter_fcstGFS_obsGFS_ExtraTC.html#sphx-
glr-generated-model-applications-tc-and-extra-tc-plotter-fcstgfs-
obsgfs-extratc-py
> .
> I have been running the global model which generates track data for
the
> post processing step following this naming convention:
> atcfunix.gfs.YYYYMMDDHH. The best track data is stored here on
WCOSS:
> /gpfs/hps3/emc/hwrf/noscrub/emc.hurpara/trak/abdeck/btk/.
>
> I have a python script that determines which file to use based on
basin
> information, storm name, and date. That is piped into the
> TC_PAIRS_BDECK_TEMPLATE while the forecast file above is piped into
the
> TC_PAIRS_ADECK_TEMPLATE this way: atcfunix.gfs.{date?fmt=%Y%m%d%H}.
>
> Inside my main script
>
>
*(/gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/METplus/scripts/run_metplus_TCPlotter.sh*,
> I have three configure files that being processed like so:*
> ${MET_PLUS}/ush/master_metplus.py -c
> ${MET_PLUS_CONF}/Plotter_fcstGFS_obsGFS_ExtraTC.conf -c
> ${MET_PLUS_TMP}/shared.conf_TC -c ${MET_PLUS_CONF}/system_TC.conf*
>
> The system_TC.conf is of little concern because it is mostly
unchanged
> except for the OUTPUT_BASE, which is changed specifically to where I
plan
> on storing the TC output. The shared_TC.conf, on the other hand, is
used
> to replace the content inside my Plotter_fcstGFS_obsGFS_ExtraTC.conf
file.
> In here I define
> the ADECK and BDECK input directories based on the where the
template files
> above are being stored. Since I have several runs encompassing the
> tropical cyclone event - 20190924 through 20190929 - I have
specified my
> INIT_BEG as 20190924 and my INIT_END as 20190929. I have also set
my
> INIT_INCREMENT to 86400 since I'm only processing the 00z cycle for
now. I
> have a couple of lines of code before the configure file that joins
these
> dates in a comma separated list such that the
CYCLONE_PLOTTER_INIT_DATE is
> "20190924,20190925,...,20190929". I'm not sure if that is allowed
though.
>
> When running the script the log that is generated produces 0 errors.
> However, I find *two issues*: *1) there is only one file being
carried
> over to generate results; that is the 20190924 date - there should
be 6*,
> and *2) The files that are created are empty and thus the plot that
is
> generated is simply a map with no data overlaid.*
>
> In attempting to debug further, I found something that looks
important
> shown here:
>
> DEBUG 3: Rejected for init mask = 0
> DEBUG 3: Rejected for valid mask = 0
> *DEBUG 2: Matching 0 ADECK tracks to 0 BDECK tracks.*
> DEBUG 1: Watch/Warning file:
>
>
/gpfs/dell2/emc/verification/noscrub/Julie.Prestopnik/met/9.1_beta2/share/met/tc_data/wwpts_us.txt
> DEBUG 5: TrackPairInfoArray: NPairs = 0, NAlloc = 0, Pairs:
> DEBUG 5:
> DEBUG 1: Output file:
>
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs/201909/AL_q2019092400.gfs.PRNA20.tcst
> 08/26 18:35:12.422 metplus.TCPairs (command_runner.py:191) DEBUG:
Finished
> running
>
>
/gpfs/dell2/emc/verification/noscrub/Julie.Prestopnik/met/9.1_beta2/bin/tc_pairs
> in 0:00:00.607349
> 08/26 18:35:12.422 metplus.CyclonePlotter
(cyclone_plotter_wrapper.py:77)
> DEBUG: Begin retrieving data...
> 08/26 18:35:12.422 metplus.CyclonePlotter
(cyclone_plotter_wrapper.py:83)
> DEBUG: Generate plot for all files in the
>
>
directory/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs
> 08/26 18:35:12.423 metplus.CyclonePlotter
(cyclone_plotter_wrapper.py:98)
> DEBUG: Parsing file
>
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs/201909/AL_q2019092400.gfs.PRNA20.tcst
> 08/26 18:35:19.778 metplus (config_launcher.py:520) DEBUG: Setting
[config]
> SCRUB_STAGING_DIR to default value: False.
> 08/26 18:35:19.779 metplus INFO:
>
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/metplus_final.conf:
> write metplus.conf here
> 08/26 18:35:19.783 metplus (met_util.py:213) DEBUG: METplus took
> 0:00:08.783240 to run.
> 08/26 18:35:19.783 metplus INFO: METplus has successfully finished
running.
>
> Specifically the line that says matching 0 ADECK tracks to 0 BDECK
tracks
> lends me to believe that this is the reason why nothing is being
added to
> my TC_PAIRS_OUTPUT_TEMPLATE file, and therefore no plots generated
on a
> map.
>
> What's interesting is the file that is extracted for BDECK is for
hurricane
> Lorenzo and includes all the dates coinciding with the ADECK file -
a case
> that is run coinciding with Lorenzo. I've checked this.
>
> One possible issue is the formatting of these files. I looked in
the test
> case that comes with the cloned met stuff. In there the
header/field info
> looks something like this:
>
>
>
*/gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/METplus/source_data/met_test/new/track_data/201412/amlq2014123118.gfso.0104*
>
> ML, 0104, 2014121318_F000_478N_1780W_FOF, 2014121318, 03, GFSO, 000,
478N,
> 1780W, 38, 986, XX, 34, NEQ, 0283, 0640, 0645, 0000, 990, 133,
-99,
> 69, 227, -99, -9999, -9999, 208, 315, 135, 156, 0063
>
> This differs from the structure of both the ADECK and BDECK input
files:
>
> *BDECK:
>
/gpfs/hps3/emc/hwrf/noscrub/emc.hurpara/trak/abdeck/btk/bal132019.dat
*
>
> AL, 13, 2019092218, , BEST, 0, 109N, 188W, 25, 1008, LO, 0,
,
> 0, 0, 0, 0, 1012, 100, 90, 40, 0, L, 0, , 0,
0,
> INVEST, S, 0, , 0, 0, 0, 0, genesis-num, 027,
>
> *ADECK:
>
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400*
>
> AL, 0912, 03, PRNA, 000, 157N, 655W, 27, 1008, XX, 34, NEQ, 0000,
0000,
> 0000, 0000, -9999, -9999, 85
>
> My suspicion is that these formats are not compatible. There are
major
> differences between these formats in my opinion. The ADECK was
generated
> automatically when making a run while BDECK is done by someone else.
Those
> files are stored whenever a storm is identified.
>
> I think what I'll need to do next is start by running TCRMW to
produce the
> ADECK files formatted by metplus. I have the grib files for that.
The
> question next, though, is the BDECK files okay for processing?
>
>
>
> --
> Edward Strobach
> EMC/NCEP/NWS/
> IMSG Contractor
> Cubicle#: 2029
> 301-683-3717
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: Tropical Cyclone verification: TCPairs and CyclonePlotter
From: Edward Strobach - NOAA Affiliate
Time: Thu Aug 27 09:52:48 2020
Hi George, I figured the BDECK must be compatible, so no worries
there.
Thanks for the link and the heads up about reformatting. I will use
the
example to modify what I currently have to see the outcome. Thanks
again.
On Wed, Aug 26, 2020 at 6:57 PM George McCabe via RT
<met_help at ucar.edu>
wrote:
> Hi Edward,
>
> The BDECK data is compatible to be processed. The use case you used
as an
> example uses extra tropical cyclone data from SBU that is
reformatted by
> TCPairs wrapper before processing. You will see your config file has
the
> following:
>
> ## tc-pairs filtering optionsTC_PAIRS_REFORMAT_DECK =
> yesTC_PAIRS_REFORMAT_TYPE = SBU
>
> Changing TC_PAIRS_REFORMAT_DECK to 'no' or 'false' will skip the
> reformatting step. There is another use case in the repo that uses
data
> that looks like yours (at least as far as I can tell).
>
>
>
https://dtcenter.github.io/METplus/generated/met_tool_wrapper/TCPairs/TCPairs_tropical.html#metplus-
configuration
>
> This use case uses BDECK data in met_test/new/hwrf/bdeck. Here is
part of
> the file called bal062018.dat:
>
> AL, 06, 2018082906, , BEST, 0, 129N, 109W, 10, 1010, DB, 0,
,
> 0, 0, 0, 0, 0, 0, 0, 0, 0, , 0, , 0,
0,
> GENESIS019, , 0, , 0, 0, 0, 0, genesis-num, 019,
>
>
> Let me know if changing that setting doesn't work and I can take a
closer
> look.
>
> Thanks,
> George
>
> On Wed, Aug 26, 2020 at 1:17 PM Edward Strobach - NOAA Affiliate via
RT <
> met_help at ucar.edu> wrote:
>
> >
> > Wed Aug 26 13:13:28 2020: Request 96458 was acted upon.
> > Transaction: Ticket created by edward.strobach at noaa.gov
> > Queue: met_help
> > Subject: Tropical Cyclone verification: TCPairs and
CyclonePlotter
> > Owner: Nobody
> > Requestors: edward.strobach at noaa.gov
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458 >
> >
> >
> > Good afternoon,
> >
> > I found it relatively straightforward to set-up some scripts to
process
> > forecast output along with obs by using this resource:
> >
> >
>
https://dtcenter.github.io/METplus/generated/model_applications/tc_and_extra_tc/Plotter_fcstGFS_obsGFS_ExtraTC.html#sphx-
glr-generated-model-applications-tc-and-extra-tc-plotter-fcstgfs-
obsgfs-extratc-py
> > .
> > I have been running the global model which generates track data
for the
> > post processing step following this naming convention:
> > atcfunix.gfs.YYYYMMDDHH. The best track data is stored here on
WCOSS:
> > /gpfs/hps3/emc/hwrf/noscrub/emc.hurpara/trak/abdeck/btk/.
> >
> > I have a python script that determines which file to use based on
basin
> > information, storm name, and date. That is piped into the
> > TC_PAIRS_BDECK_TEMPLATE while the forecast file above is piped
into the
> > TC_PAIRS_ADECK_TEMPLATE this way:
atcfunix.gfs.{date?fmt=%Y%m%d%H}.
> >
> > Inside my main script
> >
> >
>
*(/gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/METplus/scripts/run_metplus_TCPlotter.sh*,
> > I have three configure files that being processed like so:*
> > ${MET_PLUS}/ush/master_metplus.py -c
> > ${MET_PLUS_CONF}/Plotter_fcstGFS_obsGFS_ExtraTC.conf -c
> > ${MET_PLUS_TMP}/shared.conf_TC -c ${MET_PLUS_CONF}/system_TC.conf*
> >
> > The system_TC.conf is of little concern because it is mostly
unchanged
> > except for the OUTPUT_BASE, which is changed specifically to where
I plan
> > on storing the TC output. The shared_TC.conf, on the other hand,
is used
> > to replace the content inside my
Plotter_fcstGFS_obsGFS_ExtraTC.conf
> file.
> > In here I define
> > the ADECK and BDECK input directories based on the where the
template
> files
> > above are being stored. Since I have several runs encompassing
the
> > tropical cyclone event - 20190924 through 20190929 - I have
specified my
> > INIT_BEG as 20190924 and my INIT_END as 20190929. I have also set
my
> > INIT_INCREMENT to 86400 since I'm only processing the 00z cycle
for
> now. I
> > have a couple of lines of code before the configure file that
joins these
> > dates in a comma separated list such that the
CYCLONE_PLOTTER_INIT_DATE
> is
> > "20190924,20190925,...,20190929". I'm not sure if that is allowed
> though.
> >
> > When running the script the log that is generated produces 0
errors.
> > However, I find *two issues*: *1) there is only one file being
carried
> > over to generate results; that is the 20190924 date - there should
be 6*,
> > and *2) The files that are created are empty and thus the plot
that is
> > generated is simply a map with no data overlaid.*
> >
> > In attempting to debug further, I found something that looks
important
> > shown here:
> >
> > DEBUG 3: Rejected for init mask = 0
> > DEBUG 3: Rejected for valid mask = 0
> > *DEBUG 2: Matching 0 ADECK tracks to 0 BDECK tracks.*
> > DEBUG 1: Watch/Warning file:
> >
> >
>
/gpfs/dell2/emc/verification/noscrub/Julie.Prestopnik/met/9.1_beta2/share/met/tc_data/wwpts_us.txt
> > DEBUG 5: TrackPairInfoArray: NPairs = 0, NAlloc = 0, Pairs:
> > DEBUG 5:
> > DEBUG 1: Output file:
> >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs/201909/AL_q2019092400.gfs.PRNA20.tcst
> > 08/26 18:35:12.422 metplus.TCPairs (command_runner.py:191) DEBUG:
> Finished
> > running
> >
> >
>
/gpfs/dell2/emc/verification/noscrub/Julie.Prestopnik/met/9.1_beta2/bin/tc_pairs
> > in 0:00:00.607349
> > 08/26 18:35:12.422 metplus.CyclonePlotter
(cyclone_plotter_wrapper.py:77)
> > DEBUG: Begin retrieving data...
> > 08/26 18:35:12.422 metplus.CyclonePlotter
(cyclone_plotter_wrapper.py:83)
> > DEBUG: Generate plot for all files in the
> >
> >
>
directory/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs
> > 08/26 18:35:12.423 metplus.CyclonePlotter
(cyclone_plotter_wrapper.py:98)
> > DEBUG: Parsing file
> >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs/201909/AL_q2019092400.gfs.PRNA20.tcst
> > 08/26 18:35:19.778 metplus (config_launcher.py:520) DEBUG: Setting
> [config]
> > SCRUB_STAGING_DIR to default value: False.
> > 08/26 18:35:19.779 metplus INFO:
> >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/metplus_final.conf:
> > write metplus.conf here
> > 08/26 18:35:19.783 metplus (met_util.py:213) DEBUG: METplus took
> > 0:00:08.783240 to run.
> > 08/26 18:35:19.783 metplus INFO: METplus has successfully finished
> running.
> >
> > Specifically the line that says matching 0 ADECK tracks to 0 BDECK
tracks
> > lends me to believe that this is the reason why nothing is being
added to
> > my TC_PAIRS_OUTPUT_TEMPLATE file, and therefore no plots generated
on a
> > map.
> >
> > What's interesting is the file that is extracted for BDECK is for
> hurricane
> > Lorenzo and includes all the dates coinciding with the ADECK file
- a
> case
> > that is run coinciding with Lorenzo. I've checked this.
> >
> > One possible issue is the formatting of these files. I looked in
the
> test
> > case that comes with the cloned met stuff. In there the
header/field
> info
> > looks something like this:
> >
> >
> >
>
*/gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/METplus/source_data/met_test/new/track_data/201412/amlq2014123118.gfso.0104*
> >
> > ML, 0104, 2014121318_F000_478N_1780W_FOF, 2014121318, 03, GFSO,
000,
> 478N,
> > 1780W, 38, 986, XX, 34, NEQ, 0283, 0640, 0645, 0000, 990,
133, -99,
> > 69, 227, -99, -9999, -9999, 208, 315, 135, 156, 0063
> >
> > This differs from the structure of both the ADECK and BDECK input
files:
> >
> > *BDECK:
> >
/gpfs/hps3/emc/hwrf/noscrub/emc.hurpara/trak/abdeck/btk/bal132019.dat
*
> >
> > AL, 13, 2019092218, , BEST, 0, 109N, 188W, 25, 1008, LO,
0, ,
> > 0, 0, 0, 0, 1012, 100, 90, 40, 0, L, 0, ,
0, 0,
> > INVEST, S, 0, , 0, 0, 0, 0, genesis-num, 027,
> >
> > *ADECK:
> >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400*
> >
> > AL, 0912, 03, PRNA, 000, 157N, 655W, 27, 1008, XX, 34, NEQ,
0000,
> 0000,
> > 0000, 0000, -9999, -9999, 85
> >
> > My suspicion is that these formats are not compatible. There are
major
> > differences between these formats in my opinion. The ADECK was
generated
> > automatically when making a run while BDECK is done by someone
else.
> Those
> > files are stored whenever a storm is identified.
> >
> > I think what I'll need to do next is start by running TCRMW to
produce
> the
> > ADECK files formatted by metplus. I have the grib files for that.
The
> > question next, though, is the BDECK files okay for processing?
> >
> >
> >
> > --
> > Edward Strobach
> > EMC/NCEP/NWS/
> > IMSG Contractor
> > Cubicle#: 2029
> > 301-683-3717
> >
> >
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>
--
Edward Strobach
EMC/NCEP/NWS/
IMSG Contractor
Cubicle#: 2029
301-683-3717
------------------------------------------------
Subject: Tropical Cyclone verification: TCPairs and CyclonePlotter
From: Edward Strobach - NOAA Affiliate
Time: Thu Aug 27 10:53:15 2020
Hi,
I did decide to remove CyclonePlotter for now and use the
TCPairs_Config
file along with settings that I set in a shared_TC.conf that I store
in a
temporary directory to process the ADECK and BDECK data that I have.
Files
are generated for all the dates that span INIT_BEG and INIT_END,
thanks to
setting loop_by to INIT. However, only the header is written out, and
I
suspect it is do to what the log indicates here:
DEBUG 4: [File 1 of 5] Used 0 of 168 lines read from file
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
DEBUG 4: [File 2 of 5] Used 0 of 168 lines read from file
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
DEBUG 4: [File 3 of 5] Used 0 of 168 lines read from file
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
DEBUG 4: [File 4 of 5] Used 0 of 168 lines read from file
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
DEBUG 4: [File 5 of 5] Used 0 of 168 lines read from file
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
DEBUG 3: Used 0 of 840 lines read from 5 file(s).
DEBUG 3: Identified 0 track(s).
DEBUG 5: TrackInfoArray: NTracks = 0, NAlloc = 0, Tracks:
DEBUG 5:
DEBUG 2: Deriving 12-hour interpolated ADECK tracks.
DEBUG 2: Finished adding 0 and replacing 0 Interp12 track(s).
DEBUG 2: Deriving 0 ADECK consensus model(s).
DEBUG 2: Added 0 ADECK consensus tracks(s).
DEBUG 2: Deriving 0 ADECK lag model(s).
DEBUG 2: Added 0 ADECK lag tracks(s).
DEBUG 2: Deriving 0 CLIPER/SHIFOR baseline model(s).
DEBUG 2: Added 0 CLIPER/SHIFOR baseline track(s).
DEBUG 2: Filtering 0 ADECK tracks based on config file settings.
DEBUG 3: Total tracks read = 0
DEBUG 3: Total tracks kept = 0
DEBUG 3: Rejected for storm name = 0
DEBUG 3: Rejected for valid time = 0
DEBUG 3: Rejected for required lead times = 0
DEBUG 3: Rejected for init mask = 0
DEBUG 3: Rejected for valid mask = 0
DEBUG 2: Matching 0 ADECK tracks to 1 BDECK tracks.
I'm still thinking that the formatting is off in this file. I surmise
that
it's not able to group the data because the column information is not
in
the order that met would want it to be in. These files do not have
any
header info, so it has to rely on the correct ordering, or so I
suspect.
My interpretation, however, may be flawed. I see 6 separate TC based
metplus routines: TCPairs (I'm working on currently); CyclonePlotter
(looking to use); TCStat; TCRMW; TCMPRPlotter; and TCGen. Much, if
not
almost all of these relies on TCPairs. Is there a routine that I'm
missing
in metplus that takes a grib file and constructs ADECK files?
On Thu, Aug 27, 2020 at 11:52 AM Edward Strobach - NOAA Affiliate <
edward.strobach at noaa.gov> wrote:
> Hi George, I figured the BDECK must be compatible, so no worries
there.
> Thanks for the link and the heads up about reformatting. I will use
the
> example to modify what I currently have to see the outcome. Thanks
again.
>
> On Wed, Aug 26, 2020 at 6:57 PM George McCabe via RT
<met_help at ucar.edu>
> wrote:
>
>> Hi Edward,
>>
>> The BDECK data is compatible to be processed. The use case you used
as an
>> example uses extra tropical cyclone data from SBU that is
reformatted by
>> TCPairs wrapper before processing. You will see your config file
has the
>> following:
>>
>> ## tc-pairs filtering optionsTC_PAIRS_REFORMAT_DECK =
>> yesTC_PAIRS_REFORMAT_TYPE = SBU
>>
>> Changing TC_PAIRS_REFORMAT_DECK to 'no' or 'false' will skip the
>> reformatting step. There is another use case in the repo that uses
data
>> that looks like yours (at least as far as I can tell).
>>
>>
>>
https://dtcenter.github.io/METplus/generated/met_tool_wrapper/TCPairs/TCPairs_tropical.html#metplus-
configuration
>>
>> This use case uses BDECK data in met_test/new/hwrf/bdeck. Here is
part of
>> the file called bal062018.dat:
>>
>> AL, 06, 2018082906, , BEST, 0, 129N, 109W, 10, 1010, DB, 0,
,
>> 0, 0, 0, 0, 0, 0, 0, 0, 0, , 0, ,
0, 0,
>> GENESIS019, , 0, , 0, 0, 0, 0, genesis-num, 019,
>>
>>
>> Let me know if changing that setting doesn't work and I can take a
closer
>> look.
>>
>> Thanks,
>> George
>>
>> On Wed, Aug 26, 2020 at 1:17 PM Edward Strobach - NOAA Affiliate
via RT <
>> met_help at ucar.edu> wrote:
>>
>> >
>> > Wed Aug 26 13:13:28 2020: Request 96458 was acted upon.
>> > Transaction: Ticket created by edward.strobach at noaa.gov
>> > Queue: met_help
>> > Subject: Tropical Cyclone verification: TCPairs and
CyclonePlotter
>> > Owner: Nobody
>> > Requestors: edward.strobach at noaa.gov
>> > Status: new
>> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458 >
>> >
>> >
>> > Good afternoon,
>> >
>> > I found it relatively straightforward to set-up some scripts to
process
>> > forecast output along with obs by using this resource:
>> >
>> >
>>
https://dtcenter.github.io/METplus/generated/model_applications/tc_and_extra_tc/Plotter_fcstGFS_obsGFS_ExtraTC.html#sphx-
glr-generated-model-applications-tc-and-extra-tc-plotter-fcstgfs-
obsgfs-extratc-py
>> > .
>> > I have been running the global model which generates track data
for the
>> > post processing step following this naming convention:
>> > atcfunix.gfs.YYYYMMDDHH. The best track data is stored here on
WCOSS:
>> > /gpfs/hps3/emc/hwrf/noscrub/emc.hurpara/trak/abdeck/btk/.
>> >
>> > I have a python script that determines which file to use based on
basin
>> > information, storm name, and date. That is piped into the
>> > TC_PAIRS_BDECK_TEMPLATE while the forecast file above is piped
into the
>> > TC_PAIRS_ADECK_TEMPLATE this way:
atcfunix.gfs.{date?fmt=%Y%m%d%H}.
>> >
>> > Inside my main script
>> >
>> >
>>
*(/gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/METplus/scripts/run_metplus_TCPlotter.sh*,
>> > I have three configure files that being processed like so:*
>> > ${MET_PLUS}/ush/master_metplus.py -c
>> > ${MET_PLUS_CONF}/Plotter_fcstGFS_obsGFS_ExtraTC.conf -c
>> > ${MET_PLUS_TMP}/shared.conf_TC -c
${MET_PLUS_CONF}/system_TC.conf*
>> >
>> > The system_TC.conf is of little concern because it is mostly
unchanged
>> > except for the OUTPUT_BASE, which is changed specifically to
where I
>> plan
>> > on storing the TC output. The shared_TC.conf, on the other hand,
is
>> used
>> > to replace the content inside my
Plotter_fcstGFS_obsGFS_ExtraTC.conf
>> file.
>> > In here I define
>> > the ADECK and BDECK input directories based on the where the
template
>> files
>> > above are being stored. Since I have several runs encompassing
the
>> > tropical cyclone event - 20190924 through 20190929 - I have
specified my
>> > INIT_BEG as 20190924 and my INIT_END as 20190929. I have also
set my
>> > INIT_INCREMENT to 86400 since I'm only processing the 00z cycle
for
>> now. I
>> > have a couple of lines of code before the configure file that
joins
>> these
>> > dates in a comma separated list such that the
CYCLONE_PLOTTER_INIT_DATE
>> is
>> > "20190924,20190925,...,20190929". I'm not sure if that is
allowed
>> though.
>> >
>> > When running the script the log that is generated produces 0
errors.
>> > However, I find *two issues*: *1) there is only one file being
carried
>> > over to generate results; that is the 20190924 date - there
should be
>> 6*,
>> > and *2) The files that are created are empty and thus the plot
that is
>> > generated is simply a map with no data overlaid.*
>> >
>> > In attempting to debug further, I found something that looks
important
>> > shown here:
>> >
>> > DEBUG 3: Rejected for init mask = 0
>> > DEBUG 3: Rejected for valid mask = 0
>> > *DEBUG 2: Matching 0 ADECK tracks to 0 BDECK tracks.*
>> > DEBUG 1: Watch/Warning file:
>> >
>> >
>>
/gpfs/dell2/emc/verification/noscrub/Julie.Prestopnik/met/9.1_beta2/share/met/tc_data/wwpts_us.txt
>> > DEBUG 5: TrackPairInfoArray: NPairs = 0, NAlloc = 0, Pairs:
>> > DEBUG 5:
>> > DEBUG 1: Output file:
>> >
>> >
>>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs/201909/AL_q2019092400.gfs.PRNA20.tcst
>> > 08/26 18:35:12.422 metplus.TCPairs (command_runner.py:191) DEBUG:
>> Finished
>> > running
>> >
>> >
>>
/gpfs/dell2/emc/verification/noscrub/Julie.Prestopnik/met/9.1_beta2/bin/tc_pairs
>> > in 0:00:00.607349
>> > 08/26 18:35:12.422 metplus.CyclonePlotter
>> (cyclone_plotter_wrapper.py:77)
>> > DEBUG: Begin retrieving data...
>> > 08/26 18:35:12.422 metplus.CyclonePlotter
>> (cyclone_plotter_wrapper.py:83)
>> > DEBUG: Generate plot for all files in the
>> >
>> >
>>
directory/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs
>> > 08/26 18:35:12.423 metplus.CyclonePlotter
>> (cyclone_plotter_wrapper.py:98)
>> > DEBUG: Parsing file
>> >
>> >
>>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs/201909/AL_q2019092400.gfs.PRNA20.tcst
>> > 08/26 18:35:19.778 metplus (config_launcher.py:520) DEBUG:
Setting
>> [config]
>> > SCRUB_STAGING_DIR to default value: False.
>> > 08/26 18:35:19.779 metplus INFO:
>> >
>> >
>>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/metplus_final.conf:
>> > write metplus.conf here
>> > 08/26 18:35:19.783 metplus (met_util.py:213) DEBUG: METplus took
>> > 0:00:08.783240 to run.
>> > 08/26 18:35:19.783 metplus INFO: METplus has successfully
finished
>> running.
>> >
>> > Specifically the line that says matching 0 ADECK tracks to 0
BDECK
>> tracks
>> > lends me to believe that this is the reason why nothing is being
added
>> to
>> > my TC_PAIRS_OUTPUT_TEMPLATE file, and therefore no plots
generated on a
>> > map.
>> >
>> > What's interesting is the file that is extracted for BDECK is for
>> hurricane
>> > Lorenzo and includes all the dates coinciding with the ADECK file
- a
>> case
>> > that is run coinciding with Lorenzo. I've checked this.
>> >
>> > One possible issue is the formatting of these files. I looked in
the
>> test
>> > case that comes with the cloned met stuff. In there the
header/field
>> info
>> > looks something like this:
>> >
>> >
>> >
>>
*/gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/METplus/source_data/met_test/new/track_data/201412/amlq2014123118.gfso.0104*
>> >
>> > ML, 0104, 2014121318_F000_478N_1780W_FOF, 2014121318, 03, GFSO,
000,
>> 478N,
>> > 1780W, 38, 986, XX, 34, NEQ, 0283, 0640, 0645, 0000, 990,
133, -99,
>> > 69, 227, -99, -9999, -9999, 208, 315, 135, 156, 0063
>> >
>> > This differs from the structure of both the ADECK and BDECK input
files:
>> >
>> > *BDECK:
>> >
/gpfs/hps3/emc/hwrf/noscrub/emc.hurpara/trak/abdeck/btk/bal132019.dat
*
>> >
>> > AL, 13, 2019092218, , BEST, 0, 109N, 188W, 25, 1008, LO,
0, ,
>> > 0, 0, 0, 0, 1012, 100, 90, 40, 0, L, 0, ,
0,
>> 0,
>> > INVEST, S, 0, , 0, 0, 0, 0, genesis-num, 027,
>> >
>> > *ADECK:
>> >
>> >
>>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400*
>> >
>> > AL, 0912, 03, PRNA, 000, 157N, 655W, 27, 1008, XX, 34, NEQ,
0000,
>> 0000,
>> > 0000, 0000, -9999, -9999, 85
>> >
>> > My suspicion is that these formats are not compatible. There are
major
>> > differences between these formats in my opinion. The ADECK was
>> generated
>> > automatically when making a run while BDECK is done by someone
else.
>> Those
>> > files are stored whenever a storm is identified.
>> >
>> > I think what I'll need to do next is start by running TCRMW to
produce
>> the
>> > ADECK files formatted by metplus. I have the grib files for
that. The
>> > question next, though, is the BDECK files okay for processing?
>> >
>> >
>> >
>> > --
>> > Edward Strobach
>> > EMC/NCEP/NWS/
>> > IMSG Contractor
>> > Cubicle#: 2029
>> > 301-683-3717
>> >
>> >
>>
>> --
>> George McCabe - Software Engineer III
>> National Center for Atmospheric Research
>> Research Applications Laboratory
>> 303-497-2768
>> ---
>> My working day may not be your working day. Please do not feel
obliged to
>> reply to this email outside of your normal working hours.
>>
>>
>
> --
> Edward Strobach
> EMC/NCEP/NWS/
> IMSG Contractor
> Cubicle#: 2029
> 301-683-3717
>
--
Edward Strobach
EMC/NCEP/NWS/
IMSG Contractor
Cubicle#: 2029
301-683-3717
------------------------------------------------
Subject: Tropical Cyclone verification: TCPairs and CyclonePlotter
From: George McCabe
Time: Thu Aug 27 13:29:44 2020
Hi Edward,
>From John HG:
*The MET-TC tools read track files in ATCF format. Those track files
can be
generated from gridded model output by running software. One such
tracker
is supported to the community by the DTC and is named the "GFDL Vortex
Tracker":http://dtcenter.org/community-code/gfdl-vortex-tracker
<http://dtcenter.org/community-code/gfdl-vortex-tracker>*
*It is not expressly part of METplus but would be very useful for
tracking
storms in GRIB files.*
My guess as to why none of the tracks are being used has to do with
the
METplus configurations to filter data. I would check the values for
TC_PAIRS_CYCLONE, TC_PAIRS_BASIN, TC_PAIRS_STORM_NAME,
TC_PAIRS_STORM_ID,
and MODEL. The values you put in these lists determine which storms to
use.
You can leave them blank to use any.
Also, you could bump up the MET log verbosity and see more information
about why they are being rejected:
# to affect all MET tools set:
LOG_MET_VERBOSITY = 10
# to affect only TCPairs set:
LOG_TC_PAIRS_VERBOSITY = 10
- George
On Thu, Aug 27, 2020 at 10:53 AM Edward Strobach - NOAA Affiliate via
RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458 >
>
> Hi,
>
> I did decide to remove CyclonePlotter for now and use the
TCPairs_Config
> file along with settings that I set in a shared_TC.conf that I store
in a
> temporary directory to process the ADECK and BDECK data that I have.
Files
> are generated for all the dates that span INIT_BEG and INIT_END,
thanks to
> setting loop_by to INIT. However, only the header is written out,
and I
> suspect it is do to what the log indicates here:
>
> DEBUG 4: [File 1 of 5] Used 0 of 168 lines read from file
>
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
> DEBUG 4: [File 2 of 5] Used 0 of 168 lines read from file
>
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
> DEBUG 4: [File 3 of 5] Used 0 of 168 lines read from file
>
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
> DEBUG 4: [File 4 of 5] Used 0 of 168 lines read from file
>
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
> DEBUG 4: [File 5 of 5] Used 0 of 168 lines read from file
>
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
> DEBUG 3: Used 0 of 840 lines read from 5 file(s).
> DEBUG 3: Identified 0 track(s).
> DEBUG 5: TrackInfoArray: NTracks = 0, NAlloc = 0, Tracks:
> DEBUG 5:
> DEBUG 2: Deriving 12-hour interpolated ADECK tracks.
> DEBUG 2: Finished adding 0 and replacing 0 Interp12 track(s).
> DEBUG 2: Deriving 0 ADECK consensus model(s).
> DEBUG 2: Added 0 ADECK consensus tracks(s).
> DEBUG 2: Deriving 0 ADECK lag model(s).
> DEBUG 2: Added 0 ADECK lag tracks(s).
> DEBUG 2: Deriving 0 CLIPER/SHIFOR baseline model(s).
> DEBUG 2: Added 0 CLIPER/SHIFOR baseline track(s).
> DEBUG 2: Filtering 0 ADECK tracks based on config file settings.
> DEBUG 3: Total tracks read = 0
> DEBUG 3: Total tracks kept = 0
> DEBUG 3: Rejected for storm name = 0
> DEBUG 3: Rejected for valid time = 0
> DEBUG 3: Rejected for required lead times = 0
> DEBUG 3: Rejected for init mask = 0
> DEBUG 3: Rejected for valid mask = 0
> DEBUG 2: Matching 0 ADECK tracks to 1 BDECK tracks.
>
> I'm still thinking that the formatting is off in this file. I
surmise that
> it's not able to group the data because the column information is
not in
> the order that met would want it to be in. These files do not have
any
> header info, so it has to rely on the correct ordering, or so I
suspect.
> My interpretation, however, may be flawed. I see 6 separate TC
based
> metplus routines: TCPairs (I'm working on currently);
CyclonePlotter
> (looking to use); TCStat; TCRMW; TCMPRPlotter; and TCGen. Much, if
not
> almost all of these relies on TCPairs. Is there a routine that I'm
missing
> in metplus that takes a grib file and constructs ADECK files?
>
>
>
> On Thu, Aug 27, 2020 at 11:52 AM Edward Strobach - NOAA Affiliate <
> edward.strobach at noaa.gov> wrote:
>
> > Hi George, I figured the BDECK must be compatible, so no worries
there.
> > Thanks for the link and the heads up about reformatting. I will
use the
> > example to modify what I currently have to see the outcome.
Thanks
> again.
> >
> > On Wed, Aug 26, 2020 at 6:57 PM George McCabe via RT
<met_help at ucar.edu>
> > wrote:
> >
> >> Hi Edward,
> >>
> >> The BDECK data is compatible to be processed. The use case you
used as
> an
> >> example uses extra tropical cyclone data from SBU that is
reformatted by
> >> TCPairs wrapper before processing. You will see your config file
has the
> >> following:
> >>
> >> ## tc-pairs filtering optionsTC_PAIRS_REFORMAT_DECK =
> >> yesTC_PAIRS_REFORMAT_TYPE = SBU
> >>
> >> Changing TC_PAIRS_REFORMAT_DECK to 'no' or 'false' will skip the
> >> reformatting step. There is another use case in the repo that
uses data
> >> that looks like yours (at least as far as I can tell).
> >>
> >>
> >>
>
https://dtcenter.github.io/METplus/generated/met_tool_wrapper/TCPairs/TCPairs_tropical.html#metplus-
configuration
> >>
> >> This use case uses BDECK data in met_test/new/hwrf/bdeck. Here is
part
> of
> >> the file called bal062018.dat:
> >>
> >> AL, 06, 2018082906, , BEST, 0, 129N, 109W, 10, 1010, DB,
0, ,
> >> 0, 0, 0, 0, 0, 0, 0, 0, 0, , 0, ,
0,
> 0,
> >> GENESIS019, , 0, , 0, 0, 0, 0, genesis-num, 019,
> >>
> >>
> >> Let me know if changing that setting doesn't work and I can take
a
> closer
> >> look.
> >>
> >> Thanks,
> >> George
> >>
> >> On Wed, Aug 26, 2020 at 1:17 PM Edward Strobach - NOAA Affiliate
via RT
> <
> >> met_help at ucar.edu> wrote:
> >>
> >> >
> >> > Wed Aug 26 13:13:28 2020: Request 96458 was acted upon.
> >> > Transaction: Ticket created by edward.strobach at noaa.gov
> >> > Queue: met_help
> >> > Subject: Tropical Cyclone verification: TCPairs and
> CyclonePlotter
> >> > Owner: Nobody
> >> > Requestors: edward.strobach at noaa.gov
> >> > Status: new
> >> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458
> >
> >> >
> >> >
> >> > Good afternoon,
> >> >
> >> > I found it relatively straightforward to set-up some scripts to
> process
> >> > forecast output along with obs by using this resource:
> >> >
> >> >
> >>
>
https://dtcenter.github.io/METplus/generated/model_applications/tc_and_extra_tc/Plotter_fcstGFS_obsGFS_ExtraTC.html#sphx-
glr-generated-model-applications-tc-and-extra-tc-plotter-fcstgfs-
obsgfs-extratc-py
> >> > .
> >> > I have been running the global model which generates track data
for
> the
> >> > post processing step following this naming convention:
> >> > atcfunix.gfs.YYYYMMDDHH. The best track data is stored here on
WCOSS:
> >> > /gpfs/hps3/emc/hwrf/noscrub/emc.hurpara/trak/abdeck/btk/.
> >> >
> >> > I have a python script that determines which file to use based
on
> basin
> >> > information, storm name, and date. That is piped into the
> >> > TC_PAIRS_BDECK_TEMPLATE while the forecast file above is piped
into
> the
> >> > TC_PAIRS_ADECK_TEMPLATE this way:
atcfunix.gfs.{date?fmt=%Y%m%d%H}.
> >> >
> >> > Inside my main script
> >> >
> >> >
> >>
>
*(/gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/METplus/scripts/run_metplus_TCPlotter.sh*,
> >> > I have three configure files that being processed like so:*
> >> > ${MET_PLUS}/ush/master_metplus.py -c
> >> > ${MET_PLUS_CONF}/Plotter_fcstGFS_obsGFS_ExtraTC.conf -c
> >> > ${MET_PLUS_TMP}/shared.conf_TC -c
${MET_PLUS_CONF}/system_TC.conf*
> >> >
> >> > The system_TC.conf is of little concern because it is mostly
unchanged
> >> > except for the OUTPUT_BASE, which is changed specifically to
where I
> >> plan
> >> > on storing the TC output. The shared_TC.conf, on the other
hand, is
> >> used
> >> > to replace the content inside my
Plotter_fcstGFS_obsGFS_ExtraTC.conf
> >> file.
> >> > In here I define
> >> > the ADECK and BDECK input directories based on the where the
template
> >> files
> >> > above are being stored. Since I have several runs encompassing
the
> >> > tropical cyclone event - 20190924 through 20190929 - I have
specified
> my
> >> > INIT_BEG as 20190924 and my INIT_END as 20190929. I have also
set my
> >> > INIT_INCREMENT to 86400 since I'm only processing the 00z cycle
for
> >> now. I
> >> > have a couple of lines of code before the configure file that
joins
> >> these
> >> > dates in a comma separated list such that the
> CYCLONE_PLOTTER_INIT_DATE
> >> is
> >> > "20190924,20190925,...,20190929". I'm not sure if that is
allowed
> >> though.
> >> >
> >> > When running the script the log that is generated produces 0
errors.
> >> > However, I find *two issues*: *1) there is only one file being
> carried
> >> > over to generate results; that is the 20190924 date - there
should be
> >> 6*,
> >> > and *2) The files that are created are empty and thus the plot
that is
> >> > generated is simply a map with no data overlaid.*
> >> >
> >> > In attempting to debug further, I found something that looks
important
> >> > shown here:
> >> >
> >> > DEBUG 3: Rejected for init mask = 0
> >> > DEBUG 3: Rejected for valid mask = 0
> >> > *DEBUG 2: Matching 0 ADECK tracks to 0 BDECK tracks.*
> >> > DEBUG 1: Watch/Warning file:
> >> >
> >> >
> >>
>
/gpfs/dell2/emc/verification/noscrub/Julie.Prestopnik/met/9.1_beta2/share/met/tc_data/wwpts_us.txt
> >> > DEBUG 5: TrackPairInfoArray: NPairs = 0, NAlloc = 0, Pairs:
> >> > DEBUG 5:
> >> > DEBUG 1: Output file:
> >> >
> >> >
> >>
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs/201909/AL_q2019092400.gfs.PRNA20.tcst
> >> > 08/26 18:35:12.422 metplus.TCPairs (command_runner.py:191)
DEBUG:
> >> Finished
> >> > running
> >> >
> >> >
> >>
>
/gpfs/dell2/emc/verification/noscrub/Julie.Prestopnik/met/9.1_beta2/bin/tc_pairs
> >> > in 0:00:00.607349
> >> > 08/26 18:35:12.422 metplus.CyclonePlotter
> >> (cyclone_plotter_wrapper.py:77)
> >> > DEBUG: Begin retrieving data...
> >> > 08/26 18:35:12.422 metplus.CyclonePlotter
> >> (cyclone_plotter_wrapper.py:83)
> >> > DEBUG: Generate plot for all files in the
> >> >
> >> >
> >>
>
directory/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs
> >> > 08/26 18:35:12.423 metplus.CyclonePlotter
> >> (cyclone_plotter_wrapper.py:98)
> >> > DEBUG: Parsing file
> >> >
> >> >
> >>
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs/201909/AL_q2019092400.gfs.PRNA20.tcst
> >> > 08/26 18:35:19.778 metplus (config_launcher.py:520) DEBUG:
Setting
> >> [config]
> >> > SCRUB_STAGING_DIR to default value: False.
> >> > 08/26 18:35:19.779 metplus INFO:
> >> >
> >> >
> >>
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/metplus_final.conf:
> >> > write metplus.conf here
> >> > 08/26 18:35:19.783 metplus (met_util.py:213) DEBUG: METplus
took
> >> > 0:00:08.783240 to run.
> >> > 08/26 18:35:19.783 metplus INFO: METplus has successfully
finished
> >> running.
> >> >
> >> > Specifically the line that says matching 0 ADECK tracks to 0
BDECK
> >> tracks
> >> > lends me to believe that this is the reason why nothing is
being added
> >> to
> >> > my TC_PAIRS_OUTPUT_TEMPLATE file, and therefore no plots
generated on
> a
> >> > map.
> >> >
> >> > What's interesting is the file that is extracted for BDECK is
for
> >> hurricane
> >> > Lorenzo and includes all the dates coinciding with the ADECK
file - a
> >> case
> >> > that is run coinciding with Lorenzo. I've checked this.
> >> >
> >> > One possible issue is the formatting of these files. I looked
in the
> >> test
> >> > case that comes with the cloned met stuff. In there the
header/field
> >> info
> >> > looks something like this:
> >> >
> >> >
> >> >
> >>
>
*/gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/METplus/source_data/met_test/new/track_data/201412/amlq2014123118.gfso.0104*
> >> >
> >> > ML, 0104, 2014121318_F000_478N_1780W_FOF, 2014121318, 03, GFSO,
000,
> >> 478N,
> >> > 1780W, 38, 986, XX, 34, NEQ, 0283, 0640, 0645, 0000, 990,
133,
> -99,
> >> > 69, 227, -99, -9999, -9999, 208, 315, 135, 156, 0063
> >> >
> >> > This differs from the structure of both the ADECK and BDECK
input
> files:
> >> >
> >> > *BDECK:
> >> >
/gpfs/hps3/emc/hwrf/noscrub/emc.hurpara/trak/abdeck/btk/bal132019.dat
> *
> >> >
> >> > AL, 13, 2019092218, , BEST, 0, 109N, 188W, 25, 1008, LO,
0,
> ,
> >> > 0, 0, 0, 0, 1012, 100, 90, 40, 0, L, 0, ,
0,
> >> 0,
> >> > INVEST, S, 0, , 0, 0, 0, 0, genesis-num, 027,
> >> >
> >> > *ADECK:
> >> >
> >> >
> >>
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400*
> >> >
> >> > AL, 0912, 03, PRNA, 000, 157N, 655W, 27, 1008, XX, 34, NEQ,
0000,
> >> 0000,
> >> > 0000, 0000, -9999, -9999, 85
> >> >
> >> > My suspicion is that these formats are not compatible. There
are
> major
> >> > differences between these formats in my opinion. The ADECK was
> >> generated
> >> > automatically when making a run while BDECK is done by someone
else.
> >> Those
> >> > files are stored whenever a storm is identified.
> >> >
> >> > I think what I'll need to do next is start by running TCRMW to
produce
> >> the
> >> > ADECK files formatted by metplus. I have the grib files for
that.
> The
> >> > question next, though, is the BDECK files okay for processing?
> >> >
> >> >
> >> >
> >> > --
> >> > Edward Strobach
> >> > EMC/NCEP/NWS/
> >> > IMSG Contractor
> >> > Cubicle#: 2029
> >> > 301-683-3717
> >> >
> >> >
> >>
> >> --
> >> George McCabe - Software Engineer III
> >> National Center for Atmospheric Research
> >> Research Applications Laboratory
> >> 303-497-2768
> >> ---
> >> My working day may not be your working day. Please do not feel
obliged
> to
> >> reply to this email outside of your normal working hours.
> >>
> >>
> >
> > --
> > Edward Strobach
> > EMC/NCEP/NWS/
> > IMSG Contractor
> > Cubicle#: 2029
> > 301-683-3717
> >
>
>
> --
> Edward Strobach
> EMC/NCEP/NWS/
> IMSG Contractor
> Cubicle#: 2029
> 301-683-3717
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: Tropical Cyclone verification: TCPairs and CyclonePlotter
From: Edward Strobach - NOAA Affiliate
Time: Fri Aug 28 14:41:53 2020
Thanks for that. I left the TC_PAIRS_CYCLONE, TC_PAIRS_BASIN,
TC_PAIRS_STORM_NAME, TC_PAIRS_STORM_ID alone. I may have to develop a
way
to fill this information in without manually specifying it. If that
proves
futile, then I'll take a look at using the GFDL vortex tracker.
Thanks
On Thu, Aug 27, 2020 at 3:37 PM George McCabe via RT
<met_help at ucar.edu>
wrote:
> Hi Edward,
>
> From John HG:
>
>
>
> *The MET-TC tools read track files in ATCF format. Those track files
can be
> generated from gridded model output by running software. One such
tracker
> is supported to the community by the DTC and is named the "GFDL
Vortex
> Tracker":http://dtcenter.org/community-code/gfdl-vortex-tracker
> <http://dtcenter.org/community-code/gfdl-vortex-tracker>*
> *It is not expressly part of METplus but would be very useful for
tracking
> storms in GRIB files.*
>
> My guess as to why none of the tracks are being used has to do with
the
> METplus configurations to filter data. I would check the values for
> TC_PAIRS_CYCLONE, TC_PAIRS_BASIN, TC_PAIRS_STORM_NAME,
TC_PAIRS_STORM_ID,
> and MODEL. The values you put in these lists determine which storms
to use.
> You can leave them blank to use any.
>
> Also, you could bump up the MET log verbosity and see more
information
> about why they are being rejected:
>
> # to affect all MET tools set:
> LOG_MET_VERBOSITY = 10
>
> # to affect only TCPairs set:
> LOG_TC_PAIRS_VERBOSITY = 10
>
> - George
>
> On Thu, Aug 27, 2020 at 10:53 AM Edward Strobach - NOAA Affiliate
via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458 >
> >
> > Hi,
> >
> > I did decide to remove CyclonePlotter for now and use the
TCPairs_Config
> > file along with settings that I set in a shared_TC.conf that I
store in a
> > temporary directory to process the ADECK and BDECK data that I
have.
> Files
> > are generated for all the dates that span INIT_BEG and INIT_END,
thanks
> to
> > setting loop_by to INIT. However, only the header is written out,
and I
> > suspect it is do to what the log indicates here:
> >
> > DEBUG 4: [File 1 of 5] Used 0 of 168 lines read from file
> >
> >
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
> > DEBUG 4: [File 2 of 5] Used 0 of 168 lines read from file
> >
> >
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
> > DEBUG 4: [File 3 of 5] Used 0 of 168 lines read from file
> >
> >
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
> > DEBUG 4: [File 4 of 5] Used 0 of 168 lines read from file
> >
> >
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
> > DEBUG 4: [File 5 of 5] Used 0 of 168 lines read from file
> >
> >
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
> > DEBUG 3: Used 0 of 840 lines read from 5 file(s).
> > DEBUG 3: Identified 0 track(s).
> > DEBUG 5: TrackInfoArray: NTracks = 0, NAlloc = 0, Tracks:
> > DEBUG 5:
> > DEBUG 2: Deriving 12-hour interpolated ADECK tracks.
> > DEBUG 2: Finished adding 0 and replacing 0 Interp12 track(s).
> > DEBUG 2: Deriving 0 ADECK consensus model(s).
> > DEBUG 2: Added 0 ADECK consensus tracks(s).
> > DEBUG 2: Deriving 0 ADECK lag model(s).
> > DEBUG 2: Added 0 ADECK lag tracks(s).
> > DEBUG 2: Deriving 0 CLIPER/SHIFOR baseline model(s).
> > DEBUG 2: Added 0 CLIPER/SHIFOR baseline track(s).
> > DEBUG 2: Filtering 0 ADECK tracks based on config file settings.
> > DEBUG 3: Total tracks read = 0
> > DEBUG 3: Total tracks kept = 0
> > DEBUG 3: Rejected for storm name = 0
> > DEBUG 3: Rejected for valid time = 0
> > DEBUG 3: Rejected for required lead times = 0
> > DEBUG 3: Rejected for init mask = 0
> > DEBUG 3: Rejected for valid mask = 0
> > DEBUG 2: Matching 0 ADECK tracks to 1 BDECK tracks.
> >
> > I'm still thinking that the formatting is off in this file. I
surmise
> that
> > it's not able to group the data because the column information is
not in
> > the order that met would want it to be in. These files do not
have any
> > header info, so it has to rely on the correct ordering, or so I
suspect.
> > My interpretation, however, may be flawed. I see 6 separate TC
based
> > metplus routines: TCPairs (I'm working on currently);
CyclonePlotter
> > (looking to use); TCStat; TCRMW; TCMPRPlotter; and TCGen. Much,
if not
> > almost all of these relies on TCPairs. Is there a routine that
I'm
> missing
> > in metplus that takes a grib file and constructs ADECK files?
> >
> >
> >
> > On Thu, Aug 27, 2020 at 11:52 AM Edward Strobach - NOAA Affiliate
<
> > edward.strobach at noaa.gov> wrote:
> >
> > > Hi George, I figured the BDECK must be compatible, so no worries
there.
> > > Thanks for the link and the heads up about reformatting. I will
use
> the
> > > example to modify what I currently have to see the outcome.
Thanks
> > again.
> > >
> > > On Wed, Aug 26, 2020 at 6:57 PM George McCabe via RT <
> met_help at ucar.edu>
> > > wrote:
> > >
> > >> Hi Edward,
> > >>
> > >> The BDECK data is compatible to be processed. The use case you
used as
> > an
> > >> example uses extra tropical cyclone data from SBU that is
reformatted
> by
> > >> TCPairs wrapper before processing. You will see your config
file has
> the
> > >> following:
> > >>
> > >> ## tc-pairs filtering optionsTC_PAIRS_REFORMAT_DECK =
> > >> yesTC_PAIRS_REFORMAT_TYPE = SBU
> > >>
> > >> Changing TC_PAIRS_REFORMAT_DECK to 'no' or 'false' will skip
the
> > >> reformatting step. There is another use case in the repo that
uses
> data
> > >> that looks like yours (at least as far as I can tell).
> > >>
> > >>
> > >>
> >
>
https://dtcenter.github.io/METplus/generated/met_tool_wrapper/TCPairs/TCPairs_tropical.html#metplus-
configuration
> > >>
> > >> This use case uses BDECK data in met_test/new/hwrf/bdeck. Here
is part
> > of
> > >> the file called bal062018.dat:
> > >>
> > >> AL, 06, 2018082906, , BEST, 0, 129N, 109W, 10, 1010, DB,
0,
> ,
> > >> 0, 0, 0, 0, 0, 0, 0, 0, 0, , 0, ,
0,
> > 0,
> > >> GENESIS019, , 0, , 0, 0, 0, 0, genesis-num,
019,
> > >>
> > >>
> > >> Let me know if changing that setting doesn't work and I can
take a
> > closer
> > >> look.
> > >>
> > >> Thanks,
> > >> George
> > >>
> > >> On Wed, Aug 26, 2020 at 1:17 PM Edward Strobach - NOAA
Affiliate via
> RT
> > <
> > >> met_help at ucar.edu> wrote:
> > >>
> > >> >
> > >> > Wed Aug 26 13:13:28 2020: Request 96458 was acted upon.
> > >> > Transaction: Ticket created by edward.strobach at noaa.gov
> > >> > Queue: met_help
> > >> > Subject: Tropical Cyclone verification: TCPairs and
> > CyclonePlotter
> > >> > Owner: Nobody
> > >> > Requestors: edward.strobach at noaa.gov
> > >> > Status: new
> > >> > Ticket <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458
> > >
> > >> >
> > >> >
> > >> > Good afternoon,
> > >> >
> > >> > I found it relatively straightforward to set-up some scripts
to
> > process
> > >> > forecast output along with obs by using this resource:
> > >> >
> > >> >
> > >>
> >
>
https://dtcenter.github.io/METplus/generated/model_applications/tc_and_extra_tc/Plotter_fcstGFS_obsGFS_ExtraTC.html#sphx-
glr-generated-model-applications-tc-and-extra-tc-plotter-fcstgfs-
obsgfs-extratc-py
> > >> > .
> > >> > I have been running the global model which generates track
data for
> > the
> > >> > post processing step following this naming convention:
> > >> > atcfunix.gfs.YYYYMMDDHH. The best track data is stored here
on
> WCOSS:
> > >> > /gpfs/hps3/emc/hwrf/noscrub/emc.hurpara/trak/abdeck/btk/.
> > >> >
> > >> > I have a python script that determines which file to use
based on
> > basin
> > >> > information, storm name, and date. That is piped into the
> > >> > TC_PAIRS_BDECK_TEMPLATE while the forecast file above is
piped into
> > the
> > >> > TC_PAIRS_ADECK_TEMPLATE this way:
atcfunix.gfs.{date?fmt=%Y%m%d%H}.
> > >> >
> > >> > Inside my main script
> > >> >
> > >> >
> > >>
> >
>
*(/gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/METplus/scripts/run_metplus_TCPlotter.sh*,
> > >> > I have three configure files that being processed like so:*
> > >> > ${MET_PLUS}/ush/master_metplus.py -c
> > >> > ${MET_PLUS_CONF}/Plotter_fcstGFS_obsGFS_ExtraTC.conf -c
> > >> > ${MET_PLUS_TMP}/shared.conf_TC -c
${MET_PLUS_CONF}/system_TC.conf*
> > >> >
> > >> > The system_TC.conf is of little concern because it is mostly
> unchanged
> > >> > except for the OUTPUT_BASE, which is changed specifically to
where I
> > >> plan
> > >> > on storing the TC output. The shared_TC.conf, on the other
hand, is
> > >> used
> > >> > to replace the content inside my
Plotter_fcstGFS_obsGFS_ExtraTC.conf
> > >> file.
> > >> > In here I define
> > >> > the ADECK and BDECK input directories based on the where the
> template
> > >> files
> > >> > above are being stored. Since I have several runs
encompassing the
> > >> > tropical cyclone event - 20190924 through 20190929 - I have
> specified
> > my
> > >> > INIT_BEG as 20190924 and my INIT_END as 20190929. I have
also set
> my
> > >> > INIT_INCREMENT to 86400 since I'm only processing the 00z
cycle for
> > >> now. I
> > >> > have a couple of lines of code before the configure file that
joins
> > >> these
> > >> > dates in a comma separated list such that the
> > CYCLONE_PLOTTER_INIT_DATE
> > >> is
> > >> > "20190924,20190925,...,20190929". I'm not sure if that is
allowed
> > >> though.
> > >> >
> > >> > When running the script the log that is generated produces 0
errors.
> > >> > However, I find *two issues*: *1) there is only one file
being
> > carried
> > >> > over to generate results; that is the 20190924 date - there
should
> be
> > >> 6*,
> > >> > and *2) The files that are created are empty and thus the
plot that
> is
> > >> > generated is simply a map with no data overlaid.*
> > >> >
> > >> > In attempting to debug further, I found something that looks
> important
> > >> > shown here:
> > >> >
> > >> > DEBUG 3: Rejected for init mask = 0
> > >> > DEBUG 3: Rejected for valid mask = 0
> > >> > *DEBUG 2: Matching 0 ADECK tracks to 0 BDECK tracks.*
> > >> > DEBUG 1: Watch/Warning file:
> > >> >
> > >> >
> > >>
> >
>
/gpfs/dell2/emc/verification/noscrub/Julie.Prestopnik/met/9.1_beta2/share/met/tc_data/wwpts_us.txt
> > >> > DEBUG 5: TrackPairInfoArray: NPairs = 0, NAlloc = 0, Pairs:
> > >> > DEBUG 5:
> > >> > DEBUG 1: Output file:
> > >> >
> > >> >
> > >>
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs/201909/AL_q2019092400.gfs.PRNA20.tcst
> > >> > 08/26 18:35:12.422 metplus.TCPairs (command_runner.py:191)
DEBUG:
> > >> Finished
> > >> > running
> > >> >
> > >> >
> > >>
> >
>
/gpfs/dell2/emc/verification/noscrub/Julie.Prestopnik/met/9.1_beta2/bin/tc_pairs
> > >> > in 0:00:00.607349
> > >> > 08/26 18:35:12.422 metplus.CyclonePlotter
> > >> (cyclone_plotter_wrapper.py:77)
> > >> > DEBUG: Begin retrieving data...
> > >> > 08/26 18:35:12.422 metplus.CyclonePlotter
> > >> (cyclone_plotter_wrapper.py:83)
> > >> > DEBUG: Generate plot for all files in the
> > >> >
> > >> >
> > >>
> >
>
directory/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs
> > >> > 08/26 18:35:12.423 metplus.CyclonePlotter
> > >> (cyclone_plotter_wrapper.py:98)
> > >> > DEBUG: Parsing file
> > >> >
> > >> >
> > >>
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs/201909/AL_q2019092400.gfs.PRNA20.tcst
> > >> > 08/26 18:35:19.778 metplus (config_launcher.py:520) DEBUG:
Setting
> > >> [config]
> > >> > SCRUB_STAGING_DIR to default value: False.
> > >> > 08/26 18:35:19.779 metplus INFO:
> > >> >
> > >> >
> > >>
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/metplus_final.conf:
> > >> > write metplus.conf here
> > >> > 08/26 18:35:19.783 metplus (met_util.py:213) DEBUG: METplus
took
> > >> > 0:00:08.783240 to run.
> > >> > 08/26 18:35:19.783 metplus INFO: METplus has successfully
finished
> > >> running.
> > >> >
> > >> > Specifically the line that says matching 0 ADECK tracks to 0
BDECK
> > >> tracks
> > >> > lends me to believe that this is the reason why nothing is
being
> added
> > >> to
> > >> > my TC_PAIRS_OUTPUT_TEMPLATE file, and therefore no plots
generated
> on
> > a
> > >> > map.
> > >> >
> > >> > What's interesting is the file that is extracted for BDECK is
for
> > >> hurricane
> > >> > Lorenzo and includes all the dates coinciding with the ADECK
file -
> a
> > >> case
> > >> > that is run coinciding with Lorenzo. I've checked this.
> > >> >
> > >> > One possible issue is the formatting of these files. I
looked in
> the
> > >> test
> > >> > case that comes with the cloned met stuff. In there the
> header/field
> > >> info
> > >> > looks something like this:
> > >> >
> > >> >
> > >> >
> > >>
> >
>
*/gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/METplus/source_data/met_test/new/track_data/201412/amlq2014123118.gfso.0104*
> > >> >
> > >> > ML, 0104, 2014121318_F000_478N_1780W_FOF, 2014121318, 03,
GFSO, 000,
> > >> 478N,
> > >> > 1780W, 38, 986, XX, 34, NEQ, 0283, 0640, 0645, 0000, 990,
133,
> > -99,
> > >> > 69, 227, -99, -9999, -9999, 208, 315, 135, 156, 0063
> > >> >
> > >> > This differs from the structure of both the ADECK and BDECK
input
> > files:
> > >> >
> > >> > *BDECK:
> > >> >
>
/gpfs/hps3/emc/hwrf/noscrub/emc.hurpara/trak/abdeck/btk/bal132019.dat
> > *
> > >> >
> > >> > AL, 13, 2019092218, , BEST, 0, 109N, 188W, 25, 1008,
LO, 0,
> > ,
> > >> > 0, 0, 0, 0, 1012, 100, 90, 40, 0, L, 0,
, 0,
> > >> 0,
> > >> > INVEST, S, 0, , 0, 0, 0, 0, genesis-num,
027,
> > >> >
> > >> > *ADECK:
> > >> >
> > >> >
> > >>
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400*
> > >> >
> > >> > AL, 0912, 03, PRNA, 000, 157N, 655W, 27, 1008, XX, 34,
NEQ, 0000,
> > >> 0000,
> > >> > 0000, 0000, -9999, -9999, 85
> > >> >
> > >> > My suspicion is that these formats are not compatible. There
are
> > major
> > >> > differences between these formats in my opinion. The ADECK
was
> > >> generated
> > >> > automatically when making a run while BDECK is done by
someone else.
> > >> Those
> > >> > files are stored whenever a storm is identified.
> > >> >
> > >> > I think what I'll need to do next is start by running TCRMW
to
> produce
> > >> the
> > >> > ADECK files formatted by metplus. I have the grib files for
that.
> > The
> > >> > question next, though, is the BDECK files okay for
processing?
> > >> >
> > >> >
> > >> >
> > >> > --
> > >> > Edward Strobach
> > >> > EMC/NCEP/NWS/
> > >> > IMSG Contractor
> > >> > Cubicle#: 2029
> > >> > 301-683-3717
> > >> >
> > >> >
> > >>
> > >> --
> > >> George McCabe - Software Engineer III
> > >> National Center for Atmospheric Research
> > >> Research Applications Laboratory
> > >> 303-497-2768
> > >> ---
> > >> My working day may not be your working day. Please do not feel
obliged
> > to
> > >> reply to this email outside of your normal working hours.
> > >>
> > >>
> > >
> > > --
> > > Edward Strobach
> > > EMC/NCEP/NWS/
> > > IMSG Contractor
> > > Cubicle#: 2029
> > > 301-683-3717
> > >
> >
> >
> > --
> > Edward Strobach
> > EMC/NCEP/NWS/
> > IMSG Contractor
> > Cubicle#: 2029
> > 301-683-3717
> >
> >
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>
--
Edward Strobach
EMC/NCEP/NWS/
IMSG Contractor
Cubicle#: 2029
301-683-3717
------------------------------------------------
Subject: Tropical Cyclone verification: TCPairs and CyclonePlotter
From: Edward Strobach - NOAA Affiliate
Time: Fri Aug 28 14:53:57 2020
It seems to me, though, that these options are rather redundant. If
one
supplies a file with the Basin name, storm id, and storm name, then it
would seem to me that all one would need to do is supply the storm id
(or
name) with everything else being extracted from that using the
contents in
the file.
On Fri, Aug 28, 2020 at 4:41 PM Edward Strobach - NOAA Affiliate <
edward.strobach at noaa.gov> wrote:
> Thanks for that. I left the TC_PAIRS_CYCLONE, TC_PAIRS_BASIN,
> TC_PAIRS_STORM_NAME, TC_PAIRS_STORM_ID alone. I may have to develop
a way
> to fill this information in without manually specifying it. If that
proves
> futile, then I'll take a look at using the GFDL vortex tracker.
Thanks
>
> On Thu, Aug 27, 2020 at 3:37 PM George McCabe via RT
<met_help at ucar.edu>
> wrote:
>
>> Hi Edward,
>>
>> From John HG:
>>
>>
>>
>> *The MET-TC tools read track files in ATCF format. Those track
files can
>> be
>> generated from gridded model output by running software. One such
tracker
>> is supported to the community by the DTC and is named the "GFDL
Vortex
>> Tracker":http://dtcenter.org/community-code/gfdl-vortex-tracker
>> <http://dtcenter.org/community-code/gfdl-vortex-tracker>*
>> *It is not expressly part of METplus but would be very useful for
tracking
>> storms in GRIB files.*
>>
>> My guess as to why none of the tracks are being used has to do with
the
>> METplus configurations to filter data. I would check the values for
>> TC_PAIRS_CYCLONE, TC_PAIRS_BASIN, TC_PAIRS_STORM_NAME,
TC_PAIRS_STORM_ID,
>> and MODEL. The values you put in these lists determine which storms
to
>> use.
>> You can leave them blank to use any.
>>
>> Also, you could bump up the MET log verbosity and see more
information
>> about why they are being rejected:
>>
>> # to affect all MET tools set:
>> LOG_MET_VERBOSITY = 10
>>
>> # to affect only TCPairs set:
>> LOG_TC_PAIRS_VERBOSITY = 10
>>
>> - George
>>
>> On Thu, Aug 27, 2020 at 10:53 AM Edward Strobach - NOAA Affiliate
via RT <
>> met_help at ucar.edu> wrote:
>>
>> >
>> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458 >
>> >
>> > Hi,
>> >
>> > I did decide to remove CyclonePlotter for now and use the
TCPairs_Config
>> > file along with settings that I set in a shared_TC.conf that I
store in
>> a
>> > temporary directory to process the ADECK and BDECK data that I
have.
>> Files
>> > are generated for all the dates that span INIT_BEG and INIT_END,
thanks
>> to
>> > setting loop_by to INIT. However, only the header is written out,
and I
>> > suspect it is do to what the log indicates here:
>> >
>> > DEBUG 4: [File 1 of 5] Used 0 of 168 lines read from file
>> >
>> >
>>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
>> > DEBUG 4: [File 2 of 5] Used 0 of 168 lines read from file
>> >
>> >
>>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
>> > DEBUG 4: [File 3 of 5] Used 0 of 168 lines read from file
>> >
>> >
>>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
>> > DEBUG 4: [File 4 of 5] Used 0 of 168 lines read from file
>> >
>> >
>>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
>> > DEBUG 4: [File 5 of 5] Used 0 of 168 lines read from file
>> >
>> >
>>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
>> > DEBUG 3: Used 0 of 840 lines read from 5 file(s).
>> > DEBUG 3: Identified 0 track(s).
>> > DEBUG 5: TrackInfoArray: NTracks = 0, NAlloc = 0, Tracks:
>> > DEBUG 5:
>> > DEBUG 2: Deriving 12-hour interpolated ADECK tracks.
>> > DEBUG 2: Finished adding 0 and replacing 0 Interp12 track(s).
>> > DEBUG 2: Deriving 0 ADECK consensus model(s).
>> > DEBUG 2: Added 0 ADECK consensus tracks(s).
>> > DEBUG 2: Deriving 0 ADECK lag model(s).
>> > DEBUG 2: Added 0 ADECK lag tracks(s).
>> > DEBUG 2: Deriving 0 CLIPER/SHIFOR baseline model(s).
>> > DEBUG 2: Added 0 CLIPER/SHIFOR baseline track(s).
>> > DEBUG 2: Filtering 0 ADECK tracks based on config file settings.
>> > DEBUG 3: Total tracks read = 0
>> > DEBUG 3: Total tracks kept = 0
>> > DEBUG 3: Rejected for storm name = 0
>> > DEBUG 3: Rejected for valid time = 0
>> > DEBUG 3: Rejected for required lead times = 0
>> > DEBUG 3: Rejected for init mask = 0
>> > DEBUG 3: Rejected for valid mask = 0
>> > DEBUG 2: Matching 0 ADECK tracks to 1 BDECK tracks.
>> >
>> > I'm still thinking that the formatting is off in this file. I
surmise
>> that
>> > it's not able to group the data because the column information is
not in
>> > the order that met would want it to be in. These files do not
have any
>> > header info, so it has to rely on the correct ordering, or so I
suspect.
>> > My interpretation, however, may be flawed. I see 6 separate TC
based
>> > metplus routines: TCPairs (I'm working on currently);
CyclonePlotter
>> > (looking to use); TCStat; TCRMW; TCMPRPlotter; and TCGen. Much,
if not
>> > almost all of these relies on TCPairs. Is there a routine that
I'm
>> missing
>> > in metplus that takes a grib file and constructs ADECK files?
>> >
>> >
>> >
>> > On Thu, Aug 27, 2020 at 11:52 AM Edward Strobach - NOAA Affiliate
<
>> > edward.strobach at noaa.gov> wrote:
>> >
>> > > Hi George, I figured the BDECK must be compatible, so no
worries
>> there.
>> > > Thanks for the link and the heads up about reformatting. I
will use
>> the
>> > > example to modify what I currently have to see the outcome.
Thanks
>> > again.
>> > >
>> > > On Wed, Aug 26, 2020 at 6:57 PM George McCabe via RT <
>> met_help at ucar.edu>
>> > > wrote:
>> > >
>> > >> Hi Edward,
>> > >>
>> > >> The BDECK data is compatible to be processed. The use case you
used
>> as
>> > an
>> > >> example uses extra tropical cyclone data from SBU that is
>> reformatted by
>> > >> TCPairs wrapper before processing. You will see your config
file has
>> the
>> > >> following:
>> > >>
>> > >> ## tc-pairs filtering optionsTC_PAIRS_REFORMAT_DECK =
>> > >> yesTC_PAIRS_REFORMAT_TYPE = SBU
>> > >>
>> > >> Changing TC_PAIRS_REFORMAT_DECK to 'no' or 'false' will skip
the
>> > >> reformatting step. There is another use case in the repo that
uses
>> data
>> > >> that looks like yours (at least as far as I can tell).
>> > >>
>> > >>
>> > >>
>> >
>>
https://dtcenter.github.io/METplus/generated/met_tool_wrapper/TCPairs/TCPairs_tropical.html#metplus-
configuration
>> > >>
>> > >> This use case uses BDECK data in met_test/new/hwrf/bdeck. Here
is
>> part
>> > of
>> > >> the file called bal062018.dat:
>> > >>
>> > >> AL, 06, 2018082906, , BEST, 0, 129N, 109W, 10, 1010, DB,
0,
>> ,
>> > >> 0, 0, 0, 0, 0, 0, 0, 0, 0, , 0,
, 0,
>> > 0,
>> > >> GENESIS019, , 0, , 0, 0, 0, 0, genesis-num,
019,
>> > >>
>> > >>
>> > >> Let me know if changing that setting doesn't work and I can
take a
>> > closer
>> > >> look.
>> > >>
>> > >> Thanks,
>> > >> George
>> > >>
>> > >> On Wed, Aug 26, 2020 at 1:17 PM Edward Strobach - NOAA
Affiliate via
>> RT
>> > <
>> > >> met_help at ucar.edu> wrote:
>> > >>
>> > >> >
>> > >> > Wed Aug 26 13:13:28 2020: Request 96458 was acted upon.
>> > >> > Transaction: Ticket created by edward.strobach at noaa.gov
>> > >> > Queue: met_help
>> > >> > Subject: Tropical Cyclone verification: TCPairs and
>> > CyclonePlotter
>> > >> > Owner: Nobody
>> > >> > Requestors: edward.strobach at noaa.gov
>> > >> > Status: new
>> > >> > Ticket <URL:
>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458
>> > >
>> > >> >
>> > >> >
>> > >> > Good afternoon,
>> > >> >
>> > >> > I found it relatively straightforward to set-up some scripts
to
>> > process
>> > >> > forecast output along with obs by using this resource:
>> > >> >
>> > >> >
>> > >>
>> >
>>
https://dtcenter.github.io/METplus/generated/model_applications/tc_and_extra_tc/Plotter_fcstGFS_obsGFS_ExtraTC.html#sphx-
glr-generated-model-applications-tc-and-extra-tc-plotter-fcstgfs-
obsgfs-extratc-py
>> > >> > .
>> > >> > I have been running the global model which generates track
data for
>> > the
>> > >> > post processing step following this naming convention:
>> > >> > atcfunix.gfs.YYYYMMDDHH. The best track data is stored here
on
>> WCOSS:
>> > >> > /gpfs/hps3/emc/hwrf/noscrub/emc.hurpara/trak/abdeck/btk/.
>> > >> >
>> > >> > I have a python script that determines which file to use
based on
>> > basin
>> > >> > information, storm name, and date. That is piped into the
>> > >> > TC_PAIRS_BDECK_TEMPLATE while the forecast file above is
piped into
>> > the
>> > >> > TC_PAIRS_ADECK_TEMPLATE this way:
>> atcfunix.gfs.{date?fmt=%Y%m%d%H}.
>> > >> >
>> > >> > Inside my main script
>> > >> >
>> > >> >
>> > >>
>> >
>>
*(/gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/METplus/scripts/run_metplus_TCPlotter.sh*,
>> > >> > I have three configure files that being processed like so:*
>> > >> > ${MET_PLUS}/ush/master_metplus.py -c
>> > >> > ${MET_PLUS_CONF}/Plotter_fcstGFS_obsGFS_ExtraTC.conf -c
>> > >> > ${MET_PLUS_TMP}/shared.conf_TC -c
${MET_PLUS_CONF}/system_TC.conf*
>> > >> >
>> > >> > The system_TC.conf is of little concern because it is mostly
>> unchanged
>> > >> > except for the OUTPUT_BASE, which is changed specifically to
where
>> I
>> > >> plan
>> > >> > on storing the TC output. The shared_TC.conf, on the other
hand,
>> is
>> > >> used
>> > >> > to replace the content inside my
>> Plotter_fcstGFS_obsGFS_ExtraTC.conf
>> > >> file.
>> > >> > In here I define
>> > >> > the ADECK and BDECK input directories based on the where the
>> template
>> > >> files
>> > >> > above are being stored. Since I have several runs
encompassing the
>> > >> > tropical cyclone event - 20190924 through 20190929 - I have
>> specified
>> > my
>> > >> > INIT_BEG as 20190924 and my INIT_END as 20190929. I have
also set
>> my
>> > >> > INIT_INCREMENT to 86400 since I'm only processing the 00z
cycle for
>> > >> now. I
>> > >> > have a couple of lines of code before the configure file
that joins
>> > >> these
>> > >> > dates in a comma separated list such that the
>> > CYCLONE_PLOTTER_INIT_DATE
>> > >> is
>> > >> > "20190924,20190925,...,20190929". I'm not sure if that is
allowed
>> > >> though.
>> > >> >
>> > >> > When running the script the log that is generated produces 0
>> errors.
>> > >> > However, I find *two issues*: *1) there is only one file
being
>> > carried
>> > >> > over to generate results; that is the 20190924 date - there
should
>> be
>> > >> 6*,
>> > >> > and *2) The files that are created are empty and thus the
plot
>> that is
>> > >> > generated is simply a map with no data overlaid.*
>> > >> >
>> > >> > In attempting to debug further, I found something that looks
>> important
>> > >> > shown here:
>> > >> >
>> > >> > DEBUG 3: Rejected for init mask = 0
>> > >> > DEBUG 3: Rejected for valid mask = 0
>> > >> > *DEBUG 2: Matching 0 ADECK tracks to 0 BDECK tracks.*
>> > >> > DEBUG 1: Watch/Warning file:
>> > >> >
>> > >> >
>> > >>
>> >
>>
/gpfs/dell2/emc/verification/noscrub/Julie.Prestopnik/met/9.1_beta2/share/met/tc_data/wwpts_us.txt
>> > >> > DEBUG 5: TrackPairInfoArray: NPairs = 0, NAlloc = 0, Pairs:
>> > >> > DEBUG 5:
>> > >> > DEBUG 1: Output file:
>> > >> >
>> > >> >
>> > >>
>> >
>>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs/201909/AL_q2019092400.gfs.PRNA20.tcst
>> > >> > 08/26 18:35:12.422 metplus.TCPairs (command_runner.py:191)
DEBUG:
>> > >> Finished
>> > >> > running
>> > >> >
>> > >> >
>> > >>
>> >
>>
/gpfs/dell2/emc/verification/noscrub/Julie.Prestopnik/met/9.1_beta2/bin/tc_pairs
>> > >> > in 0:00:00.607349
>> > >> > 08/26 18:35:12.422 metplus.CyclonePlotter
>> > >> (cyclone_plotter_wrapper.py:77)
>> > >> > DEBUG: Begin retrieving data...
>> > >> > 08/26 18:35:12.422 metplus.CyclonePlotter
>> > >> (cyclone_plotter_wrapper.py:83)
>> > >> > DEBUG: Generate plot for all files in the
>> > >> >
>> > >> >
>> > >>
>> >
>>
directory/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs
>> > >> > 08/26 18:35:12.423 metplus.CyclonePlotter
>> > >> (cyclone_plotter_wrapper.py:98)
>> > >> > DEBUG: Parsing file
>> > >> >
>> > >> >
>> > >>
>> >
>>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs/201909/AL_q2019092400.gfs.PRNA20.tcst
>> > >> > 08/26 18:35:19.778 metplus (config_launcher.py:520) DEBUG:
Setting
>> > >> [config]
>> > >> > SCRUB_STAGING_DIR to default value: False.
>> > >> > 08/26 18:35:19.779 metplus INFO:
>> > >> >
>> > >> >
>> > >>
>> >
>>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/metplus_final.conf:
>> > >> > write metplus.conf here
>> > >> > 08/26 18:35:19.783 metplus (met_util.py:213) DEBUG: METplus
took
>> > >> > 0:00:08.783240 to run.
>> > >> > 08/26 18:35:19.783 metplus INFO: METplus has successfully
finished
>> > >> running.
>> > >> >
>> > >> > Specifically the line that says matching 0 ADECK tracks to 0
BDECK
>> > >> tracks
>> > >> > lends me to believe that this is the reason why nothing is
being
>> added
>> > >> to
>> > >> > my TC_PAIRS_OUTPUT_TEMPLATE file, and therefore no plots
generated
>> on
>> > a
>> > >> > map.
>> > >> >
>> > >> > What's interesting is the file that is extracted for BDECK
is for
>> > >> hurricane
>> > >> > Lorenzo and includes all the dates coinciding with the ADECK
file
>> - a
>> > >> case
>> > >> > that is run coinciding with Lorenzo. I've checked this.
>> > >> >
>> > >> > One possible issue is the formatting of these files. I
looked in
>> the
>> > >> test
>> > >> > case that comes with the cloned met stuff. In there the
>> header/field
>> > >> info
>> > >> > looks something like this:
>> > >> >
>> > >> >
>> > >> >
>> > >>
>> >
>>
*/gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/METplus/source_data/met_test/new/track_data/201412/amlq2014123118.gfso.0104*
>> > >> >
>> > >> > ML, 0104, 2014121318_F000_478N_1780W_FOF, 2014121318, 03,
GFSO,
>> 000,
>> > >> 478N,
>> > >> > 1780W, 38, 986, XX, 34, NEQ, 0283, 0640, 0645, 0000,
990, 133,
>> > -99,
>> > >> > 69, 227, -99, -9999, -9999, 208, 315, 135, 156, 0063
>> > >> >
>> > >> > This differs from the structure of both the ADECK and BDECK
input
>> > files:
>> > >> >
>> > >> > *BDECK:
>> > >> >
>>
/gpfs/hps3/emc/hwrf/noscrub/emc.hurpara/trak/abdeck/btk/bal132019.dat
>> > *
>> > >> >
>> > >> > AL, 13, 2019092218, , BEST, 0, 109N, 188W, 25, 1008,
LO, 0,
>> > ,
>> > >> > 0, 0, 0, 0, 1012, 100, 90, 40, 0, L, 0,
,
>> 0,
>> > >> 0,
>> > >> > INVEST, S, 0, , 0, 0, 0, 0, genesis-num,
027,
>> > >> >
>> > >> > *ADECK:
>> > >> >
>> > >> >
>> > >>
>> >
>>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400*
>> > >> >
>> > >> > AL, 0912, 03, PRNA, 000, 157N, 655W, 27, 1008, XX, 34,
NEQ,
>> 0000,
>> > >> 0000,
>> > >> > 0000, 0000, -9999, -9999, 85
>> > >> >
>> > >> > My suspicion is that these formats are not compatible.
There are
>> > major
>> > >> > differences between these formats in my opinion. The ADECK
was
>> > >> generated
>> > >> > automatically when making a run while BDECK is done by
someone
>> else.
>> > >> Those
>> > >> > files are stored whenever a storm is identified.
>> > >> >
>> > >> > I think what I'll need to do next is start by running TCRMW
to
>> produce
>> > >> the
>> > >> > ADECK files formatted by metplus. I have the grib files for
that.
>> > The
>> > >> > question next, though, is the BDECK files okay for
processing?
>> > >> >
>> > >> >
>> > >> >
>> > >> > --
>> > >> > Edward Strobach
>> > >> > EMC/NCEP/NWS/
>> > >> > IMSG Contractor
>> > >> > Cubicle#: 2029
>> > >> > 301-683-3717
>> > >> >
>> > >> >
>> > >>
>> > >> --
>> > >> George McCabe - Software Engineer III
>> > >> National Center for Atmospheric Research
>> > >> Research Applications Laboratory
>> > >> 303-497-2768
>> > >> ---
>> > >> My working day may not be your working day. Please do not feel
>> obliged
>> > to
>> > >> reply to this email outside of your normal working hours.
>> > >>
>> > >>
>> > >
>> > > --
>> > > Edward Strobach
>> > > EMC/NCEP/NWS/
>> > > IMSG Contractor
>> > > Cubicle#: 2029
>> > > 301-683-3717
>> > >
>> >
>> >
>> > --
>> > Edward Strobach
>> > EMC/NCEP/NWS/
>> > IMSG Contractor
>> > Cubicle#: 2029
>> > 301-683-3717
>> >
>> >
>>
>> --
>> George McCabe - Software Engineer III
>> National Center for Atmospheric Research
>> Research Applications Laboratory
>> 303-497-2768
>> ---
>> My working day may not be your working day. Please do not feel
obliged to
>> reply to this email outside of your normal working hours.
>>
>>
>
> --
> Edward Strobach
> EMC/NCEP/NWS/
> IMSG Contractor
> Cubicle#: 2029
> 301-683-3717
>
--
Edward Strobach
EMC/NCEP/NWS/
IMSG Contractor
Cubicle#: 2029
301-683-3717
------------------------------------------------
Subject: Tropical Cyclone verification: TCPairs and CyclonePlotter
From: Edward Strobach - NOAA Affiliate
Time: Fri Aug 28 15:44:52 2020
It appears that even after adding storm_id/storm_name, that met is not
able
to identify a track in the file provided. I believe it's because
neither
the storm name or storm ID is written in the ADECK file! I may have
to
write a script that parses the text file and writes in the storm name
and
id in two separate columns, respectively. Here's a line example with
what
it currently looks like for one row of the ADECK file:
AL, 13, 2019092500, 03, PRNA, 000, 133N, 313W, 58, 993, XX, 50,
NEQ,
0054, 0000, 0000, 0052, 1013, 172, 32
If I add AL132019 and LORENZO as the last two columns of this row,
then
should that remedy the problem that I have. Do you think that would
resolve it?
On Fri, Aug 28, 2020 at 4:53 PM Edward Strobach - NOAA Affiliate <
edward.strobach at noaa.gov> wrote:
> It seems to me, though, that these options are rather redundant. If
one
> supplies a file with the Basin name, storm id, and storm name, then
it
> would seem to me that all one would need to do is supply the storm
id (or
> name) with everything else being extracted from that using the
contents in
> the file.
>
> On Fri, Aug 28, 2020 at 4:41 PM Edward Strobach - NOAA Affiliate <
> edward.strobach at noaa.gov> wrote:
>
>> Thanks for that. I left the TC_PAIRS_CYCLONE, TC_PAIRS_BASIN,
>> TC_PAIRS_STORM_NAME, TC_PAIRS_STORM_ID alone. I may have to
develop a way
>> to fill this information in without manually specifying it. If
that proves
>> futile, then I'll take a look at using the GFDL vortex tracker.
Thanks
>>
>> On Thu, Aug 27, 2020 at 3:37 PM George McCabe via RT
<met_help at ucar.edu>
>> wrote:
>>
>>> Hi Edward,
>>>
>>> From John HG:
>>>
>>>
>>>
>>> *The MET-TC tools read track files in ATCF format. Those track
files can
>>> be
>>> generated from gridded model output by running software. One such
tracker
>>> is supported to the community by the DTC and is named the "GFDL
Vortex
>>> Tracker":http://dtcenter.org/community-code/gfdl-vortex-tracker
>>> <http://dtcenter.org/community-code/gfdl-vortex-tracker>*
>>> *It is not expressly part of METplus but would be very useful for
>>> tracking
>>> storms in GRIB files.*
>>>
>>> My guess as to why none of the tracks are being used has to do
with the
>>> METplus configurations to filter data. I would check the values
for
>>> TC_PAIRS_CYCLONE, TC_PAIRS_BASIN, TC_PAIRS_STORM_NAME,
TC_PAIRS_STORM_ID,
>>> and MODEL. The values you put in these lists determine which
storms to
>>> use.
>>> You can leave them blank to use any.
>>>
>>> Also, you could bump up the MET log verbosity and see more
information
>>> about why they are being rejected:
>>>
>>> # to affect all MET tools set:
>>> LOG_MET_VERBOSITY = 10
>>>
>>> # to affect only TCPairs set:
>>> LOG_TC_PAIRS_VERBOSITY = 10
>>>
>>> - George
>>>
>>> On Thu, Aug 27, 2020 at 10:53 AM Edward Strobach - NOAA Affiliate
via RT
>>> <
>>> met_help at ucar.edu> wrote:
>>>
>>> >
>>> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458 >
>>> >
>>> > Hi,
>>> >
>>> > I did decide to remove CyclonePlotter for now and use the
>>> TCPairs_Config
>>> > file along with settings that I set in a shared_TC.conf that I
store
>>> in a
>>> > temporary directory to process the ADECK and BDECK data that I
have.
>>> Files
>>> > are generated for all the dates that span INIT_BEG and INIT_END,
>>> thanks to
>>> > setting loop_by to INIT. However, only the header is written
out, and I
>>> > suspect it is do to what the log indicates here:
>>> >
>>> > DEBUG 4: [File 1 of 5] Used 0 of 168 lines read from file
>>> >
>>> >
>>>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
>>> > DEBUG 4: [File 2 of 5] Used 0 of 168 lines read from file
>>> >
>>> >
>>>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
>>> > DEBUG 4: [File 3 of 5] Used 0 of 168 lines read from file
>>> >
>>> >
>>>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
>>> > DEBUG 4: [File 4 of 5] Used 0 of 168 lines read from file
>>> >
>>> >
>>>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
>>> > DEBUG 4: [File 5 of 5] Used 0 of 168 lines read from file
>>> >
>>> >
>>>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
>>> > DEBUG 3: Used 0 of 840 lines read from 5 file(s).
>>> > DEBUG 3: Identified 0 track(s).
>>> > DEBUG 5: TrackInfoArray: NTracks = 0, NAlloc = 0, Tracks:
>>> > DEBUG 5:
>>> > DEBUG 2: Deriving 12-hour interpolated ADECK tracks.
>>> > DEBUG 2: Finished adding 0 and replacing 0 Interp12 track(s).
>>> > DEBUG 2: Deriving 0 ADECK consensus model(s).
>>> > DEBUG 2: Added 0 ADECK consensus tracks(s).
>>> > DEBUG 2: Deriving 0 ADECK lag model(s).
>>> > DEBUG 2: Added 0 ADECK lag tracks(s).
>>> > DEBUG 2: Deriving 0 CLIPER/SHIFOR baseline model(s).
>>> > DEBUG 2: Added 0 CLIPER/SHIFOR baseline track(s).
>>> > DEBUG 2: Filtering 0 ADECK tracks based on config file settings.
>>> > DEBUG 3: Total tracks read = 0
>>> > DEBUG 3: Total tracks kept = 0
>>> > DEBUG 3: Rejected for storm name = 0
>>> > DEBUG 3: Rejected for valid time = 0
>>> > DEBUG 3: Rejected for required lead times = 0
>>> > DEBUG 3: Rejected for init mask = 0
>>> > DEBUG 3: Rejected for valid mask = 0
>>> > DEBUG 2: Matching 0 ADECK tracks to 1 BDECK tracks.
>>> >
>>> > I'm still thinking that the formatting is off in this file. I
surmise
>>> that
>>> > it's not able to group the data because the column information
is not
>>> in
>>> > the order that met would want it to be in. These files do not
have any
>>> > header info, so it has to rely on the correct ordering, or so I
>>> suspect.
>>> > My interpretation, however, may be flawed. I see 6 separate TC
based
>>> > metplus routines: TCPairs (I'm working on currently);
CyclonePlotter
>>> > (looking to use); TCStat; TCRMW; TCMPRPlotter; and TCGen. Much,
if not
>>> > almost all of these relies on TCPairs. Is there a routine that
I'm
>>> missing
>>> > in metplus that takes a grib file and constructs ADECK files?
>>> >
>>> >
>>> >
>>> > On Thu, Aug 27, 2020 at 11:52 AM Edward Strobach - NOAA
Affiliate <
>>> > edward.strobach at noaa.gov> wrote:
>>> >
>>> > > Hi George, I figured the BDECK must be compatible, so no
worries
>>> there.
>>> > > Thanks for the link and the heads up about reformatting. I
will use
>>> the
>>> > > example to modify what I currently have to see the outcome.
Thanks
>>> > again.
>>> > >
>>> > > On Wed, Aug 26, 2020 at 6:57 PM George McCabe via RT <
>>> met_help at ucar.edu>
>>> > > wrote:
>>> > >
>>> > >> Hi Edward,
>>> > >>
>>> > >> The BDECK data is compatible to be processed. The use case
you used
>>> as
>>> > an
>>> > >> example uses extra tropical cyclone data from SBU that is
>>> reformatted by
>>> > >> TCPairs wrapper before processing. You will see your config
file
>>> has the
>>> > >> following:
>>> > >>
>>> > >> ## tc-pairs filtering optionsTC_PAIRS_REFORMAT_DECK =
>>> > >> yesTC_PAIRS_REFORMAT_TYPE = SBU
>>> > >>
>>> > >> Changing TC_PAIRS_REFORMAT_DECK to 'no' or 'false' will skip
the
>>> > >> reformatting step. There is another use case in the repo that
uses
>>> data
>>> > >> that looks like yours (at least as far as I can tell).
>>> > >>
>>> > >>
>>> > >>
>>> >
>>>
https://dtcenter.github.io/METplus/generated/met_tool_wrapper/TCPairs/TCPairs_tropical.html#metplus-
configuration
>>> > >>
>>> > >> This use case uses BDECK data in met_test/new/hwrf/bdeck.
Here is
>>> part
>>> > of
>>> > >> the file called bal062018.dat:
>>> > >>
>>> > >> AL, 06, 2018082906, , BEST, 0, 129N, 109W, 10, 1010,
DB,
>>> 0, ,
>>> > >> 0, 0, 0, 0, 0, 0, 0, 0, 0, , 0,
, 0,
>>> > 0,
>>> > >> GENESIS019, , 0, , 0, 0, 0, 0, genesis-num,
019,
>>> > >>
>>> > >>
>>> > >> Let me know if changing that setting doesn't work and I can
take a
>>> > closer
>>> > >> look.
>>> > >>
>>> > >> Thanks,
>>> > >> George
>>> > >>
>>> > >> On Wed, Aug 26, 2020 at 1:17 PM Edward Strobach - NOAA
Affiliate
>>> via RT
>>> > <
>>> > >> met_help at ucar.edu> wrote:
>>> > >>
>>> > >> >
>>> > >> > Wed Aug 26 13:13:28 2020: Request 96458 was acted upon.
>>> > >> > Transaction: Ticket created by edward.strobach at noaa.gov
>>> > >> > Queue: met_help
>>> > >> > Subject: Tropical Cyclone verification: TCPairs and
>>> > CyclonePlotter
>>> > >> > Owner: Nobody
>>> > >> > Requestors: edward.strobach at noaa.gov
>>> > >> > Status: new
>>> > >> > Ticket <URL:
>>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458
>>> > >
>>> > >> >
>>> > >> >
>>> > >> > Good afternoon,
>>> > >> >
>>> > >> > I found it relatively straightforward to set-up some
scripts to
>>> > process
>>> > >> > forecast output along with obs by using this resource:
>>> > >> >
>>> > >> >
>>> > >>
>>> >
>>>
https://dtcenter.github.io/METplus/generated/model_applications/tc_and_extra_tc/Plotter_fcstGFS_obsGFS_ExtraTC.html#sphx-
glr-generated-model-applications-tc-and-extra-tc-plotter-fcstgfs-
obsgfs-extratc-py
>>> > >> > .
>>> > >> > I have been running the global model which generates track
data
>>> for
>>> > the
>>> > >> > post processing step following this naming convention:
>>> > >> > atcfunix.gfs.YYYYMMDDHH. The best track data is stored
here on
>>> WCOSS:
>>> > >> > /gpfs/hps3/emc/hwrf/noscrub/emc.hurpara/trak/abdeck/btk/.
>>> > >> >
>>> > >> > I have a python script that determines which file to use
based on
>>> > basin
>>> > >> > information, storm name, and date. That is piped into the
>>> > >> > TC_PAIRS_BDECK_TEMPLATE while the forecast file above is
piped
>>> into
>>> > the
>>> > >> > TC_PAIRS_ADECK_TEMPLATE this way:
>>> atcfunix.gfs.{date?fmt=%Y%m%d%H}.
>>> > >> >
>>> > >> > Inside my main script
>>> > >> >
>>> > >> >
>>> > >>
>>> >
>>>
*(/gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/METplus/scripts/run_metplus_TCPlotter.sh*,
>>> > >> > I have three configure files that being processed like so:*
>>> > >> > ${MET_PLUS}/ush/master_metplus.py -c
>>> > >> > ${MET_PLUS_CONF}/Plotter_fcstGFS_obsGFS_ExtraTC.conf -c
>>> > >> > ${MET_PLUS_TMP}/shared.conf_TC -c
${MET_PLUS_CONF}/system_TC.conf*
>>> > >> >
>>> > >> > The system_TC.conf is of little concern because it is
mostly
>>> unchanged
>>> > >> > except for the OUTPUT_BASE, which is changed specifically
to
>>> where I
>>> > >> plan
>>> > >> > on storing the TC output. The shared_TC.conf, on the other
hand,
>>> is
>>> > >> used
>>> > >> > to replace the content inside my
>>> Plotter_fcstGFS_obsGFS_ExtraTC.conf
>>> > >> file.
>>> > >> > In here I define
>>> > >> > the ADECK and BDECK input directories based on the where
the
>>> template
>>> > >> files
>>> > >> > above are being stored. Since I have several runs
encompassing
>>> the
>>> > >> > tropical cyclone event - 20190924 through 20190929 - I have
>>> specified
>>> > my
>>> > >> > INIT_BEG as 20190924 and my INIT_END as 20190929. I have
also
>>> set my
>>> > >> > INIT_INCREMENT to 86400 since I'm only processing the 00z
cycle
>>> for
>>> > >> now. I
>>> > >> > have a couple of lines of code before the configure file
that
>>> joins
>>> > >> these
>>> > >> > dates in a comma separated list such that the
>>> > CYCLONE_PLOTTER_INIT_DATE
>>> > >> is
>>> > >> > "20190924,20190925,...,20190929". I'm not sure if that is
allowed
>>> > >> though.
>>> > >> >
>>> > >> > When running the script the log that is generated produces
0
>>> errors.
>>> > >> > However, I find *two issues*: *1) there is only one file
being
>>> > carried
>>> > >> > over to generate results; that is the 20190924 date - there
>>> should be
>>> > >> 6*,
>>> > >> > and *2) The files that are created are empty and thus the
plot
>>> that is
>>> > >> > generated is simply a map with no data overlaid.*
>>> > >> >
>>> > >> > In attempting to debug further, I found something that
looks
>>> important
>>> > >> > shown here:
>>> > >> >
>>> > >> > DEBUG 3: Rejected for init mask = 0
>>> > >> > DEBUG 3: Rejected for valid mask = 0
>>> > >> > *DEBUG 2: Matching 0 ADECK tracks to 0 BDECK tracks.*
>>> > >> > DEBUG 1: Watch/Warning file:
>>> > >> >
>>> > >> >
>>> > >>
>>> >
>>>
/gpfs/dell2/emc/verification/noscrub/Julie.Prestopnik/met/9.1_beta2/share/met/tc_data/wwpts_us.txt
>>> > >> > DEBUG 5: TrackPairInfoArray: NPairs = 0, NAlloc = 0, Pairs:
>>> > >> > DEBUG 5:
>>> > >> > DEBUG 1: Output file:
>>> > >> >
>>> > >> >
>>> > >>
>>> >
>>>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs/201909/AL_q2019092400.gfs.PRNA20.tcst
>>> > >> > 08/26 18:35:12.422 metplus.TCPairs (command_runner.py:191)
DEBUG:
>>> > >> Finished
>>> > >> > running
>>> > >> >
>>> > >> >
>>> > >>
>>> >
>>>
/gpfs/dell2/emc/verification/noscrub/Julie.Prestopnik/met/9.1_beta2/bin/tc_pairs
>>> > >> > in 0:00:00.607349
>>> > >> > 08/26 18:35:12.422 metplus.CyclonePlotter
>>> > >> (cyclone_plotter_wrapper.py:77)
>>> > >> > DEBUG: Begin retrieving data...
>>> > >> > 08/26 18:35:12.422 metplus.CyclonePlotter
>>> > >> (cyclone_plotter_wrapper.py:83)
>>> > >> > DEBUG: Generate plot for all files in the
>>> > >> >
>>> > >> >
>>> > >>
>>> >
>>>
directory/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs
>>> > >> > 08/26 18:35:12.423 metplus.CyclonePlotter
>>> > >> (cyclone_plotter_wrapper.py:98)
>>> > >> > DEBUG: Parsing file
>>> > >> >
>>> > >> >
>>> > >>
>>> >
>>>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs/201909/AL_q2019092400.gfs.PRNA20.tcst
>>> > >> > 08/26 18:35:19.778 metplus (config_launcher.py:520) DEBUG:
Setting
>>> > >> [config]
>>> > >> > SCRUB_STAGING_DIR to default value: False.
>>> > >> > 08/26 18:35:19.779 metplus INFO:
>>> > >> >
>>> > >> >
>>> > >>
>>> >
>>>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/metplus_final.conf:
>>> > >> > write metplus.conf here
>>> > >> > 08/26 18:35:19.783 metplus (met_util.py:213) DEBUG: METplus
took
>>> > >> > 0:00:08.783240 to run.
>>> > >> > 08/26 18:35:19.783 metplus INFO: METplus has successfully
finished
>>> > >> running.
>>> > >> >
>>> > >> > Specifically the line that says matching 0 ADECK tracks to
0 BDECK
>>> > >> tracks
>>> > >> > lends me to believe that this is the reason why nothing is
being
>>> added
>>> > >> to
>>> > >> > my TC_PAIRS_OUTPUT_TEMPLATE file, and therefore no plots
>>> generated on
>>> > a
>>> > >> > map.
>>> > >> >
>>> > >> > What's interesting is the file that is extracted for BDECK
is for
>>> > >> hurricane
>>> > >> > Lorenzo and includes all the dates coinciding with the
ADECK file
>>> - a
>>> > >> case
>>> > >> > that is run coinciding with Lorenzo. I've checked this.
>>> > >> >
>>> > >> > One possible issue is the formatting of these files. I
looked in
>>> the
>>> > >> test
>>> > >> > case that comes with the cloned met stuff. In there the
>>> header/field
>>> > >> info
>>> > >> > looks something like this:
>>> > >> >
>>> > >> >
>>> > >> >
>>> > >>
>>> >
>>>
*/gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/METplus/source_data/met_test/new/track_data/201412/amlq2014123118.gfso.0104*
>>> > >> >
>>> > >> > ML, 0104, 2014121318_F000_478N_1780W_FOF, 2014121318, 03,
GFSO,
>>> 000,
>>> > >> 478N,
>>> > >> > 1780W, 38, 986, XX, 34, NEQ, 0283, 0640, 0645, 0000,
990,
>>> 133,
>>> > -99,
>>> > >> > 69, 227, -99, -9999, -9999, 208, 315, 135, 156, 0063
>>> > >> >
>>> > >> > This differs from the structure of both the ADECK and BDECK
input
>>> > files:
>>> > >> >
>>> > >> > *BDECK:
>>> > >> >
>>>
/gpfs/hps3/emc/hwrf/noscrub/emc.hurpara/trak/abdeck/btk/bal132019.dat
>>> > *
>>> > >> >
>>> > >> > AL, 13, 2019092218, , BEST, 0, 109N, 188W, 25, 1008,
LO,
>>> 0,
>>> > ,
>>> > >> > 0, 0, 0, 0, 1012, 100, 90, 40, 0, L, 0,
,
>>> 0,
>>> > >> 0,
>>> > >> > INVEST, S, 0, , 0, 0, 0, 0, genesis-num,
027,
>>> > >> >
>>> > >> > *ADECK:
>>> > >> >
>>> > >> >
>>> > >>
>>> >
>>>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400*
>>> > >> >
>>> > >> > AL, 0912, 03, PRNA, 000, 157N, 655W, 27, 1008, XX, 34,
NEQ,
>>> 0000,
>>> > >> 0000,
>>> > >> > 0000, 0000, -9999, -9999, 85
>>> > >> >
>>> > >> > My suspicion is that these formats are not compatible.
There are
>>> > major
>>> > >> > differences between these formats in my opinion. The ADECK
was
>>> > >> generated
>>> > >> > automatically when making a run while BDECK is done by
someone
>>> else.
>>> > >> Those
>>> > >> > files are stored whenever a storm is identified.
>>> > >> >
>>> > >> > I think what I'll need to do next is start by running TCRMW
to
>>> produce
>>> > >> the
>>> > >> > ADECK files formatted by metplus. I have the grib files
for that.
>>> > The
>>> > >> > question next, though, is the BDECK files okay for
processing?
>>> > >> >
>>> > >> >
>>> > >> >
>>> > >> > --
>>> > >> > Edward Strobach
>>> > >> > EMC/NCEP/NWS/
>>> > >> > IMSG Contractor
>>> > >> > Cubicle#: 2029
>>> > >> > 301-683-3717
>>> > >> >
>>> > >> >
>>> > >>
>>> > >> --
>>> > >> George McCabe - Software Engineer III
>>> > >> National Center for Atmospheric Research
>>> > >> Research Applications Laboratory
>>> > >> 303-497-2768
>>> > >> ---
>>> > >> My working day may not be your working day. Please do not
feel
>>> obliged
>>> > to
>>> > >> reply to this email outside of your normal working hours.
>>> > >>
>>> > >>
>>> > >
>>> > > --
>>> > > Edward Strobach
>>> > > EMC/NCEP/NWS/
>>> > > IMSG Contractor
>>> > > Cubicle#: 2029
>>> > > 301-683-3717
>>> > >
>>> >
>>> >
>>> > --
>>> > Edward Strobach
>>> > EMC/NCEP/NWS/
>>> > IMSG Contractor
>>> > Cubicle#: 2029
>>> > 301-683-3717
>>> >
>>> >
>>>
>>> --
>>> George McCabe - Software Engineer III
>>> National Center for Atmospheric Research
>>> Research Applications Laboratory
>>> 303-497-2768
>>> ---
>>> My working day may not be your working day. Please do not feel
obliged to
>>> reply to this email outside of your normal working hours.
>>>
>>>
>>
>> --
>> Edward Strobach
>> EMC/NCEP/NWS/
>> IMSG Contractor
>> Cubicle#: 2029
>> 301-683-3717
>>
>
>
> --
> Edward Strobach
> EMC/NCEP/NWS/
> IMSG Contractor
> Cubicle#: 2029
> 301-683-3717
>
--
Edward Strobach
EMC/NCEP/NWS/
IMSG Contractor
Cubicle#: 2029
301-683-3717
------------------------------------------------
Subject: Tropical Cyclone verification: TCPairs and CyclonePlotter
From: John Halley Gotway
Time: Fri Aug 28 15:49:55 2020
Hi Ed,
You are correct. Generally the STORM_NAME is not present in the ADECK
file.
It only appears in the BDECK file.
Please try configuring it by specifying a STORM_ID. MET constructs the
storm id on the fly from the input ATCF data... as the 2-digit basin,
followed by the 2 digit storm number, followed by the 4 digit year
(e.g.
AL092011 for the 9 Atlantic Hurricane storm of 2011).
So you can specify the storm_id in the config file, but not the storm
name.
Thanks,
John
On Fri, Aug 28, 2020 at 3:45 PM Edward Strobach - NOAA Affiliate via
RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458 >
>
> It appears that even after adding storm_id/storm_name, that met is
not able
> to identify a track in the file provided. I believe it's because
neither
> the storm name or storm ID is written in the ADECK file! I may have
to
> write a script that parses the text file and writes in the storm
name and
> id in two separate columns, respectively. Here's a line example
with what
> it currently looks like for one row of the ADECK file:
>
> AL, 13, 2019092500, 03, PRNA, 000, 133N, 313W, 58, 993, XX, 50,
NEQ,
> 0054, 0000, 0000, 0052, 1013, 172, 32
>
> If I add AL132019 and LORENZO as the last two columns of this row,
then
> should that remedy the problem that I have. Do you think that would
> resolve it?
>
> On Fri, Aug 28, 2020 at 4:53 PM Edward Strobach - NOAA Affiliate <
> edward.strobach at noaa.gov> wrote:
>
> > It seems to me, though, that these options are rather redundant.
If one
> > supplies a file with the Basin name, storm id, and storm name,
then it
> > would seem to me that all one would need to do is supply the storm
id (or
> > name) with everything else being extracted from that using the
contents
> in
> > the file.
> >
> > On Fri, Aug 28, 2020 at 4:41 PM Edward Strobach - NOAA Affiliate <
> > edward.strobach at noaa.gov> wrote:
> >
> >> Thanks for that. I left the TC_PAIRS_CYCLONE, TC_PAIRS_BASIN,
> >> TC_PAIRS_STORM_NAME, TC_PAIRS_STORM_ID alone. I may have to
develop a
> way
> >> to fill this information in without manually specifying it. If
that
> proves
> >> futile, then I'll take a look at using the GFDL vortex tracker.
Thanks
> >>
> >> On Thu, Aug 27, 2020 at 3:37 PM George McCabe via RT
<met_help at ucar.edu
> >
> >> wrote:
> >>
> >>> Hi Edward,
> >>>
> >>> From John HG:
> >>>
> >>>
> >>>
> >>> *The MET-TC tools read track files in ATCF format. Those track
files
> can
> >>> be
> >>> generated from gridded model output by running software. One
such
> tracker
> >>> is supported to the community by the DTC and is named the "GFDL
Vortex
> >>> Tracker":http://dtcenter.org/community-code/gfdl-vortex-tracker
> >>> <http://dtcenter.org/community-code/gfdl-vortex-tracker>*
> >>> *It is not expressly part of METplus but would be very useful
for
> >>> tracking
> >>> storms in GRIB files.*
> >>>
> >>> My guess as to why none of the tracks are being used has to do
with the
> >>> METplus configurations to filter data. I would check the values
for
> >>> TC_PAIRS_CYCLONE, TC_PAIRS_BASIN, TC_PAIRS_STORM_NAME,
> TC_PAIRS_STORM_ID,
> >>> and MODEL. The values you put in these lists determine which
storms to
> >>> use.
> >>> You can leave them blank to use any.
> >>>
> >>> Also, you could bump up the MET log verbosity and see more
information
> >>> about why they are being rejected:
> >>>
> >>> # to affect all MET tools set:
> >>> LOG_MET_VERBOSITY = 10
> >>>
> >>> # to affect only TCPairs set:
> >>> LOG_TC_PAIRS_VERBOSITY = 10
> >>>
> >>> - George
> >>>
> >>> On Thu, Aug 27, 2020 at 10:53 AM Edward Strobach - NOAA
Affiliate via
> RT
> >>> <
> >>> met_help at ucar.edu> wrote:
> >>>
> >>> >
> >>> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458
>
> >>> >
> >>> > Hi,
> >>> >
> >>> > I did decide to remove CyclonePlotter for now and use the
> >>> TCPairs_Config
> >>> > file along with settings that I set in a shared_TC.conf that I
store
> >>> in a
> >>> > temporary directory to process the ADECK and BDECK data that I
have.
> >>> Files
> >>> > are generated for all the dates that span INIT_BEG and
INIT_END,
> >>> thanks to
> >>> > setting loop_by to INIT. However, only the header is written
out,
> and I
> >>> > suspect it is do to what the log indicates here:
> >>> >
> >>> > DEBUG 4: [File 1 of 5] Used 0 of 168 lines read from file
> >>> >
> >>> >
> >>>
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
> >>> > DEBUG 4: [File 2 of 5] Used 0 of 168 lines read from file
> >>> >
> >>> >
> >>>
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
> >>> > DEBUG 4: [File 3 of 5] Used 0 of 168 lines read from file
> >>> >
> >>> >
> >>>
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
> >>> > DEBUG 4: [File 4 of 5] Used 0 of 168 lines read from file
> >>> >
> >>> >
> >>>
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
> >>> > DEBUG 4: [File 5 of 5] Used 0 of 168 lines read from file
> >>> >
> >>> >
> >>>
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
> >>> > DEBUG 3: Used 0 of 840 lines read from 5 file(s).
> >>> > DEBUG 3: Identified 0 track(s).
> >>> > DEBUG 5: TrackInfoArray: NTracks = 0, NAlloc = 0, Tracks:
> >>> > DEBUG 5:
> >>> > DEBUG 2: Deriving 12-hour interpolated ADECK tracks.
> >>> > DEBUG 2: Finished adding 0 and replacing 0 Interp12 track(s).
> >>> > DEBUG 2: Deriving 0 ADECK consensus model(s).
> >>> > DEBUG 2: Added 0 ADECK consensus tracks(s).
> >>> > DEBUG 2: Deriving 0 ADECK lag model(s).
> >>> > DEBUG 2: Added 0 ADECK lag tracks(s).
> >>> > DEBUG 2: Deriving 0 CLIPER/SHIFOR baseline model(s).
> >>> > DEBUG 2: Added 0 CLIPER/SHIFOR baseline track(s).
> >>> > DEBUG 2: Filtering 0 ADECK tracks based on config file
settings.
> >>> > DEBUG 3: Total tracks read = 0
> >>> > DEBUG 3: Total tracks kept = 0
> >>> > DEBUG 3: Rejected for storm name = 0
> >>> > DEBUG 3: Rejected for valid time = 0
> >>> > DEBUG 3: Rejected for required lead times = 0
> >>> > DEBUG 3: Rejected for init mask = 0
> >>> > DEBUG 3: Rejected for valid mask = 0
> >>> > DEBUG 2: Matching 0 ADECK tracks to 1 BDECK tracks.
> >>> >
> >>> > I'm still thinking that the formatting is off in this file. I
> surmise
> >>> that
> >>> > it's not able to group the data because the column information
is not
> >>> in
> >>> > the order that met would want it to be in. These files do not
have
> any
> >>> > header info, so it has to rely on the correct ordering, or so
I
> >>> suspect.
> >>> > My interpretation, however, may be flawed. I see 6 separate
TC based
> >>> > metplus routines: TCPairs (I'm working on currently);
CyclonePlotter
> >>> > (looking to use); TCStat; TCRMW; TCMPRPlotter; and TCGen.
Much, if
> not
> >>> > almost all of these relies on TCPairs. Is there a routine
that I'm
> >>> missing
> >>> > in metplus that takes a grib file and constructs ADECK files?
> >>> >
> >>> >
> >>> >
> >>> > On Thu, Aug 27, 2020 at 11:52 AM Edward Strobach - NOAA
Affiliate <
> >>> > edward.strobach at noaa.gov> wrote:
> >>> >
> >>> > > Hi George, I figured the BDECK must be compatible, so no
worries
> >>> there.
> >>> > > Thanks for the link and the heads up about reformatting. I
will
> use
> >>> the
> >>> > > example to modify what I currently have to see the outcome.
Thanks
> >>> > again.
> >>> > >
> >>> > > On Wed, Aug 26, 2020 at 6:57 PM George McCabe via RT <
> >>> met_help at ucar.edu>
> >>> > > wrote:
> >>> > >
> >>> > >> Hi Edward,
> >>> > >>
> >>> > >> The BDECK data is compatible to be processed. The use case
you
> used
> >>> as
> >>> > an
> >>> > >> example uses extra tropical cyclone data from SBU that is
> >>> reformatted by
> >>> > >> TCPairs wrapper before processing. You will see your config
file
> >>> has the
> >>> > >> following:
> >>> > >>
> >>> > >> ## tc-pairs filtering optionsTC_PAIRS_REFORMAT_DECK =
> >>> > >> yesTC_PAIRS_REFORMAT_TYPE = SBU
> >>> > >>
> >>> > >> Changing TC_PAIRS_REFORMAT_DECK to 'no' or 'false' will
skip the
> >>> > >> reformatting step. There is another use case in the repo
that uses
> >>> data
> >>> > >> that looks like yours (at least as far as I can tell).
> >>> > >>
> >>> > >>
> >>> > >>
> >>> >
> >>>
>
https://dtcenter.github.io/METplus/generated/met_tool_wrapper/TCPairs/TCPairs_tropical.html#metplus-
configuration
> >>> > >>
> >>> > >> This use case uses BDECK data in met_test/new/hwrf/bdeck.
Here is
> >>> part
> >>> > of
> >>> > >> the file called bal062018.dat:
> >>> > >>
> >>> > >> AL, 06, 2018082906, , BEST, 0, 129N, 109W, 10, 1010,
DB,
> >>> 0, ,
> >>> > >> 0, 0, 0, 0, 0, 0, 0, 0, 0, , 0,
,
> 0,
> >>> > 0,
> >>> > >> GENESIS019, , 0, , 0, 0, 0, 0, genesis-
num, 019,
> >>> > >>
> >>> > >>
> >>> > >> Let me know if changing that setting doesn't work and I can
take a
> >>> > closer
> >>> > >> look.
> >>> > >>
> >>> > >> Thanks,
> >>> > >> George
> >>> > >>
> >>> > >> On Wed, Aug 26, 2020 at 1:17 PM Edward Strobach - NOAA
Affiliate
> >>> via RT
> >>> > <
> >>> > >> met_help at ucar.edu> wrote:
> >>> > >>
> >>> > >> >
> >>> > >> > Wed Aug 26 13:13:28 2020: Request 96458 was acted upon.
> >>> > >> > Transaction: Ticket created by edward.strobach at noaa.gov
> >>> > >> > Queue: met_help
> >>> > >> > Subject: Tropical Cyclone verification: TCPairs and
> >>> > CyclonePlotter
> >>> > >> > Owner: Nobody
> >>> > >> > Requestors: edward.strobach at noaa.gov
> >>> > >> > Status: new
> >>> > >> > Ticket <URL:
> >>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458
> >>> > >
> >>> > >> >
> >>> > >> >
> >>> > >> > Good afternoon,
> >>> > >> >
> >>> > >> > I found it relatively straightforward to set-up some
scripts to
> >>> > process
> >>> > >> > forecast output along with obs by using this resource:
> >>> > >> >
> >>> > >> >
> >>> > >>
> >>> >
> >>>
>
https://dtcenter.github.io/METplus/generated/model_applications/tc_and_extra_tc/Plotter_fcstGFS_obsGFS_ExtraTC.html#sphx-
glr-generated-model-applications-tc-and-extra-tc-plotter-fcstgfs-
obsgfs-extratc-py
> >>> > >> > .
> >>> > >> > I have been running the global model which generates
track data
> >>> for
> >>> > the
> >>> > >> > post processing step following this naming convention:
> >>> > >> > atcfunix.gfs.YYYYMMDDHH. The best track data is stored
here on
> >>> WCOSS:
> >>> > >> > /gpfs/hps3/emc/hwrf/noscrub/emc.hurpara/trak/abdeck/btk/.
> >>> > >> >
> >>> > >> > I have a python script that determines which file to use
based
> on
> >>> > basin
> >>> > >> > information, storm name, and date. That is piped into
the
> >>> > >> > TC_PAIRS_BDECK_TEMPLATE while the forecast file above is
piped
> >>> into
> >>> > the
> >>> > >> > TC_PAIRS_ADECK_TEMPLATE this way:
> >>> atcfunix.gfs.{date?fmt=%Y%m%d%H}.
> >>> > >> >
> >>> > >> > Inside my main script
> >>> > >> >
> >>> > >> >
> >>> > >>
> >>> >
> >>>
>
*(/gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/METplus/scripts/run_metplus_TCPlotter.sh*,
> >>> > >> > I have three configure files that being processed like
so:*
> >>> > >> > ${MET_PLUS}/ush/master_metplus.py -c
> >>> > >> > ${MET_PLUS_CONF}/Plotter_fcstGFS_obsGFS_ExtraTC.conf -c
> >>> > >> > ${MET_PLUS_TMP}/shared.conf_TC -c
> ${MET_PLUS_CONF}/system_TC.conf*
> >>> > >> >
> >>> > >> > The system_TC.conf is of little concern because it is
mostly
> >>> unchanged
> >>> > >> > except for the OUTPUT_BASE, which is changed specifically
to
> >>> where I
> >>> > >> plan
> >>> > >> > on storing the TC output. The shared_TC.conf, on the
other
> hand,
> >>> is
> >>> > >> used
> >>> > >> > to replace the content inside my
> >>> Plotter_fcstGFS_obsGFS_ExtraTC.conf
> >>> > >> file.
> >>> > >> > In here I define
> >>> > >> > the ADECK and BDECK input directories based on the where
the
> >>> template
> >>> > >> files
> >>> > >> > above are being stored. Since I have several runs
encompassing
> >>> the
> >>> > >> > tropical cyclone event - 20190924 through 20190929 - I
have
> >>> specified
> >>> > my
> >>> > >> > INIT_BEG as 20190924 and my INIT_END as 20190929. I have
also
> >>> set my
> >>> > >> > INIT_INCREMENT to 86400 since I'm only processing the 00z
cycle
> >>> for
> >>> > >> now. I
> >>> > >> > have a couple of lines of code before the configure file
that
> >>> joins
> >>> > >> these
> >>> > >> > dates in a comma separated list such that the
> >>> > CYCLONE_PLOTTER_INIT_DATE
> >>> > >> is
> >>> > >> > "20190924,20190925,...,20190929". I'm not sure if that
is
> allowed
> >>> > >> though.
> >>> > >> >
> >>> > >> > When running the script the log that is generated
produces 0
> >>> errors.
> >>> > >> > However, I find *two issues*: *1) there is only one file
being
> >>> > carried
> >>> > >> > over to generate results; that is the 20190924 date -
there
> >>> should be
> >>> > >> 6*,
> >>> > >> > and *2) The files that are created are empty and thus the
plot
> >>> that is
> >>> > >> > generated is simply a map with no data overlaid.*
> >>> > >> >
> >>> > >> > In attempting to debug further, I found something that
looks
> >>> important
> >>> > >> > shown here:
> >>> > >> >
> >>> > >> > DEBUG 3: Rejected for init mask = 0
> >>> > >> > DEBUG 3: Rejected for valid mask = 0
> >>> > >> > *DEBUG 2: Matching 0 ADECK tracks to 0 BDECK tracks.*
> >>> > >> > DEBUG 1: Watch/Warning file:
> >>> > >> >
> >>> > >> >
> >>> > >>
> >>> >
> >>>
>
/gpfs/dell2/emc/verification/noscrub/Julie.Prestopnik/met/9.1_beta2/share/met/tc_data/wwpts_us.txt
> >>> > >> > DEBUG 5: TrackPairInfoArray: NPairs = 0, NAlloc = 0,
Pairs:
> >>> > >> > DEBUG 5:
> >>> > >> > DEBUG 1: Output file:
> >>> > >> >
> >>> > >> >
> >>> > >>
> >>> >
> >>>
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs/201909/AL_q2019092400.gfs.PRNA20.tcst
> >>> > >> > 08/26 18:35:12.422 metplus.TCPairs
(command_runner.py:191)
> DEBUG:
> >>> > >> Finished
> >>> > >> > running
> >>> > >> >
> >>> > >> >
> >>> > >>
> >>> >
> >>>
>
/gpfs/dell2/emc/verification/noscrub/Julie.Prestopnik/met/9.1_beta2/bin/tc_pairs
> >>> > >> > in 0:00:00.607349
> >>> > >> > 08/26 18:35:12.422 metplus.CyclonePlotter
> >>> > >> (cyclone_plotter_wrapper.py:77)
> >>> > >> > DEBUG: Begin retrieving data...
> >>> > >> > 08/26 18:35:12.422 metplus.CyclonePlotter
> >>> > >> (cyclone_plotter_wrapper.py:83)
> >>> > >> > DEBUG: Generate plot for all files in the
> >>> > >> >
> >>> > >> >
> >>> > >>
> >>> >
> >>>
>
directory/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs
> >>> > >> > 08/26 18:35:12.423 metplus.CyclonePlotter
> >>> > >> (cyclone_plotter_wrapper.py:98)
> >>> > >> > DEBUG: Parsing file
> >>> > >> >
> >>> > >> >
> >>> > >>
> >>> >
> >>>
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs/201909/AL_q2019092400.gfs.PRNA20.tcst
> >>> > >> > 08/26 18:35:19.778 metplus (config_launcher.py:520)
DEBUG:
> Setting
> >>> > >> [config]
> >>> > >> > SCRUB_STAGING_DIR to default value: False.
> >>> > >> > 08/26 18:35:19.779 metplus INFO:
> >>> > >> >
> >>> > >> >
> >>> > >>
> >>> >
> >>>
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/metplus_final.conf:
> >>> > >> > write metplus.conf here
> >>> > >> > 08/26 18:35:19.783 metplus (met_util.py:213) DEBUG:
METplus took
> >>> > >> > 0:00:08.783240 to run.
> >>> > >> > 08/26 18:35:19.783 metplus INFO: METplus has successfully
> finished
> >>> > >> running.
> >>> > >> >
> >>> > >> > Specifically the line that says matching 0 ADECK tracks
to 0
> BDECK
> >>> > >> tracks
> >>> > >> > lends me to believe that this is the reason why nothing
is being
> >>> added
> >>> > >> to
> >>> > >> > my TC_PAIRS_OUTPUT_TEMPLATE file, and therefore no plots
> >>> generated on
> >>> > a
> >>> > >> > map.
> >>> > >> >
> >>> > >> > What's interesting is the file that is extracted for
BDECK is
> for
> >>> > >> hurricane
> >>> > >> > Lorenzo and includes all the dates coinciding with the
ADECK
> file
> >>> - a
> >>> > >> case
> >>> > >> > that is run coinciding with Lorenzo. I've checked this.
> >>> > >> >
> >>> > >> > One possible issue is the formatting of these files. I
looked
> in
> >>> the
> >>> > >> test
> >>> > >> > case that comes with the cloned met stuff. In there the
> >>> header/field
> >>> > >> info
> >>> > >> > looks something like this:
> >>> > >> >
> >>> > >> >
> >>> > >> >
> >>> > >>
> >>> >
> >>>
>
*/gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/METplus/source_data/met_test/new/track_data/201412/amlq2014123118.gfso.0104*
> >>> > >> >
> >>> > >> > ML, 0104, 2014121318_F000_478N_1780W_FOF, 2014121318, 03,
GFSO,
> >>> 000,
> >>> > >> 478N,
> >>> > >> > 1780W, 38, 986, XX, 34, NEQ, 0283, 0640, 0645, 0000,
990,
> >>> 133,
> >>> > -99,
> >>> > >> > 69, 227, -99, -9999, -9999, 208, 315, 135, 156, 0063
> >>> > >> >
> >>> > >> > This differs from the structure of both the ADECK and
BDECK
> input
> >>> > files:
> >>> > >> >
> >>> > >> > *BDECK:
> >>> > >> >
> >>>
/gpfs/hps3/emc/hwrf/noscrub/emc.hurpara/trak/abdeck/btk/bal132019.dat
> >>> > *
> >>> > >> >
> >>> > >> > AL, 13, 2019092218, , BEST, 0, 109N, 188W, 25,
1008, LO,
> >>> 0,
> >>> > ,
> >>> > >> > 0, 0, 0, 0, 1012, 100, 90, 40, 0, L,
0, ,
> >>> 0,
> >>> > >> 0,
> >>> > >> > INVEST, S, 0, , 0, 0, 0, 0, genesis-
num, 027,
> >>> > >> >
> >>> > >> > *ADECK:
> >>> > >> >
> >>> > >> >
> >>> > >>
> >>> >
> >>>
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400*
> >>> > >> >
> >>> > >> > AL, 0912, 03, PRNA, 000, 157N, 655W, 27, 1008, XX, 34,
NEQ,
> >>> 0000,
> >>> > >> 0000,
> >>> > >> > 0000, 0000, -9999, -9999, 85
> >>> > >> >
> >>> > >> > My suspicion is that these formats are not compatible.
There
> are
> >>> > major
> >>> > >> > differences between these formats in my opinion. The
ADECK was
> >>> > >> generated
> >>> > >> > automatically when making a run while BDECK is done by
someone
> >>> else.
> >>> > >> Those
> >>> > >> > files are stored whenever a storm is identified.
> >>> > >> >
> >>> > >> > I think what I'll need to do next is start by running
TCRMW to
> >>> produce
> >>> > >> the
> >>> > >> > ADECK files formatted by metplus. I have the grib files
for
> that.
> >>> > The
> >>> > >> > question next, though, is the BDECK files okay for
processing?
> >>> > >> >
> >>> > >> >
> >>> > >> >
> >>> > >> > --
> >>> > >> > Edward Strobach
> >>> > >> > EMC/NCEP/NWS/
> >>> > >> > IMSG Contractor
> >>> > >> > Cubicle#: 2029
> >>> > >> > 301-683-3717
> >>> > >> >
> >>> > >> >
> >>> > >>
> >>> > >> --
> >>> > >> George McCabe - Software Engineer III
> >>> > >> National Center for Atmospheric Research
> >>> > >> Research Applications Laboratory
> >>> > >> 303-497-2768
> >>> > >> ---
> >>> > >> My working day may not be your working day. Please do not
feel
> >>> obliged
> >>> > to
> >>> > >> reply to this email outside of your normal working hours.
> >>> > >>
> >>> > >>
> >>> > >
> >>> > > --
> >>> > > Edward Strobach
> >>> > > EMC/NCEP/NWS/
> >>> > > IMSG Contractor
> >>> > > Cubicle#: 2029
> >>> > > 301-683-3717
> >>> > >
> >>> >
> >>> >
> >>> > --
> >>> > Edward Strobach
> >>> > EMC/NCEP/NWS/
> >>> > IMSG Contractor
> >>> > Cubicle#: 2029
> >>> > 301-683-3717
> >>> >
> >>> >
> >>>
> >>> --
> >>> George McCabe - Software Engineer III
> >>> National Center for Atmospheric Research
> >>> Research Applications Laboratory
> >>> 303-497-2768
> >>> ---
> >>> My working day may not be your working day. Please do not feel
obliged
> to
> >>> reply to this email outside of your normal working hours.
> >>>
> >>>
> >>
> >> --
> >> Edward Strobach
> >> EMC/NCEP/NWS/
> >> IMSG Contractor
> >> Cubicle#: 2029
> >> 301-683-3717
> >>
> >
> >
> > --
> > Edward Strobach
> > EMC/NCEP/NWS/
> > IMSG Contractor
> > Cubicle#: 2029
> > 301-683-3717
> >
>
>
> --
> Edward Strobach
> EMC/NCEP/NWS/
> IMSG Contractor
> Cubicle#: 2029
> 301-683-3717
>
>
------------------------------------------------
Subject: Tropical Cyclone verification: TCPairs and CyclonePlotter
From: Edward Strobach - NOAA Affiliate
Time: Mon Aug 31 09:24:19 2020
Hi John,
I did try that. I rewrote the ADECK file with the storm id added at
the
end. I'm pasting an example below:
AL, 13, 2019092400, 03, PRNA, 012, 118N, 282W, 50, 993, XX, 34,
NEQ,
0168, 0115, 0046, 0134, 1013, 161, 111, AL132019
You can see that the last entry, which was not included in the
original
file, has the storm id of AL132019. I'm not very familiar with ADECK
v.
BDECK files. Do you have an example of ADECK file format along with
the
header info? I can try to rewrite the files following the appropriate
order/format, and from there pass the new ADECK file into TCPairs.
On Fri, Aug 28, 2020 at 5:50 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Hi Ed,
>
> You are correct. Generally the STORM_NAME is not present in the
ADECK file.
> It only appears in the BDECK file.
>
> Please try configuring it by specifying a STORM_ID. MET constructs
the
> storm id on the fly from the input ATCF data... as the 2-digit
basin,
> followed by the 2 digit storm number, followed by the 4 digit year
(e.g.
> AL092011 for the 9 Atlantic Hurricane storm of 2011).
>
> So you can specify the storm_id in the config file, but not the
storm name.
>
> Thanks,
> John
>
> On Fri, Aug 28, 2020 at 3:45 PM Edward Strobach - NOAA Affiliate via
RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458 >
> >
> > It appears that even after adding storm_id/storm_name, that met is
not
> able
> > to identify a track in the file provided. I believe it's because
neither
> > the storm name or storm ID is written in the ADECK file! I may
have to
> > write a script that parses the text file and writes in the storm
name and
> > id in two separate columns, respectively. Here's a line example
with
> what
> > it currently looks like for one row of the ADECK file:
> >
> > AL, 13, 2019092500, 03, PRNA, 000, 133N, 313W, 58, 993, XX,
50, NEQ,
> > 0054, 0000, 0000, 0052, 1013, 172, 32
> >
> > If I add AL132019 and LORENZO as the last two columns of this row,
then
> > should that remedy the problem that I have. Do you think that
would
> > resolve it?
> >
> > On Fri, Aug 28, 2020 at 4:53 PM Edward Strobach - NOAA Affiliate <
> > edward.strobach at noaa.gov> wrote:
> >
> > > It seems to me, though, that these options are rather redundant.
If
> one
> > > supplies a file with the Basin name, storm id, and storm name,
then it
> > > would seem to me that all one would need to do is supply the
storm id
> (or
> > > name) with everything else being extracted from that using the
contents
> > in
> > > the file.
> > >
> > > On Fri, Aug 28, 2020 at 4:41 PM Edward Strobach - NOAA Affiliate
<
> > > edward.strobach at noaa.gov> wrote:
> > >
> > >> Thanks for that. I left the TC_PAIRS_CYCLONE, TC_PAIRS_BASIN,
> > >> TC_PAIRS_STORM_NAME, TC_PAIRS_STORM_ID alone. I may have to
develop a
> > way
> > >> to fill this information in without manually specifying it. If
that
> > proves
> > >> futile, then I'll take a look at using the GFDL vortex tracker.
> Thanks
> > >>
> > >> On Thu, Aug 27, 2020 at 3:37 PM George McCabe via RT <
> met_help at ucar.edu
> > >
> > >> wrote:
> > >>
> > >>> Hi Edward,
> > >>>
> > >>> From John HG:
> > >>>
> > >>>
> > >>>
> > >>> *The MET-TC tools read track files in ATCF format. Those track
files
> > can
> > >>> be
> > >>> generated from gridded model output by running software. One
such
> > tracker
> > >>> is supported to the community by the DTC and is named the
"GFDL
> Vortex
> > >>> Tracker":http://dtcenter.org/community-code/gfdl-vortex-
tracker
> > >>> <http://dtcenter.org/community-code/gfdl-vortex-tracker>*
> > >>> *It is not expressly part of METplus but would be very useful
for
> > >>> tracking
> > >>> storms in GRIB files.*
> > >>>
> > >>> My guess as to why none of the tracks are being used has to do
with
> the
> > >>> METplus configurations to filter data. I would check the
values for
> > >>> TC_PAIRS_CYCLONE, TC_PAIRS_BASIN, TC_PAIRS_STORM_NAME,
> > TC_PAIRS_STORM_ID,
> > >>> and MODEL. The values you put in these lists determine which
storms
> to
> > >>> use.
> > >>> You can leave them blank to use any.
> > >>>
> > >>> Also, you could bump up the MET log verbosity and see more
> information
> > >>> about why they are being rejected:
> > >>>
> > >>> # to affect all MET tools set:
> > >>> LOG_MET_VERBOSITY = 10
> > >>>
> > >>> # to affect only TCPairs set:
> > >>> LOG_TC_PAIRS_VERBOSITY = 10
> > >>>
> > >>> - George
> > >>>
> > >>> On Thu, Aug 27, 2020 at 10:53 AM Edward Strobach - NOAA
Affiliate via
> > RT
> > >>> <
> > >>> met_help at ucar.edu> wrote:
> > >>>
> > >>> >
> > >>> > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458 >
> > >>> >
> > >>> > Hi,
> > >>> >
> > >>> > I did decide to remove CyclonePlotter for now and use the
> > >>> TCPairs_Config
> > >>> > file along with settings that I set in a shared_TC.conf that
I
> store
> > >>> in a
> > >>> > temporary directory to process the ADECK and BDECK data that
I
> have.
> > >>> Files
> > >>> > are generated for all the dates that span INIT_BEG and
INIT_END,
> > >>> thanks to
> > >>> > setting loop_by to INIT. However, only the header is written
out,
> > and I
> > >>> > suspect it is do to what the log indicates here:
> > >>> >
> > >>> > DEBUG 4: [File 1 of 5] Used 0 of 168 lines read from file
> > >>> >
> > >>> >
> > >>>
> >
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
> > >>> > DEBUG 4: [File 2 of 5] Used 0 of 168 lines read from file
> > >>> >
> > >>> >
> > >>>
> >
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
> > >>> > DEBUG 4: [File 3 of 5] Used 0 of 168 lines read from file
> > >>> >
> > >>> >
> > >>>
> >
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
> > >>> > DEBUG 4: [File 4 of 5] Used 0 of 168 lines read from file
> > >>> >
> > >>> >
> > >>>
> >
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
> > >>> > DEBUG 4: [File 5 of 5] Used 0 of 168 lines read from file
> > >>> >
> > >>> >
> > >>>
> >
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
> > >>> > DEBUG 3: Used 0 of 840 lines read from 5 file(s).
> > >>> > DEBUG 3: Identified 0 track(s).
> > >>> > DEBUG 5: TrackInfoArray: NTracks = 0, NAlloc = 0, Tracks:
> > >>> > DEBUG 5:
> > >>> > DEBUG 2: Deriving 12-hour interpolated ADECK tracks.
> > >>> > DEBUG 2: Finished adding 0 and replacing 0 Interp12
track(s).
> > >>> > DEBUG 2: Deriving 0 ADECK consensus model(s).
> > >>> > DEBUG 2: Added 0 ADECK consensus tracks(s).
> > >>> > DEBUG 2: Deriving 0 ADECK lag model(s).
> > >>> > DEBUG 2: Added 0 ADECK lag tracks(s).
> > >>> > DEBUG 2: Deriving 0 CLIPER/SHIFOR baseline model(s).
> > >>> > DEBUG 2: Added 0 CLIPER/SHIFOR baseline track(s).
> > >>> > DEBUG 2: Filtering 0 ADECK tracks based on config file
settings.
> > >>> > DEBUG 3: Total tracks read = 0
> > >>> > DEBUG 3: Total tracks kept = 0
> > >>> > DEBUG 3: Rejected for storm name = 0
> > >>> > DEBUG 3: Rejected for valid time = 0
> > >>> > DEBUG 3: Rejected for required lead times = 0
> > >>> > DEBUG 3: Rejected for init mask = 0
> > >>> > DEBUG 3: Rejected for valid mask = 0
> > >>> > DEBUG 2: Matching 0 ADECK tracks to 1 BDECK tracks.
> > >>> >
> > >>> > I'm still thinking that the formatting is off in this file.
I
> > surmise
> > >>> that
> > >>> > it's not able to group the data because the column
information is
> not
> > >>> in
> > >>> > the order that met would want it to be in. These files do
not have
> > any
> > >>> > header info, so it has to rely on the correct ordering, or
so I
> > >>> suspect.
> > >>> > My interpretation, however, may be flawed. I see 6 separate
TC
> based
> > >>> > metplus routines: TCPairs (I'm working on currently);
> CyclonePlotter
> > >>> > (looking to use); TCStat; TCRMW; TCMPRPlotter; and TCGen.
Much, if
> > not
> > >>> > almost all of these relies on TCPairs. Is there a routine
that I'm
> > >>> missing
> > >>> > in metplus that takes a grib file and constructs ADECK
files?
> > >>> >
> > >>> >
> > >>> >
> > >>> > On Thu, Aug 27, 2020 at 11:52 AM Edward Strobach - NOAA
Affiliate <
> > >>> > edward.strobach at noaa.gov> wrote:
> > >>> >
> > >>> > > Hi George, I figured the BDECK must be compatible, so no
worries
> > >>> there.
> > >>> > > Thanks for the link and the heads up about reformatting.
I will
> > use
> > >>> the
> > >>> > > example to modify what I currently have to see the
outcome.
> Thanks
> > >>> > again.
> > >>> > >
> > >>> > > On Wed, Aug 26, 2020 at 6:57 PM George McCabe via RT <
> > >>> met_help at ucar.edu>
> > >>> > > wrote:
> > >>> > >
> > >>> > >> Hi Edward,
> > >>> > >>
> > >>> > >> The BDECK data is compatible to be processed. The use
case you
> > used
> > >>> as
> > >>> > an
> > >>> > >> example uses extra tropical cyclone data from SBU that is
> > >>> reformatted by
> > >>> > >> TCPairs wrapper before processing. You will see your
config file
> > >>> has the
> > >>> > >> following:
> > >>> > >>
> > >>> > >> ## tc-pairs filtering optionsTC_PAIRS_REFORMAT_DECK =
> > >>> > >> yesTC_PAIRS_REFORMAT_TYPE = SBU
> > >>> > >>
> > >>> > >> Changing TC_PAIRS_REFORMAT_DECK to 'no' or 'false' will
skip the
> > >>> > >> reformatting step. There is another use case in the repo
that
> uses
> > >>> data
> > >>> > >> that looks like yours (at least as far as I can tell).
> > >>> > >>
> > >>> > >>
> > >>> > >>
> > >>> >
> > >>>
> >
>
https://dtcenter.github.io/METplus/generated/met_tool_wrapper/TCPairs/TCPairs_tropical.html#metplus-
configuration
> > >>> > >>
> > >>> > >> This use case uses BDECK data in met_test/new/hwrf/bdeck.
Here
> is
> > >>> part
> > >>> > of
> > >>> > >> the file called bal062018.dat:
> > >>> > >>
> > >>> > >> AL, 06, 2018082906, , BEST, 0, 129N, 109W, 10,
1010, DB,
> > >>> 0, ,
> > >>> > >> 0, 0, 0, 0, 0, 0, 0, 0, 0, ,
0, ,
> > 0,
> > >>> > 0,
> > >>> > >> GENESIS019, , 0, , 0, 0, 0, 0, genesis-
num,
> 019,
> > >>> > >>
> > >>> > >>
> > >>> > >> Let me know if changing that setting doesn't work and I
can
> take a
> > >>> > closer
> > >>> > >> look.
> > >>> > >>
> > >>> > >> Thanks,
> > >>> > >> George
> > >>> > >>
> > >>> > >> On Wed, Aug 26, 2020 at 1:17 PM Edward Strobach - NOAA
Affiliate
> > >>> via RT
> > >>> > <
> > >>> > >> met_help at ucar.edu> wrote:
> > >>> > >>
> > >>> > >> >
> > >>> > >> > Wed Aug 26 13:13:28 2020: Request 96458 was acted upon.
> > >>> > >> > Transaction: Ticket created by edward.strobach at noaa.gov
> > >>> > >> > Queue: met_help
> > >>> > >> > Subject: Tropical Cyclone verification: TCPairs
and
> > >>> > CyclonePlotter
> > >>> > >> > Owner: Nobody
> > >>> > >> > Requestors: edward.strobach at noaa.gov
> > >>> > >> > Status: new
> > >>> > >> > Ticket <URL:
> > >>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458
> > >>> > >
> > >>> > >> >
> > >>> > >> >
> > >>> > >> > Good afternoon,
> > >>> > >> >
> > >>> > >> > I found it relatively straightforward to set-up some
scripts
> to
> > >>> > process
> > >>> > >> > forecast output along with obs by using this resource:
> > >>> > >> >
> > >>> > >> >
> > >>> > >>
> > >>> >
> > >>>
> >
>
https://dtcenter.github.io/METplus/generated/model_applications/tc_and_extra_tc/Plotter_fcstGFS_obsGFS_ExtraTC.html#sphx-
glr-generated-model-applications-tc-and-extra-tc-plotter-fcstgfs-
obsgfs-extratc-py
> > >>> > >> > .
> > >>> > >> > I have been running the global model which generates
track
> data
> > >>> for
> > >>> > the
> > >>> > >> > post processing step following this naming convention:
> > >>> > >> > atcfunix.gfs.YYYYMMDDHH. The best track data is stored
here
> on
> > >>> WCOSS:
> > >>> > >> >
/gpfs/hps3/emc/hwrf/noscrub/emc.hurpara/trak/abdeck/btk/.
> > >>> > >> >
> > >>> > >> > I have a python script that determines which file to
use based
> > on
> > >>> > basin
> > >>> > >> > information, storm name, and date. That is piped into
the
> > >>> > >> > TC_PAIRS_BDECK_TEMPLATE while the forecast file above
is piped
> > >>> into
> > >>> > the
> > >>> > >> > TC_PAIRS_ADECK_TEMPLATE this way:
> > >>> atcfunix.gfs.{date?fmt=%Y%m%d%H}.
> > >>> > >> >
> > >>> > >> > Inside my main script
> > >>> > >> >
> > >>> > >> >
> > >>> > >>
> > >>> >
> > >>>
> >
>
*(/gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/METplus/scripts/run_metplus_TCPlotter.sh*,
> > >>> > >> > I have three configure files that being processed like
so:*
> > >>> > >> > ${MET_PLUS}/ush/master_metplus.py -c
> > >>> > >> > ${MET_PLUS_CONF}/Plotter_fcstGFS_obsGFS_ExtraTC.conf -c
> > >>> > >> > ${MET_PLUS_TMP}/shared.conf_TC -c
> > ${MET_PLUS_CONF}/system_TC.conf*
> > >>> > >> >
> > >>> > >> > The system_TC.conf is of little concern because it is
mostly
> > >>> unchanged
> > >>> > >> > except for the OUTPUT_BASE, which is changed
specifically to
> > >>> where I
> > >>> > >> plan
> > >>> > >> > on storing the TC output. The shared_TC.conf, on the
other
> > hand,
> > >>> is
> > >>> > >> used
> > >>> > >> > to replace the content inside my
> > >>> Plotter_fcstGFS_obsGFS_ExtraTC.conf
> > >>> > >> file.
> > >>> > >> > In here I define
> > >>> > >> > the ADECK and BDECK input directories based on the
where the
> > >>> template
> > >>> > >> files
> > >>> > >> > above are being stored. Since I have several runs
> encompassing
> > >>> the
> > >>> > >> > tropical cyclone event - 20190924 through 20190929 - I
have
> > >>> specified
> > >>> > my
> > >>> > >> > INIT_BEG as 20190924 and my INIT_END as 20190929. I
have also
> > >>> set my
> > >>> > >> > INIT_INCREMENT to 86400 since I'm only processing the
00z
> cycle
> > >>> for
> > >>> > >> now. I
> > >>> > >> > have a couple of lines of code before the configure
file that
> > >>> joins
> > >>> > >> these
> > >>> > >> > dates in a comma separated list such that the
> > >>> > CYCLONE_PLOTTER_INIT_DATE
> > >>> > >> is
> > >>> > >> > "20190924,20190925,...,20190929". I'm not sure if that
is
> > allowed
> > >>> > >> though.
> > >>> > >> >
> > >>> > >> > When running the script the log that is generated
produces 0
> > >>> errors.
> > >>> > >> > However, I find *two issues*: *1) there is only one
file
> being
> > >>> > carried
> > >>> > >> > over to generate results; that is the 20190924 date -
there
> > >>> should be
> > >>> > >> 6*,
> > >>> > >> > and *2) The files that are created are empty and thus
the plot
> > >>> that is
> > >>> > >> > generated is simply a map with no data overlaid.*
> > >>> > >> >
> > >>> > >> > In attempting to debug further, I found something that
looks
> > >>> important
> > >>> > >> > shown here:
> > >>> > >> >
> > >>> > >> > DEBUG 3: Rejected for init mask = 0
> > >>> > >> > DEBUG 3: Rejected for valid mask = 0
> > >>> > >> > *DEBUG 2: Matching 0 ADECK tracks to 0 BDECK tracks.*
> > >>> > >> > DEBUG 1: Watch/Warning file:
> > >>> > >> >
> > >>> > >> >
> > >>> > >>
> > >>> >
> > >>>
> >
>
/gpfs/dell2/emc/verification/noscrub/Julie.Prestopnik/met/9.1_beta2/share/met/tc_data/wwpts_us.txt
> > >>> > >> > DEBUG 5: TrackPairInfoArray: NPairs = 0, NAlloc = 0,
Pairs:
> > >>> > >> > DEBUG 5:
> > >>> > >> > DEBUG 1: Output file:
> > >>> > >> >
> > >>> > >> >
> > >>> > >>
> > >>> >
> > >>>
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs/201909/AL_q2019092400.gfs.PRNA20.tcst
> > >>> > >> > 08/26 18:35:12.422 metplus.TCPairs
(command_runner.py:191)
> > DEBUG:
> > >>> > >> Finished
> > >>> > >> > running
> > >>> > >> >
> > >>> > >> >
> > >>> > >>
> > >>> >
> > >>>
> >
>
/gpfs/dell2/emc/verification/noscrub/Julie.Prestopnik/met/9.1_beta2/bin/tc_pairs
> > >>> > >> > in 0:00:00.607349
> > >>> > >> > 08/26 18:35:12.422 metplus.CyclonePlotter
> > >>> > >> (cyclone_plotter_wrapper.py:77)
> > >>> > >> > DEBUG: Begin retrieving data...
> > >>> > >> > 08/26 18:35:12.422 metplus.CyclonePlotter
> > >>> > >> (cyclone_plotter_wrapper.py:83)
> > >>> > >> > DEBUG: Generate plot for all files in the
> > >>> > >> >
> > >>> > >> >
> > >>> > >>
> > >>> >
> > >>>
> >
>
directory/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs
> > >>> > >> > 08/26 18:35:12.423 metplus.CyclonePlotter
> > >>> > >> (cyclone_plotter_wrapper.py:98)
> > >>> > >> > DEBUG: Parsing file
> > >>> > >> >
> > >>> > >> >
> > >>> > >>
> > >>> >
> > >>>
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs/201909/AL_q2019092400.gfs.PRNA20.tcst
> > >>> > >> > 08/26 18:35:19.778 metplus (config_launcher.py:520)
DEBUG:
> > Setting
> > >>> > >> [config]
> > >>> > >> > SCRUB_STAGING_DIR to default value: False.
> > >>> > >> > 08/26 18:35:19.779 metplus INFO:
> > >>> > >> >
> > >>> > >> >
> > >>> > >>
> > >>> >
> > >>>
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/metplus_final.conf:
> > >>> > >> > write metplus.conf here
> > >>> > >> > 08/26 18:35:19.783 metplus (met_util.py:213) DEBUG:
METplus
> took
> > >>> > >> > 0:00:08.783240 to run.
> > >>> > >> > 08/26 18:35:19.783 metplus INFO: METplus has
successfully
> > finished
> > >>> > >> running.
> > >>> > >> >
> > >>> > >> > Specifically the line that says matching 0 ADECK tracks
to 0
> > BDECK
> > >>> > >> tracks
> > >>> > >> > lends me to believe that this is the reason why nothing
is
> being
> > >>> added
> > >>> > >> to
> > >>> > >> > my TC_PAIRS_OUTPUT_TEMPLATE file, and therefore no
plots
> > >>> generated on
> > >>> > a
> > >>> > >> > map.
> > >>> > >> >
> > >>> > >> > What's interesting is the file that is extracted for
BDECK is
> > for
> > >>> > >> hurricane
> > >>> > >> > Lorenzo and includes all the dates coinciding with the
ADECK
> > file
> > >>> - a
> > >>> > >> case
> > >>> > >> > that is run coinciding with Lorenzo. I've checked
this.
> > >>> > >> >
> > >>> > >> > One possible issue is the formatting of these files. I
looked
> > in
> > >>> the
> > >>> > >> test
> > >>> > >> > case that comes with the cloned met stuff. In there
the
> > >>> header/field
> > >>> > >> info
> > >>> > >> > looks something like this:
> > >>> > >> >
> > >>> > >> >
> > >>> > >> >
> > >>> > >>
> > >>> >
> > >>>
> >
>
*/gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/METplus/source_data/met_test/new/track_data/201412/amlq2014123118.gfso.0104*
> > >>> > >> >
> > >>> > >> > ML, 0104, 2014121318_F000_478N_1780W_FOF, 2014121318,
03,
> GFSO,
> > >>> 000,
> > >>> > >> 478N,
> > >>> > >> > 1780W, 38, 986, XX, 34, NEQ, 0283, 0640, 0645, 0000,
990,
> > >>> 133,
> > >>> > -99,
> > >>> > >> > 69, 227, -99, -9999, -9999, 208, 315, 135, 156,
0063
> > >>> > >> >
> > >>> > >> > This differs from the structure of both the ADECK and
BDECK
> > input
> > >>> > files:
> > >>> > >> >
> > >>> > >> > *BDECK:
> > >>> > >> >
> > >>>
/gpfs/hps3/emc/hwrf/noscrub/emc.hurpara/trak/abdeck/btk/bal132019.dat
> > >>> > *
> > >>> > >> >
> > >>> > >> > AL, 13, 2019092218, , BEST, 0, 109N, 188W, 25,
1008, LO,
> > >>> 0,
> > >>> > ,
> > >>> > >> > 0, 0, 0, 0, 1012, 100, 90, 40, 0, L,
0,
> ,
> > >>> 0,
> > >>> > >> 0,
> > >>> > >> > INVEST, S, 0, , 0, 0, 0, 0, genesis-
num,
> 027,
> > >>> > >> >
> > >>> > >> > *ADECK:
> > >>> > >> >
> > >>> > >> >
> > >>> > >>
> > >>> >
> > >>>
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400*
> > >>> > >> >
> > >>> > >> > AL, 0912, 03, PRNA, 000, 157N, 655W, 27, 1008, XX,
34, NEQ,
> > >>> 0000,
> > >>> > >> 0000,
> > >>> > >> > 0000, 0000, -9999, -9999, 85
> > >>> > >> >
> > >>> > >> > My suspicion is that these formats are not compatible.
There
> > are
> > >>> > major
> > >>> > >> > differences between these formats in my opinion. The
ADECK
> was
> > >>> > >> generated
> > >>> > >> > automatically when making a run while BDECK is done by
someone
> > >>> else.
> > >>> > >> Those
> > >>> > >> > files are stored whenever a storm is identified.
> > >>> > >> >
> > >>> > >> > I think what I'll need to do next is start by running
TCRMW to
> > >>> produce
> > >>> > >> the
> > >>> > >> > ADECK files formatted by metplus. I have the grib
files for
> > that.
> > >>> > The
> > >>> > >> > question next, though, is the BDECK files okay for
processing?
> > >>> > >> >
> > >>> > >> >
> > >>> > >> >
> > >>> > >> > --
> > >>> > >> > Edward Strobach
> > >>> > >> > EMC/NCEP/NWS/
> > >>> > >> > IMSG Contractor
> > >>> > >> > Cubicle#: 2029
> > >>> > >> > 301-683-3717
> > >>> > >> >
> > >>> > >> >
> > >>> > >>
> > >>> > >> --
> > >>> > >> George McCabe - Software Engineer III
> > >>> > >> National Center for Atmospheric Research
> > >>> > >> Research Applications Laboratory
> > >>> > >> 303-497-2768
> > >>> > >> ---
> > >>> > >> My working day may not be your working day. Please do not
feel
> > >>> obliged
> > >>> > to
> > >>> > >> reply to this email outside of your normal working hours.
> > >>> > >>
> > >>> > >>
> > >>> > >
> > >>> > > --
> > >>> > > Edward Strobach
> > >>> > > EMC/NCEP/NWS/
> > >>> > > IMSG Contractor
> > >>> > > Cubicle#: 2029
> > >>> > > 301-683-3717
> > >>> > >
> > >>> >
> > >>> >
> > >>> > --
> > >>> > Edward Strobach
> > >>> > EMC/NCEP/NWS/
> > >>> > IMSG Contractor
> > >>> > Cubicle#: 2029
> > >>> > 301-683-3717
> > >>> >
> > >>> >
> > >>>
> > >>> --
> > >>> George McCabe - Software Engineer III
> > >>> National Center for Atmospheric Research
> > >>> Research Applications Laboratory
> > >>> 303-497-2768
> > >>> ---
> > >>> My working day may not be your working day. Please do not feel
> obliged
> > to
> > >>> reply to this email outside of your normal working hours.
> > >>>
> > >>>
> > >>
> > >> --
> > >> Edward Strobach
> > >> EMC/NCEP/NWS/
> > >> IMSG Contractor
> > >> Cubicle#: 2029
> > >> 301-683-3717
> > >>
> > >
> > >
> > > --
> > > Edward Strobach
> > > EMC/NCEP/NWS/
> > > IMSG Contractor
> > > Cubicle#: 2029
> > > 301-683-3717
> > >
> >
> >
> > --
> > Edward Strobach
> > EMC/NCEP/NWS/
> > IMSG Contractor
> > Cubicle#: 2029
> > 301-683-3717
> >
> >
>
>
--
Edward Strobach
EMC/NCEP/NWS/
IMSG Contractor
Cubicle#: 2029
301-683-3717
------------------------------------------------
Subject: Tropical Cyclone verification: TCPairs and CyclonePlotter
From: John Halley Gotway
Time: Mon Aug 31 13:30:55 2020
Ed,
Here's a link to the documentation that the Navy provides about the
ATCF
file format:
https://www.nrlmry.navy.mil/atcf_web/docs/database/new/abrdeck.html
And the best source of sample data is NHC's ftp site:
https://ftp.nhc.noaa.gov/atcf/
In there...
*aid_public* contains tar files of ADECKS for all storms in the
current
season.
*btk* has uncompressed BEST track files for all storms in the current
season.
*archive* contains compressed A and B-Deck data for previous seasons.
It's pretty cool actually that this track data is so readily
available.
I have never actually modified ATCF files for my purposes. So
hopefully you
won't need to either... and can just use the output of the tracker
code
directly.
Thanks,
John
On Mon, Aug 31, 2020 at 9:24 AM Edward Strobach - NOAA Affiliate via
RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458 >
>
> Hi John,
>
> I did try that. I rewrote the ADECK file with the storm id added at
the
> end. I'm pasting an example below:
> AL, 13, 2019092400, 03, PRNA, 012, 118N, 282W, 50, 993, XX, 34,
NEQ,
> 0168, 0115, 0046, 0134, 1013, 161, 111, AL132019
>
> You can see that the last entry, which was not included in the
original
> file, has the storm id of AL132019. I'm not very familiar with
ADECK v.
> BDECK files. Do you have an example of ADECK file format along with
the
> header info? I can try to rewrite the files following the
appropriate
> order/format, and from there pass the new ADECK file into TCPairs.
>
> On Fri, Aug 28, 2020 at 5:50 PM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > Hi Ed,
> >
> > You are correct. Generally the STORM_NAME is not present in the
ADECK
> file.
> > It only appears in the BDECK file.
> >
> > Please try configuring it by specifying a STORM_ID. MET constructs
the
> > storm id on the fly from the input ATCF data... as the 2-digit
basin,
> > followed by the 2 digit storm number, followed by the 4 digit year
(e.g.
> > AL092011 for the 9 Atlantic Hurricane storm of 2011).
> >
> > So you can specify the storm_id in the config file, but not the
storm
> name.
> >
> > Thanks,
> > John
> >
> > On Fri, Aug 28, 2020 at 3:45 PM Edward Strobach - NOAA Affiliate
via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458 >
> > >
> > > It appears that even after adding storm_id/storm_name, that met
is not
> > able
> > > to identify a track in the file provided. I believe it's
because
> neither
> > > the storm name or storm ID is written in the ADECK file! I may
have to
> > > write a script that parses the text file and writes in the storm
name
> and
> > > id in two separate columns, respectively. Here's a line example
with
> > what
> > > it currently looks like for one row of the ADECK file:
> > >
> > > AL, 13, 2019092500, 03, PRNA, 000, 133N, 313W, 58, 993, XX,
50,
> NEQ,
> > > 0054, 0000, 0000, 0052, 1013, 172, 32
> > >
> > > If I add AL132019 and LORENZO as the last two columns of this
row, then
> > > should that remedy the problem that I have. Do you think that
would
> > > resolve it?
> > >
> > > On Fri, Aug 28, 2020 at 4:53 PM Edward Strobach - NOAA Affiliate
<
> > > edward.strobach at noaa.gov> wrote:
> > >
> > > > It seems to me, though, that these options are rather
redundant. If
> > one
> > > > supplies a file with the Basin name, storm id, and storm name,
then
> it
> > > > would seem to me that all one would need to do is supply the
storm id
> > (or
> > > > name) with everything else being extracted from that using the
> contents
> > > in
> > > > the file.
> > > >
> > > > On Fri, Aug 28, 2020 at 4:41 PM Edward Strobach - NOAA
Affiliate <
> > > > edward.strobach at noaa.gov> wrote:
> > > >
> > > >> Thanks for that. I left the TC_PAIRS_CYCLONE,
TC_PAIRS_BASIN,
> > > >> TC_PAIRS_STORM_NAME, TC_PAIRS_STORM_ID alone. I may have to
> develop a
> > > way
> > > >> to fill this information in without manually specifying it.
If that
> > > proves
> > > >> futile, then I'll take a look at using the GFDL vortex
tracker.
> > Thanks
> > > >>
> > > >> On Thu, Aug 27, 2020 at 3:37 PM George McCabe via RT <
> > met_help at ucar.edu
> > > >
> > > >> wrote:
> > > >>
> > > >>> Hi Edward,
> > > >>>
> > > >>> From John HG:
> > > >>>
> > > >>>
> > > >>>
> > > >>> *The MET-TC tools read track files in ATCF format. Those
track
> files
> > > can
> > > >>> be
> > > >>> generated from gridded model output by running software. One
such
> > > tracker
> > > >>> is supported to the community by the DTC and is named the
"GFDL
> > Vortex
> > > >>> Tracker":http://dtcenter.org/community-code/gfdl-vortex-
tracker
> > > >>> <http://dtcenter.org/community-code/gfdl-vortex-tracker>*
> > > >>> *It is not expressly part of METplus but would be very
useful for
> > > >>> tracking
> > > >>> storms in GRIB files.*
> > > >>>
> > > >>> My guess as to why none of the tracks are being used has to
do with
> > the
> > > >>> METplus configurations to filter data. I would check the
values for
> > > >>> TC_PAIRS_CYCLONE, TC_PAIRS_BASIN, TC_PAIRS_STORM_NAME,
> > > TC_PAIRS_STORM_ID,
> > > >>> and MODEL. The values you put in these lists determine which
storms
> > to
> > > >>> use.
> > > >>> You can leave them blank to use any.
> > > >>>
> > > >>> Also, you could bump up the MET log verbosity and see more
> > information
> > > >>> about why they are being rejected:
> > > >>>
> > > >>> # to affect all MET tools set:
> > > >>> LOG_MET_VERBOSITY = 10
> > > >>>
> > > >>> # to affect only TCPairs set:
> > > >>> LOG_TC_PAIRS_VERBOSITY = 10
> > > >>>
> > > >>> - George
> > > >>>
> > > >>> On Thu, Aug 27, 2020 at 10:53 AM Edward Strobach - NOAA
Affiliate
> via
> > > RT
> > > >>> <
> > > >>> met_help at ucar.edu> wrote:
> > > >>>
> > > >>> >
> > > >>> > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458 >
> > > >>> >
> > > >>> > Hi,
> > > >>> >
> > > >>> > I did decide to remove CyclonePlotter for now and use the
> > > >>> TCPairs_Config
> > > >>> > file along with settings that I set in a shared_TC.conf
that I
> > store
> > > >>> in a
> > > >>> > temporary directory to process the ADECK and BDECK data
that I
> > have.
> > > >>> Files
> > > >>> > are generated for all the dates that span INIT_BEG and
INIT_END,
> > > >>> thanks to
> > > >>> > setting loop_by to INIT. However, only the header is
written out,
> > > and I
> > > >>> > suspect it is do to what the log indicates here:
> > > >>> >
> > > >>> > DEBUG 4: [File 1 of 5] Used 0 of 168 lines read from file
> > > >>> >
> > > >>> >
> > > >>>
> > >
> >
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
> > > >>> > DEBUG 4: [File 2 of 5] Used 0 of 168 lines read from file
> > > >>> >
> > > >>> >
> > > >>>
> > >
> >
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
> > > >>> > DEBUG 4: [File 3 of 5] Used 0 of 168 lines read from file
> > > >>> >
> > > >>> >
> > > >>>
> > >
> >
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
> > > >>> > DEBUG 4: [File 4 of 5] Used 0 of 168 lines read from file
> > > >>> >
> > > >>> >
> > > >>>
> > >
> >
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
> > > >>> > DEBUG 4: [File 5 of 5] Used 0 of 168 lines read from file
> > > >>> >
> > > >>> >
> > > >>>
> > >
> >
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
> > > >>> > DEBUG 3: Used 0 of 840 lines read from 5 file(s).
> > > >>> > DEBUG 3: Identified 0 track(s).
> > > >>> > DEBUG 5: TrackInfoArray: NTracks = 0, NAlloc = 0, Tracks:
> > > >>> > DEBUG 5:
> > > >>> > DEBUG 2: Deriving 12-hour interpolated ADECK tracks.
> > > >>> > DEBUG 2: Finished adding 0 and replacing 0 Interp12
track(s).
> > > >>> > DEBUG 2: Deriving 0 ADECK consensus model(s).
> > > >>> > DEBUG 2: Added 0 ADECK consensus tracks(s).
> > > >>> > DEBUG 2: Deriving 0 ADECK lag model(s).
> > > >>> > DEBUG 2: Added 0 ADECK lag tracks(s).
> > > >>> > DEBUG 2: Deriving 0 CLIPER/SHIFOR baseline model(s).
> > > >>> > DEBUG 2: Added 0 CLIPER/SHIFOR baseline track(s).
> > > >>> > DEBUG 2: Filtering 0 ADECK tracks based on config file
settings.
> > > >>> > DEBUG 3: Total tracks read = 0
> > > >>> > DEBUG 3: Total tracks kept = 0
> > > >>> > DEBUG 3: Rejected for storm name = 0
> > > >>> > DEBUG 3: Rejected for valid time = 0
> > > >>> > DEBUG 3: Rejected for required lead times = 0
> > > >>> > DEBUG 3: Rejected for init mask = 0
> > > >>> > DEBUG 3: Rejected for valid mask = 0
> > > >>> > DEBUG 2: Matching 0 ADECK tracks to 1 BDECK tracks.
> > > >>> >
> > > >>> > I'm still thinking that the formatting is off in this
file. I
> > > surmise
> > > >>> that
> > > >>> > it's not able to group the data because the column
information is
> > not
> > > >>> in
> > > >>> > the order that met would want it to be in. These files do
not
> have
> > > any
> > > >>> > header info, so it has to rely on the correct ordering, or
so I
> > > >>> suspect.
> > > >>> > My interpretation, however, may be flawed. I see 6
separate TC
> > based
> > > >>> > metplus routines: TCPairs (I'm working on currently);
> > CyclonePlotter
> > > >>> > (looking to use); TCStat; TCRMW; TCMPRPlotter; and TCGen.
Much,
> if
> > > not
> > > >>> > almost all of these relies on TCPairs. Is there a routine
that
> I'm
> > > >>> missing
> > > >>> > in metplus that takes a grib file and constructs ADECK
files?
> > > >>> >
> > > >>> >
> > > >>> >
> > > >>> > On Thu, Aug 27, 2020 at 11:52 AM Edward Strobach - NOAA
> Affiliate <
> > > >>> > edward.strobach at noaa.gov> wrote:
> > > >>> >
> > > >>> > > Hi George, I figured the BDECK must be compatible, so no
> worries
> > > >>> there.
> > > >>> > > Thanks for the link and the heads up about reformatting.
I
> will
> > > use
> > > >>> the
> > > >>> > > example to modify what I currently have to see the
outcome.
> > Thanks
> > > >>> > again.
> > > >>> > >
> > > >>> > > On Wed, Aug 26, 2020 at 6:57 PM George McCabe via RT <
> > > >>> met_help at ucar.edu>
> > > >>> > > wrote:
> > > >>> > >
> > > >>> > >> Hi Edward,
> > > >>> > >>
> > > >>> > >> The BDECK data is compatible to be processed. The use
case you
> > > used
> > > >>> as
> > > >>> > an
> > > >>> > >> example uses extra tropical cyclone data from SBU that
is
> > > >>> reformatted by
> > > >>> > >> TCPairs wrapper before processing. You will see your
config
> file
> > > >>> has the
> > > >>> > >> following:
> > > >>> > >>
> > > >>> > >> ## tc-pairs filtering optionsTC_PAIRS_REFORMAT_DECK =
> > > >>> > >> yesTC_PAIRS_REFORMAT_TYPE = SBU
> > > >>> > >>
> > > >>> > >> Changing TC_PAIRS_REFORMAT_DECK to 'no' or 'false' will
skip
> the
> > > >>> > >> reformatting step. There is another use case in the
repo that
> > uses
> > > >>> data
> > > >>> > >> that looks like yours (at least as far as I can tell).
> > > >>> > >>
> > > >>> > >>
> > > >>> > >>
> > > >>> >
> > > >>>
> > >
> >
>
https://dtcenter.github.io/METplus/generated/met_tool_wrapper/TCPairs/TCPairs_tropical.html#metplus-
configuration
> > > >>> > >>
> > > >>> > >> This use case uses BDECK data in
met_test/new/hwrf/bdeck. Here
> > is
> > > >>> part
> > > >>> > of
> > > >>> > >> the file called bal062018.dat:
> > > >>> > >>
> > > >>> > >> AL, 06, 2018082906, , BEST, 0, 129N, 109W, 10,
1010, DB,
> > > >>> 0, ,
> > > >>> > >> 0, 0, 0, 0, 0, 0, 0, 0, 0, ,
0,
> ,
> > > 0,
> > > >>> > 0,
> > > >>> > >> GENESIS019, , 0, , 0, 0, 0, 0,
genesis-num,
> > 019,
> > > >>> > >>
> > > >>> > >>
> > > >>> > >> Let me know if changing that setting doesn't work and I
can
> > take a
> > > >>> > closer
> > > >>> > >> look.
> > > >>> > >>
> > > >>> > >> Thanks,
> > > >>> > >> George
> > > >>> > >>
> > > >>> > >> On Wed, Aug 26, 2020 at 1:17 PM Edward Strobach - NOAA
> Affiliate
> > > >>> via RT
> > > >>> > <
> > > >>> > >> met_help at ucar.edu> wrote:
> > > >>> > >>
> > > >>> > >> >
> > > >>> > >> > Wed Aug 26 13:13:28 2020: Request 96458 was acted
upon.
> > > >>> > >> > Transaction: Ticket created by
edward.strobach at noaa.gov
> > > >>> > >> > Queue: met_help
> > > >>> > >> > Subject: Tropical Cyclone verification: TCPairs
and
> > > >>> > CyclonePlotter
> > > >>> > >> > Owner: Nobody
> > > >>> > >> > Requestors: edward.strobach at noaa.gov
> > > >>> > >> > Status: new
> > > >>> > >> > Ticket <URL:
> > > >>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458
> > > >>> > >
> > > >>> > >> >
> > > >>> > >> >
> > > >>> > >> > Good afternoon,
> > > >>> > >> >
> > > >>> > >> > I found it relatively straightforward to set-up some
scripts
> > to
> > > >>> > process
> > > >>> > >> > forecast output along with obs by using this
resource:
> > > >>> > >> >
> > > >>> > >> >
> > > >>> > >>
> > > >>> >
> > > >>>
> > >
> >
>
https://dtcenter.github.io/METplus/generated/model_applications/tc_and_extra_tc/Plotter_fcstGFS_obsGFS_ExtraTC.html#sphx-
glr-generated-model-applications-tc-and-extra-tc-plotter-fcstgfs-
obsgfs-extratc-py
> > > >>> > >> > .
> > > >>> > >> > I have been running the global model which generates
track
> > data
> > > >>> for
> > > >>> > the
> > > >>> > >> > post processing step following this naming
convention:
> > > >>> > >> > atcfunix.gfs.YYYYMMDDHH. The best track data is
stored here
> > on
> > > >>> WCOSS:
> > > >>> > >> >
/gpfs/hps3/emc/hwrf/noscrub/emc.hurpara/trak/abdeck/btk/.
> > > >>> > >> >
> > > >>> > >> > I have a python script that determines which file to
use
> based
> > > on
> > > >>> > basin
> > > >>> > >> > information, storm name, and date. That is piped
into the
> > > >>> > >> > TC_PAIRS_BDECK_TEMPLATE while the forecast file above
is
> piped
> > > >>> into
> > > >>> > the
> > > >>> > >> > TC_PAIRS_ADECK_TEMPLATE this way:
> > > >>> atcfunix.gfs.{date?fmt=%Y%m%d%H}.
> > > >>> > >> >
> > > >>> > >> > Inside my main script
> > > >>> > >> >
> > > >>> > >> >
> > > >>> > >>
> > > >>> >
> > > >>>
> > >
> >
>
*(/gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/METplus/scripts/run_metplus_TCPlotter.sh*,
> > > >>> > >> > I have three configure files that being processed
like so:*
> > > >>> > >> > ${MET_PLUS}/ush/master_metplus.py -c
> > > >>> > >> > ${MET_PLUS_CONF}/Plotter_fcstGFS_obsGFS_ExtraTC.conf
-c
> > > >>> > >> > ${MET_PLUS_TMP}/shared.conf_TC -c
> > > ${MET_PLUS_CONF}/system_TC.conf*
> > > >>> > >> >
> > > >>> > >> > The system_TC.conf is of little concern because it is
mostly
> > > >>> unchanged
> > > >>> > >> > except for the OUTPUT_BASE, which is changed
specifically to
> > > >>> where I
> > > >>> > >> plan
> > > >>> > >> > on storing the TC output. The shared_TC.conf, on the
other
> > > hand,
> > > >>> is
> > > >>> > >> used
> > > >>> > >> > to replace the content inside my
> > > >>> Plotter_fcstGFS_obsGFS_ExtraTC.conf
> > > >>> > >> file.
> > > >>> > >> > In here I define
> > > >>> > >> > the ADECK and BDECK input directories based on the
where the
> > > >>> template
> > > >>> > >> files
> > > >>> > >> > above are being stored. Since I have several runs
> > encompassing
> > > >>> the
> > > >>> > >> > tropical cyclone event - 20190924 through 20190929 -
I have
> > > >>> specified
> > > >>> > my
> > > >>> > >> > INIT_BEG as 20190924 and my INIT_END as 20190929. I
have
> also
> > > >>> set my
> > > >>> > >> > INIT_INCREMENT to 86400 since I'm only processing the
00z
> > cycle
> > > >>> for
> > > >>> > >> now. I
> > > >>> > >> > have a couple of lines of code before the configure
file
> that
> > > >>> joins
> > > >>> > >> these
> > > >>> > >> > dates in a comma separated list such that the
> > > >>> > CYCLONE_PLOTTER_INIT_DATE
> > > >>> > >> is
> > > >>> > >> > "20190924,20190925,...,20190929". I'm not sure if
that is
> > > allowed
> > > >>> > >> though.
> > > >>> > >> >
> > > >>> > >> > When running the script the log that is generated
produces 0
> > > >>> errors.
> > > >>> > >> > However, I find *two issues*: *1) there is only one
file
> > being
> > > >>> > carried
> > > >>> > >> > over to generate results; that is the 20190924 date -
there
> > > >>> should be
> > > >>> > >> 6*,
> > > >>> > >> > and *2) The files that are created are empty and thus
the
> plot
> > > >>> that is
> > > >>> > >> > generated is simply a map with no data overlaid.*
> > > >>> > >> >
> > > >>> > >> > In attempting to debug further, I found something
that looks
> > > >>> important
> > > >>> > >> > shown here:
> > > >>> > >> >
> > > >>> > >> > DEBUG 3: Rejected for init mask = 0
> > > >>> > >> > DEBUG 3: Rejected for valid mask = 0
> > > >>> > >> > *DEBUG 2: Matching 0 ADECK tracks to 0 BDECK tracks.*
> > > >>> > >> > DEBUG 1: Watch/Warning file:
> > > >>> > >> >
> > > >>> > >> >
> > > >>> > >>
> > > >>> >
> > > >>>
> > >
> >
>
/gpfs/dell2/emc/verification/noscrub/Julie.Prestopnik/met/9.1_beta2/share/met/tc_data/wwpts_us.txt
> > > >>> > >> > DEBUG 5: TrackPairInfoArray: NPairs = 0, NAlloc = 0,
Pairs:
> > > >>> > >> > DEBUG 5:
> > > >>> > >> > DEBUG 1: Output file:
> > > >>> > >> >
> > > >>> > >> >
> > > >>> > >>
> > > >>> >
> > > >>>
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs/201909/AL_q2019092400.gfs.PRNA20.tcst
> > > >>> > >> > 08/26 18:35:12.422 metplus.TCPairs
(command_runner.py:191)
> > > DEBUG:
> > > >>> > >> Finished
> > > >>> > >> > running
> > > >>> > >> >
> > > >>> > >> >
> > > >>> > >>
> > > >>> >
> > > >>>
> > >
> >
>
/gpfs/dell2/emc/verification/noscrub/Julie.Prestopnik/met/9.1_beta2/bin/tc_pairs
> > > >>> > >> > in 0:00:00.607349
> > > >>> > >> > 08/26 18:35:12.422 metplus.CyclonePlotter
> > > >>> > >> (cyclone_plotter_wrapper.py:77)
> > > >>> > >> > DEBUG: Begin retrieving data...
> > > >>> > >> > 08/26 18:35:12.422 metplus.CyclonePlotter
> > > >>> > >> (cyclone_plotter_wrapper.py:83)
> > > >>> > >> > DEBUG: Generate plot for all files in the
> > > >>> > >> >
> > > >>> > >> >
> > > >>> > >>
> > > >>> >
> > > >>>
> > >
> >
>
directory/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs
> > > >>> > >> > 08/26 18:35:12.423 metplus.CyclonePlotter
> > > >>> > >> (cyclone_plotter_wrapper.py:98)
> > > >>> > >> > DEBUG: Parsing file
> > > >>> > >> >
> > > >>> > >> >
> > > >>> > >>
> > > >>> >
> > > >>>
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs/201909/AL_q2019092400.gfs.PRNA20.tcst
> > > >>> > >> > 08/26 18:35:19.778 metplus (config_launcher.py:520)
DEBUG:
> > > Setting
> > > >>> > >> [config]
> > > >>> > >> > SCRUB_STAGING_DIR to default value: False.
> > > >>> > >> > 08/26 18:35:19.779 metplus INFO:
> > > >>> > >> >
> > > >>> > >> >
> > > >>> > >>
> > > >>> >
> > > >>>
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/metplus_final.conf:
> > > >>> > >> > write metplus.conf here
> > > >>> > >> > 08/26 18:35:19.783 metplus (met_util.py:213) DEBUG:
METplus
> > took
> > > >>> > >> > 0:00:08.783240 to run.
> > > >>> > >> > 08/26 18:35:19.783 metplus INFO: METplus has
successfully
> > > finished
> > > >>> > >> running.
> > > >>> > >> >
> > > >>> > >> > Specifically the line that says matching 0 ADECK
tracks to 0
> > > BDECK
> > > >>> > >> tracks
> > > >>> > >> > lends me to believe that this is the reason why
nothing is
> > being
> > > >>> added
> > > >>> > >> to
> > > >>> > >> > my TC_PAIRS_OUTPUT_TEMPLATE file, and therefore no
plots
> > > >>> generated on
> > > >>> > a
> > > >>> > >> > map.
> > > >>> > >> >
> > > >>> > >> > What's interesting is the file that is extracted for
BDECK
> is
> > > for
> > > >>> > >> hurricane
> > > >>> > >> > Lorenzo and includes all the dates coinciding with
the ADECK
> > > file
> > > >>> - a
> > > >>> > >> case
> > > >>> > >> > that is run coinciding with Lorenzo. I've checked
this.
> > > >>> > >> >
> > > >>> > >> > One possible issue is the formatting of these files.
I
> looked
> > > in
> > > >>> the
> > > >>> > >> test
> > > >>> > >> > case that comes with the cloned met stuff. In there
the
> > > >>> header/field
> > > >>> > >> info
> > > >>> > >> > looks something like this:
> > > >>> > >> >
> > > >>> > >> >
> > > >>> > >> >
> > > >>> > >>
> > > >>> >
> > > >>>
> > >
> >
>
*/gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/METplus/source_data/met_test/new/track_data/201412/amlq2014123118.gfso.0104*
> > > >>> > >> >
> > > >>> > >> > ML, 0104, 2014121318_F000_478N_1780W_FOF, 2014121318,
03,
> > GFSO,
> > > >>> 000,
> > > >>> > >> 478N,
> > > >>> > >> > 1780W, 38, 986, XX, 34, NEQ, 0283, 0640, 0645,
0000,
> 990,
> > > >>> 133,
> > > >>> > -99,
> > > >>> > >> > 69, 227, -99, -9999, -9999, 208, 315, 135, 156,
0063
> > > >>> > >> >
> > > >>> > >> > This differs from the structure of both the ADECK and
BDECK
> > > input
> > > >>> > files:
> > > >>> > >> >
> > > >>> > >> > *BDECK:
> > > >>> > >> >
> > > >>>
>
/gpfs/hps3/emc/hwrf/noscrub/emc.hurpara/trak/abdeck/btk/bal132019.dat
> > > >>> > *
> > > >>> > >> >
> > > >>> > >> > AL, 13, 2019092218, , BEST, 0, 109N, 188W, 25,
1008,
> LO,
> > > >>> 0,
> > > >>> > ,
> > > >>> > >> > 0, 0, 0, 0, 1012, 100, 90, 40, 0, L,
0,
> > ,
> > > >>> 0,
> > > >>> > >> 0,
> > > >>> > >> > INVEST, S, 0, , 0, 0, 0, 0,
genesis-num,
> > 027,
> > > >>> > >> >
> > > >>> > >> > *ADECK:
> > > >>> > >> >
> > > >>> > >> >
> > > >>> > >>
> > > >>> >
> > > >>>
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400*
> > > >>> > >> >
> > > >>> > >> > AL, 0912, 03, PRNA, 000, 157N, 655W, 27, 1008, XX,
34,
> NEQ,
> > > >>> 0000,
> > > >>> > >> 0000,
> > > >>> > >> > 0000, 0000, -9999, -9999, 85
> > > >>> > >> >
> > > >>> > >> > My suspicion is that these formats are not
compatible.
> There
> > > are
> > > >>> > major
> > > >>> > >> > differences between these formats in my opinion. The
ADECK
> > was
> > > >>> > >> generated
> > > >>> > >> > automatically when making a run while BDECK is done
by
> someone
> > > >>> else.
> > > >>> > >> Those
> > > >>> > >> > files are stored whenever a storm is identified.
> > > >>> > >> >
> > > >>> > >> > I think what I'll need to do next is start by running
TCRMW
> to
> > > >>> produce
> > > >>> > >> the
> > > >>> > >> > ADECK files formatted by metplus. I have the grib
files for
> > > that.
> > > >>> > The
> > > >>> > >> > question next, though, is the BDECK files okay for
> processing?
> > > >>> > >> >
> > > >>> > >> >
> > > >>> > >> >
> > > >>> > >> > --
> > > >>> > >> > Edward Strobach
> > > >>> > >> > EMC/NCEP/NWS/
> > > >>> > >> > IMSG Contractor
> > > >>> > >> > Cubicle#: 2029
> > > >>> > >> > 301-683-3717
> > > >>> > >> >
> > > >>> > >> >
> > > >>> > >>
> > > >>> > >> --
> > > >>> > >> George McCabe - Software Engineer III
> > > >>> > >> National Center for Atmospheric Research
> > > >>> > >> Research Applications Laboratory
> > > >>> > >> 303-497-2768
> > > >>> > >> ---
> > > >>> > >> My working day may not be your working day. Please do
not feel
> > > >>> obliged
> > > >>> > to
> > > >>> > >> reply to this email outside of your normal working
hours.
> > > >>> > >>
> > > >>> > >>
> > > >>> > >
> > > >>> > > --
> > > >>> > > Edward Strobach
> > > >>> > > EMC/NCEP/NWS/
> > > >>> > > IMSG Contractor
> > > >>> > > Cubicle#: 2029
> > > >>> > > 301-683-3717
> > > >>> > >
> > > >>> >
> > > >>> >
> > > >>> > --
> > > >>> > Edward Strobach
> > > >>> > EMC/NCEP/NWS/
> > > >>> > IMSG Contractor
> > > >>> > Cubicle#: 2029
> > > >>> > 301-683-3717
> > > >>> >
> > > >>> >
> > > >>>
> > > >>> --
> > > >>> George McCabe - Software Engineer III
> > > >>> National Center for Atmospheric Research
> > > >>> Research Applications Laboratory
> > > >>> 303-497-2768
> > > >>> ---
> > > >>> My working day may not be your working day. Please do not
feel
> > obliged
> > > to
> > > >>> reply to this email outside of your normal working hours.
> > > >>>
> > > >>>
> > > >>
> > > >> --
> > > >> Edward Strobach
> > > >> EMC/NCEP/NWS/
> > > >> IMSG Contractor
> > > >> Cubicle#: 2029
> > > >> 301-683-3717
> > > >>
> > > >
> > > >
> > > > --
> > > > Edward Strobach
> > > > EMC/NCEP/NWS/
> > > > IMSG Contractor
> > > > Cubicle#: 2029
> > > > 301-683-3717
> > > >
> > >
> > >
> > > --
> > > Edward Strobach
> > > EMC/NCEP/NWS/
> > > IMSG Contractor
> > > Cubicle#: 2029
> > > 301-683-3717
> > >
> > >
> >
> >
>
> --
> Edward Strobach
> EMC/NCEP/NWS/
> IMSG Contractor
> Cubicle#: 2029
> 301-683-3717
>
>
------------------------------------------------
Subject: Tropical Cyclone verification: TCPairs and CyclonePlotter
From: Edward Strobach - NOAA Affiliate
Time: Tue Sep 01 10:44:10 2020
Thanks. Those examples are helpful and do confirm that the files
created
and passed into the TCPairs as ADECK files do follow, more or less,
the
format that's shown in the a{basin}{STORMNUM}{YYYY}.dat files. I also
found this
<https://dtcenter.org/metplus-practical-session-guide-feb-
2019/session-5-trkintfeature-relative/met-tool-tc-pairs>
to be helpful as well. When comparing the values/description in my
file
(/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092900)
with the contents in the link, it is clear that the order of my file
obeys
the following:
Basin, CY, YYYYMMDDHH, TECHNUM/MIN, TECH, TAU, LATN/S, LonE/W, VMAX,
MSLP,
TY, RAD, WINDCODE, RAD1, RAD2, RAD3, RAD4.
My first line of the referenced file has that in addition to 3 columns
at
the end - so 20 instead of 17 entries. I assume that the last 3 are
ignored.
*AL, 13, 2019092900, 03, PRNA, 000, 238N, 448W, 85, 962, XX, 34,
NEQ,
0185, 0184, 0110, 0148*, 978, 30, 32
*The good thing is that I figured it out. * Since feedback is
desirable,
I'm going to go through the process that helped me arrive at the
solution
which I detail below.
Since there can be more than one storm in the Atlantic at a given
time, I
decided to filter by *TC_PAIRS_CYCLONE* and set that to *13* since I'm
interested in processing Lorenzo. I can see that it's defined in my
master
config file as well. I did set the log verbosity to 10 to retrieve
additional info. There are no error messages; however, it says
"*Matching
0 ADECK tracks to 1 BDECK trac*k". The BDECK track, which I retrieved
from
the best track data formatted as bal132019.dat, is being processed
alongside the ADECK files, which are created during the forecast step.
Since I've run 6 forecasts that go out to 144 hours, I also have 6
ADECK
files initialized between 20190924 00Z and 20190929 00Z. In the case
of
20190929 00Z, for example, the log file indicates that the BDECK track
is
processed followed by the processing of the ADECK file. The log
correctly
identifies 119 lines of processing but says 0 of 119 lines are used:
*DEBUG 5: | | | [3] QuadInfo: Intensity = 64, ALVal =
-9999.00000,
NEVal = -9999.00000, SEVal = -9999.00000, SWVal = -9999.00000, NWVal =
-9999.00000DEBUG 5:DEBUG 5:DEBUG 5:DEBUG 2: Found 1 BDECK
track(s).DEBUG 2:
Processing 1 ADECK file(s).DEBUG 4: [File 1 of 1] Used 0 of 119 lines
read
from file
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092900"DEBUG
3: Used 0 of 119 lines read from 1 file(s).DEBUG 3: Identified 0
track(s).DEBUG 5: TrackInfoArray: NTracks = 0, NAlloc = 0,
Tracks:DEBUG
5:DEBUG 2: Deriving 12-hour interpolated ADECK tracks.DEBUG 2:
Finished
adding 0 and replacing 0 Interp12 track(s).DEBUG 2: Deriving 0 ADECK
consensus model(s).DEBUG 2: Added 0 ADECK consensus tracks(s).DEBUG 2:
Deriving 0 ADECK lag model(s).DEBUG 2: Added 0 ADECK lag
tracks(s).DEBUG 2:
Deriving 0 CLIPER/SHIFOR baseline model(s).DEBUG 2: Added 0
CLIPER/SHIFOR
baseline track(s).DEBUG 2: Filtering 0 ADECK tracks based on config
file
settings.DEBUG 3: Total tracks read = 0DEBUG 3: Total
tracks
kept = 0DEBUG 3: Rejected for storm name =
0DEBUG
3: Rejected for valid time = 0DEBUG 3: Rejected for required
lead
times = 0DEBUG 3: Rejected for init mask = 0DEBUG 3:
Rejected for
valid mask = 0DEBUG 2: Matching 0 ADECK tracks to 1 BDECK
tracks.*
This is repeated for all six forecasts. Also, all lines of the BDECK
file
are processed, so the dates in that file encompass the dates in each
of the
forecast files.
I was flummoxed as to why this would be. Under closer inspection I
found
that the directory - which is named after the model - includes the
experiment number as well (e.g. prna20). However, inside the ADECK
files
the model is PRNA, not PRNA20. Met was looking for instances of
PRNA20,
and when it couldn't it concluded that there were zero matches.
However,
when I changed my pathway to "TC_PAIRS_ADECK_INPUT_DIR =
/gpfs/dell2/emc/modeling/noscrub/${USER}/archive/${model}20" and set
model
to "prna" instead of "prna20" it worked. I now have populated files.
Moving forward I will need to keep this subtlety in mind. I plan to
use
other TC packages pretty soon here since I've been tasked to examine
some
physics changes and tune where necessary. I don't know what type of
issues
I may encounter, but I plan to use this experience as a model to
approach
possible issues related to TC verification in the future.
On Mon, Aug 31, 2020 at 3:31 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Ed,
>
> Here's a link to the documentation that the Navy provides about the
ATCF
> file format:
> https://www.nrlmry.navy.mil/atcf_web/docs/database/new/abrdeck.html
>
> And the best source of sample data is NHC's ftp site:
> https://ftp.nhc.noaa.gov/atcf/
>
> In there...
> *aid_public* contains tar files of ADECKS for all storms in the
current
> season.
> *btk* has uncompressed BEST track files for all storms in the
current
> season.
> *archive* contains compressed A and B-Deck data for previous
seasons.
>
> It's pretty cool actually that this track data is so readily
available.
>
> I have never actually modified ATCF files for my purposes. So
hopefully you
> won't need to either... and can just use the output of the tracker
code
> directly.
>
> Thanks,
> John
>
>
> On Mon, Aug 31, 2020 at 9:24 AM Edward Strobach - NOAA Affiliate via
RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458 >
> >
> > Hi John,
> >
> > I did try that. I rewrote the ADECK file with the storm id added
at the
> > end. I'm pasting an example below:
> > AL, 13, 2019092400, 03, PRNA, 012, 118N, 282W, 50, 993, XX,
34, NEQ,
> > 0168, 0115, 0046, 0134, 1013, 161, 111, AL132019
> >
> > You can see that the last entry, which was not included in the
original
> > file, has the storm id of AL132019. I'm not very familiar with
ADECK v.
> > BDECK files. Do you have an example of ADECK file format along
with the
> > header info? I can try to rewrite the files following the
appropriate
> > order/format, and from there pass the new ADECK file into TCPairs.
> >
> > On Fri, Aug 28, 2020 at 5:50 PM John Halley Gotway via RT <
> > met_help at ucar.edu>
> > wrote:
> >
> > > Hi Ed,
> > >
> > > You are correct. Generally the STORM_NAME is not present in the
ADECK
> > file.
> > > It only appears in the BDECK file.
> > >
> > > Please try configuring it by specifying a STORM_ID. MET
constructs the
> > > storm id on the fly from the input ATCF data... as the 2-digit
basin,
> > > followed by the 2 digit storm number, followed by the 4 digit
year
> (e.g.
> > > AL092011 for the 9 Atlantic Hurricane storm of 2011).
> > >
> > > So you can specify the storm_id in the config file, but not the
storm
> > name.
> > >
> > > Thanks,
> > > John
> > >
> > > On Fri, Aug 28, 2020 at 3:45 PM Edward Strobach - NOAA Affiliate
via
> RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458
>
> > > >
> > > > It appears that even after adding storm_id/storm_name, that
met is
> not
> > > able
> > > > to identify a track in the file provided. I believe it's
because
> > neither
> > > > the storm name or storm ID is written in the ADECK file! I
may have
> to
> > > > write a script that parses the text file and writes in the
storm name
> > and
> > > > id in two separate columns, respectively. Here's a line
example with
> > > what
> > > > it currently looks like for one row of the ADECK file:
> > > >
> > > > AL, 13, 2019092500, 03, PRNA, 000, 133N, 313W, 58, 993, XX,
50,
> > NEQ,
> > > > 0054, 0000, 0000, 0052, 1013, 172, 32
> > > >
> > > > If I add AL132019 and LORENZO as the last two columns of this
row,
> then
> > > > should that remedy the problem that I have. Do you think that
would
> > > > resolve it?
> > > >
> > > > On Fri, Aug 28, 2020 at 4:53 PM Edward Strobach - NOAA
Affiliate <
> > > > edward.strobach at noaa.gov> wrote:
> > > >
> > > > > It seems to me, though, that these options are rather
redundant.
> If
> > > one
> > > > > supplies a file with the Basin name, storm id, and storm
name, then
> > it
> > > > > would seem to me that all one would need to do is supply the
storm
> id
> > > (or
> > > > > name) with everything else being extracted from that using
the
> > contents
> > > > in
> > > > > the file.
> > > > >
> > > > > On Fri, Aug 28, 2020 at 4:41 PM Edward Strobach - NOAA
Affiliate <
> > > > > edward.strobach at noaa.gov> wrote:
> > > > >
> > > > >> Thanks for that. I left the TC_PAIRS_CYCLONE,
TC_PAIRS_BASIN,
> > > > >> TC_PAIRS_STORM_NAME, TC_PAIRS_STORM_ID alone. I may have
to
> > develop a
> > > > way
> > > > >> to fill this information in without manually specifying it.
If
> that
> > > > proves
> > > > >> futile, then I'll take a look at using the GFDL vortex
tracker.
> > > Thanks
> > > > >>
> > > > >> On Thu, Aug 27, 2020 at 3:37 PM George McCabe via RT <
> > > met_help at ucar.edu
> > > > >
> > > > >> wrote:
> > > > >>
> > > > >>> Hi Edward,
> > > > >>>
> > > > >>> From John HG:
> > > > >>>
> > > > >>>
> > > > >>>
> > > > >>> *The MET-TC tools read track files in ATCF format. Those
track
> > files
> > > > can
> > > > >>> be
> > > > >>> generated from gridded model output by running software.
One such
> > > > tracker
> > > > >>> is supported to the community by the DTC and is named the
"GFDL
> > > Vortex
> > > > >>> Tracker":http://dtcenter.org/community-code/gfdl-vortex-
tracker
> > > > >>> <http://dtcenter.org/community-code/gfdl-vortex-tracker>*
> > > > >>> *It is not expressly part of METplus but would be very
useful for
> > > > >>> tracking
> > > > >>> storms in GRIB files.*
> > > > >>>
> > > > >>> My guess as to why none of the tracks are being used has
to do
> with
> > > the
> > > > >>> METplus configurations to filter data. I would check the
values
> for
> > > > >>> TC_PAIRS_CYCLONE, TC_PAIRS_BASIN, TC_PAIRS_STORM_NAME,
> > > > TC_PAIRS_STORM_ID,
> > > > >>> and MODEL. The values you put in these lists determine
which
> storms
> > > to
> > > > >>> use.
> > > > >>> You can leave them blank to use any.
> > > > >>>
> > > > >>> Also, you could bump up the MET log verbosity and see more
> > > information
> > > > >>> about why they are being rejected:
> > > > >>>
> > > > >>> # to affect all MET tools set:
> > > > >>> LOG_MET_VERBOSITY = 10
> > > > >>>
> > > > >>> # to affect only TCPairs set:
> > > > >>> LOG_TC_PAIRS_VERBOSITY = 10
> > > > >>>
> > > > >>> - George
> > > > >>>
> > > > >>> On Thu, Aug 27, 2020 at 10:53 AM Edward Strobach - NOAA
Affiliate
> > via
> > > > RT
> > > > >>> <
> > > > >>> met_help at ucar.edu> wrote:
> > > > >>>
> > > > >>> >
> > > > >>> > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458
> >
> > > > >>> >
> > > > >>> > Hi,
> > > > >>> >
> > > > >>> > I did decide to remove CyclonePlotter for now and use
the
> > > > >>> TCPairs_Config
> > > > >>> > file along with settings that I set in a shared_TC.conf
that I
> > > store
> > > > >>> in a
> > > > >>> > temporary directory to process the ADECK and BDECK data
that I
> > > have.
> > > > >>> Files
> > > > >>> > are generated for all the dates that span INIT_BEG and
> INIT_END,
> > > > >>> thanks to
> > > > >>> > setting loop_by to INIT. However, only the header is
written
> out,
> > > > and I
> > > > >>> > suspect it is do to what the log indicates here:
> > > > >>> >
> > > > >>> > DEBUG 4: [File 1 of 5] Used 0 of 168 lines read from
file
> > > > >>> >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
> > > > >>> > DEBUG 4: [File 2 of 5] Used 0 of 168 lines read from
file
> > > > >>> >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
> > > > >>> > DEBUG 4: [File 3 of 5] Used 0 of 168 lines read from
file
> > > > >>> >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
> > > > >>> > DEBUG 4: [File 4 of 5] Used 0 of 168 lines read from
file
> > > > >>> >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
> > > > >>> > DEBUG 4: [File 5 of 5] Used 0 of 168 lines read from
file
> > > > >>> >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
> > > > >>> > DEBUG 3: Used 0 of 840 lines read from 5 file(s).
> > > > >>> > DEBUG 3: Identified 0 track(s).
> > > > >>> > DEBUG 5: TrackInfoArray: NTracks = 0, NAlloc = 0,
Tracks:
> > > > >>> > DEBUG 5:
> > > > >>> > DEBUG 2: Deriving 12-hour interpolated ADECK tracks.
> > > > >>> > DEBUG 2: Finished adding 0 and replacing 0 Interp12
track(s).
> > > > >>> > DEBUG 2: Deriving 0 ADECK consensus model(s).
> > > > >>> > DEBUG 2: Added 0 ADECK consensus tracks(s).
> > > > >>> > DEBUG 2: Deriving 0 ADECK lag model(s).
> > > > >>> > DEBUG 2: Added 0 ADECK lag tracks(s).
> > > > >>> > DEBUG 2: Deriving 0 CLIPER/SHIFOR baseline model(s).
> > > > >>> > DEBUG 2: Added 0 CLIPER/SHIFOR baseline track(s).
> > > > >>> > DEBUG 2: Filtering 0 ADECK tracks based on config file
> settings.
> > > > >>> > DEBUG 3: Total tracks read = 0
> > > > >>> > DEBUG 3: Total tracks kept = 0
> > > > >>> > DEBUG 3: Rejected for storm name = 0
> > > > >>> > DEBUG 3: Rejected for valid time = 0
> > > > >>> > DEBUG 3: Rejected for required lead times = 0
> > > > >>> > DEBUG 3: Rejected for init mask = 0
> > > > >>> > DEBUG 3: Rejected for valid mask = 0
> > > > >>> > DEBUG 2: Matching 0 ADECK tracks to 1 BDECK tracks.
> > > > >>> >
> > > > >>> > I'm still thinking that the formatting is off in this
file. I
> > > > surmise
> > > > >>> that
> > > > >>> > it's not able to group the data because the column
information
> is
> > > not
> > > > >>> in
> > > > >>> > the order that met would want it to be in. These files
do not
> > have
> > > > any
> > > > >>> > header info, so it has to rely on the correct ordering,
or so I
> > > > >>> suspect.
> > > > >>> > My interpretation, however, may be flawed. I see 6
separate TC
> > > based
> > > > >>> > metplus routines: TCPairs (I'm working on currently);
> > > CyclonePlotter
> > > > >>> > (looking to use); TCStat; TCRMW; TCMPRPlotter; and
TCGen.
> Much,
> > if
> > > > not
> > > > >>> > almost all of these relies on TCPairs. Is there a
routine that
> > I'm
> > > > >>> missing
> > > > >>> > in metplus that takes a grib file and constructs ADECK
files?
> > > > >>> >
> > > > >>> >
> > > > >>> >
> > > > >>> > On Thu, Aug 27, 2020 at 11:52 AM Edward Strobach - NOAA
> > Affiliate <
> > > > >>> > edward.strobach at noaa.gov> wrote:
> > > > >>> >
> > > > >>> > > Hi George, I figured the BDECK must be compatible, so
no
> > worries
> > > > >>> there.
> > > > >>> > > Thanks for the link and the heads up about
reformatting. I
> > will
> > > > use
> > > > >>> the
> > > > >>> > > example to modify what I currently have to see the
outcome.
> > > Thanks
> > > > >>> > again.
> > > > >>> > >
> > > > >>> > > On Wed, Aug 26, 2020 at 6:57 PM George McCabe via RT <
> > > > >>> met_help at ucar.edu>
> > > > >>> > > wrote:
> > > > >>> > >
> > > > >>> > >> Hi Edward,
> > > > >>> > >>
> > > > >>> > >> The BDECK data is compatible to be processed. The use
case
> you
> > > > used
> > > > >>> as
> > > > >>> > an
> > > > >>> > >> example uses extra tropical cyclone data from SBU
that is
> > > > >>> reformatted by
> > > > >>> > >> TCPairs wrapper before processing. You will see your
config
> > file
> > > > >>> has the
> > > > >>> > >> following:
> > > > >>> > >>
> > > > >>> > >> ## tc-pairs filtering optionsTC_PAIRS_REFORMAT_DECK =
> > > > >>> > >> yesTC_PAIRS_REFORMAT_TYPE = SBU
> > > > >>> > >>
> > > > >>> > >> Changing TC_PAIRS_REFORMAT_DECK to 'no' or 'false'
will skip
> > the
> > > > >>> > >> reformatting step. There is another use case in the
repo
> that
> > > uses
> > > > >>> data
> > > > >>> > >> that looks like yours (at least as far as I can
tell).
> > > > >>> > >>
> > > > >>> > >>
> > > > >>> > >>
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
https://dtcenter.github.io/METplus/generated/met_tool_wrapper/TCPairs/TCPairs_tropical.html#metplus-
configuration
> > > > >>> > >>
> > > > >>> > >> This use case uses BDECK data in
met_test/new/hwrf/bdeck.
> Here
> > > is
> > > > >>> part
> > > > >>> > of
> > > > >>> > >> the file called bal062018.dat:
> > > > >>> > >>
> > > > >>> > >> AL, 06, 2018082906, , BEST, 0, 129N, 109W, 10,
1010,
> DB,
> > > > >>> 0, ,
> > > > >>> > >> 0, 0, 0, 0, 0, 0, 0, 0, 0, ,
0,
> > ,
> > > > 0,
> > > > >>> > 0,
> > > > >>> > >> GENESIS019, , 0, , 0, 0, 0, 0,
genesis-num,
> > > 019,
> > > > >>> > >>
> > > > >>> > >>
> > > > >>> > >> Let me know if changing that setting doesn't work and
I can
> > > take a
> > > > >>> > closer
> > > > >>> > >> look.
> > > > >>> > >>
> > > > >>> > >> Thanks,
> > > > >>> > >> George
> > > > >>> > >>
> > > > >>> > >> On Wed, Aug 26, 2020 at 1:17 PM Edward Strobach -
NOAA
> > Affiliate
> > > > >>> via RT
> > > > >>> > <
> > > > >>> > >> met_help at ucar.edu> wrote:
> > > > >>> > >>
> > > > >>> > >> >
> > > > >>> > >> > Wed Aug 26 13:13:28 2020: Request 96458 was acted
upon.
> > > > >>> > >> > Transaction: Ticket created by
edward.strobach at noaa.gov
> > > > >>> > >> > Queue: met_help
> > > > >>> > >> > Subject: Tropical Cyclone verification:
TCPairs and
> > > > >>> > CyclonePlotter
> > > > >>> > >> > Owner: Nobody
> > > > >>> > >> > Requestors: edward.strobach at noaa.gov
> > > > >>> > >> > Status: new
> > > > >>> > >> > Ticket <URL:
> > > > >>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458
> > > > >>> > >
> > > > >>> > >> >
> > > > >>> > >> >
> > > > >>> > >> > Good afternoon,
> > > > >>> > >> >
> > > > >>> > >> > I found it relatively straightforward to set-up
some
> scripts
> > > to
> > > > >>> > process
> > > > >>> > >> > forecast output along with obs by using this
resource:
> > > > >>> > >> >
> > > > >>> > >> >
> > > > >>> > >>
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
https://dtcenter.github.io/METplus/generated/model_applications/tc_and_extra_tc/Plotter_fcstGFS_obsGFS_ExtraTC.html#sphx-
glr-generated-model-applications-tc-and-extra-tc-plotter-fcstgfs-
obsgfs-extratc-py
> > > > >>> > >> > .
> > > > >>> > >> > I have been running the global model which
generates track
> > > data
> > > > >>> for
> > > > >>> > the
> > > > >>> > >> > post processing step following this naming
convention:
> > > > >>> > >> > atcfunix.gfs.YYYYMMDDHH. The best track data is
stored
> here
> > > on
> > > > >>> WCOSS:
> > > > >>> > >> >
/gpfs/hps3/emc/hwrf/noscrub/emc.hurpara/trak/abdeck/btk/.
> > > > >>> > >> >
> > > > >>> > >> > I have a python script that determines which file
to use
> > based
> > > > on
> > > > >>> > basin
> > > > >>> > >> > information, storm name, and date. That is piped
into the
> > > > >>> > >> > TC_PAIRS_BDECK_TEMPLATE while the forecast file
above is
> > piped
> > > > >>> into
> > > > >>> > the
> > > > >>> > >> > TC_PAIRS_ADECK_TEMPLATE this way:
> > > > >>> atcfunix.gfs.{date?fmt=%Y%m%d%H}.
> > > > >>> > >> >
> > > > >>> > >> > Inside my main script
> > > > >>> > >> >
> > > > >>> > >> >
> > > > >>> > >>
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
*(/gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/METplus/scripts/run_metplus_TCPlotter.sh*,
> > > > >>> > >> > I have three configure files that being processed
like
> so:*
> > > > >>> > >> > ${MET_PLUS}/ush/master_metplus.py -c
> > > > >>> > >> >
${MET_PLUS_CONF}/Plotter_fcstGFS_obsGFS_ExtraTC.conf -c
> > > > >>> > >> > ${MET_PLUS_TMP}/shared.conf_TC -c
> > > > ${MET_PLUS_CONF}/system_TC.conf*
> > > > >>> > >> >
> > > > >>> > >> > The system_TC.conf is of little concern because it
is
> mostly
> > > > >>> unchanged
> > > > >>> > >> > except for the OUTPUT_BASE, which is changed
specifically
> to
> > > > >>> where I
> > > > >>> > >> plan
> > > > >>> > >> > on storing the TC output. The shared_TC.conf, on
the
> other
> > > > hand,
> > > > >>> is
> > > > >>> > >> used
> > > > >>> > >> > to replace the content inside my
> > > > >>> Plotter_fcstGFS_obsGFS_ExtraTC.conf
> > > > >>> > >> file.
> > > > >>> > >> > In here I define
> > > > >>> > >> > the ADECK and BDECK input directories based on the
where
> the
> > > > >>> template
> > > > >>> > >> files
> > > > >>> > >> > above are being stored. Since I have several runs
> > > encompassing
> > > > >>> the
> > > > >>> > >> > tropical cyclone event - 20190924 through 20190929
- I
> have
> > > > >>> specified
> > > > >>> > my
> > > > >>> > >> > INIT_BEG as 20190924 and my INIT_END as 20190929.
I have
> > also
> > > > >>> set my
> > > > >>> > >> > INIT_INCREMENT to 86400 since I'm only processing
the 00z
> > > cycle
> > > > >>> for
> > > > >>> > >> now. I
> > > > >>> > >> > have a couple of lines of code before the configure
file
> > that
> > > > >>> joins
> > > > >>> > >> these
> > > > >>> > >> > dates in a comma separated list such that the
> > > > >>> > CYCLONE_PLOTTER_INIT_DATE
> > > > >>> > >> is
> > > > >>> > >> > "20190924,20190925,...,20190929". I'm not sure if
that is
> > > > allowed
> > > > >>> > >> though.
> > > > >>> > >> >
> > > > >>> > >> > When running the script the log that is generated
> produces 0
> > > > >>> errors.
> > > > >>> > >> > However, I find *two issues*: *1) there is only
one file
> > > being
> > > > >>> > carried
> > > > >>> > >> > over to generate results; that is the 20190924 date
-
> there
> > > > >>> should be
> > > > >>> > >> 6*,
> > > > >>> > >> > and *2) The files that are created are empty and
thus the
> > plot
> > > > >>> that is
> > > > >>> > >> > generated is simply a map with no data overlaid.*
> > > > >>> > >> >
> > > > >>> > >> > In attempting to debug further, I found something
that
> looks
> > > > >>> important
> > > > >>> > >> > shown here:
> > > > >>> > >> >
> > > > >>> > >> > DEBUG 3: Rejected for init mask = 0
> > > > >>> > >> > DEBUG 3: Rejected for valid mask = 0
> > > > >>> > >> > *DEBUG 2: Matching 0 ADECK tracks to 0 BDECK
tracks.*
> > > > >>> > >> > DEBUG 1: Watch/Warning file:
> > > > >>> > >> >
> > > > >>> > >> >
> > > > >>> > >>
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
/gpfs/dell2/emc/verification/noscrub/Julie.Prestopnik/met/9.1_beta2/share/met/tc_data/wwpts_us.txt
> > > > >>> > >> > DEBUG 5: TrackPairInfoArray: NPairs = 0, NAlloc =
0,
> Pairs:
> > > > >>> > >> > DEBUG 5:
> > > > >>> > >> > DEBUG 1: Output file:
> > > > >>> > >> >
> > > > >>> > >> >
> > > > >>> > >>
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs/201909/AL_q2019092400.gfs.PRNA20.tcst
> > > > >>> > >> > 08/26 18:35:12.422 metplus.TCPairs
(command_runner.py:191)
> > > > DEBUG:
> > > > >>> > >> Finished
> > > > >>> > >> > running
> > > > >>> > >> >
> > > > >>> > >> >
> > > > >>> > >>
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
/gpfs/dell2/emc/verification/noscrub/Julie.Prestopnik/met/9.1_beta2/bin/tc_pairs
> > > > >>> > >> > in 0:00:00.607349
> > > > >>> > >> > 08/26 18:35:12.422 metplus.CyclonePlotter
> > > > >>> > >> (cyclone_plotter_wrapper.py:77)
> > > > >>> > >> > DEBUG: Begin retrieving data...
> > > > >>> > >> > 08/26 18:35:12.422 metplus.CyclonePlotter
> > > > >>> > >> (cyclone_plotter_wrapper.py:83)
> > > > >>> > >> > DEBUG: Generate plot for all files in the
> > > > >>> > >> >
> > > > >>> > >> >
> > > > >>> > >>
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
directory/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs
> > > > >>> > >> > 08/26 18:35:12.423 metplus.CyclonePlotter
> > > > >>> > >> (cyclone_plotter_wrapper.py:98)
> > > > >>> > >> > DEBUG: Parsing file
> > > > >>> > >> >
> > > > >>> > >> >
> > > > >>> > >>
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs/201909/AL_q2019092400.gfs.PRNA20.tcst
> > > > >>> > >> > 08/26 18:35:19.778 metplus (config_launcher.py:520)
DEBUG:
> > > > Setting
> > > > >>> > >> [config]
> > > > >>> > >> > SCRUB_STAGING_DIR to default value: False.
> > > > >>> > >> > 08/26 18:35:19.779 metplus INFO:
> > > > >>> > >> >
> > > > >>> > >> >
> > > > >>> > >>
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/metplus_final.conf:
> > > > >>> > >> > write metplus.conf here
> > > > >>> > >> > 08/26 18:35:19.783 metplus (met_util.py:213) DEBUG:
> METplus
> > > took
> > > > >>> > >> > 0:00:08.783240 to run.
> > > > >>> > >> > 08/26 18:35:19.783 metplus INFO: METplus has
successfully
> > > > finished
> > > > >>> > >> running.
> > > > >>> > >> >
> > > > >>> > >> > Specifically the line that says matching 0 ADECK
tracks
> to 0
> > > > BDECK
> > > > >>> > >> tracks
> > > > >>> > >> > lends me to believe that this is the reason why
nothing is
> > > being
> > > > >>> added
> > > > >>> > >> to
> > > > >>> > >> > my TC_PAIRS_OUTPUT_TEMPLATE file, and therefore no
plots
> > > > >>> generated on
> > > > >>> > a
> > > > >>> > >> > map.
> > > > >>> > >> >
> > > > >>> > >> > What's interesting is the file that is extracted
for BDECK
> > is
> > > > for
> > > > >>> > >> hurricane
> > > > >>> > >> > Lorenzo and includes all the dates coinciding with
the
> ADECK
> > > > file
> > > > >>> - a
> > > > >>> > >> case
> > > > >>> > >> > that is run coinciding with Lorenzo. I've checked
this.
> > > > >>> > >> >
> > > > >>> > >> > One possible issue is the formatting of these
files. I
> > looked
> > > > in
> > > > >>> the
> > > > >>> > >> test
> > > > >>> > >> > case that comes with the cloned met stuff. In
there the
> > > > >>> header/field
> > > > >>> > >> info
> > > > >>> > >> > looks something like this:
> > > > >>> > >> >
> > > > >>> > >> >
> > > > >>> > >> >
> > > > >>> > >>
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
*/gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/METplus/source_data/met_test/new/track_data/201412/amlq2014123118.gfso.0104*
> > > > >>> > >> >
> > > > >>> > >> > ML, 0104, 2014121318_F000_478N_1780W_FOF,
2014121318, 03,
> > > GFSO,
> > > > >>> 000,
> > > > >>> > >> 478N,
> > > > >>> > >> > 1780W, 38, 986, XX, 34, NEQ, 0283, 0640, 0645,
0000,
> > 990,
> > > > >>> 133,
> > > > >>> > -99,
> > > > >>> > >> > 69, 227, -99, -9999, -9999, 208, 315, 135,
156, 0063
> > > > >>> > >> >
> > > > >>> > >> > This differs from the structure of both the ADECK
and
> BDECK
> > > > input
> > > > >>> > files:
> > > > >>> > >> >
> > > > >>> > >> > *BDECK:
> > > > >>> > >> >
> > > > >>>
> >
/gpfs/hps3/emc/hwrf/noscrub/emc.hurpara/trak/abdeck/btk/bal132019.dat
> > > > >>> > *
> > > > >>> > >> >
> > > > >>> > >> > AL, 13, 2019092218, , BEST, 0, 109N, 188W,
25, 1008,
> > LO,
> > > > >>> 0,
> > > > >>> > ,
> > > > >>> > >> > 0, 0, 0, 0, 1012, 100, 90, 40, 0,
L, 0,
> > > ,
> > > > >>> 0,
> > > > >>> > >> 0,
> > > > >>> > >> > INVEST, S, 0, , 0, 0, 0, 0,
genesis-num,
> > > 027,
> > > > >>> > >> >
> > > > >>> > >> > *ADECK:
> > > > >>> > >> >
> > > > >>> > >> >
> > > > >>> > >>
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400*
> > > > >>> > >> >
> > > > >>> > >> > AL, 0912, 03, PRNA, 000, 157N, 655W, 27, 1008,
XX, 34,
> > NEQ,
> > > > >>> 0000,
> > > > >>> > >> 0000,
> > > > >>> > >> > 0000, 0000, -9999, -9999, 85
> > > > >>> > >> >
> > > > >>> > >> > My suspicion is that these formats are not
compatible.
> > There
> > > > are
> > > > >>> > major
> > > > >>> > >> > differences between these formats in my opinion.
The
> ADECK
> > > was
> > > > >>> > >> generated
> > > > >>> > >> > automatically when making a run while BDECK is done
by
> > someone
> > > > >>> else.
> > > > >>> > >> Those
> > > > >>> > >> > files are stored whenever a storm is identified.
> > > > >>> > >> >
> > > > >>> > >> > I think what I'll need to do next is start by
running
> TCRMW
> > to
> > > > >>> produce
> > > > >>> > >> the
> > > > >>> > >> > ADECK files formatted by metplus. I have the grib
files
> for
> > > > that.
> > > > >>> > The
> > > > >>> > >> > question next, though, is the BDECK files okay for
> > processing?
> > > > >>> > >> >
> > > > >>> > >> >
> > > > >>> > >> >
> > > > >>> > >> > --
> > > > >>> > >> > Edward Strobach
> > > > >>> > >> > EMC/NCEP/NWS/
> > > > >>> > >> > IMSG Contractor
> > > > >>> > >> > Cubicle#: 2029
> > > > >>> > >> > 301-683-3717
> > > > >>> > >> >
> > > > >>> > >> >
> > > > >>> > >>
> > > > >>> > >> --
> > > > >>> > >> George McCabe - Software Engineer III
> > > > >>> > >> National Center for Atmospheric Research
> > > > >>> > >> Research Applications Laboratory
> > > > >>> > >> 303-497-2768
> > > > >>> > >> ---
> > > > >>> > >> My working day may not be your working day. Please do
not
> feel
> > > > >>> obliged
> > > > >>> > to
> > > > >>> > >> reply to this email outside of your normal working
hours.
> > > > >>> > >>
> > > > >>> > >>
> > > > >>> > >
> > > > >>> > > --
> > > > >>> > > Edward Strobach
> > > > >>> > > EMC/NCEP/NWS/
> > > > >>> > > IMSG Contractor
> > > > >>> > > Cubicle#: 2029
> > > > >>> > > 301-683-3717
> > > > >>> > >
> > > > >>> >
> > > > >>> >
> > > > >>> > --
> > > > >>> > Edward Strobach
> > > > >>> > EMC/NCEP/NWS/
> > > > >>> > IMSG Contractor
> > > > >>> > Cubicle#: 2029
> > > > >>> > 301-683-3717
> > > > >>> >
> > > > >>> >
> > > > >>>
> > > > >>> --
> > > > >>> George McCabe - Software Engineer III
> > > > >>> National Center for Atmospheric Research
> > > > >>> Research Applications Laboratory
> > > > >>> 303-497-2768
> > > > >>> ---
> > > > >>> My working day may not be your working day. Please do not
feel
> > > obliged
> > > > to
> > > > >>> reply to this email outside of your normal working hours.
> > > > >>>
> > > > >>>
> > > > >>
> > > > >> --
> > > > >> Edward Strobach
> > > > >> EMC/NCEP/NWS/
> > > > >> IMSG Contractor
> > > > >> Cubicle#: 2029
> > > > >> 301-683-3717
> > > > >>
> > > > >
> > > > >
> > > > > --
> > > > > Edward Strobach
> > > > > EMC/NCEP/NWS/
> > > > > IMSG Contractor
> > > > > Cubicle#: 2029
> > > > > 301-683-3717
> > > > >
> > > >
> > > >
> > > > --
> > > > Edward Strobach
> > > > EMC/NCEP/NWS/
> > > > IMSG Contractor
> > > > Cubicle#: 2029
> > > > 301-683-3717
> > > >
> > > >
> > >
> > >
> >
> > --
> > Edward Strobach
> > EMC/NCEP/NWS/
> > IMSG Contractor
> > Cubicle#: 2029
> > 301-683-3717
> >
> >
>
>
--
Edward Strobach
EMC/NCEP/NWS/
IMSG Contractor
Cubicle#: 2029
301-683-3717
------------------------------------------------
Subject: Tropical Cyclone verification: TCPairs and CyclonePlotter
From: Edward Strobach - NOAA Affiliate
Time: Tue Sep 01 11:48:04 2020
The good news is I've also got CyclonePlotter to work. However, it
would
appear that CyclonePlotter is really just meant to show a quick and
dirty
plot based on the available settings and the quality of the plot -
attached below.
I noticed that I can't process all forecasts at once, rather I have to
loop
through each of my forecasts in order to generate plots for 20190924
through 20190929. Also, I have no real control over grid lines, tick
marks, legend to distinguish between best track and model, etc. As
such, I
may abandon CyclonePlotter.
Something else that's a bit odd is that TCPairs is only grouping the
data
out to 72 hours instead of 144 hours. There's no apparent setting to
extend this to 144 hours according to online sources that I've
consulted.
Is there a way to add an option such as TC_MAX_LEAD so that met knows
to
group out to 144 hours?
TC_PAIRS_CONFIG_FILE
<https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_CONFIG_FILE>
INIT_HOUR_END
<https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
INIT_HOUR_END>
INIT_INCLUDE
<https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
INIT_INCLUDE>
INIT_EXCLUDE
<https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
INIT_EXCLUDE>
TC_PAIRS_READ_ALL_FILES
<https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_READ_ALL_FILES>
TC_PAIRS_MODEL
<https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_MODEL>
TC_PAIRS_STORM_ID
<https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_STORM_ID>
TC_PAIRS_BASIN
<https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_BASIN>
TC_PAIRS_CYCLONE
<https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_CYCLONE>
TC_PAIRS_STORM_NAME
<https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_STORM_NAME>
TC_PAIRS_DLAND_FILE
<https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_DLAND_FILE>
TC_PAIRS_MISSING_VAL_TO_REPLACE
<https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_MISSING_VAL_TO_REPLACE>
TC_PAIRS_MISSING_VAL
<https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_MISSING_VAL>
TC_PAIRS_SKIP_IF_REFORMAT_EXISTS
<https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_SKIP_IF_REFORMAT_EXISTS>
TC_PAIRS_SKIP_IF_OUTPUT_EXISTS
<https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_SKIP_IF_OUTPUT_EXISTS>
TC_PAIRS_REFORMAT_DECK
<https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_REFORMAT_DECK>
TC_PAIRS_REFORMAT_TYPE
<https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_REFORMAT_TYPE>
TC_PAIRS_CUSTOM_LOOP_LIST
<https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_CUSTOM_LOOP_LIST>
On Tue, Sep 1, 2020 at 12:43 PM Edward Strobach - NOAA Affiliate <
edward.strobach at noaa.gov> wrote:
> Thanks. Those examples are helpful and do confirm that the files
created
> and passed into the TCPairs as ADECK files do follow, more or less,
the
> format that's shown in the a{basin}{STORMNUM}{YYYY}.dat files. I
also
> found this
> <https://dtcenter.org/metplus-practical-session-guide-feb-
2019/session-5-trkintfeature-relative/met-tool-tc-pairs>
> to be helpful as well. When comparing the values/description in my
file
>
(/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092900)
> with the contents in the link, it is clear that the order of my file
obeys
> the following:
>
> Basin, CY, YYYYMMDDHH, TECHNUM/MIN, TECH, TAU, LATN/S, LonE/W, VMAX,
MSLP,
> TY, RAD, WINDCODE, RAD1, RAD2, RAD3, RAD4.
>
> My first line of the referenced file has that in addition to 3
columns at
> the end - so 20 instead of 17 entries. I assume that the last 3 are
> ignored.
>
> *AL, 13, 2019092900, 03, PRNA, 000, 238N, 448W, 85, 962, XX, 34,
NEQ,
> 0185, 0184, 0110, 0148*, 978, 30, 32
>
> *The good thing is that I figured it out. * Since feedback is
desirable,
> I'm going to go through the process that helped me arrive at the
solution
> which I detail below.
>
>
> Since there can be more than one storm in the Atlantic at a given
time, I
> decided to filter by *TC_PAIRS_CYCLONE* and set that to *13* since
I'm
> interested in processing Lorenzo. I can see that it's defined in my
master
> config file as well. I did set the log verbosity to 10 to retrieve
> additional info. There are no error messages; however, it says
"*Matching
> 0 ADECK tracks to 1 BDECK trac*k". The BDECK track, which I
retrieved
> from the best track data formatted as bal132019.dat, is being
processed
> alongside the ADECK files, which are created during the forecast
step.
>
> Since I've run 6 forecasts that go out to 144 hours, I also have 6
ADECK
> files initialized between 20190924 00Z and 20190929 00Z. In the
case of
> 20190929 00Z, for example, the log file indicates that the BDECK
track is
> processed followed by the processing of the ADECK file. The log
correctly
> identifies 119 lines of processing but says 0 of 119 lines are used:
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> *DEBUG 5: | | | [3] QuadInfo: Intensity = 64, ALVal =
> -9999.00000, NEVal = -9999.00000, SEVal = -9999.00000, SWVal =
-9999.00000,
> NWVal = -9999.00000DEBUG 5:DEBUG 5:DEBUG 5:DEBUG 2: Found 1 BDECK
> track(s).DEBUG 2: Processing 1 ADECK file(s).DEBUG 4: [File 1 of 1]
Used 0
> of 119 lines read from file
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092900"DEBUG
> 3: Used 0 of 119 lines read from 1 file(s).DEBUG 3: Identified 0
> track(s).DEBUG 5: TrackInfoArray: NTracks = 0, NAlloc = 0,
Tracks:DEBUG
> 5:DEBUG 2: Deriving 12-hour interpolated ADECK tracks.DEBUG 2:
Finished
> adding 0 and replacing 0 Interp12 track(s).DEBUG 2: Deriving 0 ADECK
> consensus model(s).DEBUG 2: Added 0 ADECK consensus tracks(s).DEBUG
2:
> Deriving 0 ADECK lag model(s).DEBUG 2: Added 0 ADECK lag
tracks(s).DEBUG 2:
> Deriving 0 CLIPER/SHIFOR baseline model(s).DEBUG 2: Added 0
CLIPER/SHIFOR
> baseline track(s).DEBUG 2: Filtering 0 ADECK tracks based on config
file
> settings.DEBUG 3: Total tracks read = 0DEBUG 3: Total
tracks
> kept = 0DEBUG 3: Rejected for storm name =
0DEBUG
> 3: Rejected for valid time = 0DEBUG 3: Rejected for
required lead
> times = 0DEBUG 3: Rejected for init mask = 0DEBUG 3:
Rejected for
> valid mask = 0DEBUG 2: Matching 0 ADECK tracks to 1 BDECK
tracks.*
>
> This is repeated for all six forecasts. Also, all lines of the
BDECK file
> are processed, so the dates in that file encompass the dates in each
of the
> forecast files.
>
> I was flummoxed as to why this would be. Under closer inspection I
found
> that the directory - which is named after the model - includes the
> experiment number as well (e.g. prna20). However, inside the ADECK
files
> the model is PRNA, not PRNA20. Met was looking for instances of
PRNA20,
> and when it couldn't it concluded that there were zero matches.
However,
> when I changed my pathway to "TC_PAIRS_ADECK_INPUT_DIR =
> /gpfs/dell2/emc/modeling/noscrub/${USER}/archive/${model}20" and
set model
> to "prna" instead of "prna20" it worked. I now have populated
files.
> Moving forward I will need to keep this subtlety in mind. I plan
to use
> other TC packages pretty soon here since I've been tasked to examine
some
> physics changes and tune where necessary. I don't know what type of
issues
> I may encounter, but I plan to use this experience as a model to
approach
> possible issues related to TC verification in the future.
>
>
>
> On Mon, Aug 31, 2020 at 3:31 PM John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
>> Ed,
>>
>> Here's a link to the documentation that the Navy provides about the
ATCF
>> file format:
>> https://www.nrlmry.navy.mil/atcf_web/docs/database/new/abrdeck.html
>>
>> And the best source of sample data is NHC's ftp site:
>> https://ftp.nhc.noaa.gov/atcf/
>>
>> In there...
>> *aid_public* contains tar files of ADECKS for all storms in the
current
>> season.
>> *btk* has uncompressed BEST track files for all storms in the
current
>> season.
>> *archive* contains compressed A and B-Deck data for previous
seasons.
>>
>> It's pretty cool actually that this track data is so readily
available.
>>
>> I have never actually modified ATCF files for my purposes. So
hopefully
>> you
>> won't need to either... and can just use the output of the tracker
code
>> directly.
>>
>> Thanks,
>> John
>>
>>
>> On Mon, Aug 31, 2020 at 9:24 AM Edward Strobach - NOAA Affiliate
via RT <
>> met_help at ucar.edu> wrote:
>>
>> >
>> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458 >
>> >
>> > Hi John,
>> >
>> > I did try that. I rewrote the ADECK file with the storm id added
at the
>> > end. I'm pasting an example below:
>> > AL, 13, 2019092400, 03, PRNA, 012, 118N, 282W, 50, 993, XX,
34, NEQ,
>> > 0168, 0115, 0046, 0134, 1013, 161, 111, AL132019
>> >
>> > You can see that the last entry, which was not included in the
original
>> > file, has the storm id of AL132019. I'm not very familiar with
ADECK v.
>> > BDECK files. Do you have an example of ADECK file format along
with the
>> > header info? I can try to rewrite the files following the
appropriate
>> > order/format, and from there pass the new ADECK file into
TCPairs.
>> >
>> > On Fri, Aug 28, 2020 at 5:50 PM John Halley Gotway via RT <
>> > met_help at ucar.edu>
>> > wrote:
>> >
>> > > Hi Ed,
>> > >
>> > > You are correct. Generally the STORM_NAME is not present in the
ADECK
>> > file.
>> > > It only appears in the BDECK file.
>> > >
>> > > Please try configuring it by specifying a STORM_ID. MET
constructs the
>> > > storm id on the fly from the input ATCF data... as the 2-digit
basin,
>> > > followed by the 2 digit storm number, followed by the 4 digit
year
>> (e.g.
>> > > AL092011 for the 9 Atlantic Hurricane storm of 2011).
>> > >
>> > > So you can specify the storm_id in the config file, but not the
storm
>> > name.
>> > >
>> > > Thanks,
>> > > John
>> > >
>> > > On Fri, Aug 28, 2020 at 3:45 PM Edward Strobach - NOAA
Affiliate via
>> RT <
>> > > met_help at ucar.edu> wrote:
>> > >
>> > > >
>> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458
>
>> > > >
>> > > > It appears that even after adding storm_id/storm_name, that
met is
>> not
>> > > able
>> > > > to identify a track in the file provided. I believe it's
because
>> > neither
>> > > > the storm name or storm ID is written in the ADECK file! I
may
>> have to
>> > > > write a script that parses the text file and writes in the
storm
>> name
>> > and
>> > > > id in two separate columns, respectively. Here's a line
example
>> with
>> > > what
>> > > > it currently looks like for one row of the ADECK file:
>> > > >
>> > > > AL, 13, 2019092500, 03, PRNA, 000, 133N, 313W, 58, 993,
XX, 50,
>> > NEQ,
>> > > > 0054, 0000, 0000, 0052, 1013, 172, 32
>> > > >
>> > > > If I add AL132019 and LORENZO as the last two columns of this
row,
>> then
>> > > > should that remedy the problem that I have. Do you think
that would
>> > > > resolve it?
>> > > >
>> > > > On Fri, Aug 28, 2020 at 4:53 PM Edward Strobach - NOAA
Affiliate <
>> > > > edward.strobach at noaa.gov> wrote:
>> > > >
>> > > > > It seems to me, though, that these options are rather
redundant.
>> If
>> > > one
>> > > > > supplies a file with the Basin name, storm id, and storm
name,
>> then
>> > it
>> > > > > would seem to me that all one would need to do is supply
the
>> storm id
>> > > (or
>> > > > > name) with everything else being extracted from that using
the
>> > contents
>> > > > in
>> > > > > the file.
>> > > > >
>> > > > > On Fri, Aug 28, 2020 at 4:41 PM Edward Strobach - NOAA
Affiliate <
>> > > > > edward.strobach at noaa.gov> wrote:
>> > > > >
>> > > > >> Thanks for that. I left the TC_PAIRS_CYCLONE,
TC_PAIRS_BASIN,
>> > > > >> TC_PAIRS_STORM_NAME, TC_PAIRS_STORM_ID alone. I may have
to
>> > develop a
>> > > > way
>> > > > >> to fill this information in without manually specifying
it. If
>> that
>> > > > proves
>> > > > >> futile, then I'll take a look at using the GFDL vortex
tracker.
>> > > Thanks
>> > > > >>
>> > > > >> On Thu, Aug 27, 2020 at 3:37 PM George McCabe via RT <
>> > > met_help at ucar.edu
>> > > > >
>> > > > >> wrote:
>> > > > >>
>> > > > >>> Hi Edward,
>> > > > >>>
>> > > > >>> From John HG:
>> > > > >>>
>> > > > >>>
>> > > > >>>
>> > > > >>> *The MET-TC tools read track files in ATCF format. Those
track
>> > files
>> > > > can
>> > > > >>> be
>> > > > >>> generated from gridded model output by running software.
One
>> such
>> > > > tracker
>> > > > >>> is supported to the community by the DTC and is named the
"GFDL
>> > > Vortex
>> > > > >>> Tracker":http://dtcenter.org/community-code/gfdl-vortex-
tracker
>> > > > >>> <http://dtcenter.org/community-code/gfdl-vortex-tracker>*
>> > > > >>> *It is not expressly part of METplus but would be very
useful
>> for
>> > > > >>> tracking
>> > > > >>> storms in GRIB files.*
>> > > > >>>
>> > > > >>> My guess as to why none of the tracks are being used has
to do
>> with
>> > > the
>> > > > >>> METplus configurations to filter data. I would check the
values
>> for
>> > > > >>> TC_PAIRS_CYCLONE, TC_PAIRS_BASIN, TC_PAIRS_STORM_NAME,
>> > > > TC_PAIRS_STORM_ID,
>> > > > >>> and MODEL. The values you put in these lists determine
which
>> storms
>> > > to
>> > > > >>> use.
>> > > > >>> You can leave them blank to use any.
>> > > > >>>
>> > > > >>> Also, you could bump up the MET log verbosity and see
more
>> > > information
>> > > > >>> about why they are being rejected:
>> > > > >>>
>> > > > >>> # to affect all MET tools set:
>> > > > >>> LOG_MET_VERBOSITY = 10
>> > > > >>>
>> > > > >>> # to affect only TCPairs set:
>> > > > >>> LOG_TC_PAIRS_VERBOSITY = 10
>> > > > >>>
>> > > > >>> - George
>> > > > >>>
>> > > > >>> On Thu, Aug 27, 2020 at 10:53 AM Edward Strobach - NOAA
>> Affiliate
>> > via
>> > > > RT
>> > > > >>> <
>> > > > >>> met_help at ucar.edu> wrote:
>> > > > >>>
>> > > > >>> >
>> > > > >>> > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458
>> >
>> > > > >>> >
>> > > > >>> > Hi,
>> > > > >>> >
>> > > > >>> > I did decide to remove CyclonePlotter for now and use
the
>> > > > >>> TCPairs_Config
>> > > > >>> > file along with settings that I set in a shared_TC.conf
that I
>> > > store
>> > > > >>> in a
>> > > > >>> > temporary directory to process the ADECK and BDECK data
that I
>> > > have.
>> > > > >>> Files
>> > > > >>> > are generated for all the dates that span INIT_BEG and
>> INIT_END,
>> > > > >>> thanks to
>> > > > >>> > setting loop_by to INIT. However, only the header is
written
>> out,
>> > > > and I
>> > > > >>> > suspect it is do to what the log indicates here:
>> > > > >>> >
>> > > > >>> > DEBUG 4: [File 1 of 5] Used 0 of 168 lines read from
file
>> > > > >>> >
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
>> > > > >>> > DEBUG 4: [File 2 of 5] Used 0 of 168 lines read from
file
>> > > > >>> >
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
>> > > > >>> > DEBUG 4: [File 3 of 5] Used 0 of 168 lines read from
file
>> > > > >>> >
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
>> > > > >>> > DEBUG 4: [File 4 of 5] Used 0 of 168 lines read from
file
>> > > > >>> >
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
>> > > > >>> > DEBUG 4: [File 5 of 5] Used 0 of 168 lines read from
file
>> > > > >>> >
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
>> > > > >>> > DEBUG 3: Used 0 of 840 lines read from 5 file(s).
>> > > > >>> > DEBUG 3: Identified 0 track(s).
>> > > > >>> > DEBUG 5: TrackInfoArray: NTracks = 0, NAlloc = 0,
Tracks:
>> > > > >>> > DEBUG 5:
>> > > > >>> > DEBUG 2: Deriving 12-hour interpolated ADECK tracks.
>> > > > >>> > DEBUG 2: Finished adding 0 and replacing 0 Interp12
track(s).
>> > > > >>> > DEBUG 2: Deriving 0 ADECK consensus model(s).
>> > > > >>> > DEBUG 2: Added 0 ADECK consensus tracks(s).
>> > > > >>> > DEBUG 2: Deriving 0 ADECK lag model(s).
>> > > > >>> > DEBUG 2: Added 0 ADECK lag tracks(s).
>> > > > >>> > DEBUG 2: Deriving 0 CLIPER/SHIFOR baseline model(s).
>> > > > >>> > DEBUG 2: Added 0 CLIPER/SHIFOR baseline track(s).
>> > > > >>> > DEBUG 2: Filtering 0 ADECK tracks based on config file
>> settings.
>> > > > >>> > DEBUG 3: Total tracks read = 0
>> > > > >>> > DEBUG 3: Total tracks kept = 0
>> > > > >>> > DEBUG 3: Rejected for storm name = 0
>> > > > >>> > DEBUG 3: Rejected for valid time = 0
>> > > > >>> > DEBUG 3: Rejected for required lead times = 0
>> > > > >>> > DEBUG 3: Rejected for init mask = 0
>> > > > >>> > DEBUG 3: Rejected for valid mask = 0
>> > > > >>> > DEBUG 2: Matching 0 ADECK tracks to 1 BDECK tracks.
>> > > > >>> >
>> > > > >>> > I'm still thinking that the formatting is off in this
file. I
>> > > > surmise
>> > > > >>> that
>> > > > >>> > it's not able to group the data because the column
>> information is
>> > > not
>> > > > >>> in
>> > > > >>> > the order that met would want it to be in. These files
do not
>> > have
>> > > > any
>> > > > >>> > header info, so it has to rely on the correct ordering,
or so
>> I
>> > > > >>> suspect.
>> > > > >>> > My interpretation, however, may be flawed. I see 6
separate
>> TC
>> > > based
>> > > > >>> > metplus routines: TCPairs (I'm working on currently);
>> > > CyclonePlotter
>> > > > >>> > (looking to use); TCStat; TCRMW; TCMPRPlotter; and
TCGen.
>> Much,
>> > if
>> > > > not
>> > > > >>> > almost all of these relies on TCPairs. Is there a
routine
>> that
>> > I'm
>> > > > >>> missing
>> > > > >>> > in metplus that takes a grib file and constructs ADECK
files?
>> > > > >>> >
>> > > > >>> >
>> > > > >>> >
>> > > > >>> > On Thu, Aug 27, 2020 at 11:52 AM Edward Strobach - NOAA
>> > Affiliate <
>> > > > >>> > edward.strobach at noaa.gov> wrote:
>> > > > >>> >
>> > > > >>> > > Hi George, I figured the BDECK must be compatible, so
no
>> > worries
>> > > > >>> there.
>> > > > >>> > > Thanks for the link and the heads up about
reformatting. I
>> > will
>> > > > use
>> > > > >>> the
>> > > > >>> > > example to modify what I currently have to see the
outcome.
>> > > Thanks
>> > > > >>> > again.
>> > > > >>> > >
>> > > > >>> > > On Wed, Aug 26, 2020 at 6:57 PM George McCabe via RT
<
>> > > > >>> met_help at ucar.edu>
>> > > > >>> > > wrote:
>> > > > >>> > >
>> > > > >>> > >> Hi Edward,
>> > > > >>> > >>
>> > > > >>> > >> The BDECK data is compatible to be processed. The
use case
>> you
>> > > > used
>> > > > >>> as
>> > > > >>> > an
>> > > > >>> > >> example uses extra tropical cyclone data from SBU
that is
>> > > > >>> reformatted by
>> > > > >>> > >> TCPairs wrapper before processing. You will see your
config
>> > file
>> > > > >>> has the
>> > > > >>> > >> following:
>> > > > >>> > >>
>> > > > >>> > >> ## tc-pairs filtering optionsTC_PAIRS_REFORMAT_DECK
=
>> > > > >>> > >> yesTC_PAIRS_REFORMAT_TYPE = SBU
>> > > > >>> > >>
>> > > > >>> > >> Changing TC_PAIRS_REFORMAT_DECK to 'no' or 'false'
will
>> skip
>> > the
>> > > > >>> > >> reformatting step. There is another use case in the
repo
>> that
>> > > uses
>> > > > >>> data
>> > > > >>> > >> that looks like yours (at least as far as I can
tell).
>> > > > >>> > >>
>> > > > >>> > >>
>> > > > >>> > >>
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>>
https://dtcenter.github.io/METplus/generated/met_tool_wrapper/TCPairs/TCPairs_tropical.html#metplus-
configuration
>> > > > >>> > >>
>> > > > >>> > >> This use case uses BDECK data in
met_test/new/hwrf/bdeck.
>> Here
>> > > is
>> > > > >>> part
>> > > > >>> > of
>> > > > >>> > >> the file called bal062018.dat:
>> > > > >>> > >>
>> > > > >>> > >> AL, 06, 2018082906, , BEST, 0, 129N, 109W, 10,
1010,
>> DB,
>> > > > >>> 0, ,
>> > > > >>> > >> 0, 0, 0, 0, 0, 0, 0, 0, 0,
, 0,
>> > ,
>> > > > 0,
>> > > > >>> > 0,
>> > > > >>> > >> GENESIS019, , 0, , 0, 0, 0, 0,
>> genesis-num,
>> > > 019,
>> > > > >>> > >>
>> > > > >>> > >>
>> > > > >>> > >> Let me know if changing that setting doesn't work
and I can
>> > > take a
>> > > > >>> > closer
>> > > > >>> > >> look.
>> > > > >>> > >>
>> > > > >>> > >> Thanks,
>> > > > >>> > >> George
>> > > > >>> > >>
>> > > > >>> > >> On Wed, Aug 26, 2020 at 1:17 PM Edward Strobach -
NOAA
>> > Affiliate
>> > > > >>> via RT
>> > > > >>> > <
>> > > > >>> > >> met_help at ucar.edu> wrote:
>> > > > >>> > >>
>> > > > >>> > >> >
>> > > > >>> > >> > Wed Aug 26 13:13:28 2020: Request 96458 was acted
upon.
>> > > > >>> > >> > Transaction: Ticket created by
edward.strobach at noaa.gov
>> > > > >>> > >> > Queue: met_help
>> > > > >>> > >> > Subject: Tropical Cyclone verification:
TCPairs and
>> > > > >>> > CyclonePlotter
>> > > > >>> > >> > Owner: Nobody
>> > > > >>> > >> > Requestors: edward.strobach at noaa.gov
>> > > > >>> > >> > Status: new
>> > > > >>> > >> > Ticket <URL:
>> > > > >>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458
>> > > > >>> > >
>> > > > >>> > >> >
>> > > > >>> > >> >
>> > > > >>> > >> > Good afternoon,
>> > > > >>> > >> >
>> > > > >>> > >> > I found it relatively straightforward to set-up
some
>> scripts
>> > > to
>> > > > >>> > process
>> > > > >>> > >> > forecast output along with obs by using this
resource:
>> > > > >>> > >> >
>> > > > >>> > >> >
>> > > > >>> > >>
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>>
https://dtcenter.github.io/METplus/generated/model_applications/tc_and_extra_tc/Plotter_fcstGFS_obsGFS_ExtraTC.html#sphx-
glr-generated-model-applications-tc-and-extra-tc-plotter-fcstgfs-
obsgfs-extratc-py
>> > > > >>> > >> > .
>> > > > >>> > >> > I have been running the global model which
generates
>> track
>> > > data
>> > > > >>> for
>> > > > >>> > the
>> > > > >>> > >> > post processing step following this naming
convention:
>> > > > >>> > >> > atcfunix.gfs.YYYYMMDDHH. The best track data is
stored
>> here
>> > > on
>> > > > >>> WCOSS:
>> > > > >>> > >> >
/gpfs/hps3/emc/hwrf/noscrub/emc.hurpara/trak/abdeck/btk/.
>> > > > >>> > >> >
>> > > > >>> > >> > I have a python script that determines which file
to use
>> > based
>> > > > on
>> > > > >>> > basin
>> > > > >>> > >> > information, storm name, and date. That is piped
into
>> the
>> > > > >>> > >> > TC_PAIRS_BDECK_TEMPLATE while the forecast file
above is
>> > piped
>> > > > >>> into
>> > > > >>> > the
>> > > > >>> > >> > TC_PAIRS_ADECK_TEMPLATE this way:
>> > > > >>> atcfunix.gfs.{date?fmt=%Y%m%d%H}.
>> > > > >>> > >> >
>> > > > >>> > >> > Inside my main script
>> > > > >>> > >> >
>> > > > >>> > >> >
>> > > > >>> > >>
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>>
*(/gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/METplus/scripts/run_metplus_TCPlotter.sh*,
>> > > > >>> > >> > I have three configure files that being processed
like
>> so:*
>> > > > >>> > >> > ${MET_PLUS}/ush/master_metplus.py -c
>> > > > >>> > >> >
${MET_PLUS_CONF}/Plotter_fcstGFS_obsGFS_ExtraTC.conf -c
>> > > > >>> > >> > ${MET_PLUS_TMP}/shared.conf_TC -c
>> > > > ${MET_PLUS_CONF}/system_TC.conf*
>> > > > >>> > >> >
>> > > > >>> > >> > The system_TC.conf is of little concern because it
is
>> mostly
>> > > > >>> unchanged
>> > > > >>> > >> > except for the OUTPUT_BASE, which is changed
>> specifically to
>> > > > >>> where I
>> > > > >>> > >> plan
>> > > > >>> > >> > on storing the TC output. The shared_TC.conf, on
the
>> other
>> > > > hand,
>> > > > >>> is
>> > > > >>> > >> used
>> > > > >>> > >> > to replace the content inside my
>> > > > >>> Plotter_fcstGFS_obsGFS_ExtraTC.conf
>> > > > >>> > >> file.
>> > > > >>> > >> > In here I define
>> > > > >>> > >> > the ADECK and BDECK input directories based on the
where
>> the
>> > > > >>> template
>> > > > >>> > >> files
>> > > > >>> > >> > above are being stored. Since I have several runs
>> > > encompassing
>> > > > >>> the
>> > > > >>> > >> > tropical cyclone event - 20190924 through 20190929
- I
>> have
>> > > > >>> specified
>> > > > >>> > my
>> > > > >>> > >> > INIT_BEG as 20190924 and my INIT_END as 20190929.
I have
>> > also
>> > > > >>> set my
>> > > > >>> > >> > INIT_INCREMENT to 86400 since I'm only processing
the 00z
>> > > cycle
>> > > > >>> for
>> > > > >>> > >> now. I
>> > > > >>> > >> > have a couple of lines of code before the
configure file
>> > that
>> > > > >>> joins
>> > > > >>> > >> these
>> > > > >>> > >> > dates in a comma separated list such that the
>> > > > >>> > CYCLONE_PLOTTER_INIT_DATE
>> > > > >>> > >> is
>> > > > >>> > >> > "20190924,20190925,...,20190929". I'm not sure if
that
>> is
>> > > > allowed
>> > > > >>> > >> though.
>> > > > >>> > >> >
>> > > > >>> > >> > When running the script the log that is generated
>> produces 0
>> > > > >>> errors.
>> > > > >>> > >> > However, I find *two issues*: *1) there is only
one file
>> > > being
>> > > > >>> > carried
>> > > > >>> > >> > over to generate results; that is the 20190924
date -
>> there
>> > > > >>> should be
>> > > > >>> > >> 6*,
>> > > > >>> > >> > and *2) The files that are created are empty and
thus the
>> > plot
>> > > > >>> that is
>> > > > >>> > >> > generated is simply a map with no data overlaid.*
>> > > > >>> > >> >
>> > > > >>> > >> > In attempting to debug further, I found something
that
>> looks
>> > > > >>> important
>> > > > >>> > >> > shown here:
>> > > > >>> > >> >
>> > > > >>> > >> > DEBUG 3: Rejected for init mask = 0
>> > > > >>> > >> > DEBUG 3: Rejected for valid mask = 0
>> > > > >>> > >> > *DEBUG 2: Matching 0 ADECK tracks to 0 BDECK
tracks.*
>> > > > >>> > >> > DEBUG 1: Watch/Warning file:
>> > > > >>> > >> >
>> > > > >>> > >> >
>> > > > >>> > >>
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>>
/gpfs/dell2/emc/verification/noscrub/Julie.Prestopnik/met/9.1_beta2/share/met/tc_data/wwpts_us.txt
>> > > > >>> > >> > DEBUG 5: TrackPairInfoArray: NPairs = 0, NAlloc =
0,
>> Pairs:
>> > > > >>> > >> > DEBUG 5:
>> > > > >>> > >> > DEBUG 1: Output file:
>> > > > >>> > >> >
>> > > > >>> > >> >
>> > > > >>> > >>
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs/201909/AL_q2019092400.gfs.PRNA20.tcst
>> > > > >>> > >> > 08/26 18:35:12.422 metplus.TCPairs
>> (command_runner.py:191)
>> > > > DEBUG:
>> > > > >>> > >> Finished
>> > > > >>> > >> > running
>> > > > >>> > >> >
>> > > > >>> > >> >
>> > > > >>> > >>
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>>
/gpfs/dell2/emc/verification/noscrub/Julie.Prestopnik/met/9.1_beta2/bin/tc_pairs
>> > > > >>> > >> > in 0:00:00.607349
>> > > > >>> > >> > 08/26 18:35:12.422 metplus.CyclonePlotter
>> > > > >>> > >> (cyclone_plotter_wrapper.py:77)
>> > > > >>> > >> > DEBUG: Begin retrieving data...
>> > > > >>> > >> > 08/26 18:35:12.422 metplus.CyclonePlotter
>> > > > >>> > >> (cyclone_plotter_wrapper.py:83)
>> > > > >>> > >> > DEBUG: Generate plot for all files in the
>> > > > >>> > >> >
>> > > > >>> > >> >
>> > > > >>> > >>
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>>
directory/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs
>> > > > >>> > >> > 08/26 18:35:12.423 metplus.CyclonePlotter
>> > > > >>> > >> (cyclone_plotter_wrapper.py:98)
>> > > > >>> > >> > DEBUG: Parsing file
>> > > > >>> > >> >
>> > > > >>> > >> >
>> > > > >>> > >>
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs/201909/AL_q2019092400.gfs.PRNA20.tcst
>> > > > >>> > >> > 08/26 18:35:19.778 metplus
(config_launcher.py:520)
>> DEBUG:
>> > > > Setting
>> > > > >>> > >> [config]
>> > > > >>> > >> > SCRUB_STAGING_DIR to default value: False.
>> > > > >>> > >> > 08/26 18:35:19.779 metplus INFO:
>> > > > >>> > >> >
>> > > > >>> > >> >
>> > > > >>> > >>
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/metplus_final.conf:
>> > > > >>> > >> > write metplus.conf here
>> > > > >>> > >> > 08/26 18:35:19.783 metplus (met_util.py:213)
DEBUG:
>> METplus
>> > > took
>> > > > >>> > >> > 0:00:08.783240 to run.
>> > > > >>> > >> > 08/26 18:35:19.783 metplus INFO: METplus has
successfully
>> > > > finished
>> > > > >>> > >> running.
>> > > > >>> > >> >
>> > > > >>> > >> > Specifically the line that says matching 0 ADECK
tracks
>> to 0
>> > > > BDECK
>> > > > >>> > >> tracks
>> > > > >>> > >> > lends me to believe that this is the reason why
nothing
>> is
>> > > being
>> > > > >>> added
>> > > > >>> > >> to
>> > > > >>> > >> > my TC_PAIRS_OUTPUT_TEMPLATE file, and therefore no
plots
>> > > > >>> generated on
>> > > > >>> > a
>> > > > >>> > >> > map.
>> > > > >>> > >> >
>> > > > >>> > >> > What's interesting is the file that is extracted
for
>> BDECK
>> > is
>> > > > for
>> > > > >>> > >> hurricane
>> > > > >>> > >> > Lorenzo and includes all the dates coinciding with
the
>> ADECK
>> > > > file
>> > > > >>> - a
>> > > > >>> > >> case
>> > > > >>> > >> > that is run coinciding with Lorenzo. I've checked
this.
>> > > > >>> > >> >
>> > > > >>> > >> > One possible issue is the formatting of these
files. I
>> > looked
>> > > > in
>> > > > >>> the
>> > > > >>> > >> test
>> > > > >>> > >> > case that comes with the cloned met stuff. In
there the
>> > > > >>> header/field
>> > > > >>> > >> info
>> > > > >>> > >> > looks something like this:
>> > > > >>> > >> >
>> > > > >>> > >> >
>> > > > >>> > >> >
>> > > > >>> > >>
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>>
*/gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/METplus/source_data/met_test/new/track_data/201412/amlq2014123118.gfso.0104*
>> > > > >>> > >> >
>> > > > >>> > >> > ML, 0104, 2014121318_F000_478N_1780W_FOF,
2014121318, 03,
>> > > GFSO,
>> > > > >>> 000,
>> > > > >>> > >> 478N,
>> > > > >>> > >> > 1780W, 38, 986, XX, 34, NEQ, 0283, 0640, 0645,
0000,
>> > 990,
>> > > > >>> 133,
>> > > > >>> > -99,
>> > > > >>> > >> > 69, 227, -99, -9999, -9999, 208, 315, 135,
156, 0063
>> > > > >>> > >> >
>> > > > >>> > >> > This differs from the structure of both the ADECK
and
>> BDECK
>> > > > input
>> > > > >>> > files:
>> > > > >>> > >> >
>> > > > >>> > >> > *BDECK:
>> > > > >>> > >> >
>> > > > >>>
>> >
/gpfs/hps3/emc/hwrf/noscrub/emc.hurpara/trak/abdeck/btk/bal132019.dat
>> > > > >>> > *
>> > > > >>> > >> >
>> > > > >>> > >> > AL, 13, 2019092218, , BEST, 0, 109N, 188W,
25,
>> 1008,
>> > LO,
>> > > > >>> 0,
>> > > > >>> > ,
>> > > > >>> > >> > 0, 0, 0, 0, 1012, 100, 90, 40, 0,
L,
>> 0,
>> > > ,
>> > > > >>> 0,
>> > > > >>> > >> 0,
>> > > > >>> > >> > INVEST, S, 0, , 0, 0, 0, 0,
>> genesis-num,
>> > > 027,
>> > > > >>> > >> >
>> > > > >>> > >> > *ADECK:
>> > > > >>> > >> >
>> > > > >>> > >> >
>> > > > >>> > >>
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400*
>> > > > >>> > >> >
>> > > > >>> > >> > AL, 0912, 03, PRNA, 000, 157N, 655W, 27, 1008,
XX, 34,
>> > NEQ,
>> > > > >>> 0000,
>> > > > >>> > >> 0000,
>> > > > >>> > >> > 0000, 0000, -9999, -9999, 85
>> > > > >>> > >> >
>> > > > >>> > >> > My suspicion is that these formats are not
compatible.
>> > There
>> > > > are
>> > > > >>> > major
>> > > > >>> > >> > differences between these formats in my opinion.
The
>> ADECK
>> > > was
>> > > > >>> > >> generated
>> > > > >>> > >> > automatically when making a run while BDECK is
done by
>> > someone
>> > > > >>> else.
>> > > > >>> > >> Those
>> > > > >>> > >> > files are stored whenever a storm is identified.
>> > > > >>> > >> >
>> > > > >>> > >> > I think what I'll need to do next is start by
running
>> TCRMW
>> > to
>> > > > >>> produce
>> > > > >>> > >> the
>> > > > >>> > >> > ADECK files formatted by metplus. I have the grib
files
>> for
>> > > > that.
>> > > > >>> > The
>> > > > >>> > >> > question next, though, is the BDECK files okay for
>> > processing?
>> > > > >>> > >> >
>> > > > >>> > >> >
>> > > > >>> > >> >
>> > > > >>> > >> > --
>> > > > >>> > >> > Edward Strobach
>> > > > >>> > >> > EMC/NCEP/NWS/
>> > > > >>> > >> > IMSG Contractor
>> > > > >>> > >> > Cubicle#: 2029
>> > > > >>> > >> > 301-683-3717
>> > > > >>> > >> >
>> > > > >>> > >> >
>> > > > >>> > >>
>> > > > >>> > >> --
>> > > > >>> > >> George McCabe - Software Engineer III
>> > > > >>> > >> National Center for Atmospheric Research
>> > > > >>> > >> Research Applications Laboratory
>> > > > >>> > >> 303-497-2768
>> > > > >>> > >> ---
>> > > > >>> > >> My working day may not be your working day. Please
do not
>> feel
>> > > > >>> obliged
>> > > > >>> > to
>> > > > >>> > >> reply to this email outside of your normal working
hours.
>> > > > >>> > >>
>> > > > >>> > >>
>> > > > >>> > >
>> > > > >>> > > --
>> > > > >>> > > Edward Strobach
>> > > > >>> > > EMC/NCEP/NWS/
>> > > > >>> > > IMSG Contractor
>> > > > >>> > > Cubicle#: 2029
>> > > > >>> > > 301-683-3717
>> > > > >>> > >
>> > > > >>> >
>> > > > >>> >
>> > > > >>> > --
>> > > > >>> > Edward Strobach
>> > > > >>> > EMC/NCEP/NWS/
>> > > > >>> > IMSG Contractor
>> > > > >>> > Cubicle#: 2029
>> > > > >>> > 301-683-3717
>> > > > >>> >
>> > > > >>> >
>> > > > >>>
>> > > > >>> --
>> > > > >>> George McCabe - Software Engineer III
>> > > > >>> National Center for Atmospheric Research
>> > > > >>> Research Applications Laboratory
>> > > > >>> 303-497-2768
>> > > > >>> ---
>> > > > >>> My working day may not be your working day. Please do not
feel
>> > > obliged
>> > > > to
>> > > > >>> reply to this email outside of your normal working hours.
>> > > > >>>
>> > > > >>>
>> > > > >>
>> > > > >> --
>> > > > >> Edward Strobach
>> > > > >> EMC/NCEP/NWS/
>> > > > >> IMSG Contractor
>> > > > >> Cubicle#: 2029
>> > > > >> 301-683-3717
>> > > > >>
>> > > > >
>> > > > >
>> > > > > --
>> > > > > Edward Strobach
>> > > > > EMC/NCEP/NWS/
>> > > > > IMSG Contractor
>> > > > > Cubicle#: 2029
>> > > > > 301-683-3717
>> > > > >
>> > > >
>> > > >
>> > > > --
>> > > > Edward Strobach
>> > > > EMC/NCEP/NWS/
>> > > > IMSG Contractor
>> > > > Cubicle#: 2029
>> > > > 301-683-3717
>> > > >
>> > > >
>> > >
>> > >
>> >
>> > --
>> > Edward Strobach
>> > EMC/NCEP/NWS/
>> > IMSG Contractor
>> > Cubicle#: 2029
>> > 301-683-3717
>> >
>> >
>>
>>
>
> --
> Edward Strobach
> EMC/NCEP/NWS/
> IMSG Contractor
> Cubicle#: 2029
> 301-683-3717
>
--
Edward Strobach
EMC/NCEP/NWS/
IMSG Contractor
Cubicle#: 2029
301-683-3717
------------------------------------------------
Subject: Tropical Cyclone verification: TCPairs and CyclonePlotter
From: Edward Strobach - NOAA Affiliate
Time: Tue Sep 01 11:56:58 2020
I forgot to add something..
What's interesting is that the tutorial going over PROCESS_LIST =
TCPairs,
CyclonePlotter indicates that the plot below could be generated. I've
used
all the settings in the tutorial and never got that plot. Of course
that
wouldn't be possible based on the ASCII file that is generated in the
CyclonePlotter step. It only contains lat lon info. I'm not sure
what
lead_group = 0 and lead_group = 6 means. It does appear - according
to the
valid time in the text file - that it does go out to 144 hours, but
alternates between lead group 0 and lead group 6.
model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
valid_time: 20190924_000000 lat: 11.4 lon: -25.7 lead_group: 0
first_point:True
model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
valid_time: 20190924_060000 lat: 11.6 lon: -27.4 lead_group: 6
first_point:False
model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
valid_time: 20190924_120000 lat: 11.8 lon: -28.2 lead_group: 0
first_point:False
model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
valid_time: 20190924_180000 lat: 12.7 lon: -28.9 lead_group: 6
first_point:False
model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
valid_time: 20190925_000000 lat: 13.1 lon: -30.6 lead_group: 0
first_point:False
model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
valid_time: 20190925_060000 lat: 13.5 lon: -31.8 lead_group: 6
first_point:False
model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
valid_time: 20190925_120000 lat: 13.6 lon: -33.0 lead_group: 0
first_point:False
model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
valid_time: 20190925_180000 lat: 14.0 lon: -34.0 lead_group: 6
first_point:False
model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
valid_time: 20190926_000000 lat: 14.5 lon: -35.1 lead_group: 0
first_point:False
model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
valid_time: 20190926_060000 lat: 14.8 lon: -36.2 lead_group: 6
first_point:False
model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
valid_time: 20190926_120000 lat: 15.3 lon: -37.0 lead_group: 0
first_point:False
model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
valid_time: 20190926_180000 lat: 16.0 lon: -37.7 lead_group: 6
first_point:False
model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
valid_time: 20190927_000000 lat: 16.9 lon: -38.2 lead_group: 0
first_point:False
model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
valid_time: 20190927_060000 lat: 17.8 lon: -38.7 lead_group: 6
first_point:False
model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
valid_time: 20190927_120000 lat: 18.8 lon: -39.2 lead_group: 0
first_point:False
model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
valid_time: 20190927_180000 lat: 19.8 lon: -39.6 lead_group: 6
first_point:False
model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
valid_time: 20190928_000000 lat: 20.8 lon: -39.8 lead_group: 0
first_point:False
model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
valid_time: 20190928_060000 lat: 21.8 lon: -40.1 lead_group: 6
first_point:False
model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
valid_time: 20190928_120000 lat: 22.8 lon: -40.2 lead_group: 0
first_point:False
model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
valid_time: 20190928_180000 lat: 24.0 lon: -40.4 lead_group: 6
first_point:False
model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
valid_time: 20190929_000000 lat: 25.1 lon: -40.4 lead_group: 0
first_point:False
model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
valid_time: 20190929_060000 lat: 26.4 lon: -40.7 lead_group: 6
first_point:False
model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
valid_time: 20190929_120000 lat: 27.6 lon: -41.2 lead_group: 0
first_point:False
model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
valid_time: 20190929_180000 lat: 28.5 lon: -41.7 lead_group: 6
first_point:False
model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
valid_time: 20190930_000000 lat: 29.3 lon: -42.0 lead_group: 0
first_point:False
On Tue, Sep 1, 2020 at 1:47 PM Edward Strobach - NOAA Affiliate <
edward.strobach at noaa.gov> wrote:
> The good news is I've also got CyclonePlotter to work. However, it
would
> appear that CyclonePlotter is really just meant to show a quick and
dirty
> plot based on the available settings and the quality of the plot -
> attached below.
>
> I noticed that I can't process all forecasts at once, rather I have
to
> loop through each of my forecasts in order to generate plots for
20190924
> through 20190929. Also, I have no real control over grid lines,
tick
> marks, legend to distinguish between best track and model, etc. As
such, I
> may abandon CyclonePlotter.
>
> Something else that's a bit odd is that TCPairs is only grouping the
data
> out to 72 hours instead of 144 hours. There's no apparent setting
to
> extend this to 144 hours according to online sources that I've
consulted.
> Is there a way to add an option such as TC_MAX_LEAD so that met
knows to
> group out to 144 hours?
>
> TC_PAIRS_CONFIG_FILE
> <https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_CONFIG_FILE>
> INIT_HOUR_END
> <https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
INIT_HOUR_END>
> INIT_INCLUDE
> <https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
INIT_INCLUDE>
> INIT_EXCLUDE
> <https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
INIT_EXCLUDE>
> TC_PAIRS_READ_ALL_FILES
> <https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_READ_ALL_FILES>
> TC_PAIRS_MODEL
> <https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_MODEL>
> TC_PAIRS_STORM_ID
> <https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_STORM_ID>
> TC_PAIRS_BASIN
> <https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_BASIN>
> TC_PAIRS_CYCLONE
> <https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_CYCLONE>
> TC_PAIRS_STORM_NAME
> <https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_STORM_NAME>
> TC_PAIRS_DLAND_FILE
> <https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_DLAND_FILE>
> TC_PAIRS_MISSING_VAL_TO_REPLACE
> <https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_MISSING_VAL_TO_REPLACE>
> TC_PAIRS_MISSING_VAL
> <https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_MISSING_VAL>
> TC_PAIRS_SKIP_IF_REFORMAT_EXISTS
> <https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_SKIP_IF_REFORMAT_EXISTS>
> TC_PAIRS_SKIP_IF_OUTPUT_EXISTS
> <https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_SKIP_IF_OUTPUT_EXISTS>
> TC_PAIRS_REFORMAT_DECK
> <https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_REFORMAT_DECK>
> TC_PAIRS_REFORMAT_TYPE
> <https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_REFORMAT_TYPE>
> TC_PAIRS_CUSTOM_LOOP_LIST
> <https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_CUSTOM_LOOP_LIST>
>
> On Tue, Sep 1, 2020 at 12:43 PM Edward Strobach - NOAA Affiliate <
> edward.strobach at noaa.gov> wrote:
>
>> Thanks. Those examples are helpful and do confirm that the files
created
>> and passed into the TCPairs as ADECK files do follow, more or less,
the
>> format that's shown in the a{basin}{STORMNUM}{YYYY}.dat files. I
also
>> found this
>> <https://dtcenter.org/metplus-practical-session-guide-feb-
2019/session-5-trkintfeature-relative/met-tool-tc-pairs>
>> to be helpful as well. When comparing the values/description in my
file
>>
(/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092900)
>> with the contents in the link, it is clear that the order of my
file obeys
>> the following:
>>
>> Basin, CY, YYYYMMDDHH, TECHNUM/MIN, TECH, TAU, LATN/S, LonE/W,
VMAX,
>> MSLP, TY, RAD, WINDCODE, RAD1, RAD2, RAD3, RAD4.
>>
>> My first line of the referenced file has that in addition to 3
columns at
>> the end - so 20 instead of 17 entries. I assume that the last 3
are
>> ignored.
>>
>> *AL, 13, 2019092900, 03, PRNA, 000, 238N, 448W, 85, 962, XX,
34, NEQ,
>> 0185, 0184, 0110, 0148*, 978, 30, 32
>>
>> *The good thing is that I figured it out. * Since feedback is
desirable,
>> I'm going to go through the process that helped me arrive at the
solution
>> which I detail below.
>>
>>
>> Since there can be more than one storm in the Atlantic at a given
time, I
>> decided to filter by *TC_PAIRS_CYCLONE* and set that to *13* since
I'm
>> interested in processing Lorenzo. I can see that it's defined in
my master
>> config file as well. I did set the log verbosity to 10 to retrieve
>> additional info. There are no error messages; however, it says
"*Matching
>> 0 ADECK tracks to 1 BDECK trac*k". The BDECK track, which I
retrieved
>> from the best track data formatted as bal132019.dat, is being
processed
>> alongside the ADECK files, which are created during the forecast
step.
>>
>> Since I've run 6 forecasts that go out to 144 hours, I also have 6
ADECK
>> files initialized between 20190924 00Z and 20190929 00Z. In the
case of
>> 20190929 00Z, for example, the log file indicates that the BDECK
track is
>> processed followed by the processing of the ADECK file. The log
correctly
>> identifies 119 lines of processing but says 0 of 119 lines are
used:
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> *DEBUG 5: | | | [3] QuadInfo: Intensity = 64, ALVal =
>> -9999.00000, NEVal = -9999.00000, SEVal = -9999.00000, SWVal =
-9999.00000,
>> NWVal = -9999.00000DEBUG 5:DEBUG 5:DEBUG 5:DEBUG 2: Found 1 BDECK
>> track(s).DEBUG 2: Processing 1 ADECK file(s).DEBUG 4: [File 1 of 1]
Used 0
>> of 119 lines read from file
>>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092900"DEBUG
>> 3: Used 0 of 119 lines read from 1 file(s).DEBUG 3: Identified 0
>> track(s).DEBUG 5: TrackInfoArray: NTracks = 0, NAlloc = 0,
Tracks:DEBUG
>> 5:DEBUG 2: Deriving 12-hour interpolated ADECK tracks.DEBUG 2:
Finished
>> adding 0 and replacing 0 Interp12 track(s).DEBUG 2: Deriving 0
ADECK
>> consensus model(s).DEBUG 2: Added 0 ADECK consensus tracks(s).DEBUG
2:
>> Deriving 0 ADECK lag model(s).DEBUG 2: Added 0 ADECK lag
tracks(s).DEBUG 2:
>> Deriving 0 CLIPER/SHIFOR baseline model(s).DEBUG 2: Added 0
CLIPER/SHIFOR
>> baseline track(s).DEBUG 2: Filtering 0 ADECK tracks based on config
file
>> settings.DEBUG 3: Total tracks read = 0DEBUG 3:
Total tracks
>> kept = 0DEBUG 3: Rejected for storm name =
0DEBUG
>> 3: Rejected for valid time = 0DEBUG 3: Rejected for
required lead
>> times = 0DEBUG 3: Rejected for init mask = 0DEBUG 3:
Rejected for
>> valid mask = 0DEBUG 2: Matching 0 ADECK tracks to 1 BDECK
tracks.*
>>
>> This is repeated for all six forecasts. Also, all lines of the
BDECK
>> file are processed, so the dates in that file encompass the dates
in each
>> of the forecast files.
>>
>> I was flummoxed as to why this would be. Under closer inspection I
found
>> that the directory - which is named after the model - includes the
>> experiment number as well (e.g. prna20). However, inside the ADECK
files
>> the model is PRNA, not PRNA20. Met was looking for instances of
PRNA20,
>> and when it couldn't it concluded that there were zero matches.
However,
>> when I changed my pathway to "TC_PAIRS_ADECK_INPUT_DIR =
>> /gpfs/dell2/emc/modeling/noscrub/${USER}/archive/${model}20" and
set model
>> to "prna" instead of "prna20" it worked. I now have populated
files.
>> Moving forward I will need to keep this subtlety in mind. I plan
to use
>> other TC packages pretty soon here since I've been tasked to
examine some
>> physics changes and tune where necessary. I don't know what type
of issues
>> I may encounter, but I plan to use this experience as a model to
approach
>> possible issues related to TC verification in the future.
>>
>>
>>
>> On Mon, Aug 31, 2020 at 3:31 PM John Halley Gotway via RT <
>> met_help at ucar.edu> wrote:
>>
>>> Ed,
>>>
>>> Here's a link to the documentation that the Navy provides about
the ATCF
>>> file format:
>>>
https://www.nrlmry.navy.mil/atcf_web/docs/database/new/abrdeck.html
>>>
>>> And the best source of sample data is NHC's ftp site:
>>> https://ftp.nhc.noaa.gov/atcf/
>>>
>>> In there...
>>> *aid_public* contains tar files of ADECKS for all storms in the
current
>>> season.
>>> *btk* has uncompressed BEST track files for all storms in the
current
>>> season.
>>> *archive* contains compressed A and B-Deck data for previous
seasons.
>>>
>>> It's pretty cool actually that this track data is so readily
available.
>>>
>>> I have never actually modified ATCF files for my purposes. So
hopefully
>>> you
>>> won't need to either... and can just use the output of the tracker
code
>>> directly.
>>>
>>> Thanks,
>>> John
>>>
>>>
>>> On Mon, Aug 31, 2020 at 9:24 AM Edward Strobach - NOAA Affiliate
via RT <
>>> met_help at ucar.edu> wrote:
>>>
>>> >
>>> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458 >
>>> >
>>> > Hi John,
>>> >
>>> > I did try that. I rewrote the ADECK file with the storm id
added at
>>> the
>>> > end. I'm pasting an example below:
>>> > AL, 13, 2019092400, 03, PRNA, 012, 118N, 282W, 50, 993, XX,
34,
>>> NEQ,
>>> > 0168, 0115, 0046, 0134, 1013, 161, 111, AL132019
>>> >
>>> > You can see that the last entry, which was not included in the
original
>>> > file, has the storm id of AL132019. I'm not very familiar with
ADECK
>>> v.
>>> > BDECK files. Do you have an example of ADECK file format along
with
>>> the
>>> > header info? I can try to rewrite the files following the
appropriate
>>> > order/format, and from there pass the new ADECK file into
TCPairs.
>>> >
>>> > On Fri, Aug 28, 2020 at 5:50 PM John Halley Gotway via RT <
>>> > met_help at ucar.edu>
>>> > wrote:
>>> >
>>> > > Hi Ed,
>>> > >
>>> > > You are correct. Generally the STORM_NAME is not present in
the ADECK
>>> > file.
>>> > > It only appears in the BDECK file.
>>> > >
>>> > > Please try configuring it by specifying a STORM_ID. MET
constructs
>>> the
>>> > > storm id on the fly from the input ATCF data... as the 2-digit
basin,
>>> > > followed by the 2 digit storm number, followed by the 4 digit
year
>>> (e.g.
>>> > > AL092011 for the 9 Atlantic Hurricane storm of 2011).
>>> > >
>>> > > So you can specify the storm_id in the config file, but not
the storm
>>> > name.
>>> > >
>>> > > Thanks,
>>> > > John
>>> > >
>>> > > On Fri, Aug 28, 2020 at 3:45 PM Edward Strobach - NOAA
Affiliate via
>>> RT <
>>> > > met_help at ucar.edu> wrote:
>>> > >
>>> > > >
>>> > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458 >
>>> > > >
>>> > > > It appears that even after adding storm_id/storm_name, that
met is
>>> not
>>> > > able
>>> > > > to identify a track in the file provided. I believe it's
because
>>> > neither
>>> > > > the storm name or storm ID is written in the ADECK file! I
may
>>> have to
>>> > > > write a script that parses the text file and writes in the
storm
>>> name
>>> > and
>>> > > > id in two separate columns, respectively. Here's a line
example
>>> with
>>> > > what
>>> > > > it currently looks like for one row of the ADECK file:
>>> > > >
>>> > > > AL, 13, 2019092500, 03, PRNA, 000, 133N, 313W, 58, 993,
XX, 50,
>>> > NEQ,
>>> > > > 0054, 0000, 0000, 0052, 1013, 172, 32
>>> > > >
>>> > > > If I add AL132019 and LORENZO as the last two columns of
this row,
>>> then
>>> > > > should that remedy the problem that I have. Do you think
that
>>> would
>>> > > > resolve it?
>>> > > >
>>> > > > On Fri, Aug 28, 2020 at 4:53 PM Edward Strobach - NOAA
Affiliate <
>>> > > > edward.strobach at noaa.gov> wrote:
>>> > > >
>>> > > > > It seems to me, though, that these options are rather
>>> redundant. If
>>> > > one
>>> > > > > supplies a file with the Basin name, storm id, and storm
name,
>>> then
>>> > it
>>> > > > > would seem to me that all one would need to do is supply
the
>>> storm id
>>> > > (or
>>> > > > > name) with everything else being extracted from that using
the
>>> > contents
>>> > > > in
>>> > > > > the file.
>>> > > > >
>>> > > > > On Fri, Aug 28, 2020 at 4:41 PM Edward Strobach - NOAA
Affiliate
>>> <
>>> > > > > edward.strobach at noaa.gov> wrote:
>>> > > > >
>>> > > > >> Thanks for that. I left the TC_PAIRS_CYCLONE,
TC_PAIRS_BASIN,
>>> > > > >> TC_PAIRS_STORM_NAME, TC_PAIRS_STORM_ID alone. I may have
to
>>> > develop a
>>> > > > way
>>> > > > >> to fill this information in without manually specifying
it. If
>>> that
>>> > > > proves
>>> > > > >> futile, then I'll take a look at using the GFDL vortex
tracker.
>>> > > Thanks
>>> > > > >>
>>> > > > >> On Thu, Aug 27, 2020 at 3:37 PM George McCabe via RT <
>>> > > met_help at ucar.edu
>>> > > > >
>>> > > > >> wrote:
>>> > > > >>
>>> > > > >>> Hi Edward,
>>> > > > >>>
>>> > > > >>> From John HG:
>>> > > > >>>
>>> > > > >>>
>>> > > > >>>
>>> > > > >>> *The MET-TC tools read track files in ATCF format. Those
track
>>> > files
>>> > > > can
>>> > > > >>> be
>>> > > > >>> generated from gridded model output by running software.
One
>>> such
>>> > > > tracker
>>> > > > >>> is supported to the community by the DTC and is named
the "GFDL
>>> > > Vortex
>>> > > > >>> Tracker":
>>> http://dtcenter.org/community-code/gfdl-vortex-tracker
>>> > > > >>> <http://dtcenter.org/community-code/gfdl-vortex-
tracker>*
>>> > > > >>> *It is not expressly part of METplus but would be very
useful
>>> for
>>> > > > >>> tracking
>>> > > > >>> storms in GRIB files.*
>>> > > > >>>
>>> > > > >>> My guess as to why none of the tracks are being used has
to do
>>> with
>>> > > the
>>> > > > >>> METplus configurations to filter data. I would check the
>>> values for
>>> > > > >>> TC_PAIRS_CYCLONE, TC_PAIRS_BASIN, TC_PAIRS_STORM_NAME,
>>> > > > TC_PAIRS_STORM_ID,
>>> > > > >>> and MODEL. The values you put in these lists determine
which
>>> storms
>>> > > to
>>> > > > >>> use.
>>> > > > >>> You can leave them blank to use any.
>>> > > > >>>
>>> > > > >>> Also, you could bump up the MET log verbosity and see
more
>>> > > information
>>> > > > >>> about why they are being rejected:
>>> > > > >>>
>>> > > > >>> # to affect all MET tools set:
>>> > > > >>> LOG_MET_VERBOSITY = 10
>>> > > > >>>
>>> > > > >>> # to affect only TCPairs set:
>>> > > > >>> LOG_TC_PAIRS_VERBOSITY = 10
>>> > > > >>>
>>> > > > >>> - George
>>> > > > >>>
>>> > > > >>> On Thu, Aug 27, 2020 at 10:53 AM Edward Strobach - NOAA
>>> Affiliate
>>> > via
>>> > > > RT
>>> > > > >>> <
>>> > > > >>> met_help at ucar.edu> wrote:
>>> > > > >>>
>>> > > > >>> >
>>> > > > >>> > <URL:
>>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458 >
>>> > > > >>> >
>>> > > > >>> > Hi,
>>> > > > >>> >
>>> > > > >>> > I did decide to remove CyclonePlotter for now and use
the
>>> > > > >>> TCPairs_Config
>>> > > > >>> > file along with settings that I set in a
shared_TC.conf that
>>> I
>>> > > store
>>> > > > >>> in a
>>> > > > >>> > temporary directory to process the ADECK and BDECK
data that
>>> I
>>> > > have.
>>> > > > >>> Files
>>> > > > >>> > are generated for all the dates that span INIT_BEG and
>>> INIT_END,
>>> > > > >>> thanks to
>>> > > > >>> > setting loop_by to INIT. However, only the header is
written
>>> out,
>>> > > > and I
>>> > > > >>> > suspect it is do to what the log indicates here:
>>> > > > >>> >
>>> > > > >>> > DEBUG 4: [File 1 of 5] Used 0 of 168 lines read from
file
>>> > > > >>> >
>>> > > > >>> >
>>> > > > >>>
>>> > > >
>>> > >
>>> >
>>>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
>>> > > > >>> > DEBUG 4: [File 2 of 5] Used 0 of 168 lines read from
file
>>> > > > >>> >
>>> > > > >>> >
>>> > > > >>>
>>> > > >
>>> > >
>>> >
>>>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
>>> > > > >>> > DEBUG 4: [File 3 of 5] Used 0 of 168 lines read from
file
>>> > > > >>> >
>>> > > > >>> >
>>> > > > >>>
>>> > > >
>>> > >
>>> >
>>>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
>>> > > > >>> > DEBUG 4: [File 4 of 5] Used 0 of 168 lines read from
file
>>> > > > >>> >
>>> > > > >>> >
>>> > > > >>>
>>> > > >
>>> > >
>>> >
>>>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
>>> > > > >>> > DEBUG 4: [File 5 of 5] Used 0 of 168 lines read from
file
>>> > > > >>> >
>>> > > > >>> >
>>> > > > >>>
>>> > > >
>>> > >
>>> >
>>>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
>>> > > > >>> > DEBUG 3: Used 0 of 840 lines read from 5 file(s).
>>> > > > >>> > DEBUG 3: Identified 0 track(s).
>>> > > > >>> > DEBUG 5: TrackInfoArray: NTracks = 0, NAlloc = 0,
Tracks:
>>> > > > >>> > DEBUG 5:
>>> > > > >>> > DEBUG 2: Deriving 12-hour interpolated ADECK tracks.
>>> > > > >>> > DEBUG 2: Finished adding 0 and replacing 0 Interp12
track(s).
>>> > > > >>> > DEBUG 2: Deriving 0 ADECK consensus model(s).
>>> > > > >>> > DEBUG 2: Added 0 ADECK consensus tracks(s).
>>> > > > >>> > DEBUG 2: Deriving 0 ADECK lag model(s).
>>> > > > >>> > DEBUG 2: Added 0 ADECK lag tracks(s).
>>> > > > >>> > DEBUG 2: Deriving 0 CLIPER/SHIFOR baseline model(s).
>>> > > > >>> > DEBUG 2: Added 0 CLIPER/SHIFOR baseline track(s).
>>> > > > >>> > DEBUG 2: Filtering 0 ADECK tracks based on config file
>>> settings.
>>> > > > >>> > DEBUG 3: Total tracks read = 0
>>> > > > >>> > DEBUG 3: Total tracks kept = 0
>>> > > > >>> > DEBUG 3: Rejected for storm name = 0
>>> > > > >>> > DEBUG 3: Rejected for valid time = 0
>>> > > > >>> > DEBUG 3: Rejected for required lead times = 0
>>> > > > >>> > DEBUG 3: Rejected for init mask = 0
>>> > > > >>> > DEBUG 3: Rejected for valid mask = 0
>>> > > > >>> > DEBUG 2: Matching 0 ADECK tracks to 1 BDECK tracks.
>>> > > > >>> >
>>> > > > >>> > I'm still thinking that the formatting is off in this
file.
>>> I
>>> > > > surmise
>>> > > > >>> that
>>> > > > >>> > it's not able to group the data because the column
>>> information is
>>> > > not
>>> > > > >>> in
>>> > > > >>> > the order that met would want it to be in. These
files do
>>> not
>>> > have
>>> > > > any
>>> > > > >>> > header info, so it has to rely on the correct
ordering, or
>>> so I
>>> > > > >>> suspect.
>>> > > > >>> > My interpretation, however, may be flawed. I see 6
separate
>>> TC
>>> > > based
>>> > > > >>> > metplus routines: TCPairs (I'm working on currently);
>>> > > CyclonePlotter
>>> > > > >>> > (looking to use); TCStat; TCRMW; TCMPRPlotter; and
TCGen.
>>> Much,
>>> > if
>>> > > > not
>>> > > > >>> > almost all of these relies on TCPairs. Is there a
routine
>>> that
>>> > I'm
>>> > > > >>> missing
>>> > > > >>> > in metplus that takes a grib file and constructs ADECK
files?
>>> > > > >>> >
>>> > > > >>> >
>>> > > > >>> >
>>> > > > >>> > On Thu, Aug 27, 2020 at 11:52 AM Edward Strobach -
NOAA
>>> > Affiliate <
>>> > > > >>> > edward.strobach at noaa.gov> wrote:
>>> > > > >>> >
>>> > > > >>> > > Hi George, I figured the BDECK must be compatible,
so no
>>> > worries
>>> > > > >>> there.
>>> > > > >>> > > Thanks for the link and the heads up about
reformatting. I
>>> > will
>>> > > > use
>>> > > > >>> the
>>> > > > >>> > > example to modify what I currently have to see the
outcome.
>>> > > Thanks
>>> > > > >>> > again.
>>> > > > >>> > >
>>> > > > >>> > > On Wed, Aug 26, 2020 at 6:57 PM George McCabe via RT
<
>>> > > > >>> met_help at ucar.edu>
>>> > > > >>> > > wrote:
>>> > > > >>> > >
>>> > > > >>> > >> Hi Edward,
>>> > > > >>> > >>
>>> > > > >>> > >> The BDECK data is compatible to be processed. The
use
>>> case you
>>> > > > used
>>> > > > >>> as
>>> > > > >>> > an
>>> > > > >>> > >> example uses extra tropical cyclone data from SBU
that is
>>> > > > >>> reformatted by
>>> > > > >>> > >> TCPairs wrapper before processing. You will see
your
>>> config
>>> > file
>>> > > > >>> has the
>>> > > > >>> > >> following:
>>> > > > >>> > >>
>>> > > > >>> > >> ## tc-pairs filtering optionsTC_PAIRS_REFORMAT_DECK
=
>>> > > > >>> > >> yesTC_PAIRS_REFORMAT_TYPE = SBU
>>> > > > >>> > >>
>>> > > > >>> > >> Changing TC_PAIRS_REFORMAT_DECK to 'no' or 'false'
will
>>> skip
>>> > the
>>> > > > >>> > >> reformatting step. There is another use case in the
repo
>>> that
>>> > > uses
>>> > > > >>> data
>>> > > > >>> > >> that looks like yours (at least as far as I can
tell).
>>> > > > >>> > >>
>>> > > > >>> > >>
>>> > > > >>> > >>
>>> > > > >>> >
>>> > > > >>>
>>> > > >
>>> > >
>>> >
>>>
https://dtcenter.github.io/METplus/generated/met_tool_wrapper/TCPairs/TCPairs_tropical.html#metplus-
configuration
>>> > > > >>> > >>
>>> > > > >>> > >> This use case uses BDECK data in
met_test/new/hwrf/bdeck.
>>> Here
>>> > > is
>>> > > > >>> part
>>> > > > >>> > of
>>> > > > >>> > >> the file called bal062018.dat:
>>> > > > >>> > >>
>>> > > > >>> > >> AL, 06, 2018082906, , BEST, 0, 129N, 109W,
10,
>>> 1010, DB,
>>> > > > >>> 0, ,
>>> > > > >>> > >> 0, 0, 0, 0, 0, 0, 0, 0, 0,
, 0,
>>> > ,
>>> > > > 0,
>>> > > > >>> > 0,
>>> > > > >>> > >> GENESIS019, , 0, , 0, 0, 0, 0,
>>> genesis-num,
>>> > > 019,
>>> > > > >>> > >>
>>> > > > >>> > >>
>>> > > > >>> > >> Let me know if changing that setting doesn't work
and I
>>> can
>>> > > take a
>>> > > > >>> > closer
>>> > > > >>> > >> look.
>>> > > > >>> > >>
>>> > > > >>> > >> Thanks,
>>> > > > >>> > >> George
>>> > > > >>> > >>
>>> > > > >>> > >> On Wed, Aug 26, 2020 at 1:17 PM Edward Strobach -
NOAA
>>> > Affiliate
>>> > > > >>> via RT
>>> > > > >>> > <
>>> > > > >>> > >> met_help at ucar.edu> wrote:
>>> > > > >>> > >>
>>> > > > >>> > >> >
>>> > > > >>> > >> > Wed Aug 26 13:13:28 2020: Request 96458 was acted
upon.
>>> > > > >>> > >> > Transaction: Ticket created by
edward.strobach at noaa.gov
>>> > > > >>> > >> > Queue: met_help
>>> > > > >>> > >> > Subject: Tropical Cyclone verification:
TCPairs and
>>> > > > >>> > CyclonePlotter
>>> > > > >>> > >> > Owner: Nobody
>>> > > > >>> > >> > Requestors: edward.strobach at noaa.gov
>>> > > > >>> > >> > Status: new
>>> > > > >>> > >> > Ticket <URL:
>>> > > > >>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458
>>> > > > >>> > >
>>> > > > >>> > >> >
>>> > > > >>> > >> >
>>> > > > >>> > >> > Good afternoon,
>>> > > > >>> > >> >
>>> > > > >>> > >> > I found it relatively straightforward to set-up
some
>>> scripts
>>> > > to
>>> > > > >>> > process
>>> > > > >>> > >> > forecast output along with obs by using this
resource:
>>> > > > >>> > >> >
>>> > > > >>> > >> >
>>> > > > >>> > >>
>>> > > > >>> >
>>> > > > >>>
>>> > > >
>>> > >
>>> >
>>>
https://dtcenter.github.io/METplus/generated/model_applications/tc_and_extra_tc/Plotter_fcstGFS_obsGFS_ExtraTC.html#sphx-
glr-generated-model-applications-tc-and-extra-tc-plotter-fcstgfs-
obsgfs-extratc-py
>>> > > > >>> > >> > .
>>> > > > >>> > >> > I have been running the global model which
generates
>>> track
>>> > > data
>>> > > > >>> for
>>> > > > >>> > the
>>> > > > >>> > >> > post processing step following this naming
convention:
>>> > > > >>> > >> > atcfunix.gfs.YYYYMMDDHH. The best track data is
stored
>>> here
>>> > > on
>>> > > > >>> WCOSS:
>>> > > > >>> > >> >
>>> /gpfs/hps3/emc/hwrf/noscrub/emc.hurpara/trak/abdeck/btk/.
>>> > > > >>> > >> >
>>> > > > >>> > >> > I have a python script that determines which file
to use
>>> > based
>>> > > > on
>>> > > > >>> > basin
>>> > > > >>> > >> > information, storm name, and date. That is piped
into
>>> the
>>> > > > >>> > >> > TC_PAIRS_BDECK_TEMPLATE while the forecast file
above is
>>> > piped
>>> > > > >>> into
>>> > > > >>> > the
>>> > > > >>> > >> > TC_PAIRS_ADECK_TEMPLATE this way:
>>> > > > >>> atcfunix.gfs.{date?fmt=%Y%m%d%H}.
>>> > > > >>> > >> >
>>> > > > >>> > >> > Inside my main script
>>> > > > >>> > >> >
>>> > > > >>> > >> >
>>> > > > >>> > >>
>>> > > > >>> >
>>> > > > >>>
>>> > > >
>>> > >
>>> >
>>>
*(/gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/METplus/scripts/run_metplus_TCPlotter.sh*,
>>> > > > >>> > >> > I have three configure files that being processed
like
>>> so:*
>>> > > > >>> > >> > ${MET_PLUS}/ush/master_metplus.py -c
>>> > > > >>> > >> >
${MET_PLUS_CONF}/Plotter_fcstGFS_obsGFS_ExtraTC.conf -c
>>> > > > >>> > >> > ${MET_PLUS_TMP}/shared.conf_TC -c
>>> > > > ${MET_PLUS_CONF}/system_TC.conf*
>>> > > > >>> > >> >
>>> > > > >>> > >> > The system_TC.conf is of little concern because
it is
>>> mostly
>>> > > > >>> unchanged
>>> > > > >>> > >> > except for the OUTPUT_BASE, which is changed
>>> specifically to
>>> > > > >>> where I
>>> > > > >>> > >> plan
>>> > > > >>> > >> > on storing the TC output. The shared_TC.conf, on
the
>>> other
>>> > > > hand,
>>> > > > >>> is
>>> > > > >>> > >> used
>>> > > > >>> > >> > to replace the content inside my
>>> > > > >>> Plotter_fcstGFS_obsGFS_ExtraTC.conf
>>> > > > >>> > >> file.
>>> > > > >>> > >> > In here I define
>>> > > > >>> > >> > the ADECK and BDECK input directories based on
the
>>> where the
>>> > > > >>> template
>>> > > > >>> > >> files
>>> > > > >>> > >> > above are being stored. Since I have several
runs
>>> > > encompassing
>>> > > > >>> the
>>> > > > >>> > >> > tropical cyclone event - 20190924 through
20190929 - I
>>> have
>>> > > > >>> specified
>>> > > > >>> > my
>>> > > > >>> > >> > INIT_BEG as 20190924 and my INIT_END as 20190929.
I
>>> have
>>> > also
>>> > > > >>> set my
>>> > > > >>> > >> > INIT_INCREMENT to 86400 since I'm only processing
the
>>> 00z
>>> > > cycle
>>> > > > >>> for
>>> > > > >>> > >> now. I
>>> > > > >>> > >> > have a couple of lines of code before the
configure file
>>> > that
>>> > > > >>> joins
>>> > > > >>> > >> these
>>> > > > >>> > >> > dates in a comma separated list such that the
>>> > > > >>> > CYCLONE_PLOTTER_INIT_DATE
>>> > > > >>> > >> is
>>> > > > >>> > >> > "20190924,20190925,...,20190929". I'm not sure
if that
>>> is
>>> > > > allowed
>>> > > > >>> > >> though.
>>> > > > >>> > >> >
>>> > > > >>> > >> > When running the script the log that is generated
>>> produces 0
>>> > > > >>> errors.
>>> > > > >>> > >> > However, I find *two issues*: *1) there is only
one
>>> file
>>> > > being
>>> > > > >>> > carried
>>> > > > >>> > >> > over to generate results; that is the 20190924
date -
>>> there
>>> > > > >>> should be
>>> > > > >>> > >> 6*,
>>> > > > >>> > >> > and *2) The files that are created are empty and
thus
>>> the
>>> > plot
>>> > > > >>> that is
>>> > > > >>> > >> > generated is simply a map with no data overlaid.*
>>> > > > >>> > >> >
>>> > > > >>> > >> > In attempting to debug further, I found something
that
>>> looks
>>> > > > >>> important
>>> > > > >>> > >> > shown here:
>>> > > > >>> > >> >
>>> > > > >>> > >> > DEBUG 3: Rejected for init mask = 0
>>> > > > >>> > >> > DEBUG 3: Rejected for valid mask = 0
>>> > > > >>> > >> > *DEBUG 2: Matching 0 ADECK tracks to 0 BDECK
tracks.*
>>> > > > >>> > >> > DEBUG 1: Watch/Warning file:
>>> > > > >>> > >> >
>>> > > > >>> > >> >
>>> > > > >>> > >>
>>> > > > >>> >
>>> > > > >>>
>>> > > >
>>> > >
>>> >
>>>
/gpfs/dell2/emc/verification/noscrub/Julie.Prestopnik/met/9.1_beta2/share/met/tc_data/wwpts_us.txt
>>> > > > >>> > >> > DEBUG 5: TrackPairInfoArray: NPairs = 0, NAlloc =
0,
>>> Pairs:
>>> > > > >>> > >> > DEBUG 5:
>>> > > > >>> > >> > DEBUG 1: Output file:
>>> > > > >>> > >> >
>>> > > > >>> > >> >
>>> > > > >>> > >>
>>> > > > >>> >
>>> > > > >>>
>>> > > >
>>> > >
>>> >
>>>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs/201909/AL_q2019092400.gfs.PRNA20.tcst
>>> > > > >>> > >> > 08/26 18:35:12.422 metplus.TCPairs
>>> (command_runner.py:191)
>>> > > > DEBUG:
>>> > > > >>> > >> Finished
>>> > > > >>> > >> > running
>>> > > > >>> > >> >
>>> > > > >>> > >> >
>>> > > > >>> > >>
>>> > > > >>> >
>>> > > > >>>
>>> > > >
>>> > >
>>> >
>>>
/gpfs/dell2/emc/verification/noscrub/Julie.Prestopnik/met/9.1_beta2/bin/tc_pairs
>>> > > > >>> > >> > in 0:00:00.607349
>>> > > > >>> > >> > 08/26 18:35:12.422 metplus.CyclonePlotter
>>> > > > >>> > >> (cyclone_plotter_wrapper.py:77)
>>> > > > >>> > >> > DEBUG: Begin retrieving data...
>>> > > > >>> > >> > 08/26 18:35:12.422 metplus.CyclonePlotter
>>> > > > >>> > >> (cyclone_plotter_wrapper.py:83)
>>> > > > >>> > >> > DEBUG: Generate plot for all files in the
>>> > > > >>> > >> >
>>> > > > >>> > >> >
>>> > > > >>> > >>
>>> > > > >>> >
>>> > > > >>>
>>> > > >
>>> > >
>>> >
>>>
directory/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs
>>> > > > >>> > >> > 08/26 18:35:12.423 metplus.CyclonePlotter
>>> > > > >>> > >> (cyclone_plotter_wrapper.py:98)
>>> > > > >>> > >> > DEBUG: Parsing file
>>> > > > >>> > >> >
>>> > > > >>> > >> >
>>> > > > >>> > >>
>>> > > > >>> >
>>> > > > >>>
>>> > > >
>>> > >
>>> >
>>>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs/201909/AL_q2019092400.gfs.PRNA20.tcst
>>> > > > >>> > >> > 08/26 18:35:19.778 metplus
(config_launcher.py:520)
>>> DEBUG:
>>> > > > Setting
>>> > > > >>> > >> [config]
>>> > > > >>> > >> > SCRUB_STAGING_DIR to default value: False.
>>> > > > >>> > >> > 08/26 18:35:19.779 metplus INFO:
>>> > > > >>> > >> >
>>> > > > >>> > >> >
>>> > > > >>> > >>
>>> > > > >>> >
>>> > > > >>>
>>> > > >
>>> > >
>>> >
>>>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/metplus_final.conf:
>>> > > > >>> > >> > write metplus.conf here
>>> > > > >>> > >> > 08/26 18:35:19.783 metplus (met_util.py:213)
DEBUG:
>>> METplus
>>> > > took
>>> > > > >>> > >> > 0:00:08.783240 to run.
>>> > > > >>> > >> > 08/26 18:35:19.783 metplus INFO: METplus has
>>> successfully
>>> > > > finished
>>> > > > >>> > >> running.
>>> > > > >>> > >> >
>>> > > > >>> > >> > Specifically the line that says matching 0 ADECK
tracks
>>> to 0
>>> > > > BDECK
>>> > > > >>> > >> tracks
>>> > > > >>> > >> > lends me to believe that this is the reason why
nothing
>>> is
>>> > > being
>>> > > > >>> added
>>> > > > >>> > >> to
>>> > > > >>> > >> > my TC_PAIRS_OUTPUT_TEMPLATE file, and therefore
no plots
>>> > > > >>> generated on
>>> > > > >>> > a
>>> > > > >>> > >> > map.
>>> > > > >>> > >> >
>>> > > > >>> > >> > What's interesting is the file that is extracted
for
>>> BDECK
>>> > is
>>> > > > for
>>> > > > >>> > >> hurricane
>>> > > > >>> > >> > Lorenzo and includes all the dates coinciding
with the
>>> ADECK
>>> > > > file
>>> > > > >>> - a
>>> > > > >>> > >> case
>>> > > > >>> > >> > that is run coinciding with Lorenzo. I've
checked this.
>>> > > > >>> > >> >
>>> > > > >>> > >> > One possible issue is the formatting of these
files. I
>>> > looked
>>> > > > in
>>> > > > >>> the
>>> > > > >>> > >> test
>>> > > > >>> > >> > case that comes with the cloned met stuff. In
there the
>>> > > > >>> header/field
>>> > > > >>> > >> info
>>> > > > >>> > >> > looks something like this:
>>> > > > >>> > >> >
>>> > > > >>> > >> >
>>> > > > >>> > >> >
>>> > > > >>> > >>
>>> > > > >>> >
>>> > > > >>>
>>> > > >
>>> > >
>>> >
>>>
*/gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/METplus/source_data/met_test/new/track_data/201412/amlq2014123118.gfso.0104*
>>> > > > >>> > >> >
>>> > > > >>> > >> > ML, 0104, 2014121318_F000_478N_1780W_FOF,
2014121318,
>>> 03,
>>> > > GFSO,
>>> > > > >>> 000,
>>> > > > >>> > >> 478N,
>>> > > > >>> > >> > 1780W, 38, 986, XX, 34, NEQ, 0283, 0640, 0645,
0000,
>>> > 990,
>>> > > > >>> 133,
>>> > > > >>> > -99,
>>> > > > >>> > >> > 69, 227, -99, -9999, -9999, 208, 315, 135,
156,
>>> 0063
>>> > > > >>> > >> >
>>> > > > >>> > >> > This differs from the structure of both the ADECK
and
>>> BDECK
>>> > > > input
>>> > > > >>> > files:
>>> > > > >>> > >> >
>>> > > > >>> > >> > *BDECK:
>>> > > > >>> > >> >
>>> > > > >>>
>>> >
/gpfs/hps3/emc/hwrf/noscrub/emc.hurpara/trak/abdeck/btk/bal132019.dat
>>> > > > >>> > *
>>> > > > >>> > >> >
>>> > > > >>> > >> > AL, 13, 2019092218, , BEST, 0, 109N, 188W,
25,
>>> 1008,
>>> > LO,
>>> > > > >>> 0,
>>> > > > >>> > ,
>>> > > > >>> > >> > 0, 0, 0, 0, 1012, 100, 90, 40, 0,
L,
>>> 0,
>>> > > ,
>>> > > > >>> 0,
>>> > > > >>> > >> 0,
>>> > > > >>> > >> > INVEST, S, 0, , 0, 0, 0, 0,
>>> genesis-num,
>>> > > 027,
>>> > > > >>> > >> >
>>> > > > >>> > >> > *ADECK:
>>> > > > >>> > >> >
>>> > > > >>> > >> >
>>> > > > >>> > >>
>>> > > > >>> >
>>> > > > >>>
>>> > > >
>>> > >
>>> >
>>>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400*
>>> > > > >>> > >> >
>>> > > > >>> > >> > AL, 0912, 03, PRNA, 000, 157N, 655W, 27, 1008,
XX,
>>> 34,
>>> > NEQ,
>>> > > > >>> 0000,
>>> > > > >>> > >> 0000,
>>> > > > >>> > >> > 0000, 0000, -9999, -9999, 85
>>> > > > >>> > >> >
>>> > > > >>> > >> > My suspicion is that these formats are not
compatible.
>>> > There
>>> > > > are
>>> > > > >>> > major
>>> > > > >>> > >> > differences between these formats in my opinion.
The
>>> ADECK
>>> > > was
>>> > > > >>> > >> generated
>>> > > > >>> > >> > automatically when making a run while BDECK is
done by
>>> > someone
>>> > > > >>> else.
>>> > > > >>> > >> Those
>>> > > > >>> > >> > files are stored whenever a storm is identified.
>>> > > > >>> > >> >
>>> > > > >>> > >> > I think what I'll need to do next is start by
running
>>> TCRMW
>>> > to
>>> > > > >>> produce
>>> > > > >>> > >> the
>>> > > > >>> > >> > ADECK files formatted by metplus. I have the
grib
>>> files for
>>> > > > that.
>>> > > > >>> > The
>>> > > > >>> > >> > question next, though, is the BDECK files okay
for
>>> > processing?
>>> > > > >>> > >> >
>>> > > > >>> > >> >
>>> > > > >>> > >> >
>>> > > > >>> > >> > --
>>> > > > >>> > >> > Edward Strobach
>>> > > > >>> > >> > EMC/NCEP/NWS/
>>> > > > >>> > >> > IMSG Contractor
>>> > > > >>> > >> > Cubicle#: 2029
>>> > > > >>> > >> > 301-683-3717
>>> > > > >>> > >> >
>>> > > > >>> > >> >
>>> > > > >>> > >>
>>> > > > >>> > >> --
>>> > > > >>> > >> George McCabe - Software Engineer III
>>> > > > >>> > >> National Center for Atmospheric Research
>>> > > > >>> > >> Research Applications Laboratory
>>> > > > >>> > >> 303-497-2768
>>> > > > >>> > >> ---
>>> > > > >>> > >> My working day may not be your working day. Please
do not
>>> feel
>>> > > > >>> obliged
>>> > > > >>> > to
>>> > > > >>> > >> reply to this email outside of your normal working
hours.
>>> > > > >>> > >>
>>> > > > >>> > >>
>>> > > > >>> > >
>>> > > > >>> > > --
>>> > > > >>> > > Edward Strobach
>>> > > > >>> > > EMC/NCEP/NWS/
>>> > > > >>> > > IMSG Contractor
>>> > > > >>> > > Cubicle#: 2029
>>> > > > >>> > > 301-683-3717
>>> > > > >>> > >
>>> > > > >>> >
>>> > > > >>> >
>>> > > > >>> > --
>>> > > > >>> > Edward Strobach
>>> > > > >>> > EMC/NCEP/NWS/
>>> > > > >>> > IMSG Contractor
>>> > > > >>> > Cubicle#: 2029
>>> > > > >>> > 301-683-3717
>>> > > > >>> >
>>> > > > >>> >
>>> > > > >>>
>>> > > > >>> --
>>> > > > >>> George McCabe - Software Engineer III
>>> > > > >>> National Center for Atmospheric Research
>>> > > > >>> Research Applications Laboratory
>>> > > > >>> 303-497-2768
>>> > > > >>> ---
>>> > > > >>> My working day may not be your working day. Please do
not feel
>>> > > obliged
>>> > > > to
>>> > > > >>> reply to this email outside of your normal working
hours.
>>> > > > >>>
>>> > > > >>>
>>> > > > >>
>>> > > > >> --
>>> > > > >> Edward Strobach
>>> > > > >> EMC/NCEP/NWS/
>>> > > > >> IMSG Contractor
>>> > > > >> Cubicle#: 2029
>>> > > > >> 301-683-3717
>>> > > > >>
>>> > > > >
>>> > > > >
>>> > > > > --
>>> > > > > Edward Strobach
>>> > > > > EMC/NCEP/NWS/
>>> > > > > IMSG Contractor
>>> > > > > Cubicle#: 2029
>>> > > > > 301-683-3717
>>> > > > >
>>> > > >
>>> > > >
>>> > > > --
>>> > > > Edward Strobach
>>> > > > EMC/NCEP/NWS/
>>> > > > IMSG Contractor
>>> > > > Cubicle#: 2029
>>> > > > 301-683-3717
>>> > > >
>>> > > >
>>> > >
>>> > >
>>> >
>>> > --
>>> > Edward Strobach
>>> > EMC/NCEP/NWS/
>>> > IMSG Contractor
>>> > Cubicle#: 2029
>>> > 301-683-3717
>>> >
>>> >
>>>
>>>
>>
>> --
>> Edward Strobach
>> EMC/NCEP/NWS/
>> IMSG Contractor
>> Cubicle#: 2029
>> 301-683-3717
>>
>
>
> --
> Edward Strobach
> EMC/NCEP/NWS/
> IMSG Contractor
> Cubicle#: 2029
> 301-683-3717
>
--
Edward Strobach
EMC/NCEP/NWS/
IMSG Contractor
Cubicle#: 2029
301-683-3717
------------------------------------------------
Subject: Tropical Cyclone verification: TCPairs and CyclonePlotter
From: Minna Win
Time: Tue Sep 01 12:04:49 2020
Hi Ed,
CyclonePlotter was created to replicate the plots generated by one of
our
NOAA partners and not designed to be configurable.
Regards,
Minna
---------------
Minna Win
National Center for Atmospheric Research
Developmental Testbed Center
Phone: 303-497-8423
Fax: 303-497-8401
On Tue, Sep 1, 2020 at 11:57 AM Edward Strobach - NOAA Affiliate via
RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458 >
>
> I forgot to add something..
>
> What's interesting is that the tutorial going over PROCESS_LIST =
TCPairs,
> CyclonePlotter indicates that the plot below could be generated.
I've used
> all the settings in the tutorial and never got that plot. Of
course that
> wouldn't be possible based on the ASCII file that is generated in
the
> CyclonePlotter step. It only contains lat lon info. I'm not sure
what
> lead_group = 0 and lead_group = 6 means. It does appear - according
to the
> valid time in the text file - that it does go out to 144 hours, but
> alternates between lead group 0 and lead group 6.
>
> model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
> valid_time: 20190924_000000 lat: 11.4 lon: -25.7 lead_group: 0
> first_point:True
> model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
> valid_time: 20190924_060000 lat: 11.6 lon: -27.4 lead_group: 6
> first_point:False
> model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
> valid_time: 20190924_120000 lat: 11.8 lon: -28.2 lead_group: 0
> first_point:False
> model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
> valid_time: 20190924_180000 lat: 12.7 lon: -28.9 lead_group: 6
> first_point:False
> model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
> valid_time: 20190925_000000 lat: 13.1 lon: -30.6 lead_group: 0
> first_point:False
> model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
> valid_time: 20190925_060000 lat: 13.5 lon: -31.8 lead_group: 6
> first_point:False
> model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
> valid_time: 20190925_120000 lat: 13.6 lon: -33.0 lead_group: 0
> first_point:False
> model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
> valid_time: 20190925_180000 lat: 14.0 lon: -34.0 lead_group: 6
> first_point:False
> model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
> valid_time: 20190926_000000 lat: 14.5 lon: -35.1 lead_group: 0
> first_point:False
> model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
> valid_time: 20190926_060000 lat: 14.8 lon: -36.2 lead_group: 6
> first_point:False
> model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
> valid_time: 20190926_120000 lat: 15.3 lon: -37.0 lead_group: 0
> first_point:False
> model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
> valid_time: 20190926_180000 lat: 16.0 lon: -37.7 lead_group: 6
> first_point:False
> model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
> valid_time: 20190927_000000 lat: 16.9 lon: -38.2 lead_group: 0
> first_point:False
> model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
> valid_time: 20190927_060000 lat: 17.8 lon: -38.7 lead_group: 6
> first_point:False
> model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
> valid_time: 20190927_120000 lat: 18.8 lon: -39.2 lead_group: 0
> first_point:False
> model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
> valid_time: 20190927_180000 lat: 19.8 lon: -39.6 lead_group: 6
> first_point:False
> model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
> valid_time: 20190928_000000 lat: 20.8 lon: -39.8 lead_group: 0
> first_point:False
> model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
> valid_time: 20190928_060000 lat: 21.8 lon: -40.1 lead_group: 6
> first_point:False
> model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
> valid_time: 20190928_120000 lat: 22.8 lon: -40.2 lead_group: 0
> first_point:False
> model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
> valid_time: 20190928_180000 lat: 24.0 lon: -40.4 lead_group: 6
> first_point:False
> model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
> valid_time: 20190929_000000 lat: 25.1 lon: -40.4 lead_group: 0
> first_point:False
> model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
> valid_time: 20190929_060000 lat: 26.4 lon: -40.7 lead_group: 6
> first_point:False
> model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
> valid_time: 20190929_120000 lat: 27.6 lon: -41.2 lead_group: 0
> first_point:False
> model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
> valid_time: 20190929_180000 lat: 28.5 lon: -41.7 lead_group: 6
> first_point:False
> model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
> valid_time: 20190930_000000 lat: 29.3 lon: -42.0 lead_group: 0
> first_point:False
>
> On Tue, Sep 1, 2020 at 1:47 PM Edward Strobach - NOAA Affiliate <
> edward.strobach at noaa.gov> wrote:
>
> > The good news is I've also got CyclonePlotter to work. However,
it
> would
> > appear that CyclonePlotter is really just meant to show a quick
and dirty
> > plot based on the available settings and the quality of the plot
-
> > attached below.
> >
> > I noticed that I can't process all forecasts at once, rather I
have to
> > loop through each of my forecasts in order to generate plots for
20190924
> > through 20190929. Also, I have no real control over grid lines,
tick
> > marks, legend to distinguish between best track and model, etc.
As
> such, I
> > may abandon CyclonePlotter.
> >
> > Something else that's a bit odd is that TCPairs is only grouping
the data
> > out to 72 hours instead of 144 hours. There's no apparent setting
to
> > extend this to 144 hours according to online sources that I've
consulted.
> > Is there a way to add an option such as TC_MAX_LEAD so that met
knows to
> > group out to 144 hours?
> >
> > TC_PAIRS_CONFIG_FILE
> > <
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_CONFIG_FILE
> >
> > INIT_HOUR_END
> > <
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
INIT_HOUR_END
> >
> > INIT_INCLUDE
> > <
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
INIT_INCLUDE
> >
> > INIT_EXCLUDE
> > <
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
INIT_EXCLUDE
> >
> > TC_PAIRS_READ_ALL_FILES
> > <
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_READ_ALL_FILES
> >
> > TC_PAIRS_MODEL
> > <
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_MODEL
> >
> > TC_PAIRS_STORM_ID
> > <
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_STORM_ID
> >
> > TC_PAIRS_BASIN
> > <
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_BASIN
> >
> > TC_PAIRS_CYCLONE
> > <
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_CYCLONE
> >
> > TC_PAIRS_STORM_NAME
> > <
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_STORM_NAME
> >
> > TC_PAIRS_DLAND_FILE
> > <
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_DLAND_FILE
> >
> > TC_PAIRS_MISSING_VAL_TO_REPLACE
> > <
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_MISSING_VAL_TO_REPLACE
> >
> > TC_PAIRS_MISSING_VAL
> > <
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_MISSING_VAL
> >
> > TC_PAIRS_SKIP_IF_REFORMAT_EXISTS
> > <
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_SKIP_IF_REFORMAT_EXISTS
> >
> > TC_PAIRS_SKIP_IF_OUTPUT_EXISTS
> > <
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_SKIP_IF_OUTPUT_EXISTS
> >
> > TC_PAIRS_REFORMAT_DECK
> > <
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_REFORMAT_DECK
> >
> > TC_PAIRS_REFORMAT_TYPE
> > <
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_REFORMAT_TYPE
> >
> > TC_PAIRS_CUSTOM_LOOP_LIST
> > <
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_CUSTOM_LOOP_LIST
> >
> >
> > On Tue, Sep 1, 2020 at 12:43 PM Edward Strobach - NOAA Affiliate <
> > edward.strobach at noaa.gov> wrote:
> >
> >> Thanks. Those examples are helpful and do confirm that the files
created
> >> and passed into the TCPairs as ADECK files do follow, more or
less, the
> >> format that's shown in the a{basin}{STORMNUM}{YYYY}.dat files. I
also
> >> found this
> >> <
> https://dtcenter.org/metplus-practical-session-guide-feb-
2019/session-5-trkintfeature-relative/met-tool-tc-pairs
> >
> >> to be helpful as well. When comparing the values/description in
my file
> >>
>
(/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092900)
> >> with the contents in the link, it is clear that the order of my
file
> obeys
> >> the following:
> >>
> >> Basin, CY, YYYYMMDDHH, TECHNUM/MIN, TECH, TAU, LATN/S, LonE/W,
VMAX,
> >> MSLP, TY, RAD, WINDCODE, RAD1, RAD2, RAD3, RAD4.
> >>
> >> My first line of the referenced file has that in addition to 3
columns
> at
> >> the end - so 20 instead of 17 entries. I assume that the last 3
are
> >> ignored.
> >>
> >> *AL, 13, 2019092900, 03, PRNA, 000, 238N, 448W, 85, 962, XX,
34,
> NEQ,
> >> 0185, 0184, 0110, 0148*, 978, 30, 32
> >>
> >> *The good thing is that I figured it out. * Since feedback is
desirable,
> >> I'm going to go through the process that helped me arrive at the
> solution
> >> which I detail below.
> >>
> >>
> >> Since there can be more than one storm in the Atlantic at a given
time,
> I
> >> decided to filter by *TC_PAIRS_CYCLONE* and set that to *13*
since I'm
> >> interested in processing Lorenzo. I can see that it's defined in
my
> master
> >> config file as well. I did set the log verbosity to 10 to
retrieve
> >> additional info. There are no error messages; however, it says
> "*Matching
> >> 0 ADECK tracks to 1 BDECK trac*k". The BDECK track, which I
retrieved
> >> from the best track data formatted as bal132019.dat, is being
processed
> >> alongside the ADECK files, which are created during the forecast
step.
> >>
> >> Since I've run 6 forecasts that go out to 144 hours, I also have
6 ADECK
> >> files initialized between 20190924 00Z and 20190929 00Z. In the
case
> of
> >> 20190929 00Z, for example, the log file indicates that the BDECK
track
> is
> >> processed followed by the processing of the ADECK file. The log
> correctly
> >> identifies 119 lines of processing but says 0 of 119 lines are
used:
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> *DEBUG 5: | | | [3] QuadInfo: Intensity = 64, ALVal =
> >> -9999.00000, NEVal = -9999.00000, SEVal = -9999.00000, SWVal =
> -9999.00000,
> >> NWVal = -9999.00000DEBUG 5:DEBUG 5:DEBUG 5:DEBUG 2: Found 1 BDECK
> >> track(s).DEBUG 2: Processing 1 ADECK file(s).DEBUG 4: [File 1 of
1]
> Used 0
> >> of 119 lines read from file
> >>
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092900"DEBUG
> >> 3: Used 0 of 119 lines read from 1 file(s).DEBUG 3: Identified 0
> >> track(s).DEBUG 5: TrackInfoArray: NTracks = 0, NAlloc = 0,
Tracks:DEBUG
> >> 5:DEBUG 2: Deriving 12-hour interpolated ADECK tracks.DEBUG 2:
Finished
> >> adding 0 and replacing 0 Interp12 track(s).DEBUG 2: Deriving 0
ADECK
> >> consensus model(s).DEBUG 2: Added 0 ADECK consensus
tracks(s).DEBUG 2:
> >> Deriving 0 ADECK lag model(s).DEBUG 2: Added 0 ADECK lag
> tracks(s).DEBUG 2:
> >> Deriving 0 CLIPER/SHIFOR baseline model(s).DEBUG 2: Added 0
> CLIPER/SHIFOR
> >> baseline track(s).DEBUG 2: Filtering 0 ADECK tracks based on
config file
> >> settings.DEBUG 3: Total tracks read = 0DEBUG 3:
Total
> tracks
> >> kept = 0DEBUG 3: Rejected for storm name
=
> 0DEBUG
> >> 3: Rejected for valid time = 0DEBUG 3: Rejected for
required
> lead
> >> times = 0DEBUG 3: Rejected for init mask = 0DEBUG 3:
Rejected
> for
> >> valid mask = 0DEBUG 2: Matching 0 ADECK tracks to 1
BDECK
> tracks.*
> >>
> >> This is repeated for all six forecasts. Also, all lines of the
BDECK
> >> file are processed, so the dates in that file encompass the dates
in
> each
> >> of the forecast files.
> >>
> >> I was flummoxed as to why this would be. Under closer inspection
I
> found
> >> that the directory - which is named after the model - includes
the
> >> experiment number as well (e.g. prna20). However, inside the
ADECK
> files
> >> the model is PRNA, not PRNA20. Met was looking for instances of
PRNA20,
> >> and when it couldn't it concluded that there were zero matches.
> However,
> >> when I changed my pathway to "TC_PAIRS_ADECK_INPUT_DIR =
> >> /gpfs/dell2/emc/modeling/noscrub/${USER}/archive/${model}20" and
set
> model
> >> to "prna" instead of "prna20" it worked. I now have populated
files.
> >> Moving forward I will need to keep this subtlety in mind. I
plan to
> use
> >> other TC packages pretty soon here since I've been tasked to
examine
> some
> >> physics changes and tune where necessary. I don't know what type
of
> issues
> >> I may encounter, but I plan to use this experience as a model to
> approach
> >> possible issues related to TC verification in the future.
> >>
> >>
> >>
> >> On Mon, Aug 31, 2020 at 3:31 PM John Halley Gotway via RT <
> >> met_help at ucar.edu> wrote:
> >>
> >>> Ed,
> >>>
> >>> Here's a link to the documentation that the Navy provides about
the
> ATCF
> >>> file format:
> >>>
https://www.nrlmry.navy.mil/atcf_web/docs/database/new/abrdeck.html
> >>>
> >>> And the best source of sample data is NHC's ftp site:
> >>> https://ftp.nhc.noaa.gov/atcf/
> >>>
> >>> In there...
> >>> *aid_public* contains tar files of ADECKS for all storms in the
current
> >>> season.
> >>> *btk* has uncompressed BEST track files for all storms in the
current
> >>> season.
> >>> *archive* contains compressed A and B-Deck data for previous
seasons.
> >>>
> >>> It's pretty cool actually that this track data is so readily
available.
> >>>
> >>> I have never actually modified ATCF files for my purposes. So
hopefully
> >>> you
> >>> won't need to either... and can just use the output of the
tracker code
> >>> directly.
> >>>
> >>> Thanks,
> >>> John
> >>>
> >>>
> >>> On Mon, Aug 31, 2020 at 9:24 AM Edward Strobach - NOAA Affiliate
via
> RT <
> >>> met_help at ucar.edu> wrote:
> >>>
> >>> >
> >>> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458
>
> >>> >
> >>> > Hi John,
> >>> >
> >>> > I did try that. I rewrote the ADECK file with the storm id
added at
> >>> the
> >>> > end. I'm pasting an example below:
> >>> > AL, 13, 2019092400, 03, PRNA, 012, 118N, 282W, 50, 993, XX,
34,
> >>> NEQ,
> >>> > 0168, 0115, 0046, 0134, 1013, 161, 111, AL132019
> >>> >
> >>> > You can see that the last entry, which was not included in the
> original
> >>> > file, has the storm id of AL132019. I'm not very familiar
with ADECK
> >>> v.
> >>> > BDECK files. Do you have an example of ADECK file format
along with
> >>> the
> >>> > header info? I can try to rewrite the files following the
> appropriate
> >>> > order/format, and from there pass the new ADECK file into
TCPairs.
> >>> >
> >>> > On Fri, Aug 28, 2020 at 5:50 PM John Halley Gotway via RT <
> >>> > met_help at ucar.edu>
> >>> > wrote:
> >>> >
> >>> > > Hi Ed,
> >>> > >
> >>> > > You are correct. Generally the STORM_NAME is not present in
the
> ADECK
> >>> > file.
> >>> > > It only appears in the BDECK file.
> >>> > >
> >>> > > Please try configuring it by specifying a STORM_ID. MET
constructs
> >>> the
> >>> > > storm id on the fly from the input ATCF data... as the 2-
digit
> basin,
> >>> > > followed by the 2 digit storm number, followed by the 4
digit year
> >>> (e.g.
> >>> > > AL092011 for the 9 Atlantic Hurricane storm of 2011).
> >>> > >
> >>> > > So you can specify the storm_id in the config file, but not
the
> storm
> >>> > name.
> >>> > >
> >>> > > Thanks,
> >>> > > John
> >>> > >
> >>> > > On Fri, Aug 28, 2020 at 3:45 PM Edward Strobach - NOAA
Affiliate
> via
> >>> RT <
> >>> > > met_help at ucar.edu> wrote:
> >>> > >
> >>> > > >
> >>> > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458 >
> >>> > > >
> >>> > > > It appears that even after adding storm_id/storm_name,
that met
> is
> >>> not
> >>> > > able
> >>> > > > to identify a track in the file provided. I believe it's
because
> >>> > neither
> >>> > > > the storm name or storm ID is written in the ADECK file!
I may
> >>> have to
> >>> > > > write a script that parses the text file and writes in the
storm
> >>> name
> >>> > and
> >>> > > > id in two separate columns, respectively. Here's a line
example
> >>> with
> >>> > > what
> >>> > > > it currently looks like for one row of the ADECK file:
> >>> > > >
> >>> > > > AL, 13, 2019092500, 03, PRNA, 000, 133N, 313W, 58, 993,
XX,
> 50,
> >>> > NEQ,
> >>> > > > 0054, 0000, 0000, 0052, 1013, 172, 32
> >>> > > >
> >>> > > > If I add AL132019 and LORENZO as the last two columns of
this
> row,
> >>> then
> >>> > > > should that remedy the problem that I have. Do you think
that
> >>> would
> >>> > > > resolve it?
> >>> > > >
> >>> > > > On Fri, Aug 28, 2020 at 4:53 PM Edward Strobach - NOAA
Affiliate
> <
> >>> > > > edward.strobach at noaa.gov> wrote:
> >>> > > >
> >>> > > > > It seems to me, though, that these options are rather
> >>> redundant. If
> >>> > > one
> >>> > > > > supplies a file with the Basin name, storm id, and storm
name,
> >>> then
> >>> > it
> >>> > > > > would seem to me that all one would need to do is supply
the
> >>> storm id
> >>> > > (or
> >>> > > > > name) with everything else being extracted from that
using the
> >>> > contents
> >>> > > > in
> >>> > > > > the file.
> >>> > > > >
> >>> > > > > On Fri, Aug 28, 2020 at 4:41 PM Edward Strobach - NOAA
> Affiliate
> >>> <
> >>> > > > > edward.strobach at noaa.gov> wrote:
> >>> > > > >
> >>> > > > >> Thanks for that. I left the TC_PAIRS_CYCLONE,
TC_PAIRS_BASIN,
> >>> > > > >> TC_PAIRS_STORM_NAME, TC_PAIRS_STORM_ID alone. I may
have to
> >>> > develop a
> >>> > > > way
> >>> > > > >> to fill this information in without manually specifying
it.
> If
> >>> that
> >>> > > > proves
> >>> > > > >> futile, then I'll take a look at using the GFDL vortex
> tracker.
> >>> > > Thanks
> >>> > > > >>
> >>> > > > >> On Thu, Aug 27, 2020 at 3:37 PM George McCabe via RT <
> >>> > > met_help at ucar.edu
> >>> > > > >
> >>> > > > >> wrote:
> >>> > > > >>
> >>> > > > >>> Hi Edward,
> >>> > > > >>>
> >>> > > > >>> From John HG:
> >>> > > > >>>
> >>> > > > >>>
> >>> > > > >>>
> >>> > > > >>> *The MET-TC tools read track files in ATCF format.
Those
> track
> >>> > files
> >>> > > > can
> >>> > > > >>> be
> >>> > > > >>> generated from gridded model output by running
software. One
> >>> such
> >>> > > > tracker
> >>> > > > >>> is supported to the community by the DTC and is named
the
> "GFDL
> >>> > > Vortex
> >>> > > > >>> Tracker":
> >>> http://dtcenter.org/community-code/gfdl-vortex-tracker
> >>> > > > >>> <http://dtcenter.org/community-code/gfdl-vortex-
tracker>*
> >>> > > > >>> *It is not expressly part of METplus but would be very
useful
> >>> for
> >>> > > > >>> tracking
> >>> > > > >>> storms in GRIB files.*
> >>> > > > >>>
> >>> > > > >>> My guess as to why none of the tracks are being used
has to
> do
> >>> with
> >>> > > the
> >>> > > > >>> METplus configurations to filter data. I would check
the
> >>> values for
> >>> > > > >>> TC_PAIRS_CYCLONE, TC_PAIRS_BASIN, TC_PAIRS_STORM_NAME,
> >>> > > > TC_PAIRS_STORM_ID,
> >>> > > > >>> and MODEL. The values you put in these lists determine
which
> >>> storms
> >>> > > to
> >>> > > > >>> use.
> >>> > > > >>> You can leave them blank to use any.
> >>> > > > >>>
> >>> > > > >>> Also, you could bump up the MET log verbosity and see
more
> >>> > > information
> >>> > > > >>> about why they are being rejected:
> >>> > > > >>>
> >>> > > > >>> # to affect all MET tools set:
> >>> > > > >>> LOG_MET_VERBOSITY = 10
> >>> > > > >>>
> >>> > > > >>> # to affect only TCPairs set:
> >>> > > > >>> LOG_TC_PAIRS_VERBOSITY = 10
> >>> > > > >>>
> >>> > > > >>> - George
> >>> > > > >>>
> >>> > > > >>> On Thu, Aug 27, 2020 at 10:53 AM Edward Strobach -
NOAA
> >>> Affiliate
> >>> > via
> >>> > > > RT
> >>> > > > >>> <
> >>> > > > >>> met_help at ucar.edu> wrote:
> >>> > > > >>>
> >>> > > > >>> >
> >>> > > > >>> > <URL:
> >>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458 >
> >>> > > > >>> >
> >>> > > > >>> > Hi,
> >>> > > > >>> >
> >>> > > > >>> > I did decide to remove CyclonePlotter for now and
use the
> >>> > > > >>> TCPairs_Config
> >>> > > > >>> > file along with settings that I set in a
shared_TC.conf
> that
> >>> I
> >>> > > store
> >>> > > > >>> in a
> >>> > > > >>> > temporary directory to process the ADECK and BDECK
data
> that
> >>> I
> >>> > > have.
> >>> > > > >>> Files
> >>> > > > >>> > are generated for all the dates that span INIT_BEG
and
> >>> INIT_END,
> >>> > > > >>> thanks to
> >>> > > > >>> > setting loop_by to INIT. However, only the header is
> written
> >>> out,
> >>> > > > and I
> >>> > > > >>> > suspect it is do to what the log indicates here:
> >>> > > > >>> >
> >>> > > > >>> > DEBUG 4: [File 1 of 5] Used 0 of 168 lines read from
file
> >>> > > > >>> >
> >>> > > > >>> >
> >>> > > > >>>
> >>> > > >
> >>> > >
> >>> >
> >>>
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
> >>> > > > >>> > DEBUG 4: [File 2 of 5] Used 0 of 168 lines read from
file
> >>> > > > >>> >
> >>> > > > >>> >
> >>> > > > >>>
> >>> > > >
> >>> > >
> >>> >
> >>>
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
> >>> > > > >>> > DEBUG 4: [File 3 of 5] Used 0 of 168 lines read from
file
> >>> > > > >>> >
> >>> > > > >>> >
> >>> > > > >>>
> >>> > > >
> >>> > >
> >>> >
> >>>
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
> >>> > > > >>> > DEBUG 4: [File 4 of 5] Used 0 of 168 lines read from
file
> >>> > > > >>> >
> >>> > > > >>> >
> >>> > > > >>>
> >>> > > >
> >>> > >
> >>> >
> >>>
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
> >>> > > > >>> > DEBUG 4: [File 5 of 5] Used 0 of 168 lines read from
file
> >>> > > > >>> >
> >>> > > > >>> >
> >>> > > > >>>
> >>> > > >
> >>> > >
> >>> >
> >>>
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
> >>> > > > >>> > DEBUG 3: Used 0 of 840 lines read from 5 file(s).
> >>> > > > >>> > DEBUG 3: Identified 0 track(s).
> >>> > > > >>> > DEBUG 5: TrackInfoArray: NTracks = 0, NAlloc = 0,
Tracks:
> >>> > > > >>> > DEBUG 5:
> >>> > > > >>> > DEBUG 2: Deriving 12-hour interpolated ADECK tracks.
> >>> > > > >>> > DEBUG 2: Finished adding 0 and replacing 0 Interp12
> track(s).
> >>> > > > >>> > DEBUG 2: Deriving 0 ADECK consensus model(s).
> >>> > > > >>> > DEBUG 2: Added 0 ADECK consensus tracks(s).
> >>> > > > >>> > DEBUG 2: Deriving 0 ADECK lag model(s).
> >>> > > > >>> > DEBUG 2: Added 0 ADECK lag tracks(s).
> >>> > > > >>> > DEBUG 2: Deriving 0 CLIPER/SHIFOR baseline model(s).
> >>> > > > >>> > DEBUG 2: Added 0 CLIPER/SHIFOR baseline track(s).
> >>> > > > >>> > DEBUG 2: Filtering 0 ADECK tracks based on config
file
> >>> settings.
> >>> > > > >>> > DEBUG 3: Total tracks read = 0
> >>> > > > >>> > DEBUG 3: Total tracks kept = 0
> >>> > > > >>> > DEBUG 3: Rejected for storm name = 0
> >>> > > > >>> > DEBUG 3: Rejected for valid time = 0
> >>> > > > >>> > DEBUG 3: Rejected for required lead times = 0
> >>> > > > >>> > DEBUG 3: Rejected for init mask = 0
> >>> > > > >>> > DEBUG 3: Rejected for valid mask = 0
> >>> > > > >>> > DEBUG 2: Matching 0 ADECK tracks to 1 BDECK tracks.
> >>> > > > >>> >
> >>> > > > >>> > I'm still thinking that the formatting is off in
this file.
> >>> I
> >>> > > > surmise
> >>> > > > >>> that
> >>> > > > >>> > it's not able to group the data because the column
> >>> information is
> >>> > > not
> >>> > > > >>> in
> >>> > > > >>> > the order that met would want it to be in. These
files do
> >>> not
> >>> > have
> >>> > > > any
> >>> > > > >>> > header info, so it has to rely on the correct
ordering, or
> >>> so I
> >>> > > > >>> suspect.
> >>> > > > >>> > My interpretation, however, may be flawed. I see 6
> separate
> >>> TC
> >>> > > based
> >>> > > > >>> > metplus routines: TCPairs (I'm working on
currently);
> >>> > > CyclonePlotter
> >>> > > > >>> > (looking to use); TCStat; TCRMW; TCMPRPlotter; and
TCGen.
> >>> Much,
> >>> > if
> >>> > > > not
> >>> > > > >>> > almost all of these relies on TCPairs. Is there a
routine
> >>> that
> >>> > I'm
> >>> > > > >>> missing
> >>> > > > >>> > in metplus that takes a grib file and constructs
ADECK
> files?
> >>> > > > >>> >
> >>> > > > >>> >
> >>> > > > >>> >
> >>> > > > >>> > On Thu, Aug 27, 2020 at 11:52 AM Edward Strobach -
NOAA
> >>> > Affiliate <
> >>> > > > >>> > edward.strobach at noaa.gov> wrote:
> >>> > > > >>> >
> >>> > > > >>> > > Hi George, I figured the BDECK must be compatible,
so no
> >>> > worries
> >>> > > > >>> there.
> >>> > > > >>> > > Thanks for the link and the heads up about
> reformatting. I
> >>> > will
> >>> > > > use
> >>> > > > >>> the
> >>> > > > >>> > > example to modify what I currently have to see the
> outcome.
> >>> > > Thanks
> >>> > > > >>> > again.
> >>> > > > >>> > >
> >>> > > > >>> > > On Wed, Aug 26, 2020 at 6:57 PM George McCabe via
RT <
> >>> > > > >>> met_help at ucar.edu>
> >>> > > > >>> > > wrote:
> >>> > > > >>> > >
> >>> > > > >>> > >> Hi Edward,
> >>> > > > >>> > >>
> >>> > > > >>> > >> The BDECK data is compatible to be processed. The
use
> >>> case you
> >>> > > > used
> >>> > > > >>> as
> >>> > > > >>> > an
> >>> > > > >>> > >> example uses extra tropical cyclone data from SBU
that
> is
> >>> > > > >>> reformatted by
> >>> > > > >>> > >> TCPairs wrapper before processing. You will see
your
> >>> config
> >>> > file
> >>> > > > >>> has the
> >>> > > > >>> > >> following:
> >>> > > > >>> > >>
> >>> > > > >>> > >> ## tc-pairs filtering
optionsTC_PAIRS_REFORMAT_DECK =
> >>> > > > >>> > >> yesTC_PAIRS_REFORMAT_TYPE = SBU
> >>> > > > >>> > >>
> >>> > > > >>> > >> Changing TC_PAIRS_REFORMAT_DECK to 'no' or
'false' will
> >>> skip
> >>> > the
> >>> > > > >>> > >> reformatting step. There is another use case in
the repo
> >>> that
> >>> > > uses
> >>> > > > >>> data
> >>> > > > >>> > >> that looks like yours (at least as far as I can
tell).
> >>> > > > >>> > >>
> >>> > > > >>> > >>
> >>> > > > >>> > >>
> >>> > > > >>> >
> >>> > > > >>>
> >>> > > >
> >>> > >
> >>> >
> >>>
>
https://dtcenter.github.io/METplus/generated/met_tool_wrapper/TCPairs/TCPairs_tropical.html#metplus-
configuration
> >>> > > > >>> > >>
> >>> > > > >>> > >> This use case uses BDECK data in
> met_test/new/hwrf/bdeck.
> >>> Here
> >>> > > is
> >>> > > > >>> part
> >>> > > > >>> > of
> >>> > > > >>> > >> the file called bal062018.dat:
> >>> > > > >>> > >>
> >>> > > > >>> > >> AL, 06, 2018082906, , BEST, 0, 129N, 109W,
10,
> >>> 1010, DB,
> >>> > > > >>> 0, ,
> >>> > > > >>> > >> 0, 0, 0, 0, 0, 0, 0, 0, 0,
,
> 0,
> >>> > ,
> >>> > > > 0,
> >>> > > > >>> > 0,
> >>> > > > >>> > >> GENESIS019, , 0, , 0, 0, 0, 0,
> >>> genesis-num,
> >>> > > 019,
> >>> > > > >>> > >>
> >>> > > > >>> > >>
> >>> > > > >>> > >> Let me know if changing that setting doesn't work
and I
> >>> can
> >>> > > take a
> >>> > > > >>> > closer
> >>> > > > >>> > >> look.
> >>> > > > >>> > >>
> >>> > > > >>> > >> Thanks,
> >>> > > > >>> > >> George
> >>> > > > >>> > >>
> >>> > > > >>> > >> On Wed, Aug 26, 2020 at 1:17 PM Edward Strobach -
NOAA
> >>> > Affiliate
> >>> > > > >>> via RT
> >>> > > > >>> > <
> >>> > > > >>> > >> met_help at ucar.edu> wrote:
> >>> > > > >>> > >>
> >>> > > > >>> > >> >
> >>> > > > >>> > >> > Wed Aug 26 13:13:28 2020: Request 96458 was
acted
> upon.
> >>> > > > >>> > >> > Transaction: Ticket created by
> edward.strobach at noaa.gov
> >>> > > > >>> > >> > Queue: met_help
> >>> > > > >>> > >> > Subject: Tropical Cyclone verification:
TCPairs
> and
> >>> > > > >>> > CyclonePlotter
> >>> > > > >>> > >> > Owner: Nobody
> >>> > > > >>> > >> > Requestors: edward.strobach at noaa.gov
> >>> > > > >>> > >> > Status: new
> >>> > > > >>> > >> > Ticket <URL:
> >>> > > > >>>
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458
> >>> > > > >>> > >
> >>> > > > >>> > >> >
> >>> > > > >>> > >> >
> >>> > > > >>> > >> > Good afternoon,
> >>> > > > >>> > >> >
> >>> > > > >>> > >> > I found it relatively straightforward to set-up
some
> >>> scripts
> >>> > > to
> >>> > > > >>> > process
> >>> > > > >>> > >> > forecast output along with obs by using this
resource:
> >>> > > > >>> > >> >
> >>> > > > >>> > >> >
> >>> > > > >>> > >>
> >>> > > > >>> >
> >>> > > > >>>
> >>> > > >
> >>> > >
> >>> >
> >>>
>
https://dtcenter.github.io/METplus/generated/model_applications/tc_and_extra_tc/Plotter_fcstGFS_obsGFS_ExtraTC.html#sphx-
glr-generated-model-applications-tc-and-extra-tc-plotter-fcstgfs-
obsgfs-extratc-py
> >>> > > > >>> > >> > .
> >>> > > > >>> > >> > I have been running the global model which
generates
> >>> track
> >>> > > data
> >>> > > > >>> for
> >>> > > > >>> > the
> >>> > > > >>> > >> > post processing step following this naming
convention:
> >>> > > > >>> > >> > atcfunix.gfs.YYYYMMDDHH. The best track data
is
> stored
> >>> here
> >>> > > on
> >>> > > > >>> WCOSS:
> >>> > > > >>> > >> >
> >>> /gpfs/hps3/emc/hwrf/noscrub/emc.hurpara/trak/abdeck/btk/.
> >>> > > > >>> > >> >
> >>> > > > >>> > >> > I have a python script that determines which
file to
> use
> >>> > based
> >>> > > > on
> >>> > > > >>> > basin
> >>> > > > >>> > >> > information, storm name, and date. That is
piped into
> >>> the
> >>> > > > >>> > >> > TC_PAIRS_BDECK_TEMPLATE while the forecast file
above
> is
> >>> > piped
> >>> > > > >>> into
> >>> > > > >>> > the
> >>> > > > >>> > >> > TC_PAIRS_ADECK_TEMPLATE this way:
> >>> > > > >>> atcfunix.gfs.{date?fmt=%Y%m%d%H}.
> >>> > > > >>> > >> >
> >>> > > > >>> > >> > Inside my main script
> >>> > > > >>> > >> >
> >>> > > > >>> > >> >
> >>> > > > >>> > >>
> >>> > > > >>> >
> >>> > > > >>>
> >>> > > >
> >>> > >
> >>> >
> >>>
>
*(/gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/METplus/scripts/run_metplus_TCPlotter.sh*,
> >>> > > > >>> > >> > I have three configure files that being
processed like
> >>> so:*
> >>> > > > >>> > >> > ${MET_PLUS}/ush/master_metplus.py -c
> >>> > > > >>> > >> >
${MET_PLUS_CONF}/Plotter_fcstGFS_obsGFS_ExtraTC.conf
> -c
> >>> > > > >>> > >> > ${MET_PLUS_TMP}/shared.conf_TC -c
> >>> > > > ${MET_PLUS_CONF}/system_TC.conf*
> >>> > > > >>> > >> >
> >>> > > > >>> > >> > The system_TC.conf is of little concern because
it is
> >>> mostly
> >>> > > > >>> unchanged
> >>> > > > >>> > >> > except for the OUTPUT_BASE, which is changed
> >>> specifically to
> >>> > > > >>> where I
> >>> > > > >>> > >> plan
> >>> > > > >>> > >> > on storing the TC output. The shared_TC.conf,
on the
> >>> other
> >>> > > > hand,
> >>> > > > >>> is
> >>> > > > >>> > >> used
> >>> > > > >>> > >> > to replace the content inside my
> >>> > > > >>> Plotter_fcstGFS_obsGFS_ExtraTC.conf
> >>> > > > >>> > >> file.
> >>> > > > >>> > >> > In here I define
> >>> > > > >>> > >> > the ADECK and BDECK input directories based on
the
> >>> where the
> >>> > > > >>> template
> >>> > > > >>> > >> files
> >>> > > > >>> > >> > above are being stored. Since I have several
runs
> >>> > > encompassing
> >>> > > > >>> the
> >>> > > > >>> > >> > tropical cyclone event - 20190924 through
20190929 - I
> >>> have
> >>> > > > >>> specified
> >>> > > > >>> > my
> >>> > > > >>> > >> > INIT_BEG as 20190924 and my INIT_END as
20190929. I
> >>> have
> >>> > also
> >>> > > > >>> set my
> >>> > > > >>> > >> > INIT_INCREMENT to 86400 since I'm only
processing the
> >>> 00z
> >>> > > cycle
> >>> > > > >>> for
> >>> > > > >>> > >> now. I
> >>> > > > >>> > >> > have a couple of lines of code before the
configure
> file
> >>> > that
> >>> > > > >>> joins
> >>> > > > >>> > >> these
> >>> > > > >>> > >> > dates in a comma separated list such that the
> >>> > > > >>> > CYCLONE_PLOTTER_INIT_DATE
> >>> > > > >>> > >> is
> >>> > > > >>> > >> > "20190924,20190925,...,20190929". I'm not sure
if
> that
> >>> is
> >>> > > > allowed
> >>> > > > >>> > >> though.
> >>> > > > >>> > >> >
> >>> > > > >>> > >> > When running the script the log that is
generated
> >>> produces 0
> >>> > > > >>> errors.
> >>> > > > >>> > >> > However, I find *two issues*: *1) there is
only one
> >>> file
> >>> > > being
> >>> > > > >>> > carried
> >>> > > > >>> > >> > over to generate results; that is the 20190924
date -
> >>> there
> >>> > > > >>> should be
> >>> > > > >>> > >> 6*,
> >>> > > > >>> > >> > and *2) The files that are created are empty
and thus
> >>> the
> >>> > plot
> >>> > > > >>> that is
> >>> > > > >>> > >> > generated is simply a map with no data
overlaid.*
> >>> > > > >>> > >> >
> >>> > > > >>> > >> > In attempting to debug further, I found
something that
> >>> looks
> >>> > > > >>> important
> >>> > > > >>> > >> > shown here:
> >>> > > > >>> > >> >
> >>> > > > >>> > >> > DEBUG 3: Rejected for init mask = 0
> >>> > > > >>> > >> > DEBUG 3: Rejected for valid mask = 0
> >>> > > > >>> > >> > *DEBUG 2: Matching 0 ADECK tracks to 0 BDECK
tracks.*
> >>> > > > >>> > >> > DEBUG 1: Watch/Warning file:
> >>> > > > >>> > >> >
> >>> > > > >>> > >> >
> >>> > > > >>> > >>
> >>> > > > >>> >
> >>> > > > >>>
> >>> > > >
> >>> > >
> >>> >
> >>>
>
/gpfs/dell2/emc/verification/noscrub/Julie.Prestopnik/met/9.1_beta2/share/met/tc_data/wwpts_us.txt
> >>> > > > >>> > >> > DEBUG 5: TrackPairInfoArray: NPairs = 0, NAlloc
= 0,
> >>> Pairs:
> >>> > > > >>> > >> > DEBUG 5:
> >>> > > > >>> > >> > DEBUG 1: Output file:
> >>> > > > >>> > >> >
> >>> > > > >>> > >> >
> >>> > > > >>> > >>
> >>> > > > >>> >
> >>> > > > >>>
> >>> > > >
> >>> > >
> >>> >
> >>>
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs/201909/AL_q2019092400.gfs.PRNA20.tcst
> >>> > > > >>> > >> > 08/26 18:35:12.422 metplus.TCPairs
> >>> (command_runner.py:191)
> >>> > > > DEBUG:
> >>> > > > >>> > >> Finished
> >>> > > > >>> > >> > running
> >>> > > > >>> > >> >
> >>> > > > >>> > >> >
> >>> > > > >>> > >>
> >>> > > > >>> >
> >>> > > > >>>
> >>> > > >
> >>> > >
> >>> >
> >>>
>
/gpfs/dell2/emc/verification/noscrub/Julie.Prestopnik/met/9.1_beta2/bin/tc_pairs
> >>> > > > >>> > >> > in 0:00:00.607349
> >>> > > > >>> > >> > 08/26 18:35:12.422 metplus.CyclonePlotter
> >>> > > > >>> > >> (cyclone_plotter_wrapper.py:77)
> >>> > > > >>> > >> > DEBUG: Begin retrieving data...
> >>> > > > >>> > >> > 08/26 18:35:12.422 metplus.CyclonePlotter
> >>> > > > >>> > >> (cyclone_plotter_wrapper.py:83)
> >>> > > > >>> > >> > DEBUG: Generate plot for all files in the
> >>> > > > >>> > >> >
> >>> > > > >>> > >> >
> >>> > > > >>> > >>
> >>> > > > >>> >
> >>> > > > >>>
> >>> > > >
> >>> > >
> >>> >
> >>>
>
directory/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs
> >>> > > > >>> > >> > 08/26 18:35:12.423 metplus.CyclonePlotter
> >>> > > > >>> > >> (cyclone_plotter_wrapper.py:98)
> >>> > > > >>> > >> > DEBUG: Parsing file
> >>> > > > >>> > >> >
> >>> > > > >>> > >> >
> >>> > > > >>> > >>
> >>> > > > >>> >
> >>> > > > >>>
> >>> > > >
> >>> > >
> >>> >
> >>>
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs/201909/AL_q2019092400.gfs.PRNA20.tcst
> >>> > > > >>> > >> > 08/26 18:35:19.778 metplus
(config_launcher.py:520)
> >>> DEBUG:
> >>> > > > Setting
> >>> > > > >>> > >> [config]
> >>> > > > >>> > >> > SCRUB_STAGING_DIR to default value: False.
> >>> > > > >>> > >> > 08/26 18:35:19.779 metplus INFO:
> >>> > > > >>> > >> >
> >>> > > > >>> > >> >
> >>> > > > >>> > >>
> >>> > > > >>> >
> >>> > > > >>>
> >>> > > >
> >>> > >
> >>> >
> >>>
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/metplus_final.conf:
> >>> > > > >>> > >> > write metplus.conf here
> >>> > > > >>> > >> > 08/26 18:35:19.783 metplus (met_util.py:213)
DEBUG:
> >>> METplus
> >>> > > took
> >>> > > > >>> > >> > 0:00:08.783240 to run.
> >>> > > > >>> > >> > 08/26 18:35:19.783 metplus INFO: METplus has
> >>> successfully
> >>> > > > finished
> >>> > > > >>> > >> running.
> >>> > > > >>> > >> >
> >>> > > > >>> > >> > Specifically the line that says matching 0
ADECK
> tracks
> >>> to 0
> >>> > > > BDECK
> >>> > > > >>> > >> tracks
> >>> > > > >>> > >> > lends me to believe that this is the reason why
> nothing
> >>> is
> >>> > > being
> >>> > > > >>> added
> >>> > > > >>> > >> to
> >>> > > > >>> > >> > my TC_PAIRS_OUTPUT_TEMPLATE file, and therefore
no
> plots
> >>> > > > >>> generated on
> >>> > > > >>> > a
> >>> > > > >>> > >> > map.
> >>> > > > >>> > >> >
> >>> > > > >>> > >> > What's interesting is the file that is
extracted for
> >>> BDECK
> >>> > is
> >>> > > > for
> >>> > > > >>> > >> hurricane
> >>> > > > >>> > >> > Lorenzo and includes all the dates coinciding
with the
> >>> ADECK
> >>> > > > file
> >>> > > > >>> - a
> >>> > > > >>> > >> case
> >>> > > > >>> > >> > that is run coinciding with Lorenzo. I've
checked
> this.
> >>> > > > >>> > >> >
> >>> > > > >>> > >> > One possible issue is the formatting of these
files.
> I
> >>> > looked
> >>> > > > in
> >>> > > > >>> the
> >>> > > > >>> > >> test
> >>> > > > >>> > >> > case that comes with the cloned met stuff. In
there
> the
> >>> > > > >>> header/field
> >>> > > > >>> > >> info
> >>> > > > >>> > >> > looks something like this:
> >>> > > > >>> > >> >
> >>> > > > >>> > >> >
> >>> > > > >>> > >> >
> >>> > > > >>> > >>
> >>> > > > >>> >
> >>> > > > >>>
> >>> > > >
> >>> > >
> >>> >
> >>>
>
*/gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/METplus/source_data/met_test/new/track_data/201412/amlq2014123118.gfso.0104*
> >>> > > > >>> > >> >
> >>> > > > >>> > >> > ML, 0104, 2014121318_F000_478N_1780W_FOF,
2014121318,
> >>> 03,
> >>> > > GFSO,
> >>> > > > >>> 000,
> >>> > > > >>> > >> 478N,
> >>> > > > >>> > >> > 1780W, 38, 986, XX, 34, NEQ, 0283, 0640,
0645,
> 0000,
> >>> > 990,
> >>> > > > >>> 133,
> >>> > > > >>> > -99,
> >>> > > > >>> > >> > 69, 227, -99, -9999, -9999, 208, 315, 135,
156,
> >>> 0063
> >>> > > > >>> > >> >
> >>> > > > >>> > >> > This differs from the structure of both the
ADECK and
> >>> BDECK
> >>> > > > input
> >>> > > > >>> > files:
> >>> > > > >>> > >> >
> >>> > > > >>> > >> > *BDECK:
> >>> > > > >>> > >> >
> >>> > > > >>>
> >>> >
/gpfs/hps3/emc/hwrf/noscrub/emc.hurpara/trak/abdeck/btk/bal132019.dat
> >>> > > > >>> > *
> >>> > > > >>> > >> >
> >>> > > > >>> > >> > AL, 13, 2019092218, , BEST, 0, 109N, 188W,
25,
> >>> 1008,
> >>> > LO,
> >>> > > > >>> 0,
> >>> > > > >>> > ,
> >>> > > > >>> > >> > 0, 0, 0, 0, 1012, 100, 90, 40,
0, L,
> >>> 0,
> >>> > > ,
> >>> > > > >>> 0,
> >>> > > > >>> > >> 0,
> >>> > > > >>> > >> > INVEST, S, 0, , 0, 0, 0, 0,
> >>> genesis-num,
> >>> > > 027,
> >>> > > > >>> > >> >
> >>> > > > >>> > >> > *ADECK:
> >>> > > > >>> > >> >
> >>> > > > >>> > >> >
> >>> > > > >>> > >>
> >>> > > > >>> >
> >>> > > > >>>
> >>> > > >
> >>> > >
> >>> >
> >>>
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400*
> >>> > > > >>> > >> >
> >>> > > > >>> > >> > AL, 0912, 03, PRNA, 000, 157N, 655W, 27,
1008, XX,
> >>> 34,
> >>> > NEQ,
> >>> > > > >>> 0000,
> >>> > > > >>> > >> 0000,
> >>> > > > >>> > >> > 0000, 0000, -9999, -9999, 85
> >>> > > > >>> > >> >
> >>> > > > >>> > >> > My suspicion is that these formats are not
compatible.
> >>> > There
> >>> > > > are
> >>> > > > >>> > major
> >>> > > > >>> > >> > differences between these formats in my
opinion. The
> >>> ADECK
> >>> > > was
> >>> > > > >>> > >> generated
> >>> > > > >>> > >> > automatically when making a run while BDECK is
done by
> >>> > someone
> >>> > > > >>> else.
> >>> > > > >>> > >> Those
> >>> > > > >>> > >> > files are stored whenever a storm is
identified.
> >>> > > > >>> > >> >
> >>> > > > >>> > >> > I think what I'll need to do next is start by
running
> >>> TCRMW
> >>> > to
> >>> > > > >>> produce
> >>> > > > >>> > >> the
> >>> > > > >>> > >> > ADECK files formatted by metplus. I have the
grib
> >>> files for
> >>> > > > that.
> >>> > > > >>> > The
> >>> > > > >>> > >> > question next, though, is the BDECK files okay
for
> >>> > processing?
> >>> > > > >>> > >> >
> >>> > > > >>> > >> >
> >>> > > > >>> > >> >
> >>> > > > >>> > >> > --
> >>> > > > >>> > >> > Edward Strobach
> >>> > > > >>> > >> > EMC/NCEP/NWS/
> >>> > > > >>> > >> > IMSG Contractor
> >>> > > > >>> > >> > Cubicle#: 2029
> >>> > > > >>> > >> > 301-683-3717
> >>> > > > >>> > >> >
> >>> > > > >>> > >> >
> >>> > > > >>> > >>
> >>> > > > >>> > >> --
> >>> > > > >>> > >> George McCabe - Software Engineer III
> >>> > > > >>> > >> National Center for Atmospheric Research
> >>> > > > >>> > >> Research Applications Laboratory
> >>> > > > >>> > >> 303-497-2768
> >>> > > > >>> > >> ---
> >>> > > > >>> > >> My working day may not be your working day.
Please do
> not
> >>> feel
> >>> > > > >>> obliged
> >>> > > > >>> > to
> >>> > > > >>> > >> reply to this email outside of your normal
working
> hours.
> >>> > > > >>> > >>
> >>> > > > >>> > >>
> >>> > > > >>> > >
> >>> > > > >>> > > --
> >>> > > > >>> > > Edward Strobach
> >>> > > > >>> > > EMC/NCEP/NWS/
> >>> > > > >>> > > IMSG Contractor
> >>> > > > >>> > > Cubicle#: 2029
> >>> > > > >>> > > 301-683-3717
> >>> > > > >>> > >
> >>> > > > >>> >
> >>> > > > >>> >
> >>> > > > >>> > --
> >>> > > > >>> > Edward Strobach
> >>> > > > >>> > EMC/NCEP/NWS/
> >>> > > > >>> > IMSG Contractor
> >>> > > > >>> > Cubicle#: 2029
> >>> > > > >>> > 301-683-3717
> >>> > > > >>> >
> >>> > > > >>> >
> >>> > > > >>>
> >>> > > > >>> --
> >>> > > > >>> George McCabe - Software Engineer III
> >>> > > > >>> National Center for Atmospheric Research
> >>> > > > >>> Research Applications Laboratory
> >>> > > > >>> 303-497-2768
> >>> > > > >>> ---
> >>> > > > >>> My working day may not be your working day. Please do
not
> feel
> >>> > > obliged
> >>> > > > to
> >>> > > > >>> reply to this email outside of your normal working
hours.
> >>> > > > >>>
> >>> > > > >>>
> >>> > > > >>
> >>> > > > >> --
> >>> > > > >> Edward Strobach
> >>> > > > >> EMC/NCEP/NWS/
> >>> > > > >> IMSG Contractor
> >>> > > > >> Cubicle#: 2029
> >>> > > > >> 301-683-3717
> >>> > > > >>
> >>> > > > >
> >>> > > > >
> >>> > > > > --
> >>> > > > > Edward Strobach
> >>> > > > > EMC/NCEP/NWS/
> >>> > > > > IMSG Contractor
> >>> > > > > Cubicle#: 2029
> >>> > > > > 301-683-3717
> >>> > > > >
> >>> > > >
> >>> > > >
> >>> > > > --
> >>> > > > Edward Strobach
> >>> > > > EMC/NCEP/NWS/
> >>> > > > IMSG Contractor
> >>> > > > Cubicle#: 2029
> >>> > > > 301-683-3717
> >>> > > >
> >>> > > >
> >>> > >
> >>> > >
> >>> >
> >>> > --
> >>> > Edward Strobach
> >>> > EMC/NCEP/NWS/
> >>> > IMSG Contractor
> >>> > Cubicle#: 2029
> >>> > 301-683-3717
> >>> >
> >>> >
> >>>
> >>>
> >>
> >> --
> >> Edward Strobach
> >> EMC/NCEP/NWS/
> >> IMSG Contractor
> >> Cubicle#: 2029
> >> 301-683-3717
> >>
> >
> >
> > --
> > Edward Strobach
> > EMC/NCEP/NWS/
> > IMSG Contractor
> > Cubicle#: 2029
> > 301-683-3717
> >
>
>
> --
> Edward Strobach
> EMC/NCEP/NWS/
> IMSG Contractor
> Cubicle#: 2029
> 301-683-3717
>
>
------------------------------------------------
Subject: Tropical Cyclone verification: TCPairs and CyclonePlotter
From: John Halley Gotway
Time: Tue Sep 01 17:08:50 2020
Ed,
OK, I'm trying to catch up on this message history. So it sounds like
tc_pairs was filtering out your track data based on the model name.
From
tc_pairs perspective, if the model entry is defined in the config
file,
then that filter is applied. If the model entry is left empty, then
all
data is used regardless of model name.
Since you're running this through METplus, I suppose it was METplus
that
was setting the model config option to the string which produced no
pairs.
This question of "Why am I getting no output data?" comes up in
several
contexts. For point-stat, we have users run at verbosity level 3 (-v
3) so
see counts for why obs were or were not used for each vx task. Last
week,
the same question came up for ensemble-stat, but we DO NOT have the
reason
counting logic there. And here the confusion was caused by tc_pairs.
I assume if you saw a log message saying something like:
*Rejected 119 lines based on MODEL name.*
That would have pointed you in the right direction much faster.
So we could enhance...
- Ensemble-Stat to add rejection reason counts when verifying against
point
obs.
- TC-Pairs to add rejection reason counts for which ATCF lines were
used.
- TC-Stat to add rejection reason counts for which TCST lines were
used.
- Stat-Analysis to add rejection reason counts for which STAT lines
were
used.
Do you think any of these potential enhancements are worth doing?
Thanks,
John
On Tue, Sep 1, 2020 at 12:05 PM Minna Win via RT <met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458 >
>
> Hi Ed,
>
> CyclonePlotter was created to replicate the plots generated by one
of our
> NOAA partners and not designed to be configurable.
>
> Regards,
> Minna
> ---------------
> Minna Win
> National Center for Atmospheric Research
> Developmental Testbed Center
> Phone: 303-497-8423
> Fax: 303-497-8401
>
>
>
> On Tue, Sep 1, 2020 at 11:57 AM Edward Strobach - NOAA Affiliate via
RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458 >
> >
> > I forgot to add something..
> >
> > What's interesting is that the tutorial going over PROCESS_LIST =
> TCPairs,
> > CyclonePlotter indicates that the plot below could be generated.
I've
> used
> > all the settings in the tutorial and never got that plot. Of
course
> that
> > wouldn't be possible based on the ASCII file that is generated in
the
> > CyclonePlotter step. It only contains lat lon info. I'm not sure
what
> > lead_group = 0 and lead_group = 6 means. It does appear -
according to
> the
> > valid time in the text file - that it does go out to 144 hours,
but
> > alternates between lead group 0 and lead group 6.
> >
> > model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
> > valid_time: 20190924_000000 lat: 11.4 lon: -25.7 lead_group:
0
> > first_point:True
> > model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
> > valid_time: 20190924_060000 lat: 11.6 lon: -27.4 lead_group:
6
> > first_point:False
> > model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
> > valid_time: 20190924_120000 lat: 11.8 lon: -28.2 lead_group:
0
> > first_point:False
> > model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
> > valid_time: 20190924_180000 lat: 12.7 lon: -28.9 lead_group:
6
> > first_point:False
> > model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
> > valid_time: 20190925_000000 lat: 13.1 lon: -30.6 lead_group:
0
> > first_point:False
> > model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
> > valid_time: 20190925_060000 lat: 13.5 lon: -31.8 lead_group:
6
> > first_point:False
> > model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
> > valid_time: 20190925_120000 lat: 13.6 lon: -33.0 lead_group:
0
> > first_point:False
> > model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
> > valid_time: 20190925_180000 lat: 14.0 lon: -34.0 lead_group:
6
> > first_point:False
> > model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
> > valid_time: 20190926_000000 lat: 14.5 lon: -35.1 lead_group:
0
> > first_point:False
> > model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
> > valid_time: 20190926_060000 lat: 14.8 lon: -36.2 lead_group:
6
> > first_point:False
> > model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
> > valid_time: 20190926_120000 lat: 15.3 lon: -37.0 lead_group:
0
> > first_point:False
> > model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
> > valid_time: 20190926_180000 lat: 16.0 lon: -37.7 lead_group:
6
> > first_point:False
> > model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
> > valid_time: 20190927_000000 lat: 16.9 lon: -38.2 lead_group:
0
> > first_point:False
> > model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
> > valid_time: 20190927_060000 lat: 17.8 lon: -38.7 lead_group:
6
> > first_point:False
> > model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
> > valid_time: 20190927_120000 lat: 18.8 lon: -39.2 lead_group:
0
> > first_point:False
> > model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
> > valid_time: 20190927_180000 lat: 19.8 lon: -39.6 lead_group:
6
> > first_point:False
> > model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
> > valid_time: 20190928_000000 lat: 20.8 lon: -39.8 lead_group:
0
> > first_point:False
> > model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
> > valid_time: 20190928_060000 lat: 21.8 lon: -40.1 lead_group:
6
> > first_point:False
> > model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
> > valid_time: 20190928_120000 lat: 22.8 lon: -40.2 lead_group:
0
> > first_point:False
> > model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
> > valid_time: 20190928_180000 lat: 24.0 lon: -40.4 lead_group:
6
> > first_point:False
> > model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
> > valid_time: 20190929_000000 lat: 25.1 lon: -40.4 lead_group:
0
> > first_point:False
> > model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
> > valid_time: 20190929_060000 lat: 26.4 lon: -40.7 lead_group:
6
> > first_point:False
> > model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
> > valid_time: 20190929_120000 lat: 27.6 lon: -41.2 lead_group:
0
> > first_point:False
> > model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
> > valid_time: 20190929_180000 lat: 28.5 lon: -41.7 lead_group:
6
> > first_point:False
> > model_name: PRNA storm_id: AL132019 init_time: 20190924_000000
> > valid_time: 20190930_000000 lat: 29.3 lon: -42.0 lead_group:
0
> > first_point:False
> >
> > On Tue, Sep 1, 2020 at 1:47 PM Edward Strobach - NOAA Affiliate <
> > edward.strobach at noaa.gov> wrote:
> >
> > > The good news is I've also got CyclonePlotter to work. However,
it
> > would
> > > appear that CyclonePlotter is really just meant to show a quick
and
> dirty
> > > plot based on the available settings and the quality of the
plot -
> > > attached below.
> > >
> > > I noticed that I can't process all forecasts at once, rather I
have to
> > > loop through each of my forecasts in order to generate plots for
> 20190924
> > > through 20190929. Also, I have no real control over grid lines,
tick
> > > marks, legend to distinguish between best track and model, etc.
As
> > such, I
> > > may abandon CyclonePlotter.
> > >
> > > Something else that's a bit odd is that TCPairs is only grouping
the
> data
> > > out to 72 hours instead of 144 hours. There's no apparent
setting to
> > > extend this to 144 hours according to online sources that I've
> consulted.
> > > Is there a way to add an option such as TC_MAX_LEAD so that met
knows
> to
> > > group out to 144 hours?
> > >
> > > TC_PAIRS_CONFIG_FILE
> > > <
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_CONFIG_FILE
> > >
> > > INIT_HOUR_END
> > > <
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
INIT_HOUR_END
> > >
> > > INIT_INCLUDE
> > > <
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
INIT_INCLUDE
> > >
> > > INIT_EXCLUDE
> > > <
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
INIT_EXCLUDE
> > >
> > > TC_PAIRS_READ_ALL_FILES
> > > <
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_READ_ALL_FILES
> > >
> > > TC_PAIRS_MODEL
> > > <
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_MODEL
> > >
> > > TC_PAIRS_STORM_ID
> > > <
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_STORM_ID
> > >
> > > TC_PAIRS_BASIN
> > > <
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_BASIN
> > >
> > > TC_PAIRS_CYCLONE
> > > <
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_CYCLONE
> > >
> > > TC_PAIRS_STORM_NAME
> > > <
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_STORM_NAME
> > >
> > > TC_PAIRS_DLAND_FILE
> > > <
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_DLAND_FILE
> > >
> > > TC_PAIRS_MISSING_VAL_TO_REPLACE
> > > <
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_MISSING_VAL_TO_REPLACE
> > >
> > > TC_PAIRS_MISSING_VAL
> > > <
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_MISSING_VAL
> > >
> > > TC_PAIRS_SKIP_IF_REFORMAT_EXISTS
> > > <
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_SKIP_IF_REFORMAT_EXISTS
> > >
> > > TC_PAIRS_SKIP_IF_OUTPUT_EXISTS
> > > <
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_SKIP_IF_OUTPUT_EXISTS
> > >
> > > TC_PAIRS_REFORMAT_DECK
> > > <
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_REFORMAT_DECK
> > >
> > > TC_PAIRS_REFORMAT_TYPE
> > > <
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_REFORMAT_TYPE
> > >
> > > TC_PAIRS_CUSTOM_LOOP_LIST
> > > <
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_CUSTOM_LOOP_LIST
> > >
> > >
> > > On Tue, Sep 1, 2020 at 12:43 PM Edward Strobach - NOAA Affiliate
<
> > > edward.strobach at noaa.gov> wrote:
> > >
> > >> Thanks. Those examples are helpful and do confirm that the
files
> created
> > >> and passed into the TCPairs as ADECK files do follow, more or
less,
> the
> > >> format that's shown in the a{basin}{STORMNUM}{YYYY}.dat files.
I also
> > >> found this
> > >> <
> >
> https://dtcenter.org/metplus-practical-session-guide-feb-
2019/session-5-trkintfeature-relative/met-tool-tc-pairs
> > >
> > >> to be helpful as well. When comparing the values/description
in my
> file
> > >>
> >
>
(/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092900)
> > >> with the contents in the link, it is clear that the order of my
file
> > obeys
> > >> the following:
> > >>
> > >> Basin, CY, YYYYMMDDHH, TECHNUM/MIN, TECH, TAU, LATN/S, LonE/W,
VMAX,
> > >> MSLP, TY, RAD, WINDCODE, RAD1, RAD2, RAD3, RAD4.
> > >>
> > >> My first line of the referenced file has that in addition to 3
columns
> > at
> > >> the end - so 20 instead of 17 entries. I assume that the last
3 are
> > >> ignored.
> > >>
> > >> *AL, 13, 2019092900, 03, PRNA, 000, 238N, 448W, 85, 962, XX,
34,
> > NEQ,
> > >> 0185, 0184, 0110, 0148*, 978, 30, 32
> > >>
> > >> *The good thing is that I figured it out. * Since feedback is
> desirable,
> > >> I'm going to go through the process that helped me arrive at
the
> > solution
> > >> which I detail below.
> > >>
> > >>
> > >> Since there can be more than one storm in the Atlantic at a
given
> time,
> > I
> > >> decided to filter by *TC_PAIRS_CYCLONE* and set that to *13*
since I'm
> > >> interested in processing Lorenzo. I can see that it's defined
in my
> > master
> > >> config file as well. I did set the log verbosity to 10 to
retrieve
> > >> additional info. There are no error messages; however, it says
> > "*Matching
> > >> 0 ADECK tracks to 1 BDECK trac*k". The BDECK track, which I
retrieved
> > >> from the best track data formatted as bal132019.dat, is being
> processed
> > >> alongside the ADECK files, which are created during the
forecast step.
> > >>
> > >> Since I've run 6 forecasts that go out to 144 hours, I also
have 6
> ADECK
> > >> files initialized between 20190924 00Z and 20190929 00Z. In
the case
> > of
> > >> 20190929 00Z, for example, the log file indicates that the
BDECK track
> > is
> > >> processed followed by the processing of the ADECK file. The
log
> > correctly
> > >> identifies 119 lines of processing but says 0 of 119 lines are
used:
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >> *DEBUG 5: | | | [3] QuadInfo: Intensity = 64, ALVal =
> > >> -9999.00000, NEVal = -9999.00000, SEVal = -9999.00000, SWVal =
> > -9999.00000,
> > >> NWVal = -9999.00000DEBUG 5:DEBUG 5:DEBUG 5:DEBUG 2: Found 1
BDECK
> > >> track(s).DEBUG 2: Processing 1 ADECK file(s).DEBUG 4: [File 1
of 1]
> > Used 0
> > >> of 119 lines read from file
> > >>
> >
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092900"DEBUG
> > >> 3: Used 0 of 119 lines read from 1 file(s).DEBUG 3: Identified
0
> > >> track(s).DEBUG 5: TrackInfoArray: NTracks = 0, NAlloc = 0,
> Tracks:DEBUG
> > >> 5:DEBUG 2: Deriving 12-hour interpolated ADECK tracks.DEBUG 2:
> Finished
> > >> adding 0 and replacing 0 Interp12 track(s).DEBUG 2: Deriving 0
ADECK
> > >> consensus model(s).DEBUG 2: Added 0 ADECK consensus
tracks(s).DEBUG 2:
> > >> Deriving 0 ADECK lag model(s).DEBUG 2: Added 0 ADECK lag
> > tracks(s).DEBUG 2:
> > >> Deriving 0 CLIPER/SHIFOR baseline model(s).DEBUG 2: Added 0
> > CLIPER/SHIFOR
> > >> baseline track(s).DEBUG 2: Filtering 0 ADECK tracks based on
config
> file
> > >> settings.DEBUG 3: Total tracks read = 0DEBUG 3:
Total
> > tracks
> > >> kept = 0DEBUG 3: Rejected for storm name
=
> > 0DEBUG
> > >> 3: Rejected for valid time = 0DEBUG 3: Rejected for
required
> > lead
> > >> times = 0DEBUG 3: Rejected for init mask = 0DEBUG 3:
> Rejected
> > for
> > >> valid mask = 0DEBUG 2: Matching 0 ADECK tracks to 1
BDECK
> > tracks.*
> > >>
> > >> This is repeated for all six forecasts. Also, all lines of the
BDECK
> > >> file are processed, so the dates in that file encompass the
dates in
> > each
> > >> of the forecast files.
> > >>
> > >> I was flummoxed as to why this would be. Under closer
inspection I
> > found
> > >> that the directory - which is named after the model - includes
the
> > >> experiment number as well (e.g. prna20). However, inside the
ADECK
> > files
> > >> the model is PRNA, not PRNA20. Met was looking for instances
of
> PRNA20,
> > >> and when it couldn't it concluded that there were zero matches.
> > However,
> > >> when I changed my pathway to "TC_PAIRS_ADECK_INPUT_DIR =
> > >> /gpfs/dell2/emc/modeling/noscrub/${USER}/archive/${model}20"
and set
> > model
> > >> to "prna" instead of "prna20" it worked. I now have populated
files.
> > >> Moving forward I will need to keep this subtlety in mind. I
plan to
> > use
> > >> other TC packages pretty soon here since I've been tasked to
examine
> > some
> > >> physics changes and tune where necessary. I don't know what
type of
> > issues
> > >> I may encounter, but I plan to use this experience as a model
to
> > approach
> > >> possible issues related to TC verification in the future.
> > >>
> > >>
> > >>
> > >> On Mon, Aug 31, 2020 at 3:31 PM John Halley Gotway via RT <
> > >> met_help at ucar.edu> wrote:
> > >>
> > >>> Ed,
> > >>>
> > >>> Here's a link to the documentation that the Navy provides
about the
> > ATCF
> > >>> file format:
> > >>>
https://www.nrlmry.navy.mil/atcf_web/docs/database/new/abrdeck.html
> > >>>
> > >>> And the best source of sample data is NHC's ftp site:
> > >>> https://ftp.nhc.noaa.gov/atcf/
> > >>>
> > >>> In there...
> > >>> *aid_public* contains tar files of ADECKS for all storms in
the
> current
> > >>> season.
> > >>> *btk* has uncompressed BEST track files for all storms in the
current
> > >>> season.
> > >>> *archive* contains compressed A and B-Deck data for previous
seasons.
> > >>>
> > >>> It's pretty cool actually that this track data is so readily
> available.
> > >>>
> > >>> I have never actually modified ATCF files for my purposes. So
> hopefully
> > >>> you
> > >>> won't need to either... and can just use the output of the
tracker
> code
> > >>> directly.
> > >>>
> > >>> Thanks,
> > >>> John
> > >>>
> > >>>
> > >>> On Mon, Aug 31, 2020 at 9:24 AM Edward Strobach - NOAA
Affiliate via
> > RT <
> > >>> met_help at ucar.edu> wrote:
> > >>>
> > >>> >
> > >>> > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458 >
> > >>> >
> > >>> > Hi John,
> > >>> >
> > >>> > I did try that. I rewrote the ADECK file with the storm id
added
> at
> > >>> the
> > >>> > end. I'm pasting an example below:
> > >>> > AL, 13, 2019092400, 03, PRNA, 012, 118N, 282W, 50, 993,
XX, 34,
> > >>> NEQ,
> > >>> > 0168, 0115, 0046, 0134, 1013, 161, 111, AL132019
> > >>> >
> > >>> > You can see that the last entry, which was not included in
the
> > original
> > >>> > file, has the storm id of AL132019. I'm not very familiar
with
> ADECK
> > >>> v.
> > >>> > BDECK files. Do you have an example of ADECK file format
along
> with
> > >>> the
> > >>> > header info? I can try to rewrite the files following the
> > appropriate
> > >>> > order/format, and from there pass the new ADECK file into
TCPairs.
> > >>> >
> > >>> > On Fri, Aug 28, 2020 at 5:50 PM John Halley Gotway via RT <
> > >>> > met_help at ucar.edu>
> > >>> > wrote:
> > >>> >
> > >>> > > Hi Ed,
> > >>> > >
> > >>> > > You are correct. Generally the STORM_NAME is not present
in the
> > ADECK
> > >>> > file.
> > >>> > > It only appears in the BDECK file.
> > >>> > >
> > >>> > > Please try configuring it by specifying a STORM_ID. MET
> constructs
> > >>> the
> > >>> > > storm id on the fly from the input ATCF data... as the 2-
digit
> > basin,
> > >>> > > followed by the 2 digit storm number, followed by the 4
digit
> year
> > >>> (e.g.
> > >>> > > AL092011 for the 9 Atlantic Hurricane storm of 2011).
> > >>> > >
> > >>> > > So you can specify the storm_id in the config file, but
not the
> > storm
> > >>> > name.
> > >>> > >
> > >>> > > Thanks,
> > >>> > > John
> > >>> > >
> > >>> > > On Fri, Aug 28, 2020 at 3:45 PM Edward Strobach - NOAA
Affiliate
> > via
> > >>> RT <
> > >>> > > met_help at ucar.edu> wrote:
> > >>> > >
> > >>> > > >
> > >>> > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458
> >
> > >>> > > >
> > >>> > > > It appears that even after adding storm_id/storm_name,
that met
> > is
> > >>> not
> > >>> > > able
> > >>> > > > to identify a track in the file provided. I believe
it's
> because
> > >>> > neither
> > >>> > > > the storm name or storm ID is written in the ADECK file!
I may
> > >>> have to
> > >>> > > > write a script that parses the text file and writes in
the
> storm
> > >>> name
> > >>> > and
> > >>> > > > id in two separate columns, respectively. Here's a line
> example
> > >>> with
> > >>> > > what
> > >>> > > > it currently looks like for one row of the ADECK file:
> > >>> > > >
> > >>> > > > AL, 13, 2019092500, 03, PRNA, 000, 133N, 313W, 58,
993, XX,
> > 50,
> > >>> > NEQ,
> > >>> > > > 0054, 0000, 0000, 0052, 1013, 172, 32
> > >>> > > >
> > >>> > > > If I add AL132019 and LORENZO as the last two columns of
this
> > row,
> > >>> then
> > >>> > > > should that remedy the problem that I have. Do you
think that
> > >>> would
> > >>> > > > resolve it?
> > >>> > > >
> > >>> > > > On Fri, Aug 28, 2020 at 4:53 PM Edward Strobach - NOAA
> Affiliate
> > <
> > >>> > > > edward.strobach at noaa.gov> wrote:
> > >>> > > >
> > >>> > > > > It seems to me, though, that these options are rather
> > >>> redundant. If
> > >>> > > one
> > >>> > > > > supplies a file with the Basin name, storm id, and
storm
> name,
> > >>> then
> > >>> > it
> > >>> > > > > would seem to me that all one would need to do is
supply the
> > >>> storm id
> > >>> > > (or
> > >>> > > > > name) with everything else being extracted from that
using
> the
> > >>> > contents
> > >>> > > > in
> > >>> > > > > the file.
> > >>> > > > >
> > >>> > > > > On Fri, Aug 28, 2020 at 4:41 PM Edward Strobach - NOAA
> > Affiliate
> > >>> <
> > >>> > > > > edward.strobach at noaa.gov> wrote:
> > >>> > > > >
> > >>> > > > >> Thanks for that. I left the TC_PAIRS_CYCLONE,
> TC_PAIRS_BASIN,
> > >>> > > > >> TC_PAIRS_STORM_NAME, TC_PAIRS_STORM_ID alone. I may
have to
> > >>> > develop a
> > >>> > > > way
> > >>> > > > >> to fill this information in without manually
specifying it.
> > If
> > >>> that
> > >>> > > > proves
> > >>> > > > >> futile, then I'll take a look at using the GFDL
vortex
> > tracker.
> > >>> > > Thanks
> > >>> > > > >>
> > >>> > > > >> On Thu, Aug 27, 2020 at 3:37 PM George McCabe via RT
<
> > >>> > > met_help at ucar.edu
> > >>> > > > >
> > >>> > > > >> wrote:
> > >>> > > > >>
> > >>> > > > >>> Hi Edward,
> > >>> > > > >>>
> > >>> > > > >>> From John HG:
> > >>> > > > >>>
> > >>> > > > >>>
> > >>> > > > >>>
> > >>> > > > >>> *The MET-TC tools read track files in ATCF format.
Those
> > track
> > >>> > files
> > >>> > > > can
> > >>> > > > >>> be
> > >>> > > > >>> generated from gridded model output by running
software.
> One
> > >>> such
> > >>> > > > tracker
> > >>> > > > >>> is supported to the community by the DTC and is
named the
> > "GFDL
> > >>> > > Vortex
> > >>> > > > >>> Tracker":
> > >>> http://dtcenter.org/community-code/gfdl-vortex-tracker
> > >>> > > > >>> <http://dtcenter.org/community-code/gfdl-vortex-
tracker>*
> > >>> > > > >>> *It is not expressly part of METplus but would be
very
> useful
> > >>> for
> > >>> > > > >>> tracking
> > >>> > > > >>> storms in GRIB files.*
> > >>> > > > >>>
> > >>> > > > >>> My guess as to why none of the tracks are being used
has to
> > do
> > >>> with
> > >>> > > the
> > >>> > > > >>> METplus configurations to filter data. I would check
the
> > >>> values for
> > >>> > > > >>> TC_PAIRS_CYCLONE, TC_PAIRS_BASIN,
TC_PAIRS_STORM_NAME,
> > >>> > > > TC_PAIRS_STORM_ID,
> > >>> > > > >>> and MODEL. The values you put in these lists
determine
> which
> > >>> storms
> > >>> > > to
> > >>> > > > >>> use.
> > >>> > > > >>> You can leave them blank to use any.
> > >>> > > > >>>
> > >>> > > > >>> Also, you could bump up the MET log verbosity and
see more
> > >>> > > information
> > >>> > > > >>> about why they are being rejected:
> > >>> > > > >>>
> > >>> > > > >>> # to affect all MET tools set:
> > >>> > > > >>> LOG_MET_VERBOSITY = 10
> > >>> > > > >>>
> > >>> > > > >>> # to affect only TCPairs set:
> > >>> > > > >>> LOG_TC_PAIRS_VERBOSITY = 10
> > >>> > > > >>>
> > >>> > > > >>> - George
> > >>> > > > >>>
> > >>> > > > >>> On Thu, Aug 27, 2020 at 10:53 AM Edward Strobach -
NOAA
> > >>> Affiliate
> > >>> > via
> > >>> > > > RT
> > >>> > > > >>> <
> > >>> > > > >>> met_help at ucar.edu> wrote:
> > >>> > > > >>>
> > >>> > > > >>> >
> > >>> > > > >>> > <URL:
> > >>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458 >
> > >>> > > > >>> >
> > >>> > > > >>> > Hi,
> > >>> > > > >>> >
> > >>> > > > >>> > I did decide to remove CyclonePlotter for now and
use the
> > >>> > > > >>> TCPairs_Config
> > >>> > > > >>> > file along with settings that I set in a
shared_TC.conf
> > that
> > >>> I
> > >>> > > store
> > >>> > > > >>> in a
> > >>> > > > >>> > temporary directory to process the ADECK and BDECK
data
> > that
> > >>> I
> > >>> > > have.
> > >>> > > > >>> Files
> > >>> > > > >>> > are generated for all the dates that span INIT_BEG
and
> > >>> INIT_END,
> > >>> > > > >>> thanks to
> > >>> > > > >>> > setting loop_by to INIT. However, only the header
is
> > written
> > >>> out,
> > >>> > > > and I
> > >>> > > > >>> > suspect it is do to what the log indicates here:
> > >>> > > > >>> >
> > >>> > > > >>> > DEBUG 4: [File 1 of 5] Used 0 of 168 lines read
from file
> > >>> > > > >>> >
> > >>> > > > >>> >
> > >>> > > > >>>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
> > >>> > > > >>> > DEBUG 4: [File 2 of 5] Used 0 of 168 lines read
from file
> > >>> > > > >>> >
> > >>> > > > >>> >
> > >>> > > > >>>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
> > >>> > > > >>> > DEBUG 4: [File 3 of 5] Used 0 of 168 lines read
from file
> > >>> > > > >>> >
> > >>> > > > >>> >
> > >>> > > > >>>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
> > >>> > > > >>> > DEBUG 4: [File 4 of 5] Used 0 of 168 lines read
from file
> > >>> > > > >>> >
> > >>> > > > >>> >
> > >>> > > > >>>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
> > >>> > > > >>> > DEBUG 4: [File 5 of 5] Used 0 of 168 lines read
from file
> > >>> > > > >>> >
> > >>> > > > >>> >
> > >>> > > > >>>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
> > >>> > > > >>> > DEBUG 3: Used 0 of 840 lines read from 5 file(s).
> > >>> > > > >>> > DEBUG 3: Identified 0 track(s).
> > >>> > > > >>> > DEBUG 5: TrackInfoArray: NTracks = 0, NAlloc = 0,
Tracks:
> > >>> > > > >>> > DEBUG 5:
> > >>> > > > >>> > DEBUG 2: Deriving 12-hour interpolated ADECK
tracks.
> > >>> > > > >>> > DEBUG 2: Finished adding 0 and replacing 0
Interp12
> > track(s).
> > >>> > > > >>> > DEBUG 2: Deriving 0 ADECK consensus model(s).
> > >>> > > > >>> > DEBUG 2: Added 0 ADECK consensus tracks(s).
> > >>> > > > >>> > DEBUG 2: Deriving 0 ADECK lag model(s).
> > >>> > > > >>> > DEBUG 2: Added 0 ADECK lag tracks(s).
> > >>> > > > >>> > DEBUG 2: Deriving 0 CLIPER/SHIFOR baseline
model(s).
> > >>> > > > >>> > DEBUG 2: Added 0 CLIPER/SHIFOR baseline track(s).
> > >>> > > > >>> > DEBUG 2: Filtering 0 ADECK tracks based on config
file
> > >>> settings.
> > >>> > > > >>> > DEBUG 3: Total tracks read = 0
> > >>> > > > >>> > DEBUG 3: Total tracks kept = 0
> > >>> > > > >>> > DEBUG 3: Rejected for storm name = 0
> > >>> > > > >>> > DEBUG 3: Rejected for valid time = 0
> > >>> > > > >>> > DEBUG 3: Rejected for required lead times = 0
> > >>> > > > >>> > DEBUG 3: Rejected for init mask = 0
> > >>> > > > >>> > DEBUG 3: Rejected for valid mask = 0
> > >>> > > > >>> > DEBUG 2: Matching 0 ADECK tracks to 1 BDECK
tracks.
> > >>> > > > >>> >
> > >>> > > > >>> > I'm still thinking that the formatting is off in
this
> file.
> > >>> I
> > >>> > > > surmise
> > >>> > > > >>> that
> > >>> > > > >>> > it's not able to group the data because the column
> > >>> information is
> > >>> > > not
> > >>> > > > >>> in
> > >>> > > > >>> > the order that met would want it to be in. These
files
> do
> > >>> not
> > >>> > have
> > >>> > > > any
> > >>> > > > >>> > header info, so it has to rely on the correct
ordering,
> or
> > >>> so I
> > >>> > > > >>> suspect.
> > >>> > > > >>> > My interpretation, however, may be flawed. I see
6
> > separate
> > >>> TC
> > >>> > > based
> > >>> > > > >>> > metplus routines: TCPairs (I'm working on
currently);
> > >>> > > CyclonePlotter
> > >>> > > > >>> > (looking to use); TCStat; TCRMW; TCMPRPlotter; and
TCGen.
> > >>> Much,
> > >>> > if
> > >>> > > > not
> > >>> > > > >>> > almost all of these relies on TCPairs. Is there a
> routine
> > >>> that
> > >>> > I'm
> > >>> > > > >>> missing
> > >>> > > > >>> > in metplus that takes a grib file and constructs
ADECK
> > files?
> > >>> > > > >>> >
> > >>> > > > >>> >
> > >>> > > > >>> >
> > >>> > > > >>> > On Thu, Aug 27, 2020 at 11:52 AM Edward Strobach -
NOAA
> > >>> > Affiliate <
> > >>> > > > >>> > edward.strobach at noaa.gov> wrote:
> > >>> > > > >>> >
> > >>> > > > >>> > > Hi George, I figured the BDECK must be
compatible, so
> no
> > >>> > worries
> > >>> > > > >>> there.
> > >>> > > > >>> > > Thanks for the link and the heads up about
> > reformatting. I
> > >>> > will
> > >>> > > > use
> > >>> > > > >>> the
> > >>> > > > >>> > > example to modify what I currently have to see
the
> > outcome.
> > >>> > > Thanks
> > >>> > > > >>> > again.
> > >>> > > > >>> > >
> > >>> > > > >>> > > On Wed, Aug 26, 2020 at 6:57 PM George McCabe
via RT <
> > >>> > > > >>> met_help at ucar.edu>
> > >>> > > > >>> > > wrote:
> > >>> > > > >>> > >
> > >>> > > > >>> > >> Hi Edward,
> > >>> > > > >>> > >>
> > >>> > > > >>> > >> The BDECK data is compatible to be processed.
The use
> > >>> case you
> > >>> > > > used
> > >>> > > > >>> as
> > >>> > > > >>> > an
> > >>> > > > >>> > >> example uses extra tropical cyclone data from
SBU that
> > is
> > >>> > > > >>> reformatted by
> > >>> > > > >>> > >> TCPairs wrapper before processing. You will see
your
> > >>> config
> > >>> > file
> > >>> > > > >>> has the
> > >>> > > > >>> > >> following:
> > >>> > > > >>> > >>
> > >>> > > > >>> > >> ## tc-pairs filtering
optionsTC_PAIRS_REFORMAT_DECK =
> > >>> > > > >>> > >> yesTC_PAIRS_REFORMAT_TYPE = SBU
> > >>> > > > >>> > >>
> > >>> > > > >>> > >> Changing TC_PAIRS_REFORMAT_DECK to 'no' or
'false'
> will
> > >>> skip
> > >>> > the
> > >>> > > > >>> > >> reformatting step. There is another use case in
the
> repo
> > >>> that
> > >>> > > uses
> > >>> > > > >>> data
> > >>> > > > >>> > >> that looks like yours (at least as far as I can
tell).
> > >>> > > > >>> > >>
> > >>> > > > >>> > >>
> > >>> > > > >>> > >>
> > >>> > > > >>> >
> > >>> > > > >>>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
>
https://dtcenter.github.io/METplus/generated/met_tool_wrapper/TCPairs/TCPairs_tropical.html#metplus-
configuration
> > >>> > > > >>> > >>
> > >>> > > > >>> > >> This use case uses BDECK data in
> > met_test/new/hwrf/bdeck.
> > >>> Here
> > >>> > > is
> > >>> > > > >>> part
> > >>> > > > >>> > of
> > >>> > > > >>> > >> the file called bal062018.dat:
> > >>> > > > >>> > >>
> > >>> > > > >>> > >> AL, 06, 2018082906, , BEST, 0, 129N, 109W,
10,
> > >>> 1010, DB,
> > >>> > > > >>> 0, ,
> > >>> > > > >>> > >> 0, 0, 0, 0, 0, 0, 0, 0,
0, ,
> > 0,
> > >>> > ,
> > >>> > > > 0,
> > >>> > > > >>> > 0,
> > >>> > > > >>> > >> GENESIS019, , 0, , 0, 0, 0, 0,
> > >>> genesis-num,
> > >>> > > 019,
> > >>> > > > >>> > >>
> > >>> > > > >>> > >>
> > >>> > > > >>> > >> Let me know if changing that setting doesn't
work and
> I
> > >>> can
> > >>> > > take a
> > >>> > > > >>> > closer
> > >>> > > > >>> > >> look.
> > >>> > > > >>> > >>
> > >>> > > > >>> > >> Thanks,
> > >>> > > > >>> > >> George
> > >>> > > > >>> > >>
> > >>> > > > >>> > >> On Wed, Aug 26, 2020 at 1:17 PM Edward Strobach
- NOAA
> > >>> > Affiliate
> > >>> > > > >>> via RT
> > >>> > > > >>> > <
> > >>> > > > >>> > >> met_help at ucar.edu> wrote:
> > >>> > > > >>> > >>
> > >>> > > > >>> > >> >
> > >>> > > > >>> > >> > Wed Aug 26 13:13:28 2020: Request 96458 was
acted
> > upon.
> > >>> > > > >>> > >> > Transaction: Ticket created by
> > edward.strobach at noaa.gov
> > >>> > > > >>> > >> > Queue: met_help
> > >>> > > > >>> > >> > Subject: Tropical Cyclone verification:
TCPairs
> > and
> > >>> > > > >>> > CyclonePlotter
> > >>> > > > >>> > >> > Owner: Nobody
> > >>> > > > >>> > >> > Requestors: edward.strobach at noaa.gov
> > >>> > > > >>> > >> > Status: new
> > >>> > > > >>> > >> > Ticket <URL:
> > >>> > > > >>>
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458
> > >>> > > > >>> > >
> > >>> > > > >>> > >> >
> > >>> > > > >>> > >> >
> > >>> > > > >>> > >> > Good afternoon,
> > >>> > > > >>> > >> >
> > >>> > > > >>> > >> > I found it relatively straightforward to set-
up some
> > >>> scripts
> > >>> > > to
> > >>> > > > >>> > process
> > >>> > > > >>> > >> > forecast output along with obs by using this
> resource:
> > >>> > > > >>> > >> >
> > >>> > > > >>> > >> >
> > >>> > > > >>> > >>
> > >>> > > > >>> >
> > >>> > > > >>>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
>
https://dtcenter.github.io/METplus/generated/model_applications/tc_and_extra_tc/Plotter_fcstGFS_obsGFS_ExtraTC.html#sphx-
glr-generated-model-applications-tc-and-extra-tc-plotter-fcstgfs-
obsgfs-extratc-py
> > >>> > > > >>> > >> > .
> > >>> > > > >>> > >> > I have been running the global model which
generates
> > >>> track
> > >>> > > data
> > >>> > > > >>> for
> > >>> > > > >>> > the
> > >>> > > > >>> > >> > post processing step following this naming
> convention:
> > >>> > > > >>> > >> > atcfunix.gfs.YYYYMMDDHH. The best track data
is
> > stored
> > >>> here
> > >>> > > on
> > >>> > > > >>> WCOSS:
> > >>> > > > >>> > >> >
> > >>> /gpfs/hps3/emc/hwrf/noscrub/emc.hurpara/trak/abdeck/btk/.
> > >>> > > > >>> > >> >
> > >>> > > > >>> > >> > I have a python script that determines which
file to
> > use
> > >>> > based
> > >>> > > > on
> > >>> > > > >>> > basin
> > >>> > > > >>> > >> > information, storm name, and date. That is
piped
> into
> > >>> the
> > >>> > > > >>> > >> > TC_PAIRS_BDECK_TEMPLATE while the forecast
file
> above
> > is
> > >>> > piped
> > >>> > > > >>> into
> > >>> > > > >>> > the
> > >>> > > > >>> > >> > TC_PAIRS_ADECK_TEMPLATE this way:
> > >>> > > > >>> atcfunix.gfs.{date?fmt=%Y%m%d%H}.
> > >>> > > > >>> > >> >
> > >>> > > > >>> > >> > Inside my main script
> > >>> > > > >>> > >> >
> > >>> > > > >>> > >> >
> > >>> > > > >>> > >>
> > >>> > > > >>> >
> > >>> > > > >>>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
>
*(/gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/METplus/scripts/run_metplus_TCPlotter.sh*,
> > >>> > > > >>> > >> > I have three configure files that being
processed
> like
> > >>> so:*
> > >>> > > > >>> > >> > ${MET_PLUS}/ush/master_metplus.py -c
> > >>> > > > >>> > >> >
${MET_PLUS_CONF}/Plotter_fcstGFS_obsGFS_ExtraTC.conf
> > -c
> > >>> > > > >>> > >> > ${MET_PLUS_TMP}/shared.conf_TC -c
> > >>> > > > ${MET_PLUS_CONF}/system_TC.conf*
> > >>> > > > >>> > >> >
> > >>> > > > >>> > >> > The system_TC.conf is of little concern
because it
> is
> > >>> mostly
> > >>> > > > >>> unchanged
> > >>> > > > >>> > >> > except for the OUTPUT_BASE, which is changed
> > >>> specifically to
> > >>> > > > >>> where I
> > >>> > > > >>> > >> plan
> > >>> > > > >>> > >> > on storing the TC output. The
shared_TC.conf, on
> the
> > >>> other
> > >>> > > > hand,
> > >>> > > > >>> is
> > >>> > > > >>> > >> used
> > >>> > > > >>> > >> > to replace the content inside my
> > >>> > > > >>> Plotter_fcstGFS_obsGFS_ExtraTC.conf
> > >>> > > > >>> > >> file.
> > >>> > > > >>> > >> > In here I define
> > >>> > > > >>> > >> > the ADECK and BDECK input directories based
on the
> > >>> where the
> > >>> > > > >>> template
> > >>> > > > >>> > >> files
> > >>> > > > >>> > >> > above are being stored. Since I have several
runs
> > >>> > > encompassing
> > >>> > > > >>> the
> > >>> > > > >>> > >> > tropical cyclone event - 20190924 through
20190929
> - I
> > >>> have
> > >>> > > > >>> specified
> > >>> > > > >>> > my
> > >>> > > > >>> > >> > INIT_BEG as 20190924 and my INIT_END as
20190929. I
> > >>> have
> > >>> > also
> > >>> > > > >>> set my
> > >>> > > > >>> > >> > INIT_INCREMENT to 86400 since I'm only
processing
> the
> > >>> 00z
> > >>> > > cycle
> > >>> > > > >>> for
> > >>> > > > >>> > >> now. I
> > >>> > > > >>> > >> > have a couple of lines of code before the
configure
> > file
> > >>> > that
> > >>> > > > >>> joins
> > >>> > > > >>> > >> these
> > >>> > > > >>> > >> > dates in a comma separated list such that the
> > >>> > > > >>> > CYCLONE_PLOTTER_INIT_DATE
> > >>> > > > >>> > >> is
> > >>> > > > >>> > >> > "20190924,20190925,...,20190929". I'm not
sure if
> > that
> > >>> is
> > >>> > > > allowed
> > >>> > > > >>> > >> though.
> > >>> > > > >>> > >> >
> > >>> > > > >>> > >> > When running the script the log that is
generated
> > >>> produces 0
> > >>> > > > >>> errors.
> > >>> > > > >>> > >> > However, I find *two issues*: *1) there is
only one
> > >>> file
> > >>> > > being
> > >>> > > > >>> > carried
> > >>> > > > >>> > >> > over to generate results; that is the
20190924 date
> -
> > >>> there
> > >>> > > > >>> should be
> > >>> > > > >>> > >> 6*,
> > >>> > > > >>> > >> > and *2) The files that are created are empty
and
> thus
> > >>> the
> > >>> > plot
> > >>> > > > >>> that is
> > >>> > > > >>> > >> > generated is simply a map with no data
overlaid.*
> > >>> > > > >>> > >> >
> > >>> > > > >>> > >> > In attempting to debug further, I found
something
> that
> > >>> looks
> > >>> > > > >>> important
> > >>> > > > >>> > >> > shown here:
> > >>> > > > >>> > >> >
> > >>> > > > >>> > >> > DEBUG 3: Rejected for init mask = 0
> > >>> > > > >>> > >> > DEBUG 3: Rejected for valid mask = 0
> > >>> > > > >>> > >> > *DEBUG 2: Matching 0 ADECK tracks to 0 BDECK
> tracks.*
> > >>> > > > >>> > >> > DEBUG 1: Watch/Warning file:
> > >>> > > > >>> > >> >
> > >>> > > > >>> > >> >
> > >>> > > > >>> > >>
> > >>> > > > >>> >
> > >>> > > > >>>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
>
/gpfs/dell2/emc/verification/noscrub/Julie.Prestopnik/met/9.1_beta2/share/met/tc_data/wwpts_us.txt
> > >>> > > > >>> > >> > DEBUG 5: TrackPairInfoArray: NPairs = 0,
NAlloc = 0,
> > >>> Pairs:
> > >>> > > > >>> > >> > DEBUG 5:
> > >>> > > > >>> > >> > DEBUG 1: Output file:
> > >>> > > > >>> > >> >
> > >>> > > > >>> > >> >
> > >>> > > > >>> > >>
> > >>> > > > >>> >
> > >>> > > > >>>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs/201909/AL_q2019092400.gfs.PRNA20.tcst
> > >>> > > > >>> > >> > 08/26 18:35:12.422 metplus.TCPairs
> > >>> (command_runner.py:191)
> > >>> > > > DEBUG:
> > >>> > > > >>> > >> Finished
> > >>> > > > >>> > >> > running
> > >>> > > > >>> > >> >
> > >>> > > > >>> > >> >
> > >>> > > > >>> > >>
> > >>> > > > >>> >
> > >>> > > > >>>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
>
/gpfs/dell2/emc/verification/noscrub/Julie.Prestopnik/met/9.1_beta2/bin/tc_pairs
> > >>> > > > >>> > >> > in 0:00:00.607349
> > >>> > > > >>> > >> > 08/26 18:35:12.422 metplus.CyclonePlotter
> > >>> > > > >>> > >> (cyclone_plotter_wrapper.py:77)
> > >>> > > > >>> > >> > DEBUG: Begin retrieving data...
> > >>> > > > >>> > >> > 08/26 18:35:12.422 metplus.CyclonePlotter
> > >>> > > > >>> > >> (cyclone_plotter_wrapper.py:83)
> > >>> > > > >>> > >> > DEBUG: Generate plot for all files in the
> > >>> > > > >>> > >> >
> > >>> > > > >>> > >> >
> > >>> > > > >>> > >>
> > >>> > > > >>> >
> > >>> > > > >>>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
>
directory/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs
> > >>> > > > >>> > >> > 08/26 18:35:12.423 metplus.CyclonePlotter
> > >>> > > > >>> > >> (cyclone_plotter_wrapper.py:98)
> > >>> > > > >>> > >> > DEBUG: Parsing file
> > >>> > > > >>> > >> >
> > >>> > > > >>> > >> >
> > >>> > > > >>> > >>
> > >>> > > > >>> >
> > >>> > > > >>>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs/201909/AL_q2019092400.gfs.PRNA20.tcst
> > >>> > > > >>> > >> > 08/26 18:35:19.778 metplus
(config_launcher.py:520)
> > >>> DEBUG:
> > >>> > > > Setting
> > >>> > > > >>> > >> [config]
> > >>> > > > >>> > >> > SCRUB_STAGING_DIR to default value: False.
> > >>> > > > >>> > >> > 08/26 18:35:19.779 metplus INFO:
> > >>> > > > >>> > >> >
> > >>> > > > >>> > >> >
> > >>> > > > >>> > >>
> > >>> > > > >>> >
> > >>> > > > >>>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/metplus_final.conf:
> > >>> > > > >>> > >> > write metplus.conf here
> > >>> > > > >>> > >> > 08/26 18:35:19.783 metplus (met_util.py:213)
DEBUG:
> > >>> METplus
> > >>> > > took
> > >>> > > > >>> > >> > 0:00:08.783240 to run.
> > >>> > > > >>> > >> > 08/26 18:35:19.783 metplus INFO: METplus has
> > >>> successfully
> > >>> > > > finished
> > >>> > > > >>> > >> running.
> > >>> > > > >>> > >> >
> > >>> > > > >>> > >> > Specifically the line that says matching 0
ADECK
> > tracks
> > >>> to 0
> > >>> > > > BDECK
> > >>> > > > >>> > >> tracks
> > >>> > > > >>> > >> > lends me to believe that this is the reason
why
> > nothing
> > >>> is
> > >>> > > being
> > >>> > > > >>> added
> > >>> > > > >>> > >> to
> > >>> > > > >>> > >> > my TC_PAIRS_OUTPUT_TEMPLATE file, and
therefore no
> > plots
> > >>> > > > >>> generated on
> > >>> > > > >>> > a
> > >>> > > > >>> > >> > map.
> > >>> > > > >>> > >> >
> > >>> > > > >>> > >> > What's interesting is the file that is
extracted for
> > >>> BDECK
> > >>> > is
> > >>> > > > for
> > >>> > > > >>> > >> hurricane
> > >>> > > > >>> > >> > Lorenzo and includes all the dates coinciding
with
> the
> > >>> ADECK
> > >>> > > > file
> > >>> > > > >>> - a
> > >>> > > > >>> > >> case
> > >>> > > > >>> > >> > that is run coinciding with Lorenzo. I've
checked
> > this.
> > >>> > > > >>> > >> >
> > >>> > > > >>> > >> > One possible issue is the formatting of these
files.
> > I
> > >>> > looked
> > >>> > > > in
> > >>> > > > >>> the
> > >>> > > > >>> > >> test
> > >>> > > > >>> > >> > case that comes with the cloned met stuff.
In there
> > the
> > >>> > > > >>> header/field
> > >>> > > > >>> > >> info
> > >>> > > > >>> > >> > looks something like this:
> > >>> > > > >>> > >> >
> > >>> > > > >>> > >> >
> > >>> > > > >>> > >> >
> > >>> > > > >>> > >>
> > >>> > > > >>> >
> > >>> > > > >>>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
>
*/gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/METplus/source_data/met_test/new/track_data/201412/amlq2014123118.gfso.0104*
> > >>> > > > >>> > >> >
> > >>> > > > >>> > >> > ML, 0104, 2014121318_F000_478N_1780W_FOF,
> 2014121318,
> > >>> 03,
> > >>> > > GFSO,
> > >>> > > > >>> 000,
> > >>> > > > >>> > >> 478N,
> > >>> > > > >>> > >> > 1780W, 38, 986, XX, 34, NEQ, 0283, 0640,
0645,
> > 0000,
> > >>> > 990,
> > >>> > > > >>> 133,
> > >>> > > > >>> > -99,
> > >>> > > > >>> > >> > 69, 227, -99, -9999, -9999, 208, 315,
135, 156,
> > >>> 0063
> > >>> > > > >>> > >> >
> > >>> > > > >>> > >> > This differs from the structure of both the
ADECK
> and
> > >>> BDECK
> > >>> > > > input
> > >>> > > > >>> > files:
> > >>> > > > >>> > >> >
> > >>> > > > >>> > >> > *BDECK:
> > >>> > > > >>> > >> >
> > >>> > > > >>>
> > >>> >
>
/gpfs/hps3/emc/hwrf/noscrub/emc.hurpara/trak/abdeck/btk/bal132019.dat
> > >>> > > > >>> > *
> > >>> > > > >>> > >> >
> > >>> > > > >>> > >> > AL, 13, 2019092218, , BEST, 0, 109N,
188W, 25,
> > >>> 1008,
> > >>> > LO,
> > >>> > > > >>> 0,
> > >>> > > > >>> > ,
> > >>> > > > >>> > >> > 0, 0, 0, 0, 1012, 100, 90, 40,
0,
> L,
> > >>> 0,
> > >>> > > ,
> > >>> > > > >>> 0,
> > >>> > > > >>> > >> 0,
> > >>> > > > >>> > >> > INVEST, S, 0, , 0, 0, 0, 0,
> > >>> genesis-num,
> > >>> > > 027,
> > >>> > > > >>> > >> >
> > >>> > > > >>> > >> > *ADECK:
> > >>> > > > >>> > >> >
> > >>> > > > >>> > >> >
> > >>> > > > >>> > >>
> > >>> > > > >>> >
> > >>> > > > >>>
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400*
> > >>> > > > >>> > >> >
> > >>> > > > >>> > >> > AL, 0912, 03, PRNA, 000, 157N, 655W, 27,
1008, XX,
> > >>> 34,
> > >>> > NEQ,
> > >>> > > > >>> 0000,
> > >>> > > > >>> > >> 0000,
> > >>> > > > >>> > >> > 0000, 0000, -9999, -9999, 85
> > >>> > > > >>> > >> >
> > >>> > > > >>> > >> > My suspicion is that these formats are not
> compatible.
> > >>> > There
> > >>> > > > are
> > >>> > > > >>> > major
> > >>> > > > >>> > >> > differences between these formats in my
opinion.
> The
> > >>> ADECK
> > >>> > > was
> > >>> > > > >>> > >> generated
> > >>> > > > >>> > >> > automatically when making a run while BDECK
is done
> by
> > >>> > someone
> > >>> > > > >>> else.
> > >>> > > > >>> > >> Those
> > >>> > > > >>> > >> > files are stored whenever a storm is
identified.
> > >>> > > > >>> > >> >
> > >>> > > > >>> > >> > I think what I'll need to do next is start by
> running
> > >>> TCRMW
> > >>> > to
> > >>> > > > >>> produce
> > >>> > > > >>> > >> the
> > >>> > > > >>> > >> > ADECK files formatted by metplus. I have the
grib
> > >>> files for
> > >>> > > > that.
> > >>> > > > >>> > The
> > >>> > > > >>> > >> > question next, though, is the BDECK files
okay for
> > >>> > processing?
> > >>> > > > >>> > >> >
> > >>> > > > >>> > >> >
> > >>> > > > >>> > >> >
> > >>> > > > >>> > >> > --
> > >>> > > > >>> > >> > Edward Strobach
> > >>> > > > >>> > >> > EMC/NCEP/NWS/
> > >>> > > > >>> > >> > IMSG Contractor
> > >>> > > > >>> > >> > Cubicle#: 2029
> > >>> > > > >>> > >> > 301-683-3717
> > >>> > > > >>> > >> >
> > >>> > > > >>> > >> >
> > >>> > > > >>> > >>
> > >>> > > > >>> > >> --
> > >>> > > > >>> > >> George McCabe - Software Engineer III
> > >>> > > > >>> > >> National Center for Atmospheric Research
> > >>> > > > >>> > >> Research Applications Laboratory
> > >>> > > > >>> > >> 303-497-2768
> > >>> > > > >>> > >> ---
> > >>> > > > >>> > >> My working day may not be your working day.
Please do
> > not
> > >>> feel
> > >>> > > > >>> obliged
> > >>> > > > >>> > to
> > >>> > > > >>> > >> reply to this email outside of your normal
working
> > hours.
> > >>> > > > >>> > >>
> > >>> > > > >>> > >>
> > >>> > > > >>> > >
> > >>> > > > >>> > > --
> > >>> > > > >>> > > Edward Strobach
> > >>> > > > >>> > > EMC/NCEP/NWS/
> > >>> > > > >>> > > IMSG Contractor
> > >>> > > > >>> > > Cubicle#: 2029
> > >>> > > > >>> > > 301-683-3717
> > >>> > > > >>> > >
> > >>> > > > >>> >
> > >>> > > > >>> >
> > >>> > > > >>> > --
> > >>> > > > >>> > Edward Strobach
> > >>> > > > >>> > EMC/NCEP/NWS/
> > >>> > > > >>> > IMSG Contractor
> > >>> > > > >>> > Cubicle#: 2029
> > >>> > > > >>> > 301-683-3717
> > >>> > > > >>> >
> > >>> > > > >>> >
> > >>> > > > >>>
> > >>> > > > >>> --
> > >>> > > > >>> George McCabe - Software Engineer III
> > >>> > > > >>> National Center for Atmospheric Research
> > >>> > > > >>> Research Applications Laboratory
> > >>> > > > >>> 303-497-2768
> > >>> > > > >>> ---
> > >>> > > > >>> My working day may not be your working day. Please
do not
> > feel
> > >>> > > obliged
> > >>> > > > to
> > >>> > > > >>> reply to this email outside of your normal working
hours.
> > >>> > > > >>>
> > >>> > > > >>>
> > >>> > > > >>
> > >>> > > > >> --
> > >>> > > > >> Edward Strobach
> > >>> > > > >> EMC/NCEP/NWS/
> > >>> > > > >> IMSG Contractor
> > >>> > > > >> Cubicle#: 2029
> > >>> > > > >> 301-683-3717
> > >>> > > > >>
> > >>> > > > >
> > >>> > > > >
> > >>> > > > > --
> > >>> > > > > Edward Strobach
> > >>> > > > > EMC/NCEP/NWS/
> > >>> > > > > IMSG Contractor
> > >>> > > > > Cubicle#: 2029
> > >>> > > > > 301-683-3717
> > >>> > > > >
> > >>> > > >
> > >>> > > >
> > >>> > > > --
> > >>> > > > Edward Strobach
> > >>> > > > EMC/NCEP/NWS/
> > >>> > > > IMSG Contractor
> > >>> > > > Cubicle#: 2029
> > >>> > > > 301-683-3717
> > >>> > > >
> > >>> > > >
> > >>> > >
> > >>> > >
> > >>> >
> > >>> > --
> > >>> > Edward Strobach
> > >>> > EMC/NCEP/NWS/
> > >>> > IMSG Contractor
> > >>> > Cubicle#: 2029
> > >>> > 301-683-3717
> > >>> >
> > >>> >
> > >>>
> > >>>
> > >>
> > >> --
> > >> Edward Strobach
> > >> EMC/NCEP/NWS/
> > >> IMSG Contractor
> > >> Cubicle#: 2029
> > >> 301-683-3717
> > >>
> > >
> > >
> > > --
> > > Edward Strobach
> > > EMC/NCEP/NWS/
> > > IMSG Contractor
> > > Cubicle#: 2029
> > > 301-683-3717
> > >
> >
> >
> > --
> > Edward Strobach
> > EMC/NCEP/NWS/
> > IMSG Contractor
> > Cubicle#: 2029
> > 301-683-3717
> >
> >
>
>
------------------------------------------------
Subject: Tropical Cyclone verification: TCPairs and CyclonePlotter
From: Edward Strobach - NOAA Affiliate
Time: Tue Sep 01 17:30:02 2020
Hi John,
Yes, that would be helpful I think. Even though it took some time to
figure out why, especially when I thought that everything was set up
right,
it came down to a slight difference in model name representation. I
wasn't
aware of the model name making an impact since I was basing my
diagnosis on
data filtering options. The other enhancements sound similar and
think
that they would be also helpful. If a message was pointing to the
source
of error, that would definitely be helpful.
What I've been doing to improve debugging is going into the referenced
python script. Sometimes it helps while other times it doesn't.
That's
actually what I'm working on right now. If a message does appear
obscure,
I'll make sure to let you all know. I realize that one of the goals
is to
have an intuitive log file. I've been pretty happy with many of the
log
messages since most of them do point out what the issues are. Thanks
On Tue, Sep 1, 2020 at 7:09 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Ed,
>
> OK, I'm trying to catch up on this message history. So it sounds
like
> tc_pairs was filtering out your track data based on the model name.
From
> tc_pairs perspective, if the model entry is defined in the config
file,
> then that filter is applied. If the model entry is left empty, then
all
> data is used regardless of model name.
>
> Since you're running this through METplus, I suppose it was METplus
that
> was setting the model config option to the string which produced no
pairs.
>
> This question of "Why am I getting no output data?" comes up in
several
> contexts. For point-stat, we have users run at verbosity level 3 (-v
3) so
> see counts for why obs were or were not used for each vx task. Last
week,
> the same question came up for ensemble-stat, but we DO NOT have the
reason
> counting logic there. And here the confusion was caused by tc_pairs.
>
> I assume if you saw a log message saying something like:
> *Rejected 119 lines based on MODEL name.*
> That would have pointed you in the right direction much faster.
>
> So we could enhance...
> - Ensemble-Stat to add rejection reason counts when verifying
against point
> obs.
> - TC-Pairs to add rejection reason counts for which ATCF lines were
used.
> - TC-Stat to add rejection reason counts for which TCST lines were
used.
> - Stat-Analysis to add rejection reason counts for which STAT lines
were
> used.
>
> Do you think any of these potential enhancements are worth doing?
>
> Thanks,
> John
>
> On Tue, Sep 1, 2020 at 12:05 PM Minna Win via RT <met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458 >
> >
> > Hi Ed,
> >
> > CyclonePlotter was created to replicate the plots generated by one
of our
> > NOAA partners and not designed to be configurable.
> >
> > Regards,
> > Minna
> > ---------------
> > Minna Win
> > National Center for Atmospheric Research
> > Developmental Testbed Center
> > Phone: 303-497-8423
> > Fax: 303-497-8401
> >
> >
> >
> > On Tue, Sep 1, 2020 at 11:57 AM Edward Strobach - NOAA Affiliate
via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458 >
> > >
> > > I forgot to add something..
> > >
> > > What's interesting is that the tutorial going over PROCESS_LIST
=
> > TCPairs,
> > > CyclonePlotter indicates that the plot below could be generated.
I've
> > used
> > > all the settings in the tutorial and never got that plot. Of
course
> > that
> > > wouldn't be possible based on the ASCII file that is generated
in the
> > > CyclonePlotter step. It only contains lat lon info. I'm not
sure what
> > > lead_group = 0 and lead_group = 6 means. It does appear -
according to
> > the
> > > valid time in the text file - that it does go out to 144 hours,
but
> > > alternates between lead group 0 and lead group 6.
> > >
> > > model_name: PRNA storm_id: AL132019 init_time:
20190924_000000
> > > valid_time: 20190924_000000 lat: 11.4 lon: -25.7
lead_group: 0
> > > first_point:True
> > > model_name: PRNA storm_id: AL132019 init_time:
20190924_000000
> > > valid_time: 20190924_060000 lat: 11.6 lon: -27.4
lead_group: 6
> > > first_point:False
> > > model_name: PRNA storm_id: AL132019 init_time:
20190924_000000
> > > valid_time: 20190924_120000 lat: 11.8 lon: -28.2
lead_group: 0
> > > first_point:False
> > > model_name: PRNA storm_id: AL132019 init_time:
20190924_000000
> > > valid_time: 20190924_180000 lat: 12.7 lon: -28.9
lead_group: 6
> > > first_point:False
> > > model_name: PRNA storm_id: AL132019 init_time:
20190924_000000
> > > valid_time: 20190925_000000 lat: 13.1 lon: -30.6
lead_group: 0
> > > first_point:False
> > > model_name: PRNA storm_id: AL132019 init_time:
20190924_000000
> > > valid_time: 20190925_060000 lat: 13.5 lon: -31.8
lead_group: 6
> > > first_point:False
> > > model_name: PRNA storm_id: AL132019 init_time:
20190924_000000
> > > valid_time: 20190925_120000 lat: 13.6 lon: -33.0
lead_group: 0
> > > first_point:False
> > > model_name: PRNA storm_id: AL132019 init_time:
20190924_000000
> > > valid_time: 20190925_180000 lat: 14.0 lon: -34.0
lead_group: 6
> > > first_point:False
> > > model_name: PRNA storm_id: AL132019 init_time:
20190924_000000
> > > valid_time: 20190926_000000 lat: 14.5 lon: -35.1
lead_group: 0
> > > first_point:False
> > > model_name: PRNA storm_id: AL132019 init_time:
20190924_000000
> > > valid_time: 20190926_060000 lat: 14.8 lon: -36.2
lead_group: 6
> > > first_point:False
> > > model_name: PRNA storm_id: AL132019 init_time:
20190924_000000
> > > valid_time: 20190926_120000 lat: 15.3 lon: -37.0
lead_group: 0
> > > first_point:False
> > > model_name: PRNA storm_id: AL132019 init_time:
20190924_000000
> > > valid_time: 20190926_180000 lat: 16.0 lon: -37.7
lead_group: 6
> > > first_point:False
> > > model_name: PRNA storm_id: AL132019 init_time:
20190924_000000
> > > valid_time: 20190927_000000 lat: 16.9 lon: -38.2
lead_group: 0
> > > first_point:False
> > > model_name: PRNA storm_id: AL132019 init_time:
20190924_000000
> > > valid_time: 20190927_060000 lat: 17.8 lon: -38.7
lead_group: 6
> > > first_point:False
> > > model_name: PRNA storm_id: AL132019 init_time:
20190924_000000
> > > valid_time: 20190927_120000 lat: 18.8 lon: -39.2
lead_group: 0
> > > first_point:False
> > > model_name: PRNA storm_id: AL132019 init_time:
20190924_000000
> > > valid_time: 20190927_180000 lat: 19.8 lon: -39.6
lead_group: 6
> > > first_point:False
> > > model_name: PRNA storm_id: AL132019 init_time:
20190924_000000
> > > valid_time: 20190928_000000 lat: 20.8 lon: -39.8
lead_group: 0
> > > first_point:False
> > > model_name: PRNA storm_id: AL132019 init_time:
20190924_000000
> > > valid_time: 20190928_060000 lat: 21.8 lon: -40.1
lead_group: 6
> > > first_point:False
> > > model_name: PRNA storm_id: AL132019 init_time:
20190924_000000
> > > valid_time: 20190928_120000 lat: 22.8 lon: -40.2
lead_group: 0
> > > first_point:False
> > > model_name: PRNA storm_id: AL132019 init_time:
20190924_000000
> > > valid_time: 20190928_180000 lat: 24.0 lon: -40.4
lead_group: 6
> > > first_point:False
> > > model_name: PRNA storm_id: AL132019 init_time:
20190924_000000
> > > valid_time: 20190929_000000 lat: 25.1 lon: -40.4
lead_group: 0
> > > first_point:False
> > > model_name: PRNA storm_id: AL132019 init_time:
20190924_000000
> > > valid_time: 20190929_060000 lat: 26.4 lon: -40.7
lead_group: 6
> > > first_point:False
> > > model_name: PRNA storm_id: AL132019 init_time:
20190924_000000
> > > valid_time: 20190929_120000 lat: 27.6 lon: -41.2
lead_group: 0
> > > first_point:False
> > > model_name: PRNA storm_id: AL132019 init_time:
20190924_000000
> > > valid_time: 20190929_180000 lat: 28.5 lon: -41.7
lead_group: 6
> > > first_point:False
> > > model_name: PRNA storm_id: AL132019 init_time:
20190924_000000
> > > valid_time: 20190930_000000 lat: 29.3 lon: -42.0
lead_group: 0
> > > first_point:False
> > >
> > > On Tue, Sep 1, 2020 at 1:47 PM Edward Strobach - NOAA Affiliate
<
> > > edward.strobach at noaa.gov> wrote:
> > >
> > > > The good news is I've also got CyclonePlotter to work.
However, it
> > > would
> > > > appear that CyclonePlotter is really just meant to show a
quick and
> > dirty
> > > > plot based on the available settings and the quality of the
plot -
> > > > attached below.
> > > >
> > > > I noticed that I can't process all forecasts at once, rather I
have
> to
> > > > loop through each of my forecasts in order to generate plots
for
> > 20190924
> > > > through 20190929. Also, I have no real control over grid
lines, tick
> > > > marks, legend to distinguish between best track and model,
etc. As
> > > such, I
> > > > may abandon CyclonePlotter.
> > > >
> > > > Something else that's a bit odd is that TCPairs is only
grouping the
> > data
> > > > out to 72 hours instead of 144 hours. There's no apparent
setting to
> > > > extend this to 144 hours according to online sources that I've
> > consulted.
> > > > Is there a way to add an option such as TC_MAX_LEAD so that
met knows
> > to
> > > > group out to 144 hours?
> > > >
> > > > TC_PAIRS_CONFIG_FILE
> > > > <
> > >
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_CONFIG_FILE
> > > >
> > > > INIT_HOUR_END
> > > > <
> > >
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
INIT_HOUR_END
> > > >
> > > > INIT_INCLUDE
> > > > <
> > >
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
INIT_INCLUDE
> > > >
> > > > INIT_EXCLUDE
> > > > <
> > >
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
INIT_EXCLUDE
> > > >
> > > > TC_PAIRS_READ_ALL_FILES
> > > > <
> > >
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_READ_ALL_FILES
> > > >
> > > > TC_PAIRS_MODEL
> > > > <
> > >
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_MODEL
> > > >
> > > > TC_PAIRS_STORM_ID
> > > > <
> > >
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_STORM_ID
> > > >
> > > > TC_PAIRS_BASIN
> > > > <
> > >
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_BASIN
> > > >
> > > > TC_PAIRS_CYCLONE
> > > > <
> > >
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_CYCLONE
> > > >
> > > > TC_PAIRS_STORM_NAME
> > > > <
> > >
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_STORM_NAME
> > > >
> > > > TC_PAIRS_DLAND_FILE
> > > > <
> > >
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_DLAND_FILE
> > > >
> > > > TC_PAIRS_MISSING_VAL_TO_REPLACE
> > > > <
> > >
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_MISSING_VAL_TO_REPLACE
> > > >
> > > > TC_PAIRS_MISSING_VAL
> > > > <
> > >
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_MISSING_VAL
> > > >
> > > > TC_PAIRS_SKIP_IF_REFORMAT_EXISTS
> > > > <
> > >
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_SKIP_IF_REFORMAT_EXISTS
> > > >
> > > > TC_PAIRS_SKIP_IF_OUTPUT_EXISTS
> > > > <
> > >
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_SKIP_IF_OUTPUT_EXISTS
> > > >
> > > > TC_PAIRS_REFORMAT_DECK
> > > > <
> > >
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_REFORMAT_DECK
> > > >
> > > > TC_PAIRS_REFORMAT_TYPE
> > > > <
> > >
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_REFORMAT_TYPE
> > > >
> > > > TC_PAIRS_CUSTOM_LOOP_LIST
> > > > <
> > >
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_CUSTOM_LOOP_LIST
> > > >
> > > >
> > > > On Tue, Sep 1, 2020 at 12:43 PM Edward Strobach - NOAA
Affiliate <
> > > > edward.strobach at noaa.gov> wrote:
> > > >
> > > >> Thanks. Those examples are helpful and do confirm that the
files
> > created
> > > >> and passed into the TCPairs as ADECK files do follow, more or
less,
> > the
> > > >> format that's shown in the a{basin}{STORMNUM}{YYYY}.dat
files. I
> also
> > > >> found this
> > > >> <
> > >
> >
> https://dtcenter.org/metplus-practical-session-guide-feb-
2019/session-5-trkintfeature-relative/met-tool-tc-pairs
> > > >
> > > >> to be helpful as well. When comparing the values/description
in my
> > file
> > > >>
> > >
> >
>
(/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092900)
> > > >> with the contents in the link, it is clear that the order of
my file
> > > obeys
> > > >> the following:
> > > >>
> > > >> Basin, CY, YYYYMMDDHH, TECHNUM/MIN, TECH, TAU, LATN/S,
LonE/W, VMAX,
> > > >> MSLP, TY, RAD, WINDCODE, RAD1, RAD2, RAD3, RAD4.
> > > >>
> > > >> My first line of the referenced file has that in addition to
3
> columns
> > > at
> > > >> the end - so 20 instead of 17 entries. I assume that the
last 3 are
> > > >> ignored.
> > > >>
> > > >> *AL, 13, 2019092900, 03, PRNA, 000, 238N, 448W, 85, 962,
XX, 34,
> > > NEQ,
> > > >> 0185, 0184, 0110, 0148*, 978, 30, 32
> > > >>
> > > >> *The good thing is that I figured it out. * Since feedback is
> > desirable,
> > > >> I'm going to go through the process that helped me arrive at
the
> > > solution
> > > >> which I detail below.
> > > >>
> > > >>
> > > >> Since there can be more than one storm in the Atlantic at a
given
> > time,
> > > I
> > > >> decided to filter by *TC_PAIRS_CYCLONE* and set that to *13*
since
> I'm
> > > >> interested in processing Lorenzo. I can see that it's
defined in my
> > > master
> > > >> config file as well. I did set the log verbosity to 10 to
retrieve
> > > >> additional info. There are no error messages; however, it
says
> > > "*Matching
> > > >> 0 ADECK tracks to 1 BDECK trac*k". The BDECK track, which I
> retrieved
> > > >> from the best track data formatted as bal132019.dat, is being
> > processed
> > > >> alongside the ADECK files, which are created during the
forecast
> step.
> > > >>
> > > >> Since I've run 6 forecasts that go out to 144 hours, I also
have 6
> > ADECK
> > > >> files initialized between 20190924 00Z and 20190929 00Z. In
the
> case
> > > of
> > > >> 20190929 00Z, for example, the log file indicates that the
BDECK
> track
> > > is
> > > >> processed followed by the processing of the ADECK file. The
log
> > > correctly
> > > >> identifies 119 lines of processing but says 0 of 119 lines
are used:
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >>
> > > >> *DEBUG 5: | | | [3] QuadInfo: Intensity = 64, ALVal
=
> > > >> -9999.00000, NEVal = -9999.00000, SEVal = -9999.00000, SWVal
=
> > > -9999.00000,
> > > >> NWVal = -9999.00000DEBUG 5:DEBUG 5:DEBUG 5:DEBUG 2: Found 1
BDECK
> > > >> track(s).DEBUG 2: Processing 1 ADECK file(s).DEBUG 4: [File 1
of 1]
> > > Used 0
> > > >> of 119 lines read from file
> > > >>
> > >
> >
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092900"DEBUG
> > > >> 3: Used 0 of 119 lines read from 1 file(s).DEBUG 3:
Identified 0
> > > >> track(s).DEBUG 5: TrackInfoArray: NTracks = 0, NAlloc = 0,
> > Tracks:DEBUG
> > > >> 5:DEBUG 2: Deriving 12-hour interpolated ADECK tracks.DEBUG
2:
> > Finished
> > > >> adding 0 and replacing 0 Interp12 track(s).DEBUG 2: Deriving
0 ADECK
> > > >> consensus model(s).DEBUG 2: Added 0 ADECK consensus
tracks(s).DEBUG
> 2:
> > > >> Deriving 0 ADECK lag model(s).DEBUG 2: Added 0 ADECK lag
> > > tracks(s).DEBUG 2:
> > > >> Deriving 0 CLIPER/SHIFOR baseline model(s).DEBUG 2: Added 0
> > > CLIPER/SHIFOR
> > > >> baseline track(s).DEBUG 2: Filtering 0 ADECK tracks based on
config
> > file
> > > >> settings.DEBUG 3: Total tracks read = 0DEBUG
3: Total
> > > tracks
> > > >> kept = 0DEBUG 3: Rejected for storm name
=
> > > 0DEBUG
> > > >> 3: Rejected for valid time = 0DEBUG 3: Rejected for
> required
> > > lead
> > > >> times = 0DEBUG 3: Rejected for init mask = 0DEBUG
3:
> > Rejected
> > > for
> > > >> valid mask = 0DEBUG 2: Matching 0 ADECK tracks to 1
BDECK
> > > tracks.*
> > > >>
> > > >> This is repeated for all six forecasts. Also, all lines of
the
> BDECK
> > > >> file are processed, so the dates in that file encompass the
dates in
> > > each
> > > >> of the forecast files.
> > > >>
> > > >> I was flummoxed as to why this would be. Under closer
inspection I
> > > found
> > > >> that the directory - which is named after the model -
includes the
> > > >> experiment number as well (e.g. prna20). However, inside the
ADECK
> > > files
> > > >> the model is PRNA, not PRNA20. Met was looking for instances
of
> > PRNA20,
> > > >> and when it couldn't it concluded that there were zero
matches.
> > > However,
> > > >> when I changed my pathway to "TC_PAIRS_ADECK_INPUT_DIR =
> > > >> /gpfs/dell2/emc/modeling/noscrub/${USER}/archive/${model}20"
and
> set
> > > model
> > > >> to "prna" instead of "prna20" it worked. I now have
populated
> files.
> > > >> Moving forward I will need to keep this subtlety in mind. I
plan
> to
> > > use
> > > >> other TC packages pretty soon here since I've been tasked to
examine
> > > some
> > > >> physics changes and tune where necessary. I don't know what
type of
> > > issues
> > > >> I may encounter, but I plan to use this experience as a model
to
> > > approach
> > > >> possible issues related to TC verification in the future.
> > > >>
> > > >>
> > > >>
> > > >> On Mon, Aug 31, 2020 at 3:31 PM John Halley Gotway via RT <
> > > >> met_help at ucar.edu> wrote:
> > > >>
> > > >>> Ed,
> > > >>>
> > > >>> Here's a link to the documentation that the Navy provides
about the
> > > ATCF
> > > >>> file format:
> > > >>>
> https://www.nrlmry.navy.mil/atcf_web/docs/database/new/abrdeck.html
> > > >>>
> > > >>> And the best source of sample data is NHC's ftp site:
> > > >>> https://ftp.nhc.noaa.gov/atcf/
> > > >>>
> > > >>> In there...
> > > >>> *aid_public* contains tar files of ADECKS for all storms in
the
> > current
> > > >>> season.
> > > >>> *btk* has uncompressed BEST track files for all storms in
the
> current
> > > >>> season.
> > > >>> *archive* contains compressed A and B-Deck data for previous
> seasons.
> > > >>>
> > > >>> It's pretty cool actually that this track data is so readily
> > available.
> > > >>>
> > > >>> I have never actually modified ATCF files for my purposes.
So
> > hopefully
> > > >>> you
> > > >>> won't need to either... and can just use the output of the
tracker
> > code
> > > >>> directly.
> > > >>>
> > > >>> Thanks,
> > > >>> John
> > > >>>
> > > >>>
> > > >>> On Mon, Aug 31, 2020 at 9:24 AM Edward Strobach - NOAA
Affiliate
> via
> > > RT <
> > > >>> met_help at ucar.edu> wrote:
> > > >>>
> > > >>> >
> > > >>> > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458 >
> > > >>> >
> > > >>> > Hi John,
> > > >>> >
> > > >>> > I did try that. I rewrote the ADECK file with the storm
id added
> > at
> > > >>> the
> > > >>> > end. I'm pasting an example below:
> > > >>> > AL, 13, 2019092400, 03, PRNA, 012, 118N, 282W, 50, 993,
XX,
> 34,
> > > >>> NEQ,
> > > >>> > 0168, 0115, 0046, 0134, 1013, 161, 111, AL132019
> > > >>> >
> > > >>> > You can see that the last entry, which was not included in
the
> > > original
> > > >>> > file, has the storm id of AL132019. I'm not very familiar
with
> > ADECK
> > > >>> v.
> > > >>> > BDECK files. Do you have an example of ADECK file format
along
> > with
> > > >>> the
> > > >>> > header info? I can try to rewrite the files following the
> > > appropriate
> > > >>> > order/format, and from there pass the new ADECK file into
> TCPairs.
> > > >>> >
> > > >>> > On Fri, Aug 28, 2020 at 5:50 PM John Halley Gotway via RT
<
> > > >>> > met_help at ucar.edu>
> > > >>> > wrote:
> > > >>> >
> > > >>> > > Hi Ed,
> > > >>> > >
> > > >>> > > You are correct. Generally the STORM_NAME is not present
in the
> > > ADECK
> > > >>> > file.
> > > >>> > > It only appears in the BDECK file.
> > > >>> > >
> > > >>> > > Please try configuring it by specifying a STORM_ID. MET
> > constructs
> > > >>> the
> > > >>> > > storm id on the fly from the input ATCF data... as the
2-digit
> > > basin,
> > > >>> > > followed by the 2 digit storm number, followed by the 4
digit
> > year
> > > >>> (e.g.
> > > >>> > > AL092011 for the 9 Atlantic Hurricane storm of 2011).
> > > >>> > >
> > > >>> > > So you can specify the storm_id in the config file, but
not the
> > > storm
> > > >>> > name.
> > > >>> > >
> > > >>> > > Thanks,
> > > >>> > > John
> > > >>> > >
> > > >>> > > On Fri, Aug 28, 2020 at 3:45 PM Edward Strobach - NOAA
> Affiliate
> > > via
> > > >>> RT <
> > > >>> > > met_help at ucar.edu> wrote:
> > > >>> > >
> > > >>> > > >
> > > >>> > > > <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458
> > >
> > > >>> > > >
> > > >>> > > > It appears that even after adding storm_id/storm_name,
that
> met
> > > is
> > > >>> not
> > > >>> > > able
> > > >>> > > > to identify a track in the file provided. I believe
it's
> > because
> > > >>> > neither
> > > >>> > > > the storm name or storm ID is written in the ADECK
file! I
> may
> > > >>> have to
> > > >>> > > > write a script that parses the text file and writes in
the
> > storm
> > > >>> name
> > > >>> > and
> > > >>> > > > id in two separate columns, respectively. Here's a
line
> > example
> > > >>> with
> > > >>> > > what
> > > >>> > > > it currently looks like for one row of the ADECK file:
> > > >>> > > >
> > > >>> > > > AL, 13, 2019092500, 03, PRNA, 000, 133N, 313W, 58,
993,
> XX,
> > > 50,
> > > >>> > NEQ,
> > > >>> > > > 0054, 0000, 0000, 0052, 1013, 172, 32
> > > >>> > > >
> > > >>> > > > If I add AL132019 and LORENZO as the last two columns
of this
> > > row,
> > > >>> then
> > > >>> > > > should that remedy the problem that I have. Do you
think
> that
> > > >>> would
> > > >>> > > > resolve it?
> > > >>> > > >
> > > >>> > > > On Fri, Aug 28, 2020 at 4:53 PM Edward Strobach - NOAA
> > Affiliate
> > > <
> > > >>> > > > edward.strobach at noaa.gov> wrote:
> > > >>> > > >
> > > >>> > > > > It seems to me, though, that these options are
rather
> > > >>> redundant. If
> > > >>> > > one
> > > >>> > > > > supplies a file with the Basin name, storm id, and
storm
> > name,
> > > >>> then
> > > >>> > it
> > > >>> > > > > would seem to me that all one would need to do is
supply
> the
> > > >>> storm id
> > > >>> > > (or
> > > >>> > > > > name) with everything else being extracted from that
using
> > the
> > > >>> > contents
> > > >>> > > > in
> > > >>> > > > > the file.
> > > >>> > > > >
> > > >>> > > > > On Fri, Aug 28, 2020 at 4:41 PM Edward Strobach -
NOAA
> > > Affiliate
> > > >>> <
> > > >>> > > > > edward.strobach at noaa.gov> wrote:
> > > >>> > > > >
> > > >>> > > > >> Thanks for that. I left the TC_PAIRS_CYCLONE,
> > TC_PAIRS_BASIN,
> > > >>> > > > >> TC_PAIRS_STORM_NAME, TC_PAIRS_STORM_ID alone. I
may have
> to
> > > >>> > develop a
> > > >>> > > > way
> > > >>> > > > >> to fill this information in without manually
specifying
> it.
> > > If
> > > >>> that
> > > >>> > > > proves
> > > >>> > > > >> futile, then I'll take a look at using the GFDL
vortex
> > > tracker.
> > > >>> > > Thanks
> > > >>> > > > >>
> > > >>> > > > >> On Thu, Aug 27, 2020 at 3:37 PM George McCabe via
RT <
> > > >>> > > met_help at ucar.edu
> > > >>> > > > >
> > > >>> > > > >> wrote:
> > > >>> > > > >>
> > > >>> > > > >>> Hi Edward,
> > > >>> > > > >>>
> > > >>> > > > >>> From John HG:
> > > >>> > > > >>>
> > > >>> > > > >>>
> > > >>> > > > >>>
> > > >>> > > > >>> *The MET-TC tools read track files in ATCF format.
Those
> > > track
> > > >>> > files
> > > >>> > > > can
> > > >>> > > > >>> be
> > > >>> > > > >>> generated from gridded model output by running
software.
> > One
> > > >>> such
> > > >>> > > > tracker
> > > >>> > > > >>> is supported to the community by the DTC and is
named the
> > > "GFDL
> > > >>> > > Vortex
> > > >>> > > > >>> Tracker":
> > > >>> http://dtcenter.org/community-code/gfdl-vortex-tracker
> > > >>> > > > >>> <http://dtcenter.org/community-code/gfdl-vortex-
tracker
> >*
> > > >>> > > > >>> *It is not expressly part of METplus but would be
very
> > useful
> > > >>> for
> > > >>> > > > >>> tracking
> > > >>> > > > >>> storms in GRIB files.*
> > > >>> > > > >>>
> > > >>> > > > >>> My guess as to why none of the tracks are being
used has
> to
> > > do
> > > >>> with
> > > >>> > > the
> > > >>> > > > >>> METplus configurations to filter data. I would
check the
> > > >>> values for
> > > >>> > > > >>> TC_PAIRS_CYCLONE, TC_PAIRS_BASIN,
TC_PAIRS_STORM_NAME,
> > > >>> > > > TC_PAIRS_STORM_ID,
> > > >>> > > > >>> and MODEL. The values you put in these lists
determine
> > which
> > > >>> storms
> > > >>> > > to
> > > >>> > > > >>> use.
> > > >>> > > > >>> You can leave them blank to use any.
> > > >>> > > > >>>
> > > >>> > > > >>> Also, you could bump up the MET log verbosity and
see
> more
> > > >>> > > information
> > > >>> > > > >>> about why they are being rejected:
> > > >>> > > > >>>
> > > >>> > > > >>> # to affect all MET tools set:
> > > >>> > > > >>> LOG_MET_VERBOSITY = 10
> > > >>> > > > >>>
> > > >>> > > > >>> # to affect only TCPairs set:
> > > >>> > > > >>> LOG_TC_PAIRS_VERBOSITY = 10
> > > >>> > > > >>>
> > > >>> > > > >>> - George
> > > >>> > > > >>>
> > > >>> > > > >>> On Thu, Aug 27, 2020 at 10:53 AM Edward Strobach -
NOAA
> > > >>> Affiliate
> > > >>> > via
> > > >>> > > > RT
> > > >>> > > > >>> <
> > > >>> > > > >>> met_help at ucar.edu> wrote:
> > > >>> > > > >>>
> > > >>> > > > >>> >
> > > >>> > > > >>> > <URL:
> > > >>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458 >
> > > >>> > > > >>> >
> > > >>> > > > >>> > Hi,
> > > >>> > > > >>> >
> > > >>> > > > >>> > I did decide to remove CyclonePlotter for now
and use
> the
> > > >>> > > > >>> TCPairs_Config
> > > >>> > > > >>> > file along with settings that I set in a
shared_TC.conf
> > > that
> > > >>> I
> > > >>> > > store
> > > >>> > > > >>> in a
> > > >>> > > > >>> > temporary directory to process the ADECK and
BDECK data
> > > that
> > > >>> I
> > > >>> > > have.
> > > >>> > > > >>> Files
> > > >>> > > > >>> > are generated for all the dates that span
INIT_BEG and
> > > >>> INIT_END,
> > > >>> > > > >>> thanks to
> > > >>> > > > >>> > setting loop_by to INIT. However, only the
header is
> > > written
> > > >>> out,
> > > >>> > > > and I
> > > >>> > > > >>> > suspect it is do to what the log indicates here:
> > > >>> > > > >>> >
> > > >>> > > > >>> > DEBUG 4: [File 1 of 5] Used 0 of 168 lines read
from
> file
> > > >>> > > > >>> >
> > > >>> > > > >>> >
> > > >>> > > > >>>
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
> > > >>> > > > >>> > DEBUG 4: [File 2 of 5] Used 0 of 168 lines read
from
> file
> > > >>> > > > >>> >
> > > >>> > > > >>> >
> > > >>> > > > >>>
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
> > > >>> > > > >>> > DEBUG 4: [File 3 of 5] Used 0 of 168 lines read
from
> file
> > > >>> > > > >>> >
> > > >>> > > > >>> >
> > > >>> > > > >>>
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
> > > >>> > > > >>> > DEBUG 4: [File 4 of 5] Used 0 of 168 lines read
from
> file
> > > >>> > > > >>> >
> > > >>> > > > >>> >
> > > >>> > > > >>>
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
> > > >>> > > > >>> > DEBUG 4: [File 5 of 5] Used 0 of 168 lines read
from
> file
> > > >>> > > > >>> >
> > > >>> > > > >>> >
> > > >>> > > > >>>
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
> > > >>> > > > >>> > DEBUG 3: Used 0 of 840 lines read from 5
file(s).
> > > >>> > > > >>> > DEBUG 3: Identified 0 track(s).
> > > >>> > > > >>> > DEBUG 5: TrackInfoArray: NTracks = 0, NAlloc =
0,
> Tracks:
> > > >>> > > > >>> > DEBUG 5:
> > > >>> > > > >>> > DEBUG 2: Deriving 12-hour interpolated ADECK
tracks.
> > > >>> > > > >>> > DEBUG 2: Finished adding 0 and replacing 0
Interp12
> > > track(s).
> > > >>> > > > >>> > DEBUG 2: Deriving 0 ADECK consensus model(s).
> > > >>> > > > >>> > DEBUG 2: Added 0 ADECK consensus tracks(s).
> > > >>> > > > >>> > DEBUG 2: Deriving 0 ADECK lag model(s).
> > > >>> > > > >>> > DEBUG 2: Added 0 ADECK lag tracks(s).
> > > >>> > > > >>> > DEBUG 2: Deriving 0 CLIPER/SHIFOR baseline
model(s).
> > > >>> > > > >>> > DEBUG 2: Added 0 CLIPER/SHIFOR baseline
track(s).
> > > >>> > > > >>> > DEBUG 2: Filtering 0 ADECK tracks based on
config file
> > > >>> settings.
> > > >>> > > > >>> > DEBUG 3: Total tracks read = 0
> > > >>> > > > >>> > DEBUG 3: Total tracks kept = 0
> > > >>> > > > >>> > DEBUG 3: Rejected for storm name = 0
> > > >>> > > > >>> > DEBUG 3: Rejected for valid time = 0
> > > >>> > > > >>> > DEBUG 3: Rejected for required lead times = 0
> > > >>> > > > >>> > DEBUG 3: Rejected for init mask = 0
> > > >>> > > > >>> > DEBUG 3: Rejected for valid mask = 0
> > > >>> > > > >>> > DEBUG 2: Matching 0 ADECK tracks to 1 BDECK
tracks.
> > > >>> > > > >>> >
> > > >>> > > > >>> > I'm still thinking that the formatting is off in
this
> > file.
> > > >>> I
> > > >>> > > > surmise
> > > >>> > > > >>> that
> > > >>> > > > >>> > it's not able to group the data because the
column
> > > >>> information is
> > > >>> > > not
> > > >>> > > > >>> in
> > > >>> > > > >>> > the order that met would want it to be in.
These files
> > do
> > > >>> not
> > > >>> > have
> > > >>> > > > any
> > > >>> > > > >>> > header info, so it has to rely on the correct
ordering,
> > or
> > > >>> so I
> > > >>> > > > >>> suspect.
> > > >>> > > > >>> > My interpretation, however, may be flawed. I
see 6
> > > separate
> > > >>> TC
> > > >>> > > based
> > > >>> > > > >>> > metplus routines: TCPairs (I'm working on
currently);
> > > >>> > > CyclonePlotter
> > > >>> > > > >>> > (looking to use); TCStat; TCRMW; TCMPRPlotter;
and
> TCGen.
> > > >>> Much,
> > > >>> > if
> > > >>> > > > not
> > > >>> > > > >>> > almost all of these relies on TCPairs. Is there
a
> > routine
> > > >>> that
> > > >>> > I'm
> > > >>> > > > >>> missing
> > > >>> > > > >>> > in metplus that takes a grib file and constructs
ADECK
> > > files?
> > > >>> > > > >>> >
> > > >>> > > > >>> >
> > > >>> > > > >>> >
> > > >>> > > > >>> > On Thu, Aug 27, 2020 at 11:52 AM Edward Strobach
- NOAA
> > > >>> > Affiliate <
> > > >>> > > > >>> > edward.strobach at noaa.gov> wrote:
> > > >>> > > > >>> >
> > > >>> > > > >>> > > Hi George, I figured the BDECK must be
compatible, so
> > no
> > > >>> > worries
> > > >>> > > > >>> there.
> > > >>> > > > >>> > > Thanks for the link and the heads up about
> > > reformatting. I
> > > >>> > will
> > > >>> > > > use
> > > >>> > > > >>> the
> > > >>> > > > >>> > > example to modify what I currently have to see
the
> > > outcome.
> > > >>> > > Thanks
> > > >>> > > > >>> > again.
> > > >>> > > > >>> > >
> > > >>> > > > >>> > > On Wed, Aug 26, 2020 at 6:57 PM George McCabe
via RT
> <
> > > >>> > > > >>> met_help at ucar.edu>
> > > >>> > > > >>> > > wrote:
> > > >>> > > > >>> > >
> > > >>> > > > >>> > >> Hi Edward,
> > > >>> > > > >>> > >>
> > > >>> > > > >>> > >> The BDECK data is compatible to be processed.
The
> use
> > > >>> case you
> > > >>> > > > used
> > > >>> > > > >>> as
> > > >>> > > > >>> > an
> > > >>> > > > >>> > >> example uses extra tropical cyclone data from
SBU
> that
> > > is
> > > >>> > > > >>> reformatted by
> > > >>> > > > >>> > >> TCPairs wrapper before processing. You will
see your
> > > >>> config
> > > >>> > file
> > > >>> > > > >>> has the
> > > >>> > > > >>> > >> following:
> > > >>> > > > >>> > >>
> > > >>> > > > >>> > >> ## tc-pairs filtering
optionsTC_PAIRS_REFORMAT_DECK
> =
> > > >>> > > > >>> > >> yesTC_PAIRS_REFORMAT_TYPE = SBU
> > > >>> > > > >>> > >>
> > > >>> > > > >>> > >> Changing TC_PAIRS_REFORMAT_DECK to 'no' or
'false'
> > will
> > > >>> skip
> > > >>> > the
> > > >>> > > > >>> > >> reformatting step. There is another use case
in the
> > repo
> > > >>> that
> > > >>> > > uses
> > > >>> > > > >>> data
> > > >>> > > > >>> > >> that looks like yours (at least as far as I
can
> tell).
> > > >>> > > > >>> > >>
> > > >>> > > > >>> > >>
> > > >>> > > > >>> > >>
> > > >>> > > > >>> >
> > > >>> > > > >>>
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
https://dtcenter.github.io/METplus/generated/met_tool_wrapper/TCPairs/TCPairs_tropical.html#metplus-
configuration
> > > >>> > > > >>> > >>
> > > >>> > > > >>> > >> This use case uses BDECK data in
> > > met_test/new/hwrf/bdeck.
> > > >>> Here
> > > >>> > > is
> > > >>> > > > >>> part
> > > >>> > > > >>> > of
> > > >>> > > > >>> > >> the file called bal062018.dat:
> > > >>> > > > >>> > >>
> > > >>> > > > >>> > >> AL, 06, 2018082906, , BEST, 0, 129N,
109W, 10,
> > > >>> 1010, DB,
> > > >>> > > > >>> 0, ,
> > > >>> > > > >>> > >> 0, 0, 0, 0, 0, 0, 0, 0,
0,
> ,
> > > 0,
> > > >>> > ,
> > > >>> > > > 0,
> > > >>> > > > >>> > 0,
> > > >>> > > > >>> > >> GENESIS019, , 0, , 0, 0, 0,
0,
> > > >>> genesis-num,
> > > >>> > > 019,
> > > >>> > > > >>> > >>
> > > >>> > > > >>> > >>
> > > >>> > > > >>> > >> Let me know if changing that setting doesn't
work
> and
> > I
> > > >>> can
> > > >>> > > take a
> > > >>> > > > >>> > closer
> > > >>> > > > >>> > >> look.
> > > >>> > > > >>> > >>
> > > >>> > > > >>> > >> Thanks,
> > > >>> > > > >>> > >> George
> > > >>> > > > >>> > >>
> > > >>> > > > >>> > >> On Wed, Aug 26, 2020 at 1:17 PM Edward
Strobach -
> NOAA
> > > >>> > Affiliate
> > > >>> > > > >>> via RT
> > > >>> > > > >>> > <
> > > >>> > > > >>> > >> met_help at ucar.edu> wrote:
> > > >>> > > > >>> > >>
> > > >>> > > > >>> > >> >
> > > >>> > > > >>> > >> > Wed Aug 26 13:13:28 2020: Request 96458 was
acted
> > > upon.
> > > >>> > > > >>> > >> > Transaction: Ticket created by
> > > edward.strobach at noaa.gov
> > > >>> > > > >>> > >> > Queue: met_help
> > > >>> > > > >>> > >> > Subject: Tropical Cyclone
verification:
> TCPairs
> > > and
> > > >>> > > > >>> > CyclonePlotter
> > > >>> > > > >>> > >> > Owner: Nobody
> > > >>> > > > >>> > >> > Requestors: edward.strobach at noaa.gov
> > > >>> > > > >>> > >> > Status: new
> > > >>> > > > >>> > >> > Ticket <URL:
> > > >>> > > > >>>
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458
> > > >>> > > > >>> > >
> > > >>> > > > >>> > >> >
> > > >>> > > > >>> > >> >
> > > >>> > > > >>> > >> > Good afternoon,
> > > >>> > > > >>> > >> >
> > > >>> > > > >>> > >> > I found it relatively straightforward to
set-up
> some
> > > >>> scripts
> > > >>> > > to
> > > >>> > > > >>> > process
> > > >>> > > > >>> > >> > forecast output along with obs by using
this
> > resource:
> > > >>> > > > >>> > >> >
> > > >>> > > > >>> > >> >
> > > >>> > > > >>> > >>
> > > >>> > > > >>> >
> > > >>> > > > >>>
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
https://dtcenter.github.io/METplus/generated/model_applications/tc_and_extra_tc/Plotter_fcstGFS_obsGFS_ExtraTC.html#sphx-
glr-generated-model-applications-tc-and-extra-tc-plotter-fcstgfs-
obsgfs-extratc-py
> > > >>> > > > >>> > >> > .
> > > >>> > > > >>> > >> > I have been running the global model which
> generates
> > > >>> track
> > > >>> > > data
> > > >>> > > > >>> for
> > > >>> > > > >>> > the
> > > >>> > > > >>> > >> > post processing step following this naming
> > convention:
> > > >>> > > > >>> > >> > atcfunix.gfs.YYYYMMDDHH. The best track
data is
> > > stored
> > > >>> here
> > > >>> > > on
> > > >>> > > > >>> WCOSS:
> > > >>> > > > >>> > >> >
> > > >>> /gpfs/hps3/emc/hwrf/noscrub/emc.hurpara/trak/abdeck/btk/.
> > > >>> > > > >>> > >> >
> > > >>> > > > >>> > >> > I have a python script that determines
which file
> to
> > > use
> > > >>> > based
> > > >>> > > > on
> > > >>> > > > >>> > basin
> > > >>> > > > >>> > >> > information, storm name, and date. That is
piped
> > into
> > > >>> the
> > > >>> > > > >>> > >> > TC_PAIRS_BDECK_TEMPLATE while the forecast
file
> > above
> > > is
> > > >>> > piped
> > > >>> > > > >>> into
> > > >>> > > > >>> > the
> > > >>> > > > >>> > >> > TC_PAIRS_ADECK_TEMPLATE this way:
> > > >>> > > > >>> atcfunix.gfs.{date?fmt=%Y%m%d%H}.
> > > >>> > > > >>> > >> >
> > > >>> > > > >>> > >> > Inside my main script
> > > >>> > > > >>> > >> >
> > > >>> > > > >>> > >> >
> > > >>> > > > >>> > >>
> > > >>> > > > >>> >
> > > >>> > > > >>>
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
*(/gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/METplus/scripts/run_metplus_TCPlotter.sh*,
> > > >>> > > > >>> > >> > I have three configure files that being
processed
> > like
> > > >>> so:*
> > > >>> > > > >>> > >> > ${MET_PLUS}/ush/master_metplus.py -c
> > > >>> > > > >>> > >> >
> ${MET_PLUS_CONF}/Plotter_fcstGFS_obsGFS_ExtraTC.conf
> > > -c
> > > >>> > > > >>> > >> > ${MET_PLUS_TMP}/shared.conf_TC -c
> > > >>> > > > ${MET_PLUS_CONF}/system_TC.conf*
> > > >>> > > > >>> > >> >
> > > >>> > > > >>> > >> > The system_TC.conf is of little concern
because it
> > is
> > > >>> mostly
> > > >>> > > > >>> unchanged
> > > >>> > > > >>> > >> > except for the OUTPUT_BASE, which is
changed
> > > >>> specifically to
> > > >>> > > > >>> where I
> > > >>> > > > >>> > >> plan
> > > >>> > > > >>> > >> > on storing the TC output. The
shared_TC.conf, on
> > the
> > > >>> other
> > > >>> > > > hand,
> > > >>> > > > >>> is
> > > >>> > > > >>> > >> used
> > > >>> > > > >>> > >> > to replace the content inside my
> > > >>> > > > >>> Plotter_fcstGFS_obsGFS_ExtraTC.conf
> > > >>> > > > >>> > >> file.
> > > >>> > > > >>> > >> > In here I define
> > > >>> > > > >>> > >> > the ADECK and BDECK input directories based
on the
> > > >>> where the
> > > >>> > > > >>> template
> > > >>> > > > >>> > >> files
> > > >>> > > > >>> > >> > above are being stored. Since I have
several runs
> > > >>> > > encompassing
> > > >>> > > > >>> the
> > > >>> > > > >>> > >> > tropical cyclone event - 20190924 through
20190929
> > - I
> > > >>> have
> > > >>> > > > >>> specified
> > > >>> > > > >>> > my
> > > >>> > > > >>> > >> > INIT_BEG as 20190924 and my INIT_END as
> 20190929. I
> > > >>> have
> > > >>> > also
> > > >>> > > > >>> set my
> > > >>> > > > >>> > >> > INIT_INCREMENT to 86400 since I'm only
processing
> > the
> > > >>> 00z
> > > >>> > > cycle
> > > >>> > > > >>> for
> > > >>> > > > >>> > >> now. I
> > > >>> > > > >>> > >> > have a couple of lines of code before the
> configure
> > > file
> > > >>> > that
> > > >>> > > > >>> joins
> > > >>> > > > >>> > >> these
> > > >>> > > > >>> > >> > dates in a comma separated list such that
the
> > > >>> > > > >>> > CYCLONE_PLOTTER_INIT_DATE
> > > >>> > > > >>> > >> is
> > > >>> > > > >>> > >> > "20190924,20190925,...,20190929". I'm not
sure if
> > > that
> > > >>> is
> > > >>> > > > allowed
> > > >>> > > > >>> > >> though.
> > > >>> > > > >>> > >> >
> > > >>> > > > >>> > >> > When running the script the log that is
generated
> > > >>> produces 0
> > > >>> > > > >>> errors.
> > > >>> > > > >>> > >> > However, I find *two issues*: *1) there is
only
> one
> > > >>> file
> > > >>> > > being
> > > >>> > > > >>> > carried
> > > >>> > > > >>> > >> > over to generate results; that is the
20190924
> date
> > -
> > > >>> there
> > > >>> > > > >>> should be
> > > >>> > > > >>> > >> 6*,
> > > >>> > > > >>> > >> > and *2) The files that are created are
empty and
> > thus
> > > >>> the
> > > >>> > plot
> > > >>> > > > >>> that is
> > > >>> > > > >>> > >> > generated is simply a map with no data
overlaid.*
> > > >>> > > > >>> > >> >
> > > >>> > > > >>> > >> > In attempting to debug further, I found
something
> > that
> > > >>> looks
> > > >>> > > > >>> important
> > > >>> > > > >>> > >> > shown here:
> > > >>> > > > >>> > >> >
> > > >>> > > > >>> > >> > DEBUG 3: Rejected for init mask =
0
> > > >>> > > > >>> > >> > DEBUG 3: Rejected for valid mask =
0
> > > >>> > > > >>> > >> > *DEBUG 2: Matching 0 ADECK tracks to 0
BDECK
> > tracks.*
> > > >>> > > > >>> > >> > DEBUG 1: Watch/Warning file:
> > > >>> > > > >>> > >> >
> > > >>> > > > >>> > >> >
> > > >>> > > > >>> > >>
> > > >>> > > > >>> >
> > > >>> > > > >>>
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
/gpfs/dell2/emc/verification/noscrub/Julie.Prestopnik/met/9.1_beta2/share/met/tc_data/wwpts_us.txt
> > > >>> > > > >>> > >> > DEBUG 5: TrackPairInfoArray: NPairs = 0,
NAlloc =
> 0,
> > > >>> Pairs:
> > > >>> > > > >>> > >> > DEBUG 5:
> > > >>> > > > >>> > >> > DEBUG 1: Output file:
> > > >>> > > > >>> > >> >
> > > >>> > > > >>> > >> >
> > > >>> > > > >>> > >>
> > > >>> > > > >>> >
> > > >>> > > > >>>
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs/201909/AL_q2019092400.gfs.PRNA20.tcst
> > > >>> > > > >>> > >> > 08/26 18:35:12.422 metplus.TCPairs
> > > >>> (command_runner.py:191)
> > > >>> > > > DEBUG:
> > > >>> > > > >>> > >> Finished
> > > >>> > > > >>> > >> > running
> > > >>> > > > >>> > >> >
> > > >>> > > > >>> > >> >
> > > >>> > > > >>> > >>
> > > >>> > > > >>> >
> > > >>> > > > >>>
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
/gpfs/dell2/emc/verification/noscrub/Julie.Prestopnik/met/9.1_beta2/bin/tc_pairs
> > > >>> > > > >>> > >> > in 0:00:00.607349
> > > >>> > > > >>> > >> > 08/26 18:35:12.422 metplus.CyclonePlotter
> > > >>> > > > >>> > >> (cyclone_plotter_wrapper.py:77)
> > > >>> > > > >>> > >> > DEBUG: Begin retrieving data...
> > > >>> > > > >>> > >> > 08/26 18:35:12.422 metplus.CyclonePlotter
> > > >>> > > > >>> > >> (cyclone_plotter_wrapper.py:83)
> > > >>> > > > >>> > >> > DEBUG: Generate plot for all files in the
> > > >>> > > > >>> > >> >
> > > >>> > > > >>> > >> >
> > > >>> > > > >>> > >>
> > > >>> > > > >>> >
> > > >>> > > > >>>
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
directory/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs
> > > >>> > > > >>> > >> > 08/26 18:35:12.423 metplus.CyclonePlotter
> > > >>> > > > >>> > >> (cyclone_plotter_wrapper.py:98)
> > > >>> > > > >>> > >> > DEBUG: Parsing file
> > > >>> > > > >>> > >> >
> > > >>> > > > >>> > >> >
> > > >>> > > > >>> > >>
> > > >>> > > > >>> >
> > > >>> > > > >>>
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs/201909/AL_q2019092400.gfs.PRNA20.tcst
> > > >>> > > > >>> > >> > 08/26 18:35:19.778 metplus
> (config_launcher.py:520)
> > > >>> DEBUG:
> > > >>> > > > Setting
> > > >>> > > > >>> > >> [config]
> > > >>> > > > >>> > >> > SCRUB_STAGING_DIR to default value: False.
> > > >>> > > > >>> > >> > 08/26 18:35:19.779 metplus INFO:
> > > >>> > > > >>> > >> >
> > > >>> > > > >>> > >> >
> > > >>> > > > >>> > >>
> > > >>> > > > >>> >
> > > >>> > > > >>>
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/metplus_final.conf:
> > > >>> > > > >>> > >> > write metplus.conf here
> > > >>> > > > >>> > >> > 08/26 18:35:19.783 metplus
(met_util.py:213)
> DEBUG:
> > > >>> METplus
> > > >>> > > took
> > > >>> > > > >>> > >> > 0:00:08.783240 to run.
> > > >>> > > > >>> > >> > 08/26 18:35:19.783 metplus INFO: METplus
has
> > > >>> successfully
> > > >>> > > > finished
> > > >>> > > > >>> > >> running.
> > > >>> > > > >>> > >> >
> > > >>> > > > >>> > >> > Specifically the line that says matching 0
ADECK
> > > tracks
> > > >>> to 0
> > > >>> > > > BDECK
> > > >>> > > > >>> > >> tracks
> > > >>> > > > >>> > >> > lends me to believe that this is the reason
why
> > > nothing
> > > >>> is
> > > >>> > > being
> > > >>> > > > >>> added
> > > >>> > > > >>> > >> to
> > > >>> > > > >>> > >> > my TC_PAIRS_OUTPUT_TEMPLATE file, and
therefore no
> > > plots
> > > >>> > > > >>> generated on
> > > >>> > > > >>> > a
> > > >>> > > > >>> > >> > map.
> > > >>> > > > >>> > >> >
> > > >>> > > > >>> > >> > What's interesting is the file that is
extracted
> for
> > > >>> BDECK
> > > >>> > is
> > > >>> > > > for
> > > >>> > > > >>> > >> hurricane
> > > >>> > > > >>> > >> > Lorenzo and includes all the dates
coinciding with
> > the
> > > >>> ADECK
> > > >>> > > > file
> > > >>> > > > >>> - a
> > > >>> > > > >>> > >> case
> > > >>> > > > >>> > >> > that is run coinciding with Lorenzo. I've
checked
> > > this.
> > > >>> > > > >>> > >> >
> > > >>> > > > >>> > >> > One possible issue is the formatting of
these
> files.
> > > I
> > > >>> > looked
> > > >>> > > > in
> > > >>> > > > >>> the
> > > >>> > > > >>> > >> test
> > > >>> > > > >>> > >> > case that comes with the cloned met stuff.
In
> there
> > > the
> > > >>> > > > >>> header/field
> > > >>> > > > >>> > >> info
> > > >>> > > > >>> > >> > looks something like this:
> > > >>> > > > >>> > >> >
> > > >>> > > > >>> > >> >
> > > >>> > > > >>> > >> >
> > > >>> > > > >>> > >>
> > > >>> > > > >>> >
> > > >>> > > > >>>
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
*/gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/METplus/source_data/met_test/new/track_data/201412/amlq2014123118.gfso.0104*
> > > >>> > > > >>> > >> >
> > > >>> > > > >>> > >> > ML, 0104, 2014121318_F000_478N_1780W_FOF,
> > 2014121318,
> > > >>> 03,
> > > >>> > > GFSO,
> > > >>> > > > >>> 000,
> > > >>> > > > >>> > >> 478N,
> > > >>> > > > >>> > >> > 1780W, 38, 986, XX, 34, NEQ, 0283, 0640,
0645,
> > > 0000,
> > > >>> > 990,
> > > >>> > > > >>> 133,
> > > >>> > > > >>> > -99,
> > > >>> > > > >>> > >> > 69, 227, -99, -9999, -9999, 208, 315,
135,
> 156,
> > > >>> 0063
> > > >>> > > > >>> > >> >
> > > >>> > > > >>> > >> > This differs from the structure of both the
ADECK
> > and
> > > >>> BDECK
> > > >>> > > > input
> > > >>> > > > >>> > files:
> > > >>> > > > >>> > >> >
> > > >>> > > > >>> > >> > *BDECK:
> > > >>> > > > >>> > >> >
> > > >>> > > > >>>
> > > >>> >
> >
/gpfs/hps3/emc/hwrf/noscrub/emc.hurpara/trak/abdeck/btk/bal132019.dat
> > > >>> > > > >>> > *
> > > >>> > > > >>> > >> >
> > > >>> > > > >>> > >> > AL, 13, 2019092218, , BEST, 0, 109N,
188W,
> 25,
> > > >>> 1008,
> > > >>> > LO,
> > > >>> > > > >>> 0,
> > > >>> > > > >>> > ,
> > > >>> > > > >>> > >> > 0, 0, 0, 0, 1012, 100, 90, 40,
0,
> > L,
> > > >>> 0,
> > > >>> > > ,
> > > >>> > > > >>> 0,
> > > >>> > > > >>> > >> 0,
> > > >>> > > > >>> > >> > INVEST, S, 0, , 0, 0, 0,
0,
> > > >>> genesis-num,
> > > >>> > > 027,
> > > >>> > > > >>> > >> >
> > > >>> > > > >>> > >> > *ADECK:
> > > >>> > > > >>> > >> >
> > > >>> > > > >>> > >> >
> > > >>> > > > >>> > >>
> > > >>> > > > >>> >
> > > >>> > > > >>>
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400*
> > > >>> > > > >>> > >> >
> > > >>> > > > >>> > >> > AL, 0912, 03, PRNA, 000, 157N, 655W, 27,
1008,
> XX,
> > > >>> 34,
> > > >>> > NEQ,
> > > >>> > > > >>> 0000,
> > > >>> > > > >>> > >> 0000,
> > > >>> > > > >>> > >> > 0000, 0000, -9999, -9999, 85
> > > >>> > > > >>> > >> >
> > > >>> > > > >>> > >> > My suspicion is that these formats are not
> > compatible.
> > > >>> > There
> > > >>> > > > are
> > > >>> > > > >>> > major
> > > >>> > > > >>> > >> > differences between these formats in my
opinion.
> > The
> > > >>> ADECK
> > > >>> > > was
> > > >>> > > > >>> > >> generated
> > > >>> > > > >>> > >> > automatically when making a run while BDECK
is
> done
> > by
> > > >>> > someone
> > > >>> > > > >>> else.
> > > >>> > > > >>> > >> Those
> > > >>> > > > >>> > >> > files are stored whenever a storm is
identified.
> > > >>> > > > >>> > >> >
> > > >>> > > > >>> > >> > I think what I'll need to do next is start
by
> > running
> > > >>> TCRMW
> > > >>> > to
> > > >>> > > > >>> produce
> > > >>> > > > >>> > >> the
> > > >>> > > > >>> > >> > ADECK files formatted by metplus. I have
the grib
> > > >>> files for
> > > >>> > > > that.
> > > >>> > > > >>> > The
> > > >>> > > > >>> > >> > question next, though, is the BDECK files
okay for
> > > >>> > processing?
> > > >>> > > > >>> > >> >
> > > >>> > > > >>> > >> >
> > > >>> > > > >>> > >> >
> > > >>> > > > >>> > >> > --
> > > >>> > > > >>> > >> > Edward Strobach
> > > >>> > > > >>> > >> > EMC/NCEP/NWS/
> > > >>> > > > >>> > >> > IMSG Contractor
> > > >>> > > > >>> > >> > Cubicle#: 2029
> > > >>> > > > >>> > >> > 301-683-3717
> > > >>> > > > >>> > >> >
> > > >>> > > > >>> > >> >
> > > >>> > > > >>> > >>
> > > >>> > > > >>> > >> --
> > > >>> > > > >>> > >> George McCabe - Software Engineer III
> > > >>> > > > >>> > >> National Center for Atmospheric Research
> > > >>> > > > >>> > >> Research Applications Laboratory
> > > >>> > > > >>> > >> 303-497-2768
> > > >>> > > > >>> > >> ---
> > > >>> > > > >>> > >> My working day may not be your working day.
Please
> do
> > > not
> > > >>> feel
> > > >>> > > > >>> obliged
> > > >>> > > > >>> > to
> > > >>> > > > >>> > >> reply to this email outside of your normal
working
> > > hours.
> > > >>> > > > >>> > >>
> > > >>> > > > >>> > >>
> > > >>> > > > >>> > >
> > > >>> > > > >>> > > --
> > > >>> > > > >>> > > Edward Strobach
> > > >>> > > > >>> > > EMC/NCEP/NWS/
> > > >>> > > > >>> > > IMSG Contractor
> > > >>> > > > >>> > > Cubicle#: 2029
> > > >>> > > > >>> > > 301-683-3717
> > > >>> > > > >>> > >
> > > >>> > > > >>> >
> > > >>> > > > >>> >
> > > >>> > > > >>> > --
> > > >>> > > > >>> > Edward Strobach
> > > >>> > > > >>> > EMC/NCEP/NWS/
> > > >>> > > > >>> > IMSG Contractor
> > > >>> > > > >>> > Cubicle#: 2029
> > > >>> > > > >>> > 301-683-3717
> > > >>> > > > >>> >
> > > >>> > > > >>> >
> > > >>> > > > >>>
> > > >>> > > > >>> --
> > > >>> > > > >>> George McCabe - Software Engineer III
> > > >>> > > > >>> National Center for Atmospheric Research
> > > >>> > > > >>> Research Applications Laboratory
> > > >>> > > > >>> 303-497-2768
> > > >>> > > > >>> ---
> > > >>> > > > >>> My working day may not be your working day. Please
do not
> > > feel
> > > >>> > > obliged
> > > >>> > > > to
> > > >>> > > > >>> reply to this email outside of your normal working
hours.
> > > >>> > > > >>>
> > > >>> > > > >>>
> > > >>> > > > >>
> > > >>> > > > >> --
> > > >>> > > > >> Edward Strobach
> > > >>> > > > >> EMC/NCEP/NWS/
> > > >>> > > > >> IMSG Contractor
> > > >>> > > > >> Cubicle#: 2029
> > > >>> > > > >> 301-683-3717
> > > >>> > > > >>
> > > >>> > > > >
> > > >>> > > > >
> > > >>> > > > > --
> > > >>> > > > > Edward Strobach
> > > >>> > > > > EMC/NCEP/NWS/
> > > >>> > > > > IMSG Contractor
> > > >>> > > > > Cubicle#: 2029
> > > >>> > > > > 301-683-3717
> > > >>> > > > >
> > > >>> > > >
> > > >>> > > >
> > > >>> > > > --
> > > >>> > > > Edward Strobach
> > > >>> > > > EMC/NCEP/NWS/
> > > >>> > > > IMSG Contractor
> > > >>> > > > Cubicle#: 2029
> > > >>> > > > 301-683-3717
> > > >>> > > >
> > > >>> > > >
> > > >>> > >
> > > >>> > >
> > > >>> >
> > > >>> > --
> > > >>> > Edward Strobach
> > > >>> > EMC/NCEP/NWS/
> > > >>> > IMSG Contractor
> > > >>> > Cubicle#: 2029
> > > >>> > 301-683-3717
> > > >>> >
> > > >>> >
> > > >>>
> > > >>>
> > > >>
> > > >> --
> > > >> Edward Strobach
> > > >> EMC/NCEP/NWS/
> > > >> IMSG Contractor
> > > >> Cubicle#: 2029
> > > >> 301-683-3717
> > > >>
> > > >
> > > >
> > > > --
> > > > Edward Strobach
> > > > EMC/NCEP/NWS/
> > > > IMSG Contractor
> > > > Cubicle#: 2029
> > > > 301-683-3717
> > > >
> > >
> > >
> > > --
> > > Edward Strobach
> > > EMC/NCEP/NWS/
> > > IMSG Contractor
> > > Cubicle#: 2029
> > > 301-683-3717
> > >
> > >
> >
> >
>
>
--
Edward Strobach
EMC/NCEP/NWS/
IMSG Contractor
Cubicle#: 2029
301-683-3717
------------------------------------------------
Subject: Tropical Cyclone verification: TCPairs and CyclonePlotter
From: John Halley Gotway
Time: Wed Sep 02 17:05:30 2020
Ed,
OK, I wrote up the following GitHub issue to track this feature
request:
https://github.com/dtcenter/MET/issues/1481
I assigned it a "Medium" priority and noted that we'd need to figure
out
how to pay for it.
I'll go ahead and resolve this issue.
Thanks,
John
On Tue, Sep 1, 2020 at 5:30 PM Edward Strobach - NOAA Affiliate via RT
<
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458 >
>
> Hi John,
>
> Yes, that would be helpful I think. Even though it took some time
to
> figure out why, especially when I thought that everything was set up
right,
> it came down to a slight difference in model name representation. I
wasn't
> aware of the model name making an impact since I was basing my
diagnosis on
> data filtering options. The other enhancements sound similar and
think
> that they would be also helpful. If a message was pointing to the
source
> of error, that would definitely be helpful.
>
> What I've been doing to improve debugging is going into the
referenced
> python script. Sometimes it helps while other times it doesn't.
That's
> actually what I'm working on right now. If a message does appear
obscure,
> I'll make sure to let you all know. I realize that one of the goals
is to
> have an intuitive log file. I've been pretty happy with many of the
log
> messages since most of them do point out what the issues are.
Thanks
>
> On Tue, Sep 1, 2020 at 7:09 PM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > Ed,
> >
> > OK, I'm trying to catch up on this message history. So it sounds
like
> > tc_pairs was filtering out your track data based on the model
name. From
> > tc_pairs perspective, if the model entry is defined in the config
file,
> > then that filter is applied. If the model entry is left empty,
then all
> > data is used regardless of model name.
> >
> > Since you're running this through METplus, I suppose it was
METplus that
> > was setting the model config option to the string which produced
no
> pairs.
> >
> > This question of "Why am I getting no output data?" comes up in
several
> > contexts. For point-stat, we have users run at verbosity level 3
(-v 3)
> so
> > see counts for why obs were or were not used for each vx task.
Last week,
> > the same question came up for ensemble-stat, but we DO NOT have
the
> reason
> > counting logic there. And here the confusion was caused by
tc_pairs.
> >
> > I assume if you saw a log message saying something like:
> > *Rejected 119 lines based on MODEL name.*
> > That would have pointed you in the right direction much faster.
> >
> > So we could enhance...
> > - Ensemble-Stat to add rejection reason counts when verifying
against
> point
> > obs.
> > - TC-Pairs to add rejection reason counts for which ATCF lines
were used.
> > - TC-Stat to add rejection reason counts for which TCST lines were
used.
> > - Stat-Analysis to add rejection reason counts for which STAT
lines were
> > used.
> >
> > Do you think any of these potential enhancements are worth doing?
> >
> > Thanks,
> > John
> >
> > On Tue, Sep 1, 2020 at 12:05 PM Minna Win via RT
<met_help at ucar.edu>
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458 >
> > >
> > > Hi Ed,
> > >
> > > CyclonePlotter was created to replicate the plots generated by
one of
> our
> > > NOAA partners and not designed to be configurable.
> > >
> > > Regards,
> > > Minna
> > > ---------------
> > > Minna Win
> > > National Center for Atmospheric Research
> > > Developmental Testbed Center
> > > Phone: 303-497-8423
> > > Fax: 303-497-8401
> > >
> > >
> > >
> > > On Tue, Sep 1, 2020 at 11:57 AM Edward Strobach - NOAA Affiliate
via
> RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458
>
> > > >
> > > > I forgot to add something..
> > > >
> > > > What's interesting is that the tutorial going over
PROCESS_LIST =
> > > TCPairs,
> > > > CyclonePlotter indicates that the plot below could be
generated.
> I've
> > > used
> > > > all the settings in the tutorial and never got that plot. Of
course
> > > that
> > > > wouldn't be possible based on the ASCII file that is generated
in the
> > > > CyclonePlotter step. It only contains lat lon info. I'm not
sure
> what
> > > > lead_group = 0 and lead_group = 6 means. It does appear -
according
> to
> > > the
> > > > valid time in the text file - that it does go out to 144
hours, but
> > > > alternates between lead group 0 and lead group 6.
> > > >
> > > > model_name: PRNA storm_id: AL132019 init_time:
20190924_000000
> > > > valid_time: 20190924_000000 lat: 11.4 lon: -25.7
lead_group: 0
> > > > first_point:True
> > > > model_name: PRNA storm_id: AL132019 init_time:
20190924_000000
> > > > valid_time: 20190924_060000 lat: 11.6 lon: -27.4
lead_group: 6
> > > > first_point:False
> > > > model_name: PRNA storm_id: AL132019 init_time:
20190924_000000
> > > > valid_time: 20190924_120000 lat: 11.8 lon: -28.2
lead_group: 0
> > > > first_point:False
> > > > model_name: PRNA storm_id: AL132019 init_time:
20190924_000000
> > > > valid_time: 20190924_180000 lat: 12.7 lon: -28.9
lead_group: 6
> > > > first_point:False
> > > > model_name: PRNA storm_id: AL132019 init_time:
20190924_000000
> > > > valid_time: 20190925_000000 lat: 13.1 lon: -30.6
lead_group: 0
> > > > first_point:False
> > > > model_name: PRNA storm_id: AL132019 init_time:
20190924_000000
> > > > valid_time: 20190925_060000 lat: 13.5 lon: -31.8
lead_group: 6
> > > > first_point:False
> > > > model_name: PRNA storm_id: AL132019 init_time:
20190924_000000
> > > > valid_time: 20190925_120000 lat: 13.6 lon: -33.0
lead_group: 0
> > > > first_point:False
> > > > model_name: PRNA storm_id: AL132019 init_time:
20190924_000000
> > > > valid_time: 20190925_180000 lat: 14.0 lon: -34.0
lead_group: 6
> > > > first_point:False
> > > > model_name: PRNA storm_id: AL132019 init_time:
20190924_000000
> > > > valid_time: 20190926_000000 lat: 14.5 lon: -35.1
lead_group: 0
> > > > first_point:False
> > > > model_name: PRNA storm_id: AL132019 init_time:
20190924_000000
> > > > valid_time: 20190926_060000 lat: 14.8 lon: -36.2
lead_group: 6
> > > > first_point:False
> > > > model_name: PRNA storm_id: AL132019 init_time:
20190924_000000
> > > > valid_time: 20190926_120000 lat: 15.3 lon: -37.0
lead_group: 0
> > > > first_point:False
> > > > model_name: PRNA storm_id: AL132019 init_time:
20190924_000000
> > > > valid_time: 20190926_180000 lat: 16.0 lon: -37.7
lead_group: 6
> > > > first_point:False
> > > > model_name: PRNA storm_id: AL132019 init_time:
20190924_000000
> > > > valid_time: 20190927_000000 lat: 16.9 lon: -38.2
lead_group: 0
> > > > first_point:False
> > > > model_name: PRNA storm_id: AL132019 init_time:
20190924_000000
> > > > valid_time: 20190927_060000 lat: 17.8 lon: -38.7
lead_group: 6
> > > > first_point:False
> > > > model_name: PRNA storm_id: AL132019 init_time:
20190924_000000
> > > > valid_time: 20190927_120000 lat: 18.8 lon: -39.2
lead_group: 0
> > > > first_point:False
> > > > model_name: PRNA storm_id: AL132019 init_time:
20190924_000000
> > > > valid_time: 20190927_180000 lat: 19.8 lon: -39.6
lead_group: 6
> > > > first_point:False
> > > > model_name: PRNA storm_id: AL132019 init_time:
20190924_000000
> > > > valid_time: 20190928_000000 lat: 20.8 lon: -39.8
lead_group: 0
> > > > first_point:False
> > > > model_name: PRNA storm_id: AL132019 init_time:
20190924_000000
> > > > valid_time: 20190928_060000 lat: 21.8 lon: -40.1
lead_group: 6
> > > > first_point:False
> > > > model_name: PRNA storm_id: AL132019 init_time:
20190924_000000
> > > > valid_time: 20190928_120000 lat: 22.8 lon: -40.2
lead_group: 0
> > > > first_point:False
> > > > model_name: PRNA storm_id: AL132019 init_time:
20190924_000000
> > > > valid_time: 20190928_180000 lat: 24.0 lon: -40.4
lead_group: 6
> > > > first_point:False
> > > > model_name: PRNA storm_id: AL132019 init_time:
20190924_000000
> > > > valid_time: 20190929_000000 lat: 25.1 lon: -40.4
lead_group: 0
> > > > first_point:False
> > > > model_name: PRNA storm_id: AL132019 init_time:
20190924_000000
> > > > valid_time: 20190929_060000 lat: 26.4 lon: -40.7
lead_group: 6
> > > > first_point:False
> > > > model_name: PRNA storm_id: AL132019 init_time:
20190924_000000
> > > > valid_time: 20190929_120000 lat: 27.6 lon: -41.2
lead_group: 0
> > > > first_point:False
> > > > model_name: PRNA storm_id: AL132019 init_time:
20190924_000000
> > > > valid_time: 20190929_180000 lat: 28.5 lon: -41.7
lead_group: 6
> > > > first_point:False
> > > > model_name: PRNA storm_id: AL132019 init_time:
20190924_000000
> > > > valid_time: 20190930_000000 lat: 29.3 lon: -42.0
lead_group: 0
> > > > first_point:False
> > > >
> > > > On Tue, Sep 1, 2020 at 1:47 PM Edward Strobach - NOAA
Affiliate <
> > > > edward.strobach at noaa.gov> wrote:
> > > >
> > > > > The good news is I've also got CyclonePlotter to work.
However,
> it
> > > > would
> > > > > appear that CyclonePlotter is really just meant to show a
quick and
> > > dirty
> > > > > plot based on the available settings and the quality of the
plot -
> > > > > attached below.
> > > > >
> > > > > I noticed that I can't process all forecasts at once, rather
I have
> > to
> > > > > loop through each of my forecasts in order to generate plots
for
> > > 20190924
> > > > > through 20190929. Also, I have no real control over grid
lines,
> tick
> > > > > marks, legend to distinguish between best track and model,
etc. As
> > > > such, I
> > > > > may abandon CyclonePlotter.
> > > > >
> > > > > Something else that's a bit odd is that TCPairs is only
grouping
> the
> > > data
> > > > > out to 72 hours instead of 144 hours. There's no apparent
setting
> to
> > > > > extend this to 144 hours according to online sources that
I've
> > > consulted.
> > > > > Is there a way to add an option such as TC_MAX_LEAD so that
met
> knows
> > > to
> > > > > group out to 144 hours?
> > > > >
> > > > > TC_PAIRS_CONFIG_FILE
> > > > > <
> > > >
> > >
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_CONFIG_FILE
> > > > >
> > > > > INIT_HOUR_END
> > > > > <
> > > >
> > >
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
INIT_HOUR_END
> > > > >
> > > > > INIT_INCLUDE
> > > > > <
> > > >
> > >
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
INIT_INCLUDE
> > > > >
> > > > > INIT_EXCLUDE
> > > > > <
> > > >
> > >
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
INIT_EXCLUDE
> > > > >
> > > > > TC_PAIRS_READ_ALL_FILES
> > > > > <
> > > >
> > >
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_READ_ALL_FILES
> > > > >
> > > > > TC_PAIRS_MODEL
> > > > > <
> > > >
> > >
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_MODEL
> > > > >
> > > > > TC_PAIRS_STORM_ID
> > > > > <
> > > >
> > >
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_STORM_ID
> > > > >
> > > > > TC_PAIRS_BASIN
> > > > > <
> > > >
> > >
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_BASIN
> > > > >
> > > > > TC_PAIRS_CYCLONE
> > > > > <
> > > >
> > >
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_CYCLONE
> > > > >
> > > > > TC_PAIRS_STORM_NAME
> > > > > <
> > > >
> > >
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_STORM_NAME
> > > > >
> > > > > TC_PAIRS_DLAND_FILE
> > > > > <
> > > >
> > >
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_DLAND_FILE
> > > > >
> > > > > TC_PAIRS_MISSING_VAL_TO_REPLACE
> > > > > <
> > > >
> > >
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_MISSING_VAL_TO_REPLACE
> > > > >
> > > > > TC_PAIRS_MISSING_VAL
> > > > > <
> > > >
> > >
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_MISSING_VAL
> > > > >
> > > > > TC_PAIRS_SKIP_IF_REFORMAT_EXISTS
> > > > > <
> > > >
> > >
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_SKIP_IF_REFORMAT_EXISTS
> > > > >
> > > > > TC_PAIRS_SKIP_IF_OUTPUT_EXISTS
> > > > > <
> > > >
> > >
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_SKIP_IF_OUTPUT_EXISTS
> > > > >
> > > > > TC_PAIRS_REFORMAT_DECK
> > > > > <
> > > >
> > >
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_REFORMAT_DECK
> > > > >
> > > > > TC_PAIRS_REFORMAT_TYPE
> > > > > <
> > > >
> > >
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_REFORMAT_TYPE
> > > > >
> > > > > TC_PAIRS_CUSTOM_LOOP_LIST
> > > > > <
> > > >
> > >
> >
> https://dtcenter.github.io/METplus/Users_Guide/glossary.html#term-
TC_PAIRS_CUSTOM_LOOP_LIST
> > > > >
> > > > >
> > > > > On Tue, Sep 1, 2020 at 12:43 PM Edward Strobach - NOAA
Affiliate <
> > > > > edward.strobach at noaa.gov> wrote:
> > > > >
> > > > >> Thanks. Those examples are helpful and do confirm that the
files
> > > created
> > > > >> and passed into the TCPairs as ADECK files do follow, more
or
> less,
> > > the
> > > > >> format that's shown in the a{basin}{STORMNUM}{YYYY}.dat
files. I
> > also
> > > > >> found this
> > > > >> <
> > > >
> > >
> >
> https://dtcenter.org/metplus-practical-session-guide-feb-
2019/session-5-trkintfeature-relative/met-tool-tc-pairs
> > > > >
> > > > >> to be helpful as well. When comparing the
values/description in
> my
> > > file
> > > > >>
> > > >
> > >
> >
>
(/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092900)
> > > > >> with the contents in the link, it is clear that the order
of my
> file
> > > > obeys
> > > > >> the following:
> > > > >>
> > > > >> Basin, CY, YYYYMMDDHH, TECHNUM/MIN, TECH, TAU, LATN/S,
LonE/W,
> VMAX,
> > > > >> MSLP, TY, RAD, WINDCODE, RAD1, RAD2, RAD3, RAD4.
> > > > >>
> > > > >> My first line of the referenced file has that in addition
to 3
> > columns
> > > > at
> > > > >> the end - so 20 instead of 17 entries. I assume that the
last 3
> are
> > > > >> ignored.
> > > > >>
> > > > >> *AL, 13, 2019092900, 03, PRNA, 000, 238N, 448W, 85, 962,
XX,
> 34,
> > > > NEQ,
> > > > >> 0185, 0184, 0110, 0148*, 978, 30, 32
> > > > >>
> > > > >> *The good thing is that I figured it out. * Since feedback
is
> > > desirable,
> > > > >> I'm going to go through the process that helped me arrive
at the
> > > > solution
> > > > >> which I detail below.
> > > > >>
> > > > >>
> > > > >> Since there can be more than one storm in the Atlantic at a
given
> > > time,
> > > > I
> > > > >> decided to filter by *TC_PAIRS_CYCLONE* and set that to
*13* since
> > I'm
> > > > >> interested in processing Lorenzo. I can see that it's
defined in
> my
> > > > master
> > > > >> config file as well. I did set the log verbosity to 10 to
retrieve
> > > > >> additional info. There are no error messages; however, it
says
> > > > "*Matching
> > > > >> 0 ADECK tracks to 1 BDECK trac*k". The BDECK track, which
I
> > retrieved
> > > > >> from the best track data formatted as bal132019.dat, is
being
> > > processed
> > > > >> alongside the ADECK files, which are created during the
forecast
> > step.
> > > > >>
> > > > >> Since I've run 6 forecasts that go out to 144 hours, I also
have 6
> > > ADECK
> > > > >> files initialized between 20190924 00Z and 20190929 00Z.
In the
> > case
> > > > of
> > > > >> 20190929 00Z, for example, the log file indicates that the
BDECK
> > track
> > > > is
> > > > >> processed followed by the processing of the ADECK file.
The log
> > > > correctly
> > > > >> identifies 119 lines of processing but says 0 of 119 lines
are
> used:
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >> *DEBUG 5: | | | [3] QuadInfo: Intensity = 64,
ALVal =
> > > > >> -9999.00000, NEVal = -9999.00000, SEVal = -9999.00000,
SWVal =
> > > > -9999.00000,
> > > > >> NWVal = -9999.00000DEBUG 5:DEBUG 5:DEBUG 5:DEBUG 2: Found 1
BDECK
> > > > >> track(s).DEBUG 2: Processing 1 ADECK file(s).DEBUG 4: [File
1 of
> 1]
> > > > Used 0
> > > > >> of 119 lines read from file
> > > > >>
> > > >
> > >
> >
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092900"DEBUG
> > > > >> 3: Used 0 of 119 lines read from 1 file(s).DEBUG 3:
Identified 0
> > > > >> track(s).DEBUG 5: TrackInfoArray: NTracks = 0, NAlloc = 0,
> > > Tracks:DEBUG
> > > > >> 5:DEBUG 2: Deriving 12-hour interpolated ADECK tracks.DEBUG
2:
> > > Finished
> > > > >> adding 0 and replacing 0 Interp12 track(s).DEBUG 2:
Deriving 0
> ADECK
> > > > >> consensus model(s).DEBUG 2: Added 0 ADECK consensus
> tracks(s).DEBUG
> > 2:
> > > > >> Deriving 0 ADECK lag model(s).DEBUG 2: Added 0 ADECK lag
> > > > tracks(s).DEBUG 2:
> > > > >> Deriving 0 CLIPER/SHIFOR baseline model(s).DEBUG 2: Added 0
> > > > CLIPER/SHIFOR
> > > > >> baseline track(s).DEBUG 2: Filtering 0 ADECK tracks based
on
> config
> > > file
> > > > >> settings.DEBUG 3: Total tracks read = 0DEBUG
3:
> Total
> > > > tracks
> > > > >> kept = 0DEBUG 3: Rejected for storm name
=
> > > > 0DEBUG
> > > > >> 3: Rejected for valid time = 0DEBUG 3: Rejected
for
> > required
> > > > lead
> > > > >> times = 0DEBUG 3: Rejected for init mask = 0DEBUG
3:
> > > Rejected
> > > > for
> > > > >> valid mask = 0DEBUG 2: Matching 0 ADECK tracks to
1 BDECK
> > > > tracks.*
> > > > >>
> > > > >> This is repeated for all six forecasts. Also, all lines of
the
> > BDECK
> > > > >> file are processed, so the dates in that file encompass the
dates
> in
> > > > each
> > > > >> of the forecast files.
> > > > >>
> > > > >> I was flummoxed as to why this would be. Under closer
inspection
> I
> > > > found
> > > > >> that the directory - which is named after the model -
includes the
> > > > >> experiment number as well (e.g. prna20). However, inside
the
> ADECK
> > > > files
> > > > >> the model is PRNA, not PRNA20. Met was looking for
instances of
> > > PRNA20,
> > > > >> and when it couldn't it concluded that there were zero
matches.
> > > > However,
> > > > >> when I changed my pathway to "TC_PAIRS_ADECK_INPUT_DIR =
> > > > >>
/gpfs/dell2/emc/modeling/noscrub/${USER}/archive/${model}20" and
> > set
> > > > model
> > > > >> to "prna" instead of "prna20" it worked. I now have
populated
> > files.
> > > > >> Moving forward I will need to keep this subtlety in mind.
I plan
> > to
> > > > use
> > > > >> other TC packages pretty soon here since I've been tasked
to
> examine
> > > > some
> > > > >> physics changes and tune where necessary. I don't know
what type
> of
> > > > issues
> > > > >> I may encounter, but I plan to use this experience as a
model to
> > > > approach
> > > > >> possible issues related to TC verification in the future.
> > > > >>
> > > > >>
> > > > >>
> > > > >> On Mon, Aug 31, 2020 at 3:31 PM John Halley Gotway via RT <
> > > > >> met_help at ucar.edu> wrote:
> > > > >>
> > > > >>> Ed,
> > > > >>>
> > > > >>> Here's a link to the documentation that the Navy provides
about
> the
> > > > ATCF
> > > > >>> file format:
> > > > >>>
> >
https://www.nrlmry.navy.mil/atcf_web/docs/database/new/abrdeck.html
> > > > >>>
> > > > >>> And the best source of sample data is NHC's ftp site:
> > > > >>> https://ftp.nhc.noaa.gov/atcf/
> > > > >>>
> > > > >>> In there...
> > > > >>> *aid_public* contains tar files of ADECKS for all storms
in the
> > > current
> > > > >>> season.
> > > > >>> *btk* has uncompressed BEST track files for all storms in
the
> > current
> > > > >>> season.
> > > > >>> *archive* contains compressed A and B-Deck data for
previous
> > seasons.
> > > > >>>
> > > > >>> It's pretty cool actually that this track data is so
readily
> > > available.
> > > > >>>
> > > > >>> I have never actually modified ATCF files for my purposes.
So
> > > hopefully
> > > > >>> you
> > > > >>> won't need to either... and can just use the output of the
> tracker
> > > code
> > > > >>> directly.
> > > > >>>
> > > > >>> Thanks,
> > > > >>> John
> > > > >>>
> > > > >>>
> > > > >>> On Mon, Aug 31, 2020 at 9:24 AM Edward Strobach - NOAA
Affiliate
> > via
> > > > RT <
> > > > >>> met_help at ucar.edu> wrote:
> > > > >>>
> > > > >>> >
> > > > >>> > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458
> >
> > > > >>> >
> > > > >>> > Hi John,
> > > > >>> >
> > > > >>> > I did try that. I rewrote the ADECK file with the storm
id
> added
> > > at
> > > > >>> the
> > > > >>> > end. I'm pasting an example below:
> > > > >>> > AL, 13, 2019092400, 03, PRNA, 012, 118N, 282W, 50,
993, XX,
> > 34,
> > > > >>> NEQ,
> > > > >>> > 0168, 0115, 0046, 0134, 1013, 161, 111, AL132019
> > > > >>> >
> > > > >>> > You can see that the last entry, which was not included
in the
> > > > original
> > > > >>> > file, has the storm id of AL132019. I'm not very
familiar with
> > > ADECK
> > > > >>> v.
> > > > >>> > BDECK files. Do you have an example of ADECK file
format along
> > > with
> > > > >>> the
> > > > >>> > header info? I can try to rewrite the files following
the
> > > > appropriate
> > > > >>> > order/format, and from there pass the new ADECK file
into
> > TCPairs.
> > > > >>> >
> > > > >>> > On Fri, Aug 28, 2020 at 5:50 PM John Halley Gotway via
RT <
> > > > >>> > met_help at ucar.edu>
> > > > >>> > wrote:
> > > > >>> >
> > > > >>> > > Hi Ed,
> > > > >>> > >
> > > > >>> > > You are correct. Generally the STORM_NAME is not
present in
> the
> > > > ADECK
> > > > >>> > file.
> > > > >>> > > It only appears in the BDECK file.
> > > > >>> > >
> > > > >>> > > Please try configuring it by specifying a STORM_ID.
MET
> > > constructs
> > > > >>> the
> > > > >>> > > storm id on the fly from the input ATCF data... as the
> 2-digit
> > > > basin,
> > > > >>> > > followed by the 2 digit storm number, followed by the
4 digit
> > > year
> > > > >>> (e.g.
> > > > >>> > > AL092011 for the 9 Atlantic Hurricane storm of 2011).
> > > > >>> > >
> > > > >>> > > So you can specify the storm_id in the config file,
but not
> the
> > > > storm
> > > > >>> > name.
> > > > >>> > >
> > > > >>> > > Thanks,
> > > > >>> > > John
> > > > >>> > >
> > > > >>> > > On Fri, Aug 28, 2020 at 3:45 PM Edward Strobach - NOAA
> > Affiliate
> > > > via
> > > > >>> RT <
> > > > >>> > > met_help at ucar.edu> wrote:
> > > > >>> > >
> > > > >>> > > >
> > > > >>> > > > <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458
> > > >
> > > > >>> > > >
> > > > >>> > > > It appears that even after adding
storm_id/storm_name, that
> > met
> > > > is
> > > > >>> not
> > > > >>> > > able
> > > > >>> > > > to identify a track in the file provided. I believe
it's
> > > because
> > > > >>> > neither
> > > > >>> > > > the storm name or storm ID is written in the ADECK
file! I
> > may
> > > > >>> have to
> > > > >>> > > > write a script that parses the text file and writes
in the
> > > storm
> > > > >>> name
> > > > >>> > and
> > > > >>> > > > id in two separate columns, respectively. Here's a
line
> > > example
> > > > >>> with
> > > > >>> > > what
> > > > >>> > > > it currently looks like for one row of the ADECK
file:
> > > > >>> > > >
> > > > >>> > > > AL, 13, 2019092500, 03, PRNA, 000, 133N, 313W, 58,
993,
> > XX,
> > > > 50,
> > > > >>> > NEQ,
> > > > >>> > > > 0054, 0000, 0000, 0052, 1013, 172, 32
> > > > >>> > > >
> > > > >>> > > > If I add AL132019 and LORENZO as the last two
columns of
> this
> > > > row,
> > > > >>> then
> > > > >>> > > > should that remedy the problem that I have. Do you
think
> > that
> > > > >>> would
> > > > >>> > > > resolve it?
> > > > >>> > > >
> > > > >>> > > > On Fri, Aug 28, 2020 at 4:53 PM Edward Strobach -
NOAA
> > > Affiliate
> > > > <
> > > > >>> > > > edward.strobach at noaa.gov> wrote:
> > > > >>> > > >
> > > > >>> > > > > It seems to me, though, that these options are
rather
> > > > >>> redundant. If
> > > > >>> > > one
> > > > >>> > > > > supplies a file with the Basin name, storm id, and
storm
> > > name,
> > > > >>> then
> > > > >>> > it
> > > > >>> > > > > would seem to me that all one would need to do is
supply
> > the
> > > > >>> storm id
> > > > >>> > > (or
> > > > >>> > > > > name) with everything else being extracted from
that
> using
> > > the
> > > > >>> > contents
> > > > >>> > > > in
> > > > >>> > > > > the file.
> > > > >>> > > > >
> > > > >>> > > > > On Fri, Aug 28, 2020 at 4:41 PM Edward Strobach -
NOAA
> > > > Affiliate
> > > > >>> <
> > > > >>> > > > > edward.strobach at noaa.gov> wrote:
> > > > >>> > > > >
> > > > >>> > > > >> Thanks for that. I left the TC_PAIRS_CYCLONE,
> > > TC_PAIRS_BASIN,
> > > > >>> > > > >> TC_PAIRS_STORM_NAME, TC_PAIRS_STORM_ID alone. I
may
> have
> > to
> > > > >>> > develop a
> > > > >>> > > > way
> > > > >>> > > > >> to fill this information in without manually
specifying
> > it.
> > > > If
> > > > >>> that
> > > > >>> > > > proves
> > > > >>> > > > >> futile, then I'll take a look at using the GFDL
vortex
> > > > tracker.
> > > > >>> > > Thanks
> > > > >>> > > > >>
> > > > >>> > > > >> On Thu, Aug 27, 2020 at 3:37 PM George McCabe via
RT <
> > > > >>> > > met_help at ucar.edu
> > > > >>> > > > >
> > > > >>> > > > >> wrote:
> > > > >>> > > > >>
> > > > >>> > > > >>> Hi Edward,
> > > > >>> > > > >>>
> > > > >>> > > > >>> From John HG:
> > > > >>> > > > >>>
> > > > >>> > > > >>>
> > > > >>> > > > >>>
> > > > >>> > > > >>> *The MET-TC tools read track files in ATCF
format.
> Those
> > > > track
> > > > >>> > files
> > > > >>> > > > can
> > > > >>> > > > >>> be
> > > > >>> > > > >>> generated from gridded model output by running
> software.
> > > One
> > > > >>> such
> > > > >>> > > > tracker
> > > > >>> > > > >>> is supported to the community by the DTC and is
named
> the
> > > > "GFDL
> > > > >>> > > Vortex
> > > > >>> > > > >>> Tracker":
> > > > >>> http://dtcenter.org/community-code/gfdl-vortex-tracker
> > > > >>> > > > >>> <
> http://dtcenter.org/community-code/gfdl-vortex-tracker
> > >*
> > > > >>> > > > >>> *It is not expressly part of METplus but would
be very
> > > useful
> > > > >>> for
> > > > >>> > > > >>> tracking
> > > > >>> > > > >>> storms in GRIB files.*
> > > > >>> > > > >>>
> > > > >>> > > > >>> My guess as to why none of the tracks are being
used
> has
> > to
> > > > do
> > > > >>> with
> > > > >>> > > the
> > > > >>> > > > >>> METplus configurations to filter data. I would
check
> the
> > > > >>> values for
> > > > >>> > > > >>> TC_PAIRS_CYCLONE, TC_PAIRS_BASIN,
TC_PAIRS_STORM_NAME,
> > > > >>> > > > TC_PAIRS_STORM_ID,
> > > > >>> > > > >>> and MODEL. The values you put in these lists
determine
> > > which
> > > > >>> storms
> > > > >>> > > to
> > > > >>> > > > >>> use.
> > > > >>> > > > >>> You can leave them blank to use any.
> > > > >>> > > > >>>
> > > > >>> > > > >>> Also, you could bump up the MET log verbosity
and see
> > more
> > > > >>> > > information
> > > > >>> > > > >>> about why they are being rejected:
> > > > >>> > > > >>>
> > > > >>> > > > >>> # to affect all MET tools set:
> > > > >>> > > > >>> LOG_MET_VERBOSITY = 10
> > > > >>> > > > >>>
> > > > >>> > > > >>> # to affect only TCPairs set:
> > > > >>> > > > >>> LOG_TC_PAIRS_VERBOSITY = 10
> > > > >>> > > > >>>
> > > > >>> > > > >>> - George
> > > > >>> > > > >>>
> > > > >>> > > > >>> On Thu, Aug 27, 2020 at 10:53 AM Edward Strobach
- NOAA
> > > > >>> Affiliate
> > > > >>> > via
> > > > >>> > > > RT
> > > > >>> > > > >>> <
> > > > >>> > > > >>> met_help at ucar.edu> wrote:
> > > > >>> > > > >>>
> > > > >>> > > > >>> >
> > > > >>> > > > >>> > <URL:
> > > > >>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458 >
> > > > >>> > > > >>> >
> > > > >>> > > > >>> > Hi,
> > > > >>> > > > >>> >
> > > > >>> > > > >>> > I did decide to remove CyclonePlotter for now
and use
> > the
> > > > >>> > > > >>> TCPairs_Config
> > > > >>> > > > >>> > file along with settings that I set in a
> shared_TC.conf
> > > > that
> > > > >>> I
> > > > >>> > > store
> > > > >>> > > > >>> in a
> > > > >>> > > > >>> > temporary directory to process the ADECK and
BDECK
> data
> > > > that
> > > > >>> I
> > > > >>> > > have.
> > > > >>> > > > >>> Files
> > > > >>> > > > >>> > are generated for all the dates that span
INIT_BEG
> and
> > > > >>> INIT_END,
> > > > >>> > > > >>> thanks to
> > > > >>> > > > >>> > setting loop_by to INIT. However, only the
header is
> > > > written
> > > > >>> out,
> > > > >>> > > > and I
> > > > >>> > > > >>> > suspect it is do to what the log indicates
here:
> > > > >>> > > > >>> >
> > > > >>> > > > >>> > DEBUG 4: [File 1 of 5] Used 0 of 168 lines
read from
> > file
> > > > >>> > > > >>> >
> > > > >>> > > > >>> >
> > > > >>> > > > >>>
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
> > > > >>> > > > >>> > DEBUG 4: [File 2 of 5] Used 0 of 168 lines
read from
> > file
> > > > >>> > > > >>> >
> > > > >>> > > > >>> >
> > > > >>> > > > >>>
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
> > > > >>> > > > >>> > DEBUG 4: [File 3 of 5] Used 0 of 168 lines
read from
> > file
> > > > >>> > > > >>> >
> > > > >>> > > > >>> >
> > > > >>> > > > >>>
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
> > > > >>> > > > >>> > DEBUG 4: [File 4 of 5] Used 0 of 168 lines
read from
> > file
> > > > >>> > > > >>> >
> > > > >>> > > > >>> >
> > > > >>> > > > >>>
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
> > > > >>> > > > >>> > DEBUG 4: [File 5 of 5] Used 0 of 168 lines
read from
> > file
> > > > >>> > > > >>> >
> > > > >>> > > > >>> >
> > > > >>> > > > >>>
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
"/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400"
> > > > >>> > > > >>> > DEBUG 3: Used 0 of 840 lines read from 5
file(s).
> > > > >>> > > > >>> > DEBUG 3: Identified 0 track(s).
> > > > >>> > > > >>> > DEBUG 5: TrackInfoArray: NTracks = 0, NAlloc =
0,
> > Tracks:
> > > > >>> > > > >>> > DEBUG 5:
> > > > >>> > > > >>> > DEBUG 2: Deriving 12-hour interpolated ADECK
tracks.
> > > > >>> > > > >>> > DEBUG 2: Finished adding 0 and replacing 0
Interp12
> > > > track(s).
> > > > >>> > > > >>> > DEBUG 2: Deriving 0 ADECK consensus model(s).
> > > > >>> > > > >>> > DEBUG 2: Added 0 ADECK consensus tracks(s).
> > > > >>> > > > >>> > DEBUG 2: Deriving 0 ADECK lag model(s).
> > > > >>> > > > >>> > DEBUG 2: Added 0 ADECK lag tracks(s).
> > > > >>> > > > >>> > DEBUG 2: Deriving 0 CLIPER/SHIFOR baseline
model(s).
> > > > >>> > > > >>> > DEBUG 2: Added 0 CLIPER/SHIFOR baseline
track(s).
> > > > >>> > > > >>> > DEBUG 2: Filtering 0 ADECK tracks based on
config
> file
> > > > >>> settings.
> > > > >>> > > > >>> > DEBUG 3: Total tracks read = 0
> > > > >>> > > > >>> > DEBUG 3: Total tracks kept = 0
> > > > >>> > > > >>> > DEBUG 3: Rejected for storm name = 0
> > > > >>> > > > >>> > DEBUG 3: Rejected for valid time = 0
> > > > >>> > > > >>> > DEBUG 3: Rejected for required lead times = 0
> > > > >>> > > > >>> > DEBUG 3: Rejected for init mask = 0
> > > > >>> > > > >>> > DEBUG 3: Rejected for valid mask = 0
> > > > >>> > > > >>> > DEBUG 2: Matching 0 ADECK tracks to 1 BDECK
tracks.
> > > > >>> > > > >>> >
> > > > >>> > > > >>> > I'm still thinking that the formatting is off
in this
> > > file.
> > > > >>> I
> > > > >>> > > > surmise
> > > > >>> > > > >>> that
> > > > >>> > > > >>> > it's not able to group the data because the
column
> > > > >>> information is
> > > > >>> > > not
> > > > >>> > > > >>> in
> > > > >>> > > > >>> > the order that met would want it to be in.
These
> files
> > > do
> > > > >>> not
> > > > >>> > have
> > > > >>> > > > any
> > > > >>> > > > >>> > header info, so it has to rely on the correct
> ordering,
> > > or
> > > > >>> so I
> > > > >>> > > > >>> suspect.
> > > > >>> > > > >>> > My interpretation, however, may be flawed. I
see 6
> > > > separate
> > > > >>> TC
> > > > >>> > > based
> > > > >>> > > > >>> > metplus routines: TCPairs (I'm working on
> currently);
> > > > >>> > > CyclonePlotter
> > > > >>> > > > >>> > (looking to use); TCStat; TCRMW; TCMPRPlotter;
and
> > TCGen.
> > > > >>> Much,
> > > > >>> > if
> > > > >>> > > > not
> > > > >>> > > > >>> > almost all of these relies on TCPairs. Is
there a
> > > routine
> > > > >>> that
> > > > >>> > I'm
> > > > >>> > > > >>> missing
> > > > >>> > > > >>> > in metplus that takes a grib file and
constructs
> ADECK
> > > > files?
> > > > >>> > > > >>> >
> > > > >>> > > > >>> >
> > > > >>> > > > >>> >
> > > > >>> > > > >>> > On Thu, Aug 27, 2020 at 11:52 AM Edward
Strobach -
> NOAA
> > > > >>> > Affiliate <
> > > > >>> > > > >>> > edward.strobach at noaa.gov> wrote:
> > > > >>> > > > >>> >
> > > > >>> > > > >>> > > Hi George, I figured the BDECK must be
compatible,
> so
> > > no
> > > > >>> > worries
> > > > >>> > > > >>> there.
> > > > >>> > > > >>> > > Thanks for the link and the heads up about
> > > > reformatting. I
> > > > >>> > will
> > > > >>> > > > use
> > > > >>> > > > >>> the
> > > > >>> > > > >>> > > example to modify what I currently have to
see the
> > > > outcome.
> > > > >>> > > Thanks
> > > > >>> > > > >>> > again.
> > > > >>> > > > >>> > >
> > > > >>> > > > >>> > > On Wed, Aug 26, 2020 at 6:57 PM George
McCabe via
> RT
> > <
> > > > >>> > > > >>> met_help at ucar.edu>
> > > > >>> > > > >>> > > wrote:
> > > > >>> > > > >>> > >
> > > > >>> > > > >>> > >> Hi Edward,
> > > > >>> > > > >>> > >>
> > > > >>> > > > >>> > >> The BDECK data is compatible to be
processed. The
> > use
> > > > >>> case you
> > > > >>> > > > used
> > > > >>> > > > >>> as
> > > > >>> > > > >>> > an
> > > > >>> > > > >>> > >> example uses extra tropical cyclone data
from SBU
> > that
> > > > is
> > > > >>> > > > >>> reformatted by
> > > > >>> > > > >>> > >> TCPairs wrapper before processing. You will
see
> your
> > > > >>> config
> > > > >>> > file
> > > > >>> > > > >>> has the
> > > > >>> > > > >>> > >> following:
> > > > >>> > > > >>> > >>
> > > > >>> > > > >>> > >> ## tc-pairs filtering
> optionsTC_PAIRS_REFORMAT_DECK
> > =
> > > > >>> > > > >>> > >> yesTC_PAIRS_REFORMAT_TYPE = SBU
> > > > >>> > > > >>> > >>
> > > > >>> > > > >>> > >> Changing TC_PAIRS_REFORMAT_DECK to 'no' or
'false'
> > > will
> > > > >>> skip
> > > > >>> > the
> > > > >>> > > > >>> > >> reformatting step. There is another use
case in
> the
> > > repo
> > > > >>> that
> > > > >>> > > uses
> > > > >>> > > > >>> data
> > > > >>> > > > >>> > >> that looks like yours (at least as far as I
can
> > tell).
> > > > >>> > > > >>> > >>
> > > > >>> > > > >>> > >>
> > > > >>> > > > >>> > >>
> > > > >>> > > > >>> >
> > > > >>> > > > >>>
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
https://dtcenter.github.io/METplus/generated/met_tool_wrapper/TCPairs/TCPairs_tropical.html#metplus-
configuration
> > > > >>> > > > >>> > >>
> > > > >>> > > > >>> > >> This use case uses BDECK data in
> > > > met_test/new/hwrf/bdeck.
> > > > >>> Here
> > > > >>> > > is
> > > > >>> > > > >>> part
> > > > >>> > > > >>> > of
> > > > >>> > > > >>> > >> the file called bal062018.dat:
> > > > >>> > > > >>> > >>
> > > > >>> > > > >>> > >> AL, 06, 2018082906, , BEST, 0, 129N,
109W,
> 10,
> > > > >>> 1010, DB,
> > > > >>> > > > >>> 0, ,
> > > > >>> > > > >>> > >> 0, 0, 0, 0, 0, 0, 0, 0,
0,
> > ,
> > > > 0,
> > > > >>> > ,
> > > > >>> > > > 0,
> > > > >>> > > > >>> > 0,
> > > > >>> > > > >>> > >> GENESIS019, , 0, , 0, 0, 0,
0,
> > > > >>> genesis-num,
> > > > >>> > > 019,
> > > > >>> > > > >>> > >>
> > > > >>> > > > >>> > >>
> > > > >>> > > > >>> > >> Let me know if changing that setting
doesn't work
> > and
> > > I
> > > > >>> can
> > > > >>> > > take a
> > > > >>> > > > >>> > closer
> > > > >>> > > > >>> > >> look.
> > > > >>> > > > >>> > >>
> > > > >>> > > > >>> > >> Thanks,
> > > > >>> > > > >>> > >> George
> > > > >>> > > > >>> > >>
> > > > >>> > > > >>> > >> On Wed, Aug 26, 2020 at 1:17 PM Edward
Strobach -
> > NOAA
> > > > >>> > Affiliate
> > > > >>> > > > >>> via RT
> > > > >>> > > > >>> > <
> > > > >>> > > > >>> > >> met_help at ucar.edu> wrote:
> > > > >>> > > > >>> > >>
> > > > >>> > > > >>> > >> >
> > > > >>> > > > >>> > >> > Wed Aug 26 13:13:28 2020: Request 96458
was
> acted
> > > > upon.
> > > > >>> > > > >>> > >> > Transaction: Ticket created by
> > > > edward.strobach at noaa.gov
> > > > >>> > > > >>> > >> > Queue: met_help
> > > > >>> > > > >>> > >> > Subject: Tropical Cyclone
verification:
> > TCPairs
> > > > and
> > > > >>> > > > >>> > CyclonePlotter
> > > > >>> > > > >>> > >> > Owner: Nobody
> > > > >>> > > > >>> > >> > Requestors: edward.strobach at noaa.gov
> > > > >>> > > > >>> > >> > Status: new
> > > > >>> > > > >>> > >> > Ticket <URL:
> > > > >>> > > > >>>
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=96458
> > > > >>> > > > >>> > >
> > > > >>> > > > >>> > >> >
> > > > >>> > > > >>> > >> >
> > > > >>> > > > >>> > >> > Good afternoon,
> > > > >>> > > > >>> > >> >
> > > > >>> > > > >>> > >> > I found it relatively straightforward to
set-up
> > some
> > > > >>> scripts
> > > > >>> > > to
> > > > >>> > > > >>> > process
> > > > >>> > > > >>> > >> > forecast output along with obs by using
this
> > > resource:
> > > > >>> > > > >>> > >> >
> > > > >>> > > > >>> > >> >
> > > > >>> > > > >>> > >>
> > > > >>> > > > >>> >
> > > > >>> > > > >>>
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
https://dtcenter.github.io/METplus/generated/model_applications/tc_and_extra_tc/Plotter_fcstGFS_obsGFS_ExtraTC.html#sphx-
glr-generated-model-applications-tc-and-extra-tc-plotter-fcstgfs-
obsgfs-extratc-py
> > > > >>> > > > >>> > >> > .
> > > > >>> > > > >>> > >> > I have been running the global model
which
> > generates
> > > > >>> track
> > > > >>> > > data
> > > > >>> > > > >>> for
> > > > >>> > > > >>> > the
> > > > >>> > > > >>> > >> > post processing step following this
naming
> > > convention:
> > > > >>> > > > >>> > >> > atcfunix.gfs.YYYYMMDDHH. The best track
data is
> > > > stored
> > > > >>> here
> > > > >>> > > on
> > > > >>> > > > >>> WCOSS:
> > > > >>> > > > >>> > >> >
> > > > >>> /gpfs/hps3/emc/hwrf/noscrub/emc.hurpara/trak/abdeck/btk/.
> > > > >>> > > > >>> > >> >
> > > > >>> > > > >>> > >> > I have a python script that determines
which
> file
> > to
> > > > use
> > > > >>> > based
> > > > >>> > > > on
> > > > >>> > > > >>> > basin
> > > > >>> > > > >>> > >> > information, storm name, and date. That
is
> piped
> > > into
> > > > >>> the
> > > > >>> > > > >>> > >> > TC_PAIRS_BDECK_TEMPLATE while the
forecast file
> > > above
> > > > is
> > > > >>> > piped
> > > > >>> > > > >>> into
> > > > >>> > > > >>> > the
> > > > >>> > > > >>> > >> > TC_PAIRS_ADECK_TEMPLATE this way:
> > > > >>> > > > >>> atcfunix.gfs.{date?fmt=%Y%m%d%H}.
> > > > >>> > > > >>> > >> >
> > > > >>> > > > >>> > >> > Inside my main script
> > > > >>> > > > >>> > >> >
> > > > >>> > > > >>> > >> >
> > > > >>> > > > >>> > >>
> > > > >>> > > > >>> >
> > > > >>> > > > >>>
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
*(/gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/METplus/scripts/run_metplus_TCPlotter.sh*,
> > > > >>> > > > >>> > >> > I have three configure files that being
> processed
> > > like
> > > > >>> so:*
> > > > >>> > > > >>> > >> > ${MET_PLUS}/ush/master_metplus.py -c
> > > > >>> > > > >>> > >> >
> > ${MET_PLUS_CONF}/Plotter_fcstGFS_obsGFS_ExtraTC.conf
> > > > -c
> > > > >>> > > > >>> > >> > ${MET_PLUS_TMP}/shared.conf_TC -c
> > > > >>> > > > ${MET_PLUS_CONF}/system_TC.conf*
> > > > >>> > > > >>> > >> >
> > > > >>> > > > >>> > >> > The system_TC.conf is of little concern
because
> it
> > > is
> > > > >>> mostly
> > > > >>> > > > >>> unchanged
> > > > >>> > > > >>> > >> > except for the OUTPUT_BASE, which is
changed
> > > > >>> specifically to
> > > > >>> > > > >>> where I
> > > > >>> > > > >>> > >> plan
> > > > >>> > > > >>> > >> > on storing the TC output. The
shared_TC.conf,
> on
> > > the
> > > > >>> other
> > > > >>> > > > hand,
> > > > >>> > > > >>> is
> > > > >>> > > > >>> > >> used
> > > > >>> > > > >>> > >> > to replace the content inside my
> > > > >>> > > > >>> Plotter_fcstGFS_obsGFS_ExtraTC.conf
> > > > >>> > > > >>> > >> file.
> > > > >>> > > > >>> > >> > In here I define
> > > > >>> > > > >>> > >> > the ADECK and BDECK input directories
based on
> the
> > > > >>> where the
> > > > >>> > > > >>> template
> > > > >>> > > > >>> > >> files
> > > > >>> > > > >>> > >> > above are being stored. Since I have
several
> runs
> > > > >>> > > encompassing
> > > > >>> > > > >>> the
> > > > >>> > > > >>> > >> > tropical cyclone event - 20190924 through
> 20190929
> > > - I
> > > > >>> have
> > > > >>> > > > >>> specified
> > > > >>> > > > >>> > my
> > > > >>> > > > >>> > >> > INIT_BEG as 20190924 and my INIT_END as
> > 20190929. I
> > > > >>> have
> > > > >>> > also
> > > > >>> > > > >>> set my
> > > > >>> > > > >>> > >> > INIT_INCREMENT to 86400 since I'm only
> processing
> > > the
> > > > >>> 00z
> > > > >>> > > cycle
> > > > >>> > > > >>> for
> > > > >>> > > > >>> > >> now. I
> > > > >>> > > > >>> > >> > have a couple of lines of code before the
> > configure
> > > > file
> > > > >>> > that
> > > > >>> > > > >>> joins
> > > > >>> > > > >>> > >> these
> > > > >>> > > > >>> > >> > dates in a comma separated list such that
the
> > > > >>> > > > >>> > CYCLONE_PLOTTER_INIT_DATE
> > > > >>> > > > >>> > >> is
> > > > >>> > > > >>> > >> > "20190924,20190925,...,20190929". I'm
not sure
> if
> > > > that
> > > > >>> is
> > > > >>> > > > allowed
> > > > >>> > > > >>> > >> though.
> > > > >>> > > > >>> > >> >
> > > > >>> > > > >>> > >> > When running the script the log that is
> generated
> > > > >>> produces 0
> > > > >>> > > > >>> errors.
> > > > >>> > > > >>> > >> > However, I find *two issues*: *1) there
is only
> > one
> > > > >>> file
> > > > >>> > > being
> > > > >>> > > > >>> > carried
> > > > >>> > > > >>> > >> > over to generate results; that is the
20190924
> > date
> > > -
> > > > >>> there
> > > > >>> > > > >>> should be
> > > > >>> > > > >>> > >> 6*,
> > > > >>> > > > >>> > >> > and *2) The files that are created are
empty and
> > > thus
> > > > >>> the
> > > > >>> > plot
> > > > >>> > > > >>> that is
> > > > >>> > > > >>> > >> > generated is simply a map with no data
> overlaid.*
> > > > >>> > > > >>> > >> >
> > > > >>> > > > >>> > >> > In attempting to debug further, I found
> something
> > > that
> > > > >>> looks
> > > > >>> > > > >>> important
> > > > >>> > > > >>> > >> > shown here:
> > > > >>> > > > >>> > >> >
> > > > >>> > > > >>> > >> > DEBUG 3: Rejected for init mask
= 0
> > > > >>> > > > >>> > >> > DEBUG 3: Rejected for valid mask
= 0
> > > > >>> > > > >>> > >> > *DEBUG 2: Matching 0 ADECK tracks to 0
BDECK
> > > tracks.*
> > > > >>> > > > >>> > >> > DEBUG 1: Watch/Warning file:
> > > > >>> > > > >>> > >> >
> > > > >>> > > > >>> > >> >
> > > > >>> > > > >>> > >>
> > > > >>> > > > >>> >
> > > > >>> > > > >>>
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
/gpfs/dell2/emc/verification/noscrub/Julie.Prestopnik/met/9.1_beta2/share/met/tc_data/wwpts_us.txt
> > > > >>> > > > >>> > >> > DEBUG 5: TrackPairInfoArray: NPairs = 0,
NAlloc
> =
> > 0,
> > > > >>> Pairs:
> > > > >>> > > > >>> > >> > DEBUG 5:
> > > > >>> > > > >>> > >> > DEBUG 1: Output file:
> > > > >>> > > > >>> > >> >
> > > > >>> > > > >>> > >> >
> > > > >>> > > > >>> > >>
> > > > >>> > > > >>> >
> > > > >>> > > > >>>
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs/201909/AL_q2019092400.gfs.PRNA20.tcst
> > > > >>> > > > >>> > >> > 08/26 18:35:12.422 metplus.TCPairs
> > > > >>> (command_runner.py:191)
> > > > >>> > > > DEBUG:
> > > > >>> > > > >>> > >> Finished
> > > > >>> > > > >>> > >> > running
> > > > >>> > > > >>> > >> >
> > > > >>> > > > >>> > >> >
> > > > >>> > > > >>> > >>
> > > > >>> > > > >>> >
> > > > >>> > > > >>>
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
/gpfs/dell2/emc/verification/noscrub/Julie.Prestopnik/met/9.1_beta2/bin/tc_pairs
> > > > >>> > > > >>> > >> > in 0:00:00.607349
> > > > >>> > > > >>> > >> > 08/26 18:35:12.422 metplus.CyclonePlotter
> > > > >>> > > > >>> > >> (cyclone_plotter_wrapper.py:77)
> > > > >>> > > > >>> > >> > DEBUG: Begin retrieving data...
> > > > >>> > > > >>> > >> > 08/26 18:35:12.422 metplus.CyclonePlotter
> > > > >>> > > > >>> > >> (cyclone_plotter_wrapper.py:83)
> > > > >>> > > > >>> > >> > DEBUG: Generate plot for all files in the
> > > > >>> > > > >>> > >> >
> > > > >>> > > > >>> > >> >
> > > > >>> > > > >>> > >>
> > > > >>> > > > >>> >
> > > > >>> > > > >>>
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
directory/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs
> > > > >>> > > > >>> > >> > 08/26 18:35:12.423 metplus.CyclonePlotter
> > > > >>> > > > >>> > >> (cyclone_plotter_wrapper.py:98)
> > > > >>> > > > >>> > >> > DEBUG: Parsing file
> > > > >>> > > > >>> > >> >
> > > > >>> > > > >>> > >> >
> > > > >>> > > > >>> > >>
> > > > >>> > > > >>> >
> > > > >>> > > > >>>
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/tc_pairs/201909/AL_q2019092400.gfs.PRNA20.tcst
> > > > >>> > > > >>> > >> > 08/26 18:35:19.778 metplus
> > (config_launcher.py:520)
> > > > >>> DEBUG:
> > > > >>> > > > Setting
> > > > >>> > > > >>> > >> [config]
> > > > >>> > > > >>> > >> > SCRUB_STAGING_DIR to default value:
False.
> > > > >>> > > > >>> > >> > 08/26 18:35:19.779 metplus INFO:
> > > > >>> > > > >>> > >> >
> > > > >>> > > > >>> > >> >
> > > > >>> > > > >>> > >>
> > > > >>> > > > >>> >
> > > > >>> > > > >>>
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_TC/TCPlotter/metplus_final.conf:
> > > > >>> > > > >>> > >> > write metplus.conf here
> > > > >>> > > > >>> > >> > 08/26 18:35:19.783 metplus
(met_util.py:213)
> > DEBUG:
> > > > >>> METplus
> > > > >>> > > took
> > > > >>> > > > >>> > >> > 0:00:08.783240 to run.
> > > > >>> > > > >>> > >> > 08/26 18:35:19.783 metplus INFO: METplus
has
> > > > >>> successfully
> > > > >>> > > > finished
> > > > >>> > > > >>> > >> running.
> > > > >>> > > > >>> > >> >
> > > > >>> > > > >>> > >> > Specifically the line that says matching
0 ADECK
> > > > tracks
> > > > >>> to 0
> > > > >>> > > > BDECK
> > > > >>> > > > >>> > >> tracks
> > > > >>> > > > >>> > >> > lends me to believe that this is the
reason why
> > > > nothing
> > > > >>> is
> > > > >>> > > being
> > > > >>> > > > >>> added
> > > > >>> > > > >>> > >> to
> > > > >>> > > > >>> > >> > my TC_PAIRS_OUTPUT_TEMPLATE file, and
therefore
> no
> > > > plots
> > > > >>> > > > >>> generated on
> > > > >>> > > > >>> > a
> > > > >>> > > > >>> > >> > map.
> > > > >>> > > > >>> > >> >
> > > > >>> > > > >>> > >> > What's interesting is the file that is
extracted
> > for
> > > > >>> BDECK
> > > > >>> > is
> > > > >>> > > > for
> > > > >>> > > > >>> > >> hurricane
> > > > >>> > > > >>> > >> > Lorenzo and includes all the dates
coinciding
> with
> > > the
> > > > >>> ADECK
> > > > >>> > > > file
> > > > >>> > > > >>> - a
> > > > >>> > > > >>> > >> case
> > > > >>> > > > >>> > >> > that is run coinciding with Lorenzo.
I've
> checked
> > > > this.
> > > > >>> > > > >>> > >> >
> > > > >>> > > > >>> > >> > One possible issue is the formatting of
these
> > files.
> > > > I
> > > > >>> > looked
> > > > >>> > > > in
> > > > >>> > > > >>> the
> > > > >>> > > > >>> > >> test
> > > > >>> > > > >>> > >> > case that comes with the cloned met
stuff. In
> > there
> > > > the
> > > > >>> > > > >>> header/field
> > > > >>> > > > >>> > >> info
> > > > >>> > > > >>> > >> > looks something like this:
> > > > >>> > > > >>> > >> >
> > > > >>> > > > >>> > >> >
> > > > >>> > > > >>> > >> >
> > > > >>> > > > >>> > >>
> > > > >>> > > > >>> >
> > > > >>> > > > >>>
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
*/gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/METplus/source_data/met_test/new/track_data/201412/amlq2014123118.gfso.0104*
> > > > >>> > > > >>> > >> >
> > > > >>> > > > >>> > >> > ML, 0104, 2014121318_F000_478N_1780W_FOF,
> > > 2014121318,
> > > > >>> 03,
> > > > >>> > > GFSO,
> > > > >>> > > > >>> 000,
> > > > >>> > > > >>> > >> 478N,
> > > > >>> > > > >>> > >> > 1780W, 38, 986, XX, 34, NEQ, 0283,
0640,
> 0645,
> > > > 0000,
> > > > >>> > 990,
> > > > >>> > > > >>> 133,
> > > > >>> > > > >>> > -99,
> > > > >>> > > > >>> > >> > 69, 227, -99, -9999, -9999, 208, 315,
135,
> > 156,
> > > > >>> 0063
> > > > >>> > > > >>> > >> >
> > > > >>> > > > >>> > >> > This differs from the structure of both
the
> ADECK
> > > and
> > > > >>> BDECK
> > > > >>> > > > input
> > > > >>> > > > >>> > files:
> > > > >>> > > > >>> > >> >
> > > > >>> > > > >>> > >> > *BDECK:
> > > > >>> > > > >>> > >> >
> > > > >>> > > > >>>
> > > > >>> >
> > >
/gpfs/hps3/emc/hwrf/noscrub/emc.hurpara/trak/abdeck/btk/bal132019.dat
> > > > >>> > > > >>> > *
> > > > >>> > > > >>> > >> >
> > > > >>> > > > >>> > >> > AL, 13, 2019092218, , BEST, 0, 109N,
188W,
> > 25,
> > > > >>> 1008,
> > > > >>> > LO,
> > > > >>> > > > >>> 0,
> > > > >>> > > > >>> > ,
> > > > >>> > > > >>> > >> > 0, 0, 0, 0, 1012, 100, 90,
40, 0,
> > > L,
> > > > >>> 0,
> > > > >>> > > ,
> > > > >>> > > > >>> 0,
> > > > >>> > > > >>> > >> 0,
> > > > >>> > > > >>> > >> > INVEST, S, 0, , 0, 0, 0,
0,
> > > > >>> genesis-num,
> > > > >>> > > 027,
> > > > >>> > > > >>> > >> >
> > > > >>> > > > >>> > >> > *ADECK:
> > > > >>> > > > >>> > >> >
> > > > >>> > > > >>> > >> >
> > > > >>> > > > >>> > >>
> > > > >>> > > > >>> >
> > > > >>> > > > >>>
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/archive/prna20/atcfunix.gfs.2019092400*
> > > > >>> > > > >>> > >> >
> > > > >>> > > > >>> > >> > AL, 0912, 03, PRNA, 000, 157N, 655W,
27, 1008,
> > XX,
> > > > >>> 34,
> > > > >>> > NEQ,
> > > > >>> > > > >>> 0000,
> > > > >>> > > > >>> > >> 0000,
> > > > >>> > > > >>> > >> > 0000, 0000, -9999, -9999, 85
> > > > >>> > > > >>> > >> >
> > > > >>> > > > >>> > >> > My suspicion is that these formats are
not
> > > compatible.
> > > > >>> > There
> > > > >>> > > > are
> > > > >>> > > > >>> > major
> > > > >>> > > > >>> > >> > differences between these formats in my
opinion.
> > > The
> > > > >>> ADECK
> > > > >>> > > was
> > > > >>> > > > >>> > >> generated
> > > > >>> > > > >>> > >> > automatically when making a run while
BDECK is
> > done
> > > by
> > > > >>> > someone
> > > > >>> > > > >>> else.
> > > > >>> > > > >>> > >> Those
> > > > >>> > > > >>> > >> > files are stored whenever a storm is
identified.
> > > > >>> > > > >>> > >> >
> > > > >>> > > > >>> > >> > I think what I'll need to do next is
start by
> > > running
> > > > >>> TCRMW
> > > > >>> > to
> > > > >>> > > > >>> produce
> > > > >>> > > > >>> > >> the
> > > > >>> > > > >>> > >> > ADECK files formatted by metplus. I have
the
> grib
> > > > >>> files for
> > > > >>> > > > that.
> > > > >>> > > > >>> > The
> > > > >>> > > > >>> > >> > question next, though, is the BDECK files
okay
> for
> > > > >>> > processing?
> > > > >>> > > > >>> > >> >
> > > > >>> > > > >>> > >> >
> > > > >>> > > > >>> > >> >
> > > > >>> > > > >>> > >> > --
> > > > >>> > > > >>> > >> > Edward Strobach
> > > > >>> > > > >>> > >> > EMC/NCEP/NWS/
> > > > >>> > > > >>> > >> > IMSG Contractor
> > > > >>> > > > >>> > >> > Cubicle#: 2029
> > > > >>> > > > >>> > >> > 301-683-3717
> > > > >>> > > > >>> > >> >
> > > > >>> > > > >>> > >> >
> > > > >>> > > > >>> > >>
> > > > >>> > > > >>> > >> --
> > > > >>> > > > >>> > >> George McCabe - Software Engineer III
> > > > >>> > > > >>> > >> National Center for Atmospheric Research
> > > > >>> > > > >>> > >> Research Applications Laboratory
> > > > >>> > > > >>> > >> 303-497-2768
> > > > >>> > > > >>> > >> ---
> > > > >>> > > > >>> > >> My working day may not be your working day.
Please
> > do
> > > > not
> > > > >>> feel
> > > > >>> > > > >>> obliged
> > > > >>> > > > >>> > to
> > > > >>> > > > >>> > >> reply to this email outside of your normal
working
> > > > hours.
> > > > >>> > > > >>> > >>
> > > > >>> > > > >>> > >>
> > > > >>> > > > >>> > >
> > > > >>> > > > >>> > > --
> > > > >>> > > > >>> > > Edward Strobach
> > > > >>> > > > >>> > > EMC/NCEP/NWS/
> > > > >>> > > > >>> > > IMSG Contractor
> > > > >>> > > > >>> > > Cubicle#: 2029
> > > > >>> > > > >>> > > 301-683-3717
> > > > >>> > > > >>> > >
> > > > >>> > > > >>> >
> > > > >>> > > > >>> >
> > > > >>> > > > >>> > --
> > > > >>> > > > >>> > Edward Strobach
> > > > >>> > > > >>> > EMC/NCEP/NWS/
> > > > >>> > > > >>> > IMSG Contractor
> > > > >>> > > > >>> > Cubicle#: 2029
> > > > >>> > > > >>> > 301-683-3717
> > > > >>> > > > >>> >
> > > > >>> > > > >>> >
> > > > >>> > > > >>>
> > > > >>> > > > >>> --
> > > > >>> > > > >>> George McCabe - Software Engineer III
> > > > >>> > > > >>> National Center for Atmospheric Research
> > > > >>> > > > >>> Research Applications Laboratory
> > > > >>> > > > >>> 303-497-2768
> > > > >>> > > > >>> ---
> > > > >>> > > > >>> My working day may not be your working day.
Please do
> not
> > > > feel
> > > > >>> > > obliged
> > > > >>> > > > to
> > > > >>> > > > >>> reply to this email outside of your normal
working
> hours.
> > > > >>> > > > >>>
> > > > >>> > > > >>>
> > > > >>> > > > >>
> > > > >>> > > > >> --
> > > > >>> > > > >> Edward Strobach
> > > > >>> > > > >> EMC/NCEP/NWS/
> > > > >>> > > > >> IMSG Contractor
> > > > >>> > > > >> Cubicle#: 2029
> > > > >>> > > > >> 301-683-3717
> > > > >>> > > > >>
> > > > >>> > > > >
> > > > >>> > > > >
> > > > >>> > > > > --
> > > > >>> > > > > Edward Strobach
> > > > >>> > > > > EMC/NCEP/NWS/
> > > > >>> > > > > IMSG Contractor
> > > > >>> > > > > Cubicle#: 2029
> > > > >>> > > > > 301-683-3717
> > > > >>> > > > >
> > > > >>> > > >
> > > > >>> > > >
> > > > >>> > > > --
> > > > >>> > > > Edward Strobach
> > > > >>> > > > EMC/NCEP/NWS/
> > > > >>> > > > IMSG Contractor
> > > > >>> > > > Cubicle#: 2029
> > > > >>> > > > 301-683-3717
> > > > >>> > > >
> > > > >>> > > >
> > > > >>> > >
> > > > >>> > >
> > > > >>> >
> > > > >>> > --
> > > > >>> > Edward Strobach
> > > > >>> > EMC/NCEP/NWS/
> > > > >>> > IMSG Contractor
> > > > >>> > Cubicle#: 2029
> > > > >>> > 301-683-3717
> > > > >>> >
> > > > >>> >
> > > > >>>
> > > > >>>
> > > > >>
> > > > >> --
> > > > >> Edward Strobach
> > > > >> EMC/NCEP/NWS/
> > > > >> IMSG Contractor
> > > > >> Cubicle#: 2029
> > > > >> 301-683-3717
> > > > >>
> > > > >
> > > > >
> > > > > --
> > > > > Edward Strobach
> > > > > EMC/NCEP/NWS/
> > > > > IMSG Contractor
> > > > > Cubicle#: 2029
> > > > > 301-683-3717
> > > > >
> > > >
> > > >
> > > > --
> > > > Edward Strobach
> > > > EMC/NCEP/NWS/
> > > > IMSG Contractor
> > > > Cubicle#: 2029
> > > > 301-683-3717
> > > >
> > > >
> > >
> > >
> >
> >
>
> --
> Edward Strobach
> EMC/NCEP/NWS/
> IMSG Contractor
> Cubicle#: 2029
> 301-683-3717
>
>
------------------------------------------------
More information about the Met_help
mailing list