[Met_help] [rt.rap.ucar.edu #99846] History for ask a question about METplus config of TC

George McCabe via RT met_help at ucar.edu
Wed Jun 16 15:23:27 MDT 2021


----------------------------------------------------------------
  Initial Request
----------------------------------------------------------------

Hi,

This is Yan, again. I ran the METplus config file after our discussion,
however, it seems that the tc_pairs tcst file wasn't generated.

Here is the script in my directory on Jet:
I set TC_PAIRS_STORM_ID = AL292020, so the code can find the input adeck
and bdeck data correctly
*----
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test.conf

The output directory is
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs
and the tc_pairs data: al292020_pairs.dat.tcst

log file
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210510214154
*-----

The error starts:[image: image.png]

Then I checked al292020_pairs.dat.tcst, and it is empty.

I wonder if there is anything I missed?

Many thanks.
Best,
Yan


----------------------------------------------------------------
  Complete Ticket History
----------------------------------------------------------------

Subject: ask a question about METplus config of TC
From: John Halley Gotway
Time: Tue May 11 07:44:27 2021

Hi Yan,

I'm going to assign this ticket to George and ask him to take a look
and
debug this runtime error.

Thanks,
John HG

On Mon, May 10, 2021 at 3:49 PM Yan Jin - NOAA Affiliate via RT <
met_help at ucar.edu> wrote:

>
> Mon May 10 15:49:51 2021: Request 99846 was acted upon.
> Transaction: Ticket created by yan.jin at noaa.gov
>        Queue: met_help
>      Subject: ask a question about METplus config of TC
>        Owner: Nobody
>   Requestors: yan.jin at noaa.gov
>       Status: new
>  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846 >
>
>
> Hi,
>
> This is Yan, again. I ran the METplus config file after our
discussion,
> however, it seems that the tc_pairs tcst file wasn't generated.
>
> Here is the script in my directory on Jet:
> I set TC_PAIRS_STORM_ID = AL292020, so the code can find the input
adeck
> and bdeck data correctly
> *----
>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test.conf
>
> The output directory is
> /mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs
> and the tc_pairs data: al292020_pairs.dat.tcst
>
> log file
>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210510214154
> *-----
>
> The error starts:[image: image.png]
>
> Then I checked al292020_pairs.dat.tcst, and it is empty.
>
> I wonder if there is anything I missed?
>
> Many thanks.
> Best,
> Yan
>
>

------------------------------------------------
Subject: ask a question about METplus config of TC
From: Yan Jin - NOAA Affiliate
Time: Tue May 11 07:46:25 2021

Hi John,

Thanks so much!

Best,
Yan


On Tue, May 11, 2021 at 9:44 AM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:

> Hi Yan,
>
> I'm going to assign this ticket to George and ask him to take a look
and
> debug this runtime error.
>
> Thanks,
> John HG
>
> On Mon, May 10, 2021 at 3:49 PM Yan Jin - NOAA Affiliate via RT <
> met_help at ucar.edu> wrote:
>
> >
> > Mon May 10 15:49:51 2021: Request 99846 was acted upon.
> > Transaction: Ticket created by yan.jin at noaa.gov
> >        Queue: met_help
> >      Subject: ask a question about METplus config of TC
> >        Owner: Nobody
> >   Requestors: yan.jin at noaa.gov
> >       Status: new
> >  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846 >
> >
> >
> > Hi,
> >
> > This is Yan, again. I ran the METplus config file after our
discussion,
> > however, it seems that the tc_pairs tcst file wasn't generated.
> >
> > Here is the script in my directory on Jet:
> > I set TC_PAIRS_STORM_ID = AL292020, so the code can find the input
adeck
> > and bdeck data correctly
> > *----
> >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test.conf
> >
> > The output directory is
> >
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs
> > and the tc_pairs data: al292020_pairs.dat.tcst
> >
> > log file
> >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210510214154
> > *-----
> >
> > The error starts:[image: image.png]
> >
> > Then I checked al292020_pairs.dat.tcst, and it is empty.
> >
> > I wonder if there is anything I missed?
> >
> > Many thanks.
> > Best,
> > Yan
> >
> >
>
>

------------------------------------------------
Subject: ask a question about METplus config of TC
From: John Halley Gotway
Time: Tue May 11 09:57:06 2021

Yan,

George and I tried to take a look at this, but couldn't find the data.
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210510214154

I checked on Hera and WCOSS but couldn't find the data there. I should
be
able to log on to jet, but when I try, it boots me off:

Authentication failed.
Connection to jet-rsa.boulder.rdhpcs.noaa.gov closed.

Can you confirm that you're running on jet? And any idea of jet is
down
today or something?

Thanks,
John


On Tue, May 11, 2021 at 7:46 AM Yan Jin - NOAA Affiliate via RT <
met_help at ucar.edu> wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846 >
>
> Hi John,
>
> Thanks so much!
>
> Best,
> Yan
>
>
> On Tue, May 11, 2021 at 9:44 AM John Halley Gotway via RT <
> met_help at ucar.edu>
> wrote:
>
> > Hi Yan,
> >
> > I'm going to assign this ticket to George and ask him to take a
look and
> > debug this runtime error.
> >
> > Thanks,
> > John HG
> >
> > On Mon, May 10, 2021 at 3:49 PM Yan Jin - NOAA Affiliate via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > Mon May 10 15:49:51 2021: Request 99846 was acted upon.
> > > Transaction: Ticket created by yan.jin at noaa.gov
> > >        Queue: met_help
> > >      Subject: ask a question about METplus config of TC
> > >        Owner: Nobody
> > >   Requestors: yan.jin at noaa.gov
> > >       Status: new
> > >  Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846
> >
> > >
> > >
> > > Hi,
> > >
> > > This is Yan, again. I ran the METplus config file after our
discussion,
> > > however, it seems that the tc_pairs tcst file wasn't generated.
> > >
> > > Here is the script in my directory on Jet:
> > > I set TC_PAIRS_STORM_ID = AL292020, so the code can find the
input
> adeck
> > > and bdeck data correctly
> > > *----
> > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test.conf
> > >
> > > The output directory is
> > >
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs
> > > and the tc_pairs data: al292020_pairs.dat.tcst
> > >
> > > log file
> > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210510214154
> > > *-----
> > >
> > > The error starts:[image: image.png]
> > >
> > > Then I checked al292020_pairs.dat.tcst, and it is empty.
> > >
> > > I wonder if there is anything I missed?
> > >
> > > Many thanks.
> > > Best,
> > > Yan
> > >
> > >
> >
> >
>
>

------------------------------------------------
Subject: ask a question about METplus config of TC
From: Yan Jin - NOAA Affiliate
Time: Tue May 11 10:05:06 2021

Hi John,


The directories and data I mentioned in my last email are all located
in
Jet.


Here is the email I got regarding the Jet maintenance:

***

*05/11/21 - Boulder Bastion Maintenance Downtime*
Boulder Bastion maintenance is scheduled for Tuesday 05/11/21 from
0600 ~
1000 MT. Boulder Bastion login sessions will be dropped and all new
RDHPCS
logins will need to go through the Princeton Bastion.  Logging In
<https://rdhpcs-common-docs.rdhpcs.noaa.gov/wiki/index.php/Logging_in>



*The next Jet Maintenance Downtime is TBD*

***


I tried to login Princeton but the connection is closed too.


Really appreciate your time and help on this.


Best,

Yan

On Tue, May 11, 2021 at 11:57 AM John Halley Gotway via RT <
met_help at ucar.edu> wrote:

> Yan,
>
> George and I tried to take a look at this, but couldn't find the
data.
>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210510214154
>
> I checked on Hera and WCOSS but couldn't find the data there. I
should be
> able to log on to jet, but when I try, it boots me off:
>
> Authentication failed.
> Connection to jet-rsa.boulder.rdhpcs.noaa.gov closed.
>
> Can you confirm that you're running on jet? And any idea of jet is
down
> today or something?
>
> Thanks,
> John
>
>
> On Tue, May 11, 2021 at 7:46 AM Yan Jin - NOAA Affiliate via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846 >
> >
> > Hi John,
> >
> > Thanks so much!
> >
> > Best,
> > Yan
> >
> >
> > On Tue, May 11, 2021 at 9:44 AM John Halley Gotway via RT <
> > met_help at ucar.edu>
> > wrote:
> >
> > > Hi Yan,
> > >
> > > I'm going to assign this ticket to George and ask him to take a
look
> and
> > > debug this runtime error.
> > >
> > > Thanks,
> > > John HG
> > >
> > > On Mon, May 10, 2021 at 3:49 PM Yan Jin - NOAA Affiliate via RT
<
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > Mon May 10 15:49:51 2021: Request 99846 was acted upon.
> > > > Transaction: Ticket created by yan.jin at noaa.gov
> > > >        Queue: met_help
> > > >      Subject: ask a question about METplus config of TC
> > > >        Owner: Nobody
> > > >   Requestors: yan.jin at noaa.gov
> > > >       Status: new
> > > >  Ticket <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846
> > >
> > > >
> > > >
> > > > Hi,
> > > >
> > > > This is Yan, again. I ran the METplus config file after our
> discussion,
> > > > however, it seems that the tc_pairs tcst file wasn't
generated.
> > > >
> > > > Here is the script in my directory on Jet:
> > > > I set TC_PAIRS_STORM_ID = AL292020, so the code can find the
input
> > adeck
> > > > and bdeck data correctly
> > > > *----
> > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test.conf
> > > >
> > > > The output directory is
> > > >
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs
> > > > and the tc_pairs data: al292020_pairs.dat.tcst
> > > >
> > > > log file
> > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210510214154
> > > > *-----
> > > >
> > > > The error starts:[image: image.png]
> > > >
> > > > Then I checked al292020_pairs.dat.tcst, and it is empty.
> > > >
> > > > I wonder if there is anything I missed?
> > > >
> > > > Many thanks.
> > > > Best,
> > > > Yan
> > > >
> > > >
> > >
> > >
> >
> >
>
>

------------------------------------------------
Subject: ask a question about METplus config of TC
From: George McCabe
Time: Wed May 12 09:36:01 2021

Hi Yan,

I was able to log into Jet this morning and look at your error.

It looks like the error in the call to Rscript is due to there being
no data in the input files generated by TCPairs. There are
inconsistencies in the TCMPRPlotter wrapper that prevent the log
output from the run to be seen, which would have made it more clear
why the issue was occurring.

I would recommend changing the PROCESS_LIST value in your METplus
configuration file to only run TCPairs and iterating on that until you
get output from that tool. Once you can confirm that you have output,
you can add TCMPRPlotter back to the PROCESS_LIST and test on that
portion.

I will create a GitHub issue for updating the TCMPRPlotter wrapper to
produce useful logging output that will make things more clear going
forward.

Let me know if you run into any other issues or need help configuring
TCPairs to run properly. I'm also happy to attend another working
session if that is helpful.

Thanks,
George


On Tue May 11 10:05:06 2021, yan.jin at noaa.gov wrote:
> Hi John,
>
>
> The directories and data I mentioned in my last email are all
located
> in
> Jet.
>
>
> Here is the email I got regarding the Jet maintenance:
>
> ***
>
> *05/11/21 - Boulder Bastion Maintenance Downtime*
> Boulder Bastion maintenance is scheduled for Tuesday 05/11/21 from
> 0600 ~
> 1000 MT. Boulder Bastion login sessions will be dropped and all new
> RDHPCS
> logins will need to go through the Princeton Bastion.  Logging In
> <https://rdhpcs-common-
docs.rdhpcs.noaa.gov/wiki/index.php/Logging_in>
>
>
>
> *The next Jet Maintenance Downtime is TBD*
>
> ***
>
>
> I tried to login Princeton but the connection is closed too.
>
>
> Really appreciate your time and help on this.
>
>
> Best,
>
> Yan
>
> On Tue, May 11, 2021 at 11:57 AM John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
> > Yan,
> >
> > George and I tried to take a look at this, but couldn't find the
> > data.
> >
> >
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210510214154
> >
> > I checked on Hera and WCOSS but couldn't find the data there. I
> > should be
> > able to log on to jet, but when I try, it boots me off:
> >
> > Authentication failed.
> > Connection to jet-rsa.boulder.rdhpcs.noaa.gov closed.
> >
> > Can you confirm that you're running on jet? And any idea of jet is
> > down
> > today or something?
> >
> > Thanks,
> > John
> >
> >
> > On Tue, May 11, 2021 at 7:46 AM Yan Jin - NOAA Affiliate via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846 >
> > >
> > > Hi John,
> > >
> > > Thanks so much!
> > >
> > > Best,
> > > Yan
> > >
> > >
> > > On Tue, May 11, 2021 at 9:44 AM John Halley Gotway via RT <
> > > met_help at ucar.edu>
> > > wrote:
> > >
> > > > Hi Yan,
> > > >
> > > > I'm going to assign this ticket to George and ask him to take
a
> > > > look
> > and
> > > > debug this runtime error.
> > > >
> > > > Thanks,
> > > > John HG
> > > >
> > > > On Mon, May 10, 2021 at 3:49 PM Yan Jin - NOAA Affiliate via
RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > Mon May 10 15:49:51 2021: Request 99846 was acted upon.
> > > > > Transaction: Ticket created by yan.jin at noaa.gov
> > > > >        Queue: met_help
> > > > >      Subject: ask a question about METplus config of TC
> > > > >        Owner: Nobody
> > > > >   Requestors: yan.jin at noaa.gov
> > > > >       Status: new
> > > > >  Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846
> > > >
> > > > >
> > > > >
> > > > > Hi,
> > > > >
> > > > > This is Yan, again. I ran the METplus config file after our
> > discussion,
> > > > > however, it seems that the tc_pairs tcst file wasn't
generated.
> > > > >
> > > > > Here is the script in my directory on Jet:
> > > > > I set TC_PAIRS_STORM_ID = AL292020, so the code can find the
> > > > > input
> > > adeck
> > > > > and bdeck data correctly
> > > > > *----
> > > > >
> > > > >
> > > >
> > >
> >
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test.conf
> > > > >
> > > > > The output directory is
> > > > >
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs
> > > > > and the tc_pairs data: al292020_pairs.dat.tcst
> > > > >
> > > > > log file
> > > > >
> > > > >
> > > >
> > >
> >
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210510214154
> > > > > *-----
> > > > >
> > > > > The error starts:[image: image.png]
> > > > >
> > > > > Then I checked al292020_pairs.dat.tcst, and it is empty.
> > > > >
> > > > > I wonder if there is anything I missed?
> > > > >
> > > > > Many thanks.
> > > > > Best,
> > > > > Yan
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> >
> >



------------------------------------------------
Subject: ask a question about METplus config of TC
From: Yan Jin - NOAA Affiliate
Time: Wed May 12 14:04:11 2021

Hi George,

Thanks very much for your instructions. I set "PROCESS_LIST =
TCPairs", and
didn't specify models: "MODEL =". Then run the script. However, the
output
file is still empty. It seems that tc_pairs matched 0 adeck track to 1
bdeck track. I also tried to set "MODEL = HWRF, HMON", but the output
file
is empty too.

Here is the log file:
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210512193152

And here is the output file:
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst

Many thanks.
Best,
Yan

On Wed, May 12, 2021 at 11:36 AM George McCabe via RT
<met_help at ucar.edu>
wrote:

> Hi Yan,
>
> I was able to log into Jet this morning and look at your error.
>
> It looks like the error in the call to Rscript is due to there being
no
> data in the input files generated by TCPairs. There are
inconsistencies in
> the TCMPRPlotter wrapper that prevent the log output from the run to
be
> seen, which would have made it more clear why the issue was
occurring.
>
> I would recommend changing the PROCESS_LIST value in your METplus
> configuration file to only run TCPairs and iterating on that until
you get
> output from that tool. Once you can confirm that you have output,
you can
> add TCMPRPlotter back to the PROCESS_LIST and test on that portion.
>
> I will create a GitHub issue for updating the TCMPRPlotter wrapper
to
> produce useful logging output that will make things more clear going
> forward.
>
> Let me know if you run into any other issues or need help
configuring
> TCPairs to run properly. I'm also happy to attend another working
session
> if that is helpful.
>
> Thanks,
> George
>
>
> On Tue May 11 10:05:06 2021, yan.jin at noaa.gov wrote:
> > Hi John,
> >
> >
> > The directories and data I mentioned in my last email are all
located
> > in
> > Jet.
> >
> >
> > Here is the email I got regarding the Jet maintenance:
> >
> > ***
> >
> > *05/11/21 - Boulder Bastion Maintenance Downtime*
> > Boulder Bastion maintenance is scheduled for Tuesday 05/11/21 from
> > 0600 ~
> > 1000 MT. Boulder Bastion login sessions will be dropped and all
new
> > RDHPCS
> > logins will need to go through the Princeton Bastion.  Logging In
> > <https://rdhpcs-common-
docs.rdhpcs.noaa.gov/wiki/index.php/Logging_in>
> >
> >
> >
> > *The next Jet Maintenance Downtime is TBD*
> >
> > ***
> >
> >
> > I tried to login Princeton but the connection is closed too.
> >
> >
> > Really appreciate your time and help on this.
> >
> >
> > Best,
> >
> > Yan
> >
> > On Tue, May 11, 2021 at 11:57 AM John Halley Gotway via RT <
> > met_help at ucar.edu> wrote:
> >
> > > Yan,
> > >
> > > George and I tried to take a look at this, but couldn't find the
> > > data.
> > >
> > >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210510214154
> > >
> > > I checked on Hera and WCOSS but couldn't find the data there. I
> > > should be
> > > able to log on to jet, but when I try, it boots me off:
> > >
> > > Authentication failed.
> > > Connection to jet-rsa.boulder.rdhpcs.noaa.gov closed.
> > >
> > > Can you confirm that you're running on jet? And any idea of jet
is
> > > down
> > > today or something?
> > >
> > > Thanks,
> > > John
> > >
> > >
> > > On Tue, May 11, 2021 at 7:46 AM Yan Jin - NOAA Affiliate via RT
<
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846
>
> > > >
> > > > Hi John,
> > > >
> > > > Thanks so much!
> > > >
> > > > Best,
> > > > Yan
> > > >
> > > >
> > > > On Tue, May 11, 2021 at 9:44 AM John Halley Gotway via RT <
> > > > met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > > Hi Yan,
> > > > >
> > > > > I'm going to assign this ticket to George and ask him to
take a
> > > > > look
> > > and
> > > > > debug this runtime error.
> > > > >
> > > > > Thanks,
> > > > > John HG
> > > > >
> > > > > On Mon, May 10, 2021 at 3:49 PM Yan Jin - NOAA Affiliate via
RT <
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > > >
> > > > > > Mon May 10 15:49:51 2021: Request 99846 was acted upon.
> > > > > > Transaction: Ticket created by yan.jin at noaa.gov
> > > > > >        Queue: met_help
> > > > > >      Subject: ask a question about METplus config of TC
> > > > > >        Owner: Nobody
> > > > > >   Requestors: yan.jin at noaa.gov
> > > > > >       Status: new
> > > > > >  Ticket <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846
> > > > >
> > > > > >
> > > > > >
> > > > > > Hi,
> > > > > >
> > > > > > This is Yan, again. I ran the METplus config file after
our
> > > discussion,
> > > > > > however, it seems that the tc_pairs tcst file wasn't
generated.
> > > > > >
> > > > > > Here is the script in my directory on Jet:
> > > > > > I set TC_PAIRS_STORM_ID = AL292020, so the code can find
the
> > > > > > input
> > > > adeck
> > > > > > and bdeck data correctly
> > > > > > *----
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test.conf
> > > > > >
> > > > > > The output directory is
> > > > > >
> /mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs
> > > > > > and the tc_pairs data: al292020_pairs.dat.tcst
> > > > > >
> > > > > > log file
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210510214154
> > > > > > *-----
> > > > > >
> > > > > > The error starts:[image: image.png]
> > > > > >
> > > > > > Then I checked al292020_pairs.dat.tcst, and it is empty.
> > > > > >
> > > > > > I wonder if there is anything I missed?
> > > > > >
> > > > > > Many thanks.
> > > > > > Best,
> > > > > > Yan
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
>
>
>
>

------------------------------------------------
Subject: ask a question about METplus config of TC
From: George McCabe
Time: Wed May 12 15:19:59 2021

Hi Yan,

It looks like the values set for init_beg and init_end in the TCpairs
config file are both "20200101_000000" because the INIT_BEG and
INIT_END
time is 2020. There are no adeck tracks for this time, so all of the
data
is being filtered out.

This is a little confusing in METplus 3.1 because INIT_BEG and
INIT_END
control the time looping of execution (if applicable for the use case)
AND
the values passed to the wrapped TCPairs config file. In METplus 4.0.0
(which was released yesterday and should be available on jet soon!),
you
can set INIT_BEG and INIT_END to control time looping (if desired) and
use
TC_PAIRS_INIT_BEG and TC_PAIRS_INIT_END to specify the values to pass
to
the TCPairs config file.

If you were using METplus 4.0.0, I would recommend setting something
like:


LOOP_BY = INIT
INIT_TIME_FMT = %Y
INIT_BEG = 2020
INIT_END = 2020
INIT_INCREMENT = 21600

TC_PAIRS_INIT_BEG = 20201030_000000
TC_PAIRS_INIT_END = 20201031_000000


This would run once using 2020 to find filenames and process data from
the
files that are initialized between 20201030_000000 and
20201031_000000.

If using METplus 3.1, you will have to use the INIT_BEG and INIT_END
values
to set the values in the wrapped TCPairs config. This is OK for this
use
case because you have LOOP_ORDER set to processes and TCPairs only
runs
once with that setting.

LOOP_BY = INIT
INIT_TIME_FMT = %Y%m%d
INIT_BEG = 20201030
INIT_END = 20201031

This configuration should run and only use tracks initialized between
2020-10-30 @ 0Z and 2020-10-31 @ 0Z. If you need to specify the init
time
range with hour precision, you can do that as well. Here is an
example:

LOOP_BY = INIT
INIT_TIME_FMT = %Y%m%d%H
INIT_BEG = 2020103012
INIT_END = 2020103112

This configuration will only use tracks initialized between 2020-10-30
@
12Z and 2020-10-31 @ 12Z.

You can also restrict the data used by valid time as well by using
TC_PAIRS_VALID_BEG and TC_PAIRS_VALID_END in METplus 4.0.0 or
VALID_BEG and
VALID_END for 3.1.

Let me know if you are still unable to produce output with these
changes. I
was able to run tc_pairs by hand on Jet and produce a file with output
tracks. You can view it here if you'd like:

/lfs1/HFIP/dtc-hurr/George.Mccabe/al292020_pairs.dat.tcst

These are the commands I used:

export BASIN="[]";
export CYCLONE="[]";
export DLAND_FILE="/contrib/met/9.1/share/met/tc_data/
dland_global_tenth_degree.nc";
export INIT_BEG="20201030_000000";
export INIT_END="20201031_000000";
export INIT_EXCLUDE="[]";
export INIT_INCLUDE="[]";
export MODEL="[]";
export STORM_ID="[\"AL292020\"]";
export STORM_NAME="[]";
export VALID_BEG="";
export VALID_END="";
/contrib/met/9.1/bin/tc_pairs -v 4 -bdeck
/mnt/lfs4/HFIP/hwrf-data/hwrf-input/abdeck/btk/bal292020.dat -adeck
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/data/aal292020.dat -config
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/use_cases/model_applications/tc_and_extra_tc/TCPairsETCConfig
-out /lfs1/HFIP/dtc-hurr/George.Mccabe/al292020_pairs.dat

I took the values from your METplus log file and modified them to
write to
my directory and use a different init time range.

Thanks,
George




On Wed, May 12, 2021 at 2:04 PM Yan Jin - NOAA Affiliate via RT <
met_help at ucar.edu> wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846 >
>
> Hi George,
>
> Thanks very much for your instructions. I set "PROCESS_LIST =
TCPairs", and
> didn't specify models: "MODEL =". Then run the script. However, the
output
> file is still empty. It seems that tc_pairs matched 0 adeck track to
1
> bdeck track. I also tried to set "MODEL = HWRF, HMON", but the
output file
> is empty too.
>
> Here is the log file:
>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210512193152
>
> And here is the output file:
>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
>
> Many thanks.
> Best,
> Yan
>
> On Wed, May 12, 2021 at 11:36 AM George McCabe via RT
<met_help at ucar.edu>
> wrote:
>
> > Hi Yan,
> >
> > I was able to log into Jet this morning and look at your error.
> >
> > It looks like the error in the call to Rscript is due to there
being no
> > data in the input files generated by TCPairs. There are
inconsistencies
> in
> > the TCMPRPlotter wrapper that prevent the log output from the run
to be
> > seen, which would have made it more clear why the issue was
occurring.
> >
> > I would recommend changing the PROCESS_LIST value in your METplus
> > configuration file to only run TCPairs and iterating on that until
you
> get
> > output from that tool. Once you can confirm that you have output,
you can
> > add TCMPRPlotter back to the PROCESS_LIST and test on that
portion.
> >
> > I will create a GitHub issue for updating the TCMPRPlotter wrapper
to
> > produce useful logging output that will make things more clear
going
> > forward.
> >
> > Let me know if you run into any other issues or need help
configuring
> > TCPairs to run properly. I'm also happy to attend another working
session
> > if that is helpful.
> >
> > Thanks,
> > George
> >
> >
> > On Tue May 11 10:05:06 2021, yan.jin at noaa.gov wrote:
> > > Hi John,
> > >
> > >
> > > The directories and data I mentioned in my last email are all
located
> > > in
> > > Jet.
> > >
> > >
> > > Here is the email I got regarding the Jet maintenance:
> > >
> > > ***
> > >
> > > *05/11/21 - Boulder Bastion Maintenance Downtime*
> > > Boulder Bastion maintenance is scheduled for Tuesday 05/11/21
from
> > > 0600 ~
> > > 1000 MT. Boulder Bastion login sessions will be dropped and all
new
> > > RDHPCS
> > > logins will need to go through the Princeton Bastion.  Logging
In
> > > <https://rdhpcs-common-
docs.rdhpcs.noaa.gov/wiki/index.php/Logging_in>
> > >
> > >
> > >
> > > *The next Jet Maintenance Downtime is TBD*
> > >
> > > ***
> > >
> > >
> > > I tried to login Princeton but the connection is closed too.
> > >
> > >
> > > Really appreciate your time and help on this.
> > >
> > >
> > > Best,
> > >
> > > Yan
> > >
> > > On Tue, May 11, 2021 at 11:57 AM John Halley Gotway via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > > Yan,
> > > >
> > > > George and I tried to take a look at this, but couldn't find
the
> > > > data.
> > > >
> > > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210510214154
> > > >
> > > > I checked on Hera and WCOSS but couldn't find the data there.
I
> > > > should be
> > > > able to log on to jet, but when I try, it boots me off:
> > > >
> > > > Authentication failed.
> > > > Connection to jet-rsa.boulder.rdhpcs.noaa.gov closed.
> > > >
> > > > Can you confirm that you're running on jet? And any idea of
jet is
> > > > down
> > > > today or something?
> > > >
> > > > Thanks,
> > > > John
> > > >
> > > >
> > > > On Tue, May 11, 2021 at 7:46 AM Yan Jin - NOAA Affiliate via
RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846 >
> > > > >
> > > > > Hi John,
> > > > >
> > > > > Thanks so much!
> > > > >
> > > > > Best,
> > > > > Yan
> > > > >
> > > > >
> > > > > On Tue, May 11, 2021 at 9:44 AM John Halley Gotway via RT <
> > > > > met_help at ucar.edu>
> > > > > wrote:
> > > > >
> > > > > > Hi Yan,
> > > > > >
> > > > > > I'm going to assign this ticket to George and ask him to
take a
> > > > > > look
> > > > and
> > > > > > debug this runtime error.
> > > > > >
> > > > > > Thanks,
> > > > > > John HG
> > > > > >
> > > > > > On Mon, May 10, 2021 at 3:49 PM Yan Jin - NOAA Affiliate
via RT <
> > > > > > met_help at ucar.edu> wrote:
> > > > > >
> > > > > > >
> > > > > > > Mon May 10 15:49:51 2021: Request 99846 was acted upon.
> > > > > > > Transaction: Ticket created by yan.jin at noaa.gov
> > > > > > >        Queue: met_help
> > > > > > >      Subject: ask a question about METplus config of TC
> > > > > > >        Owner: Nobody
> > > > > > >   Requestors: yan.jin at noaa.gov
> > > > > > >       Status: new
> > > > > > >  Ticket <URL:
> > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846
> > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > This is Yan, again. I ran the METplus config file after
our
> > > > discussion,
> > > > > > > however, it seems that the tc_pairs tcst file wasn't
generated.
> > > > > > >
> > > > > > > Here is the script in my directory on Jet:
> > > > > > > I set TC_PAIRS_STORM_ID = AL292020, so the code can find
the
> > > > > > > input
> > > > > adeck
> > > > > > > and bdeck data correctly
> > > > > > > *----
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test.conf
> > > > > > >
> > > > > > > The output directory is
> > > > > > >
> >
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs
> > > > > > > and the tc_pairs data: al292020_pairs.dat.tcst
> > > > > > >
> > > > > > > log file
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210510214154
> > > > > > > *-----
> > > > > > >
> > > > > > > The error starts:[image: image.png]
> > > > > > >
> > > > > > > Then I checked al292020_pairs.dat.tcst, and it is empty.
> > > > > > >
> > > > > > > I wonder if there is anything I missed?
> > > > > > >
> > > > > > > Many thanks.
> > > > > > > Best,
> > > > > > > Yan
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > >
> >
> >
> >
> >
>
>

--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.

------------------------------------------------
Subject: ask a question about METplus config of TC
From: Yan Jin - NOAA Affiliate
Time: Thu May 13 12:36:52 2021

Hi George,

Thanks so much. Now tc_pairs works, and the output file can be
generated.
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst

Then I set "PROCESS_LIST = TCPairs, TCMPRPlotter", and I kept most
parameters that initialized with "TCMPR_PLOTTER" the same as the
default
configuration.

my METplus config file:
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test.conf

then run script
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/user_config/run.sh

However, the Rscript cannot plot figures.
I checked the log file:
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210513182819
[image: image.png]

It seems that the error "NameError: name 'produtil' is not defined"?

Could you give me some suggestions?

Many thanks.
Best,
Yan

On Wed, May 12, 2021 at 5:20 PM George McCabe via RT
<met_help at ucar.edu>
wrote:

> Hi Yan,
>
> It looks like the values set for init_beg and init_end in the
TCpairs
> config file are both "20200101_000000" because the INIT_BEG and
INIT_END
> time is 2020. There are no adeck tracks for this time, so all of the
data
> is being filtered out.
>
> This is a little confusing in METplus 3.1 because INIT_BEG and
INIT_END
> control the time looping of execution (if applicable for the use
case) AND
> the values passed to the wrapped TCPairs config file. In METplus
4.0.0
> (which was released yesterday and should be available on jet soon!),
you
> can set INIT_BEG and INIT_END to control time looping (if desired)
and use
> TC_PAIRS_INIT_BEG and TC_PAIRS_INIT_END to specify the values to
pass to
> the TCPairs config file.
>
> If you were using METplus 4.0.0, I would recommend setting something
like:
>
>
> LOOP_BY = INIT
> INIT_TIME_FMT = %Y
> INIT_BEG = 2020
> INIT_END = 2020
> INIT_INCREMENT = 21600
>
> TC_PAIRS_INIT_BEG = 20201030_000000
> TC_PAIRS_INIT_END = 20201031_000000
>
>
> This would run once using 2020 to find filenames and process data
from the
> files that are initialized between 20201030_000000 and
20201031_000000.
>
> If using METplus 3.1, you will have to use the INIT_BEG and INIT_END
values
> to set the values in the wrapped TCPairs config. This is OK for this
use
> case because you have LOOP_ORDER set to processes and TCPairs only
runs
> once with that setting.
>
> LOOP_BY = INIT
> INIT_TIME_FMT = %Y%m%d
> INIT_BEG = 20201030
> INIT_END = 20201031
>
> This configuration should run and only use tracks initialized
between
> 2020-10-30 @ 0Z and 2020-10-31 @ 0Z. If you need to specify the init
time
> range with hour precision, you can do that as well. Here is an
example:
>
> LOOP_BY = INIT
> INIT_TIME_FMT = %Y%m%d%H
> INIT_BEG = 2020103012
> INIT_END = 2020103112
>
> This configuration will only use tracks initialized between 2020-10-
30 @
> 12Z and 2020-10-31 @ 12Z.
>
> You can also restrict the data used by valid time as well by using
> TC_PAIRS_VALID_BEG and TC_PAIRS_VALID_END in METplus 4.0.0 or
VALID_BEG and
> VALID_END for 3.1.
>
> Let me know if you are still unable to produce output with these
changes. I
> was able to run tc_pairs by hand on Jet and produce a file with
output
> tracks. You can view it here if you'd like:
>
> /lfs1/HFIP/dtc-hurr/George.Mccabe/al292020_pairs.dat.tcst
>
> These are the commands I used:
>
> export BASIN="[]";
> export CYCLONE="[]";
> export DLAND_FILE="/contrib/met/9.1/share/met/tc_data/
> dland_global_tenth_degree.nc";
> export INIT_BEG="20201030_000000";
> export INIT_END="20201031_000000";
> export INIT_EXCLUDE="[]";
> export INIT_INCLUDE="[]";
> export MODEL="[]";
> export STORM_ID="[\"AL292020\"]";
> export STORM_NAME="[]";
> export VALID_BEG="";
> export VALID_END="";
> /contrib/met/9.1/bin/tc_pairs -v 4 -bdeck
> /mnt/lfs4/HFIP/hwrf-data/hwrf-input/abdeck/btk/bal292020.dat -adeck
> /mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/data/aal292020.dat
-config
>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/use_cases/model_applications/tc_and_extra_tc/TCPairsETCConfig
> -out /lfs1/HFIP/dtc-hurr/George.Mccabe/al292020_pairs.dat
>
> I took the values from your METplus log file and modified them to
write to
> my directory and use a different init time range.
>
> Thanks,
> George
>
>
>
>
> On Wed, May 12, 2021 at 2:04 PM Yan Jin - NOAA Affiliate via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846 >
> >
> > Hi George,
> >
> > Thanks very much for your instructions. I set "PROCESS_LIST =
TCPairs",
> and
> > didn't specify models: "MODEL =". Then run the script. However,
the
> output
> > file is still empty. It seems that tc_pairs matched 0 adeck track
to 1
> > bdeck track. I also tried to set "MODEL = HWRF, HMON", but the
output
> file
> > is empty too.
> >
> > Here is the log file:
> >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210512193152
> >
> > And here is the output file:
> >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> >
> > Many thanks.
> > Best,
> > Yan
> >
> > On Wed, May 12, 2021 at 11:36 AM George McCabe via RT
<met_help at ucar.edu
> >
> > wrote:
> >
> > > Hi Yan,
> > >
> > > I was able to log into Jet this morning and look at your error.
> > >
> > > It looks like the error in the call to Rscript is due to there
being no
> > > data in the input files generated by TCPairs. There are
inconsistencies
> > in
> > > the TCMPRPlotter wrapper that prevent the log output from the
run to be
> > > seen, which would have made it more clear why the issue was
occurring.
> > >
> > > I would recommend changing the PROCESS_LIST value in your
METplus
> > > configuration file to only run TCPairs and iterating on that
until you
> > get
> > > output from that tool. Once you can confirm that you have
output, you
> can
> > > add TCMPRPlotter back to the PROCESS_LIST and test on that
portion.
> > >
> > > I will create a GitHub issue for updating the TCMPRPlotter
wrapper to
> > > produce useful logging output that will make things more clear
going
> > > forward.
> > >
> > > Let me know if you run into any other issues or need help
configuring
> > > TCPairs to run properly. I'm also happy to attend another
working
> session
> > > if that is helpful.
> > >
> > > Thanks,
> > > George
> > >
> > >
> > > On Tue May 11 10:05:06 2021, yan.jin at noaa.gov wrote:
> > > > Hi John,
> > > >
> > > >
> > > > The directories and data I mentioned in my last email are all
located
> > > > in
> > > > Jet.
> > > >
> > > >
> > > > Here is the email I got regarding the Jet maintenance:
> > > >
> > > > ***
> > > >
> > > > *05/11/21 - Boulder Bastion Maintenance Downtime*
> > > > Boulder Bastion maintenance is scheduled for Tuesday 05/11/21
from
> > > > 0600 ~
> > > > 1000 MT. Boulder Bastion login sessions will be dropped and
all new
> > > > RDHPCS
> > > > logins will need to go through the Princeton Bastion.  Logging
In
> > > > <
> https://rdhpcs-common-
docs.rdhpcs.noaa.gov/wiki/index.php/Logging_in>
> > > >
> > > >
> > > >
> > > > *The next Jet Maintenance Downtime is TBD*
> > > >
> > > > ***
> > > >
> > > >
> > > > I tried to login Princeton but the connection is closed too.
> > > >
> > > >
> > > > Really appreciate your time and help on this.
> > > >
> > > >
> > > > Best,
> > > >
> > > > Yan
> > > >
> > > > On Tue, May 11, 2021 at 11:57 AM John Halley Gotway via RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > > > Yan,
> > > > >
> > > > > George and I tried to take a look at this, but couldn't find
the
> > > > > data.
> > > > >
> > > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210510214154
> > > > >
> > > > > I checked on Hera and WCOSS but couldn't find the data
there. I
> > > > > should be
> > > > > able to log on to jet, but when I try, it boots me off:
> > > > >
> > > > > Authentication failed.
> > > > > Connection to jet-rsa.boulder.rdhpcs.noaa.gov closed.
> > > > >
> > > > > Can you confirm that you're running on jet? And any idea of
jet is
> > > > > down
> > > > > today or something?
> > > > >
> > > > > Thanks,
> > > > > John
> > > > >
> > > > >
> > > > > On Tue, May 11, 2021 at 7:46 AM Yan Jin - NOAA Affiliate via
RT <
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > > >
> > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846 >
> > > > > >
> > > > > > Hi John,
> > > > > >
> > > > > > Thanks so much!
> > > > > >
> > > > > > Best,
> > > > > > Yan
> > > > > >
> > > > > >
> > > > > > On Tue, May 11, 2021 at 9:44 AM John Halley Gotway via RT
<
> > > > > > met_help at ucar.edu>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi Yan,
> > > > > > >
> > > > > > > I'm going to assign this ticket to George and ask him to
take a
> > > > > > > look
> > > > > and
> > > > > > > debug this runtime error.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > John HG
> > > > > > >
> > > > > > > On Mon, May 10, 2021 at 3:49 PM Yan Jin - NOAA Affiliate
via
> RT <
> > > > > > > met_help at ucar.edu> wrote:
> > > > > > >
> > > > > > > >
> > > > > > > > Mon May 10 15:49:51 2021: Request 99846 was acted
upon.
> > > > > > > > Transaction: Ticket created by yan.jin at noaa.gov
> > > > > > > >        Queue: met_help
> > > > > > > >      Subject: ask a question about METplus config of
TC
> > > > > > > >        Owner: Nobody
> > > > > > > >   Requestors: yan.jin at noaa.gov
> > > > > > > >       Status: new
> > > > > > > >  Ticket <URL:
> > > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846
> > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Hi,
> > > > > > > >
> > > > > > > > This is Yan, again. I ran the METplus config file
after our
> > > > > discussion,
> > > > > > > > however, it seems that the tc_pairs tcst file wasn't
> generated.
> > > > > > > >
> > > > > > > > Here is the script in my directory on Jet:
> > > > > > > > I set TC_PAIRS_STORM_ID = AL292020, so the code can
find the
> > > > > > > > input
> > > > > > adeck
> > > > > > > > and bdeck data correctly
> > > > > > > > *----
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test.conf
> > > > > > > >
> > > > > > > > The output directory is
> > > > > > > >
> > >
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs
> > > > > > > > and the tc_pairs data: al292020_pairs.dat.tcst
> > > > > > > >
> > > > > > > > log file
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210510214154
> > > > > > > > *-----
> > > > > > > >
> > > > > > > > The error starts:[image: image.png]
> > > > > > > >
> > > > > > > > Then I checked al292020_pairs.dat.tcst, and it is
empty.
> > > > > > > >
> > > > > > > > I wonder if there is anything I missed?
> > > > > > > >
> > > > > > > > Many thanks.
> > > > > > > > Best,
> > > > > > > > Yan
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > >
> > >
> > >
> > >
> >
> >
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>

------------------------------------------------
Subject: ask a question about METplus config of TC
From: George McCabe
Time: Thu May 13 13:04:34 2021

Hi Yan,

The produtil is due to a bug in the wrapper in the logic that reports
when
an error occurs. I ran the command on Jet by hand and it looks like it
is
still not finding any input data to plot. You can run the following
commands (after loading all of the required modules):

export MET_INSTALL_DIR=/contrib/met/9.1

Rscript /contrib/met/9.1/share/met/Rscripts/plot_tcmpr.R \
-config
/contrib/met/METplus/METplus-
3.1.1/parm/use_cases/model_applications/tc_and_extra_tc/TCMPRPlotterConfig_customize
\
-subtitle "Your subtitle goes here" -ylab "Your y-label goes here" \
-dep AMAX_WIND-BMAX_WIND,TK_ERR -legend "Your legend text goes
here..." \
-plot MEAN,MEDIAN,BOXPLOT \
-lookin
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
\
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
\
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
\
-outdir
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tcmpr_plots


Here is the output from that run:

Reading plot configuration file:
/contrib/met/METplus/METplus-
3.1.1/parm/use_cases/model_applications/tc_and_extra_tc/TCMPRPlotterConfig_customize
No plotting configuration file specified.
CALLING:  /contrib/met/9.1/bin/tc_stat -lookin
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
-lookin
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
-lookin
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
-job filter -dump_row /tmp/plot_tcmpr_116966.tcst  -event_equal true
-v 3
DEBUG 1: [Source 1] Lookin:
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
DEBUG 1: [Source 2] Lookin:
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
DEBUG 1: [Source 3] Lookin:
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
DEBUG 2:
DEBUG 2: Processing Job 1: -job filter -event_equal true -dump_row
/tmp/plot_tcmpr_116966.tcst
DEBUG 3: Applying track-based event equalization logic.
DEBUG 3: Reading file 1 of 3:
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
DEBUG 3: Reading file 2 of 3:
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
DEBUG 3: Reading file 3 of 3:
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
DEBUG 3: For track-based event equalization, identified 0 common cases
for
146 models: AC00 AEMI AEMN AP01 AP02 AP03 AP04 AP05 AP06 AP07 AP08
AP09
AP0I AP10 AP11 AP12 AP13 AP14 AP15 AP16 AP17 AP18 AP19 AP1I AP20 AP21
AP22
AP23 AP24 AP25 AP26 AP27 AP28 AP29 AP30 API CARQ CEM2 CEMI CEMN CLP5
CMC
CMC2 CMCI COTC COTI CTCX DRCL DSHP EC00 EEMN EGRR EMN2 EMN3 EMN4 EMNI
EMX
EMX2 EMXI EN01 EN02 EN03 EN04 EN05 EN06 EN07 EN08 EN09 EN0I EN10 EN11
EN12
EN13 EN14 EN15 EN16 EN17 EN18 EN19 EN1I EN20 EN21 EN22 EN23 EN24 EN25
ENI
EP01 EP02 EP03 EP04 EP05 EP06 EP07 EP08 EP09 EP0I EP10 EP11 EP12 EP13
EP14
EP15 EP16 EP17 EP18 EP19 EP1I EP20 EP21 EP22 EP23 EP24 EP25 EPI GFEX
GFSI
GFSO HMNI HMON HWF2 HWFI HWRF ICON IVCN LGEM NGX NGX2 NGXI NVGM OCD5
RVCN
SHF5 SHIP TABD TABM TABS TCLP TVCA TVCE TVCN TVCX UKX UKX2 UKXI XTRP
DEBUG 1: Event equalization of tracks found no common cases.
DEBUG 1: Creating output dump file: /tmp/plot_tcmpr_116966.tcst
FILTER: -job filter -line_type TCMPR -event_equal true -dump_row
/tmp/plot_tcmpr_116966.tcst

DEBUG 2: Job 1 used 0 out of 0 lines read.
DEBUG 3: Total lines read                 = 0
DEBUG 3: Total lines kept                 = 0
DEBUG 3: Rejected for track watch/warn    = 0
DEBUG 3: Rejected for init threshold      = 0
DEBUG 3: Rejected for init string         = 0
DEBUG 3: Rejected for out init mask       = 0
DEBUG 3: Rejected for water only          = 0
DEBUG 3: Rejected for rapid inten         = 0
DEBUG 3: Rejected for landfall            = 0
DEBUG 3: Rejected for amodel              = 0
DEBUG 3: Rejected for bmodel              = 0
DEBUG 3: Rejected for desc                = 0
DEBUG 3: Rejected for storm id            = 0
DEBUG 3: Rejected for basin               = 0
DEBUG 3: Rejected for cyclone             = 0
DEBUG 3: Rejected for storm name          = 0
DEBUG 3: Rejected for init time           = 0
DEBUG 3: Rejected for init hour           = 0
DEBUG 3: Rejected for lead time           = 0
DEBUG 3: Rejected for required lead times = 0
DEBUG 3: Rejected for valid time          = 0
DEBUG 3: Rejected for valid hour          = 0
DEBUG 3: Rejected for init mask           = 0
DEBUG 3: Rejected for valid mask          = 0
DEBUG 3: Rejected for line type           = 0
DEBUG 3: Rejected for numeric threshold   = 0
DEBUG 3: Rejected for string matching     = 0
DEBUG 3: Rejected for match points        = 0
DEBUG 3: Rejected for event equal         = 0
DEBUG 3: Rejected for out init mask       = 0
DEBUG 3: Rejected for out valid mask      = 0
Reading track data: /tmp/plot_tcmpr_116966.tcst
Error in read.table(tcst_tmp_file, header = TRUE) :
  no lines available in input
Execution halted



It sounds like filter arguments may need to be provided to produce TC-
Stat
output (which is called by the R script). You can set this in your
METplus
config file with the TCMPR_PLOTTER_FILTER variable, but it will be
difficult to debug since the log output is not shown. I would
recommend
modifying the Rscript call from this email and add the -filter
argument
with any filter information to pass to TCStat. I am not sure settings
would
help you produce output, but I have reached out to other METplus team
members for suggestions.

Thanks,
George



On Thu, May 13, 2021 at 12:37 PM Yan Jin - NOAA Affiliate via RT <
met_help at ucar.edu> wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846 >
>
> Hi George,
>
> Thanks so much. Now tc_pairs works, and the output file can be
generated.
>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
>
> Then I set "PROCESS_LIST = TCPairs, TCMPRPlotter", and I kept most
> parameters that initialized with "TCMPR_PLOTTER" the same as the
default
> configuration.
>
> my METplus config file:
>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test.conf
>
> then run script
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/user_config/run.sh
>
> However, the Rscript cannot plot figures.
> I checked the log file:
>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210513182819
> [image: image.png]
>
> It seems that the error "NameError: name 'produtil' is not defined"?
>
> Could you give me some suggestions?
>
> Many thanks.
> Best,
> Yan
>
> On Wed, May 12, 2021 at 5:20 PM George McCabe via RT
<met_help at ucar.edu>
> wrote:
>
> > Hi Yan,
> >
> > It looks like the values set for init_beg and init_end in the
TCpairs
> > config file are both "20200101_000000" because the INIT_BEG and
INIT_END
> > time is 2020. There are no adeck tracks for this time, so all of
the data
> > is being filtered out.
> >
> > This is a little confusing in METplus 3.1 because INIT_BEG and
INIT_END
> > control the time looping of execution (if applicable for the use
case)
> AND
> > the values passed to the wrapped TCPairs config file. In METplus
4.0.0
> > (which was released yesterday and should be available on jet
soon!), you
> > can set INIT_BEG and INIT_END to control time looping (if desired)
and
> use
> > TC_PAIRS_INIT_BEG and TC_PAIRS_INIT_END to specify the values to
pass to
> > the TCPairs config file.
> >
> > If you were using METplus 4.0.0, I would recommend setting
something
> like:
> >
> >
> > LOOP_BY = INIT
> > INIT_TIME_FMT = %Y
> > INIT_BEG = 2020
> > INIT_END = 2020
> > INIT_INCREMENT = 21600
> >
> > TC_PAIRS_INIT_BEG = 20201030_000000
> > TC_PAIRS_INIT_END = 20201031_000000
> >
> >
> > This would run once using 2020 to find filenames and process data
from
> the
> > files that are initialized between 20201030_000000 and
20201031_000000.
> >
> > If using METplus 3.1, you will have to use the INIT_BEG and
INIT_END
> values
> > to set the values in the wrapped TCPairs config. This is OK for
this use
> > case because you have LOOP_ORDER set to processes and TCPairs only
runs
> > once with that setting.
> >
> > LOOP_BY = INIT
> > INIT_TIME_FMT = %Y%m%d
> > INIT_BEG = 20201030
> > INIT_END = 20201031
> >
> > This configuration should run and only use tracks initialized
between
> > 2020-10-30 @ 0Z and 2020-10-31 @ 0Z. If you need to specify the
init time
> > range with hour precision, you can do that as well. Here is an
example:
> >
> > LOOP_BY = INIT
> > INIT_TIME_FMT = %Y%m%d%H
> > INIT_BEG = 2020103012
> > INIT_END = 2020103112
> >
> > This configuration will only use tracks initialized between 2020-
10-30 @
> > 12Z and 2020-10-31 @ 12Z.
> >
> > You can also restrict the data used by valid time as well by using
> > TC_PAIRS_VALID_BEG and TC_PAIRS_VALID_END in METplus 4.0.0 or
VALID_BEG
> and
> > VALID_END for 3.1.
> >
> > Let me know if you are still unable to produce output with these
> changes. I
> > was able to run tc_pairs by hand on Jet and produce a file with
output
> > tracks. You can view it here if you'd like:
> >
> > /lfs1/HFIP/dtc-hurr/George.Mccabe/al292020_pairs.dat.tcst
> >
> > These are the commands I used:
> >
> > export BASIN="[]";
> > export CYCLONE="[]";
> > export DLAND_FILE="/contrib/met/9.1/share/met/tc_data/
> > dland_global_tenth_degree.nc";
> > export INIT_BEG="20201030_000000";
> > export INIT_END="20201031_000000";
> > export INIT_EXCLUDE="[]";
> > export INIT_INCLUDE="[]";
> > export MODEL="[]";
> > export STORM_ID="[\"AL292020\"]";
> > export STORM_NAME="[]";
> > export VALID_BEG="";
> > export VALID_END="";
> > /contrib/met/9.1/bin/tc_pairs -v 4 -bdeck
> > /mnt/lfs4/HFIP/hwrf-data/hwrf-input/abdeck/btk/bal292020.dat
-adeck
> > /mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/data/aal292020.dat
-config
> >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/use_cases/model_applications/tc_and_extra_tc/TCPairsETCConfig
> > -out /lfs1/HFIP/dtc-hurr/George.Mccabe/al292020_pairs.dat
> >
> > I took the values from your METplus log file and modified them to
write
> to
> > my directory and use a different init time range.
> >
> > Thanks,
> > George
> >
> >
> >
> >
> > On Wed, May 12, 2021 at 2:04 PM Yan Jin - NOAA Affiliate via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846 >
> > >
> > > Hi George,
> > >
> > > Thanks very much for your instructions. I set "PROCESS_LIST =
TCPairs",
> > and
> > > didn't specify models: "MODEL =". Then run the script. However,
the
> > output
> > > file is still empty. It seems that tc_pairs matched 0 adeck
track to 1
> > > bdeck track. I also tried to set "MODEL = HWRF, HMON", but the
output
> > file
> > > is empty too.
> > >
> > > Here is the log file:
> > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210512193152
> > >
> > > And here is the output file:
> > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> > >
> > > Many thanks.
> > > Best,
> > > Yan
> > >
> > > On Wed, May 12, 2021 at 11:36 AM George McCabe via RT <
> met_help at ucar.edu
> > >
> > > wrote:
> > >
> > > > Hi Yan,
> > > >
> > > > I was able to log into Jet this morning and look at your
error.
> > > >
> > > > It looks like the error in the call to Rscript is due to there
being
> no
> > > > data in the input files generated by TCPairs. There are
> inconsistencies
> > > in
> > > > the TCMPRPlotter wrapper that prevent the log output from the
run to
> be
> > > > seen, which would have made it more clear why the issue was
> occurring.
> > > >
> > > > I would recommend changing the PROCESS_LIST value in your
METplus
> > > > configuration file to only run TCPairs and iterating on that
until
> you
> > > get
> > > > output from that tool. Once you can confirm that you have
output, you
> > can
> > > > add TCMPRPlotter back to the PROCESS_LIST and test on that
portion.
> > > >
> > > > I will create a GitHub issue for updating the TCMPRPlotter
wrapper to
> > > > produce useful logging output that will make things more clear
going
> > > > forward.
> > > >
> > > > Let me know if you run into any other issues or need help
configuring
> > > > TCPairs to run properly. I'm also happy to attend another
working
> > session
> > > > if that is helpful.
> > > >
> > > > Thanks,
> > > > George
> > > >
> > > >
> > > > On Tue May 11 10:05:06 2021, yan.jin at noaa.gov wrote:
> > > > > Hi John,
> > > > >
> > > > >
> > > > > The directories and data I mentioned in my last email are
all
> located
> > > > > in
> > > > > Jet.
> > > > >
> > > > >
> > > > > Here is the email I got regarding the Jet maintenance:
> > > > >
> > > > > ***
> > > > >
> > > > > *05/11/21 - Boulder Bastion Maintenance Downtime*
> > > > > Boulder Bastion maintenance is scheduled for Tuesday
05/11/21 from
> > > > > 0600 ~
> > > > > 1000 MT. Boulder Bastion login sessions will be dropped and
all new
> > > > > RDHPCS
> > > > > logins will need to go through the Princeton Bastion.
Logging In
> > > > > <
> > https://rdhpcs-common-
docs.rdhpcs.noaa.gov/wiki/index.php/Logging_in>
> > > > >
> > > > >
> > > > >
> > > > > *The next Jet Maintenance Downtime is TBD*
> > > > >
> > > > > ***
> > > > >
> > > > >
> > > > > I tried to login Princeton but the connection is closed too.
> > > > >
> > > > >
> > > > > Really appreciate your time and help on this.
> > > > >
> > > > >
> > > > > Best,
> > > > >
> > > > > Yan
> > > > >
> > > > > On Tue, May 11, 2021 at 11:57 AM John Halley Gotway via RT <
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > > > Yan,
> > > > > >
> > > > > > George and I tried to take a look at this, but couldn't
find the
> > > > > > data.
> > > > > >
> > > > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210510214154
> > > > > >
> > > > > > I checked on Hera and WCOSS but couldn't find the data
there. I
> > > > > > should be
> > > > > > able to log on to jet, but when I try, it boots me off:
> > > > > >
> > > > > > Authentication failed.
> > > > > > Connection to jet-rsa.boulder.rdhpcs.noaa.gov closed.
> > > > > >
> > > > > > Can you confirm that you're running on jet? And any idea
of jet
> is
> > > > > > down
> > > > > > today or something?
> > > > > >
> > > > > > Thanks,
> > > > > > John
> > > > > >
> > > > > >
> > > > > > On Tue, May 11, 2021 at 7:46 AM Yan Jin - NOAA Affiliate
via RT <
> > > > > > met_help at ucar.edu> wrote:
> > > > > >
> > > > > > >
> > > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846
> >
> > > > > > >
> > > > > > > Hi John,
> > > > > > >
> > > > > > > Thanks so much!
> > > > > > >
> > > > > > > Best,
> > > > > > > Yan
> > > > > > >
> > > > > > >
> > > > > > > On Tue, May 11, 2021 at 9:44 AM John Halley Gotway via
RT <
> > > > > > > met_help at ucar.edu>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hi Yan,
> > > > > > > >
> > > > > > > > I'm going to assign this ticket to George and ask him
to
> take a
> > > > > > > > look
> > > > > > and
> > > > > > > > debug this runtime error.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > John HG
> > > > > > > >
> > > > > > > > On Mon, May 10, 2021 at 3:49 PM Yan Jin - NOAA
Affiliate via
> > RT <
> > > > > > > > met_help at ucar.edu> wrote:
> > > > > > > >
> > > > > > > > >
> > > > > > > > > Mon May 10 15:49:51 2021: Request 99846 was acted
upon.
> > > > > > > > > Transaction: Ticket created by yan.jin at noaa.gov
> > > > > > > > >        Queue: met_help
> > > > > > > > >      Subject: ask a question about METplus config of
TC
> > > > > > > > >        Owner: Nobody
> > > > > > > > >   Requestors: yan.jin at noaa.gov
> > > > > > > > >       Status: new
> > > > > > > > >  Ticket <URL:
> > > > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846
> > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Hi,
> > > > > > > > >
> > > > > > > > > This is Yan, again. I ran the METplus config file
after our
> > > > > > discussion,
> > > > > > > > > however, it seems that the tc_pairs tcst file wasn't
> > generated.
> > > > > > > > >
> > > > > > > > > Here is the script in my directory on Jet:
> > > > > > > > > I set TC_PAIRS_STORM_ID = AL292020, so the code can
find
> the
> > > > > > > > > input
> > > > > > > adeck
> > > > > > > > > and bdeck data correctly
> > > > > > > > > *----
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test.conf
> > > > > > > > >
> > > > > > > > > The output directory is
> > > > > > > > >
> > > >
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs
> > > > > > > > > and the tc_pairs data: al292020_pairs.dat.tcst
> > > > > > > > >
> > > > > > > > > log file
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210510214154
> > > > > > > > > *-----
> > > > > > > > >
> > > > > > > > > The error starts:[image: image.png]
> > > > > > > > >
> > > > > > > > > Then I checked al292020_pairs.dat.tcst, and it is
empty.
> > > > > > > > >
> > > > > > > > > I wonder if there is anything I missed?
> > > > > > > > >
> > > > > > > > > Many thanks.
> > > > > > > > > Best,
> > > > > > > > > Yan
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> >
> > --
> > George McCabe - Software Engineer III
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > 303-497-2768
> > ---
> > My working day may not be your working day. Please do not feel
obliged to
> > reply to this email outside of your normal working hours.
> >
> >
>
>

--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.

------------------------------------------------
Subject: ask a question about METplus config of TC
From: Yan Jin - NOAA Affiliate
Time: Fri May 14 08:19:57 2021

Hi George,

Thanks so much for your help. Finally the code can run without errors.
Here
is the steps and results:

https://docs.google.com/document/d/1xRF3aiPATcX5G37ngJj-
65pl44LN7mGHo34Sf8i3zpo/edit

I also put questions in the document, I also pasted here:

Questions:

4.1: How can I do the consensus forecasts in METplus config file as I
did
in tc_pairs?

(/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_292020/TCPairsConfig_run1)

//

// Specify how consensus forecasts should be defined.

//

consensus = [

 {

  name = "CONS";

  members = ["HWRF","GFSO","HMON"];

  required = [false,false,false];

  min_req = 2;

 }

];

4.2: I am a little confused about

      ***

       LOOP_ORDER = processes

       LOOP_BY = INIT

      ***

Could you explain these two command lines and give me some examples?

Thanks very much for your time and help on my questions.

Best,
Yan

On Thu, May 13, 2021 at 3:04 PM George McCabe via RT
<met_help at ucar.edu>
wrote:

> Hi Yan,
>
> The produtil is due to a bug in the wrapper in the logic that
reports when
> an error occurs. I ran the command on Jet by hand and it looks like
it is
> still not finding any input data to plot. You can run the following
> commands (after loading all of the required modules):
>
> export MET_INSTALL_DIR=/contrib/met/9.1
>
> Rscript /contrib/met/9.1/share/met/Rscripts/plot_tcmpr.R \
> -config
>
> /contrib/met/METplus/METplus-
3.1.1/parm/use_cases/model_applications/tc_and_extra_tc/TCMPRPlotterConfig_customize
> \
> -subtitle "Your subtitle goes here" -ylab "Your y-label goes here" \
> -dep AMAX_WIND-BMAX_WIND,TK_ERR -legend "Your legend text goes
here..." \
> -plot MEAN,MEDIAN,BOXPLOT \
> -lookin
>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> \
>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
> \
>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
> \
> -outdir
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tcmpr_plots
>
>
> Here is the output from that run:
>
> Reading plot configuration file:
>
> /contrib/met/METplus/METplus-
3.1.1/parm/use_cases/model_applications/tc_and_extra_tc/TCMPRPlotterConfig_customize
> No plotting configuration file specified.
> CALLING:  /contrib/met/9.1/bin/tc_stat -lookin
>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> -lookin
>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
> -lookin
>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
> -job filter -dump_row /tmp/plot_tcmpr_116966.tcst  -event_equal true
-v 3
> DEBUG 1: [Source 1] Lookin:
>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> DEBUG 1: [Source 2] Lookin:
>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
> DEBUG 1: [Source 3] Lookin:
>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
> DEBUG 2:
> DEBUG 2: Processing Job 1: -job filter -event_equal true -dump_row
> /tmp/plot_tcmpr_116966.tcst
> DEBUG 3: Applying track-based event equalization logic.
> DEBUG 3: Reading file 1 of 3:
>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> DEBUG 3: Reading file 2 of 3:
>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
> DEBUG 3: Reading file 3 of 3:
>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
> DEBUG 3: For track-based event equalization, identified 0 common
cases for
> 146 models: AC00 AEMI AEMN AP01 AP02 AP03 AP04 AP05 AP06 AP07 AP08
AP09
> AP0I AP10 AP11 AP12 AP13 AP14 AP15 AP16 AP17 AP18 AP19 AP1I AP20
AP21 AP22
> AP23 AP24 AP25 AP26 AP27 AP28 AP29 AP30 API CARQ CEM2 CEMI CEMN CLP5
CMC
> CMC2 CMCI COTC COTI CTCX DRCL DSHP EC00 EEMN EGRR EMN2 EMN3 EMN4
EMNI EMX
> EMX2 EMXI EN01 EN02 EN03 EN04 EN05 EN06 EN07 EN08 EN09 EN0I EN10
EN11 EN12
> EN13 EN14 EN15 EN16 EN17 EN18 EN19 EN1I EN20 EN21 EN22 EN23 EN24
EN25 ENI
> EP01 EP02 EP03 EP04 EP05 EP06 EP07 EP08 EP09 EP0I EP10 EP11 EP12
EP13 EP14
> EP15 EP16 EP17 EP18 EP19 EP1I EP20 EP21 EP22 EP23 EP24 EP25 EPI GFEX
GFSI
> GFSO HMNI HMON HWF2 HWFI HWRF ICON IVCN LGEM NGX NGX2 NGXI NVGM OCD5
RVCN
> SHF5 SHIP TABD TABM TABS TCLP TVCA TVCE TVCN TVCX UKX UKX2 UKXI XTRP
> DEBUG 1: Event equalization of tracks found no common cases.
> DEBUG 1: Creating output dump file: /tmp/plot_tcmpr_116966.tcst
> FILTER: -job filter -line_type TCMPR -event_equal true -dump_row
> /tmp/plot_tcmpr_116966.tcst
>
> DEBUG 2: Job 1 used 0 out of 0 lines read.
> DEBUG 3: Total lines read                 = 0
> DEBUG 3: Total lines kept                 = 0
> DEBUG 3: Rejected for track watch/warn    = 0
> DEBUG 3: Rejected for init threshold      = 0
> DEBUG 3: Rejected for init string         = 0
> DEBUG 3: Rejected for out init mask       = 0
> DEBUG 3: Rejected for water only          = 0
> DEBUG 3: Rejected for rapid inten         = 0
> DEBUG 3: Rejected for landfall            = 0
> DEBUG 3: Rejected for amodel              = 0
> DEBUG 3: Rejected for bmodel              = 0
> DEBUG 3: Rejected for desc                = 0
> DEBUG 3: Rejected for storm id            = 0
> DEBUG 3: Rejected for basin               = 0
> DEBUG 3: Rejected for cyclone             = 0
> DEBUG 3: Rejected for storm name          = 0
> DEBUG 3: Rejected for init time           = 0
> DEBUG 3: Rejected for init hour           = 0
> DEBUG 3: Rejected for lead time           = 0
> DEBUG 3: Rejected for required lead times = 0
> DEBUG 3: Rejected for valid time          = 0
> DEBUG 3: Rejected for valid hour          = 0
> DEBUG 3: Rejected for init mask           = 0
> DEBUG 3: Rejected for valid mask          = 0
> DEBUG 3: Rejected for line type           = 0
> DEBUG 3: Rejected for numeric threshold   = 0
> DEBUG 3: Rejected for string matching     = 0
> DEBUG 3: Rejected for match points        = 0
> DEBUG 3: Rejected for event equal         = 0
> DEBUG 3: Rejected for out init mask       = 0
> DEBUG 3: Rejected for out valid mask      = 0
> Reading track data: /tmp/plot_tcmpr_116966.tcst
> Error in read.table(tcst_tmp_file, header = TRUE) :
>   no lines available in input
> Execution halted
>
>
>
> It sounds like filter arguments may need to be provided to produce
TC-Stat
> output (which is called by the R script). You can set this in your
METplus
> config file with the TCMPR_PLOTTER_FILTER variable, but it will be
> difficult to debug since the log output is not shown. I would
recommend
> modifying the Rscript call from this email and add the -filter
argument
> with any filter information to pass to TCStat. I am not sure
settings would
> help you produce output, but I have reached out to other METplus
team
> members for suggestions.
>
> Thanks,
> George
>
>
>
> On Thu, May 13, 2021 at 12:37 PM Yan Jin - NOAA Affiliate via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846 >
> >
> > Hi George,
> >
> > Thanks so much. Now tc_pairs works, and the output file can be
generated.
> >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> >
> > Then I set "PROCESS_LIST = TCPairs, TCMPRPlotter", and I kept most
> > parameters that initialized with "TCMPR_PLOTTER" the same as the
default
> > configuration.
> >
> > my METplus config file:
> >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test.conf
> >
> > then run script
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/user_config/run.sh
> >
> > However, the Rscript cannot plot figures.
> > I checked the log file:
> >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210513182819
> > [image: image.png]
> >
> > It seems that the error "NameError: name 'produtil' is not
defined"?
> >
> > Could you give me some suggestions?
> >
> > Many thanks.
> > Best,
> > Yan
> >
> > On Wed, May 12, 2021 at 5:20 PM George McCabe via RT
<met_help at ucar.edu>
> > wrote:
> >
> > > Hi Yan,
> > >
> > > It looks like the values set for init_beg and init_end in the
TCpairs
> > > config file are both "20200101_000000" because the INIT_BEG and
> INIT_END
> > > time is 2020. There are no adeck tracks for this time, so all of
the
> data
> > > is being filtered out.
> > >
> > > This is a little confusing in METplus 3.1 because INIT_BEG and
INIT_END
> > > control the time looping of execution (if applicable for the use
case)
> > AND
> > > the values passed to the wrapped TCPairs config file. In METplus
4.0.0
> > > (which was released yesterday and should be available on jet
soon!),
> you
> > > can set INIT_BEG and INIT_END to control time looping (if
desired) and
> > use
> > > TC_PAIRS_INIT_BEG and TC_PAIRS_INIT_END to specify the values to
pass
> to
> > > the TCPairs config file.
> > >
> > > If you were using METplus 4.0.0, I would recommend setting
something
> > like:
> > >
> > >
> > > LOOP_BY = INIT
> > > INIT_TIME_FMT = %Y
> > > INIT_BEG = 2020
> > > INIT_END = 2020
> > > INIT_INCREMENT = 21600
> > >
> > > TC_PAIRS_INIT_BEG = 20201030_000000
> > > TC_PAIRS_INIT_END = 20201031_000000
> > >
> > >
> > > This would run once using 2020 to find filenames and process
data from
> > the
> > > files that are initialized between 20201030_000000 and
20201031_000000.
> > >
> > > If using METplus 3.1, you will have to use the INIT_BEG and
INIT_END
> > values
> > > to set the values in the wrapped TCPairs config. This is OK for
this
> use
> > > case because you have LOOP_ORDER set to processes and TCPairs
only runs
> > > once with that setting.
> > >
> > > LOOP_BY = INIT
> > > INIT_TIME_FMT = %Y%m%d
> > > INIT_BEG = 20201030
> > > INIT_END = 20201031
> > >
> > > This configuration should run and only use tracks initialized
between
> > > 2020-10-30 @ 0Z and 2020-10-31 @ 0Z. If you need to specify the
init
> time
> > > range with hour precision, you can do that as well. Here is an
example:
> > >
> > > LOOP_BY = INIT
> > > INIT_TIME_FMT = %Y%m%d%H
> > > INIT_BEG = 2020103012
> > > INIT_END = 2020103112
> > >
> > > This configuration will only use tracks initialized between
2020-10-30
> @
> > > 12Z and 2020-10-31 @ 12Z.
> > >
> > > You can also restrict the data used by valid time as well by
using
> > > TC_PAIRS_VALID_BEG and TC_PAIRS_VALID_END in METplus 4.0.0 or
VALID_BEG
> > and
> > > VALID_END for 3.1.
> > >
> > > Let me know if you are still unable to produce output with these
> > changes. I
> > > was able to run tc_pairs by hand on Jet and produce a file with
output
> > > tracks. You can view it here if you'd like:
> > >
> > > /lfs1/HFIP/dtc-hurr/George.Mccabe/al292020_pairs.dat.tcst
> > >
> > > These are the commands I used:
> > >
> > > export BASIN="[]";
> > > export CYCLONE="[]";
> > > export DLAND_FILE="/contrib/met/9.1/share/met/tc_data/
> > > dland_global_tenth_degree.nc";
> > > export INIT_BEG="20201030_000000";
> > > export INIT_END="20201031_000000";
> > > export INIT_EXCLUDE="[]";
> > > export INIT_INCLUDE="[]";
> > > export MODEL="[]";
> > > export STORM_ID="[\"AL292020\"]";
> > > export STORM_NAME="[]";
> > > export VALID_BEG="";
> > > export VALID_END="";
> > > /contrib/met/9.1/bin/tc_pairs -v 4 -bdeck
> > > /mnt/lfs4/HFIP/hwrf-data/hwrf-input/abdeck/btk/bal292020.dat
-adeck
> > > /mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/data/aal292020.dat
-config
> > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/use_cases/model_applications/tc_and_extra_tc/TCPairsETCConfig
> > > -out /lfs1/HFIP/dtc-hurr/George.Mccabe/al292020_pairs.dat
> > >
> > > I took the values from your METplus log file and modified them
to write
> > to
> > > my directory and use a different init time range.
> > >
> > > Thanks,
> > > George
> > >
> > >
> > >
> > >
> > > On Wed, May 12, 2021 at 2:04 PM Yan Jin - NOAA Affiliate via RT
<
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846
>
> > > >
> > > > Hi George,
> > > >
> > > > Thanks very much for your instructions. I set "PROCESS_LIST =
> TCPairs",
> > > and
> > > > didn't specify models: "MODEL =". Then run the script.
However, the
> > > output
> > > > file is still empty. It seems that tc_pairs matched 0 adeck
track to
> 1
> > > > bdeck track. I also tried to set "MODEL = HWRF, HMON", but the
output
> > > file
> > > > is empty too.
> > > >
> > > > Here is the log file:
> > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210512193152
> > > >
> > > > And here is the output file:
> > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> > > >
> > > > Many thanks.
> > > > Best,
> > > > Yan
> > > >
> > > > On Wed, May 12, 2021 at 11:36 AM George McCabe via RT <
> > met_help at ucar.edu
> > > >
> > > > wrote:
> > > >
> > > > > Hi Yan,
> > > > >
> > > > > I was able to log into Jet this morning and look at your
error.
> > > > >
> > > > > It looks like the error in the call to Rscript is due to
there
> being
> > no
> > > > > data in the input files generated by TCPairs. There are
> > inconsistencies
> > > > in
> > > > > the TCMPRPlotter wrapper that prevent the log output from
the run
> to
> > be
> > > > > seen, which would have made it more clear why the issue was
> > occurring.
> > > > >
> > > > > I would recommend changing the PROCESS_LIST value in your
METplus
> > > > > configuration file to only run TCPairs and iterating on that
until
> > you
> > > > get
> > > > > output from that tool. Once you can confirm that you have
output,
> you
> > > can
> > > > > add TCMPRPlotter back to the PROCESS_LIST and test on that
portion.
> > > > >
> > > > > I will create a GitHub issue for updating the TCMPRPlotter
wrapper
> to
> > > > > produce useful logging output that will make things more
clear
> going
> > > > > forward.
> > > > >
> > > > > Let me know if you run into any other issues or need help
> configuring
> > > > > TCPairs to run properly. I'm also happy to attend another
working
> > > session
> > > > > if that is helpful.
> > > > >
> > > > > Thanks,
> > > > > George
> > > > >
> > > > >
> > > > > On Tue May 11 10:05:06 2021, yan.jin at noaa.gov wrote:
> > > > > > Hi John,
> > > > > >
> > > > > >
> > > > > > The directories and data I mentioned in my last email are
all
> > located
> > > > > > in
> > > > > > Jet.
> > > > > >
> > > > > >
> > > > > > Here is the email I got regarding the Jet maintenance:
> > > > > >
> > > > > > ***
> > > > > >
> > > > > > *05/11/21 - Boulder Bastion Maintenance Downtime*
> > > > > > Boulder Bastion maintenance is scheduled for Tuesday
05/11/21
> from
> > > > > > 0600 ~
> > > > > > 1000 MT. Boulder Bastion login sessions will be dropped
and all
> new
> > > > > > RDHPCS
> > > > > > logins will need to go through the Princeton Bastion.
Logging In
> > > > > > <
> > > https://rdhpcs-common-
docs.rdhpcs.noaa.gov/wiki/index.php/Logging_in>
> > > > > >
> > > > > >
> > > > > >
> > > > > > *The next Jet Maintenance Downtime is TBD*
> > > > > >
> > > > > > ***
> > > > > >
> > > > > >
> > > > > > I tried to login Princeton but the connection is closed
too.
> > > > > >
> > > > > >
> > > > > > Really appreciate your time and help on this.
> > > > > >
> > > > > >
> > > > > > Best,
> > > > > >
> > > > > > Yan
> > > > > >
> > > > > > On Tue, May 11, 2021 at 11:57 AM John Halley Gotway via RT
<
> > > > > > met_help at ucar.edu> wrote:
> > > > > >
> > > > > > > Yan,
> > > > > > >
> > > > > > > George and I tried to take a look at this, but couldn't
find
> the
> > > > > > > data.
> > > > > > >
> > > > > > >
> > > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210510214154
> > > > > > >
> > > > > > > I checked on Hera and WCOSS but couldn't find the data
there. I
> > > > > > > should be
> > > > > > > able to log on to jet, but when I try, it boots me off:
> > > > > > >
> > > > > > > Authentication failed.
> > > > > > > Connection to jet-rsa.boulder.rdhpcs.noaa.gov closed.
> > > > > > >
> > > > > > > Can you confirm that you're running on jet? And any idea
of jet
> > is
> > > > > > > down
> > > > > > > today or something?
> > > > > > >
> > > > > > > Thanks,
> > > > > > > John
> > > > > > >
> > > > > > >
> > > > > > > On Tue, May 11, 2021 at 7:46 AM Yan Jin - NOAA Affiliate
via
> RT <
> > > > > > > met_help at ucar.edu> wrote:
> > > > > > >
> > > > > > > >
> > > > > > > > <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846
> > >
> > > > > > > >
> > > > > > > > Hi John,
> > > > > > > >
> > > > > > > > Thanks so much!
> > > > > > > >
> > > > > > > > Best,
> > > > > > > > Yan
> > > > > > > >
> > > > > > > >
> > > > > > > > On Tue, May 11, 2021 at 9:44 AM John Halley Gotway via
RT <
> > > > > > > > met_help at ucar.edu>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi Yan,
> > > > > > > > >
> > > > > > > > > I'm going to assign this ticket to George and ask
him to
> > take a
> > > > > > > > > look
> > > > > > > and
> > > > > > > > > debug this runtime error.
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > John HG
> > > > > > > > >
> > > > > > > > > On Mon, May 10, 2021 at 3:49 PM Yan Jin - NOAA
Affiliate
> via
> > > RT <
> > > > > > > > > met_help at ucar.edu> wrote:
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Mon May 10 15:49:51 2021: Request 99846 was acted
upon.
> > > > > > > > > > Transaction: Ticket created by yan.jin at noaa.gov
> > > > > > > > > >        Queue: met_help
> > > > > > > > > >      Subject: ask a question about METplus config
of TC
> > > > > > > > > >        Owner: Nobody
> > > > > > > > > >   Requestors: yan.jin at noaa.gov
> > > > > > > > > >       Status: new
> > > > > > > > > >  Ticket <URL:
> > > > > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Hi,
> > > > > > > > > >
> > > > > > > > > > This is Yan, again. I ran the METplus config file
after
> our
> > > > > > > discussion,
> > > > > > > > > > however, it seems that the tc_pairs tcst file
wasn't
> > > generated.
> > > > > > > > > >
> > > > > > > > > > Here is the script in my directory on Jet:
> > > > > > > > > > I set TC_PAIRS_STORM_ID = AL292020, so the code
can find
> > the
> > > > > > > > > > input
> > > > > > > > adeck
> > > > > > > > > > and bdeck data correctly
> > > > > > > > > > *----
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test.conf
> > > > > > > > > >
> > > > > > > > > > The output directory is
> > > > > > > > > >
> > > > >
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs
> > > > > > > > > > and the tc_pairs data: al292020_pairs.dat.tcst
> > > > > > > > > >
> > > > > > > > > > log file
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210510214154
> > > > > > > > > > *-----
> > > > > > > > > >
> > > > > > > > > > The error starts:[image: image.png]
> > > > > > > > > >
> > > > > > > > > > Then I checked al292020_pairs.dat.tcst, and it is
empty.
> > > > > > > > > >
> > > > > > > > > > I wonder if there is anything I missed?
> > > > > > > > > >
> > > > > > > > > > Many thanks.
> > > > > > > > > > Best,
> > > > > > > > > > Yan
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > >
> > > --
> > > George McCabe - Software Engineer III
> > > National Center for Atmospheric Research
> > > Research Applications Laboratory
> > > 303-497-2768
> > > ---
> > > My working day may not be your working day. Please do not feel
obliged
> to
> > > reply to this email outside of your normal working hours.
> > >
> > >
> >
> >
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>

------------------------------------------------
Subject: ask a question about METplus config of TC
From: George McCabe
Time: Mon May 17 08:36:40 2021

Hi Yan,

4.1:
The consensus variable is not yet controlled by a METplus
configuration
variable explicitly, but you can set the value by using the
TC_PAIRS_MET_CONFIG_OVERRIDES value (in v4.0.0+). To set the consensus
value to what you have set in your config file, you would set
something
like this:

TC_PAIRS_MET_CONFIG_OVERRIDES = consensus = [ { name = "CONS"; members
=
["HWRF","GFSO","HMON"]; required = [false,false,false]; min_req = 2; }
];

I will make a note in the GitHub issue for improvements to the TCPairs
wrapper (https://github.com/dtcenter/METplus/issues/749) to add
support for
consensus explicitly via something like TC_PAIRS_CONSENSUS so that it
is
easier to configure via METplus wrappers.

4.2:
The System Configuration section of the METplus User's Guide covers
these
variables and how they affect execution. I suggest that you read
through
this section to get a full picture of how these variables work.
Regarding
your specific question, these 2 sections should help:

LOOP_BY:
https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#loop-
by
LOOP_ORDER:
https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#loop-
order

Thanks,
George

On Fri, May 14, 2021 at 8:19 AM Yan Jin - NOAA Affiliate via RT <
met_help at ucar.edu> wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846 >
>
> Hi George,
>
> Thanks so much for your help. Finally the code can run without
errors. Here
> is the steps and results:
>
>
> https://docs.google.com/document/d/1xRF3aiPATcX5G37ngJj-
65pl44LN7mGHo34Sf8i3zpo/edit
>
> I also put questions in the document, I also pasted here:
>
> Questions:
>
> 4.1: How can I do the consensus forecasts in METplus config file as
I did
> in tc_pairs?
>
>
(/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_292020/TCPairsConfig_run1)
>
> //
>
> // Specify how consensus forecasts should be defined.
>
> //
>
> consensus = [
>
>  {
>
>   name = "CONS";
>
>   members = ["HWRF","GFSO","HMON"];
>
>   required = [false,false,false];
>
>   min_req = 2;
>
>  }
>
> ];
>
> 4.2: I am a little confused about
>
>       ***
>
>        LOOP_ORDER = processes
>
>        LOOP_BY = INIT
>
>       ***
>
> Could you explain these two command lines and give me some examples?
>
> Thanks very much for your time and help on my questions.
>
> Best,
> Yan
>
> On Thu, May 13, 2021 at 3:04 PM George McCabe via RT
<met_help at ucar.edu>
> wrote:
>
> > Hi Yan,
> >
> > The produtil is due to a bug in the wrapper in the logic that
reports
> when
> > an error occurs. I ran the command on Jet by hand and it looks
like it is
> > still not finding any input data to plot. You can run the
following
> > commands (after loading all of the required modules):
> >
> > export MET_INSTALL_DIR=/contrib/met/9.1
> >
> > Rscript /contrib/met/9.1/share/met/Rscripts/plot_tcmpr.R \
> > -config
> >
> >
> /contrib/met/METplus/METplus-
3.1.1/parm/use_cases/model_applications/tc_and_extra_tc/TCMPRPlotterConfig_customize
> > \
> > -subtitle "Your subtitle goes here" -ylab "Your y-label goes here"
\
> > -dep AMAX_WIND-BMAX_WIND,TK_ERR -legend "Your legend text goes
here..." \
> > -plot MEAN,MEDIAN,BOXPLOT \
> > -lookin
> >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> > \
> >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
> > \
> >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
> > \
> > -outdir
> >
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tcmpr_plots
> >
> >
> > Here is the output from that run:
> >
> > Reading plot configuration file:
> >
> >
> /contrib/met/METplus/METplus-
3.1.1/parm/use_cases/model_applications/tc_and_extra_tc/TCMPRPlotterConfig_customize
> > No plotting configuration file specified.
> > CALLING:  /contrib/met/9.1/bin/tc_stat -lookin
> >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> > -lookin
> >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
> > -lookin
> >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
> > -job filter -dump_row /tmp/plot_tcmpr_116966.tcst  -event_equal
true -v 3
> > DEBUG 1: [Source 1] Lookin:
> >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> > DEBUG 1: [Source 2] Lookin:
> >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
> > DEBUG 1: [Source 3] Lookin:
> >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
> > DEBUG 2:
> > DEBUG 2: Processing Job 1: -job filter -event_equal true -dump_row
> > /tmp/plot_tcmpr_116966.tcst
> > DEBUG 3: Applying track-based event equalization logic.
> > DEBUG 3: Reading file 1 of 3:
> >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> > DEBUG 3: Reading file 2 of 3:
> >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
> > DEBUG 3: Reading file 3 of 3:
> >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
> > DEBUG 3: For track-based event equalization, identified 0 common
cases
> for
> > 146 models: AC00 AEMI AEMN AP01 AP02 AP03 AP04 AP05 AP06 AP07 AP08
AP09
> > AP0I AP10 AP11 AP12 AP13 AP14 AP15 AP16 AP17 AP18 AP19 AP1I AP20
AP21
> AP22
> > AP23 AP24 AP25 AP26 AP27 AP28 AP29 AP30 API CARQ CEM2 CEMI CEMN
CLP5 CMC
> > CMC2 CMCI COTC COTI CTCX DRCL DSHP EC00 EEMN EGRR EMN2 EMN3 EMN4
EMNI EMX
> > EMX2 EMXI EN01 EN02 EN03 EN04 EN05 EN06 EN07 EN08 EN09 EN0I EN10
EN11
> EN12
> > EN13 EN14 EN15 EN16 EN17 EN18 EN19 EN1I EN20 EN21 EN22 EN23 EN24
EN25 ENI
> > EP01 EP02 EP03 EP04 EP05 EP06 EP07 EP08 EP09 EP0I EP10 EP11 EP12
EP13
> EP14
> > EP15 EP16 EP17 EP18 EP19 EP1I EP20 EP21 EP22 EP23 EP24 EP25 EPI
GFEX GFSI
> > GFSO HMNI HMON HWF2 HWFI HWRF ICON IVCN LGEM NGX NGX2 NGXI NVGM
OCD5 RVCN
> > SHF5 SHIP TABD TABM TABS TCLP TVCA TVCE TVCN TVCX UKX UKX2 UKXI
XTRP
> > DEBUG 1: Event equalization of tracks found no common cases.
> > DEBUG 1: Creating output dump file: /tmp/plot_tcmpr_116966.tcst
> > FILTER: -job filter -line_type TCMPR -event_equal true -dump_row
> > /tmp/plot_tcmpr_116966.tcst
> >
> > DEBUG 2: Job 1 used 0 out of 0 lines read.
> > DEBUG 3: Total lines read                 = 0
> > DEBUG 3: Total lines kept                 = 0
> > DEBUG 3: Rejected for track watch/warn    = 0
> > DEBUG 3: Rejected for init threshold      = 0
> > DEBUG 3: Rejected for init string         = 0
> > DEBUG 3: Rejected for out init mask       = 0
> > DEBUG 3: Rejected for water only          = 0
> > DEBUG 3: Rejected for rapid inten         = 0
> > DEBUG 3: Rejected for landfall            = 0
> > DEBUG 3: Rejected for amodel              = 0
> > DEBUG 3: Rejected for bmodel              = 0
> > DEBUG 3: Rejected for desc                = 0
> > DEBUG 3: Rejected for storm id            = 0
> > DEBUG 3: Rejected for basin               = 0
> > DEBUG 3: Rejected for cyclone             = 0
> > DEBUG 3: Rejected for storm name          = 0
> > DEBUG 3: Rejected for init time           = 0
> > DEBUG 3: Rejected for init hour           = 0
> > DEBUG 3: Rejected for lead time           = 0
> > DEBUG 3: Rejected for required lead times = 0
> > DEBUG 3: Rejected for valid time          = 0
> > DEBUG 3: Rejected for valid hour          = 0
> > DEBUG 3: Rejected for init mask           = 0
> > DEBUG 3: Rejected for valid mask          = 0
> > DEBUG 3: Rejected for line type           = 0
> > DEBUG 3: Rejected for numeric threshold   = 0
> > DEBUG 3: Rejected for string matching     = 0
> > DEBUG 3: Rejected for match points        = 0
> > DEBUG 3: Rejected for event equal         = 0
> > DEBUG 3: Rejected for out init mask       = 0
> > DEBUG 3: Rejected for out valid mask      = 0
> > Reading track data: /tmp/plot_tcmpr_116966.tcst
> > Error in read.table(tcst_tmp_file, header = TRUE) :
> >   no lines available in input
> > Execution halted
> >
> >
> >
> > It sounds like filter arguments may need to be provided to produce
> TC-Stat
> > output (which is called by the R script). You can set this in your
> METplus
> > config file with the TCMPR_PLOTTER_FILTER variable, but it will be
> > difficult to debug since the log output is not shown. I would
recommend
> > modifying the Rscript call from this email and add the -filter
argument
> > with any filter information to pass to TCStat. I am not sure
settings
> would
> > help you produce output, but I have reached out to other METplus
team
> > members for suggestions.
> >
> > Thanks,
> > George
> >
> >
> >
> > On Thu, May 13, 2021 at 12:37 PM Yan Jin - NOAA Affiliate via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846 >
> > >
> > > Hi George,
> > >
> > > Thanks so much. Now tc_pairs works, and the output file can be
> generated.
> > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> > >
> > > Then I set "PROCESS_LIST = TCPairs, TCMPRPlotter", and I kept
most
> > > parameters that initialized with "TCMPR_PLOTTER" the same as the
> default
> > > configuration.
> > >
> > > my METplus config file:
> > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test.conf
> > >
> > > then run script
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/user_config/run.sh
> > >
> > > However, the Rscript cannot plot figures.
> > > I checked the log file:
> > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210513182819
> > > [image: image.png]
> > >
> > > It seems that the error "NameError: name 'produtil' is not
defined"?
> > >
> > > Could you give me some suggestions?
> > >
> > > Many thanks.
> > > Best,
> > > Yan
> > >
> > > On Wed, May 12, 2021 at 5:20 PM George McCabe via RT <
> met_help at ucar.edu>
> > > wrote:
> > >
> > > > Hi Yan,
> > > >
> > > > It looks like the values set for init_beg and init_end in the
TCpairs
> > > > config file are both "20200101_000000" because the INIT_BEG
and
> > INIT_END
> > > > time is 2020. There are no adeck tracks for this time, so all
of the
> > data
> > > > is being filtered out.
> > > >
> > > > This is a little confusing in METplus 3.1 because INIT_BEG and
> INIT_END
> > > > control the time looping of execution (if applicable for the
use
> case)
> > > AND
> > > > the values passed to the wrapped TCPairs config file. In
METplus
> 4.0.0
> > > > (which was released yesterday and should be available on jet
soon!),
> > you
> > > > can set INIT_BEG and INIT_END to control time looping (if
desired)
> and
> > > use
> > > > TC_PAIRS_INIT_BEG and TC_PAIRS_INIT_END to specify the values
to pass
> > to
> > > > the TCPairs config file.
> > > >
> > > > If you were using METplus 4.0.0, I would recommend setting
something
> > > like:
> > > >
> > > >
> > > > LOOP_BY = INIT
> > > > INIT_TIME_FMT = %Y
> > > > INIT_BEG = 2020
> > > > INIT_END = 2020
> > > > INIT_INCREMENT = 21600
> > > >
> > > > TC_PAIRS_INIT_BEG = 20201030_000000
> > > > TC_PAIRS_INIT_END = 20201031_000000
> > > >
> > > >
> > > > This would run once using 2020 to find filenames and process
data
> from
> > > the
> > > > files that are initialized between 20201030_000000 and
> 20201031_000000.
> > > >
> > > > If using METplus 3.1, you will have to use the INIT_BEG and
INIT_END
> > > values
> > > > to set the values in the wrapped TCPairs config. This is OK
for this
> > use
> > > > case because you have LOOP_ORDER set to processes and TCPairs
only
> runs
> > > > once with that setting.
> > > >
> > > > LOOP_BY = INIT
> > > > INIT_TIME_FMT = %Y%m%d
> > > > INIT_BEG = 20201030
> > > > INIT_END = 20201031
> > > >
> > > > This configuration should run and only use tracks initialized
between
> > > > 2020-10-30 @ 0Z and 2020-10-31 @ 0Z. If you need to specify
the init
> > time
> > > > range with hour precision, you can do that as well. Here is an
> example:
> > > >
> > > > LOOP_BY = INIT
> > > > INIT_TIME_FMT = %Y%m%d%H
> > > > INIT_BEG = 2020103012
> > > > INIT_END = 2020103112
> > > >
> > > > This configuration will only use tracks initialized between
> 2020-10-30
> > @
> > > > 12Z and 2020-10-31 @ 12Z.
> > > >
> > > > You can also restrict the data used by valid time as well by
using
> > > > TC_PAIRS_VALID_BEG and TC_PAIRS_VALID_END in METplus 4.0.0 or
> VALID_BEG
> > > and
> > > > VALID_END for 3.1.
> > > >
> > > > Let me know if you are still unable to produce output with
these
> > > changes. I
> > > > was able to run tc_pairs by hand on Jet and produce a file
with
> output
> > > > tracks. You can view it here if you'd like:
> > > >
> > > > /lfs1/HFIP/dtc-hurr/George.Mccabe/al292020_pairs.dat.tcst
> > > >
> > > > These are the commands I used:
> > > >
> > > > export BASIN="[]";
> > > > export CYCLONE="[]";
> > > > export DLAND_FILE="/contrib/met/9.1/share/met/tc_data/
> > > > dland_global_tenth_degree.nc";
> > > > export INIT_BEG="20201030_000000";
> > > > export INIT_END="20201031_000000";
> > > > export INIT_EXCLUDE="[]";
> > > > export INIT_INCLUDE="[]";
> > > > export MODEL="[]";
> > > > export STORM_ID="[\"AL292020\"]";
> > > > export STORM_NAME="[]";
> > > > export VALID_BEG="";
> > > > export VALID_END="";
> > > > /contrib/met/9.1/bin/tc_pairs -v 4 -bdeck
> > > > /mnt/lfs4/HFIP/hwrf-data/hwrf-input/abdeck/btk/bal292020.dat
-adeck
> > > > /mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/data/aal292020.dat
-config
> > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/use_cases/model_applications/tc_and_extra_tc/TCPairsETCConfig
> > > > -out /lfs1/HFIP/dtc-hurr/George.Mccabe/al292020_pairs.dat
> > > >
> > > > I took the values from your METplus log file and modified them
to
> write
> > > to
> > > > my directory and use a different init time range.
> > > >
> > > > Thanks,
> > > > George
> > > >
> > > >
> > > >
> > > >
> > > > On Wed, May 12, 2021 at 2:04 PM Yan Jin - NOAA Affiliate via
RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846 >
> > > > >
> > > > > Hi George,
> > > > >
> > > > > Thanks very much for your instructions. I set "PROCESS_LIST
=
> > TCPairs",
> > > > and
> > > > > didn't specify models: "MODEL =". Then run the script.
However, the
> > > > output
> > > > > file is still empty. It seems that tc_pairs matched 0 adeck
track
> to
> > 1
> > > > > bdeck track. I also tried to set "MODEL = HWRF, HMON", but
the
> output
> > > > file
> > > > > is empty too.
> > > > >
> > > > > Here is the log file:
> > > > >
> > > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210512193152
> > > > >
> > > > > And here is the output file:
> > > > >
> > > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> > > > >
> > > > > Many thanks.
> > > > > Best,
> > > > > Yan
> > > > >
> > > > > On Wed, May 12, 2021 at 11:36 AM George McCabe via RT <
> > > met_help at ucar.edu
> > > > >
> > > > > wrote:
> > > > >
> > > > > > Hi Yan,
> > > > > >
> > > > > > I was able to log into Jet this morning and look at your
error.
> > > > > >
> > > > > > It looks like the error in the call to Rscript is due to
there
> > being
> > > no
> > > > > > data in the input files generated by TCPairs. There are
> > > inconsistencies
> > > > > in
> > > > > > the TCMPRPlotter wrapper that prevent the log output from
the run
> > to
> > > be
> > > > > > seen, which would have made it more clear why the issue
was
> > > occurring.
> > > > > >
> > > > > > I would recommend changing the PROCESS_LIST value in your
METplus
> > > > > > configuration file to only run TCPairs and iterating on
that
> until
> > > you
> > > > > get
> > > > > > output from that tool. Once you can confirm that you have
output,
> > you
> > > > can
> > > > > > add TCMPRPlotter back to the PROCESS_LIST and test on that
> portion.
> > > > > >
> > > > > > I will create a GitHub issue for updating the TCMPRPlotter
> wrapper
> > to
> > > > > > produce useful logging output that will make things more
clear
> > going
> > > > > > forward.
> > > > > >
> > > > > > Let me know if you run into any other issues or need help
> > configuring
> > > > > > TCPairs to run properly. I'm also happy to attend another
working
> > > > session
> > > > > > if that is helpful.
> > > > > >
> > > > > > Thanks,
> > > > > > George
> > > > > >
> > > > > >
> > > > > > On Tue May 11 10:05:06 2021, yan.jin at noaa.gov wrote:
> > > > > > > Hi John,
> > > > > > >
> > > > > > >
> > > > > > > The directories and data I mentioned in my last email
are all
> > > located
> > > > > > > in
> > > > > > > Jet.
> > > > > > >
> > > > > > >
> > > > > > > Here is the email I got regarding the Jet maintenance:
> > > > > > >
> > > > > > > ***
> > > > > > >
> > > > > > > *05/11/21 - Boulder Bastion Maintenance Downtime*
> > > > > > > Boulder Bastion maintenance is scheduled for Tuesday
05/11/21
> > from
> > > > > > > 0600 ~
> > > > > > > 1000 MT. Boulder Bastion login sessions will be dropped
and all
> > new
> > > > > > > RDHPCS
> > > > > > > logins will need to go through the Princeton Bastion.
Logging
> In
> > > > > > > <
> > > > https://rdhpcs-common-
docs.rdhpcs.noaa.gov/wiki/index.php/Logging_in
> >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > *The next Jet Maintenance Downtime is TBD*
> > > > > > >
> > > > > > > ***
> > > > > > >
> > > > > > >
> > > > > > > I tried to login Princeton but the connection is closed
too.
> > > > > > >
> > > > > > >
> > > > > > > Really appreciate your time and help on this.
> > > > > > >
> > > > > > >
> > > > > > > Best,
> > > > > > >
> > > > > > > Yan
> > > > > > >
> > > > > > > On Tue, May 11, 2021 at 11:57 AM John Halley Gotway via
RT <
> > > > > > > met_help at ucar.edu> wrote:
> > > > > > >
> > > > > > > > Yan,
> > > > > > > >
> > > > > > > > George and I tried to take a look at this, but
couldn't find
> > the
> > > > > > > > data.
> > > > > > > >
> > > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210510214154
> > > > > > > >
> > > > > > > > I checked on Hera and WCOSS but couldn't find the data
> there. I
> > > > > > > > should be
> > > > > > > > able to log on to jet, but when I try, it boots me
off:
> > > > > > > >
> > > > > > > > Authentication failed.
> > > > > > > > Connection to jet-rsa.boulder.rdhpcs.noaa.gov closed.
> > > > > > > >
> > > > > > > > Can you confirm that you're running on jet? And any
idea of
> jet
> > > is
> > > > > > > > down
> > > > > > > > today or something?
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > John
> > > > > > > >
> > > > > > > >
> > > > > > > > On Tue, May 11, 2021 at 7:46 AM Yan Jin - NOAA
Affiliate via
> > RT <
> > > > > > > > met_help at ucar.edu> wrote:
> > > > > > > >
> > > > > > > > >
> > > > > > > > > <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846
> > > >
> > > > > > > > >
> > > > > > > > > Hi John,
> > > > > > > > >
> > > > > > > > > Thanks so much!
> > > > > > > > >
> > > > > > > > > Best,
> > > > > > > > > Yan
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Tue, May 11, 2021 at 9:44 AM John Halley Gotway
via RT <
> > > > > > > > > met_help at ucar.edu>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Hi Yan,
> > > > > > > > > >
> > > > > > > > > > I'm going to assign this ticket to George and ask
him to
> > > take a
> > > > > > > > > > look
> > > > > > > > and
> > > > > > > > > > debug this runtime error.
> > > > > > > > > >
> > > > > > > > > > Thanks,
> > > > > > > > > > John HG
> > > > > > > > > >
> > > > > > > > > > On Mon, May 10, 2021 at 3:49 PM Yan Jin - NOAA
Affiliate
> > via
> > > > RT <
> > > > > > > > > > met_help at ucar.edu> wrote:
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > Mon May 10 15:49:51 2021: Request 99846 was
acted upon.
> > > > > > > > > > > Transaction: Ticket created by yan.jin at noaa.gov
> > > > > > > > > > >        Queue: met_help
> > > > > > > > > > >      Subject: ask a question about METplus
config of TC
> > > > > > > > > > >        Owner: Nobody
> > > > > > > > > > >   Requestors: yan.jin at noaa.gov
> > > > > > > > > > >       Status: new
> > > > > > > > > > >  Ticket <URL:
> > > > > > > >
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > Hi,
> > > > > > > > > > >
> > > > > > > > > > > This is Yan, again. I ran the METplus config
file after
> > our
> > > > > > > > discussion,
> > > > > > > > > > > however, it seems that the tc_pairs tcst file
wasn't
> > > > generated.
> > > > > > > > > > >
> > > > > > > > > > > Here is the script in my directory on Jet:
> > > > > > > > > > > I set TC_PAIRS_STORM_ID = AL292020, so the code
can
> find
> > > the
> > > > > > > > > > > input
> > > > > > > > > adeck
> > > > > > > > > > > and bdeck data correctly
> > > > > > > > > > > *----
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test.conf
> > > > > > > > > > >
> > > > > > > > > > > The output directory is
> > > > > > > > > > >
> > > > > >
> /mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs
> > > > > > > > > > > and the tc_pairs data: al292020_pairs.dat.tcst
> > > > > > > > > > >
> > > > > > > > > > > log file
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210510214154
> > > > > > > > > > > *-----
> > > > > > > > > > >
> > > > > > > > > > > The error starts:[image: image.png]
> > > > > > > > > > >
> > > > > > > > > > > Then I checked al292020_pairs.dat.tcst, and it
is
> empty.
> > > > > > > > > > >
> > > > > > > > > > > I wonder if there is anything I missed?
> > > > > > > > > > >
> > > > > > > > > > > Many thanks.
> > > > > > > > > > > Best,
> > > > > > > > > > > Yan
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > > --
> > > > George McCabe - Software Engineer III
> > > > National Center for Atmospheric Research
> > > > Research Applications Laboratory
> > > > 303-497-2768
> > > > ---
> > > > My working day may not be your working day. Please do not feel
> obliged
> > to
> > > > reply to this email outside of your normal working hours.
> > > >
> > > >
> > >
> > >
> >
> > --
> > George McCabe - Software Engineer III
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > 303-497-2768
> > ---
> > My working day may not be your working day. Please do not feel
obliged to
> > reply to this email outside of your normal working hours.
> >
> >
>
>

--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.

------------------------------------------------
Subject: ask a question about METplus config of TC
From: Yan Jin - NOAA Affiliate
Time: Mon May 17 08:44:08 2021

Hi George,

Thank you. I will read the Users' Guide to get a better understanding.
Since I got answers from you I will delete my questions on the telecon
topics.

Really appreciate your time and help on my tasks.

Best,
Yan

On Mon, May 17, 2021 at 10:37 AM George McCabe via RT
<met_help at ucar.edu>
wrote:

> Hi Yan,
>
> 4.1:
> The consensus variable is not yet controlled by a METplus
configuration
> variable explicitly, but you can set the value by using the
> TC_PAIRS_MET_CONFIG_OVERRIDES value (in v4.0.0+). To set the
consensus
> value to what you have set in your config file, you would set
something
> like this:
>
> TC_PAIRS_MET_CONFIG_OVERRIDES = consensus = [ { name = "CONS";
members =
> ["HWRF","GFSO","HMON"]; required = [false,false,false]; min_req = 2;
} ];
>
> I will make a note in the GitHub issue for improvements to the
TCPairs
> wrapper (https://github.com/dtcenter/METplus/issues/749) to add
support
> for
> consensus explicitly via something like TC_PAIRS_CONSENSUS so that
it is
> easier to configure via METplus wrappers.
>
> 4.2:
> The System Configuration section of the METplus User's Guide covers
these
> variables and how they affect execution. I suggest that you read
through
> this section to get a full picture of how these variables work.
Regarding
> your specific question, these 2 sections should help:
>
> LOOP_BY:
>
>
https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#loop-
by
> LOOP_ORDER:
>
>
https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#loop-
order
>
> Thanks,
> George
>
> On Fri, May 14, 2021 at 8:19 AM Yan Jin - NOAA Affiliate via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846 >
> >
> > Hi George,
> >
> > Thanks so much for your help. Finally the code can run without
errors.
> Here
> > is the steps and results:
> >
> >
> >
> https://docs.google.com/document/d/1xRF3aiPATcX5G37ngJj-
65pl44LN7mGHo34Sf8i3zpo/edit
> >
> > I also put questions in the document, I also pasted here:
> >
> > Questions:
> >
> > 4.1: How can I do the consensus forecasts in METplus config file
as I did
> > in tc_pairs?
> >
> >
>
(/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_292020/TCPairsConfig_run1)
> >
> > //
> >
> > // Specify how consensus forecasts should be defined.
> >
> > //
> >
> > consensus = [
> >
> >  {
> >
> >   name = "CONS";
> >
> >   members = ["HWRF","GFSO","HMON"];
> >
> >   required = [false,false,false];
> >
> >   min_req = 2;
> >
> >  }
> >
> > ];
> >
> > 4.2: I am a little confused about
> >
> >       ***
> >
> >        LOOP_ORDER = processes
> >
> >        LOOP_BY = INIT
> >
> >       ***
> >
> > Could you explain these two command lines and give me some
examples?
> >
> > Thanks very much for your time and help on my questions.
> >
> > Best,
> > Yan
> >
> > On Thu, May 13, 2021 at 3:04 PM George McCabe via RT
<met_help at ucar.edu>
> > wrote:
> >
> > > Hi Yan,
> > >
> > > The produtil is due to a bug in the wrapper in the logic that
reports
> > when
> > > an error occurs. I ran the command on Jet by hand and it looks
like it
> is
> > > still not finding any input data to plot. You can run the
following
> > > commands (after loading all of the required modules):
> > >
> > > export MET_INSTALL_DIR=/contrib/met/9.1
> > >
> > > Rscript /contrib/met/9.1/share/met/Rscripts/plot_tcmpr.R \
> > > -config
> > >
> > >
> >
> /contrib/met/METplus/METplus-
3.1.1/parm/use_cases/model_applications/tc_and_extra_tc/TCMPRPlotterConfig_customize
> > > \
> > > -subtitle "Your subtitle goes here" -ylab "Your y-label goes
here" \
> > > -dep AMAX_WIND-BMAX_WIND,TK_ERR -legend "Your legend text goes
> here..." \
> > > -plot MEAN,MEDIAN,BOXPLOT \
> > > -lookin
> > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> > > \
> > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
> > > \
> > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
> > > \
> > > -outdir
> > >
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tcmpr_plots
> > >
> > >
> > > Here is the output from that run:
> > >
> > > Reading plot configuration file:
> > >
> > >
> >
> /contrib/met/METplus/METplus-
3.1.1/parm/use_cases/model_applications/tc_and_extra_tc/TCMPRPlotterConfig_customize
> > > No plotting configuration file specified.
> > > CALLING:  /contrib/met/9.1/bin/tc_stat -lookin
> > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> > > -lookin
> > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
> > > -lookin
> > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
> > > -job filter -dump_row /tmp/plot_tcmpr_116966.tcst  -event_equal
true
> -v 3
> > > DEBUG 1: [Source 1] Lookin:
> > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> > > DEBUG 1: [Source 2] Lookin:
> > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
> > > DEBUG 1: [Source 3] Lookin:
> > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
> > > DEBUG 2:
> > > DEBUG 2: Processing Job 1: -job filter -event_equal true
-dump_row
> > > /tmp/plot_tcmpr_116966.tcst
> > > DEBUG 3: Applying track-based event equalization logic.
> > > DEBUG 3: Reading file 1 of 3:
> > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> > > DEBUG 3: Reading file 2 of 3:
> > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
> > > DEBUG 3: Reading file 3 of 3:
> > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
> > > DEBUG 3: For track-based event equalization, identified 0 common
cases
> > for
> > > 146 models: AC00 AEMI AEMN AP01 AP02 AP03 AP04 AP05 AP06 AP07
AP08 AP09
> > > AP0I AP10 AP11 AP12 AP13 AP14 AP15 AP16 AP17 AP18 AP19 AP1I AP20
AP21
> > AP22
> > > AP23 AP24 AP25 AP26 AP27 AP28 AP29 AP30 API CARQ CEM2 CEMI CEMN
CLP5
> CMC
> > > CMC2 CMCI COTC COTI CTCX DRCL DSHP EC00 EEMN EGRR EMN2 EMN3 EMN4
EMNI
> EMX
> > > EMX2 EMXI EN01 EN02 EN03 EN04 EN05 EN06 EN07 EN08 EN09 EN0I EN10
EN11
> > EN12
> > > EN13 EN14 EN15 EN16 EN17 EN18 EN19 EN1I EN20 EN21 EN22 EN23 EN24
EN25
> ENI
> > > EP01 EP02 EP03 EP04 EP05 EP06 EP07 EP08 EP09 EP0I EP10 EP11 EP12
EP13
> > EP14
> > > EP15 EP16 EP17 EP18 EP19 EP1I EP20 EP21 EP22 EP23 EP24 EP25 EPI
GFEX
> GFSI
> > > GFSO HMNI HMON HWF2 HWFI HWRF ICON IVCN LGEM NGX NGX2 NGXI NVGM
OCD5
> RVCN
> > > SHF5 SHIP TABD TABM TABS TCLP TVCA TVCE TVCN TVCX UKX UKX2 UKXI
XTRP
> > > DEBUG 1: Event equalization of tracks found no common cases.
> > > DEBUG 1: Creating output dump file: /tmp/plot_tcmpr_116966.tcst
> > > FILTER: -job filter -line_type TCMPR -event_equal true -dump_row
> > > /tmp/plot_tcmpr_116966.tcst
> > >
> > > DEBUG 2: Job 1 used 0 out of 0 lines read.
> > > DEBUG 3: Total lines read                 = 0
> > > DEBUG 3: Total lines kept                 = 0
> > > DEBUG 3: Rejected for track watch/warn    = 0
> > > DEBUG 3: Rejected for init threshold      = 0
> > > DEBUG 3: Rejected for init string         = 0
> > > DEBUG 3: Rejected for out init mask       = 0
> > > DEBUG 3: Rejected for water only          = 0
> > > DEBUG 3: Rejected for rapid inten         = 0
> > > DEBUG 3: Rejected for landfall            = 0
> > > DEBUG 3: Rejected for amodel              = 0
> > > DEBUG 3: Rejected for bmodel              = 0
> > > DEBUG 3: Rejected for desc                = 0
> > > DEBUG 3: Rejected for storm id            = 0
> > > DEBUG 3: Rejected for basin               = 0
> > > DEBUG 3: Rejected for cyclone             = 0
> > > DEBUG 3: Rejected for storm name          = 0
> > > DEBUG 3: Rejected for init time           = 0
> > > DEBUG 3: Rejected for init hour           = 0
> > > DEBUG 3: Rejected for lead time           = 0
> > > DEBUG 3: Rejected for required lead times = 0
> > > DEBUG 3: Rejected for valid time          = 0
> > > DEBUG 3: Rejected for valid hour          = 0
> > > DEBUG 3: Rejected for init mask           = 0
> > > DEBUG 3: Rejected for valid mask          = 0
> > > DEBUG 3: Rejected for line type           = 0
> > > DEBUG 3: Rejected for numeric threshold   = 0
> > > DEBUG 3: Rejected for string matching     = 0
> > > DEBUG 3: Rejected for match points        = 0
> > > DEBUG 3: Rejected for event equal         = 0
> > > DEBUG 3: Rejected for out init mask       = 0
> > > DEBUG 3: Rejected for out valid mask      = 0
> > > Reading track data: /tmp/plot_tcmpr_116966.tcst
> > > Error in read.table(tcst_tmp_file, header = TRUE) :
> > >   no lines available in input
> > > Execution halted
> > >
> > >
> > >
> > > It sounds like filter arguments may need to be provided to
produce
> > TC-Stat
> > > output (which is called by the R script). You can set this in
your
> > METplus
> > > config file with the TCMPR_PLOTTER_FILTER variable, but it will
be
> > > difficult to debug since the log output is not shown. I would
recommend
> > > modifying the Rscript call from this email and add the -filter
argument
> > > with any filter information to pass to TCStat. I am not sure
settings
> > would
> > > help you produce output, but I have reached out to other METplus
team
> > > members for suggestions.
> > >
> > > Thanks,
> > > George
> > >
> > >
> > >
> > > On Thu, May 13, 2021 at 12:37 PM Yan Jin - NOAA Affiliate via RT
<
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846
>
> > > >
> > > > Hi George,
> > > >
> > > > Thanks so much. Now tc_pairs works, and the output file can be
> > generated.
> > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> > > >
> > > > Then I set "PROCESS_LIST = TCPairs, TCMPRPlotter", and I kept
most
> > > > parameters that initialized with "TCMPR_PLOTTER" the same as
the
> > default
> > > > configuration.
> > > >
> > > > my METplus config file:
> > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test.conf
> > > >
> > > > then run script
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/user_config/run.sh
> > > >
> > > > However, the Rscript cannot plot figures.
> > > > I checked the log file:
> > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210513182819
> > > > [image: image.png]
> > > >
> > > > It seems that the error "NameError: name 'produtil' is not
defined"?
> > > >
> > > > Could you give me some suggestions?
> > > >
> > > > Many thanks.
> > > > Best,
> > > > Yan
> > > >
> > > > On Wed, May 12, 2021 at 5:20 PM George McCabe via RT <
> > met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > > Hi Yan,
> > > > >
> > > > > It looks like the values set for init_beg and init_end in
the
> TCpairs
> > > > > config file are both "20200101_000000" because the INIT_BEG
and
> > > INIT_END
> > > > > time is 2020. There are no adeck tracks for this time, so
all of
> the
> > > data
> > > > > is being filtered out.
> > > > >
> > > > > This is a little confusing in METplus 3.1 because INIT_BEG
and
> > INIT_END
> > > > > control the time looping of execution (if applicable for the
use
> > case)
> > > > AND
> > > > > the values passed to the wrapped TCPairs config file. In
METplus
> > 4.0.0
> > > > > (which was released yesterday and should be available on jet
> soon!),
> > > you
> > > > > can set INIT_BEG and INIT_END to control time looping (if
desired)
> > and
> > > > use
> > > > > TC_PAIRS_INIT_BEG and TC_PAIRS_INIT_END to specify the
values to
> pass
> > > to
> > > > > the TCPairs config file.
> > > > >
> > > > > If you were using METplus 4.0.0, I would recommend setting
> something
> > > > like:
> > > > >
> > > > >
> > > > > LOOP_BY = INIT
> > > > > INIT_TIME_FMT = %Y
> > > > > INIT_BEG = 2020
> > > > > INIT_END = 2020
> > > > > INIT_INCREMENT = 21600
> > > > >
> > > > > TC_PAIRS_INIT_BEG = 20201030_000000
> > > > > TC_PAIRS_INIT_END = 20201031_000000
> > > > >
> > > > >
> > > > > This would run once using 2020 to find filenames and process
data
> > from
> > > > the
> > > > > files that are initialized between 20201030_000000 and
> > 20201031_000000.
> > > > >
> > > > > If using METplus 3.1, you will have to use the INIT_BEG and
> INIT_END
> > > > values
> > > > > to set the values in the wrapped TCPairs config. This is OK
for
> this
> > > use
> > > > > case because you have LOOP_ORDER set to processes and
TCPairs only
> > runs
> > > > > once with that setting.
> > > > >
> > > > > LOOP_BY = INIT
> > > > > INIT_TIME_FMT = %Y%m%d
> > > > > INIT_BEG = 20201030
> > > > > INIT_END = 20201031
> > > > >
> > > > > This configuration should run and only use tracks
initialized
> between
> > > > > 2020-10-30 @ 0Z and 2020-10-31 @ 0Z. If you need to specify
the
> init
> > > time
> > > > > range with hour precision, you can do that as well. Here is
an
> > example:
> > > > >
> > > > > LOOP_BY = INIT
> > > > > INIT_TIME_FMT = %Y%m%d%H
> > > > > INIT_BEG = 2020103012
> > > > > INIT_END = 2020103112
> > > > >
> > > > > This configuration will only use tracks initialized between
> > 2020-10-30
> > > @
> > > > > 12Z and 2020-10-31 @ 12Z.
> > > > >
> > > > > You can also restrict the data used by valid time as well by
using
> > > > > TC_PAIRS_VALID_BEG and TC_PAIRS_VALID_END in METplus 4.0.0
or
> > VALID_BEG
> > > > and
> > > > > VALID_END for 3.1.
> > > > >
> > > > > Let me know if you are still unable to produce output with
these
> > > > changes. I
> > > > > was able to run tc_pairs by hand on Jet and produce a file
with
> > output
> > > > > tracks. You can view it here if you'd like:
> > > > >
> > > > > /lfs1/HFIP/dtc-hurr/George.Mccabe/al292020_pairs.dat.tcst
> > > > >
> > > > > These are the commands I used:
> > > > >
> > > > > export BASIN="[]";
> > > > > export CYCLONE="[]";
> > > > > export DLAND_FILE="/contrib/met/9.1/share/met/tc_data/
> > > > > dland_global_tenth_degree.nc";
> > > > > export INIT_BEG="20201030_000000";
> > > > > export INIT_END="20201031_000000";
> > > > > export INIT_EXCLUDE="[]";
> > > > > export INIT_INCLUDE="[]";
> > > > > export MODEL="[]";
> > > > > export STORM_ID="[\"AL292020\"]";
> > > > > export STORM_NAME="[]";
> > > > > export VALID_BEG="";
> > > > > export VALID_END="";
> > > > > /contrib/met/9.1/bin/tc_pairs -v 4 -bdeck
> > > > > /mnt/lfs4/HFIP/hwrf-data/hwrf-input/abdeck/btk/bal292020.dat
-adeck
> > > > >
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/data/aal292020.dat
> -config
> > > > >
> > > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/use_cases/model_applications/tc_and_extra_tc/TCPairsETCConfig
> > > > > -out /lfs1/HFIP/dtc-hurr/George.Mccabe/al292020_pairs.dat
> > > > >
> > > > > I took the values from your METplus log file and modified
them to
> > write
> > > > to
> > > > > my directory and use a different init time range.
> > > > >
> > > > > Thanks,
> > > > > George
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Wed, May 12, 2021 at 2:04 PM Yan Jin - NOAA Affiliate via
RT <
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > > >
> > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846 >
> > > > > >
> > > > > > Hi George,
> > > > > >
> > > > > > Thanks very much for your instructions. I set
"PROCESS_LIST =
> > > TCPairs",
> > > > > and
> > > > > > didn't specify models: "MODEL =". Then run the script.
However,
> the
> > > > > output
> > > > > > file is still empty. It seems that tc_pairs matched 0
adeck track
> > to
> > > 1
> > > > > > bdeck track. I also tried to set "MODEL = HWRF, HMON", but
the
> > output
> > > > > file
> > > > > > is empty too.
> > > > > >
> > > > > > Here is the log file:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210512193152
> > > > > >
> > > > > > And here is the output file:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> > > > > >
> > > > > > Many thanks.
> > > > > > Best,
> > > > > > Yan
> > > > > >
> > > > > > On Wed, May 12, 2021 at 11:36 AM George McCabe via RT <
> > > > met_help at ucar.edu
> > > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Hi Yan,
> > > > > > >
> > > > > > > I was able to log into Jet this morning and look at your
error.
> > > > > > >
> > > > > > > It looks like the error in the call to Rscript is due to
there
> > > being
> > > > no
> > > > > > > data in the input files generated by TCPairs. There are
> > > > inconsistencies
> > > > > > in
> > > > > > > the TCMPRPlotter wrapper that prevent the log output
from the
> run
> > > to
> > > > be
> > > > > > > seen, which would have made it more clear why the issue
was
> > > > occurring.
> > > > > > >
> > > > > > > I would recommend changing the PROCESS_LIST value in
your
> METplus
> > > > > > > configuration file to only run TCPairs and iterating on
that
> > until
> > > > you
> > > > > > get
> > > > > > > output from that tool. Once you can confirm that you
have
> output,
> > > you
> > > > > can
> > > > > > > add TCMPRPlotter back to the PROCESS_LIST and test on
that
> > portion.
> > > > > > >
> > > > > > > I will create a GitHub issue for updating the
TCMPRPlotter
> > wrapper
> > > to
> > > > > > > produce useful logging output that will make things more
clear
> > > going
> > > > > > > forward.
> > > > > > >
> > > > > > > Let me know if you run into any other issues or need
help
> > > configuring
> > > > > > > TCPairs to run properly. I'm also happy to attend
another
> working
> > > > > session
> > > > > > > if that is helpful.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > George
> > > > > > >
> > > > > > >
> > > > > > > On Tue May 11 10:05:06 2021, yan.jin at noaa.gov wrote:
> > > > > > > > Hi John,
> > > > > > > >
> > > > > > > >
> > > > > > > > The directories and data I mentioned in my last email
are all
> > > > located
> > > > > > > > in
> > > > > > > > Jet.
> > > > > > > >
> > > > > > > >
> > > > > > > > Here is the email I got regarding the Jet maintenance:
> > > > > > > >
> > > > > > > > ***
> > > > > > > >
> > > > > > > > *05/11/21 - Boulder Bastion Maintenance Downtime*
> > > > > > > > Boulder Bastion maintenance is scheduled for Tuesday
05/11/21
> > > from
> > > > > > > > 0600 ~
> > > > > > > > 1000 MT. Boulder Bastion login sessions will be
dropped and
> all
> > > new
> > > > > > > > RDHPCS
> > > > > > > > logins will need to go through the Princeton Bastion.
> Logging
> > In
> > > > > > > > <
> > > > >
> https://rdhpcs-common-docs.rdhpcs.noaa.gov/wiki/index.php/Logging_in
> > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > *The next Jet Maintenance Downtime is TBD*
> > > > > > > >
> > > > > > > > ***
> > > > > > > >
> > > > > > > >
> > > > > > > > I tried to login Princeton but the connection is
closed too.
> > > > > > > >
> > > > > > > >
> > > > > > > > Really appreciate your time and help on this.
> > > > > > > >
> > > > > > > >
> > > > > > > > Best,
> > > > > > > >
> > > > > > > > Yan
> > > > > > > >
> > > > > > > > On Tue, May 11, 2021 at 11:57 AM John Halley Gotway
via RT <
> > > > > > > > met_help at ucar.edu> wrote:
> > > > > > > >
> > > > > > > > > Yan,
> > > > > > > > >
> > > > > > > > > George and I tried to take a look at this, but
couldn't
> find
> > > the
> > > > > > > > > data.
> > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210510214154
> > > > > > > > >
> > > > > > > > > I checked on Hera and WCOSS but couldn't find the
data
> > there. I
> > > > > > > > > should be
> > > > > > > > > able to log on to jet, but when I try, it boots me
off:
> > > > > > > > >
> > > > > > > > > Authentication failed.
> > > > > > > > > Connection to jet-rsa.boulder.rdhpcs.noaa.gov
closed.
> > > > > > > > >
> > > > > > > > > Can you confirm that you're running on jet? And any
idea of
> > jet
> > > > is
> > > > > > > > > down
> > > > > > > > > today or something?
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > John
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Tue, May 11, 2021 at 7:46 AM Yan Jin - NOAA
Affiliate
> via
> > > RT <
> > > > > > > > > met_help at ucar.edu> wrote:
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846
> > > > >
> > > > > > > > > >
> > > > > > > > > > Hi John,
> > > > > > > > > >
> > > > > > > > > > Thanks so much!
> > > > > > > > > >
> > > > > > > > > > Best,
> > > > > > > > > > Yan
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On Tue, May 11, 2021 at 9:44 AM John Halley Gotway
via
> RT <
> > > > > > > > > > met_help at ucar.edu>
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Hi Yan,
> > > > > > > > > > >
> > > > > > > > > > > I'm going to assign this ticket to George and
ask him
> to
> > > > take a
> > > > > > > > > > > look
> > > > > > > > > and
> > > > > > > > > > > debug this runtime error.
> > > > > > > > > > >
> > > > > > > > > > > Thanks,
> > > > > > > > > > > John HG
> > > > > > > > > > >
> > > > > > > > > > > On Mon, May 10, 2021 at 3:49 PM Yan Jin - NOAA
> Affiliate
> > > via
> > > > > RT <
> > > > > > > > > > > met_help at ucar.edu> wrote:
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > Mon May 10 15:49:51 2021: Request 99846 was
acted
> upon.
> > > > > > > > > > > > Transaction: Ticket created by
yan.jin at noaa.gov
> > > > > > > > > > > >        Queue: met_help
> > > > > > > > > > > >      Subject: ask a question about METplus
config of
> TC
> > > > > > > > > > > >        Owner: Nobody
> > > > > > > > > > > >   Requestors: yan.jin at noaa.gov
> > > > > > > > > > > >       Status: new
> > > > > > > > > > > >  Ticket <URL:
> > > > > > > > >
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > Hi,
> > > > > > > > > > > >
> > > > > > > > > > > > This is Yan, again. I ran the METplus config
file
> after
> > > our
> > > > > > > > > discussion,
> > > > > > > > > > > > however, it seems that the tc_pairs tcst file
wasn't
> > > > > generated.
> > > > > > > > > > > >
> > > > > > > > > > > > Here is the script in my directory on Jet:
> > > > > > > > > > > > I set TC_PAIRS_STORM_ID = AL292020, so the
code can
> > find
> > > > the
> > > > > > > > > > > > input
> > > > > > > > > > adeck
> > > > > > > > > > > > and bdeck data correctly
> > > > > > > > > > > > *----
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test.conf
> > > > > > > > > > > >
> > > > > > > > > > > > The output directory is
> > > > > > > > > > > >
> > > > > > >
> >
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs
> > > > > > > > > > > > and the tc_pairs data: al292020_pairs.dat.tcst
> > > > > > > > > > > >
> > > > > > > > > > > > log file
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210510214154
> > > > > > > > > > > > *-----
> > > > > > > > > > > >
> > > > > > > > > > > > The error starts:[image: image.png]
> > > > > > > > > > > >
> > > > > > > > > > > > Then I checked al292020_pairs.dat.tcst, and it
is
> > empty.
> > > > > > > > > > > >
> > > > > > > > > > > > I wonder if there is anything I missed?
> > > > > > > > > > > >
> > > > > > > > > > > > Many thanks.
> > > > > > > > > > > > Best,
> > > > > > > > > > > > Yan
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > > --
> > > > > George McCabe - Software Engineer III
> > > > > National Center for Atmospheric Research
> > > > > Research Applications Laboratory
> > > > > 303-497-2768
> > > > > ---
> > > > > My working day may not be your working day. Please do not
feel
> > obliged
> > > to
> > > > > reply to this email outside of your normal working hours.
> > > > >
> > > > >
> > > >
> > > >
> > >
> > > --
> > > George McCabe - Software Engineer III
> > > National Center for Atmospheric Research
> > > Research Applications Laboratory
> > > 303-497-2768
> > > ---
> > > My working day may not be your working day. Please do not feel
obliged
> to
> > > reply to this email outside of your normal working hours.
> > >
> > >
> >
> >
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>

------------------------------------------------
Subject: ask a question about METplus config of TC
From: Yan Jin - NOAA Affiliate
Time: Wed May 19 16:00:35 2021

Hi George,

I have a few questions regarding the TCMPR_PLOTTER_* parameters in the
METplus config file.

Here is the METplus config file I used:
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/vrfy_METplus4.0/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test.conf
**
#  Plot_TCMPR options, if left unset, default values that are
#  pre-defined in the R utility (packaged with MET) will be used.
TCMPR_PLOTTER_CONFIG_FILE = {CONFIG_DIR}/TCMPRPlotterConfig_customize
TCMPR_PLOTTER_PREFIX =
TCMPR_PLOTTER_TITLE =
TCMPR_PLOTTER_SUBTITLE =
TCMPR_PLOTTER_XLAB =
TCMPR_PLOTTER_YLAB =
TCMPR_PLOTTER_XLIM =
TCMPR_PLOTTER_YLIM =
TCMPR_PLOTTER_FILTER = -amodel HWRF,HMON
# the tcst data file to be used instead of running the MET tc_stat
tool.
TCMPR_PLOTTER_FILTERED_TCST_DATA_FILE =
# Comma separated, no whitespace.  Default is TK_ERR (track error)
unless
# otherwise indicated.
TCMPR_PLOTTER_DEP_VARS = ABS(AMAX_WIND-BMAX_WIND),TK_ERR
TCMPR_PLOTTER_SCATTER_X =
TCMPR_PLOTTER_SCATTER_Y =
TCMPR_PLOTTER_SKILL_REF =
TCMPR_PLOTTER_SERIES = AMODEL
TCMPR_PLOTTER_SERIES_CI = TRUE
TCMPR_PLOTTER_LEGEND =
TCMPR_PLOTTER_LEAD = 0,12,24,36,48,60,72,84,96,108,120
# Mean and median plots. These override the plot_tcmpr.R default of
box
plot.
# If box plot is desired, this needs to be explicitly indicated.
TCMPR_PLOTTER_PLOT_TYPES = MEAN,MEDIAN,BOXPLOT
TCMPR_PLOTTER_RP_DIFF =
TCMPR_PLOTTER_DEMO_YR =
TCMPR_PLOTTER_HFIP_BASELINE =
TCMPR_PLOTTER_FOOTNOTE_FLAG =
TCMPR_PLOTTER_PLOT_CONFIG_OPTS =
TCMPR_PLOTTER_SAVE_DATA =

#  TCMPR FLAGS no == (don't set flag), yes == (set flag)
TCMPR_PLOTTER_NO_EE = no
TCMPR_PLOTTER_NO_LOG = no
TCMPR_PLOTTER_SAVE = no
**
It generated the following 6 *.png files
in
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/vrfy_METplus4.0/Eta_use_METplus/tcmpr_plots
[image: image.png]

Then I would like to modify titles, xlab text, output file names.
Here are the modification I
did
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/vrfy_METplus4.0/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test1.conf

**
TCMPR_PLOTTER_TITLE = Absolute Intensity Error \n Eta (Atlantic 2020)
TCMPR_PLOTTER_XLAB = Lead Time (h)
**
The title of each plot becomes "Absolute Intensity Error Eta (Atlantic
2020)" with double quotes.

I also tried to set TCMPR_PLOTTER_PREFIX = Eta_ because I would like
to
save each plot as Eta_ABS_AMAX_WIND-BMAX_WIND_*.png, and
Eta_TK_ERR_*.png.
However, there is only one plot Eta_.png generated.

I would like to check how can I set those parameters if I would like
to 1)
add TC name to each plot name; 2) modify titles to each plot, e.g.:
TK_ERR*
plots with name "Track Error", and ABS_AMAX_WIND-BMAX_WIND_* plots
with
name "Absolute Intensity Error"? 3) Is it possible to modify the line
colors in this METplus config file?

Could you give me some suggestions?

Many thanks.
Best,
Yan

On Mon, May 17, 2021 at 10:37 AM George McCabe via RT
<met_help at ucar.edu>
wrote:

> Hi Yan,
>
> 4.1:
> The consensus variable is not yet controlled by a METplus
configuration
> variable explicitly, but you can set the value by using the
> TC_PAIRS_MET_CONFIG_OVERRIDES value (in v4.0.0+). To set the
consensus
> value to what you have set in your config file, you would set
something
> like this:
>
> TC_PAIRS_MET_CONFIG_OVERRIDES = consensus = [ { name = "CONS";
members =
> ["HWRF","GFSO","HMON"]; required = [false,false,false]; min_req = 2;
} ];
>
> I will make a note in the GitHub issue for improvements to the
TCPairs
> wrapper (https://github.com/dtcenter/METplus/issues/749) to add
support
> for
> consensus explicitly via something like TC_PAIRS_CONSENSUS so that
it is
> easier to configure via METplus wrappers.
>
> 4.2:
> The System Configuration section of the METplus User's Guide covers
these
> variables and how they affect execution. I suggest that you read
through
> this section to get a full picture of how these variables work.
Regarding
> your specific question, these 2 sections should help:
>
> LOOP_BY:
>
>
https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#loop-
by
> LOOP_ORDER:
>
>
https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#loop-
order
>
> Thanks,
> George
>
> On Fri, May 14, 2021 at 8:19 AM Yan Jin - NOAA Affiliate via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846 >
> >
> > Hi George,
> >
> > Thanks so much for your help. Finally the code can run without
errors.
> Here
> > is the steps and results:
> >
> >
> >
> https://docs.google.com/document/d/1xRF3aiPATcX5G37ngJj-
65pl44LN7mGHo34Sf8i3zpo/edit
> >
> > I also put questions in the document, I also pasted here:
> >
> > Questions:
> >
> > 4.1: How can I do the consensus forecasts in METplus config file
as I did
> > in tc_pairs?
> >
> >
>
(/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_292020/TCPairsConfig_run1)
> >
> > //
> >
> > // Specify how consensus forecasts should be defined.
> >
> > //
> >
> > consensus = [
> >
> >  {
> >
> >   name = "CONS";
> >
> >   members = ["HWRF","GFSO","HMON"];
> >
> >   required = [false,false,false];
> >
> >   min_req = 2;
> >
> >  }
> >
> > ];
> >
> > 4.2: I am a little confused about
> >
> >       ***
> >
> >        LOOP_ORDER = processes
> >
> >        LOOP_BY = INIT
> >
> >       ***
> >
> > Could you explain these two command lines and give me some
examples?
> >
> > Thanks very much for your time and help on my questions.
> >
> > Best,
> > Yan
> >
> > On Thu, May 13, 2021 at 3:04 PM George McCabe via RT
<met_help at ucar.edu>
> > wrote:
> >
> > > Hi Yan,
> > >
> > > The produtil is due to a bug in the wrapper in the logic that
reports
> > when
> > > an error occurs. I ran the command on Jet by hand and it looks
like it
> is
> > > still not finding any input data to plot. You can run the
following
> > > commands (after loading all of the required modules):
> > >
> > > export MET_INSTALL_DIR=/contrib/met/9.1
> > >
> > > Rscript /contrib/met/9.1/share/met/Rscripts/plot_tcmpr.R \
> > > -config
> > >
> > >
> >
> /contrib/met/METplus/METplus-
3.1.1/parm/use_cases/model_applications/tc_and_extra_tc/TCMPRPlotterConfig_customize
> > > \
> > > -subtitle "Your subtitle goes here" -ylab "Your y-label goes
here" \
> > > -dep AMAX_WIND-BMAX_WIND,TK_ERR -legend "Your legend text goes
> here..." \
> > > -plot MEAN,MEDIAN,BOXPLOT \
> > > -lookin
> > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> > > \
> > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
> > > \
> > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
> > > \
> > > -outdir
> > >
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tcmpr_plots
> > >
> > >
> > > Here is the output from that run:
> > >
> > > Reading plot configuration file:
> > >
> > >
> >
> /contrib/met/METplus/METplus-
3.1.1/parm/use_cases/model_applications/tc_and_extra_tc/TCMPRPlotterConfig_customize
> > > No plotting configuration file specified.
> > > CALLING:  /contrib/met/9.1/bin/tc_stat -lookin
> > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> > > -lookin
> > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
> > > -lookin
> > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
> > > -job filter -dump_row /tmp/plot_tcmpr_116966.tcst  -event_equal
true
> -v 3
> > > DEBUG 1: [Source 1] Lookin:
> > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> > > DEBUG 1: [Source 2] Lookin:
> > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
> > > DEBUG 1: [Source 3] Lookin:
> > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
> > > DEBUG 2:
> > > DEBUG 2: Processing Job 1: -job filter -event_equal true
-dump_row
> > > /tmp/plot_tcmpr_116966.tcst
> > > DEBUG 3: Applying track-based event equalization logic.
> > > DEBUG 3: Reading file 1 of 3:
> > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> > > DEBUG 3: Reading file 2 of 3:
> > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
> > > DEBUG 3: Reading file 3 of 3:
> > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
> > > DEBUG 3: For track-based event equalization, identified 0 common
cases
> > for
> > > 146 models: AC00 AEMI AEMN AP01 AP02 AP03 AP04 AP05 AP06 AP07
AP08 AP09
> > > AP0I AP10 AP11 AP12 AP13 AP14 AP15 AP16 AP17 AP18 AP19 AP1I AP20
AP21
> > AP22
> > > AP23 AP24 AP25 AP26 AP27 AP28 AP29 AP30 API CARQ CEM2 CEMI CEMN
CLP5
> CMC
> > > CMC2 CMCI COTC COTI CTCX DRCL DSHP EC00 EEMN EGRR EMN2 EMN3 EMN4
EMNI
> EMX
> > > EMX2 EMXI EN01 EN02 EN03 EN04 EN05 EN06 EN07 EN08 EN09 EN0I EN10
EN11
> > EN12
> > > EN13 EN14 EN15 EN16 EN17 EN18 EN19 EN1I EN20 EN21 EN22 EN23 EN24
EN25
> ENI
> > > EP01 EP02 EP03 EP04 EP05 EP06 EP07 EP08 EP09 EP0I EP10 EP11 EP12
EP13
> > EP14
> > > EP15 EP16 EP17 EP18 EP19 EP1I EP20 EP21 EP22 EP23 EP24 EP25 EPI
GFEX
> GFSI
> > > GFSO HMNI HMON HWF2 HWFI HWRF ICON IVCN LGEM NGX NGX2 NGXI NVGM
OCD5
> RVCN
> > > SHF5 SHIP TABD TABM TABS TCLP TVCA TVCE TVCN TVCX UKX UKX2 UKXI
XTRP
> > > DEBUG 1: Event equalization of tracks found no common cases.
> > > DEBUG 1: Creating output dump file: /tmp/plot_tcmpr_116966.tcst
> > > FILTER: -job filter -line_type TCMPR -event_equal true -dump_row
> > > /tmp/plot_tcmpr_116966.tcst
> > >
> > > DEBUG 2: Job 1 used 0 out of 0 lines read.
> > > DEBUG 3: Total lines read                 = 0
> > > DEBUG 3: Total lines kept                 = 0
> > > DEBUG 3: Rejected for track watch/warn    = 0
> > > DEBUG 3: Rejected for init threshold      = 0
> > > DEBUG 3: Rejected for init string         = 0
> > > DEBUG 3: Rejected for out init mask       = 0
> > > DEBUG 3: Rejected for water only          = 0
> > > DEBUG 3: Rejected for rapid inten         = 0
> > > DEBUG 3: Rejected for landfall            = 0
> > > DEBUG 3: Rejected for amodel              = 0
> > > DEBUG 3: Rejected for bmodel              = 0
> > > DEBUG 3: Rejected for desc                = 0
> > > DEBUG 3: Rejected for storm id            = 0
> > > DEBUG 3: Rejected for basin               = 0
> > > DEBUG 3: Rejected for cyclone             = 0
> > > DEBUG 3: Rejected for storm name          = 0
> > > DEBUG 3: Rejected for init time           = 0
> > > DEBUG 3: Rejected for init hour           = 0
> > > DEBUG 3: Rejected for lead time           = 0
> > > DEBUG 3: Rejected for required lead times = 0
> > > DEBUG 3: Rejected for valid time          = 0
> > > DEBUG 3: Rejected for valid hour          = 0
> > > DEBUG 3: Rejected for init mask           = 0
> > > DEBUG 3: Rejected for valid mask          = 0
> > > DEBUG 3: Rejected for line type           = 0
> > > DEBUG 3: Rejected for numeric threshold   = 0
> > > DEBUG 3: Rejected for string matching     = 0
> > > DEBUG 3: Rejected for match points        = 0
> > > DEBUG 3: Rejected for event equal         = 0
> > > DEBUG 3: Rejected for out init mask       = 0
> > > DEBUG 3: Rejected for out valid mask      = 0
> > > Reading track data: /tmp/plot_tcmpr_116966.tcst
> > > Error in read.table(tcst_tmp_file, header = TRUE) :
> > >   no lines available in input
> > > Execution halted
> > >
> > >
> > >
> > > It sounds like filter arguments may need to be provided to
produce
> > TC-Stat
> > > output (which is called by the R script). You can set this in
your
> > METplus
> > > config file with the TCMPR_PLOTTER_FILTER variable, but it will
be
> > > difficult to debug since the log output is not shown. I would
recommend
> > > modifying the Rscript call from this email and add the -filter
argument
> > > with any filter information to pass to TCStat. I am not sure
settings
> > would
> > > help you produce output, but I have reached out to other METplus
team
> > > members for suggestions.
> > >
> > > Thanks,
> > > George
> > >
> > >
> > >
> > > On Thu, May 13, 2021 at 12:37 PM Yan Jin - NOAA Affiliate via RT
<
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846
>
> > > >
> > > > Hi George,
> > > >
> > > > Thanks so much. Now tc_pairs works, and the output file can be
> > generated.
> > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> > > >
> > > > Then I set "PROCESS_LIST = TCPairs, TCMPRPlotter", and I kept
most
> > > > parameters that initialized with "TCMPR_PLOTTER" the same as
the
> > default
> > > > configuration.
> > > >
> > > > my METplus config file:
> > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test.conf
> > > >
> > > > then run script
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/user_config/run.sh
> > > >
> > > > However, the Rscript cannot plot figures.
> > > > I checked the log file:
> > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210513182819
> > > > [image: image.png]
> > > >
> > > > It seems that the error "NameError: name 'produtil' is not
defined"?
> > > >
> > > > Could you give me some suggestions?
> > > >
> > > > Many thanks.
> > > > Best,
> > > > Yan
> > > >
> > > > On Wed, May 12, 2021 at 5:20 PM George McCabe via RT <
> > met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > > Hi Yan,
> > > > >
> > > > > It looks like the values set for init_beg and init_end in
the
> TCpairs
> > > > > config file are both "20200101_000000" because the INIT_BEG
and
> > > INIT_END
> > > > > time is 2020. There are no adeck tracks for this time, so
all of
> the
> > > data
> > > > > is being filtered out.
> > > > >
> > > > > This is a little confusing in METplus 3.1 because INIT_BEG
and
> > INIT_END
> > > > > control the time looping of execution (if applicable for the
use
> > case)
> > > > AND
> > > > > the values passed to the wrapped TCPairs config file. In
METplus
> > 4.0.0
> > > > > (which was released yesterday and should be available on jet
> soon!),
> > > you
> > > > > can set INIT_BEG and INIT_END to control time looping (if
desired)
> > and
> > > > use
> > > > > TC_PAIRS_INIT_BEG and TC_PAIRS_INIT_END to specify the
values to
> pass
> > > to
> > > > > the TCPairs config file.
> > > > >
> > > > > If you were using METplus 4.0.0, I would recommend setting
> something
> > > > like:
> > > > >
> > > > >
> > > > > LOOP_BY = INIT
> > > > > INIT_TIME_FMT = %Y
> > > > > INIT_BEG = 2020
> > > > > INIT_END = 2020
> > > > > INIT_INCREMENT = 21600
> > > > >
> > > > > TC_PAIRS_INIT_BEG = 20201030_000000
> > > > > TC_PAIRS_INIT_END = 20201031_000000
> > > > >
> > > > >
> > > > > This would run once using 2020 to find filenames and process
data
> > from
> > > > the
> > > > > files that are initialized between 20201030_000000 and
> > 20201031_000000.
> > > > >
> > > > > If using METplus 3.1, you will have to use the INIT_BEG and
> INIT_END
> > > > values
> > > > > to set the values in the wrapped TCPairs config. This is OK
for
> this
> > > use
> > > > > case because you have LOOP_ORDER set to processes and
TCPairs only
> > runs
> > > > > once with that setting.
> > > > >
> > > > > LOOP_BY = INIT
> > > > > INIT_TIME_FMT = %Y%m%d
> > > > > INIT_BEG = 20201030
> > > > > INIT_END = 20201031
> > > > >
> > > > > This configuration should run and only use tracks
initialized
> between
> > > > > 2020-10-30 @ 0Z and 2020-10-31 @ 0Z. If you need to specify
the
> init
> > > time
> > > > > range with hour precision, you can do that as well. Here is
an
> > example:
> > > > >
> > > > > LOOP_BY = INIT
> > > > > INIT_TIME_FMT = %Y%m%d%H
> > > > > INIT_BEG = 2020103012
> > > > > INIT_END = 2020103112
> > > > >
> > > > > This configuration will only use tracks initialized between
> > 2020-10-30
> > > @
> > > > > 12Z and 2020-10-31 @ 12Z.
> > > > >
> > > > > You can also restrict the data used by valid time as well by
using
> > > > > TC_PAIRS_VALID_BEG and TC_PAIRS_VALID_END in METplus 4.0.0
or
> > VALID_BEG
> > > > and
> > > > > VALID_END for 3.1.
> > > > >
> > > > > Let me know if you are still unable to produce output with
these
> > > > changes. I
> > > > > was able to run tc_pairs by hand on Jet and produce a file
with
> > output
> > > > > tracks. You can view it here if you'd like:
> > > > >
> > > > > /lfs1/HFIP/dtc-hurr/George.Mccabe/al292020_pairs.dat.tcst
> > > > >
> > > > > These are the commands I used:
> > > > >
> > > > > export BASIN="[]";
> > > > > export CYCLONE="[]";
> > > > > export DLAND_FILE="/contrib/met/9.1/share/met/tc_data/
> > > > > dland_global_tenth_degree.nc";
> > > > > export INIT_BEG="20201030_000000";
> > > > > export INIT_END="20201031_000000";
> > > > > export INIT_EXCLUDE="[]";
> > > > > export INIT_INCLUDE="[]";
> > > > > export MODEL="[]";
> > > > > export STORM_ID="[\"AL292020\"]";
> > > > > export STORM_NAME="[]";
> > > > > export VALID_BEG="";
> > > > > export VALID_END="";
> > > > > /contrib/met/9.1/bin/tc_pairs -v 4 -bdeck
> > > > > /mnt/lfs4/HFIP/hwrf-data/hwrf-input/abdeck/btk/bal292020.dat
-adeck
> > > > >
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/data/aal292020.dat
> -config
> > > > >
> > > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/use_cases/model_applications/tc_and_extra_tc/TCPairsETCConfig
> > > > > -out /lfs1/HFIP/dtc-hurr/George.Mccabe/al292020_pairs.dat
> > > > >
> > > > > I took the values from your METplus log file and modified
them to
> > write
> > > > to
> > > > > my directory and use a different init time range.
> > > > >
> > > > > Thanks,
> > > > > George
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Wed, May 12, 2021 at 2:04 PM Yan Jin - NOAA Affiliate via
RT <
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > > >
> > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846 >
> > > > > >
> > > > > > Hi George,
> > > > > >
> > > > > > Thanks very much for your instructions. I set
"PROCESS_LIST =
> > > TCPairs",
> > > > > and
> > > > > > didn't specify models: "MODEL =". Then run the script.
However,
> the
> > > > > output
> > > > > > file is still empty. It seems that tc_pairs matched 0
adeck track
> > to
> > > 1
> > > > > > bdeck track. I also tried to set "MODEL = HWRF, HMON", but
the
> > output
> > > > > file
> > > > > > is empty too.
> > > > > >
> > > > > > Here is the log file:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210512193152
> > > > > >
> > > > > > And here is the output file:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> > > > > >
> > > > > > Many thanks.
> > > > > > Best,
> > > > > > Yan
> > > > > >
> > > > > > On Wed, May 12, 2021 at 11:36 AM George McCabe via RT <
> > > > met_help at ucar.edu
> > > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Hi Yan,
> > > > > > >
> > > > > > > I was able to log into Jet this morning and look at your
error.
> > > > > > >
> > > > > > > It looks like the error in the call to Rscript is due to
there
> > > being
> > > > no
> > > > > > > data in the input files generated by TCPairs. There are
> > > > inconsistencies
> > > > > > in
> > > > > > > the TCMPRPlotter wrapper that prevent the log output
from the
> run
> > > to
> > > > be
> > > > > > > seen, which would have made it more clear why the issue
was
> > > > occurring.
> > > > > > >
> > > > > > > I would recommend changing the PROCESS_LIST value in
your
> METplus
> > > > > > > configuration file to only run TCPairs and iterating on
that
> > until
> > > > you
> > > > > > get
> > > > > > > output from that tool. Once you can confirm that you
have
> output,
> > > you
> > > > > can
> > > > > > > add TCMPRPlotter back to the PROCESS_LIST and test on
that
> > portion.
> > > > > > >
> > > > > > > I will create a GitHub issue for updating the
TCMPRPlotter
> > wrapper
> > > to
> > > > > > > produce useful logging output that will make things more
clear
> > > going
> > > > > > > forward.
> > > > > > >
> > > > > > > Let me know if you run into any other issues or need
help
> > > configuring
> > > > > > > TCPairs to run properly. I'm also happy to attend
another
> working
> > > > > session
> > > > > > > if that is helpful.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > George
> > > > > > >
> > > > > > >
> > > > > > > On Tue May 11 10:05:06 2021, yan.jin at noaa.gov wrote:
> > > > > > > > Hi John,
> > > > > > > >
> > > > > > > >
> > > > > > > > The directories and data I mentioned in my last email
are all
> > > > located
> > > > > > > > in
> > > > > > > > Jet.
> > > > > > > >
> > > > > > > >
> > > > > > > > Here is the email I got regarding the Jet maintenance:
> > > > > > > >
> > > > > > > > ***
> > > > > > > >
> > > > > > > > *05/11/21 - Boulder Bastion Maintenance Downtime*
> > > > > > > > Boulder Bastion maintenance is scheduled for Tuesday
05/11/21
> > > from
> > > > > > > > 0600 ~
> > > > > > > > 1000 MT. Boulder Bastion login sessions will be
dropped and
> all
> > > new
> > > > > > > > RDHPCS
> > > > > > > > logins will need to go through the Princeton Bastion.
> Logging
> > In
> > > > > > > > <
> > > > >
> https://rdhpcs-common-docs.rdhpcs.noaa.gov/wiki/index.php/Logging_in
> > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > *The next Jet Maintenance Downtime is TBD*
> > > > > > > >
> > > > > > > > ***
> > > > > > > >
> > > > > > > >
> > > > > > > > I tried to login Princeton but the connection is
closed too.
> > > > > > > >
> > > > > > > >
> > > > > > > > Really appreciate your time and help on this.
> > > > > > > >
> > > > > > > >
> > > > > > > > Best,
> > > > > > > >
> > > > > > > > Yan
> > > > > > > >
> > > > > > > > On Tue, May 11, 2021 at 11:57 AM John Halley Gotway
via RT <
> > > > > > > > met_help at ucar.edu> wrote:
> > > > > > > >
> > > > > > > > > Yan,
> > > > > > > > >
> > > > > > > > > George and I tried to take a look at this, but
couldn't
> find
> > > the
> > > > > > > > > data.
> > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210510214154
> > > > > > > > >
> > > > > > > > > I checked on Hera and WCOSS but couldn't find the
data
> > there. I
> > > > > > > > > should be
> > > > > > > > > able to log on to jet, but when I try, it boots me
off:
> > > > > > > > >
> > > > > > > > > Authentication failed.
> > > > > > > > > Connection to jet-rsa.boulder.rdhpcs.noaa.gov
closed.
> > > > > > > > >
> > > > > > > > > Can you confirm that you're running on jet? And any
idea of
> > jet
> > > > is
> > > > > > > > > down
> > > > > > > > > today or something?
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > John
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Tue, May 11, 2021 at 7:46 AM Yan Jin - NOAA
Affiliate
> via
> > > RT <
> > > > > > > > > met_help at ucar.edu> wrote:
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846
> > > > >
> > > > > > > > > >
> > > > > > > > > > Hi John,
> > > > > > > > > >
> > > > > > > > > > Thanks so much!
> > > > > > > > > >
> > > > > > > > > > Best,
> > > > > > > > > > Yan
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On Tue, May 11, 2021 at 9:44 AM John Halley Gotway
via
> RT <
> > > > > > > > > > met_help at ucar.edu>
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Hi Yan,
> > > > > > > > > > >
> > > > > > > > > > > I'm going to assign this ticket to George and
ask him
> to
> > > > take a
> > > > > > > > > > > look
> > > > > > > > > and
> > > > > > > > > > > debug this runtime error.
> > > > > > > > > > >
> > > > > > > > > > > Thanks,
> > > > > > > > > > > John HG
> > > > > > > > > > >
> > > > > > > > > > > On Mon, May 10, 2021 at 3:49 PM Yan Jin - NOAA
> Affiliate
> > > via
> > > > > RT <
> > > > > > > > > > > met_help at ucar.edu> wrote:
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > Mon May 10 15:49:51 2021: Request 99846 was
acted
> upon.
> > > > > > > > > > > > Transaction: Ticket created by
yan.jin at noaa.gov
> > > > > > > > > > > >        Queue: met_help
> > > > > > > > > > > >      Subject: ask a question about METplus
config of
> TC
> > > > > > > > > > > >        Owner: Nobody
> > > > > > > > > > > >   Requestors: yan.jin at noaa.gov
> > > > > > > > > > > >       Status: new
> > > > > > > > > > > >  Ticket <URL:
> > > > > > > > >
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > Hi,
> > > > > > > > > > > >
> > > > > > > > > > > > This is Yan, again. I ran the METplus config
file
> after
> > > our
> > > > > > > > > discussion,
> > > > > > > > > > > > however, it seems that the tc_pairs tcst file
wasn't
> > > > > generated.
> > > > > > > > > > > >
> > > > > > > > > > > > Here is the script in my directory on Jet:
> > > > > > > > > > > > I set TC_PAIRS_STORM_ID = AL292020, so the
code can
> > find
> > > > the
> > > > > > > > > > > > input
> > > > > > > > > > adeck
> > > > > > > > > > > > and bdeck data correctly
> > > > > > > > > > > > *----
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test.conf
> > > > > > > > > > > >
> > > > > > > > > > > > The output directory is
> > > > > > > > > > > >
> > > > > > >
> >
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs
> > > > > > > > > > > > and the tc_pairs data: al292020_pairs.dat.tcst
> > > > > > > > > > > >
> > > > > > > > > > > > log file
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210510214154
> > > > > > > > > > > > *-----
> > > > > > > > > > > >
> > > > > > > > > > > > The error starts:[image: image.png]
> > > > > > > > > > > >
> > > > > > > > > > > > Then I checked al292020_pairs.dat.tcst, and it
is
> > empty.
> > > > > > > > > > > >
> > > > > > > > > > > > I wonder if there is anything I missed?
> > > > > > > > > > > >
> > > > > > > > > > > > Many thanks.
> > > > > > > > > > > > Best,
> > > > > > > > > > > > Yan
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > > --
> > > > > George McCabe - Software Engineer III
> > > > > National Center for Atmospheric Research
> > > > > Research Applications Laboratory
> > > > > 303-497-2768
> > > > > ---
> > > > > My working day may not be your working day. Please do not
feel
> > obliged
> > > to
> > > > > reply to this email outside of your normal working hours.
> > > > >
> > > > >
> > > >
> > > >
> > >
> > > --
> > > George McCabe - Software Engineer III
> > > National Center for Atmospheric Research
> > > Research Applications Laboratory
> > > 303-497-2768
> > > ---
> > > My working day may not be your working day. Please do not feel
obliged
> to
> > > reply to this email outside of your normal working hours.
> > >
> > >
> >
> >
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>

------------------------------------------------
Subject: ask a question about METplus config of TC
From: George McCabe
Time: Wed May 19 17:09:30 2021

Hi Yan,

1) add TC name to each plot name
2) modify titles to each plot, e.g.: TK_ERR* plots with name "Track
Error",
and ABS_AMAX_WIND-BMAX_WIND_* plots with name "Absolute Intensity
Error"?

It looks like the logic in the R script uses the -prefix value to
replace
the filename instead of prepending it to the plot type text as you
would
guess it would. I was told that this script was typically called
multiple
times to generate the multiple plots, which would provide more control
over
setting the output filenames, titles, etc. The bad news is it looks
like
the wrapper was written to call the R script once with all of the
configurations specified in the METplus config file. The good news is
there
are some new capabilities added to METplus 4.0.0 that could help
produce
the results you need.

Please review this section of the METplus User's Guide, specifically
the
section about specifying instance names to the processes:
https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#process-
list

Alternatively, you can use the UserScript wrapper to build custom
commands
to call the R script directly with any arguments you need if the
wrapper
doesn't support an option you need.
More information on that tool is here:
https://metplus.readthedocs.io/en/latest/Users_Guide/wrappers.html#userscript
and here:
https://metplus.readthedocs.io/en/latest/generated/met_tool_wrapper/UserScript/UserScript_run_once.html#sphx-
glr-generated-met-tool-wrapper-userscript-userscript-run-once-py

I think using different instance names to run TCMPRPlotter multiple
times
with different settings is the better approach in this case. I am
wrapping
up work for the day, but tomorrow I can log into Jet and help you
modify
your config file to do this.


3) Is it possible to modify the line colors in this METplus config
file?

Yes, you can change the line colors through the R script's config
file. You
can copy and rename this file:

https://github.com/dtcenter/MET/blob/main_v10.0/met/scripts/Rscripts/include/plot_tcmpr_config_default.R

change the values that you need to change (look for "color_list"),
then set
the METplus configuration file to point to your new file (set in
TCMPR_PLOTTER_CONFIG_FILE).


Let me know if you have any other questions. I will report back
tomorrow
when have some changes for you to test.

Thanks,
George

On Wed, May 19, 2021 at 4:00 PM Yan Jin - NOAA Affiliate via RT <
met_help at ucar.edu> wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846 >
>
> Hi George,
>
> I have a few questions regarding the TCMPR_PLOTTER_* parameters in
the
> METplus config file.
>
> Here is the METplus config file I used:
>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/vrfy_METplus4.0/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test.conf
> **
> #  Plot_TCMPR options, if left unset, default values that are
> #  pre-defined in the R utility (packaged with MET) will be used.
> TCMPR_PLOTTER_CONFIG_FILE =
{CONFIG_DIR}/TCMPRPlotterConfig_customize
> TCMPR_PLOTTER_PREFIX =
> TCMPR_PLOTTER_TITLE =
> TCMPR_PLOTTER_SUBTITLE =
> TCMPR_PLOTTER_XLAB =
> TCMPR_PLOTTER_YLAB =
> TCMPR_PLOTTER_XLIM =
> TCMPR_PLOTTER_YLIM =
> TCMPR_PLOTTER_FILTER = -amodel HWRF,HMON
> # the tcst data file to be used instead of running the MET tc_stat
tool.
> TCMPR_PLOTTER_FILTERED_TCST_DATA_FILE =
> # Comma separated, no whitespace.  Default is TK_ERR (track error)
unless
> # otherwise indicated.
> TCMPR_PLOTTER_DEP_VARS = ABS(AMAX_WIND-BMAX_WIND),TK_ERR
> TCMPR_PLOTTER_SCATTER_X =
> TCMPR_PLOTTER_SCATTER_Y =
> TCMPR_PLOTTER_SKILL_REF =
> TCMPR_PLOTTER_SERIES = AMODEL
> TCMPR_PLOTTER_SERIES_CI = TRUE
> TCMPR_PLOTTER_LEGEND =
> TCMPR_PLOTTER_LEAD = 0,12,24,36,48,60,72,84,96,108,120
> # Mean and median plots. These override the plot_tcmpr.R default of
box
> plot.
> # If box plot is desired, this needs to be explicitly indicated.
> TCMPR_PLOTTER_PLOT_TYPES = MEAN,MEDIAN,BOXPLOT
> TCMPR_PLOTTER_RP_DIFF =
> TCMPR_PLOTTER_DEMO_YR =
> TCMPR_PLOTTER_HFIP_BASELINE =
> TCMPR_PLOTTER_FOOTNOTE_FLAG =
> TCMPR_PLOTTER_PLOT_CONFIG_OPTS =
> TCMPR_PLOTTER_SAVE_DATA =
>
> #  TCMPR FLAGS no == (don't set flag), yes == (set flag)
> TCMPR_PLOTTER_NO_EE = no
> TCMPR_PLOTTER_NO_LOG = no
> TCMPR_PLOTTER_SAVE = no
> **
> It generated the following 6 *.png files
> in
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/vrfy_METplus4.0/Eta_use_METplus/tcmpr_plots
> [image: image.png]
>
> Then I would like to modify titles, xlab text, output file names.
> Here are the modification I
> did
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/vrfy_METplus4.0/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test1.conf
>
> **
> TCMPR_PLOTTER_TITLE = Absolute Intensity Error \n Eta (Atlantic
2020)
> TCMPR_PLOTTER_XLAB = Lead Time (h)
> **
> The title of each plot becomes "Absolute Intensity Error Eta
(Atlantic
> 2020)" with double quotes.
>
> I also tried to set TCMPR_PLOTTER_PREFIX = Eta_ because I would like
to
> save each plot as Eta_ABS_AMAX_WIND-BMAX_WIND_*.png, and
Eta_TK_ERR_*.png.
> However, there is only one plot Eta_.png generated.
>
> I would like to check how can I set those parameters if I would like
to 1)
> add TC name to each plot name; 2) modify titles to each plot, e.g.:
TK_ERR*
> plots with name "Track Error", and ABS_AMAX_WIND-BMAX_WIND_* plots
with
> name "Absolute Intensity Error"? 3) Is it possible to modify the
line
> colors in this METplus config file?
>
> Could you give me some suggestions?
>
> Many thanks.
> Best,
> Yan
>
> On Mon, May 17, 2021 at 10:37 AM George McCabe via RT
<met_help at ucar.edu>
> wrote:
>
> > Hi Yan,
> >
> > 4.1:
> > The consensus variable is not yet controlled by a METplus
configuration
> > variable explicitly, but you can set the value by using the
> > TC_PAIRS_MET_CONFIG_OVERRIDES value (in v4.0.0+). To set the
consensus
> > value to what you have set in your config file, you would set
something
> > like this:
> >
> > TC_PAIRS_MET_CONFIG_OVERRIDES = consensus = [ { name = "CONS";
members =
> > ["HWRF","GFSO","HMON"]; required = [false,false,false]; min_req =
2; } ];
> >
> > I will make a note in the GitHub issue for improvements to the
TCPairs
> > wrapper (https://github.com/dtcenter/METplus/issues/749) to add
support
> > for
> > consensus explicitly via something like TC_PAIRS_CONSENSUS so that
it is
> > easier to configure via METplus wrappers.
> >
> > 4.2:
> > The System Configuration section of the METplus User's Guide
covers these
> > variables and how they affect execution. I suggest that you read
through
> > this section to get a full picture of how these variables work.
Regarding
> > your specific question, these 2 sections should help:
> >
> > LOOP_BY:
> >
> >
>
https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#loop-
by
> > LOOP_ORDER:
> >
> >
>
https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#loop-
order
> >
> > Thanks,
> > George
> >
> > On Fri, May 14, 2021 at 8:19 AM Yan Jin - NOAA Affiliate via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846 >
> > >
> > > Hi George,
> > >
> > > Thanks so much for your help. Finally the code can run without
errors.
> > Here
> > > is the steps and results:
> > >
> > >
> > >
> >
> https://docs.google.com/document/d/1xRF3aiPATcX5G37ngJj-
65pl44LN7mGHo34Sf8i3zpo/edit
> > >
> > > I also put questions in the document, I also pasted here:
> > >
> > > Questions:
> > >
> > > 4.1: How can I do the consensus forecasts in METplus config file
as I
> did
> > > in tc_pairs?
> > >
> > >
> >
>
(/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_292020/TCPairsConfig_run1)
> > >
> > > //
> > >
> > > // Specify how consensus forecasts should be defined.
> > >
> > > //
> > >
> > > consensus = [
> > >
> > >  {
> > >
> > >   name = "CONS";
> > >
> > >   members = ["HWRF","GFSO","HMON"];
> > >
> > >   required = [false,false,false];
> > >
> > >   min_req = 2;
> > >
> > >  }
> > >
> > > ];
> > >
> > > 4.2: I am a little confused about
> > >
> > >       ***
> > >
> > >        LOOP_ORDER = processes
> > >
> > >        LOOP_BY = INIT
> > >
> > >       ***
> > >
> > > Could you explain these two command lines and give me some
examples?
> > >
> > > Thanks very much for your time and help on my questions.
> > >
> > > Best,
> > > Yan
> > >
> > > On Thu, May 13, 2021 at 3:04 PM George McCabe via RT <
> met_help at ucar.edu>
> > > wrote:
> > >
> > > > Hi Yan,
> > > >
> > > > The produtil is due to a bug in the wrapper in the logic that
reports
> > > when
> > > > an error occurs. I ran the command on Jet by hand and it looks
like
> it
> > is
> > > > still not finding any input data to plot. You can run the
following
> > > > commands (after loading all of the required modules):
> > > >
> > > > export MET_INSTALL_DIR=/contrib/met/9.1
> > > >
> > > > Rscript /contrib/met/9.1/share/met/Rscripts/plot_tcmpr.R \
> > > > -config
> > > >
> > > >
> > >
> >
> /contrib/met/METplus/METplus-
3.1.1/parm/use_cases/model_applications/tc_and_extra_tc/TCMPRPlotterConfig_customize
> > > > \
> > > > -subtitle "Your subtitle goes here" -ylab "Your y-label goes
here" \
> > > > -dep AMAX_WIND-BMAX_WIND,TK_ERR -legend "Your legend text goes
> > here..." \
> > > > -plot MEAN,MEDIAN,BOXPLOT \
> > > > -lookin
> > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> > > > \
> > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
> > > > \
> > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
> > > > \
> > > > -outdir
> > > >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tcmpr_plots
> > > >
> > > >
> > > > Here is the output from that run:
> > > >
> > > > Reading plot configuration file:
> > > >
> > > >
> > >
> >
> /contrib/met/METplus/METplus-
3.1.1/parm/use_cases/model_applications/tc_and_extra_tc/TCMPRPlotterConfig_customize
> > > > No plotting configuration file specified.
> > > > CALLING:  /contrib/met/9.1/bin/tc_stat -lookin
> > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> > > > -lookin
> > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
> > > > -lookin
> > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
> > > > -job filter -dump_row /tmp/plot_tcmpr_116966.tcst
-event_equal true
> > -v 3
> > > > DEBUG 1: [Source 1] Lookin:
> > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> > > > DEBUG 1: [Source 2] Lookin:
> > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
> > > > DEBUG 1: [Source 3] Lookin:
> > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
> > > > DEBUG 2:
> > > > DEBUG 2: Processing Job 1: -job filter -event_equal true
-dump_row
> > > > /tmp/plot_tcmpr_116966.tcst
> > > > DEBUG 3: Applying track-based event equalization logic.
> > > > DEBUG 3: Reading file 1 of 3:
> > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> > > > DEBUG 3: Reading file 2 of 3:
> > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
> > > > DEBUG 3: Reading file 3 of 3:
> > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
> > > > DEBUG 3: For track-based event equalization, identified 0
common
> cases
> > > for
> > > > 146 models: AC00 AEMI AEMN AP01 AP02 AP03 AP04 AP05 AP06 AP07
AP08
> AP09
> > > > AP0I AP10 AP11 AP12 AP13 AP14 AP15 AP16 AP17 AP18 AP19 AP1I
AP20 AP21
> > > AP22
> > > > AP23 AP24 AP25 AP26 AP27 AP28 AP29 AP30 API CARQ CEM2 CEMI
CEMN CLP5
> > CMC
> > > > CMC2 CMCI COTC COTI CTCX DRCL DSHP EC00 EEMN EGRR EMN2 EMN3
EMN4 EMNI
> > EMX
> > > > EMX2 EMXI EN01 EN02 EN03 EN04 EN05 EN06 EN07 EN08 EN09 EN0I
EN10 EN11
> > > EN12
> > > > EN13 EN14 EN15 EN16 EN17 EN18 EN19 EN1I EN20 EN21 EN22 EN23
EN24 EN25
> > ENI
> > > > EP01 EP02 EP03 EP04 EP05 EP06 EP07 EP08 EP09 EP0I EP10 EP11
EP12 EP13
> > > EP14
> > > > EP15 EP16 EP17 EP18 EP19 EP1I EP20 EP21 EP22 EP23 EP24 EP25
EPI GFEX
> > GFSI
> > > > GFSO HMNI HMON HWF2 HWFI HWRF ICON IVCN LGEM NGX NGX2 NGXI
NVGM OCD5
> > RVCN
> > > > SHF5 SHIP TABD TABM TABS TCLP TVCA TVCE TVCN TVCX UKX UKX2
UKXI XTRP
> > > > DEBUG 1: Event equalization of tracks found no common cases.
> > > > DEBUG 1: Creating output dump file:
/tmp/plot_tcmpr_116966.tcst
> > > > FILTER: -job filter -line_type TCMPR -event_equal true
-dump_row
> > > > /tmp/plot_tcmpr_116966.tcst
> > > >
> > > > DEBUG 2: Job 1 used 0 out of 0 lines read.
> > > > DEBUG 3: Total lines read                 = 0
> > > > DEBUG 3: Total lines kept                 = 0
> > > > DEBUG 3: Rejected for track watch/warn    = 0
> > > > DEBUG 3: Rejected for init threshold      = 0
> > > > DEBUG 3: Rejected for init string         = 0
> > > > DEBUG 3: Rejected for out init mask       = 0
> > > > DEBUG 3: Rejected for water only          = 0
> > > > DEBUG 3: Rejected for rapid inten         = 0
> > > > DEBUG 3: Rejected for landfall            = 0
> > > > DEBUG 3: Rejected for amodel              = 0
> > > > DEBUG 3: Rejected for bmodel              = 0
> > > > DEBUG 3: Rejected for desc                = 0
> > > > DEBUG 3: Rejected for storm id            = 0
> > > > DEBUG 3: Rejected for basin               = 0
> > > > DEBUG 3: Rejected for cyclone             = 0
> > > > DEBUG 3: Rejected for storm name          = 0
> > > > DEBUG 3: Rejected for init time           = 0
> > > > DEBUG 3: Rejected for init hour           = 0
> > > > DEBUG 3: Rejected for lead time           = 0
> > > > DEBUG 3: Rejected for required lead times = 0
> > > > DEBUG 3: Rejected for valid time          = 0
> > > > DEBUG 3: Rejected for valid hour          = 0
> > > > DEBUG 3: Rejected for init mask           = 0
> > > > DEBUG 3: Rejected for valid mask          = 0
> > > > DEBUG 3: Rejected for line type           = 0
> > > > DEBUG 3: Rejected for numeric threshold   = 0
> > > > DEBUG 3: Rejected for string matching     = 0
> > > > DEBUG 3: Rejected for match points        = 0
> > > > DEBUG 3: Rejected for event equal         = 0
> > > > DEBUG 3: Rejected for out init mask       = 0
> > > > DEBUG 3: Rejected for out valid mask      = 0
> > > > Reading track data: /tmp/plot_tcmpr_116966.tcst
> > > > Error in read.table(tcst_tmp_file, header = TRUE) :
> > > >   no lines available in input
> > > > Execution halted
> > > >
> > > >
> > > >
> > > > It sounds like filter arguments may need to be provided to
produce
> > > TC-Stat
> > > > output (which is called by the R script). You can set this in
your
> > > METplus
> > > > config file with the TCMPR_PLOTTER_FILTER variable, but it
will be
> > > > difficult to debug since the log output is not shown. I would
> recommend
> > > > modifying the Rscript call from this email and add the -filter
> argument
> > > > with any filter information to pass to TCStat. I am not sure
settings
> > > would
> > > > help you produce output, but I have reached out to other
METplus team
> > > > members for suggestions.
> > > >
> > > > Thanks,
> > > > George
> > > >
> > > >
> > > >
> > > > On Thu, May 13, 2021 at 12:37 PM Yan Jin - NOAA Affiliate via
RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846 >
> > > > >
> > > > > Hi George,
> > > > >
> > > > > Thanks so much. Now tc_pairs works, and the output file can
be
> > > generated.
> > > > >
> > > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> > > > >
> > > > > Then I set "PROCESS_LIST = TCPairs, TCMPRPlotter", and I
kept most
> > > > > parameters that initialized with "TCMPR_PLOTTER" the same as
the
> > > default
> > > > > configuration.
> > > > >
> > > > > my METplus config file:
> > > > >
> > > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test.conf
> > > > >
> > > > > then run script
> > > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/user_config/run.sh
> > > > >
> > > > > However, the Rscript cannot plot figures.
> > > > > I checked the log file:
> > > > >
> > > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210513182819
> > > > > [image: image.png]
> > > > >
> > > > > It seems that the error "NameError: name 'produtil' is not
> defined"?
> > > > >
> > > > > Could you give me some suggestions?
> > > > >
> > > > > Many thanks.
> > > > > Best,
> > > > > Yan
> > > > >
> > > > > On Wed, May 12, 2021 at 5:20 PM George McCabe via RT <
> > > met_help at ucar.edu>
> > > > > wrote:
> > > > >
> > > > > > Hi Yan,
> > > > > >
> > > > > > It looks like the values set for init_beg and init_end in
the
> > TCpairs
> > > > > > config file are both "20200101_000000" because the
INIT_BEG and
> > > > INIT_END
> > > > > > time is 2020. There are no adeck tracks for this time, so
all of
> > the
> > > > data
> > > > > > is being filtered out.
> > > > > >
> > > > > > This is a little confusing in METplus 3.1 because INIT_BEG
and
> > > INIT_END
> > > > > > control the time looping of execution (if applicable for
the use
> > > case)
> > > > > AND
> > > > > > the values passed to the wrapped TCPairs config file. In
METplus
> > > 4.0.0
> > > > > > (which was released yesterday and should be available on
jet
> > soon!),
> > > > you
> > > > > > can set INIT_BEG and INIT_END to control time looping (if
> desired)
> > > and
> > > > > use
> > > > > > TC_PAIRS_INIT_BEG and TC_PAIRS_INIT_END to specify the
values to
> > pass
> > > > to
> > > > > > the TCPairs config file.
> > > > > >
> > > > > > If you were using METplus 4.0.0, I would recommend setting
> > something
> > > > > like:
> > > > > >
> > > > > >
> > > > > > LOOP_BY = INIT
> > > > > > INIT_TIME_FMT = %Y
> > > > > > INIT_BEG = 2020
> > > > > > INIT_END = 2020
> > > > > > INIT_INCREMENT = 21600
> > > > > >
> > > > > > TC_PAIRS_INIT_BEG = 20201030_000000
> > > > > > TC_PAIRS_INIT_END = 20201031_000000
> > > > > >
> > > > > >
> > > > > > This would run once using 2020 to find filenames and
process data
> > > from
> > > > > the
> > > > > > files that are initialized between 20201030_000000 and
> > > 20201031_000000.
> > > > > >
> > > > > > If using METplus 3.1, you will have to use the INIT_BEG
and
> > INIT_END
> > > > > values
> > > > > > to set the values in the wrapped TCPairs config. This is
OK for
> > this
> > > > use
> > > > > > case because you have LOOP_ORDER set to processes and
TCPairs
> only
> > > runs
> > > > > > once with that setting.
> > > > > >
> > > > > > LOOP_BY = INIT
> > > > > > INIT_TIME_FMT = %Y%m%d
> > > > > > INIT_BEG = 20201030
> > > > > > INIT_END = 20201031
> > > > > >
> > > > > > This configuration should run and only use tracks
initialized
> > between
> > > > > > 2020-10-30 @ 0Z and 2020-10-31 @ 0Z. If you need to
specify the
> > init
> > > > time
> > > > > > range with hour precision, you can do that as well. Here
is an
> > > example:
> > > > > >
> > > > > > LOOP_BY = INIT
> > > > > > INIT_TIME_FMT = %Y%m%d%H
> > > > > > INIT_BEG = 2020103012
> > > > > > INIT_END = 2020103112
> > > > > >
> > > > > > This configuration will only use tracks initialized
between
> > > 2020-10-30
> > > > @
> > > > > > 12Z and 2020-10-31 @ 12Z.
> > > > > >
> > > > > > You can also restrict the data used by valid time as well
by
> using
> > > > > > TC_PAIRS_VALID_BEG and TC_PAIRS_VALID_END in METplus 4.0.0
or
> > > VALID_BEG
> > > > > and
> > > > > > VALID_END for 3.1.
> > > > > >
> > > > > > Let me know if you are still unable to produce output with
these
> > > > > changes. I
> > > > > > was able to run tc_pairs by hand on Jet and produce a file
with
> > > output
> > > > > > tracks. You can view it here if you'd like:
> > > > > >
> > > > > > /lfs1/HFIP/dtc-hurr/George.Mccabe/al292020_pairs.dat.tcst
> > > > > >
> > > > > > These are the commands I used:
> > > > > >
> > > > > > export BASIN="[]";
> > > > > > export CYCLONE="[]";
> > > > > > export DLAND_FILE="/contrib/met/9.1/share/met/tc_data/
> > > > > > dland_global_tenth_degree.nc";
> > > > > > export INIT_BEG="20201030_000000";
> > > > > > export INIT_END="20201031_000000";
> > > > > > export INIT_EXCLUDE="[]";
> > > > > > export INIT_INCLUDE="[]";
> > > > > > export MODEL="[]";
> > > > > > export STORM_ID="[\"AL292020\"]";
> > > > > > export STORM_NAME="[]";
> > > > > > export VALID_BEG="";
> > > > > > export VALID_END="";
> > > > > > /contrib/met/9.1/bin/tc_pairs -v 4 -bdeck
> > > > > > /mnt/lfs4/HFIP/hwrf-data/hwrf-
input/abdeck/btk/bal292020.dat
> -adeck
> > > > > >
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/data/aal292020.dat
> > -config
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/use_cases/model_applications/tc_and_extra_tc/TCPairsETCConfig
> > > > > > -out /lfs1/HFIP/dtc-hurr/George.Mccabe/al292020_pairs.dat
> > > > > >
> > > > > > I took the values from your METplus log file and modified
them to
> > > write
> > > > > to
> > > > > > my directory and use a different init time range.
> > > > > >
> > > > > > Thanks,
> > > > > > George
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Wed, May 12, 2021 at 2:04 PM Yan Jin - NOAA Affiliate
via RT <
> > > > > > met_help at ucar.edu> wrote:
> > > > > >
> > > > > > >
> > > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846
> >
> > > > > > >
> > > > > > > Hi George,
> > > > > > >
> > > > > > > Thanks very much for your instructions. I set
"PROCESS_LIST =
> > > > TCPairs",
> > > > > > and
> > > > > > > didn't specify models: "MODEL =". Then run the script.
However,
> > the
> > > > > > output
> > > > > > > file is still empty. It seems that tc_pairs matched 0
adeck
> track
> > > to
> > > > 1
> > > > > > > bdeck track. I also tried to set "MODEL = HWRF, HMON",
but the
> > > output
> > > > > > file
> > > > > > > is empty too.
> > > > > > >
> > > > > > > Here is the log file:
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210512193152
> > > > > > >
> > > > > > > And here is the output file:
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> > > > > > >
> > > > > > > Many thanks.
> > > > > > > Best,
> > > > > > > Yan
> > > > > > >
> > > > > > > On Wed, May 12, 2021 at 11:36 AM George McCabe via RT <
> > > > > met_help at ucar.edu
> > > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hi Yan,
> > > > > > > >
> > > > > > > > I was able to log into Jet this morning and look at
your
> error.
> > > > > > > >
> > > > > > > > It looks like the error in the call to Rscript is due
to
> there
> > > > being
> > > > > no
> > > > > > > > data in the input files generated by TCPairs. There
are
> > > > > inconsistencies
> > > > > > > in
> > > > > > > > the TCMPRPlotter wrapper that prevent the log output
from the
> > run
> > > > to
> > > > > be
> > > > > > > > seen, which would have made it more clear why the
issue was
> > > > > occurring.
> > > > > > > >
> > > > > > > > I would recommend changing the PROCESS_LIST value in
your
> > METplus
> > > > > > > > configuration file to only run TCPairs and iterating
on that
> > > until
> > > > > you
> > > > > > > get
> > > > > > > > output from that tool. Once you can confirm that you
have
> > output,
> > > > you
> > > > > > can
> > > > > > > > add TCMPRPlotter back to the PROCESS_LIST and test on
that
> > > portion.
> > > > > > > >
> > > > > > > > I will create a GitHub issue for updating the
TCMPRPlotter
> > > wrapper
> > > > to
> > > > > > > > produce useful logging output that will make things
more
> clear
> > > > going
> > > > > > > > forward.
> > > > > > > >
> > > > > > > > Let me know if you run into any other issues or need
help
> > > > configuring
> > > > > > > > TCPairs to run properly. I'm also happy to attend
another
> > working
> > > > > > session
> > > > > > > > if that is helpful.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > George
> > > > > > > >
> > > > > > > >
> > > > > > > > On Tue May 11 10:05:06 2021, yan.jin at noaa.gov wrote:
> > > > > > > > > Hi John,
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > The directories and data I mentioned in my last
email are
> all
> > > > > located
> > > > > > > > > in
> > > > > > > > > Jet.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Here is the email I got regarding the Jet
maintenance:
> > > > > > > > >
> > > > > > > > > ***
> > > > > > > > >
> > > > > > > > > *05/11/21 - Boulder Bastion Maintenance Downtime*
> > > > > > > > > Boulder Bastion maintenance is scheduled for Tuesday
> 05/11/21
> > > > from
> > > > > > > > > 0600 ~
> > > > > > > > > 1000 MT. Boulder Bastion login sessions will be
dropped and
> > all
> > > > new
> > > > > > > > > RDHPCS
> > > > > > > > > logins will need to go through the Princeton
Bastion.
> > Logging
> > > In
> > > > > > > > > <
> > > > > >
> > https://rdhpcs-common-
docs.rdhpcs.noaa.gov/wiki/index.php/Logging_in
> > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > *The next Jet Maintenance Downtime is TBD*
> > > > > > > > >
> > > > > > > > > ***
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > I tried to login Princeton but the connection is
closed
> too.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Really appreciate your time and help on this.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Best,
> > > > > > > > >
> > > > > > > > > Yan
> > > > > > > > >
> > > > > > > > > On Tue, May 11, 2021 at 11:57 AM John Halley Gotway
via RT
> <
> > > > > > > > > met_help at ucar.edu> wrote:
> > > > > > > > >
> > > > > > > > > > Yan,
> > > > > > > > > >
> > > > > > > > > > George and I tried to take a look at this, but
couldn't
> > find
> > > > the
> > > > > > > > > > data.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210510214154
> > > > > > > > > >
> > > > > > > > > > I checked on Hera and WCOSS but couldn't find the
data
> > > there. I
> > > > > > > > > > should be
> > > > > > > > > > able to log on to jet, but when I try, it boots me
off:
> > > > > > > > > >
> > > > > > > > > > Authentication failed.
> > > > > > > > > > Connection to jet-rsa.boulder.rdhpcs.noaa.gov
closed.
> > > > > > > > > >
> > > > > > > > > > Can you confirm that you're running on jet? And
any idea
> of
> > > jet
> > > > > is
> > > > > > > > > > down
> > > > > > > > > > today or something?
> > > > > > > > > >
> > > > > > > > > > Thanks,
> > > > > > > > > > John
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On Tue, May 11, 2021 at 7:46 AM Yan Jin - NOAA
Affiliate
> > via
> > > > RT <
> > > > > > > > > > met_help at ucar.edu> wrote:
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > <URL:
> > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846
> > > > > >
> > > > > > > > > > >
> > > > > > > > > > > Hi John,
> > > > > > > > > > >
> > > > > > > > > > > Thanks so much!
> > > > > > > > > > >
> > > > > > > > > > > Best,
> > > > > > > > > > > Yan
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > On Tue, May 11, 2021 at 9:44 AM John Halley
Gotway via
> > RT <
> > > > > > > > > > > met_help at ucar.edu>
> > > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Hi Yan,
> > > > > > > > > > > >
> > > > > > > > > > > > I'm going to assign this ticket to George and
ask him
> > to
> > > > > take a
> > > > > > > > > > > > look
> > > > > > > > > > and
> > > > > > > > > > > > debug this runtime error.
> > > > > > > > > > > >
> > > > > > > > > > > > Thanks,
> > > > > > > > > > > > John HG
> > > > > > > > > > > >
> > > > > > > > > > > > On Mon, May 10, 2021 at 3:49 PM Yan Jin - NOAA
> > Affiliate
> > > > via
> > > > > > RT <
> > > > > > > > > > > > met_help at ucar.edu> wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > Mon May 10 15:49:51 2021: Request 99846 was
acted
> > upon.
> > > > > > > > > > > > > Transaction: Ticket created by
yan.jin at noaa.gov
> > > > > > > > > > > > >        Queue: met_help
> > > > > > > > > > > > >      Subject: ask a question about METplus
config
> of
> > TC
> > > > > > > > > > > > >        Owner: Nobody
> > > > > > > > > > > > >   Requestors: yan.jin at noaa.gov
> > > > > > > > > > > > >       Status: new
> > > > > > > > > > > > >  Ticket <URL:
> > > > > > > > > >
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846
> > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > Hi,
> > > > > > > > > > > > >
> > > > > > > > > > > > > This is Yan, again. I ran the METplus config
file
> > after
> > > > our
> > > > > > > > > > discussion,
> > > > > > > > > > > > > however, it seems that the tc_pairs tcst
file
> wasn't
> > > > > > generated.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Here is the script in my directory on Jet:
> > > > > > > > > > > > > I set TC_PAIRS_STORM_ID = AL292020, so the
code can
> > > find
> > > > > the
> > > > > > > > > > > > > input
> > > > > > > > > > > adeck
> > > > > > > > > > > > > and bdeck data correctly
> > > > > > > > > > > > > *----
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test.conf
> > > > > > > > > > > > >
> > > > > > > > > > > > > The output directory is
> > > > > > > > > > > > >
> > > > > > > >
> > >
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs
> > > > > > > > > > > > > and the tc_pairs data:
al292020_pairs.dat.tcst
> > > > > > > > > > > > >
> > > > > > > > > > > > > log file
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210510214154
> > > > > > > > > > > > > *-----
> > > > > > > > > > > > >
> > > > > > > > > > > > > The error starts:[image: image.png]
> > > > > > > > > > > > >
> > > > > > > > > > > > > Then I checked al292020_pairs.dat.tcst, and
it is
> > > empty.
> > > > > > > > > > > > >
> > > > > > > > > > > > > I wonder if there is anything I missed?
> > > > > > > > > > > > >
> > > > > > > > > > > > > Many thanks.
> > > > > > > > > > > > > Best,
> > > > > > > > > > > > > Yan
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > > --
> > > > > > George McCabe - Software Engineer III
> > > > > > National Center for Atmospheric Research
> > > > > > Research Applications Laboratory
> > > > > > 303-497-2768
> > > > > > ---
> > > > > > My working day may not be your working day. Please do not
feel
> > > obliged
> > > > to
> > > > > > reply to this email outside of your normal working hours.
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > > --
> > > > George McCabe - Software Engineer III
> > > > National Center for Atmospheric Research
> > > > Research Applications Laboratory
> > > > 303-497-2768
> > > > ---
> > > > My working day may not be your working day. Please do not feel
> obliged
> > to
> > > > reply to this email outside of your normal working hours.
> > > >
> > > >
> > >
> > >
> >
> > --
> > George McCabe - Software Engineer III
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > 303-497-2768
> > ---
> > My working day may not be your working day. Please do not feel
obliged to
> > reply to this email outside of your normal working hours.
> >
> >
>
>

--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.

------------------------------------------------
Subject: ask a question about METplus config of TC
From: George McCabe
Time: Thu May 20 09:19:32 2021

Hi Yan,

I copied your config file and made some modifications. You can view it
on
Jet here:

/lfs1/HFIP/dtc-hurr/George.Mccabe/TCMPRPlotter_Eta.conf

Starting in METplus 4.0.0 the different section headers like dir and
filename_templates are no longer necessary. All configurations can go
under
the [config] header. I removed the sections and rearranged the
variables to
group them in a more clear way.

To call TCMPRPlotter multiple times with different settings, I added
multiple instances of the app in the PROCESS_LIST with instance names
defined:

PROCESS_LIST = TCPairs, TCMPRPlotter(abs_mean),
TCMPRPlotter(abs_median),TCMPRPlotter(abs_boxplot),
TCMPRPlotter(tk_mean),
TCMPRPlotter(tk_median),TCMPRPlotter(tk_boxplot)

Each instance will read all of the values set in the [config] section,
then
it will look for a section header that matches the string inside the
parentheses for each instance. If the section is found, it overrides
the
values set in the section to replace the values set in the [config]
section. These names can be whatever you chose, but the name in the
PROCESS_LIST must match the name of the section header.

[abs_mean]
TCMPR_PLOTTER_PREFIX = Eta_Absolute_Intensity_mean
TCMPR_PLOTTER_TITLE = Absolute Intensity Error Mean\n Eta (Atlantic
2020)
TCMPR_PLOTTER_DEP_VARS = ABS(AMAX_WIND-BMAX_WIND)
TCMPR_PLOTTER_PLOT_TYPES = MEAN

[abs_median]
TCMPR_PLOTTER_PREFIX = Eta_Absolute_Intensity_median
TCMPR_PLOTTER_TITLE = Absolute Intensity Error Median\n Eta (Atlantic
2020)
TCMPR_PLOTTER_DEP_VARS = ABS(AMAX_WIND-BMAX_WIND)
TCMPR_PLOTTER_PLOT_TYPES = MEDIAN

[abs_boxplot]
TCMPR_PLOTTER_PREFIX = Eta_Absolute_Intensity_boxplot
TCMPR_PLOTTER_TITLE = Absolute Intensity Error Boxplot\n Eta (Atlantic
2020)
TCMPR_PLOTTER_DEP_VARS = ABS(AMAX_WIND-BMAX_WIND)
TCMPR_PLOTTER_PLOT_TYPES = BOXPLOT

[tk_mean]
TCMPR_PLOTTER_PREFIX = Eta_Track_Error_mean
TCMPR_PLOTTER_TITLE = Track Error Mean\n Eta (Atlantic 2020)
TCMPR_PLOTTER_DEP_VARS = TK_ERR
TCMPR_PLOTTER_PLOT_TYPES = MEAN

[tk_median]
TCMPR_PLOTTER_PREFIX = Eta_Track_Error_median
TCMPR_PLOTTER_TITLE = Track Error Median\n Eta (Atlantic 2020)
TCMPR_PLOTTER_DEP_VARS = TK_ERR
TCMPR_PLOTTER_PLOT_TYPES = MEDIAN

[tk_boxplot]
TCMPR_PLOTTER_PREFIX = Eta_Track_Error_boxplot
TCMPR_PLOTTER_TITLE = Track Error Boxplot\n Eta (Atlantic 2020)
TCMPR_PLOTTER_DEP_VARS = TK_ERR
TCMPR_PLOTTER_PLOT_TYPES = BOXPLOT


I split up the DEP_VARS and PLOT_TYPES values into instances with one
value
of each and set the plot title and filenames appropriately for each. I
set
the X label to "Lead Time (h)" in the [config] section so it applies
to
every instance.

TCMPR_PLOTTER_XLAB = Lead Time (h)

The output images from my run is on Jet here:

/lfs1/HFIP/dtc-hurr/George.Mccabe/eta-out/tcmpr_plots

Eta_Absolute_Intensity_boxplot.log
Eta_Absolute_Intensity_boxplot.png
Eta_Absolute_Intensity_mean.png
Eta_Absolute_Intensity_median.png
Eta_Track_Error_boxplot.log
Eta_Track_Error_boxplot.png
Eta_Track_Error_mean.png
Eta_Track_Error_median.png


Here is the log file from the run:
/lfs1/HFIP/dtc-hurr/George.Mccabe/eta-
out/logs/metplus.log.20210520150804


Here is what I ran to produce the output:

- module load intel
- module load intelpython/3.6.5
- module load netcdf/4.6.1
- module load hdf5/1.10.4
- module load nco/4.9.1
- module load wgrib/1.8.1.0b
- module load wgrib2/2.0.8
- module load R/4.0.2
- module use /contrib/met/modulefiles
- module load met/10.0.0
- module use /contrib/met/METplus/modulefiles
- module load metplus/4.0.0
- run_metplus.py /lfs1/HFIP/dtc-
hurr/George.Mccabe/TCMPRPlotter_Eta.conf
config.OUTPUT_BASE=/lfs1/HFIP/dtc-hurr/George.Mccabe/eta-out


I will look into what settings are needed to change the line colors.
Also,
I have started making the modifications to the TCPairs wrapper to
properly
handle multiple storms specified in the config file. I will let you
know
when that is ready to test.

Thanks,
George



On Wed, May 19, 2021 at 5:09 PM George McCabe <mccabe at ucar.edu> wrote:

> Hi Yan,
>
> 1) add TC name to each plot name
> 2) modify titles to each plot, e.g.: TK_ERR* plots with name "Track
> Error", and ABS_AMAX_WIND-BMAX_WIND_* plots with name "Absolute
Intensity
> Error"?
>
> It looks like the logic in the R script uses the -prefix value to
replace
> the filename instead of prepending it to the plot type text as you
would
> guess it would. I was told that this script was typically called
multiple
> times to generate the multiple plots, which would provide more
control over
> setting the output filenames, titles, etc. The bad news is it looks
like
> the wrapper was written to call the R script once with all of the
> configurations specified in the METplus config file. The good news
is there
> are some new capabilities added to METplus 4.0.0 that could help
produce
> the results you need.
>
> Please review this section of the METplus User's Guide, specifically
the
> section about specifying instance names to the processes:
>
>
https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#process-
list
>
> Alternatively, you can use the UserScript wrapper to build custom
commands
> to call the R script directly with any arguments you need if the
wrapper
> doesn't support an option you need.
> More information on that tool is here:
>
https://metplus.readthedocs.io/en/latest/Users_Guide/wrappers.html#userscript
> and here:
>
https://metplus.readthedocs.io/en/latest/generated/met_tool_wrapper/UserScript/UserScript_run_once.html#sphx-
glr-generated-met-tool-wrapper-userscript-userscript-run-once-py
>
> I think using different instance names to run TCMPRPlotter multiple
times
> with different settings is the better approach in this case. I am
wrapping
> up work for the day, but tomorrow I can log into Jet and help you
modify
> your config file to do this.
>
>
> 3) Is it possible to modify the line colors in this METplus config
file?
>
> Yes, you can change the line colors through the R script's config
file.
> You can copy and rename this file:
>
>
>
https://github.com/dtcenter/MET/blob/main_v10.0/met/scripts/Rscripts/include/plot_tcmpr_config_default.R
>
> change the values that you need to change (look for "color_list"),
then
> set the METplus configuration file to point to your new file (set in
> TCMPR_PLOTTER_CONFIG_FILE).
>
>
> Let me know if you have any other questions. I will report back
tomorrow
> when have some changes for you to test.
>
> Thanks,
> George
>
> On Wed, May 19, 2021 at 4:00 PM Yan Jin - NOAA Affiliate via RT <
> met_help at ucar.edu> wrote:
>
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846 >
>>
>> Hi George,
>>
>> I have a few questions regarding the TCMPR_PLOTTER_* parameters in
the
>> METplus config file.
>>
>> Here is the METplus config file I used:
>>
>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/vrfy_METplus4.0/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test.conf
>> **
>> #  Plot_TCMPR options, if left unset, default values that are
>> #  pre-defined in the R utility (packaged with MET) will be used.
>> TCMPR_PLOTTER_CONFIG_FILE =
{CONFIG_DIR}/TCMPRPlotterConfig_customize
>> TCMPR_PLOTTER_PREFIX =
>> TCMPR_PLOTTER_TITLE =
>> TCMPR_PLOTTER_SUBTITLE =
>> TCMPR_PLOTTER_XLAB =
>> TCMPR_PLOTTER_YLAB =
>> TCMPR_PLOTTER_XLIM =
>> TCMPR_PLOTTER_YLIM =
>> TCMPR_PLOTTER_FILTER = -amodel HWRF,HMON
>> # the tcst data file to be used instead of running the MET tc_stat
tool.
>> TCMPR_PLOTTER_FILTERED_TCST_DATA_FILE =
>> # Comma separated, no whitespace.  Default is TK_ERR (track error)
unless
>> # otherwise indicated.
>> TCMPR_PLOTTER_DEP_VARS = ABS(AMAX_WIND-BMAX_WIND),TK_ERR
>> TCMPR_PLOTTER_SCATTER_X =
>> TCMPR_PLOTTER_SCATTER_Y =
>> TCMPR_PLOTTER_SKILL_REF =
>> TCMPR_PLOTTER_SERIES = AMODEL
>> TCMPR_PLOTTER_SERIES_CI = TRUE
>> TCMPR_PLOTTER_LEGEND =
>> TCMPR_PLOTTER_LEAD = 0,12,24,36,48,60,72,84,96,108,120
>> # Mean and median plots. These override the plot_tcmpr.R default of
box
>> plot.
>> # If box plot is desired, this needs to be explicitly indicated.
>> TCMPR_PLOTTER_PLOT_TYPES = MEAN,MEDIAN,BOXPLOT
>> TCMPR_PLOTTER_RP_DIFF =
>> TCMPR_PLOTTER_DEMO_YR =
>> TCMPR_PLOTTER_HFIP_BASELINE =
>> TCMPR_PLOTTER_FOOTNOTE_FLAG =
>> TCMPR_PLOTTER_PLOT_CONFIG_OPTS =
>> TCMPR_PLOTTER_SAVE_DATA =
>>
>> #  TCMPR FLAGS no == (don't set flag), yes == (set flag)
>> TCMPR_PLOTTER_NO_EE = no
>> TCMPR_PLOTTER_NO_LOG = no
>> TCMPR_PLOTTER_SAVE = no
>> **
>> It generated the following 6 *.png files
>> in
>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/vrfy_METplus4.0/Eta_use_METplus/tcmpr_plots
>> [image: image.png]
>>
>> Then I would like to modify titles, xlab text, output file names.
>> Here are the modification I
>> did
>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/vrfy_METplus4.0/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test1.conf
>>
>> **
>> TCMPR_PLOTTER_TITLE = Absolute Intensity Error \n Eta (Atlantic
2020)
>> TCMPR_PLOTTER_XLAB = Lead Time (h)
>> **
>> The title of each plot becomes "Absolute Intensity Error Eta
(Atlantic
>> 2020)" with double quotes.
>>
>> I also tried to set TCMPR_PLOTTER_PREFIX = Eta_ because I would
like to
>> save each plot as Eta_ABS_AMAX_WIND-BMAX_WIND_*.png, and
Eta_TK_ERR_*.png.
>> However, there is only one plot Eta_.png generated.
>>
>> I would like to check how can I set those parameters if I would
like to 1)
>> add TC name to each plot name; 2) modify titles to each plot, e.g.:
>> TK_ERR*
>> plots with name "Track Error", and ABS_AMAX_WIND-BMAX_WIND_* plots
with
>> name "Absolute Intensity Error"? 3) Is it possible to modify the
line
>> colors in this METplus config file?
>>
>> Could you give me some suggestions?
>>
>> Many thanks.
>> Best,
>> Yan
>>
>> On Mon, May 17, 2021 at 10:37 AM George McCabe via RT
<met_help at ucar.edu>
>> wrote:
>>
>> > Hi Yan,
>> >
>> > 4.1:
>> > The consensus variable is not yet controlled by a METplus
configuration
>> > variable explicitly, but you can set the value by using the
>> > TC_PAIRS_MET_CONFIG_OVERRIDES value (in v4.0.0+). To set the
consensus
>> > value to what you have set in your config file, you would set
something
>> > like this:
>> >
>> > TC_PAIRS_MET_CONFIG_OVERRIDES = consensus = [ { name = "CONS";
members =
>> > ["HWRF","GFSO","HMON"]; required = [false,false,false]; min_req =
2; }
>> ];
>> >
>> > I will make a note in the GitHub issue for improvements to the
TCPairs
>> > wrapper (https://github.com/dtcenter/METplus/issues/749) to add
support
>> > for
>> > consensus explicitly via something like TC_PAIRS_CONSENSUS so
that it is
>> > easier to configure via METplus wrappers.
>> >
>> > 4.2:
>> > The System Configuration section of the METplus User's Guide
covers
>> these
>> > variables and how they affect execution. I suggest that you read
through
>> > this section to get a full picture of how these variables work.
>> Regarding
>> > your specific question, these 2 sections should help:
>> >
>> > LOOP_BY:
>> >
>> >
>>
https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#loop-
by
>> > LOOP_ORDER:
>> >
>> >
>>
https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#loop-
order
>> >
>> > Thanks,
>> > George
>> >
>> > On Fri, May 14, 2021 at 8:19 AM Yan Jin - NOAA Affiliate via RT <
>> > met_help at ucar.edu> wrote:
>> >
>> > >
>> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846 >
>> > >
>> > > Hi George,
>> > >
>> > > Thanks so much for your help. Finally the code can run without
errors.
>> > Here
>> > > is the steps and results:
>> > >
>> > >
>> > >
>> >
>> https://docs.google.com/document/d/1xRF3aiPATcX5G37ngJj-
65pl44LN7mGHo34Sf8i3zpo/edit
>> > >
>> > > I also put questions in the document, I also pasted here:
>> > >
>> > > Questions:
>> > >
>> > > 4.1: How can I do the consensus forecasts in METplus config
file as I
>> did
>> > > in tc_pairs?
>> > >
>> > >
>> >
>>
(/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_292020/TCPairsConfig_run1)
>> > >
>> > > //
>> > >
>> > > // Specify how consensus forecasts should be defined.
>> > >
>> > > //
>> > >
>> > > consensus = [
>> > >
>> > >  {
>> > >
>> > >   name = "CONS";
>> > >
>> > >   members = ["HWRF","GFSO","HMON"];
>> > >
>> > >   required = [false,false,false];
>> > >
>> > >   min_req = 2;
>> > >
>> > >  }
>> > >
>> > > ];
>> > >
>> > > 4.2: I am a little confused about
>> > >
>> > >       ***
>> > >
>> > >        LOOP_ORDER = processes
>> > >
>> > >        LOOP_BY = INIT
>> > >
>> > >       ***
>> > >
>> > > Could you explain these two command lines and give me some
examples?
>> > >
>> > > Thanks very much for your time and help on my questions.
>> > >
>> > > Best,
>> > > Yan
>> > >
>> > > On Thu, May 13, 2021 at 3:04 PM George McCabe via RT <
>> met_help at ucar.edu>
>> > > wrote:
>> > >
>> > > > Hi Yan,
>> > > >
>> > > > The produtil is due to a bug in the wrapper in the logic that
>> reports
>> > > when
>> > > > an error occurs. I ran the command on Jet by hand and it
looks like
>> it
>> > is
>> > > > still not finding any input data to plot. You can run the
following
>> > > > commands (after loading all of the required modules):
>> > > >
>> > > > export MET_INSTALL_DIR=/contrib/met/9.1
>> > > >
>> > > > Rscript /contrib/met/9.1/share/met/Rscripts/plot_tcmpr.R \
>> > > > -config
>> > > >
>> > > >
>> > >
>> >
>> /contrib/met/METplus/METplus-
3.1.1/parm/use_cases/model_applications/tc_and_extra_tc/TCMPRPlotterConfig_customize
>> > > > \
>> > > > -subtitle "Your subtitle goes here" -ylab "Your y-label goes
here" \
>> > > > -dep AMAX_WIND-BMAX_WIND,TK_ERR -legend "Your legend text
goes
>> > here..." \
>> > > > -plot MEAN,MEDIAN,BOXPLOT \
>> > > > -lookin
>> > > >
>> > > >
>> > >
>> >
>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
>> > > > \
>> > > >
>> > > >
>> > >
>> >
>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
>> > > > \
>> > > >
>> > > >
>> > >
>> >
>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
>> > > > \
>> > > > -outdir
>> > > >
>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tcmpr_plots
>> > > >
>> > > >
>> > > > Here is the output from that run:
>> > > >
>> > > > Reading plot configuration file:
>> > > >
>> > > >
>> > >
>> >
>> /contrib/met/METplus/METplus-
3.1.1/parm/use_cases/model_applications/tc_and_extra_tc/TCMPRPlotterConfig_customize
>> > > > No plotting configuration file specified.
>> > > > CALLING:  /contrib/met/9.1/bin/tc_stat -lookin
>> > > >
>> > > >
>> > >
>> >
>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
>> > > > -lookin
>> > > >
>> > > >
>> > >
>> >
>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
>> > > > -lookin
>> > > >
>> > > >
>> > >
>> >
>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
>> > > > -job filter -dump_row /tmp/plot_tcmpr_116966.tcst
-event_equal true
>> > -v 3
>> > > > DEBUG 1: [Source 1] Lookin:
>> > > >
>> > > >
>> > >
>> >
>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
>> > > > DEBUG 1: [Source 2] Lookin:
>> > > >
>> > > >
>> > >
>> >
>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
>> > > > DEBUG 1: [Source 3] Lookin:
>> > > >
>> > > >
>> > >
>> >
>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
>> > > > DEBUG 2:
>> > > > DEBUG 2: Processing Job 1: -job filter -event_equal true
-dump_row
>> > > > /tmp/plot_tcmpr_116966.tcst
>> > > > DEBUG 3: Applying track-based event equalization logic.
>> > > > DEBUG 3: Reading file 1 of 3:
>> > > >
>> > > >
>> > >
>> >
>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
>> > > > DEBUG 3: Reading file 2 of 3:
>> > > >
>> > > >
>> > >
>> >
>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
>> > > > DEBUG 3: Reading file 3 of 3:
>> > > >
>> > > >
>> > >
>> >
>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
>> > > > DEBUG 3: For track-based event equalization, identified 0
common
>> cases
>> > > for
>> > > > 146 models: AC00 AEMI AEMN AP01 AP02 AP03 AP04 AP05 AP06 AP07
AP08
>> AP09
>> > > > AP0I AP10 AP11 AP12 AP13 AP14 AP15 AP16 AP17 AP18 AP19 AP1I
AP20
>> AP21
>> > > AP22
>> > > > AP23 AP24 AP25 AP26 AP27 AP28 AP29 AP30 API CARQ CEM2 CEMI
CEMN CLP5
>> > CMC
>> > > > CMC2 CMCI COTC COTI CTCX DRCL DSHP EC00 EEMN EGRR EMN2 EMN3
EMN4
>> EMNI
>> > EMX
>> > > > EMX2 EMXI EN01 EN02 EN03 EN04 EN05 EN06 EN07 EN08 EN09 EN0I
EN10
>> EN11
>> > > EN12
>> > > > EN13 EN14 EN15 EN16 EN17 EN18 EN19 EN1I EN20 EN21 EN22 EN23
EN24
>> EN25
>> > ENI
>> > > > EP01 EP02 EP03 EP04 EP05 EP06 EP07 EP08 EP09 EP0I EP10 EP11
EP12
>> EP13
>> > > EP14
>> > > > EP15 EP16 EP17 EP18 EP19 EP1I EP20 EP21 EP22 EP23 EP24 EP25
EPI GFEX
>> > GFSI
>> > > > GFSO HMNI HMON HWF2 HWFI HWRF ICON IVCN LGEM NGX NGX2 NGXI
NVGM OCD5
>> > RVCN
>> > > > SHF5 SHIP TABD TABM TABS TCLP TVCA TVCE TVCN TVCX UKX UKX2
UKXI XTRP
>> > > > DEBUG 1: Event equalization of tracks found no common cases.
>> > > > DEBUG 1: Creating output dump file:
/tmp/plot_tcmpr_116966.tcst
>> > > > FILTER: -job filter -line_type TCMPR -event_equal true
-dump_row
>> > > > /tmp/plot_tcmpr_116966.tcst
>> > > >
>> > > > DEBUG 2: Job 1 used 0 out of 0 lines read.
>> > > > DEBUG 3: Total lines read                 = 0
>> > > > DEBUG 3: Total lines kept                 = 0
>> > > > DEBUG 3: Rejected for track watch/warn    = 0
>> > > > DEBUG 3: Rejected for init threshold      = 0
>> > > > DEBUG 3: Rejected for init string         = 0
>> > > > DEBUG 3: Rejected for out init mask       = 0
>> > > > DEBUG 3: Rejected for water only          = 0
>> > > > DEBUG 3: Rejected for rapid inten         = 0
>> > > > DEBUG 3: Rejected for landfall            = 0
>> > > > DEBUG 3: Rejected for amodel              = 0
>> > > > DEBUG 3: Rejected for bmodel              = 0
>> > > > DEBUG 3: Rejected for desc                = 0
>> > > > DEBUG 3: Rejected for storm id            = 0
>> > > > DEBUG 3: Rejected for basin               = 0
>> > > > DEBUG 3: Rejected for cyclone             = 0
>> > > > DEBUG 3: Rejected for storm name          = 0
>> > > > DEBUG 3: Rejected for init time           = 0
>> > > > DEBUG 3: Rejected for init hour           = 0
>> > > > DEBUG 3: Rejected for lead time           = 0
>> > > > DEBUG 3: Rejected for required lead times = 0
>> > > > DEBUG 3: Rejected for valid time          = 0
>> > > > DEBUG 3: Rejected for valid hour          = 0
>> > > > DEBUG 3: Rejected for init mask           = 0
>> > > > DEBUG 3: Rejected for valid mask          = 0
>> > > > DEBUG 3: Rejected for line type           = 0
>> > > > DEBUG 3: Rejected for numeric threshold   = 0
>> > > > DEBUG 3: Rejected for string matching     = 0
>> > > > DEBUG 3: Rejected for match points        = 0
>> > > > DEBUG 3: Rejected for event equal         = 0
>> > > > DEBUG 3: Rejected for out init mask       = 0
>> > > > DEBUG 3: Rejected for out valid mask      = 0
>> > > > Reading track data: /tmp/plot_tcmpr_116966.tcst
>> > > > Error in read.table(tcst_tmp_file, header = TRUE) :
>> > > >   no lines available in input
>> > > > Execution halted
>> > > >
>> > > >
>> > > >
>> > > > It sounds like filter arguments may need to be provided to
produce
>> > > TC-Stat
>> > > > output (which is called by the R script). You can set this in
your
>> > > METplus
>> > > > config file with the TCMPR_PLOTTER_FILTER variable, but it
will be
>> > > > difficult to debug since the log output is not shown. I would
>> recommend
>> > > > modifying the Rscript call from this email and add the
-filter
>> argument
>> > > > with any filter information to pass to TCStat. I am not sure
>> settings
>> > > would
>> > > > help you produce output, but I have reached out to other
METplus
>> team
>> > > > members for suggestions.
>> > > >
>> > > > Thanks,
>> > > > George
>> > > >
>> > > >
>> > > >
>> > > > On Thu, May 13, 2021 at 12:37 PM Yan Jin - NOAA Affiliate via
RT <
>> > > > met_help at ucar.edu> wrote:
>> > > >
>> > > > >
>> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846 >
>> > > > >
>> > > > > Hi George,
>> > > > >
>> > > > > Thanks so much. Now tc_pairs works, and the output file can
be
>> > > generated.
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
>> > > > >
>> > > > > Then I set "PROCESS_LIST = TCPairs, TCMPRPlotter", and I
kept most
>> > > > > parameters that initialized with "TCMPR_PLOTTER" the same
as the
>> > > default
>> > > > > configuration.
>> > > > >
>> > > > > my METplus config file:
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test.conf
>> > > > >
>> > > > > then run script
>> > > > >
>> > > >
>> > >
>> >
>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/user_config/run.sh
>> > > > >
>> > > > > However, the Rscript cannot plot figures.
>> > > > > I checked the log file:
>> > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210513182819
>> > > > > [image: image.png]
>> > > > >
>> > > > > It seems that the error "NameError: name 'produtil' is not
>> defined"?
>> > > > >
>> > > > > Could you give me some suggestions?
>> > > > >
>> > > > > Many thanks.
>> > > > > Best,
>> > > > > Yan
>> > > > >
>> > > > > On Wed, May 12, 2021 at 5:20 PM George McCabe via RT <
>> > > met_help at ucar.edu>
>> > > > > wrote:
>> > > > >
>> > > > > > Hi Yan,
>> > > > > >
>> > > > > > It looks like the values set for init_beg and init_end in
the
>> > TCpairs
>> > > > > > config file are both "20200101_000000" because the
INIT_BEG and
>> > > > INIT_END
>> > > > > > time is 2020. There are no adeck tracks for this time, so
all of
>> > the
>> > > > data
>> > > > > > is being filtered out.
>> > > > > >
>> > > > > > This is a little confusing in METplus 3.1 because
INIT_BEG and
>> > > INIT_END
>> > > > > > control the time looping of execution (if applicable for
the use
>> > > case)
>> > > > > AND
>> > > > > > the values passed to the wrapped TCPairs config file. In
METplus
>> > > 4.0.0
>> > > > > > (which was released yesterday and should be available on
jet
>> > soon!),
>> > > > you
>> > > > > > can set INIT_BEG and INIT_END to control time looping (if
>> desired)
>> > > and
>> > > > > use
>> > > > > > TC_PAIRS_INIT_BEG and TC_PAIRS_INIT_END to specify the
values to
>> > pass
>> > > > to
>> > > > > > the TCPairs config file.
>> > > > > >
>> > > > > > If you were using METplus 4.0.0, I would recommend
setting
>> > something
>> > > > > like:
>> > > > > >
>> > > > > >
>> > > > > > LOOP_BY = INIT
>> > > > > > INIT_TIME_FMT = %Y
>> > > > > > INIT_BEG = 2020
>> > > > > > INIT_END = 2020
>> > > > > > INIT_INCREMENT = 21600
>> > > > > >
>> > > > > > TC_PAIRS_INIT_BEG = 20201030_000000
>> > > > > > TC_PAIRS_INIT_END = 20201031_000000
>> > > > > >
>> > > > > >
>> > > > > > This would run once using 2020 to find filenames and
process
>> data
>> > > from
>> > > > > the
>> > > > > > files that are initialized between 20201030_000000 and
>> > > 20201031_000000.
>> > > > > >
>> > > > > > If using METplus 3.1, you will have to use the INIT_BEG
and
>> > INIT_END
>> > > > > values
>> > > > > > to set the values in the wrapped TCPairs config. This is
OK for
>> > this
>> > > > use
>> > > > > > case because you have LOOP_ORDER set to processes and
TCPairs
>> only
>> > > runs
>> > > > > > once with that setting.
>> > > > > >
>> > > > > > LOOP_BY = INIT
>> > > > > > INIT_TIME_FMT = %Y%m%d
>> > > > > > INIT_BEG = 20201030
>> > > > > > INIT_END = 20201031
>> > > > > >
>> > > > > > This configuration should run and only use tracks
initialized
>> > between
>> > > > > > 2020-10-30 @ 0Z and 2020-10-31 @ 0Z. If you need to
specify the
>> > init
>> > > > time
>> > > > > > range with hour precision, you can do that as well. Here
is an
>> > > example:
>> > > > > >
>> > > > > > LOOP_BY = INIT
>> > > > > > INIT_TIME_FMT = %Y%m%d%H
>> > > > > > INIT_BEG = 2020103012
>> > > > > > INIT_END = 2020103112
>> > > > > >
>> > > > > > This configuration will only use tracks initialized
between
>> > > 2020-10-30
>> > > > @
>> > > > > > 12Z and 2020-10-31 @ 12Z.
>> > > > > >
>> > > > > > You can also restrict the data used by valid time as well
by
>> using
>> > > > > > TC_PAIRS_VALID_BEG and TC_PAIRS_VALID_END in METplus
4.0.0 or
>> > > VALID_BEG
>> > > > > and
>> > > > > > VALID_END for 3.1.
>> > > > > >
>> > > > > > Let me know if you are still unable to produce output
with these
>> > > > > changes. I
>> > > > > > was able to run tc_pairs by hand on Jet and produce a
file with
>> > > output
>> > > > > > tracks. You can view it here if you'd like:
>> > > > > >
>> > > > > > /lfs1/HFIP/dtc-hurr/George.Mccabe/al292020_pairs.dat.tcst
>> > > > > >
>> > > > > > These are the commands I used:
>> > > > > >
>> > > > > > export BASIN="[]";
>> > > > > > export CYCLONE="[]";
>> > > > > > export DLAND_FILE="/contrib/met/9.1/share/met/tc_data/
>> > > > > > dland_global_tenth_degree.nc";
>> > > > > > export INIT_BEG="20201030_000000";
>> > > > > > export INIT_END="20201031_000000";
>> > > > > > export INIT_EXCLUDE="[]";
>> > > > > > export INIT_INCLUDE="[]";
>> > > > > > export MODEL="[]";
>> > > > > > export STORM_ID="[\"AL292020\"]";
>> > > > > > export STORM_NAME="[]";
>> > > > > > export VALID_BEG="";
>> > > > > > export VALID_END="";
>> > > > > > /contrib/met/9.1/bin/tc_pairs -v 4 -bdeck
>> > > > > > /mnt/lfs4/HFIP/hwrf-data/hwrf-
input/abdeck/btk/bal292020.dat
>> -adeck
>> > > > > >
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/data/aal292020.dat
>> > -config
>> > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/use_cases/model_applications/tc_and_extra_tc/TCPairsETCConfig
>> > > > > > -out /lfs1/HFIP/dtc-hurr/George.Mccabe/al292020_pairs.dat
>> > > > > >
>> > > > > > I took the values from your METplus log file and modified
them
>> to
>> > > write
>> > > > > to
>> > > > > > my directory and use a different init time range.
>> > > > > >
>> > > > > > Thanks,
>> > > > > > George
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > >
>> > > > > > On Wed, May 12, 2021 at 2:04 PM Yan Jin - NOAA Affiliate
via RT
>> <
>> > > > > > met_help at ucar.edu> wrote:
>> > > > > >
>> > > > > > >
>> > > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846
>> >
>> > > > > > >
>> > > > > > > Hi George,
>> > > > > > >
>> > > > > > > Thanks very much for your instructions. I set
"PROCESS_LIST =
>> > > > TCPairs",
>> > > > > > and
>> > > > > > > didn't specify models: "MODEL =". Then run the script.
>> However,
>> > the
>> > > > > > output
>> > > > > > > file is still empty. It seems that tc_pairs matched 0
adeck
>> track
>> > > to
>> > > > 1
>> > > > > > > bdeck track. I also tried to set "MODEL = HWRF, HMON",
but the
>> > > output
>> > > > > > file
>> > > > > > > is empty too.
>> > > > > > >
>> > > > > > > Here is the log file:
>> > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210512193152
>> > > > > > >
>> > > > > > > And here is the output file:
>> > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
>> > > > > > >
>> > > > > > > Many thanks.
>> > > > > > > Best,
>> > > > > > > Yan
>> > > > > > >
>> > > > > > > On Wed, May 12, 2021 at 11:36 AM George McCabe via RT <
>> > > > > met_help at ucar.edu
>> > > > > > >
>> > > > > > > wrote:
>> > > > > > >
>> > > > > > > > Hi Yan,
>> > > > > > > >
>> > > > > > > > I was able to log into Jet this morning and look at
your
>> error.
>> > > > > > > >
>> > > > > > > > It looks like the error in the call to Rscript is due
to
>> there
>> > > > being
>> > > > > no
>> > > > > > > > data in the input files generated by TCPairs. There
are
>> > > > > inconsistencies
>> > > > > > > in
>> > > > > > > > the TCMPRPlotter wrapper that prevent the log output
from
>> the
>> > run
>> > > > to
>> > > > > be
>> > > > > > > > seen, which would have made it more clear why the
issue was
>> > > > > occurring.
>> > > > > > > >
>> > > > > > > > I would recommend changing the PROCESS_LIST value in
your
>> > METplus
>> > > > > > > > configuration file to only run TCPairs and iterating
on that
>> > > until
>> > > > > you
>> > > > > > > get
>> > > > > > > > output from that tool. Once you can confirm that you
have
>> > output,
>> > > > you
>> > > > > > can
>> > > > > > > > add TCMPRPlotter back to the PROCESS_LIST and test on
that
>> > > portion.
>> > > > > > > >
>> > > > > > > > I will create a GitHub issue for updating the
TCMPRPlotter
>> > > wrapper
>> > > > to
>> > > > > > > > produce useful logging output that will make things
more
>> clear
>> > > > going
>> > > > > > > > forward.
>> > > > > > > >
>> > > > > > > > Let me know if you run into any other issues or need
help
>> > > > configuring
>> > > > > > > > TCPairs to run properly. I'm also happy to attend
another
>> > working
>> > > > > > session
>> > > > > > > > if that is helpful.
>> > > > > > > >
>> > > > > > > > Thanks,
>> > > > > > > > George
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > On Tue May 11 10:05:06 2021, yan.jin at noaa.gov wrote:
>> > > > > > > > > Hi John,
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > The directories and data I mentioned in my last
email are
>> all
>> > > > > located
>> > > > > > > > > in
>> > > > > > > > > Jet.
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > Here is the email I got regarding the Jet
maintenance:
>> > > > > > > > >
>> > > > > > > > > ***
>> > > > > > > > >
>> > > > > > > > > *05/11/21 - Boulder Bastion Maintenance Downtime*
>> > > > > > > > > Boulder Bastion maintenance is scheduled for
Tuesday
>> 05/11/21
>> > > > from
>> > > > > > > > > 0600 ~
>> > > > > > > > > 1000 MT. Boulder Bastion login sessions will be
dropped
>> and
>> > all
>> > > > new
>> > > > > > > > > RDHPCS
>> > > > > > > > > logins will need to go through the Princeton
Bastion.
>> > Logging
>> > > In
>> > > > > > > > > <
>> > > > > >
>> > https://rdhpcs-common-
docs.rdhpcs.noaa.gov/wiki/index.php/Logging_in
>> > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > *The next Jet Maintenance Downtime is TBD*
>> > > > > > > > >
>> > > > > > > > > ***
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > I tried to login Princeton but the connection is
closed
>> too.
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > Really appreciate your time and help on this.
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > Best,
>> > > > > > > > >
>> > > > > > > > > Yan
>> > > > > > > > >
>> > > > > > > > > On Tue, May 11, 2021 at 11:57 AM John Halley Gotway
via
>> RT <
>> > > > > > > > > met_help at ucar.edu> wrote:
>> > > > > > > > >
>> > > > > > > > > > Yan,
>> > > > > > > > > >
>> > > > > > > > > > George and I tried to take a look at this, but
couldn't
>> > find
>> > > > the
>> > > > > > > > > > data.
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210510214154
>> > > > > > > > > >
>> > > > > > > > > > I checked on Hera and WCOSS but couldn't find the
data
>> > > there. I
>> > > > > > > > > > should be
>> > > > > > > > > > able to log on to jet, but when I try, it boots
me off:
>> > > > > > > > > >
>> > > > > > > > > > Authentication failed.
>> > > > > > > > > > Connection to jet-rsa.boulder.rdhpcs.noaa.gov
closed.
>> > > > > > > > > >
>> > > > > > > > > > Can you confirm that you're running on jet? And
any
>> idea of
>> > > jet
>> > > > > is
>> > > > > > > > > > down
>> > > > > > > > > > today or something?
>> > > > > > > > > >
>> > > > > > > > > > Thanks,
>> > > > > > > > > > John
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > On Tue, May 11, 2021 at 7:46 AM Yan Jin - NOAA
Affiliate
>> > via
>> > > > RT <
>> > > > > > > > > > met_help at ucar.edu> wrote:
>> > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > > <URL:
>> > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846
>> > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > > Hi John,
>> > > > > > > > > > >
>> > > > > > > > > > > Thanks so much!
>> > > > > > > > > > >
>> > > > > > > > > > > Best,
>> > > > > > > > > > > Yan
>> > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > > On Tue, May 11, 2021 at 9:44 AM John Halley
Gotway via
>> > RT <
>> > > > > > > > > > > met_help at ucar.edu>
>> > > > > > > > > > > wrote:
>> > > > > > > > > > >
>> > > > > > > > > > > > Hi Yan,
>> > > > > > > > > > > >
>> > > > > > > > > > > > I'm going to assign this ticket to George and
ask
>> him
>> > to
>> > > > > take a
>> > > > > > > > > > > > look
>> > > > > > > > > > and
>> > > > > > > > > > > > debug this runtime error.
>> > > > > > > > > > > >
>> > > > > > > > > > > > Thanks,
>> > > > > > > > > > > > John HG
>> > > > > > > > > > > >
>> > > > > > > > > > > > On Mon, May 10, 2021 at 3:49 PM Yan Jin -
NOAA
>> > Affiliate
>> > > > via
>> > > > > > RT <
>> > > > > > > > > > > > met_help at ucar.edu> wrote:
>> > > > > > > > > > > >
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > Mon May 10 15:49:51 2021: Request 99846 was
acted
>> > upon.
>> > > > > > > > > > > > > Transaction: Ticket created by
yan.jin at noaa.gov
>> > > > > > > > > > > > >        Queue: met_help
>> > > > > > > > > > > > >      Subject: ask a question about METplus
config
>> of
>> > TC
>> > > > > > > > > > > > >        Owner: Nobody
>> > > > > > > > > > > > >   Requestors: yan.jin at noaa.gov
>> > > > > > > > > > > > >       Status: new
>> > > > > > > > > > > > >  Ticket <URL:
>> > > > > > > > > >
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846
>> > > > > > > > > > > >
>> > > > > > > > > > > > >
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > Hi,
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > This is Yan, again. I ran the METplus
config file
>> > after
>> > > > our
>> > > > > > > > > > discussion,
>> > > > > > > > > > > > > however, it seems that the tc_pairs tcst
file
>> wasn't
>> > > > > > generated.
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > Here is the script in my directory on Jet:
>> > > > > > > > > > > > > I set TC_PAIRS_STORM_ID = AL292020, so the
code
>> can
>> > > find
>> > > > > the
>> > > > > > > > > > > > > input
>> > > > > > > > > > > adeck
>> > > > > > > > > > > > > and bdeck data correctly
>> > > > > > > > > > > > > *----
>> > > > > > > > > > > > >
>> > > > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test.conf
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > The output directory is
>> > > > > > > > > > > > >
>> > > > > > > >
>> > >
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs
>> > > > > > > > > > > > > and the tc_pairs data:
al292020_pairs.dat.tcst
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > log file
>> > > > > > > > > > > > >
>> > > > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210510214154
>> > > > > > > > > > > > > *-----
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > The error starts:[image: image.png]
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > Then I checked al292020_pairs.dat.tcst, and
it is
>> > > empty.
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > I wonder if there is anything I missed?
>> > > > > > > > > > > > >
>> > > > > > > > > > > > > Many thanks.
>> > > > > > > > > > > > > Best,
>> > > > > > > > > > > > > Yan
>> > > > > > > > > > > > >
>> > > > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > >
>> > > > > > > >
>> > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > > >
>> > > > > >
>> > > > > > --
>> > > > > > George McCabe - Software Engineer III
>> > > > > > National Center for Atmospheric Research
>> > > > > > Research Applications Laboratory
>> > > > > > 303-497-2768
>> > > > > > ---
>> > > > > > My working day may not be your working day. Please do not
feel
>> > > obliged
>> > > > to
>> > > > > > reply to this email outside of your normal working hours.
>> > > > > >
>> > > > > >
>> > > > >
>> > > > >
>> > > >
>> > > > --
>> > > > George McCabe - Software Engineer III
>> > > > National Center for Atmospheric Research
>> > > > Research Applications Laboratory
>> > > > 303-497-2768
>> > > > ---
>> > > > My working day may not be your working day. Please do not
feel
>> obliged
>> > to
>> > > > reply to this email outside of your normal working hours.
>> > > >
>> > > >
>> > >
>> > >
>> >
>> > --
>> > George McCabe - Software Engineer III
>> > National Center for Atmospheric Research
>> > Research Applications Laboratory
>> > 303-497-2768
>> > ---
>> > My working day may not be your working day. Please do not feel
obliged
>> to
>> > reply to this email outside of your normal working hours.
>> >
>> >
>>
>>
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>


--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.

------------------------------------------------
Subject: ask a question about METplus config of TC
From: George McCabe
Time: Thu May 20 09:59:10 2021

Hi Yan,

I was able to change the line colors in the images. The images can be
found
on Jet here:

/lfs1/HFIP/dtc-hurr/George.Mccabe/eta-out-colors/tcmpr_plots


This is the command I used to run:

run_metplus.py
/lfs1/HFIP/dtc-hurr/George.Mccabe/TCMPRPlotter_Eta_colors.conf
config.OUTPUT_BASE=/lfs1/HFIP/dtc-hurr/George.Mccabe/eta-out-colors


I copied the default tcmpr file from
/contrib/met/10.0.0/share/met/Rscripts/include/plot_tcmpr_config_default.R
to my directory and renamed it to plot_tcmpr_overrides.R

Then I modified the METplus config to point to that file instead of
the
file in the METplus repository:

TCMPR_PLOTTER_CONFIG_FILE =
/lfs1/HFIP/dtc-hurr/George.Mccabe/plot_tcmpr_overrides.R


The previous tcmpr parameter file changes the image resolution, so I
had to
set that in the new file to prevent the image from being very large.
Then I
changed the value of the default color list:

img_res = 72;
default_color_list  = c("blue", "green", "purple", "orange");

This variable is used to set other variables for each plot type, so
you can
modify those variables if you desire different colors for different
plots.
You only need to supply the variables you want to change from the
default
value, so your overrides file can just include the above 2 variables
to
keep the file short and clean. However, it is nice to have all of the
possible options easily available
in case you want to change other settings. How you handle this is up
to
your preference.

Let me know if you have any other questions.

Thanks,
George

On Thu, May 20, 2021 at 9:19 AM George McCabe <mccabe at ucar.edu> wrote:

> Hi Yan,
>
> I copied your config file and made some modifications. You can view
it on
> Jet here:
>
> /lfs1/HFIP/dtc-hurr/George.Mccabe/TCMPRPlotter_Eta.conf
>
> Starting in METplus 4.0.0 the different section headers like dir and
> filename_templates are no longer necessary. All configurations can
go under
> the [config] header. I removed the sections and rearranged the
variables to
> group them in a more clear way.
>
> To call TCMPRPlotter multiple times with different settings, I added
> multiple instances of the app in the PROCESS_LIST with instance
names
> defined:
>
> PROCESS_LIST = TCPairs, TCMPRPlotter(abs_mean),
> TCMPRPlotter(abs_median),TCMPRPlotter(abs_boxplot),
TCMPRPlotter(tk_mean),
> TCMPRPlotter(tk_median),TCMPRPlotter(tk_boxplot)
>
> Each instance will read all of the values set in the [config]
section,
> then it will look for a section header that matches the string
inside the
> parentheses for each instance. If the section is found, it overrides
the
> values set in the section to replace the values set in the [config]
> section. These names can be whatever you chose, but the name in the
> PROCESS_LIST must match the name of the section header.
>
> [abs_mean]
> TCMPR_PLOTTER_PREFIX = Eta_Absolute_Intensity_mean
> TCMPR_PLOTTER_TITLE = Absolute Intensity Error Mean\n Eta (Atlantic
2020)
> TCMPR_PLOTTER_DEP_VARS = ABS(AMAX_WIND-BMAX_WIND)
> TCMPR_PLOTTER_PLOT_TYPES = MEAN
>
> [abs_median]
> TCMPR_PLOTTER_PREFIX = Eta_Absolute_Intensity_median
> TCMPR_PLOTTER_TITLE = Absolute Intensity Error Median\n Eta
(Atlantic 2020)
> TCMPR_PLOTTER_DEP_VARS = ABS(AMAX_WIND-BMAX_WIND)
> TCMPR_PLOTTER_PLOT_TYPES = MEDIAN
>
> [abs_boxplot]
> TCMPR_PLOTTER_PREFIX = Eta_Absolute_Intensity_boxplot
> TCMPR_PLOTTER_TITLE = Absolute Intensity Error Boxplot\n Eta
(Atlantic
> 2020)
> TCMPR_PLOTTER_DEP_VARS = ABS(AMAX_WIND-BMAX_WIND)
> TCMPR_PLOTTER_PLOT_TYPES = BOXPLOT
>
> [tk_mean]
> TCMPR_PLOTTER_PREFIX = Eta_Track_Error_mean
> TCMPR_PLOTTER_TITLE = Track Error Mean\n Eta (Atlantic 2020)
> TCMPR_PLOTTER_DEP_VARS = TK_ERR
> TCMPR_PLOTTER_PLOT_TYPES = MEAN
>
> [tk_median]
> TCMPR_PLOTTER_PREFIX = Eta_Track_Error_median
> TCMPR_PLOTTER_TITLE = Track Error Median\n Eta (Atlantic 2020)
> TCMPR_PLOTTER_DEP_VARS = TK_ERR
> TCMPR_PLOTTER_PLOT_TYPES = MEDIAN
>
> [tk_boxplot]
> TCMPR_PLOTTER_PREFIX = Eta_Track_Error_boxplot
> TCMPR_PLOTTER_TITLE = Track Error Boxplot\n Eta (Atlantic 2020)
> TCMPR_PLOTTER_DEP_VARS = TK_ERR
> TCMPR_PLOTTER_PLOT_TYPES = BOXPLOT
>
>
> I split up the DEP_VARS and PLOT_TYPES values into instances with
one
> value of each and set the plot title and filenames appropriately for
each.
> I set the X label to "Lead Time (h)" in the [config] section so it
applies
> to every instance.
>
> TCMPR_PLOTTER_XLAB = Lead Time (h)
>
> The output images from my run is on Jet here:
>
> /lfs1/HFIP/dtc-hurr/George.Mccabe/eta-out/tcmpr_plots
>
> Eta_Absolute_Intensity_boxplot.log
> Eta_Absolute_Intensity_boxplot.png
> Eta_Absolute_Intensity_mean.png
> Eta_Absolute_Intensity_median.png
> Eta_Track_Error_boxplot.log
> Eta_Track_Error_boxplot.png
> Eta_Track_Error_mean.png
> Eta_Track_Error_median.png
>
>
> Here is the log file from the run:
> /lfs1/HFIP/dtc-hurr/George.Mccabe/eta-
out/logs/metplus.log.20210520150804
>
>
> Here is what I ran to produce the output:
>
>
>    - module load intel
>    - module load intelpython/3.6.5
>    - module load netcdf/4.6.1
>    - module load hdf5/1.10.4
>    - module load nco/4.9.1
>    - module load wgrib/1.8.1.0b
>    - module load wgrib2/2.0.8
>    - module load R/4.0.2
>    - module use /contrib/met/modulefiles
>    - module load met/10.0.0
>    - module use /contrib/met/METplus/modulefiles
>    - module load metplus/4.0.0
>    - run_metplus.py
>    /lfs1/HFIP/dtc-hurr/George.Mccabe/TCMPRPlotter_Eta.conf
>    config.OUTPUT_BASE=/lfs1/HFIP/dtc-hurr/George.Mccabe/eta-out
>
>
>
> I will look into what settings are needed to change the line colors.
Also,
> I have started making the modifications to the TCPairs wrapper to
properly
> handle multiple storms specified in the config file. I will let you
know
> when that is ready to test.
>
> Thanks,
> George
>
>
>
> On Wed, May 19, 2021 at 5:09 PM George McCabe <mccabe at ucar.edu>
wrote:
>
>> Hi Yan,
>>
>> 1) add TC name to each plot name
>> 2) modify titles to each plot, e.g.: TK_ERR* plots with name "Track
>> Error", and ABS_AMAX_WIND-BMAX_WIND_* plots with name "Absolute
Intensity
>> Error"?
>>
>> It looks like the logic in the R script uses the -prefix value to
replace
>> the filename instead of prepending it to the plot type text as you
would
>> guess it would. I was told that this script was typically called
multiple
>> times to generate the multiple plots, which would provide more
control over
>> setting the output filenames, titles, etc. The bad news is it looks
like
>> the wrapper was written to call the R script once with all of the
>> configurations specified in the METplus config file. The good news
is there
>> are some new capabilities added to METplus 4.0.0 that could help
produce
>> the results you need.
>>
>> Please review this section of the METplus User's Guide,
specifically the
>> section about specifying instance names to the processes:
>>
>>
https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#process-
list
>>
>> Alternatively, you can use the UserScript wrapper to build custom
>> commands to call the R script directly with any arguments you need
if the
>> wrapper doesn't support an option you need.
>> More information on that tool is here:
>>
https://metplus.readthedocs.io/en/latest/Users_Guide/wrappers.html#userscript
>> and here:
>>
https://metplus.readthedocs.io/en/latest/generated/met_tool_wrapper/UserScript/UserScript_run_once.html#sphx-
glr-generated-met-tool-wrapper-userscript-userscript-run-once-py
>>
>> I think using different instance names to run TCMPRPlotter multiple
times
>> with different settings is the better approach in this case. I am
wrapping
>> up work for the day, but tomorrow I can log into Jet and help you
modify
>> your config file to do this.
>>
>>
>> 3) Is it possible to modify the line colors in this METplus config
file?
>>
>> Yes, you can change the line colors through the R script's config
file.
>> You can copy and rename this file:
>>
>>
>>
https://github.com/dtcenter/MET/blob/main_v10.0/met/scripts/Rscripts/include/plot_tcmpr_config_default.R
>>
>> change the values that you need to change (look for "color_list"),
then
>> set the METplus configuration file to point to your new file (set
in
>> TCMPR_PLOTTER_CONFIG_FILE).
>>
>>
>> Let me know if you have any other questions. I will report back
tomorrow
>> when have some changes for you to test.
>>
>> Thanks,
>> George
>>
>> On Wed, May 19, 2021 at 4:00 PM Yan Jin - NOAA Affiliate via RT <
>> met_help at ucar.edu> wrote:
>>
>>>
>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846 >
>>>
>>> Hi George,
>>>
>>> I have a few questions regarding the TCMPR_PLOTTER_* parameters in
the
>>> METplus config file.
>>>
>>> Here is the METplus config file I used:
>>>
>>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/vrfy_METplus4.0/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test.conf
>>> **
>>> #  Plot_TCMPR options, if left unset, default values that are
>>> #  pre-defined in the R utility (packaged with MET) will be used.
>>> TCMPR_PLOTTER_CONFIG_FILE =
{CONFIG_DIR}/TCMPRPlotterConfig_customize
>>> TCMPR_PLOTTER_PREFIX =
>>> TCMPR_PLOTTER_TITLE =
>>> TCMPR_PLOTTER_SUBTITLE =
>>> TCMPR_PLOTTER_XLAB =
>>> TCMPR_PLOTTER_YLAB =
>>> TCMPR_PLOTTER_XLIM =
>>> TCMPR_PLOTTER_YLIM =
>>> TCMPR_PLOTTER_FILTER = -amodel HWRF,HMON
>>> # the tcst data file to be used instead of running the MET tc_stat
tool.
>>> TCMPR_PLOTTER_FILTERED_TCST_DATA_FILE =
>>> # Comma separated, no whitespace.  Default is TK_ERR (track error)
unless
>>> # otherwise indicated.
>>> TCMPR_PLOTTER_DEP_VARS = ABS(AMAX_WIND-BMAX_WIND),TK_ERR
>>> TCMPR_PLOTTER_SCATTER_X =
>>> TCMPR_PLOTTER_SCATTER_Y =
>>> TCMPR_PLOTTER_SKILL_REF =
>>> TCMPR_PLOTTER_SERIES = AMODEL
>>> TCMPR_PLOTTER_SERIES_CI = TRUE
>>> TCMPR_PLOTTER_LEGEND =
>>> TCMPR_PLOTTER_LEAD = 0,12,24,36,48,60,72,84,96,108,120
>>> # Mean and median plots. These override the plot_tcmpr.R default
of box
>>> plot.
>>> # If box plot is desired, this needs to be explicitly indicated.
>>> TCMPR_PLOTTER_PLOT_TYPES = MEAN,MEDIAN,BOXPLOT
>>> TCMPR_PLOTTER_RP_DIFF =
>>> TCMPR_PLOTTER_DEMO_YR =
>>> TCMPR_PLOTTER_HFIP_BASELINE =
>>> TCMPR_PLOTTER_FOOTNOTE_FLAG =
>>> TCMPR_PLOTTER_PLOT_CONFIG_OPTS =
>>> TCMPR_PLOTTER_SAVE_DATA =
>>>
>>> #  TCMPR FLAGS no == (don't set flag), yes == (set flag)
>>> TCMPR_PLOTTER_NO_EE = no
>>> TCMPR_PLOTTER_NO_LOG = no
>>> TCMPR_PLOTTER_SAVE = no
>>> **
>>> It generated the following 6 *.png files
>>> in
>>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/vrfy_METplus4.0/Eta_use_METplus/tcmpr_plots
>>> [image: image.png]
>>>
>>> Then I would like to modify titles, xlab text, output file names.
>>> Here are the modification I
>>> did
>>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/vrfy_METplus4.0/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test1.conf
>>>
>>> **
>>> TCMPR_PLOTTER_TITLE = Absolute Intensity Error \n Eta (Atlantic
2020)
>>> TCMPR_PLOTTER_XLAB = Lead Time (h)
>>> **
>>> The title of each plot becomes "Absolute Intensity Error Eta
(Atlantic
>>> 2020)" with double quotes.
>>>
>>> I also tried to set TCMPR_PLOTTER_PREFIX = Eta_ because I would
like to
>>> save each plot as Eta_ABS_AMAX_WIND-BMAX_WIND_*.png, and
>>> Eta_TK_ERR_*.png.
>>> However, there is only one plot Eta_.png generated.
>>>
>>> I would like to check how can I set those parameters if I would
like to
>>> 1)
>>> add TC name to each plot name; 2) modify titles to each plot,
e.g.:
>>> TK_ERR*
>>> plots with name "Track Error", and ABS_AMAX_WIND-BMAX_WIND_* plots
with
>>> name "Absolute Intensity Error"? 3) Is it possible to modify the
line
>>> colors in this METplus config file?
>>>
>>> Could you give me some suggestions?
>>>
>>> Many thanks.
>>> Best,
>>> Yan
>>>
>>> On Mon, May 17, 2021 at 10:37 AM George McCabe via RT
<met_help at ucar.edu
>>> >
>>> wrote:
>>>
>>> > Hi Yan,
>>> >
>>> > 4.1:
>>> > The consensus variable is not yet controlled by a METplus
configuration
>>> > variable explicitly, but you can set the value by using the
>>> > TC_PAIRS_MET_CONFIG_OVERRIDES value (in v4.0.0+). To set the
consensus
>>> > value to what you have set in your config file, you would set
something
>>> > like this:
>>> >
>>> > TC_PAIRS_MET_CONFIG_OVERRIDES = consensus = [ { name = "CONS";
members
>>> =
>>> > ["HWRF","GFSO","HMON"]; required = [false,false,false]; min_req
= 2; }
>>> ];
>>> >
>>> > I will make a note in the GitHub issue for improvements to the
TCPairs
>>> > wrapper (https://github.com/dtcenter/METplus/issues/749) to add
>>> support
>>> > for
>>> > consensus explicitly via something like TC_PAIRS_CONSENSUS so
that it
>>> is
>>> > easier to configure via METplus wrappers.
>>> >
>>> > 4.2:
>>> > The System Configuration section of the METplus User's Guide
covers
>>> these
>>> > variables and how they affect execution. I suggest that you read
>>> through
>>> > this section to get a full picture of how these variables work.
>>> Regarding
>>> > your specific question, these 2 sections should help:
>>> >
>>> > LOOP_BY:
>>> >
>>> >
>>>
https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#loop-
by
>>> > LOOP_ORDER:
>>> >
>>> >
>>>
https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#loop-
order
>>> >
>>> > Thanks,
>>> > George
>>> >
>>> > On Fri, May 14, 2021 at 8:19 AM Yan Jin - NOAA Affiliate via RT
<
>>> > met_help at ucar.edu> wrote:
>>> >
>>> > >
>>> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846
>
>>> > >
>>> > > Hi George,
>>> > >
>>> > > Thanks so much for your help. Finally the code can run without
>>> errors.
>>> > Here
>>> > > is the steps and results:
>>> > >
>>> > >
>>> > >
>>> >
>>> https://docs.google.com/document/d/1xRF3aiPATcX5G37ngJj-
65pl44LN7mGHo34Sf8i3zpo/edit
>>> > >
>>> > > I also put questions in the document, I also pasted here:
>>> > >
>>> > > Questions:
>>> > >
>>> > > 4.1: How can I do the consensus forecasts in METplus config
file as
>>> I did
>>> > > in tc_pairs?
>>> > >
>>> > >
>>> >
>>>
(/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_292020/TCPairsConfig_run1)
>>> > >
>>> > > //
>>> > >
>>> > > // Specify how consensus forecasts should be defined.
>>> > >
>>> > > //
>>> > >
>>> > > consensus = [
>>> > >
>>> > >  {
>>> > >
>>> > >   name = "CONS";
>>> > >
>>> > >   members = ["HWRF","GFSO","HMON"];
>>> > >
>>> > >   required = [false,false,false];
>>> > >
>>> > >   min_req = 2;
>>> > >
>>> > >  }
>>> > >
>>> > > ];
>>> > >
>>> > > 4.2: I am a little confused about
>>> > >
>>> > >       ***
>>> > >
>>> > >        LOOP_ORDER = processes
>>> > >
>>> > >        LOOP_BY = INIT
>>> > >
>>> > >       ***
>>> > >
>>> > > Could you explain these two command lines and give me some
examples?
>>> > >
>>> > > Thanks very much for your time and help on my questions.
>>> > >
>>> > > Best,
>>> > > Yan
>>> > >
>>> > > On Thu, May 13, 2021 at 3:04 PM George McCabe via RT <
>>> met_help at ucar.edu>
>>> > > wrote:
>>> > >
>>> > > > Hi Yan,
>>> > > >
>>> > > > The produtil is due to a bug in the wrapper in the logic
that
>>> reports
>>> > > when
>>> > > > an error occurs. I ran the command on Jet by hand and it
looks
>>> like it
>>> > is
>>> > > > still not finding any input data to plot. You can run the
following
>>> > > > commands (after loading all of the required modules):
>>> > > >
>>> > > > export MET_INSTALL_DIR=/contrib/met/9.1
>>> > > >
>>> > > > Rscript /contrib/met/9.1/share/met/Rscripts/plot_tcmpr.R \
>>> > > > -config
>>> > > >
>>> > > >
>>> > >
>>> >
>>> /contrib/met/METplus/METplus-
3.1.1/parm/use_cases/model_applications/tc_and_extra_tc/TCMPRPlotterConfig_customize
>>> > > > \
>>> > > > -subtitle "Your subtitle goes here" -ylab "Your y-label goes
here"
>>> \
>>> > > > -dep AMAX_WIND-BMAX_WIND,TK_ERR -legend "Your legend text
goes
>>> > here..." \
>>> > > > -plot MEAN,MEDIAN,BOXPLOT \
>>> > > > -lookin
>>> > > >
>>> > > >
>>> > >
>>> >
>>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
>>> > > > \
>>> > > >
>>> > > >
>>> > >
>>> >
>>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
>>> > > > \
>>> > > >
>>> > > >
>>> > >
>>> >
>>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
>>> > > > \
>>> > > > -outdir
>>> > > >
>>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tcmpr_plots
>>> > > >
>>> > > >
>>> > > > Here is the output from that run:
>>> > > >
>>> > > > Reading plot configuration file:
>>> > > >
>>> > > >
>>> > >
>>> >
>>> /contrib/met/METplus/METplus-
3.1.1/parm/use_cases/model_applications/tc_and_extra_tc/TCMPRPlotterConfig_customize
>>> > > > No plotting configuration file specified.
>>> > > > CALLING:  /contrib/met/9.1/bin/tc_stat -lookin
>>> > > >
>>> > > >
>>> > >
>>> >
>>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
>>> > > > -lookin
>>> > > >
>>> > > >
>>> > >
>>> >
>>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
>>> > > > -lookin
>>> > > >
>>> > > >
>>> > >
>>> >
>>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
>>> > > > -job filter -dump_row /tmp/plot_tcmpr_116966.tcst
-event_equal
>>> true
>>> > -v 3
>>> > > > DEBUG 1: [Source 1] Lookin:
>>> > > >
>>> > > >
>>> > >
>>> >
>>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
>>> > > > DEBUG 1: [Source 2] Lookin:
>>> > > >
>>> > > >
>>> > >
>>> >
>>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
>>> > > > DEBUG 1: [Source 3] Lookin:
>>> > > >
>>> > > >
>>> > >
>>> >
>>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
>>> > > > DEBUG 2:
>>> > > > DEBUG 2: Processing Job 1: -job filter -event_equal true
-dump_row
>>> > > > /tmp/plot_tcmpr_116966.tcst
>>> > > > DEBUG 3: Applying track-based event equalization logic.
>>> > > > DEBUG 3: Reading file 1 of 3:
>>> > > >
>>> > > >
>>> > >
>>> >
>>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
>>> > > > DEBUG 3: Reading file 2 of 3:
>>> > > >
>>> > > >
>>> > >
>>> >
>>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
>>> > > > DEBUG 3: Reading file 3 of 3:
>>> > > >
>>> > > >
>>> > >
>>> >
>>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
>>> > > > DEBUG 3: For track-based event equalization, identified 0
common
>>> cases
>>> > > for
>>> > > > 146 models: AC00 AEMI AEMN AP01 AP02 AP03 AP04 AP05 AP06
AP07 AP08
>>> AP09
>>> > > > AP0I AP10 AP11 AP12 AP13 AP14 AP15 AP16 AP17 AP18 AP19 AP1I
AP20
>>> AP21
>>> > > AP22
>>> > > > AP23 AP24 AP25 AP26 AP27 AP28 AP29 AP30 API CARQ CEM2 CEMI
CEMN
>>> CLP5
>>> > CMC
>>> > > > CMC2 CMCI COTC COTI CTCX DRCL DSHP EC00 EEMN EGRR EMN2 EMN3
EMN4
>>> EMNI
>>> > EMX
>>> > > > EMX2 EMXI EN01 EN02 EN03 EN04 EN05 EN06 EN07 EN08 EN09 EN0I
EN10
>>> EN11
>>> > > EN12
>>> > > > EN13 EN14 EN15 EN16 EN17 EN18 EN19 EN1I EN20 EN21 EN22 EN23
EN24
>>> EN25
>>> > ENI
>>> > > > EP01 EP02 EP03 EP04 EP05 EP06 EP07 EP08 EP09 EP0I EP10 EP11
EP12
>>> EP13
>>> > > EP14
>>> > > > EP15 EP16 EP17 EP18 EP19 EP1I EP20 EP21 EP22 EP23 EP24 EP25
EPI
>>> GFEX
>>> > GFSI
>>> > > > GFSO HMNI HMON HWF2 HWFI HWRF ICON IVCN LGEM NGX NGX2 NGXI
NVGM
>>> OCD5
>>> > RVCN
>>> > > > SHF5 SHIP TABD TABM TABS TCLP TVCA TVCE TVCN TVCX UKX UKX2
UKXI
>>> XTRP
>>> > > > DEBUG 1: Event equalization of tracks found no common cases.
>>> > > > DEBUG 1: Creating output dump file:
/tmp/plot_tcmpr_116966.tcst
>>> > > > FILTER: -job filter -line_type TCMPR -event_equal true
-dump_row
>>> > > > /tmp/plot_tcmpr_116966.tcst
>>> > > >
>>> > > > DEBUG 2: Job 1 used 0 out of 0 lines read.
>>> > > > DEBUG 3: Total lines read                 = 0
>>> > > > DEBUG 3: Total lines kept                 = 0
>>> > > > DEBUG 3: Rejected for track watch/warn    = 0
>>> > > > DEBUG 3: Rejected for init threshold      = 0
>>> > > > DEBUG 3: Rejected for init string         = 0
>>> > > > DEBUG 3: Rejected for out init mask       = 0
>>> > > > DEBUG 3: Rejected for water only          = 0
>>> > > > DEBUG 3: Rejected for rapid inten         = 0
>>> > > > DEBUG 3: Rejected for landfall            = 0
>>> > > > DEBUG 3: Rejected for amodel              = 0
>>> > > > DEBUG 3: Rejected for bmodel              = 0
>>> > > > DEBUG 3: Rejected for desc                = 0
>>> > > > DEBUG 3: Rejected for storm id            = 0
>>> > > > DEBUG 3: Rejected for basin               = 0
>>> > > > DEBUG 3: Rejected for cyclone             = 0
>>> > > > DEBUG 3: Rejected for storm name          = 0
>>> > > > DEBUG 3: Rejected for init time           = 0
>>> > > > DEBUG 3: Rejected for init hour           = 0
>>> > > > DEBUG 3: Rejected for lead time           = 0
>>> > > > DEBUG 3: Rejected for required lead times = 0
>>> > > > DEBUG 3: Rejected for valid time          = 0
>>> > > > DEBUG 3: Rejected for valid hour          = 0
>>> > > > DEBUG 3: Rejected for init mask           = 0
>>> > > > DEBUG 3: Rejected for valid mask          = 0
>>> > > > DEBUG 3: Rejected for line type           = 0
>>> > > > DEBUG 3: Rejected for numeric threshold   = 0
>>> > > > DEBUG 3: Rejected for string matching     = 0
>>> > > > DEBUG 3: Rejected for match points        = 0
>>> > > > DEBUG 3: Rejected for event equal         = 0
>>> > > > DEBUG 3: Rejected for out init mask       = 0
>>> > > > DEBUG 3: Rejected for out valid mask      = 0
>>> > > > Reading track data: /tmp/plot_tcmpr_116966.tcst
>>> > > > Error in read.table(tcst_tmp_file, header = TRUE) :
>>> > > >   no lines available in input
>>> > > > Execution halted
>>> > > >
>>> > > >
>>> > > >
>>> > > > It sounds like filter arguments may need to be provided to
produce
>>> > > TC-Stat
>>> > > > output (which is called by the R script). You can set this
in your
>>> > > METplus
>>> > > > config file with the TCMPR_PLOTTER_FILTER variable, but it
will be
>>> > > > difficult to debug since the log output is not shown. I
would
>>> recommend
>>> > > > modifying the Rscript call from this email and add the
-filter
>>> argument
>>> > > > with any filter information to pass to TCStat. I am not sure
>>> settings
>>> > > would
>>> > > > help you produce output, but I have reached out to other
METplus
>>> team
>>> > > > members for suggestions.
>>> > > >
>>> > > > Thanks,
>>> > > > George
>>> > > >
>>> > > >
>>> > > >
>>> > > > On Thu, May 13, 2021 at 12:37 PM Yan Jin - NOAA Affiliate
via RT <
>>> > > > met_help at ucar.edu> wrote:
>>> > > >
>>> > > > >
>>> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846 >
>>> > > > >
>>> > > > > Hi George,
>>> > > > >
>>> > > > > Thanks so much. Now tc_pairs works, and the output file
can be
>>> > > generated.
>>> > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
>>> > > > >
>>> > > > > Then I set "PROCESS_LIST = TCPairs, TCMPRPlotter", and I
kept
>>> most
>>> > > > > parameters that initialized with "TCMPR_PLOTTER" the same
as the
>>> > > default
>>> > > > > configuration.
>>> > > > >
>>> > > > > my METplus config file:
>>> > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test.conf
>>> > > > >
>>> > > > > then run script
>>> > > > >
>>> > > >
>>> > >
>>> >
>>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/user_config/run.sh
>>> > > > >
>>> > > > > However, the Rscript cannot plot figures.
>>> > > > > I checked the log file:
>>> > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210513182819
>>> > > > > [image: image.png]
>>> > > > >
>>> > > > > It seems that the error "NameError: name 'produtil' is not
>>> defined"?
>>> > > > >
>>> > > > > Could you give me some suggestions?
>>> > > > >
>>> > > > > Many thanks.
>>> > > > > Best,
>>> > > > > Yan
>>> > > > >
>>> > > > > On Wed, May 12, 2021 at 5:20 PM George McCabe via RT <
>>> > > met_help at ucar.edu>
>>> > > > > wrote:
>>> > > > >
>>> > > > > > Hi Yan,
>>> > > > > >
>>> > > > > > It looks like the values set for init_beg and init_end
in the
>>> > TCpairs
>>> > > > > > config file are both "20200101_000000" because the
INIT_BEG and
>>> > > > INIT_END
>>> > > > > > time is 2020. There are no adeck tracks for this time,
so all
>>> of
>>> > the
>>> > > > data
>>> > > > > > is being filtered out.
>>> > > > > >
>>> > > > > > This is a little confusing in METplus 3.1 because
INIT_BEG and
>>> > > INIT_END
>>> > > > > > control the time looping of execution (if applicable for
the
>>> use
>>> > > case)
>>> > > > > AND
>>> > > > > > the values passed to the wrapped TCPairs config file. In
>>> METplus
>>> > > 4.0.0
>>> > > > > > (which was released yesterday and should be available on
jet
>>> > soon!),
>>> > > > you
>>> > > > > > can set INIT_BEG and INIT_END to control time looping
(if
>>> desired)
>>> > > and
>>> > > > > use
>>> > > > > > TC_PAIRS_INIT_BEG and TC_PAIRS_INIT_END to specify the
values
>>> to
>>> > pass
>>> > > > to
>>> > > > > > the TCPairs config file.
>>> > > > > >
>>> > > > > > If you were using METplus 4.0.0, I would recommend
setting
>>> > something
>>> > > > > like:
>>> > > > > >
>>> > > > > >
>>> > > > > > LOOP_BY = INIT
>>> > > > > > INIT_TIME_FMT = %Y
>>> > > > > > INIT_BEG = 2020
>>> > > > > > INIT_END = 2020
>>> > > > > > INIT_INCREMENT = 21600
>>> > > > > >
>>> > > > > > TC_PAIRS_INIT_BEG = 20201030_000000
>>> > > > > > TC_PAIRS_INIT_END = 20201031_000000
>>> > > > > >
>>> > > > > >
>>> > > > > > This would run once using 2020 to find filenames and
process
>>> data
>>> > > from
>>> > > > > the
>>> > > > > > files that are initialized between 20201030_000000 and
>>> > > 20201031_000000.
>>> > > > > >
>>> > > > > > If using METplus 3.1, you will have to use the INIT_BEG
and
>>> > INIT_END
>>> > > > > values
>>> > > > > > to set the values in the wrapped TCPairs config. This is
OK for
>>> > this
>>> > > > use
>>> > > > > > case because you have LOOP_ORDER set to processes and
TCPairs
>>> only
>>> > > runs
>>> > > > > > once with that setting.
>>> > > > > >
>>> > > > > > LOOP_BY = INIT
>>> > > > > > INIT_TIME_FMT = %Y%m%d
>>> > > > > > INIT_BEG = 20201030
>>> > > > > > INIT_END = 20201031
>>> > > > > >
>>> > > > > > This configuration should run and only use tracks
initialized
>>> > between
>>> > > > > > 2020-10-30 @ 0Z and 2020-10-31 @ 0Z. If you need to
specify the
>>> > init
>>> > > > time
>>> > > > > > range with hour precision, you can do that as well. Here
is an
>>> > > example:
>>> > > > > >
>>> > > > > > LOOP_BY = INIT
>>> > > > > > INIT_TIME_FMT = %Y%m%d%H
>>> > > > > > INIT_BEG = 2020103012
>>> > > > > > INIT_END = 2020103112
>>> > > > > >
>>> > > > > > This configuration will only use tracks initialized
between
>>> > > 2020-10-30
>>> > > > @
>>> > > > > > 12Z and 2020-10-31 @ 12Z.
>>> > > > > >
>>> > > > > > You can also restrict the data used by valid time as
well by
>>> using
>>> > > > > > TC_PAIRS_VALID_BEG and TC_PAIRS_VALID_END in METplus
4.0.0 or
>>> > > VALID_BEG
>>> > > > > and
>>> > > > > > VALID_END for 3.1.
>>> > > > > >
>>> > > > > > Let me know if you are still unable to produce output
with
>>> these
>>> > > > > changes. I
>>> > > > > > was able to run tc_pairs by hand on Jet and produce a
file with
>>> > > output
>>> > > > > > tracks. You can view it here if you'd like:
>>> > > > > >
>>> > > > > > /lfs1/HFIP/dtc-
hurr/George.Mccabe/al292020_pairs.dat.tcst
>>> > > > > >
>>> > > > > > These are the commands I used:
>>> > > > > >
>>> > > > > > export BASIN="[]";
>>> > > > > > export CYCLONE="[]";
>>> > > > > > export DLAND_FILE="/contrib/met/9.1/share/met/tc_data/
>>> > > > > > dland_global_tenth_degree.nc";
>>> > > > > > export INIT_BEG="20201030_000000";
>>> > > > > > export INIT_END="20201031_000000";
>>> > > > > > export INIT_EXCLUDE="[]";
>>> > > > > > export INIT_INCLUDE="[]";
>>> > > > > > export MODEL="[]";
>>> > > > > > export STORM_ID="[\"AL292020\"]";
>>> > > > > > export STORM_NAME="[]";
>>> > > > > > export VALID_BEG="";
>>> > > > > > export VALID_END="";
>>> > > > > > /contrib/met/9.1/bin/tc_pairs -v 4 -bdeck
>>> > > > > > /mnt/lfs4/HFIP/hwrf-data/hwrf-
input/abdeck/btk/bal292020.dat
>>> -adeck
>>> > > > > >
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/data/aal292020.dat
>>> > -config
>>> > > > > >
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/use_cases/model_applications/tc_and_extra_tc/TCPairsETCConfig
>>> > > > > > -out /lfs1/HFIP/dtc-
hurr/George.Mccabe/al292020_pairs.dat
>>> > > > > >
>>> > > > > > I took the values from your METplus log file and
modified them
>>> to
>>> > > write
>>> > > > > to
>>> > > > > > my directory and use a different init time range.
>>> > > > > >
>>> > > > > > Thanks,
>>> > > > > > George
>>> > > > > >
>>> > > > > >
>>> > > > > >
>>> > > > > >
>>> > > > > > On Wed, May 12, 2021 at 2:04 PM Yan Jin - NOAA Affiliate
via
>>> RT <
>>> > > > > > met_help at ucar.edu> wrote:
>>> > > > > >
>>> > > > > > >
>>> > > > > > > <URL:
>>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846 >
>>> > > > > > >
>>> > > > > > > Hi George,
>>> > > > > > >
>>> > > > > > > Thanks very much for your instructions. I set
"PROCESS_LIST =
>>> > > > TCPairs",
>>> > > > > > and
>>> > > > > > > didn't specify models: "MODEL =". Then run the script.
>>> However,
>>> > the
>>> > > > > > output
>>> > > > > > > file is still empty. It seems that tc_pairs matched 0
adeck
>>> track
>>> > > to
>>> > > > 1
>>> > > > > > > bdeck track. I also tried to set "MODEL = HWRF, HMON",
but
>>> the
>>> > > output
>>> > > > > > file
>>> > > > > > > is empty too.
>>> > > > > > >
>>> > > > > > > Here is the log file:
>>> > > > > > >
>>> > > > > > >
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210512193152
>>> > > > > > >
>>> > > > > > > And here is the output file:
>>> > > > > > >
>>> > > > > > >
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
>>> > > > > > >
>>> > > > > > > Many thanks.
>>> > > > > > > Best,
>>> > > > > > > Yan
>>> > > > > > >
>>> > > > > > > On Wed, May 12, 2021 at 11:36 AM George McCabe via RT
<
>>> > > > > met_help at ucar.edu
>>> > > > > > >
>>> > > > > > > wrote:
>>> > > > > > >
>>> > > > > > > > Hi Yan,
>>> > > > > > > >
>>> > > > > > > > I was able to log into Jet this morning and look at
your
>>> error.
>>> > > > > > > >
>>> > > > > > > > It looks like the error in the call to Rscript is
due to
>>> there
>>> > > > being
>>> > > > > no
>>> > > > > > > > data in the input files generated by TCPairs. There
are
>>> > > > > inconsistencies
>>> > > > > > > in
>>> > > > > > > > the TCMPRPlotter wrapper that prevent the log output
from
>>> the
>>> > run
>>> > > > to
>>> > > > > be
>>> > > > > > > > seen, which would have made it more clear why the
issue was
>>> > > > > occurring.
>>> > > > > > > >
>>> > > > > > > > I would recommend changing the PROCESS_LIST value in
your
>>> > METplus
>>> > > > > > > > configuration file to only run TCPairs and iterating
on
>>> that
>>> > > until
>>> > > > > you
>>> > > > > > > get
>>> > > > > > > > output from that tool. Once you can confirm that you
have
>>> > output,
>>> > > > you
>>> > > > > > can
>>> > > > > > > > add TCMPRPlotter back to the PROCESS_LIST and test
on that
>>> > > portion.
>>> > > > > > > >
>>> > > > > > > > I will create a GitHub issue for updating the
TCMPRPlotter
>>> > > wrapper
>>> > > > to
>>> > > > > > > > produce useful logging output that will make things
more
>>> clear
>>> > > > going
>>> > > > > > > > forward.
>>> > > > > > > >
>>> > > > > > > > Let me know if you run into any other issues or need
help
>>> > > > configuring
>>> > > > > > > > TCPairs to run properly. I'm also happy to attend
another
>>> > working
>>> > > > > > session
>>> > > > > > > > if that is helpful.
>>> > > > > > > >
>>> > > > > > > > Thanks,
>>> > > > > > > > George
>>> > > > > > > >
>>> > > > > > > >
>>> > > > > > > > On Tue May 11 10:05:06 2021, yan.jin at noaa.gov wrote:
>>> > > > > > > > > Hi John,
>>> > > > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > The directories and data I mentioned in my last
email
>>> are all
>>> > > > > located
>>> > > > > > > > > in
>>> > > > > > > > > Jet.
>>> > > > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > Here is the email I got regarding the Jet
maintenance:
>>> > > > > > > > >
>>> > > > > > > > > ***
>>> > > > > > > > >
>>> > > > > > > > > *05/11/21 - Boulder Bastion Maintenance Downtime*
>>> > > > > > > > > Boulder Bastion maintenance is scheduled for
Tuesday
>>> 05/11/21
>>> > > > from
>>> > > > > > > > > 0600 ~
>>> > > > > > > > > 1000 MT. Boulder Bastion login sessions will be
dropped
>>> and
>>> > all
>>> > > > new
>>> > > > > > > > > RDHPCS
>>> > > > > > > > > logins will need to go through the Princeton
Bastion.
>>> > Logging
>>> > > In
>>> > > > > > > > > <
>>> > > > > >
>>> > https://rdhpcs-common-
docs.rdhpcs.noaa.gov/wiki/index.php/Logging_in
>>> > > >
>>> > > > > > > > >
>>> > > > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > *The next Jet Maintenance Downtime is TBD*
>>> > > > > > > > >
>>> > > > > > > > > ***
>>> > > > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > I tried to login Princeton but the connection is
closed
>>> too.
>>> > > > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > Really appreciate your time and help on this.
>>> > > > > > > > >
>>> > > > > > > > >
>>> > > > > > > > > Best,
>>> > > > > > > > >
>>> > > > > > > > > Yan
>>> > > > > > > > >
>>> > > > > > > > > On Tue, May 11, 2021 at 11:57 AM John Halley
Gotway via
>>> RT <
>>> > > > > > > > > met_help at ucar.edu> wrote:
>>> > > > > > > > >
>>> > > > > > > > > > Yan,
>>> > > > > > > > > >
>>> > > > > > > > > > George and I tried to take a look at this, but
couldn't
>>> > find
>>> > > > the
>>> > > > > > > > > > data.
>>> > > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > >
>>> > > > > > >
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210510214154
>>> > > > > > > > > >
>>> > > > > > > > > > I checked on Hera and WCOSS but couldn't find
the data
>>> > > there. I
>>> > > > > > > > > > should be
>>> > > > > > > > > > able to log on to jet, but when I try, it boots
me off:
>>> > > > > > > > > >
>>> > > > > > > > > > Authentication failed.
>>> > > > > > > > > > Connection to jet-rsa.boulder.rdhpcs.noaa.gov
closed.
>>> > > > > > > > > >
>>> > > > > > > > > > Can you confirm that you're running on jet? And
any
>>> idea of
>>> > > jet
>>> > > > > is
>>> > > > > > > > > > down
>>> > > > > > > > > > today or something?
>>> > > > > > > > > >
>>> > > > > > > > > > Thanks,
>>> > > > > > > > > > John
>>> > > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > > On Tue, May 11, 2021 at 7:46 AM Yan Jin - NOAA
>>> Affiliate
>>> > via
>>> > > > RT <
>>> > > > > > > > > > met_help at ucar.edu> wrote:
>>> > > > > > > > > >
>>> > > > > > > > > > >
>>> > > > > > > > > > > <URL:
>>> > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846
>>> > > > > >
>>> > > > > > > > > > >
>>> > > > > > > > > > > Hi John,
>>> > > > > > > > > > >
>>> > > > > > > > > > > Thanks so much!
>>> > > > > > > > > > >
>>> > > > > > > > > > > Best,
>>> > > > > > > > > > > Yan
>>> > > > > > > > > > >
>>> > > > > > > > > > >
>>> > > > > > > > > > > On Tue, May 11, 2021 at 9:44 AM John Halley
Gotway
>>> via
>>> > RT <
>>> > > > > > > > > > > met_help at ucar.edu>
>>> > > > > > > > > > > wrote:
>>> > > > > > > > > > >
>>> > > > > > > > > > > > Hi Yan,
>>> > > > > > > > > > > >
>>> > > > > > > > > > > > I'm going to assign this ticket to George
and ask
>>> him
>>> > to
>>> > > > > take a
>>> > > > > > > > > > > > look
>>> > > > > > > > > > and
>>> > > > > > > > > > > > debug this runtime error.
>>> > > > > > > > > > > >
>>> > > > > > > > > > > > Thanks,
>>> > > > > > > > > > > > John HG
>>> > > > > > > > > > > >
>>> > > > > > > > > > > > On Mon, May 10, 2021 at 3:49 PM Yan Jin -
NOAA
>>> > Affiliate
>>> > > > via
>>> > > > > > RT <
>>> > > > > > > > > > > > met_help at ucar.edu> wrote:
>>> > > > > > > > > > > >
>>> > > > > > > > > > > > >
>>> > > > > > > > > > > > > Mon May 10 15:49:51 2021: Request 99846
was acted
>>> > upon.
>>> > > > > > > > > > > > > Transaction: Ticket created by
yan.jin at noaa.gov
>>> > > > > > > > > > > > >        Queue: met_help
>>> > > > > > > > > > > > >      Subject: ask a question about METplus
>>> config of
>>> > TC
>>> > > > > > > > > > > > >        Owner: Nobody
>>> > > > > > > > > > > > >   Requestors: yan.jin at noaa.gov
>>> > > > > > > > > > > > >       Status: new
>>> > > > > > > > > > > > >  Ticket <URL:
>>> > > > > > > > > >
>>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846
>>> > > > > > > > > > > >
>>> > > > > > > > > > > > >
>>> > > > > > > > > > > > >
>>> > > > > > > > > > > > > Hi,
>>> > > > > > > > > > > > >
>>> > > > > > > > > > > > > This is Yan, again. I ran the METplus
config file
>>> > after
>>> > > > our
>>> > > > > > > > > > discussion,
>>> > > > > > > > > > > > > however, it seems that the tc_pairs tcst
file
>>> wasn't
>>> > > > > > generated.
>>> > > > > > > > > > > > >
>>> > > > > > > > > > > > > Here is the script in my directory on Jet:
>>> > > > > > > > > > > > > I set TC_PAIRS_STORM_ID = AL292020, so the
code
>>> can
>>> > > find
>>> > > > > the
>>> > > > > > > > > > > > > input
>>> > > > > > > > > > > adeck
>>> > > > > > > > > > > > > and bdeck data correctly
>>> > > > > > > > > > > > > *----
>>> > > > > > > > > > > > >
>>> > > > > > > > > > > > >
>>> > > > > > > > > > > >
>>> > > > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > >
>>> > > > > > >
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test.conf
>>> > > > > > > > > > > > >
>>> > > > > > > > > > > > > The output directory is
>>> > > > > > > > > > > > >
>>> > > > > > > >
>>> > >
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs
>>> > > > > > > > > > > > > and the tc_pairs data:
al292020_pairs.dat.tcst
>>> > > > > > > > > > > > >
>>> > > > > > > > > > > > > log file
>>> > > > > > > > > > > > >
>>> > > > > > > > > > > > >
>>> > > > > > > > > > > >
>>> > > > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > >
>>> > > > > > >
>>> > > > > >
>>> > > > >
>>> > > >
>>> > >
>>> >
>>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210510214154
>>> > > > > > > > > > > > > *-----
>>> > > > > > > > > > > > >
>>> > > > > > > > > > > > > The error starts:[image: image.png]
>>> > > > > > > > > > > > >
>>> > > > > > > > > > > > > Then I checked al292020_pairs.dat.tcst,
and it is
>>> > > empty.
>>> > > > > > > > > > > > >
>>> > > > > > > > > > > > > I wonder if there is anything I missed?
>>> > > > > > > > > > > > >
>>> > > > > > > > > > > > > Many thanks.
>>> > > > > > > > > > > > > Best,
>>> > > > > > > > > > > > > Yan
>>> > > > > > > > > > > > >
>>> > > > > > > > > > > > >
>>> > > > > > > > > > > >
>>> > > > > > > > > > > >
>>> > > > > > > > > > >
>>> > > > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > > > >
>>> > > > > > > >
>>> > > > > > > >
>>> > > > > > > >
>>> > > > > > > >
>>> > > > > > >
>>> > > > > > >
>>> > > > > >
>>> > > > > > --
>>> > > > > > George McCabe - Software Engineer III
>>> > > > > > National Center for Atmospheric Research
>>> > > > > > Research Applications Laboratory
>>> > > > > > 303-497-2768
>>> > > > > > ---
>>> > > > > > My working day may not be your working day. Please do
not feel
>>> > > obliged
>>> > > > to
>>> > > > > > reply to this email outside of your normal working
hours.
>>> > > > > >
>>> > > > > >
>>> > > > >
>>> > > > >
>>> > > >
>>> > > > --
>>> > > > George McCabe - Software Engineer III
>>> > > > National Center for Atmospheric Research
>>> > > > Research Applications Laboratory
>>> > > > 303-497-2768
>>> > > > ---
>>> > > > My working day may not be your working day. Please do not
feel
>>> obliged
>>> > to
>>> > > > reply to this email outside of your normal working hours.
>>> > > >
>>> > > >
>>> > >
>>> > >
>>> >
>>> > --
>>> > George McCabe - Software Engineer III
>>> > National Center for Atmospheric Research
>>> > Research Applications Laboratory
>>> > 303-497-2768
>>> > ---
>>> > My working day may not be your working day. Please do not feel
obliged
>>> to
>>> > reply to this email outside of your normal working hours.
>>> >
>>> >
>>>
>>>
>>
>> --
>> George McCabe - Software Engineer III
>> National Center for Atmospheric Research
>> Research Applications Laboratory
>> 303-497-2768
>> ---
>> My working day may not be your working day. Please do not feel
obliged to
>> reply to this email outside of your normal working hours.
>>
>
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>


--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.

------------------------------------------------
Subject: ask a question about METplus config of TC
From: Yan Jin - NOAA Affiliate
Time: Thu May 20 11:37:33 2021

Hi George,

Thanks so much for your help. I am reading your script now. Really
appreciate your time and patience.

Best,
Yan

On Thu, May 20, 2021 at 11:19 AM George McCabe via RT
<met_help at ucar.edu>
wrote:

> Hi Yan,
>
> I copied your config file and made some modifications. You can view
it on
> Jet here:
>
> /lfs1/HFIP/dtc-hurr/George.Mccabe/TCMPRPlotter_Eta.conf
>
> Starting in METplus 4.0.0 the different section headers like dir and
> filename_templates are no longer necessary. All configurations can
go under
> the [config] header. I removed the sections and rearranged the
variables to
> group them in a more clear way.
>
> To call TCMPRPlotter multiple times with different settings, I added
> multiple instances of the app in the PROCESS_LIST with instance
names
> defined:
>
> PROCESS_LIST = TCPairs, TCMPRPlotter(abs_mean),
> TCMPRPlotter(abs_median),TCMPRPlotter(abs_boxplot),
TCMPRPlotter(tk_mean),
> TCMPRPlotter(tk_median),TCMPRPlotter(tk_boxplot)
>
> Each instance will read all of the values set in the [config]
section, then
> it will look for a section header that matches the string inside the
> parentheses for each instance. If the section is found, it overrides
the
> values set in the section to replace the values set in the [config]
> section. These names can be whatever you chose, but the name in the
> PROCESS_LIST must match the name of the section header.
>
> [abs_mean]
> TCMPR_PLOTTER_PREFIX = Eta_Absolute_Intensity_mean
> TCMPR_PLOTTER_TITLE = Absolute Intensity Error Mean\n Eta (Atlantic
2020)
> TCMPR_PLOTTER_DEP_VARS = ABS(AMAX_WIND-BMAX_WIND)
> TCMPR_PLOTTER_PLOT_TYPES = MEAN
>
> [abs_median]
> TCMPR_PLOTTER_PREFIX = Eta_Absolute_Intensity_median
> TCMPR_PLOTTER_TITLE = Absolute Intensity Error Median\n Eta
(Atlantic 2020)
> TCMPR_PLOTTER_DEP_VARS = ABS(AMAX_WIND-BMAX_WIND)
> TCMPR_PLOTTER_PLOT_TYPES = MEDIAN
>
> [abs_boxplot]
> TCMPR_PLOTTER_PREFIX = Eta_Absolute_Intensity_boxplot
> TCMPR_PLOTTER_TITLE = Absolute Intensity Error Boxplot\n Eta
(Atlantic
> 2020)
> TCMPR_PLOTTER_DEP_VARS = ABS(AMAX_WIND-BMAX_WIND)
> TCMPR_PLOTTER_PLOT_TYPES = BOXPLOT
>
> [tk_mean]
> TCMPR_PLOTTER_PREFIX = Eta_Track_Error_mean
> TCMPR_PLOTTER_TITLE = Track Error Mean\n Eta (Atlantic 2020)
> TCMPR_PLOTTER_DEP_VARS = TK_ERR
> TCMPR_PLOTTER_PLOT_TYPES = MEAN
>
> [tk_median]
> TCMPR_PLOTTER_PREFIX = Eta_Track_Error_median
> TCMPR_PLOTTER_TITLE = Track Error Median\n Eta (Atlantic 2020)
> TCMPR_PLOTTER_DEP_VARS = TK_ERR
> TCMPR_PLOTTER_PLOT_TYPES = MEDIAN
>
> [tk_boxplot]
> TCMPR_PLOTTER_PREFIX = Eta_Track_Error_boxplot
> TCMPR_PLOTTER_TITLE = Track Error Boxplot\n Eta (Atlantic 2020)
> TCMPR_PLOTTER_DEP_VARS = TK_ERR
> TCMPR_PLOTTER_PLOT_TYPES = BOXPLOT
>
>
> I split up the DEP_VARS and PLOT_TYPES values into instances with
one value
> of each and set the plot title and filenames appropriately for each.
I set
> the X label to "Lead Time (h)" in the [config] section so it applies
to
> every instance.
>
> TCMPR_PLOTTER_XLAB = Lead Time (h)
>
> The output images from my run is on Jet here:
>
> /lfs1/HFIP/dtc-hurr/George.Mccabe/eta-out/tcmpr_plots
>
> Eta_Absolute_Intensity_boxplot.log
> Eta_Absolute_Intensity_boxplot.png
> Eta_Absolute_Intensity_mean.png
> Eta_Absolute_Intensity_median.png
> Eta_Track_Error_boxplot.log
> Eta_Track_Error_boxplot.png
> Eta_Track_Error_mean.png
> Eta_Track_Error_median.png
>
>
> Here is the log file from the run:
> /lfs1/HFIP/dtc-hurr/George.Mccabe/eta-
out/logs/metplus.log.20210520150804
>
>
> Here is what I ran to produce the output:
>
> - module load intel
> - module load intelpython/3.6.5
> - module load netcdf/4.6.1
> - module load hdf5/1.10.4
> - module load nco/4.9.1
> - module load wgrib/1.8.1.0b
> - module load wgrib2/2.0.8
> - module load R/4.0.2
> - module use /contrib/met/modulefiles
> - module load met/10.0.0
> - module use /contrib/met/METplus/modulefiles
> - module load metplus/4.0.0
> - run_metplus.py /lfs1/HFIP/dtc-
hurr/George.Mccabe/TCMPRPlotter_Eta.conf
> config.OUTPUT_BASE=/lfs1/HFIP/dtc-hurr/George.Mccabe/eta-out
>
>
> I will look into what settings are needed to change the line colors.
Also,
> I have started making the modifications to the TCPairs wrapper to
properly
> handle multiple storms specified in the config file. I will let you
know
> when that is ready to test.
>
> Thanks,
> George
>
>
>
> On Wed, May 19, 2021 at 5:09 PM George McCabe <mccabe at ucar.edu>
wrote:
>
> > Hi Yan,
> >
> > 1) add TC name to each plot name
> > 2) modify titles to each plot, e.g.: TK_ERR* plots with name
"Track
> > Error", and ABS_AMAX_WIND-BMAX_WIND_* plots with name "Absolute
Intensity
> > Error"?
> >
> > It looks like the logic in the R script uses the -prefix value to
replace
> > the filename instead of prepending it to the plot type text as you
would
> > guess it would. I was told that this script was typically called
multiple
> > times to generate the multiple plots, which would provide more
control
> over
> > setting the output filenames, titles, etc. The bad news is it
looks like
> > the wrapper was written to call the R script once with all of the
> > configurations specified in the METplus config file. The good news
is
> there
> > are some new capabilities added to METplus 4.0.0 that could help
produce
> > the results you need.
> >
> > Please review this section of the METplus User's Guide,
specifically the
> > section about specifying instance names to the processes:
> >
> >
>
https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#process-
list
> >
> > Alternatively, you can use the UserScript wrapper to build custom
> commands
> > to call the R script directly with any arguments you need if the
wrapper
> > doesn't support an option you need.
> > More information on that tool is here:
> >
>
https://metplus.readthedocs.io/en/latest/Users_Guide/wrappers.html#userscript
> > and here:
> >
>
https://metplus.readthedocs.io/en/latest/generated/met_tool_wrapper/UserScript/UserScript_run_once.html#sphx-
glr-generated-met-tool-wrapper-userscript-userscript-run-once-py
> >
> > I think using different instance names to run TCMPRPlotter
multiple times
> > with different settings is the better approach in this case. I am
> wrapping
> > up work for the day, but tomorrow I can log into Jet and help you
modify
> > your config file to do this.
> >
> >
> > 3) Is it possible to modify the line colors in this METplus config
file?
> >
> > Yes, you can change the line colors through the R script's config
file.
> > You can copy and rename this file:
> >
> >
> >
>
https://github.com/dtcenter/MET/blob/main_v10.0/met/scripts/Rscripts/include/plot_tcmpr_config_default.R
> >
> > change the values that you need to change (look for "color_list"),
then
> > set the METplus configuration file to point to your new file (set
in
> > TCMPR_PLOTTER_CONFIG_FILE).
> >
> >
> > Let me know if you have any other questions. I will report back
tomorrow
> > when have some changes for you to test.
> >
> > Thanks,
> > George
> >
> > On Wed, May 19, 2021 at 4:00 PM Yan Jin - NOAA Affiliate via RT <
> > met_help at ucar.edu> wrote:
> >
> >>
> >> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846 >
> >>
> >> Hi George,
> >>
> >> I have a few questions regarding the TCMPR_PLOTTER_* parameters
in the
> >> METplus config file.
> >>
> >> Here is the METplus config file I used:
> >>
> >>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/vrfy_METplus4.0/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test.conf
> >> **
> >> #  Plot_TCMPR options, if left unset, default values that are
> >> #  pre-defined in the R utility (packaged with MET) will be used.
> >> TCMPR_PLOTTER_CONFIG_FILE =
{CONFIG_DIR}/TCMPRPlotterConfig_customize
> >> TCMPR_PLOTTER_PREFIX =
> >> TCMPR_PLOTTER_TITLE =
> >> TCMPR_PLOTTER_SUBTITLE =
> >> TCMPR_PLOTTER_XLAB =
> >> TCMPR_PLOTTER_YLAB =
> >> TCMPR_PLOTTER_XLIM =
> >> TCMPR_PLOTTER_YLIM =
> >> TCMPR_PLOTTER_FILTER = -amodel HWRF,HMON
> >> # the tcst data file to be used instead of running the MET
tc_stat tool.
> >> TCMPR_PLOTTER_FILTERED_TCST_DATA_FILE =
> >> # Comma separated, no whitespace.  Default is TK_ERR (track
error)
> unless
> >> # otherwise indicated.
> >> TCMPR_PLOTTER_DEP_VARS = ABS(AMAX_WIND-BMAX_WIND),TK_ERR
> >> TCMPR_PLOTTER_SCATTER_X =
> >> TCMPR_PLOTTER_SCATTER_Y =
> >> TCMPR_PLOTTER_SKILL_REF =
> >> TCMPR_PLOTTER_SERIES = AMODEL
> >> TCMPR_PLOTTER_SERIES_CI = TRUE
> >> TCMPR_PLOTTER_LEGEND =
> >> TCMPR_PLOTTER_LEAD = 0,12,24,36,48,60,72,84,96,108,120
> >> # Mean and median plots. These override the plot_tcmpr.R default
of box
> >> plot.
> >> # If box plot is desired, this needs to be explicitly indicated.
> >> TCMPR_PLOTTER_PLOT_TYPES = MEAN,MEDIAN,BOXPLOT
> >> TCMPR_PLOTTER_RP_DIFF =
> >> TCMPR_PLOTTER_DEMO_YR =
> >> TCMPR_PLOTTER_HFIP_BASELINE =
> >> TCMPR_PLOTTER_FOOTNOTE_FLAG =
> >> TCMPR_PLOTTER_PLOT_CONFIG_OPTS =
> >> TCMPR_PLOTTER_SAVE_DATA =
> >>
> >> #  TCMPR FLAGS no == (don't set flag), yes == (set flag)
> >> TCMPR_PLOTTER_NO_EE = no
> >> TCMPR_PLOTTER_NO_LOG = no
> >> TCMPR_PLOTTER_SAVE = no
> >> **
> >> It generated the following 6 *.png files
> >> in
> >>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/vrfy_METplus4.0/Eta_use_METplus/tcmpr_plots
> >> [image: image.png]
> >>
> >> Then I would like to modify titles, xlab text, output file names.
> >> Here are the modification I
> >> did
> >>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/vrfy_METplus4.0/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test1.conf
> >>
> >> **
> >> TCMPR_PLOTTER_TITLE = Absolute Intensity Error \n Eta (Atlantic
2020)
> >> TCMPR_PLOTTER_XLAB = Lead Time (h)
> >> **
> >> The title of each plot becomes "Absolute Intensity Error Eta
(Atlantic
> >> 2020)" with double quotes.
> >>
> >> I also tried to set TCMPR_PLOTTER_PREFIX = Eta_ because I would
like to
> >> save each plot as Eta_ABS_AMAX_WIND-BMAX_WIND_*.png, and
> Eta_TK_ERR_*.png.
> >> However, there is only one plot Eta_.png generated.
> >>
> >> I would like to check how can I set those parameters if I would
like to
> 1)
> >> add TC name to each plot name; 2) modify titles to each plot,
e.g.:
> >> TK_ERR*
> >> plots with name "Track Error", and ABS_AMAX_WIND-BMAX_WIND_*
plots with
> >> name "Absolute Intensity Error"? 3) Is it possible to modify the
line
> >> colors in this METplus config file?
> >>
> >> Could you give me some suggestions?
> >>
> >> Many thanks.
> >> Best,
> >> Yan
> >>
> >> On Mon, May 17, 2021 at 10:37 AM George McCabe via RT <
> met_help at ucar.edu>
> >> wrote:
> >>
> >> > Hi Yan,
> >> >
> >> > 4.1:
> >> > The consensus variable is not yet controlled by a METplus
> configuration
> >> > variable explicitly, but you can set the value by using the
> >> > TC_PAIRS_MET_CONFIG_OVERRIDES value (in v4.0.0+). To set the
consensus
> >> > value to what you have set in your config file, you would set
> something
> >> > like this:
> >> >
> >> > TC_PAIRS_MET_CONFIG_OVERRIDES = consensus = [ { name = "CONS";
> members =
> >> > ["HWRF","GFSO","HMON"]; required = [false,false,false]; min_req
= 2; }
> >> ];
> >> >
> >> > I will make a note in the GitHub issue for improvements to the
TCPairs
> >> > wrapper (https://github.com/dtcenter/METplus/issues/749) to add
> support
> >> > for
> >> > consensus explicitly via something like TC_PAIRS_CONSENSUS so
that it
> is
> >> > easier to configure via METplus wrappers.
> >> >
> >> > 4.2:
> >> > The System Configuration section of the METplus User's Guide
covers
> >> these
> >> > variables and how they affect execution. I suggest that you
read
> through
> >> > this section to get a full picture of how these variables work.
> >> Regarding
> >> > your specific question, these 2 sections should help:
> >> >
> >> > LOOP_BY:
> >> >
> >> >
> >>
>
https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#loop-
by
> >> > LOOP_ORDER:
> >> >
> >> >
> >>
>
https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#loop-
order
> >> >
> >> > Thanks,
> >> > George
> >> >
> >> > On Fri, May 14, 2021 at 8:19 AM Yan Jin - NOAA Affiliate via RT
<
> >> > met_help at ucar.edu> wrote:
> >> >
> >> > >
> >> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846
>
> >> > >
> >> > > Hi George,
> >> > >
> >> > > Thanks so much for your help. Finally the code can run
without
> errors.
> >> > Here
> >> > > is the steps and results:
> >> > >
> >> > >
> >> > >
> >> >
> >>
> https://docs.google.com/document/d/1xRF3aiPATcX5G37ngJj-
65pl44LN7mGHo34Sf8i3zpo/edit
> >> > >
> >> > > I also put questions in the document, I also pasted here:
> >> > >
> >> > > Questions:
> >> > >
> >> > > 4.1: How can I do the consensus forecasts in METplus config
file as
> I
> >> did
> >> > > in tc_pairs?
> >> > >
> >> > >
> >> >
> >>
>
(/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_292020/TCPairsConfig_run1)
> >> > >
> >> > > //
> >> > >
> >> > > // Specify how consensus forecasts should be defined.
> >> > >
> >> > > //
> >> > >
> >> > > consensus = [
> >> > >
> >> > >  {
> >> > >
> >> > >   name = "CONS";
> >> > >
> >> > >   members = ["HWRF","GFSO","HMON"];
> >> > >
> >> > >   required = [false,false,false];
> >> > >
> >> > >   min_req = 2;
> >> > >
> >> > >  }
> >> > >
> >> > > ];
> >> > >
> >> > > 4.2: I am a little confused about
> >> > >
> >> > >       ***
> >> > >
> >> > >        LOOP_ORDER = processes
> >> > >
> >> > >        LOOP_BY = INIT
> >> > >
> >> > >       ***
> >> > >
> >> > > Could you explain these two command lines and give me some
examples?
> >> > >
> >> > > Thanks very much for your time and help on my questions.
> >> > >
> >> > > Best,
> >> > > Yan
> >> > >
> >> > > On Thu, May 13, 2021 at 3:04 PM George McCabe via RT <
> >> met_help at ucar.edu>
> >> > > wrote:
> >> > >
> >> > > > Hi Yan,
> >> > > >
> >> > > > The produtil is due to a bug in the wrapper in the logic
that
> >> reports
> >> > > when
> >> > > > an error occurs. I ran the command on Jet by hand and it
looks
> like
> >> it
> >> > is
> >> > > > still not finding any input data to plot. You can run the
> following
> >> > > > commands (after loading all of the required modules):
> >> > > >
> >> > > > export MET_INSTALL_DIR=/contrib/met/9.1
> >> > > >
> >> > > > Rscript /contrib/met/9.1/share/met/Rscripts/plot_tcmpr.R \
> >> > > > -config
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> /contrib/met/METplus/METplus-
3.1.1/parm/use_cases/model_applications/tc_and_extra_tc/TCMPRPlotterConfig_customize
> >> > > > \
> >> > > > -subtitle "Your subtitle goes here" -ylab "Your y-label
goes
> here" \
> >> > > > -dep AMAX_WIND-BMAX_WIND,TK_ERR -legend "Your legend text
goes
> >> > here..." \
> >> > > > -plot MEAN,MEDIAN,BOXPLOT \
> >> > > > -lookin
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> >> > > > \
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
> >> > > > \
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
> >> > > > \
> >> > > > -outdir
> >> > > >
> >>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tcmpr_plots
> >> > > >
> >> > > >
> >> > > > Here is the output from that run:
> >> > > >
> >> > > > Reading plot configuration file:
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
> /contrib/met/METplus/METplus-
3.1.1/parm/use_cases/model_applications/tc_and_extra_tc/TCMPRPlotterConfig_customize
> >> > > > No plotting configuration file specified.
> >> > > > CALLING:  /contrib/met/9.1/bin/tc_stat -lookin
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> >> > > > -lookin
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
> >> > > > -lookin
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
> >> > > > -job filter -dump_row /tmp/plot_tcmpr_116966.tcst
-event_equal
> true
> >> > -v 3
> >> > > > DEBUG 1: [Source 1] Lookin:
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> >> > > > DEBUG 1: [Source 2] Lookin:
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
> >> > > > DEBUG 1: [Source 3] Lookin:
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
> >> > > > DEBUG 2:
> >> > > > DEBUG 2: Processing Job 1: -job filter -event_equal true
-dump_row
> >> > > > /tmp/plot_tcmpr_116966.tcst
> >> > > > DEBUG 3: Applying track-based event equalization logic.
> >> > > > DEBUG 3: Reading file 1 of 3:
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> >> > > > DEBUG 3: Reading file 2 of 3:
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
> >> > > > DEBUG 3: Reading file 3 of 3:
> >> > > >
> >> > > >
> >> > >
> >> >
> >>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
> >> > > > DEBUG 3: For track-based event equalization, identified 0
common
> >> cases
> >> > > for
> >> > > > 146 models: AC00 AEMI AEMN AP01 AP02 AP03 AP04 AP05 AP06
AP07 AP08
> >> AP09
> >> > > > AP0I AP10 AP11 AP12 AP13 AP14 AP15 AP16 AP17 AP18 AP19 AP1I
AP20
> >> AP21
> >> > > AP22
> >> > > > AP23 AP24 AP25 AP26 AP27 AP28 AP29 AP30 API CARQ CEM2 CEMI
CEMN
> CLP5
> >> > CMC
> >> > > > CMC2 CMCI COTC COTI CTCX DRCL DSHP EC00 EEMN EGRR EMN2 EMN3
EMN4
> >> EMNI
> >> > EMX
> >> > > > EMX2 EMXI EN01 EN02 EN03 EN04 EN05 EN06 EN07 EN08 EN09 EN0I
EN10
> >> EN11
> >> > > EN12
> >> > > > EN13 EN14 EN15 EN16 EN17 EN18 EN19 EN1I EN20 EN21 EN22 EN23
EN24
> >> EN25
> >> > ENI
> >> > > > EP01 EP02 EP03 EP04 EP05 EP06 EP07 EP08 EP09 EP0I EP10 EP11
EP12
> >> EP13
> >> > > EP14
> >> > > > EP15 EP16 EP17 EP18 EP19 EP1I EP20 EP21 EP22 EP23 EP24 EP25
EPI
> GFEX
> >> > GFSI
> >> > > > GFSO HMNI HMON HWF2 HWFI HWRF ICON IVCN LGEM NGX NGX2 NGXI
NVGM
> OCD5
> >> > RVCN
> >> > > > SHF5 SHIP TABD TABM TABS TCLP TVCA TVCE TVCN TVCX UKX UKX2
UKXI
> XTRP
> >> > > > DEBUG 1: Event equalization of tracks found no common
cases.
> >> > > > DEBUG 1: Creating output dump file:
/tmp/plot_tcmpr_116966.tcst
> >> > > > FILTER: -job filter -line_type TCMPR -event_equal true
-dump_row
> >> > > > /tmp/plot_tcmpr_116966.tcst
> >> > > >
> >> > > > DEBUG 2: Job 1 used 0 out of 0 lines read.
> >> > > > DEBUG 3: Total lines read                 = 0
> >> > > > DEBUG 3: Total lines kept                 = 0
> >> > > > DEBUG 3: Rejected for track watch/warn    = 0
> >> > > > DEBUG 3: Rejected for init threshold      = 0
> >> > > > DEBUG 3: Rejected for init string         = 0
> >> > > > DEBUG 3: Rejected for out init mask       = 0
> >> > > > DEBUG 3: Rejected for water only          = 0
> >> > > > DEBUG 3: Rejected for rapid inten         = 0
> >> > > > DEBUG 3: Rejected for landfall            = 0
> >> > > > DEBUG 3: Rejected for amodel              = 0
> >> > > > DEBUG 3: Rejected for bmodel              = 0
> >> > > > DEBUG 3: Rejected for desc                = 0
> >> > > > DEBUG 3: Rejected for storm id            = 0
> >> > > > DEBUG 3: Rejected for basin               = 0
> >> > > > DEBUG 3: Rejected for cyclone             = 0
> >> > > > DEBUG 3: Rejected for storm name          = 0
> >> > > > DEBUG 3: Rejected for init time           = 0
> >> > > > DEBUG 3: Rejected for init hour           = 0
> >> > > > DEBUG 3: Rejected for lead time           = 0
> >> > > > DEBUG 3: Rejected for required lead times = 0
> >> > > > DEBUG 3: Rejected for valid time          = 0
> >> > > > DEBUG 3: Rejected for valid hour          = 0
> >> > > > DEBUG 3: Rejected for init mask           = 0
> >> > > > DEBUG 3: Rejected for valid mask          = 0
> >> > > > DEBUG 3: Rejected for line type           = 0
> >> > > > DEBUG 3: Rejected for numeric threshold   = 0
> >> > > > DEBUG 3: Rejected for string matching     = 0
> >> > > > DEBUG 3: Rejected for match points        = 0
> >> > > > DEBUG 3: Rejected for event equal         = 0
> >> > > > DEBUG 3: Rejected for out init mask       = 0
> >> > > > DEBUG 3: Rejected for out valid mask      = 0
> >> > > > Reading track data: /tmp/plot_tcmpr_116966.tcst
> >> > > > Error in read.table(tcst_tmp_file, header = TRUE) :
> >> > > >   no lines available in input
> >> > > > Execution halted
> >> > > >
> >> > > >
> >> > > >
> >> > > > It sounds like filter arguments may need to be provided to
produce
> >> > > TC-Stat
> >> > > > output (which is called by the R script). You can set this
in your
> >> > > METplus
> >> > > > config file with the TCMPR_PLOTTER_FILTER variable, but it
will be
> >> > > > difficult to debug since the log output is not shown. I
would
> >> recommend
> >> > > > modifying the Rscript call from this email and add the
-filter
> >> argument
> >> > > > with any filter information to pass to TCStat. I am not
sure
> >> settings
> >> > > would
> >> > > > help you produce output, but I have reached out to other
METplus
> >> team
> >> > > > members for suggestions.
> >> > > >
> >> > > > Thanks,
> >> > > > George
> >> > > >
> >> > > >
> >> > > >
> >> > > > On Thu, May 13, 2021 at 12:37 PM Yan Jin - NOAA Affiliate
via RT <
> >> > > > met_help at ucar.edu> wrote:
> >> > > >
> >> > > > >
> >> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846 >
> >> > > > >
> >> > > > > Hi George,
> >> > > > >
> >> > > > > Thanks so much. Now tc_pairs works, and the output file
can be
> >> > > generated.
> >> > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> >> > > > >
> >> > > > > Then I set "PROCESS_LIST = TCPairs, TCMPRPlotter", and I
kept
> most
> >> > > > > parameters that initialized with "TCMPR_PLOTTER" the same
as the
> >> > > default
> >> > > > > configuration.
> >> > > > >
> >> > > > > my METplus config file:
> >> > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test.conf
> >> > > > >
> >> > > > > then run script
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/user_config/run.sh
> >> > > > >
> >> > > > > However, the Rscript cannot plot figures.
> >> > > > > I checked the log file:
> >> > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210513182819
> >> > > > > [image: image.png]
> >> > > > >
> >> > > > > It seems that the error "NameError: name 'produtil' is
not
> >> defined"?
> >> > > > >
> >> > > > > Could you give me some suggestions?
> >> > > > >
> >> > > > > Many thanks.
> >> > > > > Best,
> >> > > > > Yan
> >> > > > >
> >> > > > > On Wed, May 12, 2021 at 5:20 PM George McCabe via RT <
> >> > > met_help at ucar.edu>
> >> > > > > wrote:
> >> > > > >
> >> > > > > > Hi Yan,
> >> > > > > >
> >> > > > > > It looks like the values set for init_beg and init_end
in the
> >> > TCpairs
> >> > > > > > config file are both "20200101_000000" because the
INIT_BEG
> and
> >> > > > INIT_END
> >> > > > > > time is 2020. There are no adeck tracks for this time,
so all
> of
> >> > the
> >> > > > data
> >> > > > > > is being filtered out.
> >> > > > > >
> >> > > > > > This is a little confusing in METplus 3.1 because
INIT_BEG and
> >> > > INIT_END
> >> > > > > > control the time looping of execution (if applicable
for the
> use
> >> > > case)
> >> > > > > AND
> >> > > > > > the values passed to the wrapped TCPairs config file.
In
> METplus
> >> > > 4.0.0
> >> > > > > > (which was released yesterday and should be available
on jet
> >> > soon!),
> >> > > > you
> >> > > > > > can set INIT_BEG and INIT_END to control time looping
(if
> >> desired)
> >> > > and
> >> > > > > use
> >> > > > > > TC_PAIRS_INIT_BEG and TC_PAIRS_INIT_END to specify the
values
> to
> >> > pass
> >> > > > to
> >> > > > > > the TCPairs config file.
> >> > > > > >
> >> > > > > > If you were using METplus 4.0.0, I would recommend
setting
> >> > something
> >> > > > > like:
> >> > > > > >
> >> > > > > >
> >> > > > > > LOOP_BY = INIT
> >> > > > > > INIT_TIME_FMT = %Y
> >> > > > > > INIT_BEG = 2020
> >> > > > > > INIT_END = 2020
> >> > > > > > INIT_INCREMENT = 21600
> >> > > > > >
> >> > > > > > TC_PAIRS_INIT_BEG = 20201030_000000
> >> > > > > > TC_PAIRS_INIT_END = 20201031_000000
> >> > > > > >
> >> > > > > >
> >> > > > > > This would run once using 2020 to find filenames and
process
> >> data
> >> > > from
> >> > > > > the
> >> > > > > > files that are initialized between 20201030_000000 and
> >> > > 20201031_000000.
> >> > > > > >
> >> > > > > > If using METplus 3.1, you will have to use the INIT_BEG
and
> >> > INIT_END
> >> > > > > values
> >> > > > > > to set the values in the wrapped TCPairs config. This
is OK
> for
> >> > this
> >> > > > use
> >> > > > > > case because you have LOOP_ORDER set to processes and
TCPairs
> >> only
> >> > > runs
> >> > > > > > once with that setting.
> >> > > > > >
> >> > > > > > LOOP_BY = INIT
> >> > > > > > INIT_TIME_FMT = %Y%m%d
> >> > > > > > INIT_BEG = 20201030
> >> > > > > > INIT_END = 20201031
> >> > > > > >
> >> > > > > > This configuration should run and only use tracks
initialized
> >> > between
> >> > > > > > 2020-10-30 @ 0Z and 2020-10-31 @ 0Z. If you need to
specify
> the
> >> > init
> >> > > > time
> >> > > > > > range with hour precision, you can do that as well.
Here is an
> >> > > example:
> >> > > > > >
> >> > > > > > LOOP_BY = INIT
> >> > > > > > INIT_TIME_FMT = %Y%m%d%H
> >> > > > > > INIT_BEG = 2020103012
> >> > > > > > INIT_END = 2020103112
> >> > > > > >
> >> > > > > > This configuration will only use tracks initialized
between
> >> > > 2020-10-30
> >> > > > @
> >> > > > > > 12Z and 2020-10-31 @ 12Z.
> >> > > > > >
> >> > > > > > You can also restrict the data used by valid time as
well by
> >> using
> >> > > > > > TC_PAIRS_VALID_BEG and TC_PAIRS_VALID_END in METplus
4.0.0 or
> >> > > VALID_BEG
> >> > > > > and
> >> > > > > > VALID_END for 3.1.
> >> > > > > >
> >> > > > > > Let me know if you are still unable to produce output
with
> these
> >> > > > > changes. I
> >> > > > > > was able to run tc_pairs by hand on Jet and produce a
file
> with
> >> > > output
> >> > > > > > tracks. You can view it here if you'd like:
> >> > > > > >
> >> > > > > > /lfs1/HFIP/dtc-
hurr/George.Mccabe/al292020_pairs.dat.tcst
> >> > > > > >
> >> > > > > > These are the commands I used:
> >> > > > > >
> >> > > > > > export BASIN="[]";
> >> > > > > > export CYCLONE="[]";
> >> > > > > > export DLAND_FILE="/contrib/met/9.1/share/met/tc_data/
> >> > > > > > dland_global_tenth_degree.nc";
> >> > > > > > export INIT_BEG="20201030_000000";
> >> > > > > > export INIT_END="20201031_000000";
> >> > > > > > export INIT_EXCLUDE="[]";
> >> > > > > > export INIT_INCLUDE="[]";
> >> > > > > > export MODEL="[]";
> >> > > > > > export STORM_ID="[\"AL292020\"]";
> >> > > > > > export STORM_NAME="[]";
> >> > > > > > export VALID_BEG="";
> >> > > > > > export VALID_END="";
> >> > > > > > /contrib/met/9.1/bin/tc_pairs -v 4 -bdeck
> >> > > > > > /mnt/lfs4/HFIP/hwrf-data/hwrf-
input/abdeck/btk/bal292020.dat
> >> -adeck
> >> > > > > >
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/data/aal292020.dat
> >> > -config
> >> > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/use_cases/model_applications/tc_and_extra_tc/TCPairsETCConfig
> >> > > > > > -out /lfs1/HFIP/dtc-
hurr/George.Mccabe/al292020_pairs.dat
> >> > > > > >
> >> > > > > > I took the values from your METplus log file and
modified them
> >> to
> >> > > write
> >> > > > > to
> >> > > > > > my directory and use a different init time range.
> >> > > > > >
> >> > > > > > Thanks,
> >> > > > > > George
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > > On Wed, May 12, 2021 at 2:04 PM Yan Jin - NOAA
Affiliate via
> RT
> >> <
> >> > > > > > met_help at ucar.edu> wrote:
> >> > > > > >
> >> > > > > > >
> >> > > > > > > <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846
> >> >
> >> > > > > > >
> >> > > > > > > Hi George,
> >> > > > > > >
> >> > > > > > > Thanks very much for your instructions. I set
"PROCESS_LIST
> =
> >> > > > TCPairs",
> >> > > > > > and
> >> > > > > > > didn't specify models: "MODEL =". Then run the
script.
> >> However,
> >> > the
> >> > > > > > output
> >> > > > > > > file is still empty. It seems that tc_pairs matched 0
adeck
> >> track
> >> > > to
> >> > > > 1
> >> > > > > > > bdeck track. I also tried to set "MODEL = HWRF,
HMON", but
> the
> >> > > output
> >> > > > > > file
> >> > > > > > > is empty too.
> >> > > > > > >
> >> > > > > > > Here is the log file:
> >> > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210512193152
> >> > > > > > >
> >> > > > > > > And here is the output file:
> >> > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> >> > > > > > >
> >> > > > > > > Many thanks.
> >> > > > > > > Best,
> >> > > > > > > Yan
> >> > > > > > >
> >> > > > > > > On Wed, May 12, 2021 at 11:36 AM George McCabe via RT
<
> >> > > > > met_help at ucar.edu
> >> > > > > > >
> >> > > > > > > wrote:
> >> > > > > > >
> >> > > > > > > > Hi Yan,
> >> > > > > > > >
> >> > > > > > > > I was able to log into Jet this morning and look at
your
> >> error.
> >> > > > > > > >
> >> > > > > > > > It looks like the error in the call to Rscript is
due to
> >> there
> >> > > > being
> >> > > > > no
> >> > > > > > > > data in the input files generated by TCPairs. There
are
> >> > > > > inconsistencies
> >> > > > > > > in
> >> > > > > > > > the TCMPRPlotter wrapper that prevent the log
output from
> >> the
> >> > run
> >> > > > to
> >> > > > > be
> >> > > > > > > > seen, which would have made it more clear why the
issue
> was
> >> > > > > occurring.
> >> > > > > > > >
> >> > > > > > > > I would recommend changing the PROCESS_LIST value
in your
> >> > METplus
> >> > > > > > > > configuration file to only run TCPairs and
iterating on
> that
> >> > > until
> >> > > > > you
> >> > > > > > > get
> >> > > > > > > > output from that tool. Once you can confirm that
you have
> >> > output,
> >> > > > you
> >> > > > > > can
> >> > > > > > > > add TCMPRPlotter back to the PROCESS_LIST and test
on that
> >> > > portion.
> >> > > > > > > >
> >> > > > > > > > I will create a GitHub issue for updating the
TCMPRPlotter
> >> > > wrapper
> >> > > > to
> >> > > > > > > > produce useful logging output that will make things
more
> >> clear
> >> > > > going
> >> > > > > > > > forward.
> >> > > > > > > >
> >> > > > > > > > Let me know if you run into any other issues or
need help
> >> > > > configuring
> >> > > > > > > > TCPairs to run properly. I'm also happy to attend
another
> >> > working
> >> > > > > > session
> >> > > > > > > > if that is helpful.
> >> > > > > > > >
> >> > > > > > > > Thanks,
> >> > > > > > > > George
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > > > On Tue May 11 10:05:06 2021, yan.jin at noaa.gov
wrote:
> >> > > > > > > > > Hi John,
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > > The directories and data I mentioned in my last
email
> are
> >> all
> >> > > > > located
> >> > > > > > > > > in
> >> > > > > > > > > Jet.
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > > Here is the email I got regarding the Jet
maintenance:
> >> > > > > > > > >
> >> > > > > > > > > ***
> >> > > > > > > > >
> >> > > > > > > > > *05/11/21 - Boulder Bastion Maintenance Downtime*
> >> > > > > > > > > Boulder Bastion maintenance is scheduled for
Tuesday
> >> 05/11/21
> >> > > > from
> >> > > > > > > > > 0600 ~
> >> > > > > > > > > 1000 MT. Boulder Bastion login sessions will be
dropped
> >> and
> >> > all
> >> > > > new
> >> > > > > > > > > RDHPCS
> >> > > > > > > > > logins will need to go through the Princeton
Bastion.
> >> > Logging
> >> > > In
> >> > > > > > > > > <
> >> > > > > >
> >> > https://rdhpcs-common-
docs.rdhpcs.noaa.gov/wiki/index.php/Logging_in
> >> > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > > *The next Jet Maintenance Downtime is TBD*
> >> > > > > > > > >
> >> > > > > > > > > ***
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > > I tried to login Princeton but the connection is
closed
> >> too.
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > > Really appreciate your time and help on this.
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > > Best,
> >> > > > > > > > >
> >> > > > > > > > > Yan
> >> > > > > > > > >
> >> > > > > > > > > On Tue, May 11, 2021 at 11:57 AM John Halley
Gotway via
> >> RT <
> >> > > > > > > > > met_help at ucar.edu> wrote:
> >> > > > > > > > >
> >> > > > > > > > > > Yan,
> >> > > > > > > > > >
> >> > > > > > > > > > George and I tried to take a look at this, but
> couldn't
> >> > find
> >> > > > the
> >> > > > > > > > > > data.
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210510214154
> >> > > > > > > > > >
> >> > > > > > > > > > I checked on Hera and WCOSS but couldn't find
the data
> >> > > there. I
> >> > > > > > > > > > should be
> >> > > > > > > > > > able to log on to jet, but when I try, it boots
me
> off:
> >> > > > > > > > > >
> >> > > > > > > > > > Authentication failed.
> >> > > > > > > > > > Connection to jet-rsa.boulder.rdhpcs.noaa.gov
closed.
> >> > > > > > > > > >
> >> > > > > > > > > > Can you confirm that you're running on jet? And
any
> >> idea of
> >> > > jet
> >> > > > > is
> >> > > > > > > > > > down
> >> > > > > > > > > > today or something?
> >> > > > > > > > > >
> >> > > > > > > > > > Thanks,
> >> > > > > > > > > > John
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > On Tue, May 11, 2021 at 7:46 AM Yan Jin - NOAA
> Affiliate
> >> > via
> >> > > > RT <
> >> > > > > > > > > > met_help at ucar.edu> wrote:
> >> > > > > > > > > >
> >> > > > > > > > > > >
> >> > > > > > > > > > > <URL:
> >> > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846
> >> > > > > >
> >> > > > > > > > > > >
> >> > > > > > > > > > > Hi John,
> >> > > > > > > > > > >
> >> > > > > > > > > > > Thanks so much!
> >> > > > > > > > > > >
> >> > > > > > > > > > > Best,
> >> > > > > > > > > > > Yan
> >> > > > > > > > > > >
> >> > > > > > > > > > >
> >> > > > > > > > > > > On Tue, May 11, 2021 at 9:44 AM John Halley
Gotway
> via
> >> > RT <
> >> > > > > > > > > > > met_help at ucar.edu>
> >> > > > > > > > > > > wrote:
> >> > > > > > > > > > >
> >> > > > > > > > > > > > Hi Yan,
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > I'm going to assign this ticket to George
and ask
> >> him
> >> > to
> >> > > > > take a
> >> > > > > > > > > > > > look
> >> > > > > > > > > > and
> >> > > > > > > > > > > > debug this runtime error.
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > Thanks,
> >> > > > > > > > > > > > John HG
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > On Mon, May 10, 2021 at 3:49 PM Yan Jin -
NOAA
> >> > Affiliate
> >> > > > via
> >> > > > > > RT <
> >> > > > > > > > > > > > met_help at ucar.edu> wrote:
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > > Mon May 10 15:49:51 2021: Request 99846
was
> acted
> >> > upon.
> >> > > > > > > > > > > > > Transaction: Ticket created by
yan.jin at noaa.gov
> >> > > > > > > > > > > > >        Queue: met_help
> >> > > > > > > > > > > > >      Subject: ask a question about
METplus
> config
> >> of
> >> > TC
> >> > > > > > > > > > > > >        Owner: Nobody
> >> > > > > > > > > > > > >   Requestors: yan.jin at noaa.gov
> >> > > > > > > > > > > > >       Status: new
> >> > > > > > > > > > > > >  Ticket <URL:
> >> > > > > > > > > >
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > > Hi,
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > > This is Yan, again. I ran the METplus
config
> file
> >> > after
> >> > > > our
> >> > > > > > > > > > discussion,
> >> > > > > > > > > > > > > however, it seems that the tc_pairs tcst
file
> >> wasn't
> >> > > > > > generated.
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > > Here is the script in my directory on
Jet:
> >> > > > > > > > > > > > > I set TC_PAIRS_STORM_ID = AL292020, so
the code
> >> can
> >> > > find
> >> > > > > the
> >> > > > > > > > > > > > > input
> >> > > > > > > > > > > adeck
> >> > > > > > > > > > > > > and bdeck data correctly
> >> > > > > > > > > > > > > *----
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > >
> >> > > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test.conf
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > > The output directory is
> >> > > > > > > > > > > > >
> >> > > > > > > >
> >> > >
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs
> >> > > > > > > > > > > > > and the tc_pairs data:
al292020_pairs.dat.tcst
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > > log file
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > >
> >> > > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210510214154
> >> > > > > > > > > > > > > *-----
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > > The error starts:[image: image.png]
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > > Then I checked al292020_pairs.dat.tcst,
and it
> is
> >> > > empty.
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > > I wonder if there is anything I missed?
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > > Many thanks.
> >> > > > > > > > > > > > > Best,
> >> > > > > > > > > > > > > Yan
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > > >
> >> > > > > > > > > > > >
> >> > > > > > > > > > > >
> >> > > > > > > > > > >
> >> > > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > > > --
> >> > > > > > George McCabe - Software Engineer III
> >> > > > > > National Center for Atmospheric Research
> >> > > > > > Research Applications Laboratory
> >> > > > > > 303-497-2768
> >> > > > > > ---
> >> > > > > > My working day may not be your working day. Please do
not feel
> >> > > obliged
> >> > > > to
> >> > > > > > reply to this email outside of your normal working
hours.
> >> > > > > >
> >> > > > > >
> >> > > > >
> >> > > > >
> >> > > >
> >> > > > --
> >> > > > George McCabe - Software Engineer III
> >> > > > National Center for Atmospheric Research
> >> > > > Research Applications Laboratory
> >> > > > 303-497-2768
> >> > > > ---
> >> > > > My working day may not be your working day. Please do not
feel
> >> obliged
> >> > to
> >> > > > reply to this email outside of your normal working hours.
> >> > > >
> >> > > >
> >> > >
> >> > >
> >> >
> >> > --
> >> > George McCabe - Software Engineer III
> >> > National Center for Atmospheric Research
> >> > Research Applications Laboratory
> >> > 303-497-2768
> >> > ---
> >> > My working day may not be your working day. Please do not feel
obliged
> >> to
> >> > reply to this email outside of your normal working hours.
> >> >
> >> >
> >>
> >>
> >
> > --
> > George McCabe - Software Engineer III
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > 303-497-2768
> > ---
> > My working day may not be your working day. Please do not feel
obliged to
> > reply to this email outside of your normal working hours.
> >
>
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>

------------------------------------------------
Subject: ask a question about METplus config of TC
From: Yan Jin - NOAA Affiliate
Time: Wed May 26 12:54:08 2021

Hi George,

Can I ask a question regarding the seasonal mean TC verification? For
instance, I have adeck and bdeck data over Atlantic at 2020, (Jet:
/mnt/lfs4/HFIP/hwrf-data/hwrf-input/abdeck), each has 01-99 cyclone
number.
And I would like to calculate the absolute intensity error and track
error
of seasonal mean over Atlantic. Here is my method, could you check if
it is
correct to do that?

I plan to do the May-June-July-August mean error.
$WORK:
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/vrfy_METplus4.0/METplus/parm/user_config
in this directory, the METplus config file is
TCMPRPlotter_AL_MJJA_2020


The major modification I did is:
***
MODEL = HWRF,HMON,GFSO

TC_PAIRS_DESC =

TC_PAIRS_STORM_ID =
TC_PAIRS_BASIN = AL
TC_PAIRS_CYCLONE = 01,02,03,04,05,06,07,08,09,10,11,12,13,14,15
TC_PAIRS_STORM_NAME =

TC_PAIRS_INIT_BEG = 20200515_000000
TC_PAIRS_INIT_END = 20200906_000000
***
I checked the adeck and bdeck data, and found that cyclone number 01-
15
have the initial times which are located within MJJA. So I
specified TC_PAIRS_CYCLONE with 01 to 15. I also checked the initial
time
in cyclone 01 and 15, and found the INIT_BEG and INIT_END.

It generated 15 tc_pairs.tcst files, and also 6 figures (Output
directory:
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/vrfy_METplus4.0/AL_MJJA_2020)

Here is the result
https://docs.google.com/document/d/10adrHdzNNxMJM51e5Sw9oy6uS2n1S-
qGS2c7NZdVVxc/edit?pli=1

I wonder if my method is correct? It seems that TC_PAIRS_INIT_BEG
and TC_PAIRS_INIT_END are mandatory for my case.

Many thanks.
Best,
Yan

On Thu, May 20, 2021 at 12:01 PM George McCabe via RT
<met_help at ucar.edu>
wrote:

> Hi Yan,
>
> I was able to change the line colors in the images. The images can
be found
> on Jet here:
>
> /lfs1/HFIP/dtc-hurr/George.Mccabe/eta-out-colors/tcmpr_plots
>
>
> This is the command I used to run:
>
> run_metplus.py
> /lfs1/HFIP/dtc-hurr/George.Mccabe/TCMPRPlotter_Eta_colors.conf
> config.OUTPUT_BASE=/lfs1/HFIP/dtc-hurr/George.Mccabe/eta-out-colors
>
>
> I copied the default tcmpr file from
>
/contrib/met/10.0.0/share/met/Rscripts/include/plot_tcmpr_config_default.R
> to my directory and renamed it to plot_tcmpr_overrides.R
>
> Then I modified the METplus config to point to that file instead of
the
> file in the METplus repository:
>
> TCMPR_PLOTTER_CONFIG_FILE =
> /lfs1/HFIP/dtc-hurr/George.Mccabe/plot_tcmpr_overrides.R
>
>
> The previous tcmpr parameter file changes the image resolution, so I
had to
> set that in the new file to prevent the image from being very large.
Then I
> changed the value of the default color list:
>
> img_res = 72;
> default_color_list  = c("blue", "green", "purple", "orange");
>
> This variable is used to set other variables for each plot type, so
you can
> modify those variables if you desire different colors for different
plots.
> You only need to supply the variables you want to change from the
default
> value, so your overrides file can just include the above 2 variables
to
> keep the file short and clean. However, it is nice to have all of
the
> possible options easily available
> in case you want to change other settings. How you handle this is up
to
> your preference.
>
> Let me know if you have any other questions.
>
> Thanks,
> George
>
> On Thu, May 20, 2021 at 9:19 AM George McCabe <mccabe at ucar.edu>
wrote:
>
> > Hi Yan,
> >
> > I copied your config file and made some modifications. You can
view it on
> > Jet here:
> >
> > /lfs1/HFIP/dtc-hurr/George.Mccabe/TCMPRPlotter_Eta.conf
> >
> > Starting in METplus 4.0.0 the different section headers like dir
and
> > filename_templates are no longer necessary. All configurations can
go
> under
> > the [config] header. I removed the sections and rearranged the
variables
> to
> > group them in a more clear way.
> >
> > To call TCMPRPlotter multiple times with different settings, I
added
> > multiple instances of the app in the PROCESS_LIST with instance
names
> > defined:
> >
> > PROCESS_LIST = TCPairs, TCMPRPlotter(abs_mean),
> > TCMPRPlotter(abs_median),TCMPRPlotter(abs_boxplot),
> TCMPRPlotter(tk_mean),
> > TCMPRPlotter(tk_median),TCMPRPlotter(tk_boxplot)
> >
> > Each instance will read all of the values set in the [config]
section,
> > then it will look for a section header that matches the string
inside the
> > parentheses for each instance. If the section is found, it
overrides the
> > values set in the section to replace the values set in the
[config]
> > section. These names can be whatever you chose, but the name in
the
> > PROCESS_LIST must match the name of the section header.
> >
> > [abs_mean]
> > TCMPR_PLOTTER_PREFIX = Eta_Absolute_Intensity_mean
> > TCMPR_PLOTTER_TITLE = Absolute Intensity Error Mean\n Eta
(Atlantic 2020)
> > TCMPR_PLOTTER_DEP_VARS = ABS(AMAX_WIND-BMAX_WIND)
> > TCMPR_PLOTTER_PLOT_TYPES = MEAN
> >
> > [abs_median]
> > TCMPR_PLOTTER_PREFIX = Eta_Absolute_Intensity_median
> > TCMPR_PLOTTER_TITLE = Absolute Intensity Error Median\n Eta
(Atlantic
> 2020)
> > TCMPR_PLOTTER_DEP_VARS = ABS(AMAX_WIND-BMAX_WIND)
> > TCMPR_PLOTTER_PLOT_TYPES = MEDIAN
> >
> > [abs_boxplot]
> > TCMPR_PLOTTER_PREFIX = Eta_Absolute_Intensity_boxplot
> > TCMPR_PLOTTER_TITLE = Absolute Intensity Error Boxplot\n Eta
(Atlantic
> > 2020)
> > TCMPR_PLOTTER_DEP_VARS = ABS(AMAX_WIND-BMAX_WIND)
> > TCMPR_PLOTTER_PLOT_TYPES = BOXPLOT
> >
> > [tk_mean]
> > TCMPR_PLOTTER_PREFIX = Eta_Track_Error_mean
> > TCMPR_PLOTTER_TITLE = Track Error Mean\n Eta (Atlantic 2020)
> > TCMPR_PLOTTER_DEP_VARS = TK_ERR
> > TCMPR_PLOTTER_PLOT_TYPES = MEAN
> >
> > [tk_median]
> > TCMPR_PLOTTER_PREFIX = Eta_Track_Error_median
> > TCMPR_PLOTTER_TITLE = Track Error Median\n Eta (Atlantic 2020)
> > TCMPR_PLOTTER_DEP_VARS = TK_ERR
> > TCMPR_PLOTTER_PLOT_TYPES = MEDIAN
> >
> > [tk_boxplot]
> > TCMPR_PLOTTER_PREFIX = Eta_Track_Error_boxplot
> > TCMPR_PLOTTER_TITLE = Track Error Boxplot\n Eta (Atlantic 2020)
> > TCMPR_PLOTTER_DEP_VARS = TK_ERR
> > TCMPR_PLOTTER_PLOT_TYPES = BOXPLOT
> >
> >
> > I split up the DEP_VARS and PLOT_TYPES values into instances with
one
> > value of each and set the plot title and filenames appropriately
for
> each.
> > I set the X label to "Lead Time (h)" in the [config] section so it
> applies
> > to every instance.
> >
> > TCMPR_PLOTTER_XLAB = Lead Time (h)
> >
> > The output images from my run is on Jet here:
> >
> > /lfs1/HFIP/dtc-hurr/George.Mccabe/eta-out/tcmpr_plots
> >
> > Eta_Absolute_Intensity_boxplot.log
> > Eta_Absolute_Intensity_boxplot.png
> > Eta_Absolute_Intensity_mean.png
> > Eta_Absolute_Intensity_median.png
> > Eta_Track_Error_boxplot.log
> > Eta_Track_Error_boxplot.png
> > Eta_Track_Error_mean.png
> > Eta_Track_Error_median.png
> >
> >
> > Here is the log file from the run:
> > /lfs1/HFIP/dtc-hurr/George.Mccabe/eta-
out/logs/metplus.log.20210520150804
> >
> >
> > Here is what I ran to produce the output:
> >
> >
> >    - module load intel
> >    - module load intelpython/3.6.5
> >    - module load netcdf/4.6.1
> >    - module load hdf5/1.10.4
> >    - module load nco/4.9.1
> >    - module load wgrib/1.8.1.0b
> >    - module load wgrib2/2.0.8
> >    - module load R/4.0.2
> >    - module use /contrib/met/modulefiles
> >    - module load met/10.0.0
> >    - module use /contrib/met/METplus/modulefiles
> >    - module load metplus/4.0.0
> >    - run_metplus.py
> >    /lfs1/HFIP/dtc-hurr/George.Mccabe/TCMPRPlotter_Eta.conf
> >    config.OUTPUT_BASE=/lfs1/HFIP/dtc-hurr/George.Mccabe/eta-out
> >
> >
> >
> > I will look into what settings are needed to change the line
colors.
> Also,
> > I have started making the modifications to the TCPairs wrapper to
> properly
> > handle multiple storms specified in the config file. I will let
you know
> > when that is ready to test.
> >
> > Thanks,
> > George
> >
> >
> >
> > On Wed, May 19, 2021 at 5:09 PM George McCabe <mccabe at ucar.edu>
wrote:
> >
> >> Hi Yan,
> >>
> >> 1) add TC name to each plot name
> >> 2) modify titles to each plot, e.g.: TK_ERR* plots with name
"Track
> >> Error", and ABS_AMAX_WIND-BMAX_WIND_* plots with name "Absolute
> Intensity
> >> Error"?
> >>
> >> It looks like the logic in the R script uses the -prefix value to
> replace
> >> the filename instead of prepending it to the plot type text as
you would
> >> guess it would. I was told that this script was typically called
> multiple
> >> times to generate the multiple plots, which would provide more
control
> over
> >> setting the output filenames, titles, etc. The bad news is it
looks like
> >> the wrapper was written to call the R script once with all of the
> >> configurations specified in the METplus config file. The good
news is
> there
> >> are some new capabilities added to METplus 4.0.0 that could help
produce
> >> the results you need.
> >>
> >> Please review this section of the METplus User's Guide,
specifically the
> >> section about specifying instance names to the processes:
> >>
> >>
>
https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#process-
list
> >>
> >> Alternatively, you can use the UserScript wrapper to build custom
> >> commands to call the R script directly with any arguments you
need if
> the
> >> wrapper doesn't support an option you need.
> >> More information on that tool is here:
> >>
>
https://metplus.readthedocs.io/en/latest/Users_Guide/wrappers.html#userscript
> >> and here:
> >>
>
https://metplus.readthedocs.io/en/latest/generated/met_tool_wrapper/UserScript/UserScript_run_once.html#sphx-
glr-generated-met-tool-wrapper-userscript-userscript-run-once-py
> >>
> >> I think using different instance names to run TCMPRPlotter
multiple
> times
> >> with different settings is the better approach in this case. I am
> wrapping
> >> up work for the day, but tomorrow I can log into Jet and help you
modify
> >> your config file to do this.
> >>
> >>
> >> 3) Is it possible to modify the line colors in this METplus
config file?
> >>
> >> Yes, you can change the line colors through the R script's config
file.
> >> You can copy and rename this file:
> >>
> >>
> >>
>
https://github.com/dtcenter/MET/blob/main_v10.0/met/scripts/Rscripts/include/plot_tcmpr_config_default.R
> >>
> >> change the values that you need to change (look for
"color_list"), then
> >> set the METplus configuration file to point to your new file (set
in
> >> TCMPR_PLOTTER_CONFIG_FILE).
> >>
> >>
> >> Let me know if you have any other questions. I will report back
tomorrow
> >> when have some changes for you to test.
> >>
> >> Thanks,
> >> George
> >>
> >> On Wed, May 19, 2021 at 4:00 PM Yan Jin - NOAA Affiliate via RT <
> >> met_help at ucar.edu> wrote:
> >>
> >>>
> >>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846 >
> >>>
> >>> Hi George,
> >>>
> >>> I have a few questions regarding the TCMPR_PLOTTER_* parameters
in the
> >>> METplus config file.
> >>>
> >>> Here is the METplus config file I used:
> >>>
> >>>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/vrfy_METplus4.0/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test.conf
> >>> **
> >>> #  Plot_TCMPR options, if left unset, default values that are
> >>> #  pre-defined in the R utility (packaged with MET) will be
used.
> >>> TCMPR_PLOTTER_CONFIG_FILE =
{CONFIG_DIR}/TCMPRPlotterConfig_customize
> >>> TCMPR_PLOTTER_PREFIX =
> >>> TCMPR_PLOTTER_TITLE =
> >>> TCMPR_PLOTTER_SUBTITLE =
> >>> TCMPR_PLOTTER_XLAB =
> >>> TCMPR_PLOTTER_YLAB =
> >>> TCMPR_PLOTTER_XLIM =
> >>> TCMPR_PLOTTER_YLIM =
> >>> TCMPR_PLOTTER_FILTER = -amodel HWRF,HMON
> >>> # the tcst data file to be used instead of running the MET
tc_stat
> tool.
> >>> TCMPR_PLOTTER_FILTERED_TCST_DATA_FILE =
> >>> # Comma separated, no whitespace.  Default is TK_ERR (track
error)
> unless
> >>> # otherwise indicated.
> >>> TCMPR_PLOTTER_DEP_VARS = ABS(AMAX_WIND-BMAX_WIND),TK_ERR
> >>> TCMPR_PLOTTER_SCATTER_X =
> >>> TCMPR_PLOTTER_SCATTER_Y =
> >>> TCMPR_PLOTTER_SKILL_REF =
> >>> TCMPR_PLOTTER_SERIES = AMODEL
> >>> TCMPR_PLOTTER_SERIES_CI = TRUE
> >>> TCMPR_PLOTTER_LEGEND =
> >>> TCMPR_PLOTTER_LEAD = 0,12,24,36,48,60,72,84,96,108,120
> >>> # Mean and median plots. These override the plot_tcmpr.R default
of box
> >>> plot.
> >>> # If box plot is desired, this needs to be explicitly indicated.
> >>> TCMPR_PLOTTER_PLOT_TYPES = MEAN,MEDIAN,BOXPLOT
> >>> TCMPR_PLOTTER_RP_DIFF =
> >>> TCMPR_PLOTTER_DEMO_YR =
> >>> TCMPR_PLOTTER_HFIP_BASELINE =
> >>> TCMPR_PLOTTER_FOOTNOTE_FLAG =
> >>> TCMPR_PLOTTER_PLOT_CONFIG_OPTS =
> >>> TCMPR_PLOTTER_SAVE_DATA =
> >>>
> >>> #  TCMPR FLAGS no == (don't set flag), yes == (set flag)
> >>> TCMPR_PLOTTER_NO_EE = no
> >>> TCMPR_PLOTTER_NO_LOG = no
> >>> TCMPR_PLOTTER_SAVE = no
> >>> **
> >>> It generated the following 6 *.png files
> >>> in
> >>>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/vrfy_METplus4.0/Eta_use_METplus/tcmpr_plots
> >>> [image: image.png]
> >>>
> >>> Then I would like to modify titles, xlab text, output file
names.
> >>> Here are the modification I
> >>> did
> >>>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/vrfy_METplus4.0/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test1.conf
> >>>
> >>> **
> >>> TCMPR_PLOTTER_TITLE = Absolute Intensity Error \n Eta (Atlantic
2020)
> >>> TCMPR_PLOTTER_XLAB = Lead Time (h)
> >>> **
> >>> The title of each plot becomes "Absolute Intensity Error Eta
(Atlantic
> >>> 2020)" with double quotes.
> >>>
> >>> I also tried to set TCMPR_PLOTTER_PREFIX = Eta_ because I would
like to
> >>> save each plot as Eta_ABS_AMAX_WIND-BMAX_WIND_*.png, and
> >>> Eta_TK_ERR_*.png.
> >>> However, there is only one plot Eta_.png generated.
> >>>
> >>> I would like to check how can I set those parameters if I would
like to
> >>> 1)
> >>> add TC name to each plot name; 2) modify titles to each plot,
e.g.:
> >>> TK_ERR*
> >>> plots with name "Track Error", and ABS_AMAX_WIND-BMAX_WIND_*
plots with
> >>> name "Absolute Intensity Error"? 3) Is it possible to modify the
line
> >>> colors in this METplus config file?
> >>>
> >>> Could you give me some suggestions?
> >>>
> >>> Many thanks.
> >>> Best,
> >>> Yan
> >>>
> >>> On Mon, May 17, 2021 at 10:37 AM George McCabe via RT <
> met_help at ucar.edu
> >>> >
> >>> wrote:
> >>>
> >>> > Hi Yan,
> >>> >
> >>> > 4.1:
> >>> > The consensus variable is not yet controlled by a METplus
> configuration
> >>> > variable explicitly, but you can set the value by using the
> >>> > TC_PAIRS_MET_CONFIG_OVERRIDES value (in v4.0.0+). To set the
> consensus
> >>> > value to what you have set in your config file, you would set
> something
> >>> > like this:
> >>> >
> >>> > TC_PAIRS_MET_CONFIG_OVERRIDES = consensus = [ { name = "CONS";
> members
> >>> =
> >>> > ["HWRF","GFSO","HMON"]; required = [false,false,false];
min_req = 2;
> }
> >>> ];
> >>> >
> >>> > I will make a note in the GitHub issue for improvements to the
> TCPairs
> >>> > wrapper (https://github.com/dtcenter/METplus/issues/749) to
add
> >>> support
> >>> > for
> >>> > consensus explicitly via something like TC_PAIRS_CONSENSUS so
that it
> >>> is
> >>> > easier to configure via METplus wrappers.
> >>> >
> >>> > 4.2:
> >>> > The System Configuration section of the METplus User's Guide
covers
> >>> these
> >>> > variables and how they affect execution. I suggest that you
read
> >>> through
> >>> > this section to get a full picture of how these variables
work.
> >>> Regarding
> >>> > your specific question, these 2 sections should help:
> >>> >
> >>> > LOOP_BY:
> >>> >
> >>> >
> >>>
>
https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#loop-
by
> >>> > LOOP_ORDER:
> >>> >
> >>> >
> >>>
>
https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#loop-
order
> >>> >
> >>> > Thanks,
> >>> > George
> >>> >
> >>> > On Fri, May 14, 2021 at 8:19 AM Yan Jin - NOAA Affiliate via
RT <
> >>> > met_help at ucar.edu> wrote:
> >>> >
> >>> > >
> >>> > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846 >
> >>> > >
> >>> > > Hi George,
> >>> > >
> >>> > > Thanks so much for your help. Finally the code can run
without
> >>> errors.
> >>> > Here
> >>> > > is the steps and results:
> >>> > >
> >>> > >
> >>> > >
> >>> >
> >>>
> https://docs.google.com/document/d/1xRF3aiPATcX5G37ngJj-
65pl44LN7mGHo34Sf8i3zpo/edit
> >>> > >
> >>> > > I also put questions in the document, I also pasted here:
> >>> > >
> >>> > > Questions:
> >>> > >
> >>> > > 4.1: How can I do the consensus forecasts in METplus config
file as
> >>> I did
> >>> > > in tc_pairs?
> >>> > >
> >>> > >
> >>> >
> >>>
>
(/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_292020/TCPairsConfig_run1)
> >>> > >
> >>> > > //
> >>> > >
> >>> > > // Specify how consensus forecasts should be defined.
> >>> > >
> >>> > > //
> >>> > >
> >>> > > consensus = [
> >>> > >
> >>> > >  {
> >>> > >
> >>> > >   name = "CONS";
> >>> > >
> >>> > >   members = ["HWRF","GFSO","HMON"];
> >>> > >
> >>> > >   required = [false,false,false];
> >>> > >
> >>> > >   min_req = 2;
> >>> > >
> >>> > >  }
> >>> > >
> >>> > > ];
> >>> > >
> >>> > > 4.2: I am a little confused about
> >>> > >
> >>> > >       ***
> >>> > >
> >>> > >        LOOP_ORDER = processes
> >>> > >
> >>> > >        LOOP_BY = INIT
> >>> > >
> >>> > >       ***
> >>> > >
> >>> > > Could you explain these two command lines and give me some
> examples?
> >>> > >
> >>> > > Thanks very much for your time and help on my questions.
> >>> > >
> >>> > > Best,
> >>> > > Yan
> >>> > >
> >>> > > On Thu, May 13, 2021 at 3:04 PM George McCabe via RT <
> >>> met_help at ucar.edu>
> >>> > > wrote:
> >>> > >
> >>> > > > Hi Yan,
> >>> > > >
> >>> > > > The produtil is due to a bug in the wrapper in the logic
that
> >>> reports
> >>> > > when
> >>> > > > an error occurs. I ran the command on Jet by hand and it
looks
> >>> like it
> >>> > is
> >>> > > > still not finding any input data to plot. You can run the
> following
> >>> > > > commands (after loading all of the required modules):
> >>> > > >
> >>> > > > export MET_INSTALL_DIR=/contrib/met/9.1
> >>> > > >
> >>> > > > Rscript /contrib/met/9.1/share/met/Rscripts/plot_tcmpr.R \
> >>> > > > -config
> >>> > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
> /contrib/met/METplus/METplus-
3.1.1/parm/use_cases/model_applications/tc_and_extra_tc/TCMPRPlotterConfig_customize
> >>> > > > \
> >>> > > > -subtitle "Your subtitle goes here" -ylab "Your y-label
goes
> here"
> >>> \
> >>> > > > -dep AMAX_WIND-BMAX_WIND,TK_ERR -legend "Your legend text
goes
> >>> > here..." \
> >>> > > > -plot MEAN,MEDIAN,BOXPLOT \
> >>> > > > -lookin
> >>> > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> >>> > > > \
> >>> > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
> >>> > > > \
> >>> > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
> >>> > > > \
> >>> > > > -outdir
> >>> > > >
> >>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tcmpr_plots
> >>> > > >
> >>> > > >
> >>> > > > Here is the output from that run:
> >>> > > >
> >>> > > > Reading plot configuration file:
> >>> > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
> /contrib/met/METplus/METplus-
3.1.1/parm/use_cases/model_applications/tc_and_extra_tc/TCMPRPlotterConfig_customize
> >>> > > > No plotting configuration file specified.
> >>> > > > CALLING:  /contrib/met/9.1/bin/tc_stat -lookin
> >>> > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> >>> > > > -lookin
> >>> > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
> >>> > > > -lookin
> >>> > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
> >>> > > > -job filter -dump_row /tmp/plot_tcmpr_116966.tcst
-event_equal
> >>> true
> >>> > -v 3
> >>> > > > DEBUG 1: [Source 1] Lookin:
> >>> > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> >>> > > > DEBUG 1: [Source 2] Lookin:
> >>> > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
> >>> > > > DEBUG 1: [Source 3] Lookin:
> >>> > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
> >>> > > > DEBUG 2:
> >>> > > > DEBUG 2: Processing Job 1: -job filter -event_equal true
> -dump_row
> >>> > > > /tmp/plot_tcmpr_116966.tcst
> >>> > > > DEBUG 3: Applying track-based event equalization logic.
> >>> > > > DEBUG 3: Reading file 1 of 3:
> >>> > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> >>> > > > DEBUG 3: Reading file 2 of 3:
> >>> > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
> >>> > > > DEBUG 3: Reading file 3 of 3:
> >>> > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
> >>> > > > DEBUG 3: For track-based event equalization, identified 0
common
> >>> cases
> >>> > > for
> >>> > > > 146 models: AC00 AEMI AEMN AP01 AP02 AP03 AP04 AP05 AP06
AP07
> AP08
> >>> AP09
> >>> > > > AP0I AP10 AP11 AP12 AP13 AP14 AP15 AP16 AP17 AP18 AP19
AP1I AP20
> >>> AP21
> >>> > > AP22
> >>> > > > AP23 AP24 AP25 AP26 AP27 AP28 AP29 AP30 API CARQ CEM2 CEMI
CEMN
> >>> CLP5
> >>> > CMC
> >>> > > > CMC2 CMCI COTC COTI CTCX DRCL DSHP EC00 EEMN EGRR EMN2
EMN3 EMN4
> >>> EMNI
> >>> > EMX
> >>> > > > EMX2 EMXI EN01 EN02 EN03 EN04 EN05 EN06 EN07 EN08 EN09
EN0I EN10
> >>> EN11
> >>> > > EN12
> >>> > > > EN13 EN14 EN15 EN16 EN17 EN18 EN19 EN1I EN20 EN21 EN22
EN23 EN24
> >>> EN25
> >>> > ENI
> >>> > > > EP01 EP02 EP03 EP04 EP05 EP06 EP07 EP08 EP09 EP0I EP10
EP11 EP12
> >>> EP13
> >>> > > EP14
> >>> > > > EP15 EP16 EP17 EP18 EP19 EP1I EP20 EP21 EP22 EP23 EP24
EP25 EPI
> >>> GFEX
> >>> > GFSI
> >>> > > > GFSO HMNI HMON HWF2 HWFI HWRF ICON IVCN LGEM NGX NGX2 NGXI
NVGM
> >>> OCD5
> >>> > RVCN
> >>> > > > SHF5 SHIP TABD TABM TABS TCLP TVCA TVCE TVCN TVCX UKX UKX2
UKXI
> >>> XTRP
> >>> > > > DEBUG 1: Event equalization of tracks found no common
cases.
> >>> > > > DEBUG 1: Creating output dump file:
/tmp/plot_tcmpr_116966.tcst
> >>> > > > FILTER: -job filter -line_type TCMPR -event_equal true
-dump_row
> >>> > > > /tmp/plot_tcmpr_116966.tcst
> >>> > > >
> >>> > > > DEBUG 2: Job 1 used 0 out of 0 lines read.
> >>> > > > DEBUG 3: Total lines read                 = 0
> >>> > > > DEBUG 3: Total lines kept                 = 0
> >>> > > > DEBUG 3: Rejected for track watch/warn    = 0
> >>> > > > DEBUG 3: Rejected for init threshold      = 0
> >>> > > > DEBUG 3: Rejected for init string         = 0
> >>> > > > DEBUG 3: Rejected for out init mask       = 0
> >>> > > > DEBUG 3: Rejected for water only          = 0
> >>> > > > DEBUG 3: Rejected for rapid inten         = 0
> >>> > > > DEBUG 3: Rejected for landfall            = 0
> >>> > > > DEBUG 3: Rejected for amodel              = 0
> >>> > > > DEBUG 3: Rejected for bmodel              = 0
> >>> > > > DEBUG 3: Rejected for desc                = 0
> >>> > > > DEBUG 3: Rejected for storm id            = 0
> >>> > > > DEBUG 3: Rejected for basin               = 0
> >>> > > > DEBUG 3: Rejected for cyclone             = 0
> >>> > > > DEBUG 3: Rejected for storm name          = 0
> >>> > > > DEBUG 3: Rejected for init time           = 0
> >>> > > > DEBUG 3: Rejected for init hour           = 0
> >>> > > > DEBUG 3: Rejected for lead time           = 0
> >>> > > > DEBUG 3: Rejected for required lead times = 0
> >>> > > > DEBUG 3: Rejected for valid time          = 0
> >>> > > > DEBUG 3: Rejected for valid hour          = 0
> >>> > > > DEBUG 3: Rejected for init mask           = 0
> >>> > > > DEBUG 3: Rejected for valid mask          = 0
> >>> > > > DEBUG 3: Rejected for line type           = 0
> >>> > > > DEBUG 3: Rejected for numeric threshold   = 0
> >>> > > > DEBUG 3: Rejected for string matching     = 0
> >>> > > > DEBUG 3: Rejected for match points        = 0
> >>> > > > DEBUG 3: Rejected for event equal         = 0
> >>> > > > DEBUG 3: Rejected for out init mask       = 0
> >>> > > > DEBUG 3: Rejected for out valid mask      = 0
> >>> > > > Reading track data: /tmp/plot_tcmpr_116966.tcst
> >>> > > > Error in read.table(tcst_tmp_file, header = TRUE) :
> >>> > > >   no lines available in input
> >>> > > > Execution halted
> >>> > > >
> >>> > > >
> >>> > > >
> >>> > > > It sounds like filter arguments may need to be provided to
> produce
> >>> > > TC-Stat
> >>> > > > output (which is called by the R script). You can set this
in
> your
> >>> > > METplus
> >>> > > > config file with the TCMPR_PLOTTER_FILTER variable, but it
will
> be
> >>> > > > difficult to debug since the log output is not shown. I
would
> >>> recommend
> >>> > > > modifying the Rscript call from this email and add the
-filter
> >>> argument
> >>> > > > with any filter information to pass to TCStat. I am not
sure
> >>> settings
> >>> > > would
> >>> > > > help you produce output, but I have reached out to other
METplus
> >>> team
> >>> > > > members for suggestions.
> >>> > > >
> >>> > > > Thanks,
> >>> > > > George
> >>> > > >
> >>> > > >
> >>> > > >
> >>> > > > On Thu, May 13, 2021 at 12:37 PM Yan Jin - NOAA Affiliate
via RT
> <
> >>> > > > met_help at ucar.edu> wrote:
> >>> > > >
> >>> > > > >
> >>> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846
> >
> >>> > > > >
> >>> > > > > Hi George,
> >>> > > > >
> >>> > > > > Thanks so much. Now tc_pairs works, and the output file
can be
> >>> > > generated.
> >>> > > > >
> >>> > > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> >>> > > > >
> >>> > > > > Then I set "PROCESS_LIST = TCPairs, TCMPRPlotter", and I
kept
> >>> most
> >>> > > > > parameters that initialized with "TCMPR_PLOTTER" the
same as
> the
> >>> > > default
> >>> > > > > configuration.
> >>> > > > >
> >>> > > > > my METplus config file:
> >>> > > > >
> >>> > > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test.conf
> >>> > > > >
> >>> > > > > then run script
> >>> > > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/user_config/run.sh
> >>> > > > >
> >>> > > > > However, the Rscript cannot plot figures.
> >>> > > > > I checked the log file:
> >>> > > > >
> >>> > > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210513182819
> >>> > > > > [image: image.png]
> >>> > > > >
> >>> > > > > It seems that the error "NameError: name 'produtil' is
not
> >>> defined"?
> >>> > > > >
> >>> > > > > Could you give me some suggestions?
> >>> > > > >
> >>> > > > > Many thanks.
> >>> > > > > Best,
> >>> > > > > Yan
> >>> > > > >
> >>> > > > > On Wed, May 12, 2021 at 5:20 PM George McCabe via RT <
> >>> > > met_help at ucar.edu>
> >>> > > > > wrote:
> >>> > > > >
> >>> > > > > > Hi Yan,
> >>> > > > > >
> >>> > > > > > It looks like the values set for init_beg and init_end
in the
> >>> > TCpairs
> >>> > > > > > config file are both "20200101_000000" because the
INIT_BEG
> and
> >>> > > > INIT_END
> >>> > > > > > time is 2020. There are no adeck tracks for this time,
so all
> >>> of
> >>> > the
> >>> > > > data
> >>> > > > > > is being filtered out.
> >>> > > > > >
> >>> > > > > > This is a little confusing in METplus 3.1 because
INIT_BEG
> and
> >>> > > INIT_END
> >>> > > > > > control the time looping of execution (if applicable
for the
> >>> use
> >>> > > case)
> >>> > > > > AND
> >>> > > > > > the values passed to the wrapped TCPairs config file.
In
> >>> METplus
> >>> > > 4.0.0
> >>> > > > > > (which was released yesterday and should be available
on jet
> >>> > soon!),
> >>> > > > you
> >>> > > > > > can set INIT_BEG and INIT_END to control time looping
(if
> >>> desired)
> >>> > > and
> >>> > > > > use
> >>> > > > > > TC_PAIRS_INIT_BEG and TC_PAIRS_INIT_END to specify the
values
> >>> to
> >>> > pass
> >>> > > > to
> >>> > > > > > the TCPairs config file.
> >>> > > > > >
> >>> > > > > > If you were using METplus 4.0.0, I would recommend
setting
> >>> > something
> >>> > > > > like:
> >>> > > > > >
> >>> > > > > >
> >>> > > > > > LOOP_BY = INIT
> >>> > > > > > INIT_TIME_FMT = %Y
> >>> > > > > > INIT_BEG = 2020
> >>> > > > > > INIT_END = 2020
> >>> > > > > > INIT_INCREMENT = 21600
> >>> > > > > >
> >>> > > > > > TC_PAIRS_INIT_BEG = 20201030_000000
> >>> > > > > > TC_PAIRS_INIT_END = 20201031_000000
> >>> > > > > >
> >>> > > > > >
> >>> > > > > > This would run once using 2020 to find filenames and
process
> >>> data
> >>> > > from
> >>> > > > > the
> >>> > > > > > files that are initialized between 20201030_000000 and
> >>> > > 20201031_000000.
> >>> > > > > >
> >>> > > > > > If using METplus 3.1, you will have to use the
INIT_BEG and
> >>> > INIT_END
> >>> > > > > values
> >>> > > > > > to set the values in the wrapped TCPairs config. This
is OK
> for
> >>> > this
> >>> > > > use
> >>> > > > > > case because you have LOOP_ORDER set to processes and
TCPairs
> >>> only
> >>> > > runs
> >>> > > > > > once with that setting.
> >>> > > > > >
> >>> > > > > > LOOP_BY = INIT
> >>> > > > > > INIT_TIME_FMT = %Y%m%d
> >>> > > > > > INIT_BEG = 20201030
> >>> > > > > > INIT_END = 20201031
> >>> > > > > >
> >>> > > > > > This configuration should run and only use tracks
initialized
> >>> > between
> >>> > > > > > 2020-10-30 @ 0Z and 2020-10-31 @ 0Z. If you need to
specify
> the
> >>> > init
> >>> > > > time
> >>> > > > > > range with hour precision, you can do that as well.
Here is
> an
> >>> > > example:
> >>> > > > > >
> >>> > > > > > LOOP_BY = INIT
> >>> > > > > > INIT_TIME_FMT = %Y%m%d%H
> >>> > > > > > INIT_BEG = 2020103012
> >>> > > > > > INIT_END = 2020103112
> >>> > > > > >
> >>> > > > > > This configuration will only use tracks initialized
between
> >>> > > 2020-10-30
> >>> > > > @
> >>> > > > > > 12Z and 2020-10-31 @ 12Z.
> >>> > > > > >
> >>> > > > > > You can also restrict the data used by valid time as
well by
> >>> using
> >>> > > > > > TC_PAIRS_VALID_BEG and TC_PAIRS_VALID_END in METplus
4.0.0 or
> >>> > > VALID_BEG
> >>> > > > > and
> >>> > > > > > VALID_END for 3.1.
> >>> > > > > >
> >>> > > > > > Let me know if you are still unable to produce output
with
> >>> these
> >>> > > > > changes. I
> >>> > > > > > was able to run tc_pairs by hand on Jet and produce a
file
> with
> >>> > > output
> >>> > > > > > tracks. You can view it here if you'd like:
> >>> > > > > >
> >>> > > > > > /lfs1/HFIP/dtc-
hurr/George.Mccabe/al292020_pairs.dat.tcst
> >>> > > > > >
> >>> > > > > > These are the commands I used:
> >>> > > > > >
> >>> > > > > > export BASIN="[]";
> >>> > > > > > export CYCLONE="[]";
> >>> > > > > > export DLAND_FILE="/contrib/met/9.1/share/met/tc_data/
> >>> > > > > > dland_global_tenth_degree.nc";
> >>> > > > > > export INIT_BEG="20201030_000000";
> >>> > > > > > export INIT_END="20201031_000000";
> >>> > > > > > export INIT_EXCLUDE="[]";
> >>> > > > > > export INIT_INCLUDE="[]";
> >>> > > > > > export MODEL="[]";
> >>> > > > > > export STORM_ID="[\"AL292020\"]";
> >>> > > > > > export STORM_NAME="[]";
> >>> > > > > > export VALID_BEG="";
> >>> > > > > > export VALID_END="";
> >>> > > > > > /contrib/met/9.1/bin/tc_pairs -v 4 -bdeck
> >>> > > > > > /mnt/lfs4/HFIP/hwrf-data/hwrf-
input/abdeck/btk/bal292020.dat
> >>> -adeck
> >>> > > > > >
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/data/aal292020.dat
> >>> > -config
> >>> > > > > >
> >>> > > > > >
> >>> > > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/use_cases/model_applications/tc_and_extra_tc/TCPairsETCConfig
> >>> > > > > > -out /lfs1/HFIP/dtc-
hurr/George.Mccabe/al292020_pairs.dat
> >>> > > > > >
> >>> > > > > > I took the values from your METplus log file and
modified
> them
> >>> to
> >>> > > write
> >>> > > > > to
> >>> > > > > > my directory and use a different init time range.
> >>> > > > > >
> >>> > > > > > Thanks,
> >>> > > > > > George
> >>> > > > > >
> >>> > > > > >
> >>> > > > > >
> >>> > > > > >
> >>> > > > > > On Wed, May 12, 2021 at 2:04 PM Yan Jin - NOAA
Affiliate via
> >>> RT <
> >>> > > > > > met_help at ucar.edu> wrote:
> >>> > > > > >
> >>> > > > > > >
> >>> > > > > > > <URL:
> >>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846 >
> >>> > > > > > >
> >>> > > > > > > Hi George,
> >>> > > > > > >
> >>> > > > > > > Thanks very much for your instructions. I set
> "PROCESS_LIST =
> >>> > > > TCPairs",
> >>> > > > > > and
> >>> > > > > > > didn't specify models: "MODEL =". Then run the
script.
> >>> However,
> >>> > the
> >>> > > > > > output
> >>> > > > > > > file is still empty. It seems that tc_pairs matched
0 adeck
> >>> track
> >>> > > to
> >>> > > > 1
> >>> > > > > > > bdeck track. I also tried to set "MODEL = HWRF,
HMON", but
> >>> the
> >>> > > output
> >>> > > > > > file
> >>> > > > > > > is empty too.
> >>> > > > > > >
> >>> > > > > > > Here is the log file:
> >>> > > > > > >
> >>> > > > > > >
> >>> > > > > >
> >>> > > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210512193152
> >>> > > > > > >
> >>> > > > > > > And here is the output file:
> >>> > > > > > >
> >>> > > > > > >
> >>> > > > > >
> >>> > > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> >>> > > > > > >
> >>> > > > > > > Many thanks.
> >>> > > > > > > Best,
> >>> > > > > > > Yan
> >>> > > > > > >
> >>> > > > > > > On Wed, May 12, 2021 at 11:36 AM George McCabe via
RT <
> >>> > > > > met_help at ucar.edu
> >>> > > > > > >
> >>> > > > > > > wrote:
> >>> > > > > > >
> >>> > > > > > > > Hi Yan,
> >>> > > > > > > >
> >>> > > > > > > > I was able to log into Jet this morning and look
at your
> >>> error.
> >>> > > > > > > >
> >>> > > > > > > > It looks like the error in the call to Rscript is
due to
> >>> there
> >>> > > > being
> >>> > > > > no
> >>> > > > > > > > data in the input files generated by TCPairs.
There are
> >>> > > > > inconsistencies
> >>> > > > > > > in
> >>> > > > > > > > the TCMPRPlotter wrapper that prevent the log
output from
> >>> the
> >>> > run
> >>> > > > to
> >>> > > > > be
> >>> > > > > > > > seen, which would have made it more clear why the
issue
> was
> >>> > > > > occurring.
> >>> > > > > > > >
> >>> > > > > > > > I would recommend changing the PROCESS_LIST value
in your
> >>> > METplus
> >>> > > > > > > > configuration file to only run TCPairs and
iterating on
> >>> that
> >>> > > until
> >>> > > > > you
> >>> > > > > > > get
> >>> > > > > > > > output from that tool. Once you can confirm that
you have
> >>> > output,
> >>> > > > you
> >>> > > > > > can
> >>> > > > > > > > add TCMPRPlotter back to the PROCESS_LIST and test
on
> that
> >>> > > portion.
> >>> > > > > > > >
> >>> > > > > > > > I will create a GitHub issue for updating the
> TCMPRPlotter
> >>> > > wrapper
> >>> > > > to
> >>> > > > > > > > produce useful logging output that will make
things more
> >>> clear
> >>> > > > going
> >>> > > > > > > > forward.
> >>> > > > > > > >
> >>> > > > > > > > Let me know if you run into any other issues or
need help
> >>> > > > configuring
> >>> > > > > > > > TCPairs to run properly. I'm also happy to attend
another
> >>> > working
> >>> > > > > > session
> >>> > > > > > > > if that is helpful.
> >>> > > > > > > >
> >>> > > > > > > > Thanks,
> >>> > > > > > > > George
> >>> > > > > > > >
> >>> > > > > > > >
> >>> > > > > > > > On Tue May 11 10:05:06 2021, yan.jin at noaa.gov
wrote:
> >>> > > > > > > > > Hi John,
> >>> > > > > > > > >
> >>> > > > > > > > >
> >>> > > > > > > > > The directories and data I mentioned in my last
email
> >>> are all
> >>> > > > > located
> >>> > > > > > > > > in
> >>> > > > > > > > > Jet.
> >>> > > > > > > > >
> >>> > > > > > > > >
> >>> > > > > > > > > Here is the email I got regarding the Jet
maintenance:
> >>> > > > > > > > >
> >>> > > > > > > > > ***
> >>> > > > > > > > >
> >>> > > > > > > > > *05/11/21 - Boulder Bastion Maintenance
Downtime*
> >>> > > > > > > > > Boulder Bastion maintenance is scheduled for
Tuesday
> >>> 05/11/21
> >>> > > > from
> >>> > > > > > > > > 0600 ~
> >>> > > > > > > > > 1000 MT. Boulder Bastion login sessions will be
dropped
> >>> and
> >>> > all
> >>> > > > new
> >>> > > > > > > > > RDHPCS
> >>> > > > > > > > > logins will need to go through the Princeton
Bastion.
> >>> > Logging
> >>> > > In
> >>> > > > > > > > > <
> >>> > > > > >
> >>> > https://rdhpcs-common-
docs.rdhpcs.noaa.gov/wiki/index.php/Logging_in
> >>> > > >
> >>> > > > > > > > >
> >>> > > > > > > > >
> >>> > > > > > > > >
> >>> > > > > > > > > *The next Jet Maintenance Downtime is TBD*
> >>> > > > > > > > >
> >>> > > > > > > > > ***
> >>> > > > > > > > >
> >>> > > > > > > > >
> >>> > > > > > > > > I tried to login Princeton but the connection is
closed
> >>> too.
> >>> > > > > > > > >
> >>> > > > > > > > >
> >>> > > > > > > > > Really appreciate your time and help on this.
> >>> > > > > > > > >
> >>> > > > > > > > >
> >>> > > > > > > > > Best,
> >>> > > > > > > > >
> >>> > > > > > > > > Yan
> >>> > > > > > > > >
> >>> > > > > > > > > On Tue, May 11, 2021 at 11:57 AM John Halley
Gotway via
> >>> RT <
> >>> > > > > > > > > met_help at ucar.edu> wrote:
> >>> > > > > > > > >
> >>> > > > > > > > > > Yan,
> >>> > > > > > > > > >
> >>> > > > > > > > > > George and I tried to take a look at this, but
> couldn't
> >>> > find
> >>> > > > the
> >>> > > > > > > > > > data.
> >>> > > > > > > > > >
> >>> > > > > > > > > >
> >>> > > > > > > >
> >>> > > > > > >
> >>> > > > > >
> >>> > > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210510214154
> >>> > > > > > > > > >
> >>> > > > > > > > > > I checked on Hera and WCOSS but couldn't find
the
> data
> >>> > > there. I
> >>> > > > > > > > > > should be
> >>> > > > > > > > > > able to log on to jet, but when I try, it
boots me
> off:
> >>> > > > > > > > > >
> >>> > > > > > > > > > Authentication failed.
> >>> > > > > > > > > > Connection to jet-rsa.boulder.rdhpcs.noaa.gov
> closed.
> >>> > > > > > > > > >
> >>> > > > > > > > > > Can you confirm that you're running on jet?
And any
> >>> idea of
> >>> > > jet
> >>> > > > > is
> >>> > > > > > > > > > down
> >>> > > > > > > > > > today or something?
> >>> > > > > > > > > >
> >>> > > > > > > > > > Thanks,
> >>> > > > > > > > > > John
> >>> > > > > > > > > >
> >>> > > > > > > > > >
> >>> > > > > > > > > > On Tue, May 11, 2021 at 7:46 AM Yan Jin - NOAA
> >>> Affiliate
> >>> > via
> >>> > > > RT <
> >>> > > > > > > > > > met_help at ucar.edu> wrote:
> >>> > > > > > > > > >
> >>> > > > > > > > > > >
> >>> > > > > > > > > > > <URL:
> >>> > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846
> >>> > > > > >
> >>> > > > > > > > > > >
> >>> > > > > > > > > > > Hi John,
> >>> > > > > > > > > > >
> >>> > > > > > > > > > > Thanks so much!
> >>> > > > > > > > > > >
> >>> > > > > > > > > > > Best,
> >>> > > > > > > > > > > Yan
> >>> > > > > > > > > > >
> >>> > > > > > > > > > >
> >>> > > > > > > > > > > On Tue, May 11, 2021 at 9:44 AM John Halley
Gotway
> >>> via
> >>> > RT <
> >>> > > > > > > > > > > met_help at ucar.edu>
> >>> > > > > > > > > > > wrote:
> >>> > > > > > > > > > >
> >>> > > > > > > > > > > > Hi Yan,
> >>> > > > > > > > > > > >
> >>> > > > > > > > > > > > I'm going to assign this ticket to George
and ask
> >>> him
> >>> > to
> >>> > > > > take a
> >>> > > > > > > > > > > > look
> >>> > > > > > > > > > and
> >>> > > > > > > > > > > > debug this runtime error.
> >>> > > > > > > > > > > >
> >>> > > > > > > > > > > > Thanks,
> >>> > > > > > > > > > > > John HG
> >>> > > > > > > > > > > >
> >>> > > > > > > > > > > > On Mon, May 10, 2021 at 3:49 PM Yan Jin -
NOAA
> >>> > Affiliate
> >>> > > > via
> >>> > > > > > RT <
> >>> > > > > > > > > > > > met_help at ucar.edu> wrote:
> >>> > > > > > > > > > > >
> >>> > > > > > > > > > > > >
> >>> > > > > > > > > > > > > Mon May 10 15:49:51 2021: Request 99846
was
> acted
> >>> > upon.
> >>> > > > > > > > > > > > > Transaction: Ticket created by
> yan.jin at noaa.gov
> >>> > > > > > > > > > > > >        Queue: met_help
> >>> > > > > > > > > > > > >      Subject: ask a question about
METplus
> >>> config of
> >>> > TC
> >>> > > > > > > > > > > > >        Owner: Nobody
> >>> > > > > > > > > > > > >   Requestors: yan.jin at noaa.gov
> >>> > > > > > > > > > > > >       Status: new
> >>> > > > > > > > > > > > >  Ticket <URL:
> >>> > > > > > > > > >
> >>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846
> >>> > > > > > > > > > > >
> >>> > > > > > > > > > > > >
> >>> > > > > > > > > > > > >
> >>> > > > > > > > > > > > > Hi,
> >>> > > > > > > > > > > > >
> >>> > > > > > > > > > > > > This is Yan, again. I ran the METplus
config
> file
> >>> > after
> >>> > > > our
> >>> > > > > > > > > > discussion,
> >>> > > > > > > > > > > > > however, it seems that the tc_pairs tcst
file
> >>> wasn't
> >>> > > > > > generated.
> >>> > > > > > > > > > > > >
> >>> > > > > > > > > > > > > Here is the script in my directory on
Jet:
> >>> > > > > > > > > > > > > I set TC_PAIRS_STORM_ID = AL292020, so
the code
> >>> can
> >>> > > find
> >>> > > > > the
> >>> > > > > > > > > > > > > input
> >>> > > > > > > > > > > adeck
> >>> > > > > > > > > > > > > and bdeck data correctly
> >>> > > > > > > > > > > > > *----
> >>> > > > > > > > > > > > >
> >>> > > > > > > > > > > > >
> >>> > > > > > > > > > > >
> >>> > > > > > > > > > >
> >>> > > > > > > > > >
> >>> > > > > > > >
> >>> > > > > > >
> >>> > > > > >
> >>> > > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test.conf
> >>> > > > > > > > > > > > >
> >>> > > > > > > > > > > > > The output directory is
> >>> > > > > > > > > > > > >
> >>> > > > > > > >
> >>> > >
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs
> >>> > > > > > > > > > > > > and the tc_pairs data:
al292020_pairs.dat.tcst
> >>> > > > > > > > > > > > >
> >>> > > > > > > > > > > > > log file
> >>> > > > > > > > > > > > >
> >>> > > > > > > > > > > > >
> >>> > > > > > > > > > > >
> >>> > > > > > > > > > >
> >>> > > > > > > > > >
> >>> > > > > > > >
> >>> > > > > > >
> >>> > > > > >
> >>> > > > >
> >>> > > >
> >>> > >
> >>> >
> >>>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210510214154
> >>> > > > > > > > > > > > > *-----
> >>> > > > > > > > > > > > >
> >>> > > > > > > > > > > > > The error starts:[image: image.png]
> >>> > > > > > > > > > > > >
> >>> > > > > > > > > > > > > Then I checked al292020_pairs.dat.tcst,
and it
> is
> >>> > > empty.
> >>> > > > > > > > > > > > >
> >>> > > > > > > > > > > > > I wonder if there is anything I missed?
> >>> > > > > > > > > > > > >
> >>> > > > > > > > > > > > > Many thanks.
> >>> > > > > > > > > > > > > Best,
> >>> > > > > > > > > > > > > Yan
> >>> > > > > > > > > > > > >
> >>> > > > > > > > > > > > >
> >>> > > > > > > > > > > >
> >>> > > > > > > > > > > >
> >>> > > > > > > > > > >
> >>> > > > > > > > > > >
> >>> > > > > > > > > >
> >>> > > > > > > > > >
> >>> > > > > > > >
> >>> > > > > > > >
> >>> > > > > > > >
> >>> > > > > > > >
> >>> > > > > > >
> >>> > > > > > >
> >>> > > > > >
> >>> > > > > > --
> >>> > > > > > George McCabe - Software Engineer III
> >>> > > > > > National Center for Atmospheric Research
> >>> > > > > > Research Applications Laboratory
> >>> > > > > > 303-497-2768
> >>> > > > > > ---
> >>> > > > > > My working day may not be your working day. Please do
not
> feel
> >>> > > obliged
> >>> > > > to
> >>> > > > > > reply to this email outside of your normal working
hours.
> >>> > > > > >
> >>> > > > > >
> >>> > > > >
> >>> > > > >
> >>> > > >
> >>> > > > --
> >>> > > > George McCabe - Software Engineer III
> >>> > > > National Center for Atmospheric Research
> >>> > > > Research Applications Laboratory
> >>> > > > 303-497-2768
> >>> > > > ---
> >>> > > > My working day may not be your working day. Please do not
feel
> >>> obliged
> >>> > to
> >>> > > > reply to this email outside of your normal working hours.
> >>> > > >
> >>> > > >
> >>> > >
> >>> > >
> >>> >
> >>> > --
> >>> > George McCabe - Software Engineer III
> >>> > National Center for Atmospheric Research
> >>> > Research Applications Laboratory
> >>> > 303-497-2768
> >>> > ---
> >>> > My working day may not be your working day. Please do not feel
> obliged
> >>> to
> >>> > reply to this email outside of your normal working hours.
> >>> >
> >>> >
> >>>
> >>>
> >>
> >> --
> >> George McCabe - Software Engineer III
> >> National Center for Atmospheric Research
> >> Research Applications Laboratory
> >> 303-497-2768
> >> ---
> >> My working day may not be your working day. Please do not feel
obliged
> to
> >> reply to this email outside of your normal working hours.
> >>
> >
> >
> > --
> > George McCabe - Software Engineer III
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > 303-497-2768
> > ---
> > My working day may not be your working day. Please do not feel
obliged to
> > reply to this email outside of your normal working hours.
> >
>
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>

------------------------------------------------
Subject: ask a question about METplus config of TC
From: George McCabe
Time: Wed May 26 13:36:17 2021

Hi Yan,

The way the wrapper is currently written, specifying that list of
cyclones
in TC_PAIRS_CYCLONE will result in 15 output files that all contain
all 15
cyclones. I am currently working on updating the tool to handle this
properly so that each file will only contain the cyclone in question.
Your
configuration looks correct to me, but it will not produce the correct
behavior until these updates are finished. When it is ready, I can
make it
available to you on Jet so you can test that it runs as expected.

I am actually finished with this fix but I am now working on
implementing
support for setting consensus through METplus. If you'd like, I can
put the
code from my development branch on Jet so you can test that this works
while I am still working on consensus. Let me know if you'd like me to
do
this and I can make it available to you.

Thanks,
George

- George

On Wed, May 26, 2021 at 12:54 PM Yan Jin - NOAA Affiliate via RT <
met_help at ucar.edu> wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846 >
>
> Hi George,
>
> Can I ask a question regarding the seasonal mean TC verification?
For
> instance, I have adeck and bdeck data over Atlantic at 2020, (Jet:
> /mnt/lfs4/HFIP/hwrf-data/hwrf-input/abdeck), each has 01-99 cyclone
number.
> And I would like to calculate the absolute intensity error and track
error
> of seasonal mean over Atlantic. Here is my method, could you check
if it is
> correct to do that?
>
> I plan to do the May-June-July-August mean error.
> $WORK:
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/vrfy_METplus4.0/METplus/parm/user_config
> in this directory, the METplus config file is
TCMPRPlotter_AL_MJJA_2020
>
>
> The major modification I did is:
> ***
> MODEL = HWRF,HMON,GFSO
>
> TC_PAIRS_DESC =
>
> TC_PAIRS_STORM_ID =
> TC_PAIRS_BASIN = AL
> TC_PAIRS_CYCLONE = 01,02,03,04,05,06,07,08,09,10,11,12,13,14,15
> TC_PAIRS_STORM_NAME =
>
> TC_PAIRS_INIT_BEG = 20200515_000000
> TC_PAIRS_INIT_END = 20200906_000000
> ***
> I checked the adeck and bdeck data, and found that cyclone number
01-15
> have the initial times which are located within MJJA. So I
> specified TC_PAIRS_CYCLONE with 01 to 15. I also checked the initial
time
> in cyclone 01 and 15, and found the INIT_BEG and INIT_END.
>
> It generated 15 tc_pairs.tcst files, and also 6 figures (Output
directory:
> /mnt/lfs4/HFIP/hwrfv3/Yan.Jin/vrfy_METplus4.0/AL_MJJA_2020)
>
> Here is the result
>
> https://docs.google.com/document/d/10adrHdzNNxMJM51e5Sw9oy6uS2n1S-
qGS2c7NZdVVxc/edit?pli=1
>
> I wonder if my method is correct? It seems that TC_PAIRS_INIT_BEG
> and TC_PAIRS_INIT_END are mandatory for my case.
>
> Many thanks.
> Best,
> Yan
>
> On Thu, May 20, 2021 at 12:01 PM George McCabe via RT
<met_help at ucar.edu>
> wrote:
>
> > Hi Yan,
> >
> > I was able to change the line colors in the images. The images can
be
> found
> > on Jet here:
> >
> > /lfs1/HFIP/dtc-hurr/George.Mccabe/eta-out-colors/tcmpr_plots
> >
> >
> > This is the command I used to run:
> >
> > run_metplus.py
> > /lfs1/HFIP/dtc-hurr/George.Mccabe/TCMPRPlotter_Eta_colors.conf
> > config.OUTPUT_BASE=/lfs1/HFIP/dtc-hurr/George.Mccabe/eta-out-
colors
> >
> >
> > I copied the default tcmpr file from
> >
>
/contrib/met/10.0.0/share/met/Rscripts/include/plot_tcmpr_config_default.R
> > to my directory and renamed it to plot_tcmpr_overrides.R
> >
> > Then I modified the METplus config to point to that file instead
of the
> > file in the METplus repository:
> >
> > TCMPR_PLOTTER_CONFIG_FILE =
> > /lfs1/HFIP/dtc-hurr/George.Mccabe/plot_tcmpr_overrides.R
> >
> >
> > The previous tcmpr parameter file changes the image resolution, so
I had
> to
> > set that in the new file to prevent the image from being very
large.
> Then I
> > changed the value of the default color list:
> >
> > img_res = 72;
> > default_color_list  = c("blue", "green", "purple", "orange");
> >
> > This variable is used to set other variables for each plot type,
so you
> can
> > modify those variables if you desire different colors for
different
> plots.
> > You only need to supply the variables you want to change from the
default
> > value, so your overrides file can just include the above 2
variables to
> > keep the file short and clean. However, it is nice to have all of
the
> > possible options easily available
> > in case you want to change other settings. How you handle this is
up to
> > your preference.
> >
> > Let me know if you have any other questions.
> >
> > Thanks,
> > George
> >
> > On Thu, May 20, 2021 at 9:19 AM George McCabe <mccabe at ucar.edu>
wrote:
> >
> > > Hi Yan,
> > >
> > > I copied your config file and made some modifications. You can
view it
> on
> > > Jet here:
> > >
> > > /lfs1/HFIP/dtc-hurr/George.Mccabe/TCMPRPlotter_Eta.conf
> > >
> > > Starting in METplus 4.0.0 the different section headers like dir
and
> > > filename_templates are no longer necessary. All configurations
can go
> > under
> > > the [config] header. I removed the sections and rearranged the
> variables
> > to
> > > group them in a more clear way.
> > >
> > > To call TCMPRPlotter multiple times with different settings, I
added
> > > multiple instances of the app in the PROCESS_LIST with instance
names
> > > defined:
> > >
> > > PROCESS_LIST = TCPairs, TCMPRPlotter(abs_mean),
> > > TCMPRPlotter(abs_median),TCMPRPlotter(abs_boxplot),
> > TCMPRPlotter(tk_mean),
> > > TCMPRPlotter(tk_median),TCMPRPlotter(tk_boxplot)
> > >
> > > Each instance will read all of the values set in the [config]
section,
> > > then it will look for a section header that matches the string
inside
> the
> > > parentheses for each instance. If the section is found, it
overrides
> the
> > > values set in the section to replace the values set in the
[config]
> > > section. These names can be whatever you chose, but the name in
the
> > > PROCESS_LIST must match the name of the section header.
> > >
> > > [abs_mean]
> > > TCMPR_PLOTTER_PREFIX = Eta_Absolute_Intensity_mean
> > > TCMPR_PLOTTER_TITLE = Absolute Intensity Error Mean\n Eta
(Atlantic
> 2020)
> > > TCMPR_PLOTTER_DEP_VARS = ABS(AMAX_WIND-BMAX_WIND)
> > > TCMPR_PLOTTER_PLOT_TYPES = MEAN
> > >
> > > [abs_median]
> > > TCMPR_PLOTTER_PREFIX = Eta_Absolute_Intensity_median
> > > TCMPR_PLOTTER_TITLE = Absolute Intensity Error Median\n Eta
(Atlantic
> > 2020)
> > > TCMPR_PLOTTER_DEP_VARS = ABS(AMAX_WIND-BMAX_WIND)
> > > TCMPR_PLOTTER_PLOT_TYPES = MEDIAN
> > >
> > > [abs_boxplot]
> > > TCMPR_PLOTTER_PREFIX = Eta_Absolute_Intensity_boxplot
> > > TCMPR_PLOTTER_TITLE = Absolute Intensity Error Boxplot\n Eta
(Atlantic
> > > 2020)
> > > TCMPR_PLOTTER_DEP_VARS = ABS(AMAX_WIND-BMAX_WIND)
> > > TCMPR_PLOTTER_PLOT_TYPES = BOXPLOT
> > >
> > > [tk_mean]
> > > TCMPR_PLOTTER_PREFIX = Eta_Track_Error_mean
> > > TCMPR_PLOTTER_TITLE = Track Error Mean\n Eta (Atlantic 2020)
> > > TCMPR_PLOTTER_DEP_VARS = TK_ERR
> > > TCMPR_PLOTTER_PLOT_TYPES = MEAN
> > >
> > > [tk_median]
> > > TCMPR_PLOTTER_PREFIX = Eta_Track_Error_median
> > > TCMPR_PLOTTER_TITLE = Track Error Median\n Eta (Atlantic 2020)
> > > TCMPR_PLOTTER_DEP_VARS = TK_ERR
> > > TCMPR_PLOTTER_PLOT_TYPES = MEDIAN
> > >
> > > [tk_boxplot]
> > > TCMPR_PLOTTER_PREFIX = Eta_Track_Error_boxplot
> > > TCMPR_PLOTTER_TITLE = Track Error Boxplot\n Eta (Atlantic 2020)
> > > TCMPR_PLOTTER_DEP_VARS = TK_ERR
> > > TCMPR_PLOTTER_PLOT_TYPES = BOXPLOT
> > >
> > >
> > > I split up the DEP_VARS and PLOT_TYPES values into instances
with one
> > > value of each and set the plot title and filenames appropriately
for
> > each.
> > > I set the X label to "Lead Time (h)" in the [config] section so
it
> > applies
> > > to every instance.
> > >
> > > TCMPR_PLOTTER_XLAB = Lead Time (h)
> > >
> > > The output images from my run is on Jet here:
> > >
> > > /lfs1/HFIP/dtc-hurr/George.Mccabe/eta-out/tcmpr_plots
> > >
> > > Eta_Absolute_Intensity_boxplot.log
> > > Eta_Absolute_Intensity_boxplot.png
> > > Eta_Absolute_Intensity_mean.png
> > > Eta_Absolute_Intensity_median.png
> > > Eta_Track_Error_boxplot.log
> > > Eta_Track_Error_boxplot.png
> > > Eta_Track_Error_mean.png
> > > Eta_Track_Error_median.png
> > >
> > >
> > > Here is the log file from the run:
> > >
> /lfs1/HFIP/dtc-hurr/George.Mccabe/eta-
out/logs/metplus.log.20210520150804
> > >
> > >
> > > Here is what I ran to produce the output:
> > >
> > >
> > >    - module load intel
> > >    - module load intelpython/3.6.5
> > >    - module load netcdf/4.6.1
> > >    - module load hdf5/1.10.4
> > >    - module load nco/4.9.1
> > >    - module load wgrib/1.8.1.0b
> > >    - module load wgrib2/2.0.8
> > >    - module load R/4.0.2
> > >    - module use /contrib/met/modulefiles
> > >    - module load met/10.0.0
> > >    - module use /contrib/met/METplus/modulefiles
> > >    - module load metplus/4.0.0
> > >    - run_metplus.py
> > >    /lfs1/HFIP/dtc-hurr/George.Mccabe/TCMPRPlotter_Eta.conf
> > >    config.OUTPUT_BASE=/lfs1/HFIP/dtc-hurr/George.Mccabe/eta-out
> > >
> > >
> > >
> > > I will look into what settings are needed to change the line
colors.
> > Also,
> > > I have started making the modifications to the TCPairs wrapper
to
> > properly
> > > handle multiple storms specified in the config file. I will let
you
> know
> > > when that is ready to test.
> > >
> > > Thanks,
> > > George
> > >
> > >
> > >
> > > On Wed, May 19, 2021 at 5:09 PM George McCabe <mccabe at ucar.edu>
wrote:
> > >
> > >> Hi Yan,
> > >>
> > >> 1) add TC name to each plot name
> > >> 2) modify titles to each plot, e.g.: TK_ERR* plots with name
"Track
> > >> Error", and ABS_AMAX_WIND-BMAX_WIND_* plots with name "Absolute
> > Intensity
> > >> Error"?
> > >>
> > >> It looks like the logic in the R script uses the -prefix value
to
> > replace
> > >> the filename instead of prepending it to the plot type text as
you
> would
> > >> guess it would. I was told that this script was typically
called
> > multiple
> > >> times to generate the multiple plots, which would provide more
control
> > over
> > >> setting the output filenames, titles, etc. The bad news is it
looks
> like
> > >> the wrapper was written to call the R script once with all of
the
> > >> configurations specified in the METplus config file. The good
news is
> > there
> > >> are some new capabilities added to METplus 4.0.0 that could
help
> produce
> > >> the results you need.
> > >>
> > >> Please review this section of the METplus User's Guide,
specifically
> the
> > >> section about specifying instance names to the processes:
> > >>
> > >>
> >
>
https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#process-
list
> > >>
> > >> Alternatively, you can use the UserScript wrapper to build
custom
> > >> commands to call the R script directly with any arguments you
need if
> > the
> > >> wrapper doesn't support an option you need.
> > >> More information on that tool is here:
> > >>
> >
>
https://metplus.readthedocs.io/en/latest/Users_Guide/wrappers.html#userscript
> > >> and here:
> > >>
> >
>
https://metplus.readthedocs.io/en/latest/generated/met_tool_wrapper/UserScript/UserScript_run_once.html#sphx-
glr-generated-met-tool-wrapper-userscript-userscript-run-once-py
> > >>
> > >> I think using different instance names to run TCMPRPlotter
multiple
> > times
> > >> with different settings is the better approach in this case. I
am
> > wrapping
> > >> up work for the day, but tomorrow I can log into Jet and help
you
> modify
> > >> your config file to do this.
> > >>
> > >>
> > >> 3) Is it possible to modify the line colors in this METplus
config
> file?
> > >>
> > >> Yes, you can change the line colors through the R script's
config
> file.
> > >> You can copy and rename this file:
> > >>
> > >>
> > >>
> >
>
https://github.com/dtcenter/MET/blob/main_v10.0/met/scripts/Rscripts/include/plot_tcmpr_config_default.R
> > >>
> > >> change the values that you need to change (look for
"color_list"),
> then
> > >> set the METplus configuration file to point to your new file
(set in
> > >> TCMPR_PLOTTER_CONFIG_FILE).
> > >>
> > >>
> > >> Let me know if you have any other questions. I will report back
> tomorrow
> > >> when have some changes for you to test.
> > >>
> > >> Thanks,
> > >> George
> > >>
> > >> On Wed, May 19, 2021 at 4:00 PM Yan Jin - NOAA Affiliate via RT
<
> > >> met_help at ucar.edu> wrote:
> > >>
> > >>>
> > >>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846
>
> > >>>
> > >>> Hi George,
> > >>>
> > >>> I have a few questions regarding the TCMPR_PLOTTER_*
parameters in
> the
> > >>> METplus config file.
> > >>>
> > >>> Here is the METplus config file I used:
> > >>>
> > >>>
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/vrfy_METplus4.0/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test.conf
> > >>> **
> > >>> #  Plot_TCMPR options, if left unset, default values that are
> > >>> #  pre-defined in the R utility (packaged with MET) will be
used.
> > >>> TCMPR_PLOTTER_CONFIG_FILE =
{CONFIG_DIR}/TCMPRPlotterConfig_customize
> > >>> TCMPR_PLOTTER_PREFIX =
> > >>> TCMPR_PLOTTER_TITLE =
> > >>> TCMPR_PLOTTER_SUBTITLE =
> > >>> TCMPR_PLOTTER_XLAB =
> > >>> TCMPR_PLOTTER_YLAB =
> > >>> TCMPR_PLOTTER_XLIM =
> > >>> TCMPR_PLOTTER_YLIM =
> > >>> TCMPR_PLOTTER_FILTER = -amodel HWRF,HMON
> > >>> # the tcst data file to be used instead of running the MET
tc_stat
> > tool.
> > >>> TCMPR_PLOTTER_FILTERED_TCST_DATA_FILE =
> > >>> # Comma separated, no whitespace.  Default is TK_ERR (track
error)
> > unless
> > >>> # otherwise indicated.
> > >>> TCMPR_PLOTTER_DEP_VARS = ABS(AMAX_WIND-BMAX_WIND),TK_ERR
> > >>> TCMPR_PLOTTER_SCATTER_X =
> > >>> TCMPR_PLOTTER_SCATTER_Y =
> > >>> TCMPR_PLOTTER_SKILL_REF =
> > >>> TCMPR_PLOTTER_SERIES = AMODEL
> > >>> TCMPR_PLOTTER_SERIES_CI = TRUE
> > >>> TCMPR_PLOTTER_LEGEND =
> > >>> TCMPR_PLOTTER_LEAD = 0,12,24,36,48,60,72,84,96,108,120
> > >>> # Mean and median plots. These override the plot_tcmpr.R
default of
> box
> > >>> plot.
> > >>> # If box plot is desired, this needs to be explicitly
indicated.
> > >>> TCMPR_PLOTTER_PLOT_TYPES = MEAN,MEDIAN,BOXPLOT
> > >>> TCMPR_PLOTTER_RP_DIFF =
> > >>> TCMPR_PLOTTER_DEMO_YR =
> > >>> TCMPR_PLOTTER_HFIP_BASELINE =
> > >>> TCMPR_PLOTTER_FOOTNOTE_FLAG =
> > >>> TCMPR_PLOTTER_PLOT_CONFIG_OPTS =
> > >>> TCMPR_PLOTTER_SAVE_DATA =
> > >>>
> > >>> #  TCMPR FLAGS no == (don't set flag), yes == (set flag)
> > >>> TCMPR_PLOTTER_NO_EE = no
> > >>> TCMPR_PLOTTER_NO_LOG = no
> > >>> TCMPR_PLOTTER_SAVE = no
> > >>> **
> > >>> It generated the following 6 *.png files
> > >>> in
> > >>>
> >
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/vrfy_METplus4.0/Eta_use_METplus/tcmpr_plots
> > >>> [image: image.png]
> > >>>
> > >>> Then I would like to modify titles, xlab text, output file
names.
> > >>> Here are the modification I
> > >>> did
> > >>>
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/vrfy_METplus4.0/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test1.conf
> > >>>
> > >>> **
> > >>> TCMPR_PLOTTER_TITLE = Absolute Intensity Error \n Eta
(Atlantic 2020)
> > >>> TCMPR_PLOTTER_XLAB = Lead Time (h)
> > >>> **
> > >>> The title of each plot becomes "Absolute Intensity Error Eta
> (Atlantic
> > >>> 2020)" with double quotes.
> > >>>
> > >>> I also tried to set TCMPR_PLOTTER_PREFIX = Eta_ because I
would like
> to
> > >>> save each plot as Eta_ABS_AMAX_WIND-BMAX_WIND_*.png, and
> > >>> Eta_TK_ERR_*.png.
> > >>> However, there is only one plot Eta_.png generated.
> > >>>
> > >>> I would like to check how can I set those parameters if I
would like
> to
> > >>> 1)
> > >>> add TC name to each plot name; 2) modify titles to each plot,
e.g.:
> > >>> TK_ERR*
> > >>> plots with name "Track Error", and ABS_AMAX_WIND-BMAX_WIND_*
plots
> with
> > >>> name "Absolute Intensity Error"? 3) Is it possible to modify
the line
> > >>> colors in this METplus config file?
> > >>>
> > >>> Could you give me some suggestions?
> > >>>
> > >>> Many thanks.
> > >>> Best,
> > >>> Yan
> > >>>
> > >>> On Mon, May 17, 2021 at 10:37 AM George McCabe via RT <
> > met_help at ucar.edu
> > >>> >
> > >>> wrote:
> > >>>
> > >>> > Hi Yan,
> > >>> >
> > >>> > 4.1:
> > >>> > The consensus variable is not yet controlled by a METplus
> > configuration
> > >>> > variable explicitly, but you can set the value by using the
> > >>> > TC_PAIRS_MET_CONFIG_OVERRIDES value (in v4.0.0+). To set the
> > consensus
> > >>> > value to what you have set in your config file, you would
set
> > something
> > >>> > like this:
> > >>> >
> > >>> > TC_PAIRS_MET_CONFIG_OVERRIDES = consensus = [ { name =
"CONS";
> > members
> > >>> =
> > >>> > ["HWRF","GFSO","HMON"]; required = [false,false,false];
min_req =
> 2;
> > }
> > >>> ];
> > >>> >
> > >>> > I will make a note in the GitHub issue for improvements to
the
> > TCPairs
> > >>> > wrapper (https://github.com/dtcenter/METplus/issues/749) to
add
> > >>> support
> > >>> > for
> > >>> > consensus explicitly via something like TC_PAIRS_CONSENSUS
so that
> it
> > >>> is
> > >>> > easier to configure via METplus wrappers.
> > >>> >
> > >>> > 4.2:
> > >>> > The System Configuration section of the METplus User's Guide
covers
> > >>> these
> > >>> > variables and how they affect execution. I suggest that you
read
> > >>> through
> > >>> > this section to get a full picture of how these variables
work.
> > >>> Regarding
> > >>> > your specific question, these 2 sections should help:
> > >>> >
> > >>> > LOOP_BY:
> > >>> >
> > >>> >
> > >>>
> >
>
https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#loop-
by
> > >>> > LOOP_ORDER:
> > >>> >
> > >>> >
> > >>>
> >
>
https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#loop-
order
> > >>> >
> > >>> > Thanks,
> > >>> > George
> > >>> >
> > >>> > On Fri, May 14, 2021 at 8:19 AM Yan Jin - NOAA Affiliate via
RT <
> > >>> > met_help at ucar.edu> wrote:
> > >>> >
> > >>> > >
> > >>> > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846 >
> > >>> > >
> > >>> > > Hi George,
> > >>> > >
> > >>> > > Thanks so much for your help. Finally the code can run
without
> > >>> errors.
> > >>> > Here
> > >>> > > is the steps and results:
> > >>> > >
> > >>> > >
> > >>> > >
> > >>> >
> > >>>
> >
> https://docs.google.com/document/d/1xRF3aiPATcX5G37ngJj-
65pl44LN7mGHo34Sf8i3zpo/edit
> > >>> > >
> > >>> > > I also put questions in the document, I also pasted here:
> > >>> > >
> > >>> > > Questions:
> > >>> > >
> > >>> > > 4.1: How can I do the consensus forecasts in METplus
config file
> as
> > >>> I did
> > >>> > > in tc_pairs?
> > >>> > >
> > >>> > >
> > >>> >
> > >>>
> >
>
(/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_292020/TCPairsConfig_run1)
> > >>> > >
> > >>> > > //
> > >>> > >
> > >>> > > // Specify how consensus forecasts should be defined.
> > >>> > >
> > >>> > > //
> > >>> > >
> > >>> > > consensus = [
> > >>> > >
> > >>> > >  {
> > >>> > >
> > >>> > >   name = "CONS";
> > >>> > >
> > >>> > >   members = ["HWRF","GFSO","HMON"];
> > >>> > >
> > >>> > >   required = [false,false,false];
> > >>> > >
> > >>> > >   min_req = 2;
> > >>> > >
> > >>> > >  }
> > >>> > >
> > >>> > > ];
> > >>> > >
> > >>> > > 4.2: I am a little confused about
> > >>> > >
> > >>> > >       ***
> > >>> > >
> > >>> > >        LOOP_ORDER = processes
> > >>> > >
> > >>> > >        LOOP_BY = INIT
> > >>> > >
> > >>> > >       ***
> > >>> > >
> > >>> > > Could you explain these two command lines and give me some
> > examples?
> > >>> > >
> > >>> > > Thanks very much for your time and help on my questions.
> > >>> > >
> > >>> > > Best,
> > >>> > > Yan
> > >>> > >
> > >>> > > On Thu, May 13, 2021 at 3:04 PM George McCabe via RT <
> > >>> met_help at ucar.edu>
> > >>> > > wrote:
> > >>> > >
> > >>> > > > Hi Yan,
> > >>> > > >
> > >>> > > > The produtil is due to a bug in the wrapper in the logic
that
> > >>> reports
> > >>> > > when
> > >>> > > > an error occurs. I ran the command on Jet by hand and it
looks
> > >>> like it
> > >>> > is
> > >>> > > > still not finding any input data to plot. You can run
the
> > following
> > >>> > > > commands (after loading all of the required modules):
> > >>> > > >
> > >>> > > > export MET_INSTALL_DIR=/contrib/met/9.1
> > >>> > > >
> > >>> > > > Rscript /contrib/met/9.1/share/met/Rscripts/plot_tcmpr.R
\
> > >>> > > > -config
> > >>> > > >
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> /contrib/met/METplus/METplus-
3.1.1/parm/use_cases/model_applications/tc_and_extra_tc/TCMPRPlotterConfig_customize
> > >>> > > > \
> > >>> > > > -subtitle "Your subtitle goes here" -ylab "Your y-label
goes
> > here"
> > >>> \
> > >>> > > > -dep AMAX_WIND-BMAX_WIND,TK_ERR -legend "Your legend
text goes
> > >>> > here..." \
> > >>> > > > -plot MEAN,MEDIAN,BOXPLOT \
> > >>> > > > -lookin
> > >>> > > >
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> > >>> > > > \
> > >>> > > >
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
> > >>> > > > \
> > >>> > > >
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
> > >>> > > > \
> > >>> > > > -outdir
> > >>> > > >
> > >>>
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tcmpr_plots
> > >>> > > >
> > >>> > > >
> > >>> > > > Here is the output from that run:
> > >>> > > >
> > >>> > > > Reading plot configuration file:
> > >>> > > >
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
> /contrib/met/METplus/METplus-
3.1.1/parm/use_cases/model_applications/tc_and_extra_tc/TCMPRPlotterConfig_customize
> > >>> > > > No plotting configuration file specified.
> > >>> > > > CALLING:  /contrib/met/9.1/bin/tc_stat -lookin
> > >>> > > >
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> > >>> > > > -lookin
> > >>> > > >
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
> > >>> > > > -lookin
> > >>> > > >
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
> > >>> > > > -job filter -dump_row /tmp/plot_tcmpr_116966.tcst
-event_equal
> > >>> true
> > >>> > -v 3
> > >>> > > > DEBUG 1: [Source 1] Lookin:
> > >>> > > >
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> > >>> > > > DEBUG 1: [Source 2] Lookin:
> > >>> > > >
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
> > >>> > > > DEBUG 1: [Source 3] Lookin:
> > >>> > > >
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
> > >>> > > > DEBUG 2:
> > >>> > > > DEBUG 2: Processing Job 1: -job filter -event_equal true
> > -dump_row
> > >>> > > > /tmp/plot_tcmpr_116966.tcst
> > >>> > > > DEBUG 3: Applying track-based event equalization logic.
> > >>> > > > DEBUG 3: Reading file 1 of 3:
> > >>> > > >
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> > >>> > > > DEBUG 3: Reading file 2 of 3:
> > >>> > > >
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
> > >>> > > > DEBUG 3: Reading file 3 of 3:
> > >>> > > >
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
> > >>> > > > DEBUG 3: For track-based event equalization, identified
0
> common
> > >>> cases
> > >>> > > for
> > >>> > > > 146 models: AC00 AEMI AEMN AP01 AP02 AP03 AP04 AP05 AP06
AP07
> > AP08
> > >>> AP09
> > >>> > > > AP0I AP10 AP11 AP12 AP13 AP14 AP15 AP16 AP17 AP18 AP19
AP1I
> AP20
> > >>> AP21
> > >>> > > AP22
> > >>> > > > AP23 AP24 AP25 AP26 AP27 AP28 AP29 AP30 API CARQ CEM2
CEMI CEMN
> > >>> CLP5
> > >>> > CMC
> > >>> > > > CMC2 CMCI COTC COTI CTCX DRCL DSHP EC00 EEMN EGRR EMN2
EMN3
> EMN4
> > >>> EMNI
> > >>> > EMX
> > >>> > > > EMX2 EMXI EN01 EN02 EN03 EN04 EN05 EN06 EN07 EN08 EN09
EN0I
> EN10
> > >>> EN11
> > >>> > > EN12
> > >>> > > > EN13 EN14 EN15 EN16 EN17 EN18 EN19 EN1I EN20 EN21 EN22
EN23
> EN24
> > >>> EN25
> > >>> > ENI
> > >>> > > > EP01 EP02 EP03 EP04 EP05 EP06 EP07 EP08 EP09 EP0I EP10
EP11
> EP12
> > >>> EP13
> > >>> > > EP14
> > >>> > > > EP15 EP16 EP17 EP18 EP19 EP1I EP20 EP21 EP22 EP23 EP24
EP25 EPI
> > >>> GFEX
> > >>> > GFSI
> > >>> > > > GFSO HMNI HMON HWF2 HWFI HWRF ICON IVCN LGEM NGX NGX2
NGXI NVGM
> > >>> OCD5
> > >>> > RVCN
> > >>> > > > SHF5 SHIP TABD TABM TABS TCLP TVCA TVCE TVCN TVCX UKX
UKX2 UKXI
> > >>> XTRP
> > >>> > > > DEBUG 1: Event equalization of tracks found no common
cases.
> > >>> > > > DEBUG 1: Creating output dump file:
/tmp/plot_tcmpr_116966.tcst
> > >>> > > > FILTER: -job filter -line_type TCMPR -event_equal true
> -dump_row
> > >>> > > > /tmp/plot_tcmpr_116966.tcst
> > >>> > > >
> > >>> > > > DEBUG 2: Job 1 used 0 out of 0 lines read.
> > >>> > > > DEBUG 3: Total lines read                 = 0
> > >>> > > > DEBUG 3: Total lines kept                 = 0
> > >>> > > > DEBUG 3: Rejected for track watch/warn    = 0
> > >>> > > > DEBUG 3: Rejected for init threshold      = 0
> > >>> > > > DEBUG 3: Rejected for init string         = 0
> > >>> > > > DEBUG 3: Rejected for out init mask       = 0
> > >>> > > > DEBUG 3: Rejected for water only          = 0
> > >>> > > > DEBUG 3: Rejected for rapid inten         = 0
> > >>> > > > DEBUG 3: Rejected for landfall            = 0
> > >>> > > > DEBUG 3: Rejected for amodel              = 0
> > >>> > > > DEBUG 3: Rejected for bmodel              = 0
> > >>> > > > DEBUG 3: Rejected for desc                = 0
> > >>> > > > DEBUG 3: Rejected for storm id            = 0
> > >>> > > > DEBUG 3: Rejected for basin               = 0
> > >>> > > > DEBUG 3: Rejected for cyclone             = 0
> > >>> > > > DEBUG 3: Rejected for storm name          = 0
> > >>> > > > DEBUG 3: Rejected for init time           = 0
> > >>> > > > DEBUG 3: Rejected for init hour           = 0
> > >>> > > > DEBUG 3: Rejected for lead time           = 0
> > >>> > > > DEBUG 3: Rejected for required lead times = 0
> > >>> > > > DEBUG 3: Rejected for valid time          = 0
> > >>> > > > DEBUG 3: Rejected for valid hour          = 0
> > >>> > > > DEBUG 3: Rejected for init mask           = 0
> > >>> > > > DEBUG 3: Rejected for valid mask          = 0
> > >>> > > > DEBUG 3: Rejected for line type           = 0
> > >>> > > > DEBUG 3: Rejected for numeric threshold   = 0
> > >>> > > > DEBUG 3: Rejected for string matching     = 0
> > >>> > > > DEBUG 3: Rejected for match points        = 0
> > >>> > > > DEBUG 3: Rejected for event equal         = 0
> > >>> > > > DEBUG 3: Rejected for out init mask       = 0
> > >>> > > > DEBUG 3: Rejected for out valid mask      = 0
> > >>> > > > Reading track data: /tmp/plot_tcmpr_116966.tcst
> > >>> > > > Error in read.table(tcst_tmp_file, header = TRUE) :
> > >>> > > >   no lines available in input
> > >>> > > > Execution halted
> > >>> > > >
> > >>> > > >
> > >>> > > >
> > >>> > > > It sounds like filter arguments may need to be provided
to
> > produce
> > >>> > > TC-Stat
> > >>> > > > output (which is called by the R script). You can set
this in
> > your
> > >>> > > METplus
> > >>> > > > config file with the TCMPR_PLOTTER_FILTER variable, but
it will
> > be
> > >>> > > > difficult to debug since the log output is not shown. I
would
> > >>> recommend
> > >>> > > > modifying the Rscript call from this email and add the
-filter
> > >>> argument
> > >>> > > > with any filter information to pass to TCStat. I am not
sure
> > >>> settings
> > >>> > > would
> > >>> > > > help you produce output, but I have reached out to other
> METplus
> > >>> team
> > >>> > > > members for suggestions.
> > >>> > > >
> > >>> > > > Thanks,
> > >>> > > > George
> > >>> > > >
> > >>> > > >
> > >>> > > >
> > >>> > > > On Thu, May 13, 2021 at 12:37 PM Yan Jin - NOAA
Affiliate via
> RT
> > <
> > >>> > > > met_help at ucar.edu> wrote:
> > >>> > > >
> > >>> > > > >
> > >>> > > > > <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846
> > >
> > >>> > > > >
> > >>> > > > > Hi George,
> > >>> > > > >
> > >>> > > > > Thanks so much. Now tc_pairs works, and the output
file can
> be
> > >>> > > generated.
> > >>> > > > >
> > >>> > > > >
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> > >>> > > > >
> > >>> > > > > Then I set "PROCESS_LIST = TCPairs, TCMPRPlotter", and
I kept
> > >>> most
> > >>> > > > > parameters that initialized with "TCMPR_PLOTTER" the
same as
> > the
> > >>> > > default
> > >>> > > > > configuration.
> > >>> > > > >
> > >>> > > > > my METplus config file:
> > >>> > > > >
> > >>> > > > >
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test.conf
> > >>> > > > >
> > >>> > > > > then run script
> > >>> > > > >
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/user_config/run.sh
> > >>> > > > >
> > >>> > > > > However, the Rscript cannot plot figures.
> > >>> > > > > I checked the log file:
> > >>> > > > >
> > >>> > > > >
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210513182819
> > >>> > > > > [image: image.png]
> > >>> > > > >
> > >>> > > > > It seems that the error "NameError: name 'produtil' is
not
> > >>> defined"?
> > >>> > > > >
> > >>> > > > > Could you give me some suggestions?
> > >>> > > > >
> > >>> > > > > Many thanks.
> > >>> > > > > Best,
> > >>> > > > > Yan
> > >>> > > > >
> > >>> > > > > On Wed, May 12, 2021 at 5:20 PM George McCabe via RT <
> > >>> > > met_help at ucar.edu>
> > >>> > > > > wrote:
> > >>> > > > >
> > >>> > > > > > Hi Yan,
> > >>> > > > > >
> > >>> > > > > > It looks like the values set for init_beg and
init_end in
> the
> > >>> > TCpairs
> > >>> > > > > > config file are both "20200101_000000" because the
INIT_BEG
> > and
> > >>> > > > INIT_END
> > >>> > > > > > time is 2020. There are no adeck tracks for this
time, so
> all
> > >>> of
> > >>> > the
> > >>> > > > data
> > >>> > > > > > is being filtered out.
> > >>> > > > > >
> > >>> > > > > > This is a little confusing in METplus 3.1 because
INIT_BEG
> > and
> > >>> > > INIT_END
> > >>> > > > > > control the time looping of execution (if applicable
for
> the
> > >>> use
> > >>> > > case)
> > >>> > > > > AND
> > >>> > > > > > the values passed to the wrapped TCPairs config
file. In
> > >>> METplus
> > >>> > > 4.0.0
> > >>> > > > > > (which was released yesterday and should be
available on
> jet
> > >>> > soon!),
> > >>> > > > you
> > >>> > > > > > can set INIT_BEG and INIT_END to control time
looping (if
> > >>> desired)
> > >>> > > and
> > >>> > > > > use
> > >>> > > > > > TC_PAIRS_INIT_BEG and TC_PAIRS_INIT_END to specify
the
> values
> > >>> to
> > >>> > pass
> > >>> > > > to
> > >>> > > > > > the TCPairs config file.
> > >>> > > > > >
> > >>> > > > > > If you were using METplus 4.0.0, I would recommend
setting
> > >>> > something
> > >>> > > > > like:
> > >>> > > > > >
> > >>> > > > > >
> > >>> > > > > > LOOP_BY = INIT
> > >>> > > > > > INIT_TIME_FMT = %Y
> > >>> > > > > > INIT_BEG = 2020
> > >>> > > > > > INIT_END = 2020
> > >>> > > > > > INIT_INCREMENT = 21600
> > >>> > > > > >
> > >>> > > > > > TC_PAIRS_INIT_BEG = 20201030_000000
> > >>> > > > > > TC_PAIRS_INIT_END = 20201031_000000
> > >>> > > > > >
> > >>> > > > > >
> > >>> > > > > > This would run once using 2020 to find filenames and
> process
> > >>> data
> > >>> > > from
> > >>> > > > > the
> > >>> > > > > > files that are initialized between 20201030_000000
and
> > >>> > > 20201031_000000.
> > >>> > > > > >
> > >>> > > > > > If using METplus 3.1, you will have to use the
INIT_BEG and
> > >>> > INIT_END
> > >>> > > > > values
> > >>> > > > > > to set the values in the wrapped TCPairs config.
This is OK
> > for
> > >>> > this
> > >>> > > > use
> > >>> > > > > > case because you have LOOP_ORDER set to processes
and
> TCPairs
> > >>> only
> > >>> > > runs
> > >>> > > > > > once with that setting.
> > >>> > > > > >
> > >>> > > > > > LOOP_BY = INIT
> > >>> > > > > > INIT_TIME_FMT = %Y%m%d
> > >>> > > > > > INIT_BEG = 20201030
> > >>> > > > > > INIT_END = 20201031
> > >>> > > > > >
> > >>> > > > > > This configuration should run and only use tracks
> initialized
> > >>> > between
> > >>> > > > > > 2020-10-30 @ 0Z and 2020-10-31 @ 0Z. If you need to
specify
> > the
> > >>> > init
> > >>> > > > time
> > >>> > > > > > range with hour precision, you can do that as well.
Here is
> > an
> > >>> > > example:
> > >>> > > > > >
> > >>> > > > > > LOOP_BY = INIT
> > >>> > > > > > INIT_TIME_FMT = %Y%m%d%H
> > >>> > > > > > INIT_BEG = 2020103012
> > >>> > > > > > INIT_END = 2020103112
> > >>> > > > > >
> > >>> > > > > > This configuration will only use tracks initialized
between
> > >>> > > 2020-10-30
> > >>> > > > @
> > >>> > > > > > 12Z and 2020-10-31 @ 12Z.
> > >>> > > > > >
> > >>> > > > > > You can also restrict the data used by valid time as
well
> by
> > >>> using
> > >>> > > > > > TC_PAIRS_VALID_BEG and TC_PAIRS_VALID_END in METplus
4.0.0
> or
> > >>> > > VALID_BEG
> > >>> > > > > and
> > >>> > > > > > VALID_END for 3.1.
> > >>> > > > > >
> > >>> > > > > > Let me know if you are still unable to produce
output with
> > >>> these
> > >>> > > > > changes. I
> > >>> > > > > > was able to run tc_pairs by hand on Jet and produce
a file
> > with
> > >>> > > output
> > >>> > > > > > tracks. You can view it here if you'd like:
> > >>> > > > > >
> > >>> > > > > > /lfs1/HFIP/dtc-
hurr/George.Mccabe/al292020_pairs.dat.tcst
> > >>> > > > > >
> > >>> > > > > > These are the commands I used:
> > >>> > > > > >
> > >>> > > > > > export BASIN="[]";
> > >>> > > > > > export CYCLONE="[]";
> > >>> > > > > > export
DLAND_FILE="/contrib/met/9.1/share/met/tc_data/
> > >>> > > > > > dland_global_tenth_degree.nc";
> > >>> > > > > > export INIT_BEG="20201030_000000";
> > >>> > > > > > export INIT_END="20201031_000000";
> > >>> > > > > > export INIT_EXCLUDE="[]";
> > >>> > > > > > export INIT_INCLUDE="[]";
> > >>> > > > > > export MODEL="[]";
> > >>> > > > > > export STORM_ID="[\"AL292020\"]";
> > >>> > > > > > export STORM_NAME="[]";
> > >>> > > > > > export VALID_BEG="";
> > >>> > > > > > export VALID_END="";
> > >>> > > > > > /contrib/met/9.1/bin/tc_pairs -v 4 -bdeck
> > >>> > > > > >
> /mnt/lfs4/HFIP/hwrf-data/hwrf-input/abdeck/btk/bal292020.dat
> > >>> -adeck
> > >>> > > > > >
> /mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/data/aal292020.dat
> > >>> > -config
> > >>> > > > > >
> > >>> > > > > >
> > >>> > > > >
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/use_cases/model_applications/tc_and_extra_tc/TCPairsETCConfig
> > >>> > > > > > -out /lfs1/HFIP/dtc-
hurr/George.Mccabe/al292020_pairs.dat
> > >>> > > > > >
> > >>> > > > > > I took the values from your METplus log file and
modified
> > them
> > >>> to
> > >>> > > write
> > >>> > > > > to
> > >>> > > > > > my directory and use a different init time range.
> > >>> > > > > >
> > >>> > > > > > Thanks,
> > >>> > > > > > George
> > >>> > > > > >
> > >>> > > > > >
> > >>> > > > > >
> > >>> > > > > >
> > >>> > > > > > On Wed, May 12, 2021 at 2:04 PM Yan Jin - NOAA
Affiliate
> via
> > >>> RT <
> > >>> > > > > > met_help at ucar.edu> wrote:
> > >>> > > > > >
> > >>> > > > > > >
> > >>> > > > > > > <URL:
> > >>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846 >
> > >>> > > > > > >
> > >>> > > > > > > Hi George,
> > >>> > > > > > >
> > >>> > > > > > > Thanks very much for your instructions. I set
> > "PROCESS_LIST =
> > >>> > > > TCPairs",
> > >>> > > > > > and
> > >>> > > > > > > didn't specify models: "MODEL =". Then run the
script.
> > >>> However,
> > >>> > the
> > >>> > > > > > output
> > >>> > > > > > > file is still empty. It seems that tc_pairs
matched 0
> adeck
> > >>> track
> > >>> > > to
> > >>> > > > 1
> > >>> > > > > > > bdeck track. I also tried to set "MODEL = HWRF,
HMON",
> but
> > >>> the
> > >>> > > output
> > >>> > > > > > file
> > >>> > > > > > > is empty too.
> > >>> > > > > > >
> > >>> > > > > > > Here is the log file:
> > >>> > > > > > >
> > >>> > > > > > >
> > >>> > > > > >
> > >>> > > > >
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210512193152
> > >>> > > > > > >
> > >>> > > > > > > And here is the output file:
> > >>> > > > > > >
> > >>> > > > > > >
> > >>> > > > > >
> > >>> > > > >
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> > >>> > > > > > >
> > >>> > > > > > > Many thanks.
> > >>> > > > > > > Best,
> > >>> > > > > > > Yan
> > >>> > > > > > >
> > >>> > > > > > > On Wed, May 12, 2021 at 11:36 AM George McCabe via
RT <
> > >>> > > > > met_help at ucar.edu
> > >>> > > > > > >
> > >>> > > > > > > wrote:
> > >>> > > > > > >
> > >>> > > > > > > > Hi Yan,
> > >>> > > > > > > >
> > >>> > > > > > > > I was able to log into Jet this morning and look
at
> your
> > >>> error.
> > >>> > > > > > > >
> > >>> > > > > > > > It looks like the error in the call to Rscript
is due
> to
> > >>> there
> > >>> > > > being
> > >>> > > > > no
> > >>> > > > > > > > data in the input files generated by TCPairs.
There are
> > >>> > > > > inconsistencies
> > >>> > > > > > > in
> > >>> > > > > > > > the TCMPRPlotter wrapper that prevent the log
output
> from
> > >>> the
> > >>> > run
> > >>> > > > to
> > >>> > > > > be
> > >>> > > > > > > > seen, which would have made it more clear why
the issue
> > was
> > >>> > > > > occurring.
> > >>> > > > > > > >
> > >>> > > > > > > > I would recommend changing the PROCESS_LIST
value in
> your
> > >>> > METplus
> > >>> > > > > > > > configuration file to only run TCPairs and
iterating on
> > >>> that
> > >>> > > until
> > >>> > > > > you
> > >>> > > > > > > get
> > >>> > > > > > > > output from that tool. Once you can confirm that
you
> have
> > >>> > output,
> > >>> > > > you
> > >>> > > > > > can
> > >>> > > > > > > > add TCMPRPlotter back to the PROCESS_LIST and
test on
> > that
> > >>> > > portion.
> > >>> > > > > > > >
> > >>> > > > > > > > I will create a GitHub issue for updating the
> > TCMPRPlotter
> > >>> > > wrapper
> > >>> > > > to
> > >>> > > > > > > > produce useful logging output that will make
things
> more
> > >>> clear
> > >>> > > > going
> > >>> > > > > > > > forward.
> > >>> > > > > > > >
> > >>> > > > > > > > Let me know if you run into any other issues or
need
> help
> > >>> > > > configuring
> > >>> > > > > > > > TCPairs to run properly. I'm also happy to
attend
> another
> > >>> > working
> > >>> > > > > > session
> > >>> > > > > > > > if that is helpful.
> > >>> > > > > > > >
> > >>> > > > > > > > Thanks,
> > >>> > > > > > > > George
> > >>> > > > > > > >
> > >>> > > > > > > >
> > >>> > > > > > > > On Tue May 11 10:05:06 2021, yan.jin at noaa.gov
wrote:
> > >>> > > > > > > > > Hi John,
> > >>> > > > > > > > >
> > >>> > > > > > > > >
> > >>> > > > > > > > > The directories and data I mentioned in my
last email
> > >>> are all
> > >>> > > > > located
> > >>> > > > > > > > > in
> > >>> > > > > > > > > Jet.
> > >>> > > > > > > > >
> > >>> > > > > > > > >
> > >>> > > > > > > > > Here is the email I got regarding the Jet
> maintenance:
> > >>> > > > > > > > >
> > >>> > > > > > > > > ***
> > >>> > > > > > > > >
> > >>> > > > > > > > > *05/11/21 - Boulder Bastion Maintenance
Downtime*
> > >>> > > > > > > > > Boulder Bastion maintenance is scheduled for
Tuesday
> > >>> 05/11/21
> > >>> > > > from
> > >>> > > > > > > > > 0600 ~
> > >>> > > > > > > > > 1000 MT. Boulder Bastion login sessions will
be
> dropped
> > >>> and
> > >>> > all
> > >>> > > > new
> > >>> > > > > > > > > RDHPCS
> > >>> > > > > > > > > logins will need to go through the Princeton
Bastion.
> > >>> > Logging
> > >>> > > In
> > >>> > > > > > > > > <
> > >>> > > > > >
> > >>> >
> https://rdhpcs-common-docs.rdhpcs.noaa.gov/wiki/index.php/Logging_in
> > >>> > > >
> > >>> > > > > > > > >
> > >>> > > > > > > > >
> > >>> > > > > > > > >
> > >>> > > > > > > > > *The next Jet Maintenance Downtime is TBD*
> > >>> > > > > > > > >
> > >>> > > > > > > > > ***
> > >>> > > > > > > > >
> > >>> > > > > > > > >
> > >>> > > > > > > > > I tried to login Princeton but the connection
is
> closed
> > >>> too.
> > >>> > > > > > > > >
> > >>> > > > > > > > >
> > >>> > > > > > > > > Really appreciate your time and help on this.
> > >>> > > > > > > > >
> > >>> > > > > > > > >
> > >>> > > > > > > > > Best,
> > >>> > > > > > > > >
> > >>> > > > > > > > > Yan
> > >>> > > > > > > > >
> > >>> > > > > > > > > On Tue, May 11, 2021 at 11:57 AM John Halley
Gotway
> via
> > >>> RT <
> > >>> > > > > > > > > met_help at ucar.edu> wrote:
> > >>> > > > > > > > >
> > >>> > > > > > > > > > Yan,
> > >>> > > > > > > > > >
> > >>> > > > > > > > > > George and I tried to take a look at this,
but
> > couldn't
> > >>> > find
> > >>> > > > the
> > >>> > > > > > > > > > data.
> > >>> > > > > > > > > >
> > >>> > > > > > > > > >
> > >>> > > > > > > >
> > >>> > > > > > >
> > >>> > > > > >
> > >>> > > > >
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210510214154
> > >>> > > > > > > > > >
> > >>> > > > > > > > > > I checked on Hera and WCOSS but couldn't
find the
> > data
> > >>> > > there. I
> > >>> > > > > > > > > > should be
> > >>> > > > > > > > > > able to log on to jet, but when I try, it
boots me
> > off:
> > >>> > > > > > > > > >
> > >>> > > > > > > > > > Authentication failed.
> > >>> > > > > > > > > > Connection to jet-
rsa.boulder.rdhpcs.noaa.gov
> > closed.
> > >>> > > > > > > > > >
> > >>> > > > > > > > > > Can you confirm that you're running on jet?
And any
> > >>> idea of
> > >>> > > jet
> > >>> > > > > is
> > >>> > > > > > > > > > down
> > >>> > > > > > > > > > today or something?
> > >>> > > > > > > > > >
> > >>> > > > > > > > > > Thanks,
> > >>> > > > > > > > > > John
> > >>> > > > > > > > > >
> > >>> > > > > > > > > >
> > >>> > > > > > > > > > On Tue, May 11, 2021 at 7:46 AM Yan Jin -
NOAA
> > >>> Affiliate
> > >>> > via
> > >>> > > > RT <
> > >>> > > > > > > > > > met_help at ucar.edu> wrote:
> > >>> > > > > > > > > >
> > >>> > > > > > > > > > >
> > >>> > > > > > > > > > > <URL:
> > >>> > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846
> > >>> > > > > >
> > >>> > > > > > > > > > >
> > >>> > > > > > > > > > > Hi John,
> > >>> > > > > > > > > > >
> > >>> > > > > > > > > > > Thanks so much!
> > >>> > > > > > > > > > >
> > >>> > > > > > > > > > > Best,
> > >>> > > > > > > > > > > Yan
> > >>> > > > > > > > > > >
> > >>> > > > > > > > > > >
> > >>> > > > > > > > > > > On Tue, May 11, 2021 at 9:44 AM John
Halley
> Gotway
> > >>> via
> > >>> > RT <
> > >>> > > > > > > > > > > met_help at ucar.edu>
> > >>> > > > > > > > > > > wrote:
> > >>> > > > > > > > > > >
> > >>> > > > > > > > > > > > Hi Yan,
> > >>> > > > > > > > > > > >
> > >>> > > > > > > > > > > > I'm going to assign this ticket to
George and
> ask
> > >>> him
> > >>> > to
> > >>> > > > > take a
> > >>> > > > > > > > > > > > look
> > >>> > > > > > > > > > and
> > >>> > > > > > > > > > > > debug this runtime error.
> > >>> > > > > > > > > > > >
> > >>> > > > > > > > > > > > Thanks,
> > >>> > > > > > > > > > > > John HG
> > >>> > > > > > > > > > > >
> > >>> > > > > > > > > > > > On Mon, May 10, 2021 at 3:49 PM Yan Jin
- NOAA
> > >>> > Affiliate
> > >>> > > > via
> > >>> > > > > > RT <
> > >>> > > > > > > > > > > > met_help at ucar.edu> wrote:
> > >>> > > > > > > > > > > >
> > >>> > > > > > > > > > > > >
> > >>> > > > > > > > > > > > > Mon May 10 15:49:51 2021: Request
99846 was
> > acted
> > >>> > upon.
> > >>> > > > > > > > > > > > > Transaction: Ticket created by
> > yan.jin at noaa.gov
> > >>> > > > > > > > > > > > >        Queue: met_help
> > >>> > > > > > > > > > > > >      Subject: ask a question about
METplus
> > >>> config of
> > >>> > TC
> > >>> > > > > > > > > > > > >        Owner: Nobody
> > >>> > > > > > > > > > > > >   Requestors: yan.jin at noaa.gov
> > >>> > > > > > > > > > > > >       Status: new
> > >>> > > > > > > > > > > > >  Ticket <URL:
> > >>> > > > > > > > > >
> > >>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846
> > >>> > > > > > > > > > > >
> > >>> > > > > > > > > > > > >
> > >>> > > > > > > > > > > > >
> > >>> > > > > > > > > > > > > Hi,
> > >>> > > > > > > > > > > > >
> > >>> > > > > > > > > > > > > This is Yan, again. I ran the METplus
config
> > file
> > >>> > after
> > >>> > > > our
> > >>> > > > > > > > > > discussion,
> > >>> > > > > > > > > > > > > however, it seems that the tc_pairs
tcst file
> > >>> wasn't
> > >>> > > > > > generated.
> > >>> > > > > > > > > > > > >
> > >>> > > > > > > > > > > > > Here is the script in my directory on
Jet:
> > >>> > > > > > > > > > > > > I set TC_PAIRS_STORM_ID = AL292020, so
the
> code
> > >>> can
> > >>> > > find
> > >>> > > > > the
> > >>> > > > > > > > > > > > > input
> > >>> > > > > > > > > > > adeck
> > >>> > > > > > > > > > > > > and bdeck data correctly
> > >>> > > > > > > > > > > > > *----
> > >>> > > > > > > > > > > > >
> > >>> > > > > > > > > > > > >
> > >>> > > > > > > > > > > >
> > >>> > > > > > > > > > >
> > >>> > > > > > > > > >
> > >>> > > > > > > >
> > >>> > > > > > >
> > >>> > > > > >
> > >>> > > > >
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test.conf
> > >>> > > > > > > > > > > > >
> > >>> > > > > > > > > > > > > The output directory is
> > >>> > > > > > > > > > > > >
> > >>> > > > > > > >
> > >>> > >
> /mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs
> > >>> > > > > > > > > > > > > and the tc_pairs data:
> al292020_pairs.dat.tcst
> > >>> > > > > > > > > > > > >
> > >>> > > > > > > > > > > > > log file
> > >>> > > > > > > > > > > > >
> > >>> > > > > > > > > > > > >
> > >>> > > > > > > > > > > >
> > >>> > > > > > > > > > >
> > >>> > > > > > > > > >
> > >>> > > > > > > >
> > >>> > > > > > >
> > >>> > > > > >
> > >>> > > > >
> > >>> > > >
> > >>> > >
> > >>> >
> > >>>
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210510214154
> > >>> > > > > > > > > > > > > *-----
> > >>> > > > > > > > > > > > >
> > >>> > > > > > > > > > > > > The error starts:[image: image.png]
> > >>> > > > > > > > > > > > >
> > >>> > > > > > > > > > > > > Then I checked
al292020_pairs.dat.tcst, and
> it
> > is
> > >>> > > empty.
> > >>> > > > > > > > > > > > >
> > >>> > > > > > > > > > > > > I wonder if there is anything I
missed?
> > >>> > > > > > > > > > > > >
> > >>> > > > > > > > > > > > > Many thanks.
> > >>> > > > > > > > > > > > > Best,
> > >>> > > > > > > > > > > > > Yan
> > >>> > > > > > > > > > > > >
> > >>> > > > > > > > > > > > >
> > >>> > > > > > > > > > > >
> > >>> > > > > > > > > > > >
> > >>> > > > > > > > > > >
> > >>> > > > > > > > > > >
> > >>> > > > > > > > > >
> > >>> > > > > > > > > >
> > >>> > > > > > > >
> > >>> > > > > > > >
> > >>> > > > > > > >
> > >>> > > > > > > >
> > >>> > > > > > >
> > >>> > > > > > >
> > >>> > > > > >
> > >>> > > > > > --
> > >>> > > > > > George McCabe - Software Engineer III
> > >>> > > > > > National Center for Atmospheric Research
> > >>> > > > > > Research Applications Laboratory
> > >>> > > > > > 303-497-2768
> > >>> > > > > > ---
> > >>> > > > > > My working day may not be your working day. Please
do not
> > feel
> > >>> > > obliged
> > >>> > > > to
> > >>> > > > > > reply to this email outside of your normal working
hours.
> > >>> > > > > >
> > >>> > > > > >
> > >>> > > > >
> > >>> > > > >
> > >>> > > >
> > >>> > > > --
> > >>> > > > George McCabe - Software Engineer III
> > >>> > > > National Center for Atmospheric Research
> > >>> > > > Research Applications Laboratory
> > >>> > > > 303-497-2768
> > >>> > > > ---
> > >>> > > > My working day may not be your working day. Please do
not feel
> > >>> obliged
> > >>> > to
> > >>> > > > reply to this email outside of your normal working
hours.
> > >>> > > >
> > >>> > > >
> > >>> > >
> > >>> > >
> > >>> >
> > >>> > --
> > >>> > George McCabe - Software Engineer III
> > >>> > National Center for Atmospheric Research
> > >>> > Research Applications Laboratory
> > >>> > 303-497-2768
> > >>> > ---
> > >>> > My working day may not be your working day. Please do not
feel
> > obliged
> > >>> to
> > >>> > reply to this email outside of your normal working hours.
> > >>> >
> > >>> >
> > >>>
> > >>>
> > >>
> > >> --
> > >> George McCabe - Software Engineer III
> > >> National Center for Atmospheric Research
> > >> Research Applications Laboratory
> > >> 303-497-2768
> > >> ---
> > >> My working day may not be your working day. Please do not feel
obliged
> > to
> > >> reply to this email outside of your normal working hours.
> > >>
> > >
> > >
> > > --
> > > George McCabe - Software Engineer III
> > > National Center for Atmospheric Research
> > > Research Applications Laboratory
> > > 303-497-2768
> > > ---
> > > My working day may not be your working day. Please do not feel
obliged
> to
> > > reply to this email outside of your normal working hours.
> > >
> >
> >
> > --
> > George McCabe - Software Engineer III
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > 303-497-2768
> > ---
> > My working day may not be your working day. Please do not feel
obliged to
> > reply to this email outside of your normal working hours.
> >
> >
>
>

--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.

------------------------------------------------
Subject: ask a question about METplus config of TC
From: Yan Jin - NOAA Affiliate
Time: Wed May 26 14:50:52 2021

Hi George,

Thanks so much. I am not in a hurry. It is totally fine to wait until
it is
ready after you are done with your current work.

I checked the files in
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/vrfy_METplus4.0/AL_MJJA_2020/tc_pairs,
and
each file contain the result for that cyclone number.
e.g.: al012020_pairs.dat.tcst has the initial time 20200515~20200520.

Many thanks.
Best,
Yan

On Wed, May 26, 2021 at 3:38 PM George McCabe via RT
<met_help at ucar.edu>
wrote:

> Hi Yan,
>
> The way the wrapper is currently written, specifying that list of
cyclones
> in TC_PAIRS_CYCLONE will result in 15 output files that all contain
all 15
> cyclones. I am currently working on updating the tool to handle this
> properly so that each file will only contain the cyclone in
question. Your
> configuration looks correct to me, but it will not produce the
correct
> behavior until these updates are finished. When it is ready, I can
make it
> available to you on Jet so you can test that it runs as expected.
>
> I am actually finished with this fix but I am now working on
implementing
> support for setting consensus through METplus. If you'd like, I can
put the
> code from my development branch on Jet so you can test that this
works
> while I am still working on consensus. Let me know if you'd like me
to do
> this and I can make it available to you.
>
> Thanks,
> George
>
> - George
>
> On Wed, May 26, 2021 at 12:54 PM Yan Jin - NOAA Affiliate via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846 >
> >
> > Hi George,
> >
> > Can I ask a question regarding the seasonal mean TC verification?
For
> > instance, I have adeck and bdeck data over Atlantic at 2020, (Jet:
> > /mnt/lfs4/HFIP/hwrf-data/hwrf-input/abdeck), each has 01-99
cyclone
> number.
> > And I would like to calculate the absolute intensity error and
track
> error
> > of seasonal mean over Atlantic. Here is my method, could you check
if it
> is
> > correct to do that?
> >
> > I plan to do the May-June-July-August mean error.
> > $WORK:
> >
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/vrfy_METplus4.0/METplus/parm/user_config
> > in this directory, the METplus config file is
TCMPRPlotter_AL_MJJA_2020
> >
> >
> > The major modification I did is:
> > ***
> > MODEL = HWRF,HMON,GFSO
> >
> > TC_PAIRS_DESC =
> >
> > TC_PAIRS_STORM_ID =
> > TC_PAIRS_BASIN = AL
> > TC_PAIRS_CYCLONE = 01,02,03,04,05,06,07,08,09,10,11,12,13,14,15
> > TC_PAIRS_STORM_NAME =
> >
> > TC_PAIRS_INIT_BEG = 20200515_000000
> > TC_PAIRS_INIT_END = 20200906_000000
> > ***
> > I checked the adeck and bdeck data, and found that cyclone number
01-15
> > have the initial times which are located within MJJA. So I
> > specified TC_PAIRS_CYCLONE with 01 to 15. I also checked the
initial time
> > in cyclone 01 and 15, and found the INIT_BEG and INIT_END.
> >
> > It generated 15 tc_pairs.tcst files, and also 6 figures (Output
> directory:
> > /mnt/lfs4/HFIP/hwrfv3/Yan.Jin/vrfy_METplus4.0/AL_MJJA_2020)
> >
> > Here is the result
> >
> >
> https://docs.google.com/document/d/10adrHdzNNxMJM51e5Sw9oy6uS2n1S-
qGS2c7NZdVVxc/edit?pli=1
> >
> > I wonder if my method is correct? It seems that TC_PAIRS_INIT_BEG
> > and TC_PAIRS_INIT_END are mandatory for my case.
> >
> > Many thanks.
> > Best,
> > Yan
> >
> > On Thu, May 20, 2021 at 12:01 PM George McCabe via RT
<met_help at ucar.edu
> >
> > wrote:
> >
> > > Hi Yan,
> > >
> > > I was able to change the line colors in the images. The images
can be
> > found
> > > on Jet here:
> > >
> > > /lfs1/HFIP/dtc-hurr/George.Mccabe/eta-out-colors/tcmpr_plots
> > >
> > >
> > > This is the command I used to run:
> > >
> > > run_metplus.py
> > > /lfs1/HFIP/dtc-hurr/George.Mccabe/TCMPRPlotter_Eta_colors.conf
> > > config.OUTPUT_BASE=/lfs1/HFIP/dtc-hurr/George.Mccabe/eta-out-
colors
> > >
> > >
> > > I copied the default tcmpr file from
> > >
> >
>
/contrib/met/10.0.0/share/met/Rscripts/include/plot_tcmpr_config_default.R
> > > to my directory and renamed it to plot_tcmpr_overrides.R
> > >
> > > Then I modified the METplus config to point to that file instead
of the
> > > file in the METplus repository:
> > >
> > > TCMPR_PLOTTER_CONFIG_FILE =
> > > /lfs1/HFIP/dtc-hurr/George.Mccabe/plot_tcmpr_overrides.R
> > >
> > >
> > > The previous tcmpr parameter file changes the image resolution,
so I
> had
> > to
> > > set that in the new file to prevent the image from being very
large.
> > Then I
> > > changed the value of the default color list:
> > >
> > > img_res = 72;
> > > default_color_list  = c("blue", "green", "purple", "orange");
> > >
> > > This variable is used to set other variables for each plot type,
so you
> > can
> > > modify those variables if you desire different colors for
different
> > plots.
> > > You only need to supply the variables you want to change from
the
> default
> > > value, so your overrides file can just include the above 2
variables to
> > > keep the file short and clean. However, it is nice to have all
of the
> > > possible options easily available
> > > in case you want to change other settings. How you handle this
is up to
> > > your preference.
> > >
> > > Let me know if you have any other questions.
> > >
> > > Thanks,
> > > George
> > >
> > > On Thu, May 20, 2021 at 9:19 AM George McCabe <mccabe at ucar.edu>
wrote:
> > >
> > > > Hi Yan,
> > > >
> > > > I copied your config file and made some modifications. You can
view
> it
> > on
> > > > Jet here:
> > > >
> > > > /lfs1/HFIP/dtc-hurr/George.Mccabe/TCMPRPlotter_Eta.conf
> > > >
> > > > Starting in METplus 4.0.0 the different section headers like
dir and
> > > > filename_templates are no longer necessary. All configurations
can go
> > > under
> > > > the [config] header. I removed the sections and rearranged the
> > variables
> > > to
> > > > group them in a more clear way.
> > > >
> > > > To call TCMPRPlotter multiple times with different settings, I
added
> > > > multiple instances of the app in the PROCESS_LIST with
instance names
> > > > defined:
> > > >
> > > > PROCESS_LIST = TCPairs, TCMPRPlotter(abs_mean),
> > > > TCMPRPlotter(abs_median),TCMPRPlotter(abs_boxplot),
> > > TCMPRPlotter(tk_mean),
> > > > TCMPRPlotter(tk_median),TCMPRPlotter(tk_boxplot)
> > > >
> > > > Each instance will read all of the values set in the [config]
> section,
> > > > then it will look for a section header that matches the string
inside
> > the
> > > > parentheses for each instance. If the section is found, it
overrides
> > the
> > > > values set in the section to replace the values set in the
[config]
> > > > section. These names can be whatever you chose, but the name
in the
> > > > PROCESS_LIST must match the name of the section header.
> > > >
> > > > [abs_mean]
> > > > TCMPR_PLOTTER_PREFIX = Eta_Absolute_Intensity_mean
> > > > TCMPR_PLOTTER_TITLE = Absolute Intensity Error Mean\n Eta
(Atlantic
> > 2020)
> > > > TCMPR_PLOTTER_DEP_VARS = ABS(AMAX_WIND-BMAX_WIND)
> > > > TCMPR_PLOTTER_PLOT_TYPES = MEAN
> > > >
> > > > [abs_median]
> > > > TCMPR_PLOTTER_PREFIX = Eta_Absolute_Intensity_median
> > > > TCMPR_PLOTTER_TITLE = Absolute Intensity Error Median\n Eta
(Atlantic
> > > 2020)
> > > > TCMPR_PLOTTER_DEP_VARS = ABS(AMAX_WIND-BMAX_WIND)
> > > > TCMPR_PLOTTER_PLOT_TYPES = MEDIAN
> > > >
> > > > [abs_boxplot]
> > > > TCMPR_PLOTTER_PREFIX = Eta_Absolute_Intensity_boxplot
> > > > TCMPR_PLOTTER_TITLE = Absolute Intensity Error Boxplot\n Eta
> (Atlantic
> > > > 2020)
> > > > TCMPR_PLOTTER_DEP_VARS = ABS(AMAX_WIND-BMAX_WIND)
> > > > TCMPR_PLOTTER_PLOT_TYPES = BOXPLOT
> > > >
> > > > [tk_mean]
> > > > TCMPR_PLOTTER_PREFIX = Eta_Track_Error_mean
> > > > TCMPR_PLOTTER_TITLE = Track Error Mean\n Eta (Atlantic 2020)
> > > > TCMPR_PLOTTER_DEP_VARS = TK_ERR
> > > > TCMPR_PLOTTER_PLOT_TYPES = MEAN
> > > >
> > > > [tk_median]
> > > > TCMPR_PLOTTER_PREFIX = Eta_Track_Error_median
> > > > TCMPR_PLOTTER_TITLE = Track Error Median\n Eta (Atlantic 2020)
> > > > TCMPR_PLOTTER_DEP_VARS = TK_ERR
> > > > TCMPR_PLOTTER_PLOT_TYPES = MEDIAN
> > > >
> > > > [tk_boxplot]
> > > > TCMPR_PLOTTER_PREFIX = Eta_Track_Error_boxplot
> > > > TCMPR_PLOTTER_TITLE = Track Error Boxplot\n Eta (Atlantic
2020)
> > > > TCMPR_PLOTTER_DEP_VARS = TK_ERR
> > > > TCMPR_PLOTTER_PLOT_TYPES = BOXPLOT
> > > >
> > > >
> > > > I split up the DEP_VARS and PLOT_TYPES values into instances
with one
> > > > value of each and set the plot title and filenames
appropriately for
> > > each.
> > > > I set the X label to "Lead Time (h)" in the [config] section
so it
> > > applies
> > > > to every instance.
> > > >
> > > > TCMPR_PLOTTER_XLAB = Lead Time (h)
> > > >
> > > > The output images from my run is on Jet here:
> > > >
> > > > /lfs1/HFIP/dtc-hurr/George.Mccabe/eta-out/tcmpr_plots
> > > >
> > > > Eta_Absolute_Intensity_boxplot.log
> > > > Eta_Absolute_Intensity_boxplot.png
> > > > Eta_Absolute_Intensity_mean.png
> > > > Eta_Absolute_Intensity_median.png
> > > > Eta_Track_Error_boxplot.log
> > > > Eta_Track_Error_boxplot.png
> > > > Eta_Track_Error_mean.png
> > > > Eta_Track_Error_median.png
> > > >
> > > >
> > > > Here is the log file from the run:
> > > >
> > /lfs1/HFIP/dtc-hurr/George.Mccabe/eta-
out/logs/metplus.log.20210520150804
> > > >
> > > >
> > > > Here is what I ran to produce the output:
> > > >
> > > >
> > > >    - module load intel
> > > >    - module load intelpython/3.6.5
> > > >    - module load netcdf/4.6.1
> > > >    - module load hdf5/1.10.4
> > > >    - module load nco/4.9.1
> > > >    - module load wgrib/1.8.1.0b
> > > >    - module load wgrib2/2.0.8
> > > >    - module load R/4.0.2
> > > >    - module use /contrib/met/modulefiles
> > > >    - module load met/10.0.0
> > > >    - module use /contrib/met/METplus/modulefiles
> > > >    - module load metplus/4.0.0
> > > >    - run_metplus.py
> > > >    /lfs1/HFIP/dtc-hurr/George.Mccabe/TCMPRPlotter_Eta.conf
> > > >    config.OUTPUT_BASE=/lfs1/HFIP/dtc-hurr/George.Mccabe/eta-
out
> > > >
> > > >
> > > >
> > > > I will look into what settings are needed to change the line
colors.
> > > Also,
> > > > I have started making the modifications to the TCPairs wrapper
to
> > > properly
> > > > handle multiple storms specified in the config file. I will
let you
> > know
> > > > when that is ready to test.
> > > >
> > > > Thanks,
> > > > George
> > > >
> > > >
> > > >
> > > > On Wed, May 19, 2021 at 5:09 PM George McCabe
<mccabe at ucar.edu>
> wrote:
> > > >
> > > >> Hi Yan,
> > > >>
> > > >> 1) add TC name to each plot name
> > > >> 2) modify titles to each plot, e.g.: TK_ERR* plots with name
"Track
> > > >> Error", and ABS_AMAX_WIND-BMAX_WIND_* plots with name
"Absolute
> > > Intensity
> > > >> Error"?
> > > >>
> > > >> It looks like the logic in the R script uses the -prefix
value to
> > > replace
> > > >> the filename instead of prepending it to the plot type text
as you
> > would
> > > >> guess it would. I was told that this script was typically
called
> > > multiple
> > > >> times to generate the multiple plots, which would provide
more
> control
> > > over
> > > >> setting the output filenames, titles, etc. The bad news is it
looks
> > like
> > > >> the wrapper was written to call the R script once with all of
the
> > > >> configurations specified in the METplus config file. The good
news
> is
> > > there
> > > >> are some new capabilities added to METplus 4.0.0 that could
help
> > produce
> > > >> the results you need.
> > > >>
> > > >> Please review this section of the METplus User's Guide,
specifically
> > the
> > > >> section about specifying instance names to the processes:
> > > >>
> > > >>
> > >
> >
>
https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#process-
list
> > > >>
> > > >> Alternatively, you can use the UserScript wrapper to build
custom
> > > >> commands to call the R script directly with any arguments you
need
> if
> > > the
> > > >> wrapper doesn't support an option you need.
> > > >> More information on that tool is here:
> > > >>
> > >
> >
>
https://metplus.readthedocs.io/en/latest/Users_Guide/wrappers.html#userscript
> > > >> and here:
> > > >>
> > >
> >
>
https://metplus.readthedocs.io/en/latest/generated/met_tool_wrapper/UserScript/UserScript_run_once.html#sphx-
glr-generated-met-tool-wrapper-userscript-userscript-run-once-py
> > > >>
> > > >> I think using different instance names to run TCMPRPlotter
multiple
> > > times
> > > >> with different settings is the better approach in this case.
I am
> > > wrapping
> > > >> up work for the day, but tomorrow I can log into Jet and help
you
> > modify
> > > >> your config file to do this.
> > > >>
> > > >>
> > > >> 3) Is it possible to modify the line colors in this METplus
config
> > file?
> > > >>
> > > >> Yes, you can change the line colors through the R script's
config
> > file.
> > > >> You can copy and rename this file:
> > > >>
> > > >>
> > > >>
> > >
> >
>
https://github.com/dtcenter/MET/blob/main_v10.0/met/scripts/Rscripts/include/plot_tcmpr_config_default.R
> > > >>
> > > >> change the values that you need to change (look for
"color_list"),
> > then
> > > >> set the METplus configuration file to point to your new file
(set in
> > > >> TCMPR_PLOTTER_CONFIG_FILE).
> > > >>
> > > >>
> > > >> Let me know if you have any other questions. I will report
back
> > tomorrow
> > > >> when have some changes for you to test.
> > > >>
> > > >> Thanks,
> > > >> George
> > > >>
> > > >> On Wed, May 19, 2021 at 4:00 PM Yan Jin - NOAA Affiliate via
RT <
> > > >> met_help at ucar.edu> wrote:
> > > >>
> > > >>>
> > > >>> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846 >
> > > >>>
> > > >>> Hi George,
> > > >>>
> > > >>> I have a few questions regarding the TCMPR_PLOTTER_*
parameters in
> > the
> > > >>> METplus config file.
> > > >>>
> > > >>> Here is the METplus config file I used:
> > > >>>
> > > >>>
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/vrfy_METplus4.0/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test.conf
> > > >>> **
> > > >>> #  Plot_TCMPR options, if left unset, default values that
are
> > > >>> #  pre-defined in the R utility (packaged with MET) will be
used.
> > > >>> TCMPR_PLOTTER_CONFIG_FILE =
> {CONFIG_DIR}/TCMPRPlotterConfig_customize
> > > >>> TCMPR_PLOTTER_PREFIX =
> > > >>> TCMPR_PLOTTER_TITLE =
> > > >>> TCMPR_PLOTTER_SUBTITLE =
> > > >>> TCMPR_PLOTTER_XLAB =
> > > >>> TCMPR_PLOTTER_YLAB =
> > > >>> TCMPR_PLOTTER_XLIM =
> > > >>> TCMPR_PLOTTER_YLIM =
> > > >>> TCMPR_PLOTTER_FILTER = -amodel HWRF,HMON
> > > >>> # the tcst data file to be used instead of running the MET
tc_stat
> > > tool.
> > > >>> TCMPR_PLOTTER_FILTERED_TCST_DATA_FILE =
> > > >>> # Comma separated, no whitespace.  Default is TK_ERR (track
error)
> > > unless
> > > >>> # otherwise indicated.
> > > >>> TCMPR_PLOTTER_DEP_VARS = ABS(AMAX_WIND-BMAX_WIND),TK_ERR
> > > >>> TCMPR_PLOTTER_SCATTER_X =
> > > >>> TCMPR_PLOTTER_SCATTER_Y =
> > > >>> TCMPR_PLOTTER_SKILL_REF =
> > > >>> TCMPR_PLOTTER_SERIES = AMODEL
> > > >>> TCMPR_PLOTTER_SERIES_CI = TRUE
> > > >>> TCMPR_PLOTTER_LEGEND =
> > > >>> TCMPR_PLOTTER_LEAD = 0,12,24,36,48,60,72,84,96,108,120
> > > >>> # Mean and median plots. These override the plot_tcmpr.R
default of
> > box
> > > >>> plot.
> > > >>> # If box plot is desired, this needs to be explicitly
indicated.
> > > >>> TCMPR_PLOTTER_PLOT_TYPES = MEAN,MEDIAN,BOXPLOT
> > > >>> TCMPR_PLOTTER_RP_DIFF =
> > > >>> TCMPR_PLOTTER_DEMO_YR =
> > > >>> TCMPR_PLOTTER_HFIP_BASELINE =
> > > >>> TCMPR_PLOTTER_FOOTNOTE_FLAG =
> > > >>> TCMPR_PLOTTER_PLOT_CONFIG_OPTS =
> > > >>> TCMPR_PLOTTER_SAVE_DATA =
> > > >>>
> > > >>> #  TCMPR FLAGS no == (don't set flag), yes == (set flag)
> > > >>> TCMPR_PLOTTER_NO_EE = no
> > > >>> TCMPR_PLOTTER_NO_LOG = no
> > > >>> TCMPR_PLOTTER_SAVE = no
> > > >>> **
> > > >>> It generated the following 6 *.png files
> > > >>> in
> > > >>>
> > >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/vrfy_METplus4.0/Eta_use_METplus/tcmpr_plots
> > > >>> [image: image.png]
> > > >>>
> > > >>> Then I would like to modify titles, xlab text, output file
names.
> > > >>> Here are the modification I
> > > >>> did
> > > >>>
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/vrfy_METplus4.0/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test1.conf
> > > >>>
> > > >>> **
> > > >>> TCMPR_PLOTTER_TITLE = Absolute Intensity Error \n Eta
(Atlantic
> 2020)
> > > >>> TCMPR_PLOTTER_XLAB = Lead Time (h)
> > > >>> **
> > > >>> The title of each plot becomes "Absolute Intensity Error Eta
> > (Atlantic
> > > >>> 2020)" with double quotes.
> > > >>>
> > > >>> I also tried to set TCMPR_PLOTTER_PREFIX = Eta_ because I
would
> like
> > to
> > > >>> save each plot as Eta_ABS_AMAX_WIND-BMAX_WIND_*.png, and
> > > >>> Eta_TK_ERR_*.png.
> > > >>> However, there is only one plot Eta_.png generated.
> > > >>>
> > > >>> I would like to check how can I set those parameters if I
would
> like
> > to
> > > >>> 1)
> > > >>> add TC name to each plot name; 2) modify titles to each
plot, e.g.:
> > > >>> TK_ERR*
> > > >>> plots with name "Track Error", and ABS_AMAX_WIND-BMAX_WIND_*
plots
> > with
> > > >>> name "Absolute Intensity Error"? 3) Is it possible to modify
the
> line
> > > >>> colors in this METplus config file?
> > > >>>
> > > >>> Could you give me some suggestions?
> > > >>>
> > > >>> Many thanks.
> > > >>> Best,
> > > >>> Yan
> > > >>>
> > > >>> On Mon, May 17, 2021 at 10:37 AM George McCabe via RT <
> > > met_help at ucar.edu
> > > >>> >
> > > >>> wrote:
> > > >>>
> > > >>> > Hi Yan,
> > > >>> >
> > > >>> > 4.1:
> > > >>> > The consensus variable is not yet controlled by a METplus
> > > configuration
> > > >>> > variable explicitly, but you can set the value by using
the
> > > >>> > TC_PAIRS_MET_CONFIG_OVERRIDES value (in v4.0.0+). To set
the
> > > consensus
> > > >>> > value to what you have set in your config file, you would
set
> > > something
> > > >>> > like this:
> > > >>> >
> > > >>> > TC_PAIRS_MET_CONFIG_OVERRIDES = consensus = [ { name =
"CONS";
> > > members
> > > >>> =
> > > >>> > ["HWRF","GFSO","HMON"]; required = [false,false,false];
min_req =
> > 2;
> > > }
> > > >>> ];
> > > >>> >
> > > >>> > I will make a note in the GitHub issue for improvements to
the
> > > TCPairs
> > > >>> > wrapper (https://github.com/dtcenter/METplus/issues/749)
to add
> > > >>> support
> > > >>> > for
> > > >>> > consensus explicitly via something like TC_PAIRS_CONSENSUS
so
> that
> > it
> > > >>> is
> > > >>> > easier to configure via METplus wrappers.
> > > >>> >
> > > >>> > 4.2:
> > > >>> > The System Configuration section of the METplus User's
Guide
> covers
> > > >>> these
> > > >>> > variables and how they affect execution. I suggest that
you read
> > > >>> through
> > > >>> > this section to get a full picture of how these variables
work.
> > > >>> Regarding
> > > >>> > your specific question, these 2 sections should help:
> > > >>> >
> > > >>> > LOOP_BY:
> > > >>> >
> > > >>> >
> > > >>>
> > >
> >
>
https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#loop-
by
> > > >>> > LOOP_ORDER:
> > > >>> >
> > > >>> >
> > > >>>
> > >
> >
>
https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#loop-
order
> > > >>> >
> > > >>> > Thanks,
> > > >>> > George
> > > >>> >
> > > >>> > On Fri, May 14, 2021 at 8:19 AM Yan Jin - NOAA Affiliate
via RT <
> > > >>> > met_help at ucar.edu> wrote:
> > > >>> >
> > > >>> > >
> > > >>> > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846
> >
> > > >>> > >
> > > >>> > > Hi George,
> > > >>> > >
> > > >>> > > Thanks so much for your help. Finally the code can run
without
> > > >>> errors.
> > > >>> > Here
> > > >>> > > is the steps and results:
> > > >>> > >
> > > >>> > >
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
> https://docs.google.com/document/d/1xRF3aiPATcX5G37ngJj-
65pl44LN7mGHo34Sf8i3zpo/edit
> > > >>> > >
> > > >>> > > I also put questions in the document, I also pasted
here:
> > > >>> > >
> > > >>> > > Questions:
> > > >>> > >
> > > >>> > > 4.1: How can I do the consensus forecasts in METplus
config
> file
> > as
> > > >>> I did
> > > >>> > > in tc_pairs?
> > > >>> > >
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
(/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_292020/TCPairsConfig_run1)
> > > >>> > >
> > > >>> > > //
> > > >>> > >
> > > >>> > > // Specify how consensus forecasts should be defined.
> > > >>> > >
> > > >>> > > //
> > > >>> > >
> > > >>> > > consensus = [
> > > >>> > >
> > > >>> > >  {
> > > >>> > >
> > > >>> > >   name = "CONS";
> > > >>> > >
> > > >>> > >   members = ["HWRF","GFSO","HMON"];
> > > >>> > >
> > > >>> > >   required = [false,false,false];
> > > >>> > >
> > > >>> > >   min_req = 2;
> > > >>> > >
> > > >>> > >  }
> > > >>> > >
> > > >>> > > ];
> > > >>> > >
> > > >>> > > 4.2: I am a little confused about
> > > >>> > >
> > > >>> > >       ***
> > > >>> > >
> > > >>> > >        LOOP_ORDER = processes
> > > >>> > >
> > > >>> > >        LOOP_BY = INIT
> > > >>> > >
> > > >>> > >       ***
> > > >>> > >
> > > >>> > > Could you explain these two command lines and give me
some
> > > examples?
> > > >>> > >
> > > >>> > > Thanks very much for your time and help on my questions.
> > > >>> > >
> > > >>> > > Best,
> > > >>> > > Yan
> > > >>> > >
> > > >>> > > On Thu, May 13, 2021 at 3:04 PM George McCabe via RT <
> > > >>> met_help at ucar.edu>
> > > >>> > > wrote:
> > > >>> > >
> > > >>> > > > Hi Yan,
> > > >>> > > >
> > > >>> > > > The produtil is due to a bug in the wrapper in the
logic that
> > > >>> reports
> > > >>> > > when
> > > >>> > > > an error occurs. I ran the command on Jet by hand and
it
> looks
> > > >>> like it
> > > >>> > is
> > > >>> > > > still not finding any input data to plot. You can run
the
> > > following
> > > >>> > > > commands (after loading all of the required modules):
> > > >>> > > >
> > > >>> > > > export MET_INSTALL_DIR=/contrib/met/9.1
> > > >>> > > >
> > > >>> > > > Rscript
/contrib/met/9.1/share/met/Rscripts/plot_tcmpr.R \
> > > >>> > > > -config
> > > >>> > > >
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
> /contrib/met/METplus/METplus-
3.1.1/parm/use_cases/model_applications/tc_and_extra_tc/TCMPRPlotterConfig_customize
> > > >>> > > > \
> > > >>> > > > -subtitle "Your subtitle goes here" -ylab "Your y-
label goes
> > > here"
> > > >>> \
> > > >>> > > > -dep AMAX_WIND-BMAX_WIND,TK_ERR -legend "Your legend
text
> goes
> > > >>> > here..." \
> > > >>> > > > -plot MEAN,MEDIAN,BOXPLOT \
> > > >>> > > > -lookin
> > > >>> > > >
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> > > >>> > > > \
> > > >>> > > >
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
> > > >>> > > > \
> > > >>> > > >
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
> > > >>> > > > \
> > > >>> > > > -outdir
> > > >>> > > >
> > > >>>
> >
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tcmpr_plots
> > > >>> > > >
> > > >>> > > >
> > > >>> > > > Here is the output from that run:
> > > >>> > > >
> > > >>> > > > Reading plot configuration file:
> > > >>> > > >
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
> /contrib/met/METplus/METplus-
3.1.1/parm/use_cases/model_applications/tc_and_extra_tc/TCMPRPlotterConfig_customize
> > > >>> > > > No plotting configuration file specified.
> > > >>> > > > CALLING:  /contrib/met/9.1/bin/tc_stat -lookin
> > > >>> > > >
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> > > >>> > > > -lookin
> > > >>> > > >
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
> > > >>> > > > -lookin
> > > >>> > > >
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
> > > >>> > > > -job filter -dump_row /tmp/plot_tcmpr_116966.tcst
> -event_equal
> > > >>> true
> > > >>> > -v 3
> > > >>> > > > DEBUG 1: [Source 1] Lookin:
> > > >>> > > >
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> > > >>> > > > DEBUG 1: [Source 2] Lookin:
> > > >>> > > >
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
> > > >>> > > > DEBUG 1: [Source 3] Lookin:
> > > >>> > > >
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
> > > >>> > > > DEBUG 2:
> > > >>> > > > DEBUG 2: Processing Job 1: -job filter -event_equal
true
> > > -dump_row
> > > >>> > > > /tmp/plot_tcmpr_116966.tcst
> > > >>> > > > DEBUG 3: Applying track-based event equalization
logic.
> > > >>> > > > DEBUG 3: Reading file 1 of 3:
> > > >>> > > >
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> > > >>> > > > DEBUG 3: Reading file 2 of 3:
> > > >>> > > >
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
> > > >>> > > > DEBUG 3: Reading file 3 of 3:
> > > >>> > > >
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
> > > >>> > > > DEBUG 3: For track-based event equalization,
identified 0
> > common
> > > >>> cases
> > > >>> > > for
> > > >>> > > > 146 models: AC00 AEMI AEMN AP01 AP02 AP03 AP04 AP05
AP06 AP07
> > > AP08
> > > >>> AP09
> > > >>> > > > AP0I AP10 AP11 AP12 AP13 AP14 AP15 AP16 AP17 AP18 AP19
AP1I
> > AP20
> > > >>> AP21
> > > >>> > > AP22
> > > >>> > > > AP23 AP24 AP25 AP26 AP27 AP28 AP29 AP30 API CARQ CEM2
CEMI
> CEMN
> > > >>> CLP5
> > > >>> > CMC
> > > >>> > > > CMC2 CMCI COTC COTI CTCX DRCL DSHP EC00 EEMN EGRR EMN2
EMN3
> > EMN4
> > > >>> EMNI
> > > >>> > EMX
> > > >>> > > > EMX2 EMXI EN01 EN02 EN03 EN04 EN05 EN06 EN07 EN08 EN09
EN0I
> > EN10
> > > >>> EN11
> > > >>> > > EN12
> > > >>> > > > EN13 EN14 EN15 EN16 EN17 EN18 EN19 EN1I EN20 EN21 EN22
EN23
> > EN24
> > > >>> EN25
> > > >>> > ENI
> > > >>> > > > EP01 EP02 EP03 EP04 EP05 EP06 EP07 EP08 EP09 EP0I EP10
EP11
> > EP12
> > > >>> EP13
> > > >>> > > EP14
> > > >>> > > > EP15 EP16 EP17 EP18 EP19 EP1I EP20 EP21 EP22 EP23 EP24
EP25
> EPI
> > > >>> GFEX
> > > >>> > GFSI
> > > >>> > > > GFSO HMNI HMON HWF2 HWFI HWRF ICON IVCN LGEM NGX NGX2
NGXI
> NVGM
> > > >>> OCD5
> > > >>> > RVCN
> > > >>> > > > SHF5 SHIP TABD TABM TABS TCLP TVCA TVCE TVCN TVCX UKX
UKX2
> UKXI
> > > >>> XTRP
> > > >>> > > > DEBUG 1: Event equalization of tracks found no common
cases.
> > > >>> > > > DEBUG 1: Creating output dump file:
> /tmp/plot_tcmpr_116966.tcst
> > > >>> > > > FILTER: -job filter -line_type TCMPR -event_equal true
> > -dump_row
> > > >>> > > > /tmp/plot_tcmpr_116966.tcst
> > > >>> > > >
> > > >>> > > > DEBUG 2: Job 1 used 0 out of 0 lines read.
> > > >>> > > > DEBUG 3: Total lines read                 = 0
> > > >>> > > > DEBUG 3: Total lines kept                 = 0
> > > >>> > > > DEBUG 3: Rejected for track watch/warn    = 0
> > > >>> > > > DEBUG 3: Rejected for init threshold      = 0
> > > >>> > > > DEBUG 3: Rejected for init string         = 0
> > > >>> > > > DEBUG 3: Rejected for out init mask       = 0
> > > >>> > > > DEBUG 3: Rejected for water only          = 0
> > > >>> > > > DEBUG 3: Rejected for rapid inten         = 0
> > > >>> > > > DEBUG 3: Rejected for landfall            = 0
> > > >>> > > > DEBUG 3: Rejected for amodel              = 0
> > > >>> > > > DEBUG 3: Rejected for bmodel              = 0
> > > >>> > > > DEBUG 3: Rejected for desc                = 0
> > > >>> > > > DEBUG 3: Rejected for storm id            = 0
> > > >>> > > > DEBUG 3: Rejected for basin               = 0
> > > >>> > > > DEBUG 3: Rejected for cyclone             = 0
> > > >>> > > > DEBUG 3: Rejected for storm name          = 0
> > > >>> > > > DEBUG 3: Rejected for init time           = 0
> > > >>> > > > DEBUG 3: Rejected for init hour           = 0
> > > >>> > > > DEBUG 3: Rejected for lead time           = 0
> > > >>> > > > DEBUG 3: Rejected for required lead times = 0
> > > >>> > > > DEBUG 3: Rejected for valid time          = 0
> > > >>> > > > DEBUG 3: Rejected for valid hour          = 0
> > > >>> > > > DEBUG 3: Rejected for init mask           = 0
> > > >>> > > > DEBUG 3: Rejected for valid mask          = 0
> > > >>> > > > DEBUG 3: Rejected for line type           = 0
> > > >>> > > > DEBUG 3: Rejected for numeric threshold   = 0
> > > >>> > > > DEBUG 3: Rejected for string matching     = 0
> > > >>> > > > DEBUG 3: Rejected for match points        = 0
> > > >>> > > > DEBUG 3: Rejected for event equal         = 0
> > > >>> > > > DEBUG 3: Rejected for out init mask       = 0
> > > >>> > > > DEBUG 3: Rejected for out valid mask      = 0
> > > >>> > > > Reading track data: /tmp/plot_tcmpr_116966.tcst
> > > >>> > > > Error in read.table(tcst_tmp_file, header = TRUE) :
> > > >>> > > >   no lines available in input
> > > >>> > > > Execution halted
> > > >>> > > >
> > > >>> > > >
> > > >>> > > >
> > > >>> > > > It sounds like filter arguments may need to be
provided to
> > > produce
> > > >>> > > TC-Stat
> > > >>> > > > output (which is called by the R script). You can set
this in
> > > your
> > > >>> > > METplus
> > > >>> > > > config file with the TCMPR_PLOTTER_FILTER variable,
but it
> will
> > > be
> > > >>> > > > difficult to debug since the log output is not shown.
I would
> > > >>> recommend
> > > >>> > > > modifying the Rscript call from this email and add the
> -filter
> > > >>> argument
> > > >>> > > > with any filter information to pass to TCStat. I am
not sure
> > > >>> settings
> > > >>> > > would
> > > >>> > > > help you produce output, but I have reached out to
other
> > METplus
> > > >>> team
> > > >>> > > > members for suggestions.
> > > >>> > > >
> > > >>> > > > Thanks,
> > > >>> > > > George
> > > >>> > > >
> > > >>> > > >
> > > >>> > > >
> > > >>> > > > On Thu, May 13, 2021 at 12:37 PM Yan Jin - NOAA
Affiliate via
> > RT
> > > <
> > > >>> > > > met_help at ucar.edu> wrote:
> > > >>> > > >
> > > >>> > > > >
> > > >>> > > > > <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846
> > > >
> > > >>> > > > >
> > > >>> > > > > Hi George,
> > > >>> > > > >
> > > >>> > > > > Thanks so much. Now tc_pairs works, and the output
file can
> > be
> > > >>> > > generated.
> > > >>> > > > >
> > > >>> > > > >
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> > > >>> > > > >
> > > >>> > > > > Then I set "PROCESS_LIST = TCPairs, TCMPRPlotter",
and I
> kept
> > > >>> most
> > > >>> > > > > parameters that initialized with "TCMPR_PLOTTER" the
same
> as
> > > the
> > > >>> > > default
> > > >>> > > > > configuration.
> > > >>> > > > >
> > > >>> > > > > my METplus config file:
> > > >>> > > > >
> > > >>> > > > >
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test.conf
> > > >>> > > > >
> > > >>> > > > > then run script
> > > >>> > > > >
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/user_config/run.sh
> > > >>> > > > >
> > > >>> > > > > However, the Rscript cannot plot figures.
> > > >>> > > > > I checked the log file:
> > > >>> > > > >
> > > >>> > > > >
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210513182819
> > > >>> > > > > [image: image.png]
> > > >>> > > > >
> > > >>> > > > > It seems that the error "NameError: name 'produtil'
is not
> > > >>> defined"?
> > > >>> > > > >
> > > >>> > > > > Could you give me some suggestions?
> > > >>> > > > >
> > > >>> > > > > Many thanks.
> > > >>> > > > > Best,
> > > >>> > > > > Yan
> > > >>> > > > >
> > > >>> > > > > On Wed, May 12, 2021 at 5:20 PM George McCabe via RT
<
> > > >>> > > met_help at ucar.edu>
> > > >>> > > > > wrote:
> > > >>> > > > >
> > > >>> > > > > > Hi Yan,
> > > >>> > > > > >
> > > >>> > > > > > It looks like the values set for init_beg and
init_end in
> > the
> > > >>> > TCpairs
> > > >>> > > > > > config file are both "20200101_000000" because the
> INIT_BEG
> > > and
> > > >>> > > > INIT_END
> > > >>> > > > > > time is 2020. There are no adeck tracks for this
time, so
> > all
> > > >>> of
> > > >>> > the
> > > >>> > > > data
> > > >>> > > > > > is being filtered out.
> > > >>> > > > > >
> > > >>> > > > > > This is a little confusing in METplus 3.1 because
> INIT_BEG
> > > and
> > > >>> > > INIT_END
> > > >>> > > > > > control the time looping of execution (if
applicable for
> > the
> > > >>> use
> > > >>> > > case)
> > > >>> > > > > AND
> > > >>> > > > > > the values passed to the wrapped TCPairs config
file. In
> > > >>> METplus
> > > >>> > > 4.0.0
> > > >>> > > > > > (which was released yesterday and should be
available on
> > jet
> > > >>> > soon!),
> > > >>> > > > you
> > > >>> > > > > > can set INIT_BEG and INIT_END to control time
looping (if
> > > >>> desired)
> > > >>> > > and
> > > >>> > > > > use
> > > >>> > > > > > TC_PAIRS_INIT_BEG and TC_PAIRS_INIT_END to specify
the
> > values
> > > >>> to
> > > >>> > pass
> > > >>> > > > to
> > > >>> > > > > > the TCPairs config file.
> > > >>> > > > > >
> > > >>> > > > > > If you were using METplus 4.0.0, I would recommend
> setting
> > > >>> > something
> > > >>> > > > > like:
> > > >>> > > > > >
> > > >>> > > > > >
> > > >>> > > > > > LOOP_BY = INIT
> > > >>> > > > > > INIT_TIME_FMT = %Y
> > > >>> > > > > > INIT_BEG = 2020
> > > >>> > > > > > INIT_END = 2020
> > > >>> > > > > > INIT_INCREMENT = 21600
> > > >>> > > > > >
> > > >>> > > > > > TC_PAIRS_INIT_BEG = 20201030_000000
> > > >>> > > > > > TC_PAIRS_INIT_END = 20201031_000000
> > > >>> > > > > >
> > > >>> > > > > >
> > > >>> > > > > > This would run once using 2020 to find filenames
and
> > process
> > > >>> data
> > > >>> > > from
> > > >>> > > > > the
> > > >>> > > > > > files that are initialized between 20201030_000000
and
> > > >>> > > 20201031_000000.
> > > >>> > > > > >
> > > >>> > > > > > If using METplus 3.1, you will have to use the
INIT_BEG
> and
> > > >>> > INIT_END
> > > >>> > > > > values
> > > >>> > > > > > to set the values in the wrapped TCPairs config.
This is
> OK
> > > for
> > > >>> > this
> > > >>> > > > use
> > > >>> > > > > > case because you have LOOP_ORDER set to processes
and
> > TCPairs
> > > >>> only
> > > >>> > > runs
> > > >>> > > > > > once with that setting.
> > > >>> > > > > >
> > > >>> > > > > > LOOP_BY = INIT
> > > >>> > > > > > INIT_TIME_FMT = %Y%m%d
> > > >>> > > > > > INIT_BEG = 20201030
> > > >>> > > > > > INIT_END = 20201031
> > > >>> > > > > >
> > > >>> > > > > > This configuration should run and only use tracks
> > initialized
> > > >>> > between
> > > >>> > > > > > 2020-10-30 @ 0Z and 2020-10-31 @ 0Z. If you need
to
> specify
> > > the
> > > >>> > init
> > > >>> > > > time
> > > >>> > > > > > range with hour precision, you can do that as
well. Here
> is
> > > an
> > > >>> > > example:
> > > >>> > > > > >
> > > >>> > > > > > LOOP_BY = INIT
> > > >>> > > > > > INIT_TIME_FMT = %Y%m%d%H
> > > >>> > > > > > INIT_BEG = 2020103012
> > > >>> > > > > > INIT_END = 2020103112
> > > >>> > > > > >
> > > >>> > > > > > This configuration will only use tracks
initialized
> between
> > > >>> > > 2020-10-30
> > > >>> > > > @
> > > >>> > > > > > 12Z and 2020-10-31 @ 12Z.
> > > >>> > > > > >
> > > >>> > > > > > You can also restrict the data used by valid time
as well
> > by
> > > >>> using
> > > >>> > > > > > TC_PAIRS_VALID_BEG and TC_PAIRS_VALID_END in
METplus
> 4.0.0
> > or
> > > >>> > > VALID_BEG
> > > >>> > > > > and
> > > >>> > > > > > VALID_END for 3.1.
> > > >>> > > > > >
> > > >>> > > > > > Let me know if you are still unable to produce
output
> with
> > > >>> these
> > > >>> > > > > changes. I
> > > >>> > > > > > was able to run tc_pairs by hand on Jet and
produce a
> file
> > > with
> > > >>> > > output
> > > >>> > > > > > tracks. You can view it here if you'd like:
> > > >>> > > > > >
> > > >>> > > > > > /lfs1/HFIP/dtc-
hurr/George.Mccabe/al292020_pairs.dat.tcst
> > > >>> > > > > >
> > > >>> > > > > > These are the commands I used:
> > > >>> > > > > >
> > > >>> > > > > > export BASIN="[]";
> > > >>> > > > > > export CYCLONE="[]";
> > > >>> > > > > > export
DLAND_FILE="/contrib/met/9.1/share/met/tc_data/
> > > >>> > > > > > dland_global_tenth_degree.nc";
> > > >>> > > > > > export INIT_BEG="20201030_000000";
> > > >>> > > > > > export INIT_END="20201031_000000";
> > > >>> > > > > > export INIT_EXCLUDE="[]";
> > > >>> > > > > > export INIT_INCLUDE="[]";
> > > >>> > > > > > export MODEL="[]";
> > > >>> > > > > > export STORM_ID="[\"AL292020\"]";
> > > >>> > > > > > export STORM_NAME="[]";
> > > >>> > > > > > export VALID_BEG="";
> > > >>> > > > > > export VALID_END="";
> > > >>> > > > > > /contrib/met/9.1/bin/tc_pairs -v 4 -bdeck
> > > >>> > > > > >
> > /mnt/lfs4/HFIP/hwrf-data/hwrf-input/abdeck/btk/bal292020.dat
> > > >>> -adeck
> > > >>> > > > > >
> > /mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/data/aal292020.dat
> > > >>> > -config
> > > >>> > > > > >
> > > >>> > > > > >
> > > >>> > > > >
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/use_cases/model_applications/tc_and_extra_tc/TCPairsETCConfig
> > > >>> > > > > > -out /lfs1/HFIP/dtc-
hurr/George.Mccabe/al292020_pairs.dat
> > > >>> > > > > >
> > > >>> > > > > > I took the values from your METplus log file and
modified
> > > them
> > > >>> to
> > > >>> > > write
> > > >>> > > > > to
> > > >>> > > > > > my directory and use a different init time range.
> > > >>> > > > > >
> > > >>> > > > > > Thanks,
> > > >>> > > > > > George
> > > >>> > > > > >
> > > >>> > > > > >
> > > >>> > > > > >
> > > >>> > > > > >
> > > >>> > > > > > On Wed, May 12, 2021 at 2:04 PM Yan Jin - NOAA
Affiliate
> > via
> > > >>> RT <
> > > >>> > > > > > met_help at ucar.edu> wrote:
> > > >>> > > > > >
> > > >>> > > > > > >
> > > >>> > > > > > > <URL:
> > > >>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846 >
> > > >>> > > > > > >
> > > >>> > > > > > > Hi George,
> > > >>> > > > > > >
> > > >>> > > > > > > Thanks very much for your instructions. I set
> > > "PROCESS_LIST =
> > > >>> > > > TCPairs",
> > > >>> > > > > > and
> > > >>> > > > > > > didn't specify models: "MODEL =". Then run the
script.
> > > >>> However,
> > > >>> > the
> > > >>> > > > > > output
> > > >>> > > > > > > file is still empty. It seems that tc_pairs
matched 0
> > adeck
> > > >>> track
> > > >>> > > to
> > > >>> > > > 1
> > > >>> > > > > > > bdeck track. I also tried to set "MODEL = HWRF,
HMON",
> > but
> > > >>> the
> > > >>> > > output
> > > >>> > > > > > file
> > > >>> > > > > > > is empty too.
> > > >>> > > > > > >
> > > >>> > > > > > > Here is the log file:
> > > >>> > > > > > >
> > > >>> > > > > > >
> > > >>> > > > > >
> > > >>> > > > >
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210512193152
> > > >>> > > > > > >
> > > >>> > > > > > > And here is the output file:
> > > >>> > > > > > >
> > > >>> > > > > > >
> > > >>> > > > > >
> > > >>> > > > >
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> > > >>> > > > > > >
> > > >>> > > > > > > Many thanks.
> > > >>> > > > > > > Best,
> > > >>> > > > > > > Yan
> > > >>> > > > > > >
> > > >>> > > > > > > On Wed, May 12, 2021 at 11:36 AM George McCabe
via RT <
> > > >>> > > > > met_help at ucar.edu
> > > >>> > > > > > >
> > > >>> > > > > > > wrote:
> > > >>> > > > > > >
> > > >>> > > > > > > > Hi Yan,
> > > >>> > > > > > > >
> > > >>> > > > > > > > I was able to log into Jet this morning and
look at
> > your
> > > >>> error.
> > > >>> > > > > > > >
> > > >>> > > > > > > > It looks like the error in the call to Rscript
is due
> > to
> > > >>> there
> > > >>> > > > being
> > > >>> > > > > no
> > > >>> > > > > > > > data in the input files generated by TCPairs.
There
> are
> > > >>> > > > > inconsistencies
> > > >>> > > > > > > in
> > > >>> > > > > > > > the TCMPRPlotter wrapper that prevent the log
output
> > from
> > > >>> the
> > > >>> > run
> > > >>> > > > to
> > > >>> > > > > be
> > > >>> > > > > > > > seen, which would have made it more clear why
the
> issue
> > > was
> > > >>> > > > > occurring.
> > > >>> > > > > > > >
> > > >>> > > > > > > > I would recommend changing the PROCESS_LIST
value in
> > your
> > > >>> > METplus
> > > >>> > > > > > > > configuration file to only run TCPairs and
iterating
> on
> > > >>> that
> > > >>> > > until
> > > >>> > > > > you
> > > >>> > > > > > > get
> > > >>> > > > > > > > output from that tool. Once you can confirm
that you
> > have
> > > >>> > output,
> > > >>> > > > you
> > > >>> > > > > > can
> > > >>> > > > > > > > add TCMPRPlotter back to the PROCESS_LIST and
test on
> > > that
> > > >>> > > portion.
> > > >>> > > > > > > >
> > > >>> > > > > > > > I will create a GitHub issue for updating the
> > > TCMPRPlotter
> > > >>> > > wrapper
> > > >>> > > > to
> > > >>> > > > > > > > produce useful logging output that will make
things
> > more
> > > >>> clear
> > > >>> > > > going
> > > >>> > > > > > > > forward.
> > > >>> > > > > > > >
> > > >>> > > > > > > > Let me know if you run into any other issues
or need
> > help
> > > >>> > > > configuring
> > > >>> > > > > > > > TCPairs to run properly. I'm also happy to
attend
> > another
> > > >>> > working
> > > >>> > > > > > session
> > > >>> > > > > > > > if that is helpful.
> > > >>> > > > > > > >
> > > >>> > > > > > > > Thanks,
> > > >>> > > > > > > > George
> > > >>> > > > > > > >
> > > >>> > > > > > > >
> > > >>> > > > > > > > On Tue May 11 10:05:06 2021, yan.jin at noaa.gov
wrote:
> > > >>> > > > > > > > > Hi John,
> > > >>> > > > > > > > >
> > > >>> > > > > > > > >
> > > >>> > > > > > > > > The directories and data I mentioned in my
last
> email
> > > >>> are all
> > > >>> > > > > located
> > > >>> > > > > > > > > in
> > > >>> > > > > > > > > Jet.
> > > >>> > > > > > > > >
> > > >>> > > > > > > > >
> > > >>> > > > > > > > > Here is the email I got regarding the Jet
> > maintenance:
> > > >>> > > > > > > > >
> > > >>> > > > > > > > > ***
> > > >>> > > > > > > > >
> > > >>> > > > > > > > > *05/11/21 - Boulder Bastion Maintenance
Downtime*
> > > >>> > > > > > > > > Boulder Bastion maintenance is scheduled for
> Tuesday
> > > >>> 05/11/21
> > > >>> > > > from
> > > >>> > > > > > > > > 0600 ~
> > > >>> > > > > > > > > 1000 MT. Boulder Bastion login sessions will
be
> > dropped
> > > >>> and
> > > >>> > all
> > > >>> > > > new
> > > >>> > > > > > > > > RDHPCS
> > > >>> > > > > > > > > logins will need to go through the Princeton
> Bastion.
> > > >>> > Logging
> > > >>> > > In
> > > >>> > > > > > > > > <
> > > >>> > > > > >
> > > >>> >
> > https://rdhpcs-common-
docs.rdhpcs.noaa.gov/wiki/index.php/Logging_in
> > > >>> > > >
> > > >>> > > > > > > > >
> > > >>> > > > > > > > >
> > > >>> > > > > > > > >
> > > >>> > > > > > > > > *The next Jet Maintenance Downtime is TBD*
> > > >>> > > > > > > > >
> > > >>> > > > > > > > > ***
> > > >>> > > > > > > > >
> > > >>> > > > > > > > >
> > > >>> > > > > > > > > I tried to login Princeton but the
connection is
> > closed
> > > >>> too.
> > > >>> > > > > > > > >
> > > >>> > > > > > > > >
> > > >>> > > > > > > > > Really appreciate your time and help on
this.
> > > >>> > > > > > > > >
> > > >>> > > > > > > > >
> > > >>> > > > > > > > > Best,
> > > >>> > > > > > > > >
> > > >>> > > > > > > > > Yan
> > > >>> > > > > > > > >
> > > >>> > > > > > > > > On Tue, May 11, 2021 at 11:57 AM John Halley
Gotway
> > via
> > > >>> RT <
> > > >>> > > > > > > > > met_help at ucar.edu> wrote:
> > > >>> > > > > > > > >
> > > >>> > > > > > > > > > Yan,
> > > >>> > > > > > > > > >
> > > >>> > > > > > > > > > George and I tried to take a look at this,
but
> > > couldn't
> > > >>> > find
> > > >>> > > > the
> > > >>> > > > > > > > > > data.
> > > >>> > > > > > > > > >
> > > >>> > > > > > > > > >
> > > >>> > > > > > > >
> > > >>> > > > > > >
> > > >>> > > > > >
> > > >>> > > > >
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210510214154
> > > >>> > > > > > > > > >
> > > >>> > > > > > > > > > I checked on Hera and WCOSS but couldn't
find the
> > > data
> > > >>> > > there. I
> > > >>> > > > > > > > > > should be
> > > >>> > > > > > > > > > able to log on to jet, but when I try, it
boots
> me
> > > off:
> > > >>> > > > > > > > > >
> > > >>> > > > > > > > > > Authentication failed.
> > > >>> > > > > > > > > > Connection to jet-
rsa.boulder.rdhpcs.noaa.gov
> > > closed.
> > > >>> > > > > > > > > >
> > > >>> > > > > > > > > > Can you confirm that you're running on
jet? And
> any
> > > >>> idea of
> > > >>> > > jet
> > > >>> > > > > is
> > > >>> > > > > > > > > > down
> > > >>> > > > > > > > > > today or something?
> > > >>> > > > > > > > > >
> > > >>> > > > > > > > > > Thanks,
> > > >>> > > > > > > > > > John
> > > >>> > > > > > > > > >
> > > >>> > > > > > > > > >
> > > >>> > > > > > > > > > On Tue, May 11, 2021 at 7:46 AM Yan Jin -
NOAA
> > > >>> Affiliate
> > > >>> > via
> > > >>> > > > RT <
> > > >>> > > > > > > > > > met_help at ucar.edu> wrote:
> > > >>> > > > > > > > > >
> > > >>> > > > > > > > > > >
> > > >>> > > > > > > > > > > <URL:
> > > >>> > > >
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846
> > > >>> > > > > >
> > > >>> > > > > > > > > > >
> > > >>> > > > > > > > > > > Hi John,
> > > >>> > > > > > > > > > >
> > > >>> > > > > > > > > > > Thanks so much!
> > > >>> > > > > > > > > > >
> > > >>> > > > > > > > > > > Best,
> > > >>> > > > > > > > > > > Yan
> > > >>> > > > > > > > > > >
> > > >>> > > > > > > > > > >
> > > >>> > > > > > > > > > > On Tue, May 11, 2021 at 9:44 AM John
Halley
> > Gotway
> > > >>> via
> > > >>> > RT <
> > > >>> > > > > > > > > > > met_help at ucar.edu>
> > > >>> > > > > > > > > > > wrote:
> > > >>> > > > > > > > > > >
> > > >>> > > > > > > > > > > > Hi Yan,
> > > >>> > > > > > > > > > > >
> > > >>> > > > > > > > > > > > I'm going to assign this ticket to
George and
> > ask
> > > >>> him
> > > >>> > to
> > > >>> > > > > take a
> > > >>> > > > > > > > > > > > look
> > > >>> > > > > > > > > > and
> > > >>> > > > > > > > > > > > debug this runtime error.
> > > >>> > > > > > > > > > > >
> > > >>> > > > > > > > > > > > Thanks,
> > > >>> > > > > > > > > > > > John HG
> > > >>> > > > > > > > > > > >
> > > >>> > > > > > > > > > > > On Mon, May 10, 2021 at 3:49 PM Yan
Jin -
> NOAA
> > > >>> > Affiliate
> > > >>> > > > via
> > > >>> > > > > > RT <
> > > >>> > > > > > > > > > > > met_help at ucar.edu> wrote:
> > > >>> > > > > > > > > > > >
> > > >>> > > > > > > > > > > > >
> > > >>> > > > > > > > > > > > > Mon May 10 15:49:51 2021: Request
99846 was
> > > acted
> > > >>> > upon.
> > > >>> > > > > > > > > > > > > Transaction: Ticket created by
> > > yan.jin at noaa.gov
> > > >>> > > > > > > > > > > > >        Queue: met_help
> > > >>> > > > > > > > > > > > >      Subject: ask a question about
METplus
> > > >>> config of
> > > >>> > TC
> > > >>> > > > > > > > > > > > >        Owner: Nobody
> > > >>> > > > > > > > > > > > >   Requestors: yan.jin at noaa.gov
> > > >>> > > > > > > > > > > > >       Status: new
> > > >>> > > > > > > > > > > > >  Ticket <URL:
> > > >>> > > > > > > > > >
> > > >>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846
> > > >>> > > > > > > > > > > >
> > > >>> > > > > > > > > > > > >
> > > >>> > > > > > > > > > > > >
> > > >>> > > > > > > > > > > > > Hi,
> > > >>> > > > > > > > > > > > >
> > > >>> > > > > > > > > > > > > This is Yan, again. I ran the
METplus
> config
> > > file
> > > >>> > after
> > > >>> > > > our
> > > >>> > > > > > > > > > discussion,
> > > >>> > > > > > > > > > > > > however, it seems that the tc_pairs
tcst
> file
> > > >>> wasn't
> > > >>> > > > > > generated.
> > > >>> > > > > > > > > > > > >
> > > >>> > > > > > > > > > > > > Here is the script in my directory
on Jet:
> > > >>> > > > > > > > > > > > > I set TC_PAIRS_STORM_ID = AL292020,
so the
> > code
> > > >>> can
> > > >>> > > find
> > > >>> > > > > the
> > > >>> > > > > > > > > > > > > input
> > > >>> > > > > > > > > > > adeck
> > > >>> > > > > > > > > > > > > and bdeck data correctly
> > > >>> > > > > > > > > > > > > *----
> > > >>> > > > > > > > > > > > >
> > > >>> > > > > > > > > > > > >
> > > >>> > > > > > > > > > > >
> > > >>> > > > > > > > > > >
> > > >>> > > > > > > > > >
> > > >>> > > > > > > >
> > > >>> > > > > > >
> > > >>> > > > > >
> > > >>> > > > >
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test.conf
> > > >>> > > > > > > > > > > > >
> > > >>> > > > > > > > > > > > > The output directory is
> > > >>> > > > > > > > > > > > >
> > > >>> > > > > > > >
> > > >>> > >
> >
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs
> > > >>> > > > > > > > > > > > > and the tc_pairs data:
> > al292020_pairs.dat.tcst
> > > >>> > > > > > > > > > > > >
> > > >>> > > > > > > > > > > > > log file
> > > >>> > > > > > > > > > > > >
> > > >>> > > > > > > > > > > > >
> > > >>> > > > > > > > > > > >
> > > >>> > > > > > > > > > >
> > > >>> > > > > > > > > >
> > > >>> > > > > > > >
> > > >>> > > > > > >
> > > >>> > > > > >
> > > >>> > > > >
> > > >>> > > >
> > > >>> > >
> > > >>> >
> > > >>>
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210510214154
> > > >>> > > > > > > > > > > > > *-----
> > > >>> > > > > > > > > > > > >
> > > >>> > > > > > > > > > > > > The error starts:[image: image.png]
> > > >>> > > > > > > > > > > > >
> > > >>> > > > > > > > > > > > > Then I checked
al292020_pairs.dat.tcst, and
> > it
> > > is
> > > >>> > > empty.
> > > >>> > > > > > > > > > > > >
> > > >>> > > > > > > > > > > > > I wonder if there is anything I
missed?
> > > >>> > > > > > > > > > > > >
> > > >>> > > > > > > > > > > > > Many thanks.
> > > >>> > > > > > > > > > > > > Best,
> > > >>> > > > > > > > > > > > > Yan
> > > >>> > > > > > > > > > > > >
> > > >>> > > > > > > > > > > > >
> > > >>> > > > > > > > > > > >
> > > >>> > > > > > > > > > > >
> > > >>> > > > > > > > > > >
> > > >>> > > > > > > > > > >
> > > >>> > > > > > > > > >
> > > >>> > > > > > > > > >
> > > >>> > > > > > > >
> > > >>> > > > > > > >
> > > >>> > > > > > > >
> > > >>> > > > > > > >
> > > >>> > > > > > >
> > > >>> > > > > > >
> > > >>> > > > > >
> > > >>> > > > > > --
> > > >>> > > > > > George McCabe - Software Engineer III
> > > >>> > > > > > National Center for Atmospheric Research
> > > >>> > > > > > Research Applications Laboratory
> > > >>> > > > > > 303-497-2768
> > > >>> > > > > > ---
> > > >>> > > > > > My working day may not be your working day. Please
do not
> > > feel
> > > >>> > > obliged
> > > >>> > > > to
> > > >>> > > > > > reply to this email outside of your normal working
hours.
> > > >>> > > > > >
> > > >>> > > > > >
> > > >>> > > > >
> > > >>> > > > >
> > > >>> > > >
> > > >>> > > > --
> > > >>> > > > George McCabe - Software Engineer III
> > > >>> > > > National Center for Atmospheric Research
> > > >>> > > > Research Applications Laboratory
> > > >>> > > > 303-497-2768
> > > >>> > > > ---
> > > >>> > > > My working day may not be your working day. Please do
not
> feel
> > > >>> obliged
> > > >>> > to
> > > >>> > > > reply to this email outside of your normal working
hours.
> > > >>> > > >
> > > >>> > > >
> > > >>> > >
> > > >>> > >
> > > >>> >
> > > >>> > --
> > > >>> > George McCabe - Software Engineer III
> > > >>> > National Center for Atmospheric Research
> > > >>> > Research Applications Laboratory
> > > >>> > 303-497-2768
> > > >>> > ---
> > > >>> > My working day may not be your working day. Please do not
feel
> > > obliged
> > > >>> to
> > > >>> > reply to this email outside of your normal working hours.
> > > >>> >
> > > >>> >
> > > >>>
> > > >>>
> > > >>
> > > >> --
> > > >> George McCabe - Software Engineer III
> > > >> National Center for Atmospheric Research
> > > >> Research Applications Laboratory
> > > >> 303-497-2768
> > > >> ---
> > > >> My working day may not be your working day. Please do not
feel
> obliged
> > > to
> > > >> reply to this email outside of your normal working hours.
> > > >>
> > > >
> > > >
> > > > --
> > > > George McCabe - Software Engineer III
> > > > National Center for Atmospheric Research
> > > > Research Applications Laboratory
> > > > 303-497-2768
> > > > ---
> > > > My working day may not be your working day. Please do not feel
> obliged
> > to
> > > > reply to this email outside of your normal working hours.
> > > >
> > >
> > >
> > > --
> > > George McCabe - Software Engineer III
> > > National Center for Atmospheric Research
> > > Research Applications Laboratory
> > > 303-497-2768
> > > ---
> > > My working day may not be your working day. Please do not feel
obliged
> to
> > > reply to this email outside of your normal working hours.
> > >
> > >
> >
> >
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>

------------------------------------------------
Subject: ask a question about METplus config of TC
From: George McCabe
Time: Thu May 27 08:13:38 2021

Hi Yan,

That is great to hear! I looked at your output files and they do look
correct. It looks like I made the fix to handling the cyclone list
before
the 4.0.0 release. I did make other improvements to the TCPairs
wrapper
including adding support to set consensus through the METplus config.
I put
the code on Jet so you can test that it works for your case. To use
this
version, run the module commands to set up METplus, then instead of
running
run_metplus.py from the path, run the script from my directory,
passing in
the same config files that you were previously using.

/home/George.Mccabe/George.Mccabe/METplus.feature_749/ush/run_metplus.py
...

To set the consensus values from your previous email:

consensus = [
 {
  name = "CONS";
  members = ["HWRF","GFSO","HMON"];
  required = [false,false,false];
  min_req = 2;
 }

You can set the following in your METplus config file:

TC_PAIRS_CONSENSUS1_NAME = CONS
TC_PAIRS_CONSENSUS1_MEMBERS = HWRF, GFSO, HMON
TC_PAIRS_CONSENSUS1_REQUIRED = false, false, false
TC_PAIRS_CONSENSUS1_MIN_REQ = 2

If you need more than 1 consensus dictionary defined, you can add it
by
using TC_PAIRS_CONSENSUS2_*.

Let me know how it goes!

Thanks,
George

On Wed, May 26, 2021 at 2:57 PM Yan Jin - NOAA Affiliate via RT <
met_help at ucar.edu> wrote:

>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846 >
>
> Hi George,
>
> Thanks so much. I am not in a hurry. It is totally fine to wait
until it is
> ready after you are done with your current work.
>
> I checked the files in
> /mnt/lfs4/HFIP/hwrfv3/Yan.Jin/vrfy_METplus4.0/AL_MJJA_2020/tc_pairs,
and
> each file contain the result for that cyclone number.
> e.g.: al012020_pairs.dat.tcst has the initial time
20200515~20200520.
>
> Many thanks.
> Best,
> Yan
>
> On Wed, May 26, 2021 at 3:38 PM George McCabe via RT
<met_help at ucar.edu>
> wrote:
>
> > Hi Yan,
> >
> > The way the wrapper is currently written, specifying that list of
> cyclones
> > in TC_PAIRS_CYCLONE will result in 15 output files that all
contain all
> 15
> > cyclones. I am currently working on updating the tool to handle
this
> > properly so that each file will only contain the cyclone in
question.
> Your
> > configuration looks correct to me, but it will not produce the
correct
> > behavior until these updates are finished. When it is ready, I can
make
> it
> > available to you on Jet so you can test that it runs as expected.
> >
> > I am actually finished with this fix but I am now working on
implementing
> > support for setting consensus through METplus. If you'd like, I
can put
> the
> > code from my development branch on Jet so you can test that this
works
> > while I am still working on consensus. Let me know if you'd like
me to do
> > this and I can make it available to you.
> >
> > Thanks,
> > George
> >
> > - George
> >
> > On Wed, May 26, 2021 at 12:54 PM Yan Jin - NOAA Affiliate via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846 >
> > >
> > > Hi George,
> > >
> > > Can I ask a question regarding the seasonal mean TC
verification? For
> > > instance, I have adeck and bdeck data over Atlantic at 2020,
(Jet:
> > > /mnt/lfs4/HFIP/hwrf-data/hwrf-input/abdeck), each has 01-99
cyclone
> > number.
> > > And I would like to calculate the absolute intensity error and
track
> > error
> > > of seasonal mean over Atlantic. Here is my method, could you
check if
> it
> > is
> > > correct to do that?
> > >
> > > I plan to do the May-June-July-August mean error.
> > > $WORK:
> > >
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/vrfy_METplus4.0/METplus/parm/user_config
> > > in this directory, the METplus config file is
TCMPRPlotter_AL_MJJA_2020
> > >
> > >
> > > The major modification I did is:
> > > ***
> > > MODEL = HWRF,HMON,GFSO
> > >
> > > TC_PAIRS_DESC =
> > >
> > > TC_PAIRS_STORM_ID =
> > > TC_PAIRS_BASIN = AL
> > > TC_PAIRS_CYCLONE = 01,02,03,04,05,06,07,08,09,10,11,12,13,14,15
> > > TC_PAIRS_STORM_NAME =
> > >
> > > TC_PAIRS_INIT_BEG = 20200515_000000
> > > TC_PAIRS_INIT_END = 20200906_000000
> > > ***
> > > I checked the adeck and bdeck data, and found that cyclone
number 01-15
> > > have the initial times which are located within MJJA. So I
> > > specified TC_PAIRS_CYCLONE with 01 to 15. I also checked the
initial
> time
> > > in cyclone 01 and 15, and found the INIT_BEG and INIT_END.
> > >
> > > It generated 15 tc_pairs.tcst files, and also 6 figures (Output
> > directory:
> > > /mnt/lfs4/HFIP/hwrfv3/Yan.Jin/vrfy_METplus4.0/AL_MJJA_2020)
> > >
> > > Here is the result
> > >
> > >
> >
> https://docs.google.com/document/d/10adrHdzNNxMJM51e5Sw9oy6uS2n1S-
qGS2c7NZdVVxc/edit?pli=1
> > >
> > > I wonder if my method is correct? It seems that
TC_PAIRS_INIT_BEG
> > > and TC_PAIRS_INIT_END are mandatory for my case.
> > >
> > > Many thanks.
> > > Best,
> > > Yan
> > >
> > > On Thu, May 20, 2021 at 12:01 PM George McCabe via RT <
> met_help at ucar.edu
> > >
> > > wrote:
> > >
> > > > Hi Yan,
> > > >
> > > > I was able to change the line colors in the images. The images
can be
> > > found
> > > > on Jet here:
> > > >
> > > > /lfs1/HFIP/dtc-hurr/George.Mccabe/eta-out-colors/tcmpr_plots
> > > >
> > > >
> > > > This is the command I used to run:
> > > >
> > > > run_metplus.py
> > > > /lfs1/HFIP/dtc-hurr/George.Mccabe/TCMPRPlotter_Eta_colors.conf
> > > > config.OUTPUT_BASE=/lfs1/HFIP/dtc-hurr/George.Mccabe/eta-out-
colors
> > > >
> > > >
> > > > I copied the default tcmpr file from
> > > >
> > >
> >
>
/contrib/met/10.0.0/share/met/Rscripts/include/plot_tcmpr_config_default.R
> > > > to my directory and renamed it to plot_tcmpr_overrides.R
> > > >
> > > > Then I modified the METplus config to point to that file
instead of
> the
> > > > file in the METplus repository:
> > > >
> > > > TCMPR_PLOTTER_CONFIG_FILE =
> > > > /lfs1/HFIP/dtc-hurr/George.Mccabe/plot_tcmpr_overrides.R
> > > >
> > > >
> > > > The previous tcmpr parameter file changes the image
resolution, so I
> > had
> > > to
> > > > set that in the new file to prevent the image from being very
large.
> > > Then I
> > > > changed the value of the default color list:
> > > >
> > > > img_res = 72;
> > > > default_color_list  = c("blue", "green", "purple", "orange");
> > > >
> > > > This variable is used to set other variables for each plot
type, so
> you
> > > can
> > > > modify those variables if you desire different colors for
different
> > > plots.
> > > > You only need to supply the variables you want to change from
the
> > default
> > > > value, so your overrides file can just include the above 2
variables
> to
> > > > keep the file short and clean. However, it is nice to have all
of the
> > > > possible options easily available
> > > > in case you want to change other settings. How you handle this
is up
> to
> > > > your preference.
> > > >
> > > > Let me know if you have any other questions.
> > > >
> > > > Thanks,
> > > > George
> > > >
> > > > On Thu, May 20, 2021 at 9:19 AM George McCabe
<mccabe at ucar.edu>
> wrote:
> > > >
> > > > > Hi Yan,
> > > > >
> > > > > I copied your config file and made some modifications. You
can view
> > it
> > > on
> > > > > Jet here:
> > > > >
> > > > > /lfs1/HFIP/dtc-hurr/George.Mccabe/TCMPRPlotter_Eta.conf
> > > > >
> > > > > Starting in METplus 4.0.0 the different section headers like
dir
> and
> > > > > filename_templates are no longer necessary. All
configurations can
> go
> > > > under
> > > > > the [config] header. I removed the sections and rearranged
the
> > > variables
> > > > to
> > > > > group them in a more clear way.
> > > > >
> > > > > To call TCMPRPlotter multiple times with different settings,
I
> added
> > > > > multiple instances of the app in the PROCESS_LIST with
instance
> names
> > > > > defined:
> > > > >
> > > > > PROCESS_LIST = TCPairs, TCMPRPlotter(abs_mean),
> > > > > TCMPRPlotter(abs_median),TCMPRPlotter(abs_boxplot),
> > > > TCMPRPlotter(tk_mean),
> > > > > TCMPRPlotter(tk_median),TCMPRPlotter(tk_boxplot)
> > > > >
> > > > > Each instance will read all of the values set in the
[config]
> > section,
> > > > > then it will look for a section header that matches the
string
> inside
> > > the
> > > > > parentheses for each instance. If the section is found, it
> overrides
> > > the
> > > > > values set in the section to replace the values set in the
[config]
> > > > > section. These names can be whatever you chose, but the name
in the
> > > > > PROCESS_LIST must match the name of the section header.
> > > > >
> > > > > [abs_mean]
> > > > > TCMPR_PLOTTER_PREFIX = Eta_Absolute_Intensity_mean
> > > > > TCMPR_PLOTTER_TITLE = Absolute Intensity Error Mean\n Eta
(Atlantic
> > > 2020)
> > > > > TCMPR_PLOTTER_DEP_VARS = ABS(AMAX_WIND-BMAX_WIND)
> > > > > TCMPR_PLOTTER_PLOT_TYPES = MEAN
> > > > >
> > > > > [abs_median]
> > > > > TCMPR_PLOTTER_PREFIX = Eta_Absolute_Intensity_median
> > > > > TCMPR_PLOTTER_TITLE = Absolute Intensity Error Median\n Eta
> (Atlantic
> > > > 2020)
> > > > > TCMPR_PLOTTER_DEP_VARS = ABS(AMAX_WIND-BMAX_WIND)
> > > > > TCMPR_PLOTTER_PLOT_TYPES = MEDIAN
> > > > >
> > > > > [abs_boxplot]
> > > > > TCMPR_PLOTTER_PREFIX = Eta_Absolute_Intensity_boxplot
> > > > > TCMPR_PLOTTER_TITLE = Absolute Intensity Error Boxplot\n Eta
> > (Atlantic
> > > > > 2020)
> > > > > TCMPR_PLOTTER_DEP_VARS = ABS(AMAX_WIND-BMAX_WIND)
> > > > > TCMPR_PLOTTER_PLOT_TYPES = BOXPLOT
> > > > >
> > > > > [tk_mean]
> > > > > TCMPR_PLOTTER_PREFIX = Eta_Track_Error_mean
> > > > > TCMPR_PLOTTER_TITLE = Track Error Mean\n Eta (Atlantic 2020)
> > > > > TCMPR_PLOTTER_DEP_VARS = TK_ERR
> > > > > TCMPR_PLOTTER_PLOT_TYPES = MEAN
> > > > >
> > > > > [tk_median]
> > > > > TCMPR_PLOTTER_PREFIX = Eta_Track_Error_median
> > > > > TCMPR_PLOTTER_TITLE = Track Error Median\n Eta (Atlantic
2020)
> > > > > TCMPR_PLOTTER_DEP_VARS = TK_ERR
> > > > > TCMPR_PLOTTER_PLOT_TYPES = MEDIAN
> > > > >
> > > > > [tk_boxplot]
> > > > > TCMPR_PLOTTER_PREFIX = Eta_Track_Error_boxplot
> > > > > TCMPR_PLOTTER_TITLE = Track Error Boxplot\n Eta (Atlantic
2020)
> > > > > TCMPR_PLOTTER_DEP_VARS = TK_ERR
> > > > > TCMPR_PLOTTER_PLOT_TYPES = BOXPLOT
> > > > >
> > > > >
> > > > > I split up the DEP_VARS and PLOT_TYPES values into instances
with
> one
> > > > > value of each and set the plot title and filenames
appropriately
> for
> > > > each.
> > > > > I set the X label to "Lead Time (h)" in the [config] section
so it
> > > > applies
> > > > > to every instance.
> > > > >
> > > > > TCMPR_PLOTTER_XLAB = Lead Time (h)
> > > > >
> > > > > The output images from my run is on Jet here:
> > > > >
> > > > > /lfs1/HFIP/dtc-hurr/George.Mccabe/eta-out/tcmpr_plots
> > > > >
> > > > > Eta_Absolute_Intensity_boxplot.log
> > > > > Eta_Absolute_Intensity_boxplot.png
> > > > > Eta_Absolute_Intensity_mean.png
> > > > > Eta_Absolute_Intensity_median.png
> > > > > Eta_Track_Error_boxplot.log
> > > > > Eta_Track_Error_boxplot.png
> > > > > Eta_Track_Error_mean.png
> > > > > Eta_Track_Error_median.png
> > > > >
> > > > >
> > > > > Here is the log file from the run:
> > > > >
> > >
> /lfs1/HFIP/dtc-hurr/George.Mccabe/eta-
out/logs/metplus.log.20210520150804
> > > > >
> > > > >
> > > > > Here is what I ran to produce the output:
> > > > >
> > > > >
> > > > >    - module load intel
> > > > >    - module load intelpython/3.6.5
> > > > >    - module load netcdf/4.6.1
> > > > >    - module load hdf5/1.10.4
> > > > >    - module load nco/4.9.1
> > > > >    - module load wgrib/1.8.1.0b
> > > > >    - module load wgrib2/2.0.8
> > > > >    - module load R/4.0.2
> > > > >    - module use /contrib/met/modulefiles
> > > > >    - module load met/10.0.0
> > > > >    - module use /contrib/met/METplus/modulefiles
> > > > >    - module load metplus/4.0.0
> > > > >    - run_metplus.py
> > > > >    /lfs1/HFIP/dtc-hurr/George.Mccabe/TCMPRPlotter_Eta.conf
> > > > >    config.OUTPUT_BASE=/lfs1/HFIP/dtc-hurr/George.Mccabe/eta-
out
> > > > >
> > > > >
> > > > >
> > > > > I will look into what settings are needed to change the line
> colors.
> > > > Also,
> > > > > I have started making the modifications to the TCPairs
wrapper to
> > > > properly
> > > > > handle multiple storms specified in the config file. I will
let you
> > > know
> > > > > when that is ready to test.
> > > > >
> > > > > Thanks,
> > > > > George
> > > > >
> > > > >
> > > > >
> > > > > On Wed, May 19, 2021 at 5:09 PM George McCabe
<mccabe at ucar.edu>
> > wrote:
> > > > >
> > > > >> Hi Yan,
> > > > >>
> > > > >> 1) add TC name to each plot name
> > > > >> 2) modify titles to each plot, e.g.: TK_ERR* plots with
name
> "Track
> > > > >> Error", and ABS_AMAX_WIND-BMAX_WIND_* plots with name
"Absolute
> > > > Intensity
> > > > >> Error"?
> > > > >>
> > > > >> It looks like the logic in the R script uses the -prefix
value to
> > > > replace
> > > > >> the filename instead of prepending it to the plot type text
as you
> > > would
> > > > >> guess it would. I was told that this script was typically
called
> > > > multiple
> > > > >> times to generate the multiple plots, which would provide
more
> > control
> > > > over
> > > > >> setting the output filenames, titles, etc. The bad news is
it
> looks
> > > like
> > > > >> the wrapper was written to call the R script once with all
of the
> > > > >> configurations specified in the METplus config file. The
good news
> > is
> > > > there
> > > > >> are some new capabilities added to METplus 4.0.0 that could
help
> > > produce
> > > > >> the results you need.
> > > > >>
> > > > >> Please review this section of the METplus User's Guide,
> specifically
> > > the
> > > > >> section about specifying instance names to the processes:
> > > > >>
> > > > >>
> > > >
> > >
> >
>
https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#process-
list
> > > > >>
> > > > >> Alternatively, you can use the UserScript wrapper to build
custom
> > > > >> commands to call the R script directly with any arguments
you need
> > if
> > > > the
> > > > >> wrapper doesn't support an option you need.
> > > > >> More information on that tool is here:
> > > > >>
> > > >
> > >
> >
>
https://metplus.readthedocs.io/en/latest/Users_Guide/wrappers.html#userscript
> > > > >> and here:
> > > > >>
> > > >
> > >
> >
>
https://metplus.readthedocs.io/en/latest/generated/met_tool_wrapper/UserScript/UserScript_run_once.html#sphx-
glr-generated-met-tool-wrapper-userscript-userscript-run-once-py
> > > > >>
> > > > >> I think using different instance names to run TCMPRPlotter
> multiple
> > > > times
> > > > >> with different settings is the better approach in this
case. I am
> > > > wrapping
> > > > >> up work for the day, but tomorrow I can log into Jet and
help you
> > > modify
> > > > >> your config file to do this.
> > > > >>
> > > > >>
> > > > >> 3) Is it possible to modify the line colors in this METplus
config
> > > file?
> > > > >>
> > > > >> Yes, you can change the line colors through the R script's
config
> > > file.
> > > > >> You can copy and rename this file:
> > > > >>
> > > > >>
> > > > >>
> > > >
> > >
> >
>
https://github.com/dtcenter/MET/blob/main_v10.0/met/scripts/Rscripts/include/plot_tcmpr_config_default.R
> > > > >>
> > > > >> change the values that you need to change (look for
"color_list"),
> > > then
> > > > >> set the METplus configuration file to point to your new
file (set
> in
> > > > >> TCMPR_PLOTTER_CONFIG_FILE).
> > > > >>
> > > > >>
> > > > >> Let me know if you have any other questions. I will report
back
> > > tomorrow
> > > > >> when have some changes for you to test.
> > > > >>
> > > > >> Thanks,
> > > > >> George
> > > > >>
> > > > >> On Wed, May 19, 2021 at 4:00 PM Yan Jin - NOAA Affiliate
via RT <
> > > > >> met_help at ucar.edu> wrote:
> > > > >>
> > > > >>>
> > > > >>> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846 >
> > > > >>>
> > > > >>> Hi George,
> > > > >>>
> > > > >>> I have a few questions regarding the TCMPR_PLOTTER_*
parameters
> in
> > > the
> > > > >>> METplus config file.
> > > > >>>
> > > > >>> Here is the METplus config file I used:
> > > > >>>
> > > > >>>
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/vrfy_METplus4.0/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test.conf
> > > > >>> **
> > > > >>> #  Plot_TCMPR options, if left unset, default values that
are
> > > > >>> #  pre-defined in the R utility (packaged with MET) will
be used.
> > > > >>> TCMPR_PLOTTER_CONFIG_FILE =
> > {CONFIG_DIR}/TCMPRPlotterConfig_customize
> > > > >>> TCMPR_PLOTTER_PREFIX =
> > > > >>> TCMPR_PLOTTER_TITLE =
> > > > >>> TCMPR_PLOTTER_SUBTITLE =
> > > > >>> TCMPR_PLOTTER_XLAB =
> > > > >>> TCMPR_PLOTTER_YLAB =
> > > > >>> TCMPR_PLOTTER_XLIM =
> > > > >>> TCMPR_PLOTTER_YLIM =
> > > > >>> TCMPR_PLOTTER_FILTER = -amodel HWRF,HMON
> > > > >>> # the tcst data file to be used instead of running the MET
> tc_stat
> > > > tool.
> > > > >>> TCMPR_PLOTTER_FILTERED_TCST_DATA_FILE =
> > > > >>> # Comma separated, no whitespace.  Default is TK_ERR
(track
> error)
> > > > unless
> > > > >>> # otherwise indicated.
> > > > >>> TCMPR_PLOTTER_DEP_VARS = ABS(AMAX_WIND-BMAX_WIND),TK_ERR
> > > > >>> TCMPR_PLOTTER_SCATTER_X =
> > > > >>> TCMPR_PLOTTER_SCATTER_Y =
> > > > >>> TCMPR_PLOTTER_SKILL_REF =
> > > > >>> TCMPR_PLOTTER_SERIES = AMODEL
> > > > >>> TCMPR_PLOTTER_SERIES_CI = TRUE
> > > > >>> TCMPR_PLOTTER_LEGEND =
> > > > >>> TCMPR_PLOTTER_LEAD = 0,12,24,36,48,60,72,84,96,108,120
> > > > >>> # Mean and median plots. These override the plot_tcmpr.R
default
> of
> > > box
> > > > >>> plot.
> > > > >>> # If box plot is desired, this needs to be explicitly
indicated.
> > > > >>> TCMPR_PLOTTER_PLOT_TYPES = MEAN,MEDIAN,BOXPLOT
> > > > >>> TCMPR_PLOTTER_RP_DIFF =
> > > > >>> TCMPR_PLOTTER_DEMO_YR =
> > > > >>> TCMPR_PLOTTER_HFIP_BASELINE =
> > > > >>> TCMPR_PLOTTER_FOOTNOTE_FLAG =
> > > > >>> TCMPR_PLOTTER_PLOT_CONFIG_OPTS =
> > > > >>> TCMPR_PLOTTER_SAVE_DATA =
> > > > >>>
> > > > >>> #  TCMPR FLAGS no == (don't set flag), yes == (set flag)
> > > > >>> TCMPR_PLOTTER_NO_EE = no
> > > > >>> TCMPR_PLOTTER_NO_LOG = no
> > > > >>> TCMPR_PLOTTER_SAVE = no
> > > > >>> **
> > > > >>> It generated the following 6 *.png files
> > > > >>> in
> > > > >>>
> > > >
> >
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/vrfy_METplus4.0/Eta_use_METplus/tcmpr_plots
> > > > >>> [image: image.png]
> > > > >>>
> > > > >>> Then I would like to modify titles, xlab text, output file
names.
> > > > >>> Here are the modification I
> > > > >>> did
> > > > >>>
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/vrfy_METplus4.0/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test1.conf
> > > > >>>
> > > > >>> **
> > > > >>> TCMPR_PLOTTER_TITLE = Absolute Intensity Error \n Eta
(Atlantic
> > 2020)
> > > > >>> TCMPR_PLOTTER_XLAB = Lead Time (h)
> > > > >>> **
> > > > >>> The title of each plot becomes "Absolute Intensity Error
Eta
> > > (Atlantic
> > > > >>> 2020)" with double quotes.
> > > > >>>
> > > > >>> I also tried to set TCMPR_PLOTTER_PREFIX = Eta_ because I
would
> > like
> > > to
> > > > >>> save each plot as Eta_ABS_AMAX_WIND-BMAX_WIND_*.png, and
> > > > >>> Eta_TK_ERR_*.png.
> > > > >>> However, there is only one plot Eta_.png generated.
> > > > >>>
> > > > >>> I would like to check how can I set those parameters if I
would
> > like
> > > to
> > > > >>> 1)
> > > > >>> add TC name to each plot name; 2) modify titles to each
plot,
> e.g.:
> > > > >>> TK_ERR*
> > > > >>> plots with name "Track Error", and ABS_AMAX_WIND-
BMAX_WIND_*
> plots
> > > with
> > > > >>> name "Absolute Intensity Error"? 3) Is it possible to
modify the
> > line
> > > > >>> colors in this METplus config file?
> > > > >>>
> > > > >>> Could you give me some suggestions?
> > > > >>>
> > > > >>> Many thanks.
> > > > >>> Best,
> > > > >>> Yan
> > > > >>>
> > > > >>> On Mon, May 17, 2021 at 10:37 AM George McCabe via RT <
> > > > met_help at ucar.edu
> > > > >>> >
> > > > >>> wrote:
> > > > >>>
> > > > >>> > Hi Yan,
> > > > >>> >
> > > > >>> > 4.1:
> > > > >>> > The consensus variable is not yet controlled by a
METplus
> > > > configuration
> > > > >>> > variable explicitly, but you can set the value by using
the
> > > > >>> > TC_PAIRS_MET_CONFIG_OVERRIDES value (in v4.0.0+). To set
the
> > > > consensus
> > > > >>> > value to what you have set in your config file, you
would set
> > > > something
> > > > >>> > like this:
> > > > >>> >
> > > > >>> > TC_PAIRS_MET_CONFIG_OVERRIDES = consensus = [ { name =
"CONS";
> > > > members
> > > > >>> =
> > > > >>> > ["HWRF","GFSO","HMON"]; required = [false,false,false];
> min_req =
> > > 2;
> > > > }
> > > > >>> ];
> > > > >>> >
> > > > >>> > I will make a note in the GitHub issue for improvements
to the
> > > > TCPairs
> > > > >>> > wrapper (https://github.com/dtcenter/METplus/issues/749)
to
> add
> > > > >>> support
> > > > >>> > for
> > > > >>> > consensus explicitly via something like
TC_PAIRS_CONSENSUS so
> > that
> > > it
> > > > >>> is
> > > > >>> > easier to configure via METplus wrappers.
> > > > >>> >
> > > > >>> > 4.2:
> > > > >>> > The System Configuration section of the METplus User's
Guide
> > covers
> > > > >>> these
> > > > >>> > variables and how they affect execution. I suggest that
you
> read
> > > > >>> through
> > > > >>> > this section to get a full picture of how these
variables work.
> > > > >>> Regarding
> > > > >>> > your specific question, these 2 sections should help:
> > > > >>> >
> > > > >>> > LOOP_BY:
> > > > >>> >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#loop-
by
> > > > >>> > LOOP_ORDER:
> > > > >>> >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#loop-
order
> > > > >>> >
> > > > >>> > Thanks,
> > > > >>> > George
> > > > >>> >
> > > > >>> > On Fri, May 14, 2021 at 8:19 AM Yan Jin - NOAA Affiliate
via
> RT <
> > > > >>> > met_help at ucar.edu> wrote:
> > > > >>> >
> > > > >>> > >
> > > > >>> > > <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846
> > >
> > > > >>> > >
> > > > >>> > > Hi George,
> > > > >>> > >
> > > > >>> > > Thanks so much for your help. Finally the code can run
> without
> > > > >>> errors.
> > > > >>> > Here
> > > > >>> > > is the steps and results:
> > > > >>> > >
> > > > >>> > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
> https://docs.google.com/document/d/1xRF3aiPATcX5G37ngJj-
65pl44LN7mGHo34Sf8i3zpo/edit
> > > > >>> > >
> > > > >>> > > I also put questions in the document, I also pasted
here:
> > > > >>> > >
> > > > >>> > > Questions:
> > > > >>> > >
> > > > >>> > > 4.1: How can I do the consensus forecasts in METplus
config
> > file
> > > as
> > > > >>> I did
> > > > >>> > > in tc_pairs?
> > > > >>> > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
(/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_292020/TCPairsConfig_run1)
> > > > >>> > >
> > > > >>> > > //
> > > > >>> > >
> > > > >>> > > // Specify how consensus forecasts should be defined.
> > > > >>> > >
> > > > >>> > > //
> > > > >>> > >
> > > > >>> > > consensus = [
> > > > >>> > >
> > > > >>> > >  {
> > > > >>> > >
> > > > >>> > >   name = "CONS";
> > > > >>> > >
> > > > >>> > >   members = ["HWRF","GFSO","HMON"];
> > > > >>> > >
> > > > >>> > >   required = [false,false,false];
> > > > >>> > >
> > > > >>> > >   min_req = 2;
> > > > >>> > >
> > > > >>> > >  }
> > > > >>> > >
> > > > >>> > > ];
> > > > >>> > >
> > > > >>> > > 4.2: I am a little confused about
> > > > >>> > >
> > > > >>> > >       ***
> > > > >>> > >
> > > > >>> > >        LOOP_ORDER = processes
> > > > >>> > >
> > > > >>> > >        LOOP_BY = INIT
> > > > >>> > >
> > > > >>> > >       ***
> > > > >>> > >
> > > > >>> > > Could you explain these two command lines and give me
some
> > > > examples?
> > > > >>> > >
> > > > >>> > > Thanks very much for your time and help on my
questions.
> > > > >>> > >
> > > > >>> > > Best,
> > > > >>> > > Yan
> > > > >>> > >
> > > > >>> > > On Thu, May 13, 2021 at 3:04 PM George McCabe via RT <
> > > > >>> met_help at ucar.edu>
> > > > >>> > > wrote:
> > > > >>> > >
> > > > >>> > > > Hi Yan,
> > > > >>> > > >
> > > > >>> > > > The produtil is due to a bug in the wrapper in the
logic
> that
> > > > >>> reports
> > > > >>> > > when
> > > > >>> > > > an error occurs. I ran the command on Jet by hand
and it
> > looks
> > > > >>> like it
> > > > >>> > is
> > > > >>> > > > still not finding any input data to plot. You can
run the
> > > > following
> > > > >>> > > > commands (after loading all of the required
modules):
> > > > >>> > > >
> > > > >>> > > > export MET_INSTALL_DIR=/contrib/met/9.1
> > > > >>> > > >
> > > > >>> > > > Rscript
/contrib/met/9.1/share/met/Rscripts/plot_tcmpr.R \
> > > > >>> > > > -config
> > > > >>> > > >
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
> /contrib/met/METplus/METplus-
3.1.1/parm/use_cases/model_applications/tc_and_extra_tc/TCMPRPlotterConfig_customize
> > > > >>> > > > \
> > > > >>> > > > -subtitle "Your subtitle goes here" -ylab "Your y-
label
> goes
> > > > here"
> > > > >>> \
> > > > >>> > > > -dep AMAX_WIND-BMAX_WIND,TK_ERR -legend "Your legend
text
> > goes
> > > > >>> > here..." \
> > > > >>> > > > -plot MEAN,MEDIAN,BOXPLOT \
> > > > >>> > > > -lookin
> > > > >>> > > >
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> > > > >>> > > > \
> > > > >>> > > >
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
> > > > >>> > > > \
> > > > >>> > > >
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
> > > > >>> > > > \
> > > > >>> > > > -outdir
> > > > >>> > > >
> > > > >>>
> > >
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tcmpr_plots
> > > > >>> > > >
> > > > >>> > > >
> > > > >>> > > > Here is the output from that run:
> > > > >>> > > >
> > > > >>> > > > Reading plot configuration file:
> > > > >>> > > >
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
> /contrib/met/METplus/METplus-
3.1.1/parm/use_cases/model_applications/tc_and_extra_tc/TCMPRPlotterConfig_customize
> > > > >>> > > > No plotting configuration file specified.
> > > > >>> > > > CALLING:  /contrib/met/9.1/bin/tc_stat -lookin
> > > > >>> > > >
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> > > > >>> > > > -lookin
> > > > >>> > > >
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
> > > > >>> > > > -lookin
> > > > >>> > > >
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
> > > > >>> > > > -job filter -dump_row /tmp/plot_tcmpr_116966.tcst
> > -event_equal
> > > > >>> true
> > > > >>> > -v 3
> > > > >>> > > > DEBUG 1: [Source 1] Lookin:
> > > > >>> > > >
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> > > > >>> > > > DEBUG 1: [Source 2] Lookin:
> > > > >>> > > >
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
> > > > >>> > > > DEBUG 1: [Source 3] Lookin:
> > > > >>> > > >
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
> > > > >>> > > > DEBUG 2:
> > > > >>> > > > DEBUG 2: Processing Job 1: -job filter -event_equal
true
> > > > -dump_row
> > > > >>> > > > /tmp/plot_tcmpr_116966.tcst
> > > > >>> > > > DEBUG 3: Applying track-based event equalization
logic.
> > > > >>> > > > DEBUG 3: Reading file 1 of 3:
> > > > >>> > > >
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> > > > >>> > > > DEBUG 3: Reading file 2 of 3:
> > > > >>> > > >
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
> > > > >>> > > > DEBUG 3: Reading file 3 of 3:
> > > > >>> > > >
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
> > > > >>> > > > DEBUG 3: For track-based event equalization,
identified 0
> > > common
> > > > >>> cases
> > > > >>> > > for
> > > > >>> > > > 146 models: AC00 AEMI AEMN AP01 AP02 AP03 AP04 AP05
AP06
> AP07
> > > > AP08
> > > > >>> AP09
> > > > >>> > > > AP0I AP10 AP11 AP12 AP13 AP14 AP15 AP16 AP17 AP18
AP19 AP1I
> > > AP20
> > > > >>> AP21
> > > > >>> > > AP22
> > > > >>> > > > AP23 AP24 AP25 AP26 AP27 AP28 AP29 AP30 API CARQ
CEM2 CEMI
> > CEMN
> > > > >>> CLP5
> > > > >>> > CMC
> > > > >>> > > > CMC2 CMCI COTC COTI CTCX DRCL DSHP EC00 EEMN EGRR
EMN2 EMN3
> > > EMN4
> > > > >>> EMNI
> > > > >>> > EMX
> > > > >>> > > > EMX2 EMXI EN01 EN02 EN03 EN04 EN05 EN06 EN07 EN08
EN09 EN0I
> > > EN10
> > > > >>> EN11
> > > > >>> > > EN12
> > > > >>> > > > EN13 EN14 EN15 EN16 EN17 EN18 EN19 EN1I EN20 EN21
EN22 EN23
> > > EN24
> > > > >>> EN25
> > > > >>> > ENI
> > > > >>> > > > EP01 EP02 EP03 EP04 EP05 EP06 EP07 EP08 EP09 EP0I
EP10 EP11
> > > EP12
> > > > >>> EP13
> > > > >>> > > EP14
> > > > >>> > > > EP15 EP16 EP17 EP18 EP19 EP1I EP20 EP21 EP22 EP23
EP24 EP25
> > EPI
> > > > >>> GFEX
> > > > >>> > GFSI
> > > > >>> > > > GFSO HMNI HMON HWF2 HWFI HWRF ICON IVCN LGEM NGX
NGX2 NGXI
> > NVGM
> > > > >>> OCD5
> > > > >>> > RVCN
> > > > >>> > > > SHF5 SHIP TABD TABM TABS TCLP TVCA TVCE TVCN TVCX
UKX UKX2
> > UKXI
> > > > >>> XTRP
> > > > >>> > > > DEBUG 1: Event equalization of tracks found no
common
> cases.
> > > > >>> > > > DEBUG 1: Creating output dump file:
> > /tmp/plot_tcmpr_116966.tcst
> > > > >>> > > > FILTER: -job filter -line_type TCMPR -event_equal
true
> > > -dump_row
> > > > >>> > > > /tmp/plot_tcmpr_116966.tcst
> > > > >>> > > >
> > > > >>> > > > DEBUG 2: Job 1 used 0 out of 0 lines read.
> > > > >>> > > > DEBUG 3: Total lines read                 = 0
> > > > >>> > > > DEBUG 3: Total lines kept                 = 0
> > > > >>> > > > DEBUG 3: Rejected for track watch/warn    = 0
> > > > >>> > > > DEBUG 3: Rejected for init threshold      = 0
> > > > >>> > > > DEBUG 3: Rejected for init string         = 0
> > > > >>> > > > DEBUG 3: Rejected for out init mask       = 0
> > > > >>> > > > DEBUG 3: Rejected for water only          = 0
> > > > >>> > > > DEBUG 3: Rejected for rapid inten         = 0
> > > > >>> > > > DEBUG 3: Rejected for landfall            = 0
> > > > >>> > > > DEBUG 3: Rejected for amodel              = 0
> > > > >>> > > > DEBUG 3: Rejected for bmodel              = 0
> > > > >>> > > > DEBUG 3: Rejected for desc                = 0
> > > > >>> > > > DEBUG 3: Rejected for storm id            = 0
> > > > >>> > > > DEBUG 3: Rejected for basin               = 0
> > > > >>> > > > DEBUG 3: Rejected for cyclone             = 0
> > > > >>> > > > DEBUG 3: Rejected for storm name          = 0
> > > > >>> > > > DEBUG 3: Rejected for init time           = 0
> > > > >>> > > > DEBUG 3: Rejected for init hour           = 0
> > > > >>> > > > DEBUG 3: Rejected for lead time           = 0
> > > > >>> > > > DEBUG 3: Rejected for required lead times = 0
> > > > >>> > > > DEBUG 3: Rejected for valid time          = 0
> > > > >>> > > > DEBUG 3: Rejected for valid hour          = 0
> > > > >>> > > > DEBUG 3: Rejected for init mask           = 0
> > > > >>> > > > DEBUG 3: Rejected for valid mask          = 0
> > > > >>> > > > DEBUG 3: Rejected for line type           = 0
> > > > >>> > > > DEBUG 3: Rejected for numeric threshold   = 0
> > > > >>> > > > DEBUG 3: Rejected for string matching     = 0
> > > > >>> > > > DEBUG 3: Rejected for match points        = 0
> > > > >>> > > > DEBUG 3: Rejected for event equal         = 0
> > > > >>> > > > DEBUG 3: Rejected for out init mask       = 0
> > > > >>> > > > DEBUG 3: Rejected for out valid mask      = 0
> > > > >>> > > > Reading track data: /tmp/plot_tcmpr_116966.tcst
> > > > >>> > > > Error in read.table(tcst_tmp_file, header = TRUE) :
> > > > >>> > > >   no lines available in input
> > > > >>> > > > Execution halted
> > > > >>> > > >
> > > > >>> > > >
> > > > >>> > > >
> > > > >>> > > > It sounds like filter arguments may need to be
provided to
> > > > produce
> > > > >>> > > TC-Stat
> > > > >>> > > > output (which is called by the R script). You can
set this
> in
> > > > your
> > > > >>> > > METplus
> > > > >>> > > > config file with the TCMPR_PLOTTER_FILTER variable,
but it
> > will
> > > > be
> > > > >>> > > > difficult to debug since the log output is not
shown. I
> would
> > > > >>> recommend
> > > > >>> > > > modifying the Rscript call from this email and add
the
> > -filter
> > > > >>> argument
> > > > >>> > > > with any filter information to pass to TCStat. I am
not
> sure
> > > > >>> settings
> > > > >>> > > would
> > > > >>> > > > help you produce output, but I have reached out to
other
> > > METplus
> > > > >>> team
> > > > >>> > > > members for suggestions.
> > > > >>> > > >
> > > > >>> > > > Thanks,
> > > > >>> > > > George
> > > > >>> > > >
> > > > >>> > > >
> > > > >>> > > >
> > > > >>> > > > On Thu, May 13, 2021 at 12:37 PM Yan Jin - NOAA
Affiliate
> via
> > > RT
> > > > <
> > > > >>> > > > met_help at ucar.edu> wrote:
> > > > >>> > > >
> > > > >>> > > > >
> > > > >>> > > > > <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846
> > > > >
> > > > >>> > > > >
> > > > >>> > > > > Hi George,
> > > > >>> > > > >
> > > > >>> > > > > Thanks so much. Now tc_pairs works, and the output
file
> can
> > > be
> > > > >>> > > generated.
> > > > >>> > > > >
> > > > >>> > > > >
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> > > > >>> > > > >
> > > > >>> > > > > Then I set "PROCESS_LIST = TCPairs, TCMPRPlotter",
and I
> > kept
> > > > >>> most
> > > > >>> > > > > parameters that initialized with "TCMPR_PLOTTER"
the same
> > as
> > > > the
> > > > >>> > > default
> > > > >>> > > > > configuration.
> > > > >>> > > > >
> > > > >>> > > > > my METplus config file:
> > > > >>> > > > >
> > > > >>> > > > >
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test.conf
> > > > >>> > > > >
> > > > >>> > > > > then run script
> > > > >>> > > > >
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/user_config/run.sh
> > > > >>> > > > >
> > > > >>> > > > > However, the Rscript cannot plot figures.
> > > > >>> > > > > I checked the log file:
> > > > >>> > > > >
> > > > >>> > > > >
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210513182819
> > > > >>> > > > > [image: image.png]
> > > > >>> > > > >
> > > > >>> > > > > It seems that the error "NameError: name
'produtil' is
> not
> > > > >>> defined"?
> > > > >>> > > > >
> > > > >>> > > > > Could you give me some suggestions?
> > > > >>> > > > >
> > > > >>> > > > > Many thanks.
> > > > >>> > > > > Best,
> > > > >>> > > > > Yan
> > > > >>> > > > >
> > > > >>> > > > > On Wed, May 12, 2021 at 5:20 PM George McCabe via
RT <
> > > > >>> > > met_help at ucar.edu>
> > > > >>> > > > > wrote:
> > > > >>> > > > >
> > > > >>> > > > > > Hi Yan,
> > > > >>> > > > > >
> > > > >>> > > > > > It looks like the values set for init_beg and
init_end
> in
> > > the
> > > > >>> > TCpairs
> > > > >>> > > > > > config file are both "20200101_000000" because
the
> > INIT_BEG
> > > > and
> > > > >>> > > > INIT_END
> > > > >>> > > > > > time is 2020. There are no adeck tracks for this
time,
> so
> > > all
> > > > >>> of
> > > > >>> > the
> > > > >>> > > > data
> > > > >>> > > > > > is being filtered out.
> > > > >>> > > > > >
> > > > >>> > > > > > This is a little confusing in METplus 3.1
because
> > INIT_BEG
> > > > and
> > > > >>> > > INIT_END
> > > > >>> > > > > > control the time looping of execution (if
applicable
> for
> > > the
> > > > >>> use
> > > > >>> > > case)
> > > > >>> > > > > AND
> > > > >>> > > > > > the values passed to the wrapped TCPairs config
file.
> In
> > > > >>> METplus
> > > > >>> > > 4.0.0
> > > > >>> > > > > > (which was released yesterday and should be
available
> on
> > > jet
> > > > >>> > soon!),
> > > > >>> > > > you
> > > > >>> > > > > > can set INIT_BEG and INIT_END to control time
looping
> (if
> > > > >>> desired)
> > > > >>> > > and
> > > > >>> > > > > use
> > > > >>> > > > > > TC_PAIRS_INIT_BEG and TC_PAIRS_INIT_END to
specify the
> > > values
> > > > >>> to
> > > > >>> > pass
> > > > >>> > > > to
> > > > >>> > > > > > the TCPairs config file.
> > > > >>> > > > > >
> > > > >>> > > > > > If you were using METplus 4.0.0, I would
recommend
> > setting
> > > > >>> > something
> > > > >>> > > > > like:
> > > > >>> > > > > >
> > > > >>> > > > > >
> > > > >>> > > > > > LOOP_BY = INIT
> > > > >>> > > > > > INIT_TIME_FMT = %Y
> > > > >>> > > > > > INIT_BEG = 2020
> > > > >>> > > > > > INIT_END = 2020
> > > > >>> > > > > > INIT_INCREMENT = 21600
> > > > >>> > > > > >
> > > > >>> > > > > > TC_PAIRS_INIT_BEG = 20201030_000000
> > > > >>> > > > > > TC_PAIRS_INIT_END = 20201031_000000
> > > > >>> > > > > >
> > > > >>> > > > > >
> > > > >>> > > > > > This would run once using 2020 to find filenames
and
> > > process
> > > > >>> data
> > > > >>> > > from
> > > > >>> > > > > the
> > > > >>> > > > > > files that are initialized between
20201030_000000 and
> > > > >>> > > 20201031_000000.
> > > > >>> > > > > >
> > > > >>> > > > > > If using METplus 3.1, you will have to use the
INIT_BEG
> > and
> > > > >>> > INIT_END
> > > > >>> > > > > values
> > > > >>> > > > > > to set the values in the wrapped TCPairs config.
This
> is
> > OK
> > > > for
> > > > >>> > this
> > > > >>> > > > use
> > > > >>> > > > > > case because you have LOOP_ORDER set to
processes and
> > > TCPairs
> > > > >>> only
> > > > >>> > > runs
> > > > >>> > > > > > once with that setting.
> > > > >>> > > > > >
> > > > >>> > > > > > LOOP_BY = INIT
> > > > >>> > > > > > INIT_TIME_FMT = %Y%m%d
> > > > >>> > > > > > INIT_BEG = 20201030
> > > > >>> > > > > > INIT_END = 20201031
> > > > >>> > > > > >
> > > > >>> > > > > > This configuration should run and only use
tracks
> > > initialized
> > > > >>> > between
> > > > >>> > > > > > 2020-10-30 @ 0Z and 2020-10-31 @ 0Z. If you need
to
> > specify
> > > > the
> > > > >>> > init
> > > > >>> > > > time
> > > > >>> > > > > > range with hour precision, you can do that as
well.
> Here
> > is
> > > > an
> > > > >>> > > example:
> > > > >>> > > > > >
> > > > >>> > > > > > LOOP_BY = INIT
> > > > >>> > > > > > INIT_TIME_FMT = %Y%m%d%H
> > > > >>> > > > > > INIT_BEG = 2020103012
> > > > >>> > > > > > INIT_END = 2020103112
> > > > >>> > > > > >
> > > > >>> > > > > > This configuration will only use tracks
initialized
> > between
> > > > >>> > > 2020-10-30
> > > > >>> > > > @
> > > > >>> > > > > > 12Z and 2020-10-31 @ 12Z.
> > > > >>> > > > > >
> > > > >>> > > > > > You can also restrict the data used by valid
time as
> well
> > > by
> > > > >>> using
> > > > >>> > > > > > TC_PAIRS_VALID_BEG and TC_PAIRS_VALID_END in
METplus
> > 4.0.0
> > > or
> > > > >>> > > VALID_BEG
> > > > >>> > > > > and
> > > > >>> > > > > > VALID_END for 3.1.
> > > > >>> > > > > >
> > > > >>> > > > > > Let me know if you are still unable to produce
output
> > with
> > > > >>> these
> > > > >>> > > > > changes. I
> > > > >>> > > > > > was able to run tc_pairs by hand on Jet and
produce a
> > file
> > > > with
> > > > >>> > > output
> > > > >>> > > > > > tracks. You can view it here if you'd like:
> > > > >>> > > > > >
> > > > >>> > > > > >
> /lfs1/HFIP/dtc-hurr/George.Mccabe/al292020_pairs.dat.tcst
> > > > >>> > > > > >
> > > > >>> > > > > > These are the commands I used:
> > > > >>> > > > > >
> > > > >>> > > > > > export BASIN="[]";
> > > > >>> > > > > > export CYCLONE="[]";
> > > > >>> > > > > > export
DLAND_FILE="/contrib/met/9.1/share/met/tc_data/
> > > > >>> > > > > > dland_global_tenth_degree.nc";
> > > > >>> > > > > > export INIT_BEG="20201030_000000";
> > > > >>> > > > > > export INIT_END="20201031_000000";
> > > > >>> > > > > > export INIT_EXCLUDE="[]";
> > > > >>> > > > > > export INIT_INCLUDE="[]";
> > > > >>> > > > > > export MODEL="[]";
> > > > >>> > > > > > export STORM_ID="[\"AL292020\"]";
> > > > >>> > > > > > export STORM_NAME="[]";
> > > > >>> > > > > > export VALID_BEG="";
> > > > >>> > > > > > export VALID_END="";
> > > > >>> > > > > > /contrib/met/9.1/bin/tc_pairs -v 4 -bdeck
> > > > >>> > > > > >
> > > /mnt/lfs4/HFIP/hwrf-data/hwrf-input/abdeck/btk/bal292020.dat
> > > > >>> -adeck
> > > > >>> > > > > >
> > > /mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/data/aal292020.dat
> > > > >>> > -config
> > > > >>> > > > > >
> > > > >>> > > > > >
> > > > >>> > > > >
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/use_cases/model_applications/tc_and_extra_tc/TCPairsETCConfig
> > > > >>> > > > > > -out
> /lfs1/HFIP/dtc-hurr/George.Mccabe/al292020_pairs.dat
> > > > >>> > > > > >
> > > > >>> > > > > > I took the values from your METplus log file and
> modified
> > > > them
> > > > >>> to
> > > > >>> > > write
> > > > >>> > > > > to
> > > > >>> > > > > > my directory and use a different init time
range.
> > > > >>> > > > > >
> > > > >>> > > > > > Thanks,
> > > > >>> > > > > > George
> > > > >>> > > > > >
> > > > >>> > > > > >
> > > > >>> > > > > >
> > > > >>> > > > > >
> > > > >>> > > > > > On Wed, May 12, 2021 at 2:04 PM Yan Jin - NOAA
> Affiliate
> > > via
> > > > >>> RT <
> > > > >>> > > > > > met_help at ucar.edu> wrote:
> > > > >>> > > > > >
> > > > >>> > > > > > >
> > > > >>> > > > > > > <URL:
> > > > >>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846 >
> > > > >>> > > > > > >
> > > > >>> > > > > > > Hi George,
> > > > >>> > > > > > >
> > > > >>> > > > > > > Thanks very much for your instructions. I set
> > > > "PROCESS_LIST =
> > > > >>> > > > TCPairs",
> > > > >>> > > > > > and
> > > > >>> > > > > > > didn't specify models: "MODEL =". Then run the
> script.
> > > > >>> However,
> > > > >>> > the
> > > > >>> > > > > > output
> > > > >>> > > > > > > file is still empty. It seems that tc_pairs
matched 0
> > > adeck
> > > > >>> track
> > > > >>> > > to
> > > > >>> > > > 1
> > > > >>> > > > > > > bdeck track. I also tried to set "MODEL =
HWRF,
> HMON",
> > > but
> > > > >>> the
> > > > >>> > > output
> > > > >>> > > > > > file
> > > > >>> > > > > > > is empty too.
> > > > >>> > > > > > >
> > > > >>> > > > > > > Here is the log file:
> > > > >>> > > > > > >
> > > > >>> > > > > > >
> > > > >>> > > > > >
> > > > >>> > > > >
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210512193152
> > > > >>> > > > > > >
> > > > >>> > > > > > > And here is the output file:
> > > > >>> > > > > > >
> > > > >>> > > > > > >
> > > > >>> > > > > >
> > > > >>> > > > >
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
> > > > >>> > > > > > >
> > > > >>> > > > > > > Many thanks.
> > > > >>> > > > > > > Best,
> > > > >>> > > > > > > Yan
> > > > >>> > > > > > >
> > > > >>> > > > > > > On Wed, May 12, 2021 at 11:36 AM George McCabe
via
> RT <
> > > > >>> > > > > met_help at ucar.edu
> > > > >>> > > > > > >
> > > > >>> > > > > > > wrote:
> > > > >>> > > > > > >
> > > > >>> > > > > > > > Hi Yan,
> > > > >>> > > > > > > >
> > > > >>> > > > > > > > I was able to log into Jet this morning and
look at
> > > your
> > > > >>> error.
> > > > >>> > > > > > > >
> > > > >>> > > > > > > > It looks like the error in the call to
Rscript is
> due
> > > to
> > > > >>> there
> > > > >>> > > > being
> > > > >>> > > > > no
> > > > >>> > > > > > > > data in the input files generated by
TCPairs. There
> > are
> > > > >>> > > > > inconsistencies
> > > > >>> > > > > > > in
> > > > >>> > > > > > > > the TCMPRPlotter wrapper that prevent the
log
> output
> > > from
> > > > >>> the
> > > > >>> > run
> > > > >>> > > > to
> > > > >>> > > > > be
> > > > >>> > > > > > > > seen, which would have made it more clear
why the
> > issue
> > > > was
> > > > >>> > > > > occurring.
> > > > >>> > > > > > > >
> > > > >>> > > > > > > > I would recommend changing the PROCESS_LIST
value
> in
> > > your
> > > > >>> > METplus
> > > > >>> > > > > > > > configuration file to only run TCPairs and
> iterating
> > on
> > > > >>> that
> > > > >>> > > until
> > > > >>> > > > > you
> > > > >>> > > > > > > get
> > > > >>> > > > > > > > output from that tool. Once you can confirm
that
> you
> > > have
> > > > >>> > output,
> > > > >>> > > > you
> > > > >>> > > > > > can
> > > > >>> > > > > > > > add TCMPRPlotter back to the PROCESS_LIST
and test
> on
> > > > that
> > > > >>> > > portion.
> > > > >>> > > > > > > >
> > > > >>> > > > > > > > I will create a GitHub issue for updating
the
> > > > TCMPRPlotter
> > > > >>> > > wrapper
> > > > >>> > > > to
> > > > >>> > > > > > > > produce useful logging output that will make
things
> > > more
> > > > >>> clear
> > > > >>> > > > going
> > > > >>> > > > > > > > forward.
> > > > >>> > > > > > > >
> > > > >>> > > > > > > > Let me know if you run into any other issues
or
> need
> > > help
> > > > >>> > > > configuring
> > > > >>> > > > > > > > TCPairs to run properly. I'm also happy to
attend
> > > another
> > > > >>> > working
> > > > >>> > > > > > session
> > > > >>> > > > > > > > if that is helpful.
> > > > >>> > > > > > > >
> > > > >>> > > > > > > > Thanks,
> > > > >>> > > > > > > > George
> > > > >>> > > > > > > >
> > > > >>> > > > > > > >
> > > > >>> > > > > > > > On Tue May 11 10:05:06 2021,
yan.jin at noaa.gov
> wrote:
> > > > >>> > > > > > > > > Hi John,
> > > > >>> > > > > > > > >
> > > > >>> > > > > > > > >
> > > > >>> > > > > > > > > The directories and data I mentioned in my
last
> > email
> > > > >>> are all
> > > > >>> > > > > located
> > > > >>> > > > > > > > > in
> > > > >>> > > > > > > > > Jet.
> > > > >>> > > > > > > > >
> > > > >>> > > > > > > > >
> > > > >>> > > > > > > > > Here is the email I got regarding the Jet
> > > maintenance:
> > > > >>> > > > > > > > >
> > > > >>> > > > > > > > > ***
> > > > >>> > > > > > > > >
> > > > >>> > > > > > > > > *05/11/21 - Boulder Bastion Maintenance
Downtime*
> > > > >>> > > > > > > > > Boulder Bastion maintenance is scheduled
for
> > Tuesday
> > > > >>> 05/11/21
> > > > >>> > > > from
> > > > >>> > > > > > > > > 0600 ~
> > > > >>> > > > > > > > > 1000 MT. Boulder Bastion login sessions
will be
> > > dropped
> > > > >>> and
> > > > >>> > all
> > > > >>> > > > new
> > > > >>> > > > > > > > > RDHPCS
> > > > >>> > > > > > > > > logins will need to go through the
Princeton
> > Bastion.
> > > > >>> > Logging
> > > > >>> > > In
> > > > >>> > > > > > > > > <
> > > > >>> > > > > >
> > > > >>> >
> > > https://rdhpcs-common-
docs.rdhpcs.noaa.gov/wiki/index.php/Logging_in
> > > > >>> > > >
> > > > >>> > > > > > > > >
> > > > >>> > > > > > > > >
> > > > >>> > > > > > > > >
> > > > >>> > > > > > > > > *The next Jet Maintenance Downtime is TBD*
> > > > >>> > > > > > > > >
> > > > >>> > > > > > > > > ***
> > > > >>> > > > > > > > >
> > > > >>> > > > > > > > >
> > > > >>> > > > > > > > > I tried to login Princeton but the
connection is
> > > closed
> > > > >>> too.
> > > > >>> > > > > > > > >
> > > > >>> > > > > > > > >
> > > > >>> > > > > > > > > Really appreciate your time and help on
this.
> > > > >>> > > > > > > > >
> > > > >>> > > > > > > > >
> > > > >>> > > > > > > > > Best,
> > > > >>> > > > > > > > >
> > > > >>> > > > > > > > > Yan
> > > > >>> > > > > > > > >
> > > > >>> > > > > > > > > On Tue, May 11, 2021 at 11:57 AM John
Halley
> Gotway
> > > via
> > > > >>> RT <
> > > > >>> > > > > > > > > met_help at ucar.edu> wrote:
> > > > >>> > > > > > > > >
> > > > >>> > > > > > > > > > Yan,
> > > > >>> > > > > > > > > >
> > > > >>> > > > > > > > > > George and I tried to take a look at
this, but
> > > > couldn't
> > > > >>> > find
> > > > >>> > > > the
> > > > >>> > > > > > > > > > data.
> > > > >>> > > > > > > > > >
> > > > >>> > > > > > > > > >
> > > > >>> > > > > > > >
> > > > >>> > > > > > >
> > > > >>> > > > > >
> > > > >>> > > > >
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210510214154
> > > > >>> > > > > > > > > >
> > > > >>> > > > > > > > > > I checked on Hera and WCOSS but couldn't
find
> the
> > > > data
> > > > >>> > > there. I
> > > > >>> > > > > > > > > > should be
> > > > >>> > > > > > > > > > able to log on to jet, but when I try,
it boots
> > me
> > > > off:
> > > > >>> > > > > > > > > >
> > > > >>> > > > > > > > > > Authentication failed.
> > > > >>> > > > > > > > > > Connection to jet-
rsa.boulder.rdhpcs.noaa.gov
> > > > closed.
> > > > >>> > > > > > > > > >
> > > > >>> > > > > > > > > > Can you confirm that you're running on
jet? And
> > any
> > > > >>> idea of
> > > > >>> > > jet
> > > > >>> > > > > is
> > > > >>> > > > > > > > > > down
> > > > >>> > > > > > > > > > today or something?
> > > > >>> > > > > > > > > >
> > > > >>> > > > > > > > > > Thanks,
> > > > >>> > > > > > > > > > John
> > > > >>> > > > > > > > > >
> > > > >>> > > > > > > > > >
> > > > >>> > > > > > > > > > On Tue, May 11, 2021 at 7:46 AM Yan Jin
- NOAA
> > > > >>> Affiliate
> > > > >>> > via
> > > > >>> > > > RT <
> > > > >>> > > > > > > > > > met_help at ucar.edu> wrote:
> > > > >>> > > > > > > > > >
> > > > >>> > > > > > > > > > >
> > > > >>> > > > > > > > > > > <URL:
> > > > >>> > > >
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846
> > > > >>> > > > > >
> > > > >>> > > > > > > > > > >
> > > > >>> > > > > > > > > > > Hi John,
> > > > >>> > > > > > > > > > >
> > > > >>> > > > > > > > > > > Thanks so much!
> > > > >>> > > > > > > > > > >
> > > > >>> > > > > > > > > > > Best,
> > > > >>> > > > > > > > > > > Yan
> > > > >>> > > > > > > > > > >
> > > > >>> > > > > > > > > > >
> > > > >>> > > > > > > > > > > On Tue, May 11, 2021 at 9:44 AM John
Halley
> > > Gotway
> > > > >>> via
> > > > >>> > RT <
> > > > >>> > > > > > > > > > > met_help at ucar.edu>
> > > > >>> > > > > > > > > > > wrote:
> > > > >>> > > > > > > > > > >
> > > > >>> > > > > > > > > > > > Hi Yan,
> > > > >>> > > > > > > > > > > >
> > > > >>> > > > > > > > > > > > I'm going to assign this ticket to
George
> and
> > > ask
> > > > >>> him
> > > > >>> > to
> > > > >>> > > > > take a
> > > > >>> > > > > > > > > > > > look
> > > > >>> > > > > > > > > > and
> > > > >>> > > > > > > > > > > > debug this runtime error.
> > > > >>> > > > > > > > > > > >
> > > > >>> > > > > > > > > > > > Thanks,
> > > > >>> > > > > > > > > > > > John HG
> > > > >>> > > > > > > > > > > >
> > > > >>> > > > > > > > > > > > On Mon, May 10, 2021 at 3:49 PM Yan
Jin -
> > NOAA
> > > > >>> > Affiliate
> > > > >>> > > > via
> > > > >>> > > > > > RT <
> > > > >>> > > > > > > > > > > > met_help at ucar.edu> wrote:
> > > > >>> > > > > > > > > > > >
> > > > >>> > > > > > > > > > > > >
> > > > >>> > > > > > > > > > > > > Mon May 10 15:49:51 2021: Request
99846
> was
> > > > acted
> > > > >>> > upon.
> > > > >>> > > > > > > > > > > > > Transaction: Ticket created by
> > > > yan.jin at noaa.gov
> > > > >>> > > > > > > > > > > > >        Queue: met_help
> > > > >>> > > > > > > > > > > > >      Subject: ask a question about
> METplus
> > > > >>> config of
> > > > >>> > TC
> > > > >>> > > > > > > > > > > > >        Owner: Nobody
> > > > >>> > > > > > > > > > > > >   Requestors: yan.jin at noaa.gov
> > > > >>> > > > > > > > > > > > >       Status: new
> > > > >>> > > > > > > > > > > > >  Ticket <URL:
> > > > >>> > > > > > > > > >
> > > > >>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846
> > > > >>> > > > > > > > > > > >
> > > > >>> > > > > > > > > > > > >
> > > > >>> > > > > > > > > > > > >
> > > > >>> > > > > > > > > > > > > Hi,
> > > > >>> > > > > > > > > > > > >
> > > > >>> > > > > > > > > > > > > This is Yan, again. I ran the
METplus
> > config
> > > > file
> > > > >>> > after
> > > > >>> > > > our
> > > > >>> > > > > > > > > > discussion,
> > > > >>> > > > > > > > > > > > > however, it seems that the
tc_pairs tcst
> > file
> > > > >>> wasn't
> > > > >>> > > > > > generated.
> > > > >>> > > > > > > > > > > > >
> > > > >>> > > > > > > > > > > > > Here is the script in my directory
on
> Jet:
> > > > >>> > > > > > > > > > > > > I set TC_PAIRS_STORM_ID =
AL292020, so
> the
> > > code
> > > > >>> can
> > > > >>> > > find
> > > > >>> > > > > the
> > > > >>> > > > > > > > > > > > > input
> > > > >>> > > > > > > > > > > adeck
> > > > >>> > > > > > > > > > > > > and bdeck data correctly
> > > > >>> > > > > > > > > > > > > *----
> > > > >>> > > > > > > > > > > > >
> > > > >>> > > > > > > > > > > > >
> > > > >>> > > > > > > > > > > >
> > > > >>> > > > > > > > > > >
> > > > >>> > > > > > > > > >
> > > > >>> > > > > > > >
> > > > >>> > > > > > >
> > > > >>> > > > > >
> > > > >>> > > > >
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test.conf
> > > > >>> > > > > > > > > > > > >
> > > > >>> > > > > > > > > > > > > The output directory is
> > > > >>> > > > > > > > > > > > >
> > > > >>> > > > > > > >
> > > > >>> > >
> > >
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs
> > > > >>> > > > > > > > > > > > > and the tc_pairs data:
> > > al292020_pairs.dat.tcst
> > > > >>> > > > > > > > > > > > >
> > > > >>> > > > > > > > > > > > > log file
> > > > >>> > > > > > > > > > > > >
> > > > >>> > > > > > > > > > > > >
> > > > >>> > > > > > > > > > > >
> > > > >>> > > > > > > > > > >
> > > > >>> > > > > > > > > >
> > > > >>> > > > > > > >
> > > > >>> > > > > > >
> > > > >>> > > > > >
> > > > >>> > > > >
> > > > >>> > > >
> > > > >>> > >
> > > > >>> >
> > > > >>>
> > > >
> > >
> >
>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210510214154
> > > > >>> > > > > > > > > > > > > *-----
> > > > >>> > > > > > > > > > > > >
> > > > >>> > > > > > > > > > > > > The error starts:[image:
image.png]
> > > > >>> > > > > > > > > > > > >
> > > > >>> > > > > > > > > > > > > Then I checked
al292020_pairs.dat.tcst,
> and
> > > it
> > > > is
> > > > >>> > > empty.
> > > > >>> > > > > > > > > > > > >
> > > > >>> > > > > > > > > > > > > I wonder if there is anything I
missed?
> > > > >>> > > > > > > > > > > > >
> > > > >>> > > > > > > > > > > > > Many thanks.
> > > > >>> > > > > > > > > > > > > Best,
> > > > >>> > > > > > > > > > > > > Yan
> > > > >>> > > > > > > > > > > > >
> > > > >>> > > > > > > > > > > > >
> > > > >>> > > > > > > > > > > >
> > > > >>> > > > > > > > > > > >
> > > > >>> > > > > > > > > > >
> > > > >>> > > > > > > > > > >
> > > > >>> > > > > > > > > >
> > > > >>> > > > > > > > > >
> > > > >>> > > > > > > >
> > > > >>> > > > > > > >
> > > > >>> > > > > > > >
> > > > >>> > > > > > > >
> > > > >>> > > > > > >
> > > > >>> > > > > > >
> > > > >>> > > > > >
> > > > >>> > > > > > --
> > > > >>> > > > > > George McCabe - Software Engineer III
> > > > >>> > > > > > National Center for Atmospheric Research
> > > > >>> > > > > > Research Applications Laboratory
> > > > >>> > > > > > 303-497-2768
> > > > >>> > > > > > ---
> > > > >>> > > > > > My working day may not be your working day.
Please do
> not
> > > > feel
> > > > >>> > > obliged
> > > > >>> > > > to
> > > > >>> > > > > > reply to this email outside of your normal
working
> hours.
> > > > >>> > > > > >
> > > > >>> > > > > >
> > > > >>> > > > >
> > > > >>> > > > >
> > > > >>> > > >
> > > > >>> > > > --
> > > > >>> > > > George McCabe - Software Engineer III
> > > > >>> > > > National Center for Atmospheric Research
> > > > >>> > > > Research Applications Laboratory
> > > > >>> > > > 303-497-2768
> > > > >>> > > > ---
> > > > >>> > > > My working day may not be your working day. Please
do not
> > feel
> > > > >>> obliged
> > > > >>> > to
> > > > >>> > > > reply to this email outside of your normal working
hours.
> > > > >>> > > >
> > > > >>> > > >
> > > > >>> > >
> > > > >>> > >
> > > > >>> >
> > > > >>> > --
> > > > >>> > George McCabe - Software Engineer III
> > > > >>> > National Center for Atmospheric Research
> > > > >>> > Research Applications Laboratory
> > > > >>> > 303-497-2768
> > > > >>> > ---
> > > > >>> > My working day may not be your working day. Please do
not feel
> > > > obliged
> > > > >>> to
> > > > >>> > reply to this email outside of your normal working
hours.
> > > > >>> >
> > > > >>> >
> > > > >>>
> > > > >>>
> > > > >>
> > > > >> --
> > > > >> George McCabe - Software Engineer III
> > > > >> National Center for Atmospheric Research
> > > > >> Research Applications Laboratory
> > > > >> 303-497-2768
> > > > >> ---
> > > > >> My working day may not be your working day. Please do not
feel
> > obliged
> > > > to
> > > > >> reply to this email outside of your normal working hours.
> > > > >>
> > > > >
> > > > >
> > > > > --
> > > > > George McCabe - Software Engineer III
> > > > > National Center for Atmospheric Research
> > > > > Research Applications Laboratory
> > > > > 303-497-2768
> > > > > ---
> > > > > My working day may not be your working day. Please do not
feel
> > obliged
> > > to
> > > > > reply to this email outside of your normal working hours.
> > > > >
> > > >
> > > >
> > > > --
> > > > George McCabe - Software Engineer III
> > > > National Center for Atmospheric Research
> > > > Research Applications Laboratory
> > > > 303-497-2768
> > > > ---
> > > > My working day may not be your working day. Please do not feel
> obliged
> > to
> > > > reply to this email outside of your normal working hours.
> > > >
> > > >
> > >
> > >
> >
> > --
> > George McCabe - Software Engineer III
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > 303-497-2768
> > ---
> > My working day may not be your working day. Please do not feel
obliged to
> > reply to this email outside of your normal working hours.
> >
> >
>
>

--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.

------------------------------------------------
Subject: ask a question about METplus config of TC
From: George McCabe
Time: Thu May 27 08:30:30 2021

Hi Yan,

I am making modifications to the TCMPRPlotter wrapper and was hoping
to get
some feedback from you to make sure that the changes allow you to
configure
the tool the way you like.

I am going to add logic to look over the TCMPR_PLOTTER_DEP_VARS and
TCMPR_PLOTTER_PLOT_TYPES values to call the R script once for each
combination. This will allow you to set the filenames and plot titles
for
each run without having to create multiple instances of the wrapper.

TCMPR_PLOTTER_DEP_VARS = ABS(AMAX_WIND-BMAX_WIND), TK_ERR
TCMPR_PLOTTER_PLOT_TYPES = MEAN, MEDIAN, BOXPLOT

This would run the R script 6 times. To allow you to specify the title
for
each, I was going to add a corresponding _LABELS variable for dep and
plot
that can be referenced in the title. Instead of setting:

TCMPR_PLOTTER_TITLE = Absolute Intensity Error Mean\n Eta (Atlantic
2020)

You would be able to set:

TCMPR_PLOTTER_DEP_LABELS = Absolute Intensity Error, Track Error
TCMPR_PLOTTER_PLOT_LABELS = Mean, Median, Boxplot
TCMPR_PLOTTER_TITLE = {dep} {plot}\n Eta (Atlantic {date?fmt=%Y})

and the appropriate strings (including the year) would be used in the
title
for each run.

Question: Would it be helpful to be able to specify the formatted
label AND
the text for each dep/plot? I could set it up so that you can
reference
{dep}, {dep_label}, {plot}, and {plot_label} so you can reference each
in
the plot title and the filename, i.e.:

TCMPR_PLOTTER_PREFIX = Eta_{dep}_{plot}
TCMPR_PLOTTER_TITLE = {dep_label} {plot_label}\n Eta (Atlantic
{date?fmt=%Y})

which would result in files named like
"Eta_ABS_AMAX_WIND-BMAX_WIND_MEAN.png" that have a title " Absolute
Intensity Error Mean\n Eta (Atlantic 2020)" that changes for each
plot/dep.

Another option is to use the label values for {dep} and {plot}, but
replace
the spaces with underscores in the filenames, i.e.

TCMPR_PLOTTER_PREFIX = Eta_{dep}_{plot}

Would create files named "Eta_Absolute_Intensity_Error_Mean.png."

I'd like to implement these changes with your end goal in mind so that
you
can configure it to get the results you like. Let me know what you
think or
if you have any other suggestions to improve the logic.

Thanks,
George

On Thu, May 27, 2021 at 8:13 AM George McCabe <mccabe at ucar.edu> wrote:

> Hi Yan,
>
> That is great to hear! I looked at your output files and they do
look
> correct. It looks like I made the fix to handling the cyclone list
before
> the 4.0.0 release. I did make other improvements to the TCPairs
wrapper
> including adding support to set consensus through the METplus
config. I put
> the code on Jet so you can test that it works for your case. To use
this
> version, run the module commands to set up METplus, then instead of
running
> run_metplus.py from the path, run the script from my directory,
passing in
> the same config files that you were previously using.
>
>
/home/George.Mccabe/George.Mccabe/METplus.feature_749/ush/run_metplus.py
> ...
>
> To set the consensus values from your previous email:
>
> consensus = [
>  {
>   name = "CONS";
>   members = ["HWRF","GFSO","HMON"];
>   required = [false,false,false];
>   min_req = 2;
>  }
>
> You can set the following in your METplus config file:
>
> TC_PAIRS_CONSENSUS1_NAME = CONS
> TC_PAIRS_CONSENSUS1_MEMBERS = HWRF, GFSO, HMON
> TC_PAIRS_CONSENSUS1_REQUIRED = false, false, false
> TC_PAIRS_CONSENSUS1_MIN_REQ = 2
>
> If you need more than 1 consensus dictionary defined, you can add it
by
> using TC_PAIRS_CONSENSUS2_*.
>
> Let me know how it goes!
>
> Thanks,
> George
>
> On Wed, May 26, 2021 at 2:57 PM Yan Jin - NOAA Affiliate via RT <
> met_help at ucar.edu> wrote:
>
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846 >
>>
>> Hi George,
>>
>> Thanks so much. I am not in a hurry. It is totally fine to wait
until it
>> is
>> ready after you are done with your current work.
>>
>> I checked the files in
>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/vrfy_METplus4.0/AL_MJJA_2020/tc_pairs,
and
>> each file contain the result for that cyclone number.
>> e.g.: al012020_pairs.dat.tcst has the initial time
20200515~20200520.
>>
>> Many thanks.
>> Best,
>> Yan
>>
>> On Wed, May 26, 2021 at 3:38 PM George McCabe via RT
<met_help at ucar.edu>
>> wrote:
>>
>> > Hi Yan,
>> >
>> > The way the wrapper is currently written, specifying that list of
>> cyclones
>> > in TC_PAIRS_CYCLONE will result in 15 output files that all
contain all
>> 15
>> > cyclones. I am currently working on updating the tool to handle
this
>> > properly so that each file will only contain the cyclone in
question.
>> Your
>> > configuration looks correct to me, but it will not produce the
correct
>> > behavior until these updates are finished. When it is ready, I
can make
>> it
>> > available to you on Jet so you can test that it runs as expected.
>> >
>> > I am actually finished with this fix but I am now working on
>> implementing
>> > support for setting consensus through METplus. If you'd like, I
can put
>> the
>> > code from my development branch on Jet so you can test that this
works
>> > while I am still working on consensus. Let me know if you'd like
me to
>> do
>> > this and I can make it available to you.
>> >
>> > Thanks,
>> > George
>> >
>> > - George
>> >
>> > On Wed, May 26, 2021 at 12:54 PM Yan Jin - NOAA Affiliate via RT
<
>> > met_help at ucar.edu> wrote:
>> >
>> > >
>> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846 >
>> > >
>> > > Hi George,
>> > >
>> > > Can I ask a question regarding the seasonal mean TC
verification? For
>> > > instance, I have adeck and bdeck data over Atlantic at 2020,
(Jet:
>> > > /mnt/lfs4/HFIP/hwrf-data/hwrf-input/abdeck), each has 01-99
cyclone
>> > number.
>> > > And I would like to calculate the absolute intensity error and
track
>> > error
>> > > of seasonal mean over Atlantic. Here is my method, could you
check if
>> it
>> > is
>> > > correct to do that?
>> > >
>> > > I plan to do the May-June-July-August mean error.
>> > > $WORK:
>> > >
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/vrfy_METplus4.0/METplus/parm/user_config
>> > > in this directory, the METplus config file is
>> TCMPRPlotter_AL_MJJA_2020
>> > >
>> > >
>> > > The major modification I did is:
>> > > ***
>> > > MODEL = HWRF,HMON,GFSO
>> > >
>> > > TC_PAIRS_DESC =
>> > >
>> > > TC_PAIRS_STORM_ID =
>> > > TC_PAIRS_BASIN = AL
>> > > TC_PAIRS_CYCLONE = 01,02,03,04,05,06,07,08,09,10,11,12,13,14,15
>> > > TC_PAIRS_STORM_NAME =
>> > >
>> > > TC_PAIRS_INIT_BEG = 20200515_000000
>> > > TC_PAIRS_INIT_END = 20200906_000000
>> > > ***
>> > > I checked the adeck and bdeck data, and found that cyclone
number
>> 01-15
>> > > have the initial times which are located within MJJA. So I
>> > > specified TC_PAIRS_CYCLONE with 01 to 15. I also checked the
initial
>> time
>> > > in cyclone 01 and 15, and found the INIT_BEG and INIT_END.
>> > >
>> > > It generated 15 tc_pairs.tcst files, and also 6 figures (Output
>> > directory:
>> > > /mnt/lfs4/HFIP/hwrfv3/Yan.Jin/vrfy_METplus4.0/AL_MJJA_2020)
>> > >
>> > > Here is the result
>> > >
>> > >
>> >
>> https://docs.google.com/document/d/10adrHdzNNxMJM51e5Sw9oy6uS2n1S-
qGS2c7NZdVVxc/edit?pli=1
>> > >
>> > > I wonder if my method is correct? It seems that
TC_PAIRS_INIT_BEG
>> > > and TC_PAIRS_INIT_END are mandatory for my case.
>> > >
>> > > Many thanks.
>> > > Best,
>> > > Yan
>> > >
>> > > On Thu, May 20, 2021 at 12:01 PM George McCabe via RT <
>> met_help at ucar.edu
>> > >
>> > > wrote:
>> > >
>> > > > Hi Yan,
>> > > >
>> > > > I was able to change the line colors in the images. The
images can
>> be
>> > > found
>> > > > on Jet here:
>> > > >
>> > > > /lfs1/HFIP/dtc-hurr/George.Mccabe/eta-out-colors/tcmpr_plots
>> > > >
>> > > >
>> > > > This is the command I used to run:
>> > > >
>> > > > run_metplus.py
>> > > > /lfs1/HFIP/dtc-
hurr/George.Mccabe/TCMPRPlotter_Eta_colors.conf
>> > > > config.OUTPUT_BASE=/lfs1/HFIP/dtc-hurr/George.Mccabe/eta-out-
colors
>> > > >
>> > > >
>> > > > I copied the default tcmpr file from
>> > > >
>> > >
>> >
>>
/contrib/met/10.0.0/share/met/Rscripts/include/plot_tcmpr_config_default.R
>> > > > to my directory and renamed it to plot_tcmpr_overrides.R
>> > > >
>> > > > Then I modified the METplus config to point to that file
instead of
>> the
>> > > > file in the METplus repository:
>> > > >
>> > > > TCMPR_PLOTTER_CONFIG_FILE =
>> > > > /lfs1/HFIP/dtc-hurr/George.Mccabe/plot_tcmpr_overrides.R
>> > > >
>> > > >
>> > > > The previous tcmpr parameter file changes the image
resolution, so I
>> > had
>> > > to
>> > > > set that in the new file to prevent the image from being very
large.
>> > > Then I
>> > > > changed the value of the default color list:
>> > > >
>> > > > img_res = 72;
>> > > > default_color_list  = c("blue", "green", "purple", "orange");
>> > > >
>> > > > This variable is used to set other variables for each plot
type, so
>> you
>> > > can
>> > > > modify those variables if you desire different colors for
different
>> > > plots.
>> > > > You only need to supply the variables you want to change from
the
>> > default
>> > > > value, so your overrides file can just include the above 2
>> variables to
>> > > > keep the file short and clean. However, it is nice to have
all of
>> the
>> > > > possible options easily available
>> > > > in case you want to change other settings. How you handle
this is
>> up to
>> > > > your preference.
>> > > >
>> > > > Let me know if you have any other questions.
>> > > >
>> > > > Thanks,
>> > > > George
>> > > >
>> > > > On Thu, May 20, 2021 at 9:19 AM George McCabe
<mccabe at ucar.edu>
>> wrote:
>> > > >
>> > > > > Hi Yan,
>> > > > >
>> > > > > I copied your config file and made some modifications. You
can
>> view
>> > it
>> > > on
>> > > > > Jet here:
>> > > > >
>> > > > > /lfs1/HFIP/dtc-hurr/George.Mccabe/TCMPRPlotter_Eta.conf
>> > > > >
>> > > > > Starting in METplus 4.0.0 the different section headers
like dir
>> and
>> > > > > filename_templates are no longer necessary. All
configurations
>> can go
>> > > > under
>> > > > > the [config] header. I removed the sections and rearranged
the
>> > > variables
>> > > > to
>> > > > > group them in a more clear way.
>> > > > >
>> > > > > To call TCMPRPlotter multiple times with different
settings, I
>> added
>> > > > > multiple instances of the app in the PROCESS_LIST with
instance
>> names
>> > > > > defined:
>> > > > >
>> > > > > PROCESS_LIST = TCPairs, TCMPRPlotter(abs_mean),
>> > > > > TCMPRPlotter(abs_median),TCMPRPlotter(abs_boxplot),
>> > > > TCMPRPlotter(tk_mean),
>> > > > > TCMPRPlotter(tk_median),TCMPRPlotter(tk_boxplot)
>> > > > >
>> > > > > Each instance will read all of the values set in the
[config]
>> > section,
>> > > > > then it will look for a section header that matches the
string
>> inside
>> > > the
>> > > > > parentheses for each instance. If the section is found, it
>> overrides
>> > > the
>> > > > > values set in the section to replace the values set in the
>> [config]
>> > > > > section. These names can be whatever you chose, but the
name in
>> the
>> > > > > PROCESS_LIST must match the name of the section header.
>> > > > >
>> > > > > [abs_mean]
>> > > > > TCMPR_PLOTTER_PREFIX = Eta_Absolute_Intensity_mean
>> > > > > TCMPR_PLOTTER_TITLE = Absolute Intensity Error Mean\n Eta
>> (Atlantic
>> > > 2020)
>> > > > > TCMPR_PLOTTER_DEP_VARS = ABS(AMAX_WIND-BMAX_WIND)
>> > > > > TCMPR_PLOTTER_PLOT_TYPES = MEAN
>> > > > >
>> > > > > [abs_median]
>> > > > > TCMPR_PLOTTER_PREFIX = Eta_Absolute_Intensity_median
>> > > > > TCMPR_PLOTTER_TITLE = Absolute Intensity Error Median\n Eta
>> (Atlantic
>> > > > 2020)
>> > > > > TCMPR_PLOTTER_DEP_VARS = ABS(AMAX_WIND-BMAX_WIND)
>> > > > > TCMPR_PLOTTER_PLOT_TYPES = MEDIAN
>> > > > >
>> > > > > [abs_boxplot]
>> > > > > TCMPR_PLOTTER_PREFIX = Eta_Absolute_Intensity_boxplot
>> > > > > TCMPR_PLOTTER_TITLE = Absolute Intensity Error Boxplot\n
Eta
>> > (Atlantic
>> > > > > 2020)
>> > > > > TCMPR_PLOTTER_DEP_VARS = ABS(AMAX_WIND-BMAX_WIND)
>> > > > > TCMPR_PLOTTER_PLOT_TYPES = BOXPLOT
>> > > > >
>> > > > > [tk_mean]
>> > > > > TCMPR_PLOTTER_PREFIX = Eta_Track_Error_mean
>> > > > > TCMPR_PLOTTER_TITLE = Track Error Mean\n Eta (Atlantic
2020)
>> > > > > TCMPR_PLOTTER_DEP_VARS = TK_ERR
>> > > > > TCMPR_PLOTTER_PLOT_TYPES = MEAN
>> > > > >
>> > > > > [tk_median]
>> > > > > TCMPR_PLOTTER_PREFIX = Eta_Track_Error_median
>> > > > > TCMPR_PLOTTER_TITLE = Track Error Median\n Eta (Atlantic
2020)
>> > > > > TCMPR_PLOTTER_DEP_VARS = TK_ERR
>> > > > > TCMPR_PLOTTER_PLOT_TYPES = MEDIAN
>> > > > >
>> > > > > [tk_boxplot]
>> > > > > TCMPR_PLOTTER_PREFIX = Eta_Track_Error_boxplot
>> > > > > TCMPR_PLOTTER_TITLE = Track Error Boxplot\n Eta (Atlantic
2020)
>> > > > > TCMPR_PLOTTER_DEP_VARS = TK_ERR
>> > > > > TCMPR_PLOTTER_PLOT_TYPES = BOXPLOT
>> > > > >
>> > > > >
>> > > > > I split up the DEP_VARS and PLOT_TYPES values into
instances with
>> one
>> > > > > value of each and set the plot title and filenames
appropriately
>> for
>> > > > each.
>> > > > > I set the X label to "Lead Time (h)" in the [config]
section so it
>> > > > applies
>> > > > > to every instance.
>> > > > >
>> > > > > TCMPR_PLOTTER_XLAB = Lead Time (h)
>> > > > >
>> > > > > The output images from my run is on Jet here:
>> > > > >
>> > > > > /lfs1/HFIP/dtc-hurr/George.Mccabe/eta-out/tcmpr_plots
>> > > > >
>> > > > > Eta_Absolute_Intensity_boxplot.log
>> > > > > Eta_Absolute_Intensity_boxplot.png
>> > > > > Eta_Absolute_Intensity_mean.png
>> > > > > Eta_Absolute_Intensity_median.png
>> > > > > Eta_Track_Error_boxplot.log
>> > > > > Eta_Track_Error_boxplot.png
>> > > > > Eta_Track_Error_mean.png
>> > > > > Eta_Track_Error_median.png
>> > > > >
>> > > > >
>> > > > > Here is the log file from the run:
>> > > > >
>> > >
>> /lfs1/HFIP/dtc-hurr/George.Mccabe/eta-
out/logs/metplus.log.20210520150804
>> > > > >
>> > > > >
>> > > > > Here is what I ran to produce the output:
>> > > > >
>> > > > >
>> > > > >    - module load intel
>> > > > >    - module load intelpython/3.6.5
>> > > > >    - module load netcdf/4.6.1
>> > > > >    - module load hdf5/1.10.4
>> > > > >    - module load nco/4.9.1
>> > > > >    - module load wgrib/1.8.1.0b
>> > > > >    - module load wgrib2/2.0.8
>> > > > >    - module load R/4.0.2
>> > > > >    - module use /contrib/met/modulefiles
>> > > > >    - module load met/10.0.0
>> > > > >    - module use /contrib/met/METplus/modulefiles
>> > > > >    - module load metplus/4.0.0
>> > > > >    - run_metplus.py
>> > > > >    /lfs1/HFIP/dtc-hurr/George.Mccabe/TCMPRPlotter_Eta.conf
>> > > > >    config.OUTPUT_BASE=/lfs1/HFIP/dtc-
hurr/George.Mccabe/eta-out
>> > > > >
>> > > > >
>> > > > >
>> > > > > I will look into what settings are needed to change the
line
>> colors.
>> > > > Also,
>> > > > > I have started making the modifications to the TCPairs
wrapper to
>> > > > properly
>> > > > > handle multiple storms specified in the config file. I will
let
>> you
>> > > know
>> > > > > when that is ready to test.
>> > > > >
>> > > > > Thanks,
>> > > > > George
>> > > > >
>> > > > >
>> > > > >
>> > > > > On Wed, May 19, 2021 at 5:09 PM George McCabe
<mccabe at ucar.edu>
>> > wrote:
>> > > > >
>> > > > >> Hi Yan,
>> > > > >>
>> > > > >> 1) add TC name to each plot name
>> > > > >> 2) modify titles to each plot, e.g.: TK_ERR* plots with
name
>> "Track
>> > > > >> Error", and ABS_AMAX_WIND-BMAX_WIND_* plots with name
"Absolute
>> > > > Intensity
>> > > > >> Error"?
>> > > > >>
>> > > > >> It looks like the logic in the R script uses the -prefix
value to
>> > > > replace
>> > > > >> the filename instead of prepending it to the plot type
text as
>> you
>> > > would
>> > > > >> guess it would. I was told that this script was typically
called
>> > > > multiple
>> > > > >> times to generate the multiple plots, which would provide
more
>> > control
>> > > > over
>> > > > >> setting the output filenames, titles, etc. The bad news is
it
>> looks
>> > > like
>> > > > >> the wrapper was written to call the R script once with all
of the
>> > > > >> configurations specified in the METplus config file. The
good
>> news
>> > is
>> > > > there
>> > > > >> are some new capabilities added to METplus 4.0.0 that
could help
>> > > produce
>> > > > >> the results you need.
>> > > > >>
>> > > > >> Please review this section of the METplus User's Guide,
>> specifically
>> > > the
>> > > > >> section about specifying instance names to the processes:
>> > > > >>
>> > > > >>
>> > > >
>> > >
>> >
>>
https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#process-
list
>> > > > >>
>> > > > >> Alternatively, you can use the UserScript wrapper to build
custom
>> > > > >> commands to call the R script directly with any arguments
you
>> need
>> > if
>> > > > the
>> > > > >> wrapper doesn't support an option you need.
>> > > > >> More information on that tool is here:
>> > > > >>
>> > > >
>> > >
>> >
>>
https://metplus.readthedocs.io/en/latest/Users_Guide/wrappers.html#userscript
>> > > > >> and here:
>> > > > >>
>> > > >
>> > >
>> >
>>
https://metplus.readthedocs.io/en/latest/generated/met_tool_wrapper/UserScript/UserScript_run_once.html#sphx-
glr-generated-met-tool-wrapper-userscript-userscript-run-once-py
>> > > > >>
>> > > > >> I think using different instance names to run TCMPRPlotter
>> multiple
>> > > > times
>> > > > >> with different settings is the better approach in this
case. I am
>> > > > wrapping
>> > > > >> up work for the day, but tomorrow I can log into Jet and
help you
>> > > modify
>> > > > >> your config file to do this.
>> > > > >>
>> > > > >>
>> > > > >> 3) Is it possible to modify the line colors in this
METplus
>> config
>> > > file?
>> > > > >>
>> > > > >> Yes, you can change the line colors through the R script's
config
>> > > file.
>> > > > >> You can copy and rename this file:
>> > > > >>
>> > > > >>
>> > > > >>
>> > > >
>> > >
>> >
>>
https://github.com/dtcenter/MET/blob/main_v10.0/met/scripts/Rscripts/include/plot_tcmpr_config_default.R
>> > > > >>
>> > > > >> change the values that you need to change (look for
>> "color_list"),
>> > > then
>> > > > >> set the METplus configuration file to point to your new
file
>> (set in
>> > > > >> TCMPR_PLOTTER_CONFIG_FILE).
>> > > > >>
>> > > > >>
>> > > > >> Let me know if you have any other questions. I will report
back
>> > > tomorrow
>> > > > >> when have some changes for you to test.
>> > > > >>
>> > > > >> Thanks,
>> > > > >> George
>> > > > >>
>> > > > >> On Wed, May 19, 2021 at 4:00 PM Yan Jin - NOAA Affiliate
via RT <
>> > > > >> met_help at ucar.edu> wrote:
>> > > > >>
>> > > > >>>
>> > > > >>> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846 >
>> > > > >>>
>> > > > >>> Hi George,
>> > > > >>>
>> > > > >>> I have a few questions regarding the TCMPR_PLOTTER_*
parameters
>> in
>> > > the
>> > > > >>> METplus config file.
>> > > > >>>
>> > > > >>> Here is the METplus config file I used:
>> > > > >>>
>> > > > >>>
>> > > >
>> > >
>> >
>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/vrfy_METplus4.0/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test.conf
>> > > > >>> **
>> > > > >>> #  Plot_TCMPR options, if left unset, default values that
are
>> > > > >>> #  pre-defined in the R utility (packaged with MET) will
be
>> used.
>> > > > >>> TCMPR_PLOTTER_CONFIG_FILE =
>> > {CONFIG_DIR}/TCMPRPlotterConfig_customize
>> > > > >>> TCMPR_PLOTTER_PREFIX =
>> > > > >>> TCMPR_PLOTTER_TITLE =
>> > > > >>> TCMPR_PLOTTER_SUBTITLE =
>> > > > >>> TCMPR_PLOTTER_XLAB =
>> > > > >>> TCMPR_PLOTTER_YLAB =
>> > > > >>> TCMPR_PLOTTER_XLIM =
>> > > > >>> TCMPR_PLOTTER_YLIM =
>> > > > >>> TCMPR_PLOTTER_FILTER = -amodel HWRF,HMON
>> > > > >>> # the tcst data file to be used instead of running the
MET
>> tc_stat
>> > > > tool.
>> > > > >>> TCMPR_PLOTTER_FILTERED_TCST_DATA_FILE =
>> > > > >>> # Comma separated, no whitespace.  Default is TK_ERR
(track
>> error)
>> > > > unless
>> > > > >>> # otherwise indicated.
>> > > > >>> TCMPR_PLOTTER_DEP_VARS = ABS(AMAX_WIND-BMAX_WIND),TK_ERR
>> > > > >>> TCMPR_PLOTTER_SCATTER_X =
>> > > > >>> TCMPR_PLOTTER_SCATTER_Y =
>> > > > >>> TCMPR_PLOTTER_SKILL_REF =
>> > > > >>> TCMPR_PLOTTER_SERIES = AMODEL
>> > > > >>> TCMPR_PLOTTER_SERIES_CI = TRUE
>> > > > >>> TCMPR_PLOTTER_LEGEND =
>> > > > >>> TCMPR_PLOTTER_LEAD = 0,12,24,36,48,60,72,84,96,108,120
>> > > > >>> # Mean and median plots. These override the plot_tcmpr.R
>> default of
>> > > box
>> > > > >>> plot.
>> > > > >>> # If box plot is desired, this needs to be explicitly
indicated.
>> > > > >>> TCMPR_PLOTTER_PLOT_TYPES = MEAN,MEDIAN,BOXPLOT
>> > > > >>> TCMPR_PLOTTER_RP_DIFF =
>> > > > >>> TCMPR_PLOTTER_DEMO_YR =
>> > > > >>> TCMPR_PLOTTER_HFIP_BASELINE =
>> > > > >>> TCMPR_PLOTTER_FOOTNOTE_FLAG =
>> > > > >>> TCMPR_PLOTTER_PLOT_CONFIG_OPTS =
>> > > > >>> TCMPR_PLOTTER_SAVE_DATA =
>> > > > >>>
>> > > > >>> #  TCMPR FLAGS no == (don't set flag), yes == (set flag)
>> > > > >>> TCMPR_PLOTTER_NO_EE = no
>> > > > >>> TCMPR_PLOTTER_NO_LOG = no
>> > > > >>> TCMPR_PLOTTER_SAVE = no
>> > > > >>> **
>> > > > >>> It generated the following 6 *.png files
>> > > > >>> in
>> > > > >>>
>> > > >
>> >
>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/vrfy_METplus4.0/Eta_use_METplus/tcmpr_plots
>> > > > >>> [image: image.png]
>> > > > >>>
>> > > > >>> Then I would like to modify titles, xlab text, output
file
>> names.
>> > > > >>> Here are the modification I
>> > > > >>> did
>> > > > >>>
>> > > >
>> > >
>> >
>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/vrfy_METplus4.0/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test1.conf
>> > > > >>>
>> > > > >>> **
>> > > > >>> TCMPR_PLOTTER_TITLE = Absolute Intensity Error \n Eta
(Atlantic
>> > 2020)
>> > > > >>> TCMPR_PLOTTER_XLAB = Lead Time (h)
>> > > > >>> **
>> > > > >>> The title of each plot becomes "Absolute Intensity Error
Eta
>> > > (Atlantic
>> > > > >>> 2020)" with double quotes.
>> > > > >>>
>> > > > >>> I also tried to set TCMPR_PLOTTER_PREFIX = Eta_ because I
would
>> > like
>> > > to
>> > > > >>> save each plot as Eta_ABS_AMAX_WIND-BMAX_WIND_*.png, and
>> > > > >>> Eta_TK_ERR_*.png.
>> > > > >>> However, there is only one plot Eta_.png generated.
>> > > > >>>
>> > > > >>> I would like to check how can I set those parameters if I
would
>> > like
>> > > to
>> > > > >>> 1)
>> > > > >>> add TC name to each plot name; 2) modify titles to each
plot,
>> e.g.:
>> > > > >>> TK_ERR*
>> > > > >>> plots with name "Track Error", and ABS_AMAX_WIND-
BMAX_WIND_*
>> plots
>> > > with
>> > > > >>> name "Absolute Intensity Error"? 3) Is it possible to
modify the
>> > line
>> > > > >>> colors in this METplus config file?
>> > > > >>>
>> > > > >>> Could you give me some suggestions?
>> > > > >>>
>> > > > >>> Many thanks.
>> > > > >>> Best,
>> > > > >>> Yan
>> > > > >>>
>> > > > >>> On Mon, May 17, 2021 at 10:37 AM George McCabe via RT <
>> > > > met_help at ucar.edu
>> > > > >>> >
>> > > > >>> wrote:
>> > > > >>>
>> > > > >>> > Hi Yan,
>> > > > >>> >
>> > > > >>> > 4.1:
>> > > > >>> > The consensus variable is not yet controlled by a
METplus
>> > > > configuration
>> > > > >>> > variable explicitly, but you can set the value by using
the
>> > > > >>> > TC_PAIRS_MET_CONFIG_OVERRIDES value (in v4.0.0+). To
set the
>> > > > consensus
>> > > > >>> > value to what you have set in your config file, you
would set
>> > > > something
>> > > > >>> > like this:
>> > > > >>> >
>> > > > >>> > TC_PAIRS_MET_CONFIG_OVERRIDES = consensus = [ { name =
"CONS";
>> > > > members
>> > > > >>> =
>> > > > >>> > ["HWRF","GFSO","HMON"]; required = [false,false,false];
>> min_req =
>> > > 2;
>> > > > }
>> > > > >>> ];
>> > > > >>> >
>> > > > >>> > I will make a note in the GitHub issue for improvements
to the
>> > > > TCPairs
>> > > > >>> > wrapper
(https://github.com/dtcenter/METplus/issues/749) to
>> add
>> > > > >>> support
>> > > > >>> > for
>> > > > >>> > consensus explicitly via something like
TC_PAIRS_CONSENSUS so
>> > that
>> > > it
>> > > > >>> is
>> > > > >>> > easier to configure via METplus wrappers.
>> > > > >>> >
>> > > > >>> > 4.2:
>> > > > >>> > The System Configuration section of the METplus User's
Guide
>> > covers
>> > > > >>> these
>> > > > >>> > variables and how they affect execution. I suggest that
you
>> read
>> > > > >>> through
>> > > > >>> > this section to get a full picture of how these
variables
>> work.
>> > > > >>> Regarding
>> > > > >>> > your specific question, these 2 sections should help:
>> > > > >>> >
>> > > > >>> > LOOP_BY:
>> > > > >>> >
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>>
https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#loop-
by
>> > > > >>> > LOOP_ORDER:
>> > > > >>> >
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>>
https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#loop-
order
>> > > > >>> >
>> > > > >>> > Thanks,
>> > > > >>> > George
>> > > > >>> >
>> > > > >>> > On Fri, May 14, 2021 at 8:19 AM Yan Jin - NOAA
Affiliate via
>> RT <
>> > > > >>> > met_help at ucar.edu> wrote:
>> > > > >>> >
>> > > > >>> > >
>> > > > >>> > > <URL:
>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846
>> > >
>> > > > >>> > >
>> > > > >>> > > Hi George,
>> > > > >>> > >
>> > > > >>> > > Thanks so much for your help. Finally the code can
run
>> without
>> > > > >>> errors.
>> > > > >>> > Here
>> > > > >>> > > is the steps and results:
>> > > > >>> > >
>> > > > >>> > >
>> > > > >>> > >
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>> https://docs.google.com/document/d/1xRF3aiPATcX5G37ngJj-
65pl44LN7mGHo34Sf8i3zpo/edit
>> > > > >>> > >
>> > > > >>> > > I also put questions in the document, I also pasted
here:
>> > > > >>> > >
>> > > > >>> > > Questions:
>> > > > >>> > >
>> > > > >>> > > 4.1: How can I do the consensus forecasts in METplus
config
>> > file
>> > > as
>> > > > >>> I did
>> > > > >>> > > in tc_pairs?
>> > > > >>> > >
>> > > > >>> > >
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>>
(/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_292020/TCPairsConfig_run1)
>> > > > >>> > >
>> > > > >>> > > //
>> > > > >>> > >
>> > > > >>> > > // Specify how consensus forecasts should be defined.
>> > > > >>> > >
>> > > > >>> > > //
>> > > > >>> > >
>> > > > >>> > > consensus = [
>> > > > >>> > >
>> > > > >>> > >  {
>> > > > >>> > >
>> > > > >>> > >   name = "CONS";
>> > > > >>> > >
>> > > > >>> > >   members = ["HWRF","GFSO","HMON"];
>> > > > >>> > >
>> > > > >>> > >   required = [false,false,false];
>> > > > >>> > >
>> > > > >>> > >   min_req = 2;
>> > > > >>> > >
>> > > > >>> > >  }
>> > > > >>> > >
>> > > > >>> > > ];
>> > > > >>> > >
>> > > > >>> > > 4.2: I am a little confused about
>> > > > >>> > >
>> > > > >>> > >       ***
>> > > > >>> > >
>> > > > >>> > >        LOOP_ORDER = processes
>> > > > >>> > >
>> > > > >>> > >        LOOP_BY = INIT
>> > > > >>> > >
>> > > > >>> > >       ***
>> > > > >>> > >
>> > > > >>> > > Could you explain these two command lines and give me
some
>> > > > examples?
>> > > > >>> > >
>> > > > >>> > > Thanks very much for your time and help on my
questions.
>> > > > >>> > >
>> > > > >>> > > Best,
>> > > > >>> > > Yan
>> > > > >>> > >
>> > > > >>> > > On Thu, May 13, 2021 at 3:04 PM George McCabe via RT
<
>> > > > >>> met_help at ucar.edu>
>> > > > >>> > > wrote:
>> > > > >>> > >
>> > > > >>> > > > Hi Yan,
>> > > > >>> > > >
>> > > > >>> > > > The produtil is due to a bug in the wrapper in the
logic
>> that
>> > > > >>> reports
>> > > > >>> > > when
>> > > > >>> > > > an error occurs. I ran the command on Jet by hand
and it
>> > looks
>> > > > >>> like it
>> > > > >>> > is
>> > > > >>> > > > still not finding any input data to plot. You can
run the
>> > > > following
>> > > > >>> > > > commands (after loading all of the required
modules):
>> > > > >>> > > >
>> > > > >>> > > > export MET_INSTALL_DIR=/contrib/met/9.1
>> > > > >>> > > >
>> > > > >>> > > > Rscript
/contrib/met/9.1/share/met/Rscripts/plot_tcmpr.R \
>> > > > >>> > > > -config
>> > > > >>> > > >
>> > > > >>> > > >
>> > > > >>> > >
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>> /contrib/met/METplus/METplus-
3.1.1/parm/use_cases/model_applications/tc_and_extra_tc/TCMPRPlotterConfig_customize
>> > > > >>> > > > \
>> > > > >>> > > > -subtitle "Your subtitle goes here" -ylab "Your y-
label
>> goes
>> > > > here"
>> > > > >>> \
>> > > > >>> > > > -dep AMAX_WIND-BMAX_WIND,TK_ERR -legend "Your
legend text
>> > goes
>> > > > >>> > here..." \
>> > > > >>> > > > -plot MEAN,MEDIAN,BOXPLOT \
>> > > > >>> > > > -lookin
>> > > > >>> > > >
>> > > > >>> > > >
>> > > > >>> > >
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
>> > > > >>> > > > \
>> > > > >>> > > >
>> > > > >>> > > >
>> > > > >>> > >
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
>> > > > >>> > > > \
>> > > > >>> > > >
>> > > > >>> > > >
>> > > > >>> > >
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
>> > > > >>> > > > \
>> > > > >>> > > > -outdir
>> > > > >>> > > >
>> > > > >>>
>> > >
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tcmpr_plots
>> > > > >>> > > >
>> > > > >>> > > >
>> > > > >>> > > > Here is the output from that run:
>> > > > >>> > > >
>> > > > >>> > > > Reading plot configuration file:
>> > > > >>> > > >
>> > > > >>> > > >
>> > > > >>> > >
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>> /contrib/met/METplus/METplus-
3.1.1/parm/use_cases/model_applications/tc_and_extra_tc/TCMPRPlotterConfig_customize
>> > > > >>> > > > No plotting configuration file specified.
>> > > > >>> > > > CALLING:  /contrib/met/9.1/bin/tc_stat -lookin
>> > > > >>> > > >
>> > > > >>> > > >
>> > > > >>> > >
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
>> > > > >>> > > > -lookin
>> > > > >>> > > >
>> > > > >>> > > >
>> > > > >>> > >
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
>> > > > >>> > > > -lookin
>> > > > >>> > > >
>> > > > >>> > > >
>> > > > >>> > >
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
>> > > > >>> > > > -job filter -dump_row /tmp/plot_tcmpr_116966.tcst
>> > -event_equal
>> > > > >>> true
>> > > > >>> > -v 3
>> > > > >>> > > > DEBUG 1: [Source 1] Lookin:
>> > > > >>> > > >
>> > > > >>> > > >
>> > > > >>> > >
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
>> > > > >>> > > > DEBUG 1: [Source 2] Lookin:
>> > > > >>> > > >
>> > > > >>> > > >
>> > > > >>> > >
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
>> > > > >>> > > > DEBUG 1: [Source 3] Lookin:
>> > > > >>> > > >
>> > > > >>> > > >
>> > > > >>> > >
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
>> > > > >>> > > > DEBUG 2:
>> > > > >>> > > > DEBUG 2: Processing Job 1: -job filter -event_equal
true
>> > > > -dump_row
>> > > > >>> > > > /tmp/plot_tcmpr_116966.tcst
>> > > > >>> > > > DEBUG 3: Applying track-based event equalization
logic.
>> > > > >>> > > > DEBUG 3: Reading file 1 of 3:
>> > > > >>> > > >
>> > > > >>> > > >
>> > > > >>> > >
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
>> > > > >>> > > > DEBUG 3: Reading file 2 of 3:
>> > > > >>> > > >
>> > > > >>> > > >
>> > > > >>> > >
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020.tcst
>> > > > >>> > > > DEBUG 3: Reading file 3 of 3:
>> > > > >>> > > >
>> > > > >>> > > >
>> > > > >>> > >
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/tc_pairs.tcst
>> > > > >>> > > > DEBUG 3: For track-based event equalization,
identified 0
>> > > common
>> > > > >>> cases
>> > > > >>> > > for
>> > > > >>> > > > 146 models: AC00 AEMI AEMN AP01 AP02 AP03 AP04 AP05
AP06
>> AP07
>> > > > AP08
>> > > > >>> AP09
>> > > > >>> > > > AP0I AP10 AP11 AP12 AP13 AP14 AP15 AP16 AP17 AP18
AP19
>> AP1I
>> > > AP20
>> > > > >>> AP21
>> > > > >>> > > AP22
>> > > > >>> > > > AP23 AP24 AP25 AP26 AP27 AP28 AP29 AP30 API CARQ
CEM2 CEMI
>> > CEMN
>> > > > >>> CLP5
>> > > > >>> > CMC
>> > > > >>> > > > CMC2 CMCI COTC COTI CTCX DRCL DSHP EC00 EEMN EGRR
EMN2
>> EMN3
>> > > EMN4
>> > > > >>> EMNI
>> > > > >>> > EMX
>> > > > >>> > > > EMX2 EMXI EN01 EN02 EN03 EN04 EN05 EN06 EN07 EN08
EN09
>> EN0I
>> > > EN10
>> > > > >>> EN11
>> > > > >>> > > EN12
>> > > > >>> > > > EN13 EN14 EN15 EN16 EN17 EN18 EN19 EN1I EN20 EN21
EN22
>> EN23
>> > > EN24
>> > > > >>> EN25
>> > > > >>> > ENI
>> > > > >>> > > > EP01 EP02 EP03 EP04 EP05 EP06 EP07 EP08 EP09 EP0I
EP10
>> EP11
>> > > EP12
>> > > > >>> EP13
>> > > > >>> > > EP14
>> > > > >>> > > > EP15 EP16 EP17 EP18 EP19 EP1I EP20 EP21 EP22 EP23
EP24
>> EP25
>> > EPI
>> > > > >>> GFEX
>> > > > >>> > GFSI
>> > > > >>> > > > GFSO HMNI HMON HWF2 HWFI HWRF ICON IVCN LGEM NGX
NGX2 NGXI
>> > NVGM
>> > > > >>> OCD5
>> > > > >>> > RVCN
>> > > > >>> > > > SHF5 SHIP TABD TABM TABS TCLP TVCA TVCE TVCN TVCX
UKX UKX2
>> > UKXI
>> > > > >>> XTRP
>> > > > >>> > > > DEBUG 1: Event equalization of tracks found no
common
>> cases.
>> > > > >>> > > > DEBUG 1: Creating output dump file:
>> > /tmp/plot_tcmpr_116966.tcst
>> > > > >>> > > > FILTER: -job filter -line_type TCMPR -event_equal
true
>> > > -dump_row
>> > > > >>> > > > /tmp/plot_tcmpr_116966.tcst
>> > > > >>> > > >
>> > > > >>> > > > DEBUG 2: Job 1 used 0 out of 0 lines read.
>> > > > >>> > > > DEBUG 3: Total lines read                 = 0
>> > > > >>> > > > DEBUG 3: Total lines kept                 = 0
>> > > > >>> > > > DEBUG 3: Rejected for track watch/warn    = 0
>> > > > >>> > > > DEBUG 3: Rejected for init threshold      = 0
>> > > > >>> > > > DEBUG 3: Rejected for init string         = 0
>> > > > >>> > > > DEBUG 3: Rejected for out init mask       = 0
>> > > > >>> > > > DEBUG 3: Rejected for water only          = 0
>> > > > >>> > > > DEBUG 3: Rejected for rapid inten         = 0
>> > > > >>> > > > DEBUG 3: Rejected for landfall            = 0
>> > > > >>> > > > DEBUG 3: Rejected for amodel              = 0
>> > > > >>> > > > DEBUG 3: Rejected for bmodel              = 0
>> > > > >>> > > > DEBUG 3: Rejected for desc                = 0
>> > > > >>> > > > DEBUG 3: Rejected for storm id            = 0
>> > > > >>> > > > DEBUG 3: Rejected for basin               = 0
>> > > > >>> > > > DEBUG 3: Rejected for cyclone             = 0
>> > > > >>> > > > DEBUG 3: Rejected for storm name          = 0
>> > > > >>> > > > DEBUG 3: Rejected for init time           = 0
>> > > > >>> > > > DEBUG 3: Rejected for init hour           = 0
>> > > > >>> > > > DEBUG 3: Rejected for lead time           = 0
>> > > > >>> > > > DEBUG 3: Rejected for required lead times = 0
>> > > > >>> > > > DEBUG 3: Rejected for valid time          = 0
>> > > > >>> > > > DEBUG 3: Rejected for valid hour          = 0
>> > > > >>> > > > DEBUG 3: Rejected for init mask           = 0
>> > > > >>> > > > DEBUG 3: Rejected for valid mask          = 0
>> > > > >>> > > > DEBUG 3: Rejected for line type           = 0
>> > > > >>> > > > DEBUG 3: Rejected for numeric threshold   = 0
>> > > > >>> > > > DEBUG 3: Rejected for string matching     = 0
>> > > > >>> > > > DEBUG 3: Rejected for match points        = 0
>> > > > >>> > > > DEBUG 3: Rejected for event equal         = 0
>> > > > >>> > > > DEBUG 3: Rejected for out init mask       = 0
>> > > > >>> > > > DEBUG 3: Rejected for out valid mask      = 0
>> > > > >>> > > > Reading track data: /tmp/plot_tcmpr_116966.tcst
>> > > > >>> > > > Error in read.table(tcst_tmp_file, header = TRUE) :
>> > > > >>> > > >   no lines available in input
>> > > > >>> > > > Execution halted
>> > > > >>> > > >
>> > > > >>> > > >
>> > > > >>> > > >
>> > > > >>> > > > It sounds like filter arguments may need to be
provided to
>> > > > produce
>> > > > >>> > > TC-Stat
>> > > > >>> > > > output (which is called by the R script). You can
set
>> this in
>> > > > your
>> > > > >>> > > METplus
>> > > > >>> > > > config file with the TCMPR_PLOTTER_FILTER variable,
but it
>> > will
>> > > > be
>> > > > >>> > > > difficult to debug since the log output is not
shown. I
>> would
>> > > > >>> recommend
>> > > > >>> > > > modifying the Rscript call from this email and add
the
>> > -filter
>> > > > >>> argument
>> > > > >>> > > > with any filter information to pass to TCStat. I am
not
>> sure
>> > > > >>> settings
>> > > > >>> > > would
>> > > > >>> > > > help you produce output, but I have reached out to
other
>> > > METplus
>> > > > >>> team
>> > > > >>> > > > members for suggestions.
>> > > > >>> > > >
>> > > > >>> > > > Thanks,
>> > > > >>> > > > George
>> > > > >>> > > >
>> > > > >>> > > >
>> > > > >>> > > >
>> > > > >>> > > > On Thu, May 13, 2021 at 12:37 PM Yan Jin - NOAA
Affiliate
>> via
>> > > RT
>> > > > <
>> > > > >>> > > > met_help at ucar.edu> wrote:
>> > > > >>> > > >
>> > > > >>> > > > >
>> > > > >>> > > > > <URL:
>> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846
>> > > > >
>> > > > >>> > > > >
>> > > > >>> > > > > Hi George,
>> > > > >>> > > > >
>> > > > >>> > > > > Thanks so much. Now tc_pairs works, and the
output file
>> can
>> > > be
>> > > > >>> > > generated.
>> > > > >>> > > > >
>> > > > >>> > > > >
>> > > > >>> > > >
>> > > > >>> > >
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
>> > > > >>> > > > >
>> > > > >>> > > > > Then I set "PROCESS_LIST = TCPairs,
TCMPRPlotter", and I
>> > kept
>> > > > >>> most
>> > > > >>> > > > > parameters that initialized with "TCMPR_PLOTTER"
the
>> same
>> > as
>> > > > the
>> > > > >>> > > default
>> > > > >>> > > > > configuration.
>> > > > >>> > > > >
>> > > > >>> > > > > my METplus config file:
>> > > > >>> > > > >
>> > > > >>> > > > >
>> > > > >>> > > >
>> > > > >>> > >
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test.conf
>> > > > >>> > > > >
>> > > > >>> > > > > then run script
>> > > > >>> > > > >
>> > > > >>> > > >
>> > > > >>> > >
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/user_config/run.sh
>> > > > >>> > > > >
>> > > > >>> > > > > However, the Rscript cannot plot figures.
>> > > > >>> > > > > I checked the log file:
>> > > > >>> > > > >
>> > > > >>> > > > >
>> > > > >>> > > >
>> > > > >>> > >
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210513182819
>> > > > >>> > > > > [image: image.png]
>> > > > >>> > > > >
>> > > > >>> > > > > It seems that the error "NameError: name
'produtil' is
>> not
>> > > > >>> defined"?
>> > > > >>> > > > >
>> > > > >>> > > > > Could you give me some suggestions?
>> > > > >>> > > > >
>> > > > >>> > > > > Many thanks.
>> > > > >>> > > > > Best,
>> > > > >>> > > > > Yan
>> > > > >>> > > > >
>> > > > >>> > > > > On Wed, May 12, 2021 at 5:20 PM George McCabe via
RT <
>> > > > >>> > > met_help at ucar.edu>
>> > > > >>> > > > > wrote:
>> > > > >>> > > > >
>> > > > >>> > > > > > Hi Yan,
>> > > > >>> > > > > >
>> > > > >>> > > > > > It looks like the values set for init_beg and
>> init_end in
>> > > the
>> > > > >>> > TCpairs
>> > > > >>> > > > > > config file are both "20200101_000000" because
the
>> > INIT_BEG
>> > > > and
>> > > > >>> > > > INIT_END
>> > > > >>> > > > > > time is 2020. There are no adeck tracks for
this
>> time, so
>> > > all
>> > > > >>> of
>> > > > >>> > the
>> > > > >>> > > > data
>> > > > >>> > > > > > is being filtered out.
>> > > > >>> > > > > >
>> > > > >>> > > > > > This is a little confusing in METplus 3.1
because
>> > INIT_BEG
>> > > > and
>> > > > >>> > > INIT_END
>> > > > >>> > > > > > control the time looping of execution (if
applicable
>> for
>> > > the
>> > > > >>> use
>> > > > >>> > > case)
>> > > > >>> > > > > AND
>> > > > >>> > > > > > the values passed to the wrapped TCPairs config
file.
>> In
>> > > > >>> METplus
>> > > > >>> > > 4.0.0
>> > > > >>> > > > > > (which was released yesterday and should be
available
>> on
>> > > jet
>> > > > >>> > soon!),
>> > > > >>> > > > you
>> > > > >>> > > > > > can set INIT_BEG and INIT_END to control time
looping
>> (if
>> > > > >>> desired)
>> > > > >>> > > and
>> > > > >>> > > > > use
>> > > > >>> > > > > > TC_PAIRS_INIT_BEG and TC_PAIRS_INIT_END to
specify the
>> > > values
>> > > > >>> to
>> > > > >>> > pass
>> > > > >>> > > > to
>> > > > >>> > > > > > the TCPairs config file.
>> > > > >>> > > > > >
>> > > > >>> > > > > > If you were using METplus 4.0.0, I would
recommend
>> > setting
>> > > > >>> > something
>> > > > >>> > > > > like:
>> > > > >>> > > > > >
>> > > > >>> > > > > >
>> > > > >>> > > > > > LOOP_BY = INIT
>> > > > >>> > > > > > INIT_TIME_FMT = %Y
>> > > > >>> > > > > > INIT_BEG = 2020
>> > > > >>> > > > > > INIT_END = 2020
>> > > > >>> > > > > > INIT_INCREMENT = 21600
>> > > > >>> > > > > >
>> > > > >>> > > > > > TC_PAIRS_INIT_BEG = 20201030_000000
>> > > > >>> > > > > > TC_PAIRS_INIT_END = 20201031_000000
>> > > > >>> > > > > >
>> > > > >>> > > > > >
>> > > > >>> > > > > > This would run once using 2020 to find
filenames and
>> > > process
>> > > > >>> data
>> > > > >>> > > from
>> > > > >>> > > > > the
>> > > > >>> > > > > > files that are initialized between
20201030_000000 and
>> > > > >>> > > 20201031_000000.
>> > > > >>> > > > > >
>> > > > >>> > > > > > If using METplus 3.1, you will have to use the
>> INIT_BEG
>> > and
>> > > > >>> > INIT_END
>> > > > >>> > > > > values
>> > > > >>> > > > > > to set the values in the wrapped TCPairs
config. This
>> is
>> > OK
>> > > > for
>> > > > >>> > this
>> > > > >>> > > > use
>> > > > >>> > > > > > case because you have LOOP_ORDER set to
processes and
>> > > TCPairs
>> > > > >>> only
>> > > > >>> > > runs
>> > > > >>> > > > > > once with that setting.
>> > > > >>> > > > > >
>> > > > >>> > > > > > LOOP_BY = INIT
>> > > > >>> > > > > > INIT_TIME_FMT = %Y%m%d
>> > > > >>> > > > > > INIT_BEG = 20201030
>> > > > >>> > > > > > INIT_END = 20201031
>> > > > >>> > > > > >
>> > > > >>> > > > > > This configuration should run and only use
tracks
>> > > initialized
>> > > > >>> > between
>> > > > >>> > > > > > 2020-10-30 @ 0Z and 2020-10-31 @ 0Z. If you
need to
>> > specify
>> > > > the
>> > > > >>> > init
>> > > > >>> > > > time
>> > > > >>> > > > > > range with hour precision, you can do that as
well.
>> Here
>> > is
>> > > > an
>> > > > >>> > > example:
>> > > > >>> > > > > >
>> > > > >>> > > > > > LOOP_BY = INIT
>> > > > >>> > > > > > INIT_TIME_FMT = %Y%m%d%H
>> > > > >>> > > > > > INIT_BEG = 2020103012
>> > > > >>> > > > > > INIT_END = 2020103112
>> > > > >>> > > > > >
>> > > > >>> > > > > > This configuration will only use tracks
initialized
>> > between
>> > > > >>> > > 2020-10-30
>> > > > >>> > > > @
>> > > > >>> > > > > > 12Z and 2020-10-31 @ 12Z.
>> > > > >>> > > > > >
>> > > > >>> > > > > > You can also restrict the data used by valid
time as
>> well
>> > > by
>> > > > >>> using
>> > > > >>> > > > > > TC_PAIRS_VALID_BEG and TC_PAIRS_VALID_END in
METplus
>> > 4.0.0
>> > > or
>> > > > >>> > > VALID_BEG
>> > > > >>> > > > > and
>> > > > >>> > > > > > VALID_END for 3.1.
>> > > > >>> > > > > >
>> > > > >>> > > > > > Let me know if you are still unable to produce
output
>> > with
>> > > > >>> these
>> > > > >>> > > > > changes. I
>> > > > >>> > > > > > was able to run tc_pairs by hand on Jet and
produce a
>> > file
>> > > > with
>> > > > >>> > > output
>> > > > >>> > > > > > tracks. You can view it here if you'd like:
>> > > > >>> > > > > >
>> > > > >>> > > > > >
>> /lfs1/HFIP/dtc-hurr/George.Mccabe/al292020_pairs.dat.tcst
>> > > > >>> > > > > >
>> > > > >>> > > > > > These are the commands I used:
>> > > > >>> > > > > >
>> > > > >>> > > > > > export BASIN="[]";
>> > > > >>> > > > > > export CYCLONE="[]";
>> > > > >>> > > > > > export
DLAND_FILE="/contrib/met/9.1/share/met/tc_data/
>> > > > >>> > > > > > dland_global_tenth_degree.nc";
>> > > > >>> > > > > > export INIT_BEG="20201030_000000";
>> > > > >>> > > > > > export INIT_END="20201031_000000";
>> > > > >>> > > > > > export INIT_EXCLUDE="[]";
>> > > > >>> > > > > > export INIT_INCLUDE="[]";
>> > > > >>> > > > > > export MODEL="[]";
>> > > > >>> > > > > > export STORM_ID="[\"AL292020\"]";
>> > > > >>> > > > > > export STORM_NAME="[]";
>> > > > >>> > > > > > export VALID_BEG="";
>> > > > >>> > > > > > export VALID_END="";
>> > > > >>> > > > > > /contrib/met/9.1/bin/tc_pairs -v 4 -bdeck
>> > > > >>> > > > > >
>> > > /mnt/lfs4/HFIP/hwrf-data/hwrf-input/abdeck/btk/bal292020.dat
>> > > > >>> -adeck
>> > > > >>> > > > > >
>> > > /mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/data/aal292020.dat
>> > > > >>> > -config
>> > > > >>> > > > > >
>> > > > >>> > > > > >
>> > > > >>> > > > >
>> > > > >>> > > >
>> > > > >>> > >
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/use_cases/model_applications/tc_and_extra_tc/TCPairsETCConfig
>> > > > >>> > > > > > -out
>> /lfs1/HFIP/dtc-hurr/George.Mccabe/al292020_pairs.dat
>> > > > >>> > > > > >
>> > > > >>> > > > > > I took the values from your METplus log file
and
>> modified
>> > > > them
>> > > > >>> to
>> > > > >>> > > write
>> > > > >>> > > > > to
>> > > > >>> > > > > > my directory and use a different init time
range.
>> > > > >>> > > > > >
>> > > > >>> > > > > > Thanks,
>> > > > >>> > > > > > George
>> > > > >>> > > > > >
>> > > > >>> > > > > >
>> > > > >>> > > > > >
>> > > > >>> > > > > >
>> > > > >>> > > > > > On Wed, May 12, 2021 at 2:04 PM Yan Jin - NOAA
>> Affiliate
>> > > via
>> > > > >>> RT <
>> > > > >>> > > > > > met_help at ucar.edu> wrote:
>> > > > >>> > > > > >
>> > > > >>> > > > > > >
>> > > > >>> > > > > > > <URL:
>> > > > >>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846 >
>> > > > >>> > > > > > >
>> > > > >>> > > > > > > Hi George,
>> > > > >>> > > > > > >
>> > > > >>> > > > > > > Thanks very much for your instructions. I set
>> > > > "PROCESS_LIST =
>> > > > >>> > > > TCPairs",
>> > > > >>> > > > > > and
>> > > > >>> > > > > > > didn't specify models: "MODEL =". Then run
the
>> script.
>> > > > >>> However,
>> > > > >>> > the
>> > > > >>> > > > > > output
>> > > > >>> > > > > > > file is still empty. It seems that tc_pairs
matched
>> 0
>> > > adeck
>> > > > >>> track
>> > > > >>> > > to
>> > > > >>> > > > 1
>> > > > >>> > > > > > > bdeck track. I also tried to set "MODEL =
HWRF,
>> HMON",
>> > > but
>> > > > >>> the
>> > > > >>> > > output
>> > > > >>> > > > > > file
>> > > > >>> > > > > > > is empty too.
>> > > > >>> > > > > > >
>> > > > >>> > > > > > > Here is the log file:
>> > > > >>> > > > > > >
>> > > > >>> > > > > > >
>> > > > >>> > > > > >
>> > > > >>> > > > >
>> > > > >>> > > >
>> > > > >>> > >
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210512193152
>> > > > >>> > > > > > >
>> > > > >>> > > > > > > And here is the output file:
>> > > > >>> > > > > > >
>> > > > >>> > > > > > >
>> > > > >>> > > > > >
>> > > > >>> > > > >
>> > > > >>> > > >
>> > > > >>> > >
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs/al292020_pairs.dat.tcst
>> > > > >>> > > > > > >
>> > > > >>> > > > > > > Many thanks.
>> > > > >>> > > > > > > Best,
>> > > > >>> > > > > > > Yan
>> > > > >>> > > > > > >
>> > > > >>> > > > > > > On Wed, May 12, 2021 at 11:36 AM George
McCabe via
>> RT <
>> > > > >>> > > > > met_help at ucar.edu
>> > > > >>> > > > > > >
>> > > > >>> > > > > > > wrote:
>> > > > >>> > > > > > >
>> > > > >>> > > > > > > > Hi Yan,
>> > > > >>> > > > > > > >
>> > > > >>> > > > > > > > I was able to log into Jet this morning and
look
>> at
>> > > your
>> > > > >>> error.
>> > > > >>> > > > > > > >
>> > > > >>> > > > > > > > It looks like the error in the call to
Rscript is
>> due
>> > > to
>> > > > >>> there
>> > > > >>> > > > being
>> > > > >>> > > > > no
>> > > > >>> > > > > > > > data in the input files generated by
TCPairs.
>> There
>> > are
>> > > > >>> > > > > inconsistencies
>> > > > >>> > > > > > > in
>> > > > >>> > > > > > > > the TCMPRPlotter wrapper that prevent the
log
>> output
>> > > from
>> > > > >>> the
>> > > > >>> > run
>> > > > >>> > > > to
>> > > > >>> > > > > be
>> > > > >>> > > > > > > > seen, which would have made it more clear
why the
>> > issue
>> > > > was
>> > > > >>> > > > > occurring.
>> > > > >>> > > > > > > >
>> > > > >>> > > > > > > > I would recommend changing the PROCESS_LIST
value
>> in
>> > > your
>> > > > >>> > METplus
>> > > > >>> > > > > > > > configuration file to only run TCPairs and
>> iterating
>> > on
>> > > > >>> that
>> > > > >>> > > until
>> > > > >>> > > > > you
>> > > > >>> > > > > > > get
>> > > > >>> > > > > > > > output from that tool. Once you can confirm
that
>> you
>> > > have
>> > > > >>> > output,
>> > > > >>> > > > you
>> > > > >>> > > > > > can
>> > > > >>> > > > > > > > add TCMPRPlotter back to the PROCESS_LIST
and
>> test on
>> > > > that
>> > > > >>> > > portion.
>> > > > >>> > > > > > > >
>> > > > >>> > > > > > > > I will create a GitHub issue for updating
the
>> > > > TCMPRPlotter
>> > > > >>> > > wrapper
>> > > > >>> > > > to
>> > > > >>> > > > > > > > produce useful logging output that will
make
>> things
>> > > more
>> > > > >>> clear
>> > > > >>> > > > going
>> > > > >>> > > > > > > > forward.
>> > > > >>> > > > > > > >
>> > > > >>> > > > > > > > Let me know if you run into any other
issues or
>> need
>> > > help
>> > > > >>> > > > configuring
>> > > > >>> > > > > > > > TCPairs to run properly. I'm also happy to
attend
>> > > another
>> > > > >>> > working
>> > > > >>> > > > > > session
>> > > > >>> > > > > > > > if that is helpful.
>> > > > >>> > > > > > > >
>> > > > >>> > > > > > > > Thanks,
>> > > > >>> > > > > > > > George
>> > > > >>> > > > > > > >
>> > > > >>> > > > > > > >
>> > > > >>> > > > > > > > On Tue May 11 10:05:06 2021,
yan.jin at noaa.gov
>> wrote:
>> > > > >>> > > > > > > > > Hi John,
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > > > > The directories and data I mentioned in
my last
>> > email
>> > > > >>> are all
>> > > > >>> > > > > located
>> > > > >>> > > > > > > > > in
>> > > > >>> > > > > > > > > Jet.
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > > > > Here is the email I got regarding the Jet
>> > > maintenance:
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > > > > ***
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > > > > *05/11/21 - Boulder Bastion Maintenance
>> Downtime*
>> > > > >>> > > > > > > > > Boulder Bastion maintenance is scheduled
for
>> > Tuesday
>> > > > >>> 05/11/21
>> > > > >>> > > > from
>> > > > >>> > > > > > > > > 0600 ~
>> > > > >>> > > > > > > > > 1000 MT. Boulder Bastion login sessions
will be
>> > > dropped
>> > > > >>> and
>> > > > >>> > all
>> > > > >>> > > > new
>> > > > >>> > > > > > > > > RDHPCS
>> > > > >>> > > > > > > > > logins will need to go through the
Princeton
>> > Bastion.
>> > > > >>> > Logging
>> > > > >>> > > In
>> > > > >>> > > > > > > > > <
>> > > > >>> > > > > >
>> > > > >>> >
>> > > https://rdhpcs-common-
docs.rdhpcs.noaa.gov/wiki/index.php/Logging_in
>> > > > >>> > > >
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > > > > *The next Jet Maintenance Downtime is
TBD*
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > > > > ***
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > > > > I tried to login Princeton but the
connection is
>> > > closed
>> > > > >>> too.
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > > > > Really appreciate your time and help on
this.
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > > > > Best,
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > > > > Yan
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > > > > On Tue, May 11, 2021 at 11:57 AM John
Halley
>> Gotway
>> > > via
>> > > > >>> RT <
>> > > > >>> > > > > > > > > met_help at ucar.edu> wrote:
>> > > > >>> > > > > > > > >
>> > > > >>> > > > > > > > > > Yan,
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > > > George and I tried to take a look at
this, but
>> > > > couldn't
>> > > > >>> > find
>> > > > >>> > > > the
>> > > > >>> > > > > > > > > > data.
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > >
>> > > > >>> > > > > > >
>> > > > >>> > > > > >
>> > > > >>> > > > >
>> > > > >>> > > >
>> > > > >>> > >
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/logs/master_metplus.log.20210510214154
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > > > I checked on Hera and WCOSS but
couldn't find
>> the
>> > > > data
>> > > > >>> > > there. I
>> > > > >>> > > > > > > > > > should be
>> > > > >>> > > > > > > > > > able to log on to jet, but when I try,
it
>> boots
>> > me
>> > > > off:
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > > > Authentication failed.
>> > > > >>> > > > > > > > > > Connection to jet-
rsa.boulder.rdhpcs.noaa.gov
>> > > > closed.
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > > > Can you confirm that you're running on
jet?
>> And
>> > any
>> > > > >>> idea of
>> > > > >>> > > jet
>> > > > >>> > > > > is
>> > > > >>> > > > > > > > > > down
>> > > > >>> > > > > > > > > > today or something?
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > > > Thanks,
>> > > > >>> > > > > > > > > > John
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > > > On Tue, May 11, 2021 at 7:46 AM Yan Jin
- NOAA
>> > > > >>> Affiliate
>> > > > >>> > via
>> > > > >>> > > > RT <
>> > > > >>> > > > > > > > > > met_help at ucar.edu> wrote:
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > > > > >
>> > > > >>> > > > > > > > > > > <URL:
>> > > > >>> > > >
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846
>> > > > >>> > > > > >
>> > > > >>> > > > > > > > > > >
>> > > > >>> > > > > > > > > > > Hi John,
>> > > > >>> > > > > > > > > > >
>> > > > >>> > > > > > > > > > > Thanks so much!
>> > > > >>> > > > > > > > > > >
>> > > > >>> > > > > > > > > > > Best,
>> > > > >>> > > > > > > > > > > Yan
>> > > > >>> > > > > > > > > > >
>> > > > >>> > > > > > > > > > >
>> > > > >>> > > > > > > > > > > On Tue, May 11, 2021 at 9:44 AM John
Halley
>> > > Gotway
>> > > > >>> via
>> > > > >>> > RT <
>> > > > >>> > > > > > > > > > > met_help at ucar.edu>
>> > > > >>> > > > > > > > > > > wrote:
>> > > > >>> > > > > > > > > > >
>> > > > >>> > > > > > > > > > > > Hi Yan,
>> > > > >>> > > > > > > > > > > >
>> > > > >>> > > > > > > > > > > > I'm going to assign this ticket to
George
>> and
>> > > ask
>> > > > >>> him
>> > > > >>> > to
>> > > > >>> > > > > take a
>> > > > >>> > > > > > > > > > > > look
>> > > > >>> > > > > > > > > > and
>> > > > >>> > > > > > > > > > > > debug this runtime error.
>> > > > >>> > > > > > > > > > > >
>> > > > >>> > > > > > > > > > > > Thanks,
>> > > > >>> > > > > > > > > > > > John HG
>> > > > >>> > > > > > > > > > > >
>> > > > >>> > > > > > > > > > > > On Mon, May 10, 2021 at 3:49 PM Yan
Jin -
>> > NOAA
>> > > > >>> > Affiliate
>> > > > >>> > > > via
>> > > > >>> > > > > > RT <
>> > > > >>> > > > > > > > > > > > met_help at ucar.edu> wrote:
>> > > > >>> > > > > > > > > > > >
>> > > > >>> > > > > > > > > > > > >
>> > > > >>> > > > > > > > > > > > > Mon May 10 15:49:51 2021: Request
99846
>> was
>> > > > acted
>> > > > >>> > upon.
>> > > > >>> > > > > > > > > > > > > Transaction: Ticket created by
>> > > > yan.jin at noaa.gov
>> > > > >>> > > > > > > > > > > > >        Queue: met_help
>> > > > >>> > > > > > > > > > > > >      Subject: ask a question
about
>> METplus
>> > > > >>> config of
>> > > > >>> > TC
>> > > > >>> > > > > > > > > > > > >        Owner: Nobody
>> > > > >>> > > > > > > > > > > > >   Requestors: yan.jin at noaa.gov
>> > > > >>> > > > > > > > > > > > >       Status: new
>> > > > >>> > > > > > > > > > > > >  Ticket <URL:
>> > > > >>> > > > > > > > > >
>> > > > >>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99846
>> > > > >>> > > > > > > > > > > >
>> > > > >>> > > > > > > > > > > > >
>> > > > >>> > > > > > > > > > > > >
>> > > > >>> > > > > > > > > > > > > Hi,
>> > > > >>> > > > > > > > > > > > >
>> > > > >>> > > > > > > > > > > > > This is Yan, again. I ran the
METplus
>> > config
>> > > > file
>> > > > >>> > after
>> > > > >>> > > > our
>> > > > >>> > > > > > > > > > discussion,
>> > > > >>> > > > > > > > > > > > > however, it seems that the
tc_pairs tcst
>> > file
>> > > > >>> wasn't
>> > > > >>> > > > > > generated.
>> > > > >>> > > > > > > > > > > > >
>> > > > >>> > > > > > > > > > > > > Here is the script in my
directory on
>> Jet:
>> > > > >>> > > > > > > > > > > > > I set TC_PAIRS_STORM_ID =
AL292020, so
>> the
>> > > code
>> > > > >>> can
>> > > > >>> > > find
>> > > > >>> > > > > the
>> > > > >>> > > > > > > > > > > > > input
>> > > > >>> > > > > > > > > > > adeck
>> > > > >>> > > > > > > > > > > > > and bdeck data correctly
>> > > > >>> > > > > > > > > > > > > *----
>> > > > >>> > > > > > > > > > > > >
>> > > > >>> > > > > > > > > > > > >
>> > > > >>> > > > > > > > > > > >
>> > > > >>> > > > > > > > > > >
>> > > > >>> > > > > > > > > >
>> > > > >>> > > > > > > >
>> > > > >>> > > > > > >
>> > > > >>> > > > > >
>> > > > >>> > > > >
>> > > > >>> > > >
>> > > > >>> > >
>> > > > >>> >
>> > > > >>>
>> > > >
>> > >
>> >
>>
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/METplus/parm/user_config/Plotter_fcstGFS_obsGFS_RPlotting_test.conf
>> > > > >>> > > > > > > > > > > > >
>> > > > >>> > > > > > > > > > > > > The output directory is
>> > > > >>> > > > > > > > > > > > >
>> > > > >>> > > > > > > >
>> > > > >>> > >
>> > >
/mnt/lfs4/HFIP/hwrfv3/Yan.Jin/Verification/Eta_use_METplus/tc_pairs
>> > > > >>> > > > > > > > > > > > > and the tc_pairs data:
>> > > al292020_pairs.dat.tcst
>> > > > >>> > > > > > > > > > > > >
>> > > > >>> > > > > > > > > > > > > log file
>> > > > >>> > > > > > > > > > > > >
>> > > > >>> > > > > > > > > > > > >
>> > > > >>> > > > > > > > > > > >
>> > > > >>> > > > >