[Met_help] [rt.rap.ucar.edu #94784] History for Question to run MakePlots after StatAnalysis in METplus 3.0
Minna Win via RT
met_help at ucar.edu
Tue Oct 13 13:28:11 MDT 2020
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hi,
I’m trying to run MakePlots after StatAnalysis, so I set "PROCESS_LIST=StatAnalysis, MakePlots" and "LOOP_ORDER=processes" in my config file.
After I execute the master_metplus.py with my config files, it’s weird that looks like METplus only ran MakePlots instead of StatAnalysis first then MakePlots.
Using the same config but set StatAnalysis only, it can generate the filter output properly.
Is there any example or guidelines to setup this kind of job?
I work on cheyenne, please check /glade/work/swei/MAPP/METplus_wk/workdir_bk if you need more information.
Thank you.
Best,
Shih-Wei
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Question to run MakePlots after StatAnalysis in METplus 3.0
From: John Halley Gotway
Time: Mon Mar 30 21:39:53 2020
Shih-Wei,
I'm assigning this ticket to George McCabe. You should hear back from
him
in the next couple of days.
Thanks,
John
On Mon, Mar 30, 2020 at 5:53 PM Wei, Shih Wei via RT
<met_help at ucar.edu>
wrote:
>
> Mon Mar 30 17:53:14 2020: Request 94784 was acted upon.
> Transaction: Ticket created by swei at albany.edu
> Queue: met_help
> Subject: Question to run MakePlots after StatAnalysis in
METplus 3.0
> Owner: Nobody
> Requestors: swei at albany.edu
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94784 >
>
>
> Hi,
>
> I’m trying to run MakePlots after StatAnalysis, so I set
> "PROCESS_LIST=StatAnalysis, MakePlots" and "LOOP_ORDER=processes" in
my
> config file.
> After I execute the master_metplus.py with my config files, it’s
weird
> that looks like METplus only ran MakePlots instead of StatAnalysis
first
> then MakePlots.
> Using the same config but set StatAnalysis only, it can generate the
> filter output properly.
> Is there any example or guidelines to setup this kind of job?
> I work on cheyenne, please check
> /glade/work/swei/MAPP/METplus_wk/workdir_bk if you need more
information.
> Thank you.
>
> Best,
> Shih-Wei
>
>
>
------------------------------------------------
Subject: Question to run MakePlots after StatAnalysis in METplus 3.0
From: Minna Win
Time: Wed Apr 01 14:04:07 2020
Hello Shih Wei,
I was assigned your help ticket. I am not familiar with this plotting
script, as it was created by a NOAA colleague, so it will take me a
while
to figure out how the script works. I want to verify that you are
using
METplus v3.0, is that correct?
Regards,
Minna
---------------
Minna Win
National Center for Atmospheric Research
Developmental Testbed Center
Phone: 303-497-8423
Fax: 303-497-8401
On Tue, Mar 31, 2020 at 7:43 AM George McCabe via RT
<met_help at ucar.edu>
wrote:
>
> Tue Mar 31 07:42:45 2020: Request 94784 was acted upon.
> Transaction: Given to minnawin (Minna Win) by mccabe
> Queue: met_help
> Subject: Question to run MakePlots after StatAnalysis in
METplus 3.0
> Owner: minnawin
> Requestors: swei at albany.edu
> Status: open
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94784 >
>
>
> This transaction appears to have no content
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #94784] Question to run MakePlots after StatAnalysis in METplus 3.0
From: Wei, Shih Wei
Time: Wed Apr 01 14:41:13 2020
Hi Minna,
Yes, I’m using METplus v3.0.
Shih-Wei
> On Apr 1, 2020, at 2:04 PM, Minna Win via RT <met_help at ucar.edu>
wrote:
>
> Hello Shih Wei,
>
> I was assigned your help ticket. I am not familiar with this
plotting
> script, as it was created by a NOAA colleague, so it will take me a
while
> to figure out how the script works. I want to verify that you are
using
> METplus v3.0, is that correct?
>
> Regards,
> Minna
> ---------------
> Minna Win
> National Center for Atmospheric Research
> Developmental Testbed Center
> Phone: 303-497-8423
> Fax: 303-497-8401
>
>
>
> On Tue, Mar 31, 2020 at 7:43 AM George McCabe via RT
<met_help at ucar.edu>
> wrote:
>
>>
>> Tue Mar 31 07:42:45 2020: Request 94784 was acted upon.
>> Transaction: Given to minnawin (Minna Win) by mccabe
>> Queue: met_help
>> Subject: Question to run MakePlots after StatAnalysis in
METplus 3.0
>> Owner: minnawin
>> Requestors: swei at albany.edu
>> Status: open
>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94784 >
>>
>>
>> This transaction appears to have no content
>>
>
------------------------------------------------
Subject: Question to run MakePlots after StatAnalysis in METplus 3.0
From: Minna Win
Time: Wed Apr 01 16:18:45 2020
Hi Shih-Wei,
Could you please re-run master_metplus.py with your StatAnalysis.conf
file
with the following changes:
-set PROCESS_LIST=StatAnalysis. (let's isolate it to one wrapper to
make
sure the config file is correct. Upon inspection it looks correct)
-under your [config] section, set the logging level to debug (so we
have
more useful information in the log file):
LOG_LEVEL=debug
I suspect that StatAnalysis ran and didn't produce output. Let me
know
when you've done this so I can check your output on 'cheyenne'.
Thanks,
Minna
---------------
Minna Win
National Center for Atmospheric Research
Developmental Testbed Center
Phone: 303-497-8423
Fax: 303-497-8401
On Wed, Apr 1, 2020 at 2:41 PM Wei, Shih Wei via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94784 >
>
> Hi Minna,
>
> Yes, I’m using METplus v3.0.
>
> Shih-Wei
>
> > On Apr 1, 2020, at 2:04 PM, Minna Win via RT <met_help at ucar.edu>
wrote:
> >
> > Hello Shih Wei,
> >
> > I was assigned your help ticket. I am not familiar with this
plotting
> > script, as it was created by a NOAA colleague, so it will take me
a while
> > to figure out how the script works. I want to verify that you are
using
> > METplus v3.0, is that correct?
> >
> > Regards,
> > Minna
> > ---------------
> > Minna Win
> > National Center for Atmospheric Research
> > Developmental Testbed Center
> > Phone: 303-497-8423
> > Fax: 303-497-8401
> >
> >
> >
> > On Tue, Mar 31, 2020 at 7:43 AM George McCabe via RT
<met_help at ucar.edu>
> > wrote:
> >
> >>
> >> Tue Mar 31 07:42:45 2020: Request 94784 was acted upon.
> >> Transaction: Given to minnawin (Minna Win) by mccabe
> >> Queue: met_help
> >> Subject: Question to run MakePlots after StatAnalysis in
METplus 3.0
> >> Owner: minnawin
> >> Requestors: swei at albany.edu
> >> Status: open
> >> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94784 >
> >>
> >>
> >> This transaction appears to have no content
> >>
> >
>
>
>
>
------------------------------------------------
Subject: Question to run MakePlots after StatAnalysis in METplus 3.0
From: Wei, Shih Wei
Time: Wed Apr 01 17:13:04 2020
Hi Minna,
I just ran it again with StatAnalysis only.
It can generate output properly under
/glade/work/swei/MAPP/METplus_wk/workdir_bk/output
I tried to set LOG_LEVEL=debug, but it came out some error said it’s
not recognized.
BTW, I feel like the LOG_STAT_ANALYSIS_VERBOSITY doesn’t work
properly, because the log file looks the same no matter what value I
setup.
Now the latest log file and output are under
/glade/work/swei/MAPP/METplus_wk/workdir_bk/
Please check this log file,
/glade/work/swei/MAPP/METplus_wk/workdir_bk/logs/master_metplus.log.20200401170543
Best,
Shih-Wei
On Apr 1, 2020, at 4:18 PM, Minna Win via RT
<met_help at ucar.edu<mailto:met_help at ucar.edu>> wrote:
Hi Shih-Wei,
Could you please re-run master_metplus.py with your StatAnalysis.conf
file
with the following changes:
-set PROCESS_LIST=StatAnalysis. (let's isolate it to one wrapper to
make
sure the config file is correct. Upon inspection it looks correct)
-under your [config] section, set the logging level to debug (so we
have
more useful information in the log file):
LOG_LEVEL=debug
I suspect that StatAnalysis ran and didn't produce output. Let me
know
when you've done this so I can check your output on 'cheyenne'.
Thanks,
Minna
---------------
Minna Win
National Center for Atmospheric Research
Developmental Testbed Center
Phone: 303-497-8423
Fax: 303-497-8401
On Wed, Apr 1, 2020 at 2:41 PM Wei, Shih Wei via RT
<met_help at ucar.edu<mailto:met_help at ucar.edu>>
wrote:
<URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94784 >
Hi Minna,
Yes, I’m using METplus v3.0.
Shih-Wei
On Apr 1, 2020, at 2:04 PM, Minna Win via RT
<met_help at ucar.edu<mailto:met_help at ucar.edu>> wrote:
Hello Shih Wei,
I was assigned your help ticket. I am not familiar with this plotting
script, as it was created by a NOAA colleague, so it will take me a
while
to figure out how the script works. I want to verify that you are
using
METplus v3.0, is that correct?
Regards,
Minna
---------------
Minna Win
National Center for Atmospheric Research
Developmental Testbed Center
Phone: 303-497-8423
Fax: 303-497-8401
On Tue, Mar 31, 2020 at 7:43 AM George McCabe via RT
<met_help at ucar.edu<mailto:met_help at ucar.edu>>
wrote:
Tue Mar 31 07:42:45 2020: Request 94784 was acted upon.
Transaction: Given to minnawin (Minna Win) by mccabe
Queue: met_help
Subject: Question to run MakePlots after StatAnalysis in METplus
3.0
Owner: minnawin
Requestors: swei at albany.edu<mailto:swei at albany.edu>
Status: open
Ticket <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94784 >
This transaction appears to have no content
------------------------------------------------
Subject: Question to run MakePlots after StatAnalysis in METplus 3.0
From: Minna Win
Time: Wed Apr 01 17:35:45 2020
Hi Shih-Wei,
You need to set the log level to DEBUG (all caps). I'll run Grid-Stat
(directly and via the wrapper) locally on some sample data and see if
the
log level is sensitive to changes in values to see if this is an issue
(and
if so, if it's in the wrapper or in MET).
Regards,
Minna
---------------
Minna Win
National Center for Atmospheric Research
Developmental Testbed Center
Phone: 303-497-8423
Fax: 303-497-8401
On Wed, Apr 1, 2020 at 5:13 PM Wei, Shih Wei via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94784 >
>
> Hi Minna,
>
> I just ran it again with StatAnalysis only.
> It can generate output properly under
> /glade/work/swei/MAPP/METplus_wk/workdir_bk/output
> I tried to set LOG_LEVEL=debug, but it came out some error said it’s
not
> recognized.
> BTW, I feel like the LOG_STAT_ANALYSIS_VERBOSITY doesn’t work
properly,
> because the log file looks the same no matter what value I setup.
> Now the latest log file and output are under
> /glade/work/swei/MAPP/METplus_wk/workdir_bk/
> Please check this log file,
>
>
/glade/work/swei/MAPP/METplus_wk/workdir_bk/logs/master_metplus.log.20200401170543
>
> Best,
> Shih-Wei
>
> On Apr 1, 2020, at 4:18 PM, Minna Win via RT
<met_help at ucar.edu<mailto:
> met_help at ucar.edu>> wrote:
>
> Hi Shih-Wei,
>
> Could you please re-run master_metplus.py with your
StatAnalysis.conf file
> with the following changes:
>
> -set PROCESS_LIST=StatAnalysis. (let's isolate it to one wrapper to
make
> sure the config file is correct. Upon inspection it looks correct)
> -under your [config] section, set the logging level to debug (so we
have
> more useful information in the log file):
> LOG_LEVEL=debug
>
> I suspect that StatAnalysis ran and didn't produce output. Let me
know
> when you've done this so I can check your output on 'cheyenne'.
>
> Thanks,
> Minna
> ---------------
> Minna Win
> National Center for Atmospheric Research
> Developmental Testbed Center
> Phone: 303-497-8423
> Fax: 303-497-8401
>
>
>
> On Wed, Apr 1, 2020 at 2:41 PM Wei, Shih Wei via RT
<met_help at ucar.edu
> <mailto:met_help at ucar.edu>>
> wrote:
>
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94784 >
>
> Hi Minna,
>
> Yes, I’m using METplus v3.0.
>
> Shih-Wei
>
> On Apr 1, 2020, at 2:04 PM, Minna Win via RT
<met_help at ucar.edu<mailto:
> met_help at ucar.edu>> wrote:
>
> Hello Shih Wei,
>
> I was assigned your help ticket. I am not familiar with this
plotting
> script, as it was created by a NOAA colleague, so it will take me a
while
> to figure out how the script works. I want to verify that you are
using
> METplus v3.0, is that correct?
>
> Regards,
> Minna
> ---------------
> Minna Win
> National Center for Atmospheric Research
> Developmental Testbed Center
> Phone: 303-497-8423
> Fax: 303-497-8401
>
>
>
> On Tue, Mar 31, 2020 at 7:43 AM George McCabe via RT
<met_help at ucar.edu
> <mailto:met_help at ucar.edu>>
> wrote:
>
>
> Tue Mar 31 07:42:45 2020: Request 94784 was acted upon.
> Transaction: Given to minnawin (Minna Win) by mccabe
> Queue: met_help
> Subject: Question to run MakePlots after StatAnalysis in METplus
3.0
> Owner: minnawin
> Requestors: swei at albany.edu<mailto:swei at albany.edu>
> Status: open
> Ticket <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94784
>
>
>
> This transaction appears to have no content
>
>
>
>
>
>
>
>
>
>
------------------------------------------------
Subject: Question to run MakePlots after StatAnalysis in METplus 3.0
From: Wei, Shih Wei
Time: Wed Apr 01 18:54:50 2020
Hi Minna,
I recall John told me the verbosity level is up to 10, when I asked
him some question about grid_stat.
Is it also 10 in stat_analysis?
For series_analysis, the log file will have "DEBUG 5:” message after I
increased it to 5.
I expect that I could see the similar string in the log file of
stat_analysis, but it only shows "DEBUG 2:”
That’s why I feel like the verbosity level has no function in
stat_analysis.
Under logs in my working directory,
master_metplus.log.20200401183740 is the output for PROCESS_LIST=
StatAnalysis, MakePlots
master_metplus.log.20200401184747 is the output for PROCESS_LIST=
StatAnalysis
LOGLEVEL= DEBUG in both.
Thank you.
Best,
Shih-Wei
On Apr 1, 2020, at 5:35 PM, Minna Win via RT
<met_help at ucar.edu<mailto:met_help at ucar.edu>> wrote:
Hi Shih-Wei,
You need to set the log level to DEBUG (all caps). I'll run Grid-Stat
(directly and via the wrapper) locally on some sample data and see if
the
log level is sensitive to changes in values to see if this is an issue
(and
if so, if it's in the wrapper or in MET).
Regards,
Minna
---------------
Minna Win
National Center for Atmospheric Research
Developmental Testbed Center
Phone: 303-497-8423
Fax: 303-497-8401
On Wed, Apr 1, 2020 at 5:13 PM Wei, Shih Wei via RT
<met_help at ucar.edu<mailto:met_help at ucar.edu>>
wrote:
<URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94784 >
Hi Minna,
I just ran it again with StatAnalysis only.
It can generate output properly under
/glade/work/swei/MAPP/METplus_wk/workdir_bk/output
I tried to set LOG_LEVEL=debug, but it came out some error said it’s
not
recognized.
BTW, I feel like the LOG_STAT_ANALYSIS_VERBOSITY doesn’t work
properly,
because the log file looks the same no matter what value I setup.
Now the latest log file and output are under
/glade/work/swei/MAPP/METplus_wk/workdir_bk/
Please check this log file,
/glade/work/swei/MAPP/METplus_wk/workdir_bk/logs/master_metplus.log.20200401170543
Best,
Shih-Wei
On Apr 1, 2020, at 4:18 PM, Minna Win via RT
<met_help at ucar.edu<mailto:met_help at ucar.edu><mailto:
met_help at ucar.edu<mailto:met_help at ucar.edu>>> wrote:
Hi Shih-Wei,
Could you please re-run master_metplus.py with your StatAnalysis.conf
file
with the following changes:
-set PROCESS_LIST=StatAnalysis. (let's isolate it to one wrapper to
make
sure the config file is correct. Upon inspection it looks correct)
-under your [config] section, set the logging level to debug (so we
have
more useful information in the log file):
LOG_LEVEL=debug
I suspect that StatAnalysis ran and didn't produce output. Let me
know
when you've done this so I can check your output on 'cheyenne'.
Thanks,
Minna
---------------
Minna Win
National Center for Atmospheric Research
Developmental Testbed Center
Phone: 303-497-8423
Fax: 303-497-8401
On Wed, Apr 1, 2020 at 2:41 PM Wei, Shih Wei via RT
<met_help at ucar.edu<mailto:met_help at ucar.edu>
<mailto:met_help at ucar.edu>>
wrote:
<URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94784 >
Hi Minna,
Yes, I’m using METplus v3.0.
Shih-Wei
On Apr 1, 2020, at 2:04 PM, Minna Win via RT
<met_help at ucar.edu<mailto:met_help at ucar.edu><mailto:
met_help at ucar.edu<mailto:met_help at ucar.edu>>> wrote:
Hello Shih Wei,
I was assigned your help ticket. I am not familiar with this plotting
script, as it was created by a NOAA colleague, so it will take me a
while
to figure out how the script works. I want to verify that you are
using
METplus v3.0, is that correct?
Regards,
Minna
---------------
Minna Win
National Center for Atmospheric Research
Developmental Testbed Center
Phone: 303-497-8423
Fax: 303-497-8401
On Tue, Mar 31, 2020 at 7:43 AM George McCabe via RT
<met_help at ucar.edu<mailto:met_help at ucar.edu>
<mailto:met_help at ucar.edu>>
wrote:
Tue Mar 31 07:42:45 2020: Request 94784 was acted upon.
Transaction: Given to minnawin (Minna Win) by mccabe
Queue: met_help
Subject: Question to run MakePlots after StatAnalysis in METplus 3.0
Owner: minnawin
Requestors:
swei at albany.edu<mailto:swei at albany.edu><mailto:swei at albany.edu>
Status: open
Ticket <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=94784 >
This transaction appears to have no content
------------------------------------------------
Subject: Question to run MakePlots after StatAnalysis in METplus 3.0
From: Minna Win
Time: Tue Apr 07 09:52:08 2020
Hello Shih-Weh,
It appears that there is a bug in the StatAnalysis wrapper, I created
an issue in the METplus GitHub repository:
https://github.com/NCAR/METplus/issues/422
Thank you for bringing this issue to our attention.
Regards,
Minna
------------------------------------------------
Subject: Question to run MakePlots after StatAnalysis in METplus 3.0
From: George McCabe
Time: Wed Apr 22 11:00:47 2020
Hi Shih-Weh,
I apologize for the delayed response. Upon looking into this issue, I
noticed that two METplus configurations variables are named
incorrectly:
FCST_VAR1 should be FCST_VAR1_NAME
OBS_VAR1 should be OBS_VAR1_NAME
Changing the names of these variables caused StatAnalysis to run for
me. However, there are still errors in MakePlots that I was unable to
diagnose. It looks like a variable in plot_util.py is referenced
before it is assigned. I am unfamiliar with these plotting scripts,
but I will pass this information on to someone else who should be able
to help diagnose the issue.
Thanks,
George
On Tue Apr 07 09:52:08 2020, minnawin wrote:
> Hello Shih-Weh,
>
> It appears that there is a bug in the StatAnalysis wrapper, I
created
> an issue in the METplus GitHub repository:
> https://github.com/NCAR/METplus/issues/422
>
> Thank you for bringing this issue to our attention.
>
> Regards,
> Minna
------------------------------------------------
More information about the Met_help
mailing list