[Met_help] [rt.rap.ucar.edu #95448] History for references to a file that doesn't exist
John Halley Gotway via RT
met_help at ucar.edu
Tue Jun 2 09:31:15 MDT 2020
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Good afternoon,
I've slowly been setting up my configure file to run StatAnalysis,
MakePlots.
Everything is pretty well in terms of understanding where things go;
however, there is a reference to a file which doesn't exist in that
directory. I've searched my directory structure to determine where this
file is being referenced in order to change it from
*met_header_columns_V9.0.txt* to *met_header_columns_V8.1.txt*. Here is
the basic error..
06/01 18:01:53.787 metplus.StatAnalysis (stat_analysis_wrapper.py:1038)
DEBUG: INTERP: BILIN
06/01 18:01:53.787 metplus.StatAnalysis (stat_analysis_wrapper.py:1039)
DEBUG: REGION: FULL
06/01 18:01:53.788 metplus.StatAnalysis (stat_analysis_wrapper.py:1040)
DEBUG: FCST_LEAD: 00
06/01 18:01:53.788 metplus.StatAnalysis (stat_analysis_wrapper.py:1041)
DEBUG: MODEL: AQ
06/01 18:01:53.788 metplus.StatAnalysis (stat_analysis_wrapper.py:1042)
DEBUG: OBTYPE: ONLYSF
06/01 18:01:53.788 metplus.StatAnalysis (stat_analysis_wrapper.py:1043)
DEBUG: LINE_TYPE:
06/01 18:01:53.788 metplus.StatAnalysis (stat_analysis_wrapper.py:1044)
DEBUG: JOB: -job filter -dump_row
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_aq/prod/stat/aqm/20200525/grid2obs_step2/metplus_output/plot_by_valid/stat_analysis/OZCON_FULL/grid2obs/conus_sfc/AQ/valid20200525to20200525_valid000000to230000Z_init060000to060000Z/AQ_f00_fcstOZCONA1_obsCOPOA1_interpBILIN_regionFULL.stat
06/01 18:01:53.788 metplus.StatAnalysis (stat_analysis_wrapper.py:1045)
DEBUG: lookin directory:
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_aq/prod/stat/aqm/20200525
06/01 18:01:53.789 metplus.StatAnalysis (command_runner.py:117) INFO:
app_name is: stat_analysis, output sent to:
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_aq/prod/stat/aqm/20200525/grid2obs_step2/metplus_output/logs/master_metplus_grid2obs_conus_sfc_statanalysis_makeplots_OZCON_FULL.runon20200601180153
06/01 18:01:53.995 metplus.StatAnalysis (command_runner.py:153) INFO:
RUNNING: /usrx/local/dev/packages/met/8.1/bin/stat_analysis -lookin
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_aq/prod/stat/aqm/20200525
-config
/gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/EMC_verif-global/parm/metplus_config/metplus_use_cases/METplusV2.1/grid2obs/met_config/metV8.1/STATAnalysisConfig
>>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_aq/prod/stat/aqm/20200525/grid2obs_step2/metplus_output/logs/master_metplus_grid2obs_conus_sfc_statanalysis_makeplots_OZCON_FULL.runon20200601180153
2>&1
06/01 18:01:53.995 metplus.StatAnalysis (config_wrapper.py:102) DEBUG:
Setting [config] DO_NOT_RUN_EXE to default value: False.
DEBUG 1: Default Config File:
/usrx/local/dev/packages/met/8.1/share/met/config/STATAnalysisConfig_default
DEBUG 1: User Config File:
/gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/EMC_verif-global/parm/metplus_config/metplus_use_cases/METplusV2.1/grid2obs/met_config/metV8.1/STATAnalysisConfig
*DEBUG 2: Processing 48 STAT files.ERROR :ERROR : AsciiHeader::read() ->
trouble reading file:ERROR :
/usrx/local/dev/packages/met/8.1/share/met/table_files/met_header_columns_V9.0.txtERROR
:*
06/01 18:01:55.402 metplus.StatAnalysis (stat_analysis_wrapper.py:1007)
DEBUG: STAT_ANALYSIS RUN SETTINGS....
06/01 18:01:55.402 metplus.StatAnalysis (stat_analysis_wrapper.py:1010)
DEBUG: FCST_INIT_END:
06/01 18:01:55.402 metplus.StatAnalysis (stat_analysis_wrapper.py:1010)
DEBUG: FCST_INIT_BEG:
06/01 18:01:55.402 metplus.StatAnalysis (stat_analysis_wrapper.py:1010)
DEBUG: FCST_INIT_HOUR: "060000"
The reference to this file doesn't exist in any of the files within my
directory structure and I'm unsure how it is being referenced. My first
thought is related to default settings, but I can't seem to determine
whether that's the case.
Just in case, my run script is here:
*/gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/EMC_verif-global/scripts/job1*
my machine.conf is here:
*/gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/EMC_verif-global/parm/metplus_config/machine.conf*
my plot_grid2obs_conus_sfc.conf is here:
*/gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/EMC_verif-global/parm/plotting/examples/plot_grid2obs_conus_sfc.conf*
and my log is here:
*/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_aq/prod/stat/aqm/20200525/grid2obs_step2/metplus_output/logs/*
Again, I don't think this is a MakePlots issue, rather it seems to be a
default fallback. I just can't find the reference to this in any of the
files. Why would it assume V9.0 when it is V8.1. Any thoughts on this is
greatly appreciated.
--
Edward Strobach
EMC/NCEP/NWS/
IMSG Contractor
Cubicle#: 2029
301-683-3717
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: references to a file that doesn't exist
From: John Halley Gotway
Time: Mon Jun 01 22:52:56 2020
Ed,
Whenever we release a new version of the MET software, we add a new
file
named "met_header_columns_VX.Y.txt" where X.Y is the new version
number.
That file defines the header columns for all of the STAT line types,
and
Stat-Analysis uses that file to figure out how to parse the ASCII data
in
the .stat files.
It stands to reason that met-8.1 would include a file for
met_header_columns_V8.1.txt but would NOT include
met_header_columns_V9.0.txt. That is to say that while met-9.0 CAN
reads
met-8.1 output just fine, met-8.1 CANNOT read met-9.0 output. And I
suspect
that's what's really going on here.
Look very closely at the .stat files that are being passed to the met-
8.1
version of stat_analysis. The first column of those .stat files
indicates
the MET version number. And I suspect you'll find "V9.0" listed in at
least
one input line.
Hope that helps.
John Halley Gotway
On Mon, Jun 1, 2020 at 12:55 PM Edward Strobach - NOAA Affiliate via
RT <
met_help at ucar.edu> wrote:
>
> Mon Jun 01 12:54:35 2020: Request 95448 was acted upon.
> Transaction: Ticket created by edward.strobach at noaa.gov
> Queue: met_help
> Subject: references to a file that doesn't exist
> Owner: Nobody
> Requestors: edward.strobach at noaa.gov
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95448 >
>
>
> Good afternoon,
>
> I've slowly been setting up my configure file to run StatAnalysis,
> MakePlots.
>
> Everything is pretty well in terms of understanding where things go;
> however, there is a reference to a file which doesn't exist in that
> directory. I've searched my directory structure to determine where
this
> file is being referenced in order to change it from
> *met_header_columns_V9.0.txt* to *met_header_columns_V8.1.txt*.
Here is
> the basic error..
>
> 06/01 18:01:53.787 metplus.StatAnalysis
(stat_analysis_wrapper.py:1038)
> DEBUG: INTERP: BILIN
> 06/01 18:01:53.787 metplus.StatAnalysis
(stat_analysis_wrapper.py:1039)
> DEBUG: REGION: FULL
> 06/01 18:01:53.788 metplus.StatAnalysis
(stat_analysis_wrapper.py:1040)
> DEBUG: FCST_LEAD: 00
> 06/01 18:01:53.788 metplus.StatAnalysis
(stat_analysis_wrapper.py:1041)
> DEBUG: MODEL: AQ
> 06/01 18:01:53.788 metplus.StatAnalysis
(stat_analysis_wrapper.py:1042)
> DEBUG: OBTYPE: ONLYSF
> 06/01 18:01:53.788 metplus.StatAnalysis
(stat_analysis_wrapper.py:1043)
> DEBUG: LINE_TYPE:
> 06/01 18:01:53.788 metplus.StatAnalysis
(stat_analysis_wrapper.py:1044)
> DEBUG: JOB: -job filter -dump_row
>
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_aq/prod/stat/aqm/20200525/grid2obs_step2/metplus_output/plot_by_valid/stat_analysis/OZCON_FULL/grid2obs/conus_sfc/AQ/valid20200525to20200525_valid000000to230000Z_init060000to060000Z/AQ_f00_fcstOZCONA1_obsCOPOA1_interpBILIN_regionFULL.stat
> 06/01 18:01:53.788 metplus.StatAnalysis
(stat_analysis_wrapper.py:1045)
> DEBUG: lookin directory:
>
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_aq/prod/stat/aqm/20200525
> 06/01 18:01:53.789 metplus.StatAnalysis (command_runner.py:117)
INFO:
> app_name is: stat_analysis, output sent to:
>
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_aq/prod/stat/aqm/20200525/grid2obs_step2/metplus_output/logs/master_metplus_grid2obs_conus_sfc_statanalysis_makeplots_OZCON_FULL.runon20200601180153
> 06/01 18:01:53.995 metplus.StatAnalysis (command_runner.py:153)
INFO:
> RUNNING: /usrx/local/dev/packages/met/8.1/bin/stat_analysis -lookin
>
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_aq/prod/stat/aqm/20200525
> -config
>
> /gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/EMC_verif-
global/parm/metplus_config/metplus_use_cases/METplusV2.1/grid2obs/met_config/metV8.1/STATAnalysisConfig
> >>
>
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_aq/prod/stat/aqm/20200525/grid2obs_step2/metplus_output/logs/master_metplus_grid2obs_conus_sfc_statanalysis_makeplots_OZCON_FULL.runon20200601180153
> 2>&1
> 06/01 18:01:53.995 metplus.StatAnalysis (config_wrapper.py:102)
DEBUG:
> Setting [config] DO_NOT_RUN_EXE to default value: False.
> DEBUG 1: Default Config File:
>
>
/usrx/local/dev/packages/met/8.1/share/met/config/STATAnalysisConfig_default
> DEBUG 1: User Config File:
>
> /gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/EMC_verif-
global/parm/metplus_config/metplus_use_cases/METplusV2.1/grid2obs/met_config/metV8.1/STATAnalysisConfig
>
>
>
>
> *DEBUG 2: Processing 48 STAT files.ERROR :ERROR :
AsciiHeader::read() ->
> trouble reading file:ERROR :
>
>
/usrx/local/dev/packages/met/8.1/share/met/table_files/met_header_columns_V9.0.txtERROR
> :*
> 06/01 18:01:55.402 metplus.StatAnalysis
(stat_analysis_wrapper.py:1007)
> DEBUG: STAT_ANALYSIS RUN SETTINGS....
> 06/01 18:01:55.402 metplus.StatAnalysis
(stat_analysis_wrapper.py:1010)
> DEBUG: FCST_INIT_END:
> 06/01 18:01:55.402 metplus.StatAnalysis
(stat_analysis_wrapper.py:1010)
> DEBUG: FCST_INIT_BEG:
> 06/01 18:01:55.402 metplus.StatAnalysis
(stat_analysis_wrapper.py:1010)
> DEBUG: FCST_INIT_HOUR: "060000"
>
>
> The reference to this file doesn't exist in any of the files within
my
> directory structure and I'm unsure how it is being referenced. My
first
> thought is related to default settings, but I can't seem to
determine
> whether that's the case.
>
> Just in case, my run script is here:
>
> */gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/EMC_verif-
global/scripts/job1*
> my machine.conf is here:
>
> */gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/EMC_verif-
global/parm/metplus_config/machine.conf*
> my plot_grid2obs_conus_sfc.conf is here:
>
> */gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/EMC_verif-
global/parm/plotting/examples/plot_grid2obs_conus_sfc.conf*
> and my log is here:
>
>
*/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_aq/prod/stat/aqm/20200525/grid2obs_step2/metplus_output/logs/*
>
> Again, I don't think this is a MakePlots issue, rather it seems to
be a
> default fallback. I just can't find the reference to this in any of
the
> files. Why would it assume V9.0 when it is V8.1. Any thoughts on
this is
> greatly appreciated.
> --
> Edward Strobach
> EMC/NCEP/NWS/
> IMSG Contractor
> Cubicle#: 2029
> 301-683-3717
>
>
------------------------------------------------
Subject: references to a file that doesn't exist
From: Edward Strobach - NOAA Affiliate
Time: Tue Jun 02 06:33:43 2020
Oh...that's a problem. I broke the process into parts:
The first part takes the air quality data, converts prepbufr to netcdf
and
generates .stat files. The process list is as follows: PB2NC,
PointStat,
StatAnalysis. This was done with Met 9.0/Metplus3.0.
The second step was to take the .stat files, run them through stat
analysis
again along with MakePlots. The process list is as follows:
StatAnalysis,
MakePlots. This was done with Met8.1/Metplus2.1. I guess what I'll
want
to do is convert the second step over to Met 9.0/Metplus3.0 in order
to
make it work. That's probably how I need to handle this problem.
On Tue, Jun 2, 2020 at 12:53 AM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
> Ed,
>
> Whenever we release a new version of the MET software, we add a new
file
> named "met_header_columns_VX.Y.txt" where X.Y is the new version
number.
> That file defines the header columns for all of the STAT line types,
and
> Stat-Analysis uses that file to figure out how to parse the ASCII
data in
> the .stat files.
>
> It stands to reason that met-8.1 would include a file for
> met_header_columns_V8.1.txt but would NOT include
> met_header_columns_V9.0.txt. That is to say that while met-9.0 CAN
reads
> met-8.1 output just fine, met-8.1 CANNOT read met-9.0 output. And I
suspect
> that's what's really going on here.
>
> Look very closely at the .stat files that are being passed to the
met-8.1
> version of stat_analysis. The first column of those .stat files
indicates
> the MET version number. And I suspect you'll find "V9.0" listed in
at least
> one input line.
>
> Hope that helps.
>
> John Halley Gotway
>
> On Mon, Jun 1, 2020 at 12:55 PM Edward Strobach - NOAA Affiliate via
RT <
> met_help at ucar.edu> wrote:
>
> >
> > Mon Jun 01 12:54:35 2020: Request 95448 was acted upon.
> > Transaction: Ticket created by edward.strobach at noaa.gov
> > Queue: met_help
> > Subject: references to a file that doesn't exist
> > Owner: Nobody
> > Requestors: edward.strobach at noaa.gov
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95448 >
> >
> >
> > Good afternoon,
> >
> > I've slowly been setting up my configure file to run StatAnalysis,
> > MakePlots.
> >
> > Everything is pretty well in terms of understanding where things
go;
> > however, there is a reference to a file which doesn't exist in
that
> > directory. I've searched my directory structure to determine
where this
> > file is being referenced in order to change it from
> > *met_header_columns_V9.0.txt* to *met_header_columns_V8.1.txt*.
Here is
> > the basic error..
> >
> > 06/01 18:01:53.787 metplus.StatAnalysis
(stat_analysis_wrapper.py:1038)
> > DEBUG: INTERP: BILIN
> > 06/01 18:01:53.787 metplus.StatAnalysis
(stat_analysis_wrapper.py:1039)
> > DEBUG: REGION: FULL
> > 06/01 18:01:53.788 metplus.StatAnalysis
(stat_analysis_wrapper.py:1040)
> > DEBUG: FCST_LEAD: 00
> > 06/01 18:01:53.788 metplus.StatAnalysis
(stat_analysis_wrapper.py:1041)
> > DEBUG: MODEL: AQ
> > 06/01 18:01:53.788 metplus.StatAnalysis
(stat_analysis_wrapper.py:1042)
> > DEBUG: OBTYPE: ONLYSF
> > 06/01 18:01:53.788 metplus.StatAnalysis
(stat_analysis_wrapper.py:1043)
> > DEBUG: LINE_TYPE:
> > 06/01 18:01:53.788 metplus.StatAnalysis
(stat_analysis_wrapper.py:1044)
> > DEBUG: JOB: -job filter -dump_row
> >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_aq/prod/stat/aqm/20200525/grid2obs_step2/metplus_output/plot_by_valid/stat_analysis/OZCON_FULL/grid2obs/conus_sfc/AQ/valid20200525to20200525_valid000000to230000Z_init060000to060000Z/AQ_f00_fcstOZCONA1_obsCOPOA1_interpBILIN_regionFULL.stat
> > 06/01 18:01:53.788 metplus.StatAnalysis
(stat_analysis_wrapper.py:1045)
> > DEBUG: lookin directory:
> >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_aq/prod/stat/aqm/20200525
> > 06/01 18:01:53.789 metplus.StatAnalysis (command_runner.py:117)
INFO:
> > app_name is: stat_analysis, output sent to:
> >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_aq/prod/stat/aqm/20200525/grid2obs_step2/metplus_output/logs/master_metplus_grid2obs_conus_sfc_statanalysis_makeplots_OZCON_FULL.runon20200601180153
> > 06/01 18:01:53.995 metplus.StatAnalysis (command_runner.py:153)
INFO:
> > RUNNING: /usrx/local/dev/packages/met/8.1/bin/stat_analysis
-lookin
> >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_aq/prod/stat/aqm/20200525
> > -config
> >
> >
> /gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/EMC_verif-
global/parm/metplus_config/metplus_use_cases/METplusV2.1/grid2obs/met_config/metV8.1/STATAnalysisConfig
> > >>
> >
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_aq/prod/stat/aqm/20200525/grid2obs_step2/metplus_output/logs/master_metplus_grid2obs_conus_sfc_statanalysis_makeplots_OZCON_FULL.runon20200601180153
> > 2>&1
> > 06/01 18:01:53.995 metplus.StatAnalysis (config_wrapper.py:102)
DEBUG:
> > Setting [config] DO_NOT_RUN_EXE to default value: False.
> > DEBUG 1: Default Config File:
> >
> >
>
/usrx/local/dev/packages/met/8.1/share/met/config/STATAnalysisConfig_default
> > DEBUG 1: User Config File:
> >
> >
> /gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/EMC_verif-
global/parm/metplus_config/metplus_use_cases/METplusV2.1/grid2obs/met_config/metV8.1/STATAnalysisConfig
> >
> >
> >
> >
> > *DEBUG 2: Processing 48 STAT files.ERROR :ERROR :
AsciiHeader::read()
> ->
> > trouble reading file:ERROR :
> >
> >
>
/usrx/local/dev/packages/met/8.1/share/met/table_files/met_header_columns_V9.0.txtERROR
> > :*
> > 06/01 18:01:55.402 metplus.StatAnalysis
(stat_analysis_wrapper.py:1007)
> > DEBUG: STAT_ANALYSIS RUN SETTINGS....
> > 06/01 18:01:55.402 metplus.StatAnalysis
(stat_analysis_wrapper.py:1010)
> > DEBUG: FCST_INIT_END:
> > 06/01 18:01:55.402 metplus.StatAnalysis
(stat_analysis_wrapper.py:1010)
> > DEBUG: FCST_INIT_BEG:
> > 06/01 18:01:55.402 metplus.StatAnalysis
(stat_analysis_wrapper.py:1010)
> > DEBUG: FCST_INIT_HOUR: "060000"
> >
> >
> > The reference to this file doesn't exist in any of the files
within my
> > directory structure and I'm unsure how it is being referenced. My
first
> > thought is related to default settings, but I can't seem to
determine
> > whether that's the case.
> >
> > Just in case, my run script is here:
> >
> >
> */gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/EMC_verif-
global/scripts/job1*
> > my machine.conf is here:
> >
> >
> */gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/EMC_verif-
global/parm/metplus_config/machine.conf*
> > my plot_grid2obs_conus_sfc.conf is here:
> >
> >
> */gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/EMC_verif-
global/parm/plotting/examples/plot_grid2obs_conus_sfc.conf*
> > and my log is here:
> >
> >
>
*/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_aq/prod/stat/aqm/20200525/grid2obs_step2/metplus_output/logs/*
> >
> > Again, I don't think this is a MakePlots issue, rather it seems to
be a
> > default fallback. I just can't find the reference to this in any
of the
> > files. Why would it assume V9.0 when it is V8.1. Any thoughts on
this
> is
> > greatly appreciated.
> > --
> > Edward Strobach
> > EMC/NCEP/NWS/
> > IMSG Contractor
> > Cubicle#: 2029
> > 301-683-3717
> >
> >
>
>
--
Edward Strobach
EMC/NCEP/NWS/
IMSG Contractor
Cubicle#: 2029
301-683-3717
------------------------------------------------
Subject: references to a file that doesn't exist
From: Edward Strobach - NOAA Affiliate
Time: Tue Jun 02 07:31:56 2020
As I make these changes, I did have one follow question:
Since VALID_HOUR_METHOD and INIT_HOUR_METHOD is no longer used, what
should
I do to specify "LOOP" v. "GROUP". What I was doing before was
grouping
valid times for each initial time. I see that VALID_HOUR_BEG,
INIT_HOUR_BEG, etc. has been replaced in list form (e.g.
FCST_VALID_HOUR_LIST and FCST_INIT_HOUR_LIST). I don't see how I can
"LOOP" v. "GROUP" this way.
On Tue, Jun 2, 2020 at 8:33 AM Edward Strobach - NOAA Affiliate <
edward.strobach at noaa.gov> wrote:
> Oh...that's a problem. I broke the process into parts:
>
> The first part takes the air quality data, converts prepbufr to
netcdf and
> generates .stat files. The process list is as follows: PB2NC,
PointStat,
> StatAnalysis. This was done with Met 9.0/Metplus3.0.
>
> The second step was to take the .stat files, run them through stat
> analysis again along with MakePlots. The process list is as
follows:
> StatAnalysis, MakePlots. This was done with Met8.1/Metplus2.1. I
guess
> what I'll want to do is convert the second step over to Met
9.0/Metplus3.0
> in order to make it work. That's probably how I need to handle this
> problem.
>
> On Tue, Jun 2, 2020 at 12:53 AM John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
>> Ed,
>>
>> Whenever we release a new version of the MET software, we add a new
file
>> named "met_header_columns_VX.Y.txt" where X.Y is the new version
number.
>> That file defines the header columns for all of the STAT line
types, and
>> Stat-Analysis uses that file to figure out how to parse the ASCII
data in
>> the .stat files.
>>
>> It stands to reason that met-8.1 would include a file for
>> met_header_columns_V8.1.txt but would NOT include
>> met_header_columns_V9.0.txt. That is to say that while met-9.0 CAN
reads
>> met-8.1 output just fine, met-8.1 CANNOT read met-9.0 output. And I
>> suspect
>> that's what's really going on here.
>>
>> Look very closely at the .stat files that are being passed to the
met-8.1
>> version of stat_analysis. The first column of those .stat files
indicates
>> the MET version number. And I suspect you'll find "V9.0" listed in
at
>> least
>> one input line.
>>
>> Hope that helps.
>>
>> John Halley Gotway
>>
>> On Mon, Jun 1, 2020 at 12:55 PM Edward Strobach - NOAA Affiliate
via RT <
>> met_help at ucar.edu> wrote:
>>
>> >
>> > Mon Jun 01 12:54:35 2020: Request 95448 was acted upon.
>> > Transaction: Ticket created by edward.strobach at noaa.gov
>> > Queue: met_help
>> > Subject: references to a file that doesn't exist
>> > Owner: Nobody
>> > Requestors: edward.strobach at noaa.gov
>> > Status: new
>> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95448 >
>> >
>> >
>> > Good afternoon,
>> >
>> > I've slowly been setting up my configure file to run
StatAnalysis,
>> > MakePlots.
>> >
>> > Everything is pretty well in terms of understanding where things
go;
>> > however, there is a reference to a file which doesn't exist in
that
>> > directory. I've searched my directory structure to determine
where this
>> > file is being referenced in order to change it from
>> > *met_header_columns_V9.0.txt* to *met_header_columns_V8.1.txt*.
Here is
>> > the basic error..
>> >
>> > 06/01 18:01:53.787 metplus.StatAnalysis
(stat_analysis_wrapper.py:1038)
>> > DEBUG: INTERP: BILIN
>> > 06/01 18:01:53.787 metplus.StatAnalysis
(stat_analysis_wrapper.py:1039)
>> > DEBUG: REGION: FULL
>> > 06/01 18:01:53.788 metplus.StatAnalysis
(stat_analysis_wrapper.py:1040)
>> > DEBUG: FCST_LEAD: 00
>> > 06/01 18:01:53.788 metplus.StatAnalysis
(stat_analysis_wrapper.py:1041)
>> > DEBUG: MODEL: AQ
>> > 06/01 18:01:53.788 metplus.StatAnalysis
(stat_analysis_wrapper.py:1042)
>> > DEBUG: OBTYPE: ONLYSF
>> > 06/01 18:01:53.788 metplus.StatAnalysis
(stat_analysis_wrapper.py:1043)
>> > DEBUG: LINE_TYPE:
>> > 06/01 18:01:53.788 metplus.StatAnalysis
(stat_analysis_wrapper.py:1044)
>> > DEBUG: JOB: -job filter -dump_row
>> >
>> >
>>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_aq/prod/stat/aqm/20200525/grid2obs_step2/metplus_output/plot_by_valid/stat_analysis/OZCON_FULL/grid2obs/conus_sfc/AQ/valid20200525to20200525_valid000000to230000Z_init060000to060000Z/AQ_f00_fcstOZCONA1_obsCOPOA1_interpBILIN_regionFULL.stat
>> > 06/01 18:01:53.788 metplus.StatAnalysis
(stat_analysis_wrapper.py:1045)
>> > DEBUG: lookin directory:
>> >
>> >
>>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_aq/prod/stat/aqm/20200525
>> > 06/01 18:01:53.789 metplus.StatAnalysis (command_runner.py:117)
INFO:
>> > app_name is: stat_analysis, output sent to:
>> >
>> >
>>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_aq/prod/stat/aqm/20200525/grid2obs_step2/metplus_output/logs/master_metplus_grid2obs_conus_sfc_statanalysis_makeplots_OZCON_FULL.runon20200601180153
>> > 06/01 18:01:53.995 metplus.StatAnalysis (command_runner.py:153)
INFO:
>> > RUNNING: /usrx/local/dev/packages/met/8.1/bin/stat_analysis
-lookin
>> >
>> >
>>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_aq/prod/stat/aqm/20200525
>> > -config
>> >
>> >
>> /gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/EMC_verif-
global/parm/metplus_config/metplus_use_cases/METplusV2.1/grid2obs/met_config/metV8.1/STATAnalysisConfig
>> > >>
>> >
>> >
>>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_aq/prod/stat/aqm/20200525/grid2obs_step2/metplus_output/logs/master_metplus_grid2obs_conus_sfc_statanalysis_makeplots_OZCON_FULL.runon20200601180153
>> > 2>&1
>> > 06/01 18:01:53.995 metplus.StatAnalysis (config_wrapper.py:102)
DEBUG:
>> > Setting [config] DO_NOT_RUN_EXE to default value: False.
>> > DEBUG 1: Default Config File:
>> >
>> >
>>
/usrx/local/dev/packages/met/8.1/share/met/config/STATAnalysisConfig_default
>> > DEBUG 1: User Config File:
>> >
>> >
>> /gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/EMC_verif-
global/parm/metplus_config/metplus_use_cases/METplusV2.1/grid2obs/met_config/metV8.1/STATAnalysisConfig
>> >
>> >
>> >
>> >
>> > *DEBUG 2: Processing 48 STAT files.ERROR :ERROR :
AsciiHeader::read()
>> ->
>> > trouble reading file:ERROR :
>> >
>> >
>>
/usrx/local/dev/packages/met/8.1/share/met/table_files/met_header_columns_V9.0.txtERROR
>> > :*
>> > 06/01 18:01:55.402 metplus.StatAnalysis
(stat_analysis_wrapper.py:1007)
>> > DEBUG: STAT_ANALYSIS RUN SETTINGS....
>> > 06/01 18:01:55.402 metplus.StatAnalysis
(stat_analysis_wrapper.py:1010)
>> > DEBUG: FCST_INIT_END:
>> > 06/01 18:01:55.402 metplus.StatAnalysis
(stat_analysis_wrapper.py:1010)
>> > DEBUG: FCST_INIT_BEG:
>> > 06/01 18:01:55.402 metplus.StatAnalysis
(stat_analysis_wrapper.py:1010)
>> > DEBUG: FCST_INIT_HOUR: "060000"
>> >
>> >
>> > The reference to this file doesn't exist in any of the files
within my
>> > directory structure and I'm unsure how it is being referenced.
My first
>> > thought is related to default settings, but I can't seem to
determine
>> > whether that's the case.
>> >
>> > Just in case, my run script is here:
>> >
>> >
>> */gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/EMC_verif-
global/scripts/job1*
>> > my machine.conf is here:
>> >
>> >
>> */gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/EMC_verif-
global/parm/metplus_config/machine.conf*
>> > my plot_grid2obs_conus_sfc.conf is here:
>> >
>> >
>> */gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/EMC_verif-
global/parm/plotting/examples/plot_grid2obs_conus_sfc.conf*
>> > and my log is here:
>> >
>> >
>>
*/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_aq/prod/stat/aqm/20200525/grid2obs_step2/metplus_output/logs/*
>> >
>> > Again, I don't think this is a MakePlots issue, rather it seems
to be a
>> > default fallback. I just can't find the reference to this in any
of the
>> > files. Why would it assume V9.0 when it is V8.1. Any thoughts
on this
>> is
>> > greatly appreciated.
>> > --
>> > Edward Strobach
>> > EMC/NCEP/NWS/
>> > IMSG Contractor
>> > Cubicle#: 2029
>> > 301-683-3717
>> >
>> >
>>
>>
>
> --
> Edward Strobach
> EMC/NCEP/NWS/
> IMSG Contractor
> Cubicle#: 2029
> 301-683-3717
>
--
Edward Strobach
EMC/NCEP/NWS/
IMSG Contractor
Cubicle#: 2029
301-683-3717
------------------------------------------------
Subject: references to a file that doesn't exist
From: Edward Strobach - NOAA Affiliate
Time: Tue Jun 02 07:41:28 2020
Perhaps through LOOP_LIST_ITEMS and GROUP_LIST_ITEMS?
On Tue, Jun 2, 2020 at 9:31 AM Edward Strobach - NOAA Affiliate <
edward.strobach at noaa.gov> wrote:
> As I make these changes, I did have one follow question:
> Since VALID_HOUR_METHOD and INIT_HOUR_METHOD is no longer used, what
> should I do to specify "LOOP" v. "GROUP". What I was doing before
was
> grouping valid times for each initial time. I see that
VALID_HOUR_BEG,
> INIT_HOUR_BEG, etc. has been replaced in list form (e.g.
> FCST_VALID_HOUR_LIST and FCST_INIT_HOUR_LIST). I don't see how I
can
> "LOOP" v. "GROUP" this way.
>
> On Tue, Jun 2, 2020 at 8:33 AM Edward Strobach - NOAA Affiliate <
> edward.strobach at noaa.gov> wrote:
>
>> Oh...that's a problem. I broke the process into parts:
>>
>> The first part takes the air quality data, converts prepbufr to
netcdf
>> and generates .stat files. The process list is as follows: PB2NC,
>> PointStat, StatAnalysis. This was done with Met 9.0/Metplus3.0.
>>
>> The second step was to take the .stat files, run them through stat
>> analysis again along with MakePlots. The process list is as
follows:
>> StatAnalysis, MakePlots. This was done with Met8.1/Metplus2.1. I
guess
>> what I'll want to do is convert the second step over to Met
9.0/Metplus3.0
>> in order to make it work. That's probably how I need to handle this
>> problem.
>>
>> On Tue, Jun 2, 2020 at 12:53 AM John Halley Gotway via RT <
>> met_help at ucar.edu> wrote:
>>
>>> Ed,
>>>
>>> Whenever we release a new version of the MET software, we add a
new file
>>> named "met_header_columns_VX.Y.txt" where X.Y is the new version
number.
>>> That file defines the header columns for all of the STAT line
types, and
>>> Stat-Analysis uses that file to figure out how to parse the ASCII
data in
>>> the .stat files.
>>>
>>> It stands to reason that met-8.1 would include a file for
>>> met_header_columns_V8.1.txt but would NOT include
>>> met_header_columns_V9.0.txt. That is to say that while met-9.0 CAN
reads
>>> met-8.1 output just fine, met-8.1 CANNOT read met-9.0 output. And
I
>>> suspect
>>> that's what's really going on here.
>>>
>>> Look very closely at the .stat files that are being passed to the
met-8.1
>>> version of stat_analysis. The first column of those .stat files
indicates
>>> the MET version number. And I suspect you'll find "V9.0" listed in
at
>>> least
>>> one input line.
>>>
>>> Hope that helps.
>>>
>>> John Halley Gotway
>>>
>>> On Mon, Jun 1, 2020 at 12:55 PM Edward Strobach - NOAA Affiliate
via RT <
>>> met_help at ucar.edu> wrote:
>>>
>>> >
>>> > Mon Jun 01 12:54:35 2020: Request 95448 was acted upon.
>>> > Transaction: Ticket created by edward.strobach at noaa.gov
>>> > Queue: met_help
>>> > Subject: references to a file that doesn't exist
>>> > Owner: Nobody
>>> > Requestors: edward.strobach at noaa.gov
>>> > Status: new
>>> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95448
>>> >
>>> >
>>> >
>>> > Good afternoon,
>>> >
>>> > I've slowly been setting up my configure file to run
StatAnalysis,
>>> > MakePlots.
>>> >
>>> > Everything is pretty well in terms of understanding where things
go;
>>> > however, there is a reference to a file which doesn't exist in
that
>>> > directory. I've searched my directory structure to determine
where
>>> this
>>> > file is being referenced in order to change it from
>>> > *met_header_columns_V9.0.txt* to *met_header_columns_V8.1.txt*.
Here
>>> is
>>> > the basic error..
>>> >
>>> > 06/01 18:01:53.787 metplus.StatAnalysis
(stat_analysis_wrapper.py:1038)
>>> > DEBUG: INTERP: BILIN
>>> > 06/01 18:01:53.787 metplus.StatAnalysis
(stat_analysis_wrapper.py:1039)
>>> > DEBUG: REGION: FULL
>>> > 06/01 18:01:53.788 metplus.StatAnalysis
(stat_analysis_wrapper.py:1040)
>>> > DEBUG: FCST_LEAD: 00
>>> > 06/01 18:01:53.788 metplus.StatAnalysis
(stat_analysis_wrapper.py:1041)
>>> > DEBUG: MODEL: AQ
>>> > 06/01 18:01:53.788 metplus.StatAnalysis
(stat_analysis_wrapper.py:1042)
>>> > DEBUG: OBTYPE: ONLYSF
>>> > 06/01 18:01:53.788 metplus.StatAnalysis
(stat_analysis_wrapper.py:1043)
>>> > DEBUG: LINE_TYPE:
>>> > 06/01 18:01:53.788 metplus.StatAnalysis
(stat_analysis_wrapper.py:1044)
>>> > DEBUG: JOB: -job filter -dump_row
>>> >
>>> >
>>>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_aq/prod/stat/aqm/20200525/grid2obs_step2/metplus_output/plot_by_valid/stat_analysis/OZCON_FULL/grid2obs/conus_sfc/AQ/valid20200525to20200525_valid000000to230000Z_init060000to060000Z/AQ_f00_fcstOZCONA1_obsCOPOA1_interpBILIN_regionFULL.stat
>>> > 06/01 18:01:53.788 metplus.StatAnalysis
(stat_analysis_wrapper.py:1045)
>>> > DEBUG: lookin directory:
>>> >
>>> >
>>>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_aq/prod/stat/aqm/20200525
>>> > 06/01 18:01:53.789 metplus.StatAnalysis (command_runner.py:117)
INFO:
>>> > app_name is: stat_analysis, output sent to:
>>> >
>>> >
>>>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_aq/prod/stat/aqm/20200525/grid2obs_step2/metplus_output/logs/master_metplus_grid2obs_conus_sfc_statanalysis_makeplots_OZCON_FULL.runon20200601180153
>>> > 06/01 18:01:53.995 metplus.StatAnalysis (command_runner.py:153)
INFO:
>>> > RUNNING: /usrx/local/dev/packages/met/8.1/bin/stat_analysis
-lookin
>>> >
>>> >
>>>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_aq/prod/stat/aqm/20200525
>>> > -config
>>> >
>>> >
>>> /gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/EMC_verif-
global/parm/metplus_config/metplus_use_cases/METplusV2.1/grid2obs/met_config/metV8.1/STATAnalysisConfig
>>> > >>
>>> >
>>> >
>>>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_aq/prod/stat/aqm/20200525/grid2obs_step2/metplus_output/logs/master_metplus_grid2obs_conus_sfc_statanalysis_makeplots_OZCON_FULL.runon20200601180153
>>> > 2>&1
>>> > 06/01 18:01:53.995 metplus.StatAnalysis (config_wrapper.py:102)
DEBUG:
>>> > Setting [config] DO_NOT_RUN_EXE to default value: False.
>>> > DEBUG 1: Default Config File:
>>> >
>>> >
>>>
/usrx/local/dev/packages/met/8.1/share/met/config/STATAnalysisConfig_default
>>> > DEBUG 1: User Config File:
>>> >
>>> >
>>> /gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/EMC_verif-
global/parm/metplus_config/metplus_use_cases/METplusV2.1/grid2obs/met_config/metV8.1/STATAnalysisConfig
>>> >
>>> >
>>> >
>>> >
>>> > *DEBUG 2: Processing 48 STAT files.ERROR :ERROR :
>>> AsciiHeader::read() ->
>>> > trouble reading file:ERROR :
>>> >
>>> >
>>>
/usrx/local/dev/packages/met/8.1/share/met/table_files/met_header_columns_V9.0.txtERROR
>>> > :*
>>> > 06/01 18:01:55.402 metplus.StatAnalysis
(stat_analysis_wrapper.py:1007)
>>> > DEBUG: STAT_ANALYSIS RUN SETTINGS....
>>> > 06/01 18:01:55.402 metplus.StatAnalysis
(stat_analysis_wrapper.py:1010)
>>> > DEBUG: FCST_INIT_END:
>>> > 06/01 18:01:55.402 metplus.StatAnalysis
(stat_analysis_wrapper.py:1010)
>>> > DEBUG: FCST_INIT_BEG:
>>> > 06/01 18:01:55.402 metplus.StatAnalysis
(stat_analysis_wrapper.py:1010)
>>> > DEBUG: FCST_INIT_HOUR: "060000"
>>> >
>>> >
>>> > The reference to this file doesn't exist in any of the files
within my
>>> > directory structure and I'm unsure how it is being referenced.
My
>>> first
>>> > thought is related to default settings, but I can't seem to
determine
>>> > whether that's the case.
>>> >
>>> > Just in case, my run script is here:
>>> >
>>> >
>>> */gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/EMC_verif-
global/scripts/job1*
>>> > my machine.conf is here:
>>> >
>>> >
>>> */gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/EMC_verif-
global/parm/metplus_config/machine.conf*
>>> > my plot_grid2obs_conus_sfc.conf is here:
>>> >
>>> >
>>> */gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/EMC_verif-
global/parm/plotting/examples/plot_grid2obs_conus_sfc.conf*
>>> > and my log is here:
>>> >
>>> >
>>>
*/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_aq/prod/stat/aqm/20200525/grid2obs_step2/metplus_output/logs/*
>>> >
>>> > Again, I don't think this is a MakePlots issue, rather it seems
to be a
>>> > default fallback. I just can't find the reference to this in
any of
>>> the
>>> > files. Why would it assume V9.0 when it is V8.1. Any thoughts
on
>>> this is
>>> > greatly appreciated.
>>> > --
>>> > Edward Strobach
>>> > EMC/NCEP/NWS/
>>> > IMSG Contractor
>>> > Cubicle#: 2029
>>> > 301-683-3717
>>> >
>>> >
>>>
>>>
>>
>> --
>> Edward Strobach
>> EMC/NCEP/NWS/
>> IMSG Contractor
>> Cubicle#: 2029
>> 301-683-3717
>>
>
>
> --
> Edward Strobach
> EMC/NCEP/NWS/
> IMSG Contractor
> Cubicle#: 2029
> 301-683-3717
>
--
Edward Strobach
EMC/NCEP/NWS/
IMSG Contractor
Cubicle#: 2029
301-683-3717
------------------------------------------------
Subject: references to a file that doesn't exist
From: George McCabe
Time: Tue Jun 02 07:55:57 2020
Hi Edward,
I have not used that tool very much, but as I understand it you are
correct
that you should use LOOP_LIST_ITEMS and GROUP_LIST_ITEMS.
See
https://github.com/NCAR/METplus/blob/master_v3.0/parm/use_cases/met_tool_wrapper/StatAnalysis/StatAnalysis.conf
In this example, we have
...
MODEL_LIST = {MODEL1}
...
FCST_VALID_HOUR_LIST = 12
FCST_INIT_HOUR_LIST = 00, 12
...
GROUP_LIST_ITEMS = FCST_INIT_HOUR_LIST
LOOP_LIST_ITEMS = FCST_VALID_HOUR_LIST, MODEL_LIST
You list the _LIST variable names in the group or loop item lists. I
was
under the impression that if a _LIST value only had 1 item, then you
didn't
have to mention it in the _LIST_ITEMS section, but I may be wrong.
Mallory
wrote these tools so she would be the best resource with any questions
about it.
Thanks,
George
On Tue, Jun 2, 2020 at 7:41 AM Edward Strobach - NOAA Affiliate via RT
<
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95448 >
>
> Perhaps through LOOP_LIST_ITEMS and GROUP_LIST_ITEMS?
>
> On Tue, Jun 2, 2020 at 9:31 AM Edward Strobach - NOAA Affiliate <
> edward.strobach at noaa.gov> wrote:
>
> > As I make these changes, I did have one follow question:
> > Since VALID_HOUR_METHOD and INIT_HOUR_METHOD is no longer used,
what
> > should I do to specify "LOOP" v. "GROUP". What I was doing before
was
> > grouping valid times for each initial time. I see that
VALID_HOUR_BEG,
> > INIT_HOUR_BEG, etc. has been replaced in list form (e.g.
> > FCST_VALID_HOUR_LIST and FCST_INIT_HOUR_LIST). I don't see how I
can
> > "LOOP" v. "GROUP" this way.
> >
> > On Tue, Jun 2, 2020 at 8:33 AM Edward Strobach - NOAA Affiliate <
> > edward.strobach at noaa.gov> wrote:
> >
> >> Oh...that's a problem. I broke the process into parts:
> >>
> >> The first part takes the air quality data, converts prepbufr to
netcdf
> >> and generates .stat files. The process list is as follows:
PB2NC,
> >> PointStat, StatAnalysis. This was done with Met 9.0/Metplus3.0.
> >>
> >> The second step was to take the .stat files, run them through
stat
> >> analysis again along with MakePlots. The process list is as
follows:
> >> StatAnalysis, MakePlots. This was done with Met8.1/Metplus2.1.
I
> guess
> >> what I'll want to do is convert the second step over to Met
> 9.0/Metplus3.0
> >> in order to make it work. That's probably how I need to handle
this
> >> problem.
> >>
> >> On Tue, Jun 2, 2020 at 12:53 AM John Halley Gotway via RT <
> >> met_help at ucar.edu> wrote:
> >>
> >>> Ed,
> >>>
> >>> Whenever we release a new version of the MET software, we add a
new
> file
> >>> named "met_header_columns_VX.Y.txt" where X.Y is the new version
> number.
> >>> That file defines the header columns for all of the STAT line
types,
> and
> >>> Stat-Analysis uses that file to figure out how to parse the
ASCII data
> in
> >>> the .stat files.
> >>>
> >>> It stands to reason that met-8.1 would include a file for
> >>> met_header_columns_V8.1.txt but would NOT include
> >>> met_header_columns_V9.0.txt. That is to say that while met-9.0
CAN
> reads
> >>> met-8.1 output just fine, met-8.1 CANNOT read met-9.0 output.
And I
> >>> suspect
> >>> that's what's really going on here.
> >>>
> >>> Look very closely at the .stat files that are being passed to
the
> met-8.1
> >>> version of stat_analysis. The first column of those .stat files
> indicates
> >>> the MET version number. And I suspect you'll find "V9.0" listed
in at
> >>> least
> >>> one input line.
> >>>
> >>> Hope that helps.
> >>>
> >>> John Halley Gotway
> >>>
> >>> On Mon, Jun 1, 2020 at 12:55 PM Edward Strobach - NOAA Affiliate
via
> RT <
> >>> met_help at ucar.edu> wrote:
> >>>
> >>> >
> >>> > Mon Jun 01 12:54:35 2020: Request 95448 was acted upon.
> >>> > Transaction: Ticket created by edward.strobach at noaa.gov
> >>> > Queue: met_help
> >>> > Subject: references to a file that doesn't exist
> >>> > Owner: Nobody
> >>> > Requestors: edward.strobach at noaa.gov
> >>> > Status: new
> >>> > Ticket <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95448
> >>> >
> >>> >
> >>> >
> >>> > Good afternoon,
> >>> >
> >>> > I've slowly been setting up my configure file to run
StatAnalysis,
> >>> > MakePlots.
> >>> >
> >>> > Everything is pretty well in terms of understanding where
things go;
> >>> > however, there is a reference to a file which doesn't exist in
that
> >>> > directory. I've searched my directory structure to determine
where
> >>> this
> >>> > file is being referenced in order to change it from
> >>> > *met_header_columns_V9.0.txt* to
*met_header_columns_V8.1.txt*. Here
> >>> is
> >>> > the basic error..
> >>> >
> >>> > 06/01 18:01:53.787 metplus.StatAnalysis
> (stat_analysis_wrapper.py:1038)
> >>> > DEBUG: INTERP: BILIN
> >>> > 06/01 18:01:53.787 metplus.StatAnalysis
> (stat_analysis_wrapper.py:1039)
> >>> > DEBUG: REGION: FULL
> >>> > 06/01 18:01:53.788 metplus.StatAnalysis
> (stat_analysis_wrapper.py:1040)
> >>> > DEBUG: FCST_LEAD: 00
> >>> > 06/01 18:01:53.788 metplus.StatAnalysis
> (stat_analysis_wrapper.py:1041)
> >>> > DEBUG: MODEL: AQ
> >>> > 06/01 18:01:53.788 metplus.StatAnalysis
> (stat_analysis_wrapper.py:1042)
> >>> > DEBUG: OBTYPE: ONLYSF
> >>> > 06/01 18:01:53.788 metplus.StatAnalysis
> (stat_analysis_wrapper.py:1043)
> >>> > DEBUG: LINE_TYPE:
> >>> > 06/01 18:01:53.788 metplus.StatAnalysis
> (stat_analysis_wrapper.py:1044)
> >>> > DEBUG: JOB: -job filter -dump_row
> >>> >
> >>> >
> >>>
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_aq/prod/stat/aqm/20200525/grid2obs_step2/metplus_output/plot_by_valid/stat_analysis/OZCON_FULL/grid2obs/conus_sfc/AQ/valid20200525to20200525_valid000000to230000Z_init060000to060000Z/AQ_f00_fcstOZCONA1_obsCOPOA1_interpBILIN_regionFULL.stat
> >>> > 06/01 18:01:53.788 metplus.StatAnalysis
> (stat_analysis_wrapper.py:1045)
> >>> > DEBUG: lookin directory:
> >>> >
> >>> >
> >>>
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_aq/prod/stat/aqm/20200525
> >>> > 06/01 18:01:53.789 metplus.StatAnalysis
(command_runner.py:117) INFO:
> >>> > app_name is: stat_analysis, output sent to:
> >>> >
> >>> >
> >>>
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_aq/prod/stat/aqm/20200525/grid2obs_step2/metplus_output/logs/master_metplus_grid2obs_conus_sfc_statanalysis_makeplots_OZCON_FULL.runon20200601180153
> >>> > 06/01 18:01:53.995 metplus.StatAnalysis
(command_runner.py:153) INFO:
> >>> > RUNNING: /usrx/local/dev/packages/met/8.1/bin/stat_analysis
-lookin
> >>> >
> >>> >
> >>>
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_aq/prod/stat/aqm/20200525
> >>> > -config
> >>> >
> >>> >
> >>>
> /gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/EMC_verif-
global/parm/metplus_config/metplus_use_cases/METplusV2.1/grid2obs/met_config/metV8.1/STATAnalysisConfig
> >>> > >>
> >>> >
> >>> >
> >>>
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_aq/prod/stat/aqm/20200525/grid2obs_step2/metplus_output/logs/master_metplus_grid2obs_conus_sfc_statanalysis_makeplots_OZCON_FULL.runon20200601180153
> >>> > 2>&1
> >>> > 06/01 18:01:53.995 metplus.StatAnalysis
(config_wrapper.py:102)
> DEBUG:
> >>> > Setting [config] DO_NOT_RUN_EXE to default value: False.
> >>> > DEBUG 1: Default Config File:
> >>> >
> >>> >
> >>>
>
/usrx/local/dev/packages/met/8.1/share/met/config/STATAnalysisConfig_default
> >>> > DEBUG 1: User Config File:
> >>> >
> >>> >
> >>>
> /gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/EMC_verif-
global/parm/metplus_config/metplus_use_cases/METplusV2.1/grid2obs/met_config/metV8.1/STATAnalysisConfig
> >>> >
> >>> >
> >>> >
> >>> >
> >>> > *DEBUG 2: Processing 48 STAT files.ERROR :ERROR :
> >>> AsciiHeader::read() ->
> >>> > trouble reading file:ERROR :
> >>> >
> >>> >
> >>>
>
/usrx/local/dev/packages/met/8.1/share/met/table_files/met_header_columns_V9.0.txtERROR
> >>> > :*
> >>> > 06/01 18:01:55.402 metplus.StatAnalysis
> (stat_analysis_wrapper.py:1007)
> >>> > DEBUG: STAT_ANALYSIS RUN SETTINGS....
> >>> > 06/01 18:01:55.402 metplus.StatAnalysis
> (stat_analysis_wrapper.py:1010)
> >>> > DEBUG: FCST_INIT_END:
> >>> > 06/01 18:01:55.402 metplus.StatAnalysis
> (stat_analysis_wrapper.py:1010)
> >>> > DEBUG: FCST_INIT_BEG:
> >>> > 06/01 18:01:55.402 metplus.StatAnalysis
> (stat_analysis_wrapper.py:1010)
> >>> > DEBUG: FCST_INIT_HOUR: "060000"
> >>> >
> >>> >
> >>> > The reference to this file doesn't exist in any of the files
within
> my
> >>> > directory structure and I'm unsure how it is being referenced.
My
> >>> first
> >>> > thought is related to default settings, but I can't seem to
determine
> >>> > whether that's the case.
> >>> >
> >>> > Just in case, my run script is here:
> >>> >
> >>> >
> >>>
> */gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/EMC_verif-
global/scripts/job1*
> >>> > my machine.conf is here:
> >>> >
> >>> >
> >>>
> */gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/EMC_verif-
global/parm/metplus_config/machine.conf*
> >>> > my plot_grid2obs_conus_sfc.conf is here:
> >>> >
> >>> >
> >>>
> */gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/EMC_verif-
global/parm/plotting/examples/plot_grid2obs_conus_sfc.conf*
> >>> > and my log is here:
> >>> >
> >>> >
> >>>
>
*/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_aq/prod/stat/aqm/20200525/grid2obs_step2/metplus_output/logs/*
> >>> >
> >>> > Again, I don't think this is a MakePlots issue, rather it
seems to
> be a
> >>> > default fallback. I just can't find the reference to this in
any of
> >>> the
> >>> > files. Why would it assume V9.0 when it is V8.1. Any
thoughts on
> >>> this is
> >>> > greatly appreciated.
> >>> > --
> >>> > Edward Strobach
> >>> > EMC/NCEP/NWS/
> >>> > IMSG Contractor
> >>> > Cubicle#: 2029
> >>> > 301-683-3717
> >>> >
> >>> >
> >>>
> >>>
> >>
> >> --
> >> Edward Strobach
> >> EMC/NCEP/NWS/
> >> IMSG Contractor
> >> Cubicle#: 2029
> >> 301-683-3717
> >>
> >
> >
> > --
> > Edward Strobach
> > EMC/NCEP/NWS/
> > IMSG Contractor
> > Cubicle#: 2029
> > 301-683-3717
> >
>
>
> --
> Edward Strobach
> EMC/NCEP/NWS/
> IMSG Contractor
> Cubicle#: 2029
> 301-683-3717
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: references to a file that doesn't exist
From: Edward Strobach - NOAA Affiliate
Time: Tue Jun 02 08:01:25 2020
Thanks for the clarification. That makes a lot more sense. It also
seems
like best practice to be consistent with the met/metplus version for
all
*related* processes tasks, even when separated.
On Tue, Jun 2, 2020 at 9:56 AM George McCabe via RT
<met_help at ucar.edu>
wrote:
> Hi Edward,
>
> I have not used that tool very much, but as I understand it you are
correct
> that you should use LOOP_LIST_ITEMS and GROUP_LIST_ITEMS.
>
> See
>
>
https://github.com/NCAR/METplus/blob/master_v3.0/parm/use_cases/met_tool_wrapper/StatAnalysis/StatAnalysis.conf
>
> In this example, we have
>
> ...
> MODEL_LIST = {MODEL1}
> ...
> FCST_VALID_HOUR_LIST = 12
> FCST_INIT_HOUR_LIST = 00, 12
> ...
> GROUP_LIST_ITEMS = FCST_INIT_HOUR_LIST
> LOOP_LIST_ITEMS = FCST_VALID_HOUR_LIST, MODEL_LIST
>
> You list the _LIST variable names in the group or loop item lists. I
was
> under the impression that if a _LIST value only had 1 item, then you
didn't
> have to mention it in the _LIST_ITEMS section, but I may be wrong.
Mallory
> wrote these tools so she would be the best resource with any
questions
> about it.
>
> Thanks,
> George
>
> On Tue, Jun 2, 2020 at 7:41 AM Edward Strobach - NOAA Affiliate via
RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95448 >
> >
> > Perhaps through LOOP_LIST_ITEMS and GROUP_LIST_ITEMS?
> >
> > On Tue, Jun 2, 2020 at 9:31 AM Edward Strobach - NOAA Affiliate <
> > edward.strobach at noaa.gov> wrote:
> >
> > > As I make these changes, I did have one follow question:
> > > Since VALID_HOUR_METHOD and INIT_HOUR_METHOD is no longer used,
what
> > > should I do to specify "LOOP" v. "GROUP". What I was doing
before was
> > > grouping valid times for each initial time. I see that
VALID_HOUR_BEG,
> > > INIT_HOUR_BEG, etc. has been replaced in list form (e.g.
> > > FCST_VALID_HOUR_LIST and FCST_INIT_HOUR_LIST). I don't see how
I can
> > > "LOOP" v. "GROUP" this way.
> > >
> > > On Tue, Jun 2, 2020 at 8:33 AM Edward Strobach - NOAA Affiliate
<
> > > edward.strobach at noaa.gov> wrote:
> > >
> > >> Oh...that's a problem. I broke the process into parts:
> > >>
> > >> The first part takes the air quality data, converts prepbufr to
netcdf
> > >> and generates .stat files. The process list is as follows:
PB2NC,
> > >> PointStat, StatAnalysis. This was done with Met
9.0/Metplus3.0.
> > >>
> > >> The second step was to take the .stat files, run them through
stat
> > >> analysis again along with MakePlots. The process list is as
follows:
> > >> StatAnalysis, MakePlots. This was done with
Met8.1/Metplus2.1. I
> > guess
> > >> what I'll want to do is convert the second step over to Met
> > 9.0/Metplus3.0
> > >> in order to make it work. That's probably how I need to handle
this
> > >> problem.
> > >>
> > >> On Tue, Jun 2, 2020 at 12:53 AM John Halley Gotway via RT <
> > >> met_help at ucar.edu> wrote:
> > >>
> > >>> Ed,
> > >>>
> > >>> Whenever we release a new version of the MET software, we add
a new
> > file
> > >>> named "met_header_columns_VX.Y.txt" where X.Y is the new
version
> > number.
> > >>> That file defines the header columns for all of the STAT line
types,
> > and
> > >>> Stat-Analysis uses that file to figure out how to parse the
ASCII
> data
> > in
> > >>> the .stat files.
> > >>>
> > >>> It stands to reason that met-8.1 would include a file for
> > >>> met_header_columns_V8.1.txt but would NOT include
> > >>> met_header_columns_V9.0.txt. That is to say that while met-9.0
CAN
> > reads
> > >>> met-8.1 output just fine, met-8.1 CANNOT read met-9.0 output.
And I
> > >>> suspect
> > >>> that's what's really going on here.
> > >>>
> > >>> Look very closely at the .stat files that are being passed to
the
> > met-8.1
> > >>> version of stat_analysis. The first column of those .stat
files
> > indicates
> > >>> the MET version number. And I suspect you'll find "V9.0"
listed in at
> > >>> least
> > >>> one input line.
> > >>>
> > >>> Hope that helps.
> > >>>
> > >>> John Halley Gotway
> > >>>
> > >>> On Mon, Jun 1, 2020 at 12:55 PM Edward Strobach - NOAA
Affiliate via
> > RT <
> > >>> met_help at ucar.edu> wrote:
> > >>>
> > >>> >
> > >>> > Mon Jun 01 12:54:35 2020: Request 95448 was acted upon.
> > >>> > Transaction: Ticket created by edward.strobach at noaa.gov
> > >>> > Queue: met_help
> > >>> > Subject: references to a file that doesn't exist
> > >>> > Owner: Nobody
> > >>> > Requestors: edward.strobach at noaa.gov
> > >>> > Status: new
> > >>> > Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95448
> > >>> >
> > >>> >
> > >>> >
> > >>> > Good afternoon,
> > >>> >
> > >>> > I've slowly been setting up my configure file to run
StatAnalysis,
> > >>> > MakePlots.
> > >>> >
> > >>> > Everything is pretty well in terms of understanding where
things
> go;
> > >>> > however, there is a reference to a file which doesn't exist
in that
> > >>> > directory. I've searched my directory structure to
determine where
> > >>> this
> > >>> > file is being referenced in order to change it from
> > >>> > *met_header_columns_V9.0.txt* to
*met_header_columns_V8.1.txt*.
> Here
> > >>> is
> > >>> > the basic error..
> > >>> >
> > >>> > 06/01 18:01:53.787 metplus.StatAnalysis
> > (stat_analysis_wrapper.py:1038)
> > >>> > DEBUG: INTERP: BILIN
> > >>> > 06/01 18:01:53.787 metplus.StatAnalysis
> > (stat_analysis_wrapper.py:1039)
> > >>> > DEBUG: REGION: FULL
> > >>> > 06/01 18:01:53.788 metplus.StatAnalysis
> > (stat_analysis_wrapper.py:1040)
> > >>> > DEBUG: FCST_LEAD: 00
> > >>> > 06/01 18:01:53.788 metplus.StatAnalysis
> > (stat_analysis_wrapper.py:1041)
> > >>> > DEBUG: MODEL: AQ
> > >>> > 06/01 18:01:53.788 metplus.StatAnalysis
> > (stat_analysis_wrapper.py:1042)
> > >>> > DEBUG: OBTYPE: ONLYSF
> > >>> > 06/01 18:01:53.788 metplus.StatAnalysis
> > (stat_analysis_wrapper.py:1043)
> > >>> > DEBUG: LINE_TYPE:
> > >>> > 06/01 18:01:53.788 metplus.StatAnalysis
> > (stat_analysis_wrapper.py:1044)
> > >>> > DEBUG: JOB: -job filter -dump_row
> > >>> >
> > >>> >
> > >>>
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_aq/prod/stat/aqm/20200525/grid2obs_step2/metplus_output/plot_by_valid/stat_analysis/OZCON_FULL/grid2obs/conus_sfc/AQ/valid20200525to20200525_valid000000to230000Z_init060000to060000Z/AQ_f00_fcstOZCONA1_obsCOPOA1_interpBILIN_regionFULL.stat
> > >>> > 06/01 18:01:53.788 metplus.StatAnalysis
> > (stat_analysis_wrapper.py:1045)
> > >>> > DEBUG: lookin directory:
> > >>> >
> > >>> >
> > >>>
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_aq/prod/stat/aqm/20200525
> > >>> > 06/01 18:01:53.789 metplus.StatAnalysis
(command_runner.py:117)
> INFO:
> > >>> > app_name is: stat_analysis, output sent to:
> > >>> >
> > >>> >
> > >>>
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_aq/prod/stat/aqm/20200525/grid2obs_step2/metplus_output/logs/master_metplus_grid2obs_conus_sfc_statanalysis_makeplots_OZCON_FULL.runon20200601180153
> > >>> > 06/01 18:01:53.995 metplus.StatAnalysis
(command_runner.py:153)
> INFO:
> > >>> > RUNNING: /usrx/local/dev/packages/met/8.1/bin/stat_analysis
-lookin
> > >>> >
> > >>> >
> > >>>
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_aq/prod/stat/aqm/20200525
> > >>> > -config
> > >>> >
> > >>> >
> > >>>
> >
> /gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/EMC_verif-
global/parm/metplus_config/metplus_use_cases/METplusV2.1/grid2obs/met_config/metV8.1/STATAnalysisConfig
> > >>> > >>
> > >>> >
> > >>> >
> > >>>
> >
>
/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_aq/prod/stat/aqm/20200525/grid2obs_step2/metplus_output/logs/master_metplus_grid2obs_conus_sfc_statanalysis_makeplots_OZCON_FULL.runon20200601180153
> > >>> > 2>&1
> > >>> > 06/01 18:01:53.995 metplus.StatAnalysis
(config_wrapper.py:102)
> > DEBUG:
> > >>> > Setting [config] DO_NOT_RUN_EXE to default value: False.
> > >>> > DEBUG 1: Default Config File:
> > >>> >
> > >>> >
> > >>>
> >
>
/usrx/local/dev/packages/met/8.1/share/met/config/STATAnalysisConfig_default
> > >>> > DEBUG 1: User Config File:
> > >>> >
> > >>> >
> > >>>
> >
> /gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/EMC_verif-
global/parm/metplus_config/metplus_use_cases/METplusV2.1/grid2obs/met_config/metV8.1/STATAnalysisConfig
> > >>> >
> > >>> >
> > >>> >
> > >>> >
> > >>> > *DEBUG 2: Processing 48 STAT files.ERROR :ERROR :
> > >>> AsciiHeader::read() ->
> > >>> > trouble reading file:ERROR :
> > >>> >
> > >>> >
> > >>>
> >
>
/usrx/local/dev/packages/met/8.1/share/met/table_files/met_header_columns_V9.0.txtERROR
> > >>> > :*
> > >>> > 06/01 18:01:55.402 metplus.StatAnalysis
> > (stat_analysis_wrapper.py:1007)
> > >>> > DEBUG: STAT_ANALYSIS RUN SETTINGS....
> > >>> > 06/01 18:01:55.402 metplus.StatAnalysis
> > (stat_analysis_wrapper.py:1010)
> > >>> > DEBUG: FCST_INIT_END:
> > >>> > 06/01 18:01:55.402 metplus.StatAnalysis
> > (stat_analysis_wrapper.py:1010)
> > >>> > DEBUG: FCST_INIT_BEG:
> > >>> > 06/01 18:01:55.402 metplus.StatAnalysis
> > (stat_analysis_wrapper.py:1010)
> > >>> > DEBUG: FCST_INIT_HOUR: "060000"
> > >>> >
> > >>> >
> > >>> > The reference to this file doesn't exist in any of the files
within
> > my
> > >>> > directory structure and I'm unsure how it is being
referenced. My
> > >>> first
> > >>> > thought is related to default settings, but I can't seem to
> determine
> > >>> > whether that's the case.
> > >>> >
> > >>> > Just in case, my run script is here:
> > >>> >
> > >>> >
> > >>>
> >
> */gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/EMC_verif-
global/scripts/job1*
> > >>> > my machine.conf is here:
> > >>> >
> > >>> >
> > >>>
> >
> */gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/EMC_verif-
global/parm/metplus_config/machine.conf*
> > >>> > my plot_grid2obs_conus_sfc.conf is here:
> > >>> >
> > >>> >
> > >>>
> >
> */gpfs/dell2/emc/modeling/save/Edward.Strobach/MetPlus/EMC_verif-
global/parm/plotting/examples/plot_grid2obs_conus_sfc.conf*
> > >>> > and my log is here:
> > >>> >
> > >>> >
> > >>>
> >
>
*/gpfs/dell2/emc/modeling/noscrub/Edward.Strobach/metplus_aq/prod/stat/aqm/20200525/grid2obs_step2/metplus_output/logs/*
> > >>> >
> > >>> > Again, I don't think this is a MakePlots issue, rather it
seems to
> > be a
> > >>> > default fallback. I just can't find the reference to this
in any
> of
> > >>> the
> > >>> > files. Why would it assume V9.0 when it is V8.1. Any
thoughts on
> > >>> this is
> > >>> > greatly appreciated.
> > >>> > --
> > >>> > Edward Strobach
> > >>> > EMC/NCEP/NWS/
> > >>> > IMSG Contractor
> > >>> > Cubicle#: 2029
> > >>> > 301-683-3717
> > >>> >
> > >>> >
> > >>>
> > >>>
> > >>
> > >> --
> > >> Edward Strobach
> > >> EMC/NCEP/NWS/
> > >> IMSG Contractor
> > >> Cubicle#: 2029
> > >> 301-683-3717
> > >>
> > >
> > >
> > > --
> > > Edward Strobach
> > > EMC/NCEP/NWS/
> > > IMSG Contractor
> > > Cubicle#: 2029
> > > 301-683-3717
> > >
> >
> >
> > --
> > Edward Strobach
> > EMC/NCEP/NWS/
> > IMSG Contractor
> > Cubicle#: 2029
> > 301-683-3717
> >
> >
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>
--
Edward Strobach
EMC/NCEP/NWS/
IMSG Contractor
Cubicle#: 2029
301-683-3717
------------------------------------------------
More information about the Met_help
mailing list