[Met_help] [rt.rap.ucar.edu #89789] History for Using FCST_PCP_COMBINE in METplus
George McCabe via RT
met_help at ucar.edu
Tue Jun 4 11:17:30 MDT 2019
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hello. I haven't been able to get the FCST_PCP_COMBINE feature to work
in METplus, running out of ideas and hope you can help.
I've put some examples in
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/ (current test
directory: gyre:/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/test).
test_fv3gfs_24h.ksh is the simple test script. The goal is to compute a
24h verification ending at 12Z 20190415.
'test1' did not need FCST_PCP_COMBINE - the config file
fv3gfs_24h.conf.test1 simply had LEAD_SEQ set to '24',
FCST_PCP_COMBINE_RUN was not set. The FV3GFS had two precip buckets, 6h
and continuously-accumulating, and METplus easily found the 24h forecast
accumulation and went on with the grid_stat.
'test2' had 'LEAD_SEQ = 36', and I added the following in the config
(https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test2):
FCST_PCP_COMBINE_RUN = True
FCST_PCP_COMBINE_INPUT_DATATYPE = GRIB
FCST_PCP_COMBINE_INPUT_DIR = /gpfs/dell1/nco/ops/com/gfs/para
FCST_PCP_COMBINE_OUTPUT_DIR = {OUTPUT_BASE}/qpf_grib/bucket
FCST_GRID_STAT_INPUT_DIR = {OUTPUT_BASE}/qpf_grib/bucket
FCST_MAX_FORECAST = 36
FCST_INIT_INTERVAL = 6
FCST_LEVEL = 36
Here the goal was to see if I could get METplus to subtract the 00-12h
accumulation from gfs.t00z.pgrb2.0p25.f012 from the 00-36h accumulation
in gfs.t00z.pgrb2.0p25.f036. Thinking that FCST_LEVEL is the forecast
bucket length, I figured setting it to '36' and artificially specifying
FCST_MAX_FORECAST = 36 (instead of the full 384h) should work for this
limited test where LEAD_SEQ is set to 36 (is there a way to specify that
the bucket never gets emptied?). It failed with the follow err msg
(full message in
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test2):
04/16 20:01:42.754 jlog (master_metplus.py:217) CRITICAL:
master_metplus failed: ''
Traceback (most recent call last):
File "/meso/save/Ying.Lin/metplus/ush/master_metplus.py", line 213,
in <module>
main()
File "/meso/save/Ying.Lin/metplus/ush/master_metplus.py", line 170,
in main
process.run_at_time(-1, run_time)
File
"/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/ush/reformat_gridded_wrapper.py",
line 80, in run_at_time
self.run_at_time_once(task_info, var_info, rl)
File
"/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/ush/pcp_combine_wrapper.py",
line 419, in run_at_time_once
....
'test3' is an attempt from a different angle, thinking that if the
'continuously running' bucket didn't work, maybe the 6-hourly bucket
would. In
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test3,
I changed LEAD_SEQ to 24, and set FCST_LEVEL = 6. It produced err msgs
(https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test3)
that appeared to be the same as that from 'test2'.
Earlier on I tried using the FCST_PCP_COMBINE feature with the NAM
'CONUSNEST' output (with simple regular 6-hourly bucket) and got the
same err msg, so I don't think FV3GFS's dual-bucket system is the cause.
Also posted on
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/ are the two
metplus_final.conf files from both 'test2' and 'test3'.
I found in
/global/noscrub/Julie.Prestopnik/METplus/METplus-2.0.4/parm/use_cases/qpf/examples/
that hrefmean-vs-mrms-qpe.conf and nationalblend-vs-mrms-qpe.conf had
"FCST_PCP_COMBINE_RUN = True", that's where I borrowed the
FCST_PCP_COMBINE_* set up in my fv3gfs_24h.conf.test2 and
fv3gfs_24h.conf.test3. Haven't been able to run the hrefmean and
nationalblend tests myself however - got err msg of " CRITICAL:
master_metplus failed: No option 'GEMPAKTOCF_CLASSPATH' in section:
'exe'" from both attempts.
Thank you for looking into this -
Ying
--
Ying Lin
NCEP/EMC/Verification, Post-processing and Product Generation Branch
NCWCP Cubicle No. 2015
Ying.Lin at noaa.gov
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Using FCST_PCP_COMBINE in METplus
From: George McCabe
Time: Wed Apr 17 13:32:30 2019
Hi Ying,
You will need to set FCST_PCP_COMBINE_METHOD = SUBTRACT to config
METplus
to subtract accumulations files to get the accumulation you desire.
The
following use case has an example of this configuration:
parm/use_cases/grid_to_grid/examples/precip_continous.conf
I also noticed that your config file has the following items in the
[config] section:
FCST_PCP_COMBINE_INPUT_DIR = /gpfs/dell1/nco/ops/com/gfs/para
FCST_PCP_COMBINE_OUTPUT_DIR = {OUTPUT_BASE}/qpf_grib/bucket
FCST_GRID_STAT_INPUT_DIR = {OUTPUT_BASE}/qpf_grib/bucket
You should move these to the [dir] section.
Please let me know if you continue to have issues. The error message
you
provided has more information at the bottom that was not included. The
next
release of METplus contains more extensive error checking so the error
message from that version should be much more helpful.
Thanks,
George
On Wed, Apr 17, 2019 at 12:43 PM Ying Lin via RT <met_help at ucar.edu>
wrote:
>
> Wed Apr 17 12:43:02 2019: Request 89789 was acted upon.
> Transaction: Ticket created by ying.lin at noaa.gov
> Queue: met_help
> Subject: Using FCST_PCP_COMBINE in METplus
> Owner: Nobody
> Requestors: ying.lin at noaa.gov
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789 >
>
>
> Hello. I haven't been able to get the FCST_PCP_COMBINE feature to
work
> in METplus, running out of ideas and hope you can help.
>
> I've put some examples in
> https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/ (current
test
> directory: gyre:/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/test).
> test_fv3gfs_24h.ksh is the simple test script. The goal is to
compute a
> 24h verification ending at 12Z 20190415.
>
> 'test1' did not need FCST_PCP_COMBINE - the config file
> fv3gfs_24h.conf.test1 simply had LEAD_SEQ set to '24',
> FCST_PCP_COMBINE_RUN was not set. The FV3GFS had two precip
buckets, 6h
> and continuously-accumulating, and METplus easily found the 24h
forecast
> accumulation and went on with the grid_stat.
>
> 'test2' had 'LEAD_SEQ = 36', and I added the following in the config
> (
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test2
> ):
> FCST_PCP_COMBINE_RUN = True
> FCST_PCP_COMBINE_INPUT_DATATYPE = GRIB
> FCST_PCP_COMBINE_INPUT_DIR = /gpfs/dell1/nco/ops/com/gfs/para
> FCST_PCP_COMBINE_OUTPUT_DIR = {OUTPUT_BASE}/qpf_grib/bucket
> FCST_GRID_STAT_INPUT_DIR = {OUTPUT_BASE}/qpf_grib/bucket
> FCST_MAX_FORECAST = 36
> FCST_INIT_INTERVAL = 6
> FCST_LEVEL = 36
> Here the goal was to see if I could get METplus to subtract the 00-
12h
> accumulation from gfs.t00z.pgrb2.0p25.f012 from the 00-36h
accumulation
> in gfs.t00z.pgrb2.0p25.f036. Thinking that FCST_LEVEL is the
forecast
> bucket length, I figured setting it to '36' and artificially
specifying
> FCST_MAX_FORECAST = 36 (instead of the full 384h) should work for
this
> limited test where LEAD_SEQ is set to 36 (is there a way to specify
that
> the bucket never gets emptied?). It failed with the follow err msg
> (full message in
>
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test2
> ):
>
> 04/16 20:01:42.754 jlog (master_metplus.py:217) CRITICAL:
> master_metplus failed: ''
> Traceback (most recent call last):
> File "/meso/save/Ying.Lin/metplus/ush/master_metplus.py", line
213,
> in <module>
> main()
> File "/meso/save/Ying.Lin/metplus/ush/master_metplus.py", line
170,
> in main
> process.run_at_time(-1, run_time)
> File
>
"/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/ush/reformat_gridded_wrapper.py",
>
> line 80, in run_at_time
> self.run_at_time_once(task_info, var_info, rl)
> File
>
"/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/ush/pcp_combine_wrapper.py",
> line 419, in run_at_time_once
> ....
>
> 'test3' is an attempt from a different angle, thinking that if the
> 'continuously running' bucket didn't work, maybe the 6-hourly bucket
> would. In
>
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test3,
>
> I changed LEAD_SEQ to 24, and set FCST_LEVEL = 6. It produced err
msgs
> (
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test3)
>
> that appeared to be the same as that from 'test2'.
>
> Earlier on I tried using the FCST_PCP_COMBINE feature with the NAM
> 'CONUSNEST' output (with simple regular 6-hourly bucket) and got the
> same err msg, so I don't think FV3GFS's dual-bucket system is the
cause.
>
> Also posted on
> https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/ are the
two
> metplus_final.conf files from both 'test2' and 'test3'.
>
> I found in
> /global/noscrub/Julie.Prestopnik/METplus/METplus-
2.0.4/parm/use_cases/qpf/examples/
>
> that hrefmean-vs-mrms-qpe.conf and nationalblend-vs-mrms-qpe.conf
had
> "FCST_PCP_COMBINE_RUN = True", that's where I borrowed the
> FCST_PCP_COMBINE_* set up in my fv3gfs_24h.conf.test2 and
> fv3gfs_24h.conf.test3. Haven't been able to run the hrefmean and
> nationalblend tests myself however - got err msg of " CRITICAL:
> master_metplus failed: No option 'GEMPAKTOCF_CLASSPATH' in section:
> 'exe'" from both attempts.
>
> Thank you for looking into this -
>
> Ying
>
> --
> Ying Lin
> NCEP/EMC/Verification, Post-processing and Product Generation Branch
> NCWCP Cubicle No. 2015
> Ying.Lin at noaa.gov
>
>
>
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #89789] Using FCST_PCP_COMBINE in METplus
From: Ying Lin
Time: Thu Apr 18 13:37:28 2019
Thank you very much George, this is very helpful.
About the example in
parm/use_cases/grid_to_grid/examples/precip_continuous.conf
I tried running it and it seems to have run through the pcp_combine
step
fine: it did this:
/global/noscrub/Julie.Prestopnik/met/8.0/bin/pcp_combine -subtract
/global/noscrub/Julie.Prestopnik/METplus/METplus-
2.0.4_sample_data/grid_to_grid/prfv3rt1/20180308/gfs.t00z.pgrb.1p00.f048
48
/global/noscrub/Julie.Prestopnik/METplus/METplus-
2.0.4_sample_data/grid_to_grid/prfv3rt1/20180308/gfs.t00z.pgrb.1p00.f024
24 /meso/noscrub/Ying.Lin/metplus.out/gfs/bucket/gfs.2018030900_A024h
Several questions about the example:
1) The precip_continuous.conf has
LEAD_SEQ = 24
I thought it meant forecast lead of 24h, e.g. look for 00-24h forecast
valid at 2018030900 (given in the config). If so, why doesn't it
simply use gfs.t00z.pgrb.1p00.f024, which already have the 00-24h
forecast? If it subtracts gfs.t00z.pgrb.1p00.f024 from
gfs.t00z.pgrb.1p00.f048, the resulting file would have 24-48h forecast
valid at 2018031000. I did
ncdump -h
/meso/noscrub/Ying.Lin/metplus.out/gfs/bucket/gfs.2018030900_A024h
and got
APCP_24:init_time = "20180308_000000" ;
APCP_24:init_time_ut = "1520467200" ;
APCP_24:valid_time = "20180310_000000" ;
APCP_24:valid_time_ut = "1520640000" ;
APCP_24:accum_time = "240000" ;
it seems to mean that the file contains APCP_24h with initial time of
00z 03/08 and the valid time is 00Z 03/10 (that'll be APCP_48h instead
of 24h?)
2) The precip_continuous.conf has
FCST_LEVEL = 6
I thought FCST_LEVEL is the forecast bucket length. In the sample GFS
files, there isn't a 6-hourly bucket - gfs.t00z.pgrb.1p00.f024
contains
00-24h accumulation and gfs.t00z.pgrb.1p00.f048 contains 00-48h
accumulation. Does FCST_LEVEL mean something other than the forecast
bucket length?
3) precip_continuous.conf has
PCP_COMBINE_METHOD = SUBTRACT
which should work for the example since the verifying analysis is the
24h CCPA file. But what if I need to use 'SUBTRACT' for the forecast
field but 'SUM' for the analysis field (e.g. summing up 6h CCPAs into
a
24h total)? At one point I tried
PCP_COMBINE_METHOD = SUBTRACT, SUM
and got
ERROR: Invalid PCP_COMBINE_METHOD specified
After several failed attempts getting my own config file to work, I
starting from precip_continuous.conf (which worked through the
PcpCombine step; I think that's the only step it runs since the config
has only PcpCombine in the PROCESS_LIST) and tried to set a config
file
as close to precip_continuous.conf as possible
(/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/parm/ylin_config/models/fv3gfs_24h.conf,
also uploaded to
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf).
The resulting err msg:
04/18 18:35:33.912 jlog (master_metplus.py:217) CRITICAL:
master_metplus failed: ''
Traceback (most recent call last):
File "/meso/save/Ying.Lin/metplus/ush/master_metplus.py", line 213,
in <module>
main()
File "/meso/save/Ying.Lin/metplus/ush/master_metplus.py", line 170,
in main
process.run_at_time(-1, run_time)
File
"/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/ush/reformat_gridded_wrapper.py",
line 80, in run_at_time
Full output file:
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs.out.test5;
metplus_final.conf:
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/metplus_final.conf
The quoted err msg above appears to the the same as in my email
yesterday, so there must still be something wrong with the format of
my
config, though I think I eliminated the problems you pointed out in
your
reply. There wouldn't be a ccpa_conus_1.0d_* of the valid day I
specified, but I think the processing failed way before that point.
Thank you again -
Ying
On 4/17/19 3:32 PM, George McCabe via RT wrote:
> Hi Ying,
>
> You will need to set FCST_PCP_COMBINE_METHOD = SUBTRACT to config
METplus
> to subtract accumulations files to get the accumulation you desire.
The
> following use case has an example of this configuration:
>
> parm/use_cases/grid_to_grid/examples/precip_continous.conf
>
> I also noticed that your config file has the following items in the
> [config] section:
>
> FCST_PCP_COMBINE_INPUT_DIR = /gpfs/dell1/nco/ops/com/gfs/para
> FCST_PCP_COMBINE_OUTPUT_DIR = {OUTPUT_BASE}/qpf_grib/bucket
> FCST_GRID_STAT_INPUT_DIR = {OUTPUT_BASE}/qpf_grib/bucket
>
> You should move these to the [dir] section.
>
> Please let me know if you continue to have issues. The error message
you
> provided has more information at the bottom that was not included.
The next
> release of METplus contains more extensive error checking so the
error
> message from that version should be much more helpful.
>
> Thanks,
> George
>
> On Wed, Apr 17, 2019 at 12:43 PM Ying Lin via RT <met_help at ucar.edu>
wrote:
>
>> Wed Apr 17 12:43:02 2019: Request 89789 was acted upon.
>> Transaction: Ticket created by ying.lin at noaa.gov
>> Queue: met_help
>> Subject: Using FCST_PCP_COMBINE in METplus
>> Owner: Nobody
>> Requestors: ying.lin at noaa.gov
>> Status: new
>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789 >
>>
>>
>> Hello. I haven't been able to get the FCST_PCP_COMBINE feature to
work
>> in METplus, running out of ideas and hope you can help.
>>
>> I've put some examples in
>> https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/ (current
test
>> directory: gyre:/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/test).
>> test_fv3gfs_24h.ksh is the simple test script. The goal is to
compute a
>> 24h verification ending at 12Z 20190415.
>>
>> 'test1' did not need FCST_PCP_COMBINE - the config file
>> fv3gfs_24h.conf.test1 simply had LEAD_SEQ set to '24',
>> FCST_PCP_COMBINE_RUN was not set. The FV3GFS had two precip
buckets, 6h
>> and continuously-accumulating, and METplus easily found the 24h
forecast
>> accumulation and went on with the grid_stat.
>>
>> 'test2' had 'LEAD_SEQ = 36', and I added the following in the
config
>> (
>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test2
>> ):
>> FCST_PCP_COMBINE_RUN = True
>> FCST_PCP_COMBINE_INPUT_DATATYPE = GRIB
>> FCST_PCP_COMBINE_INPUT_DIR = /gpfs/dell1/nco/ops/com/gfs/para
>> FCST_PCP_COMBINE_OUTPUT_DIR = {OUTPUT_BASE}/qpf_grib/bucket
>> FCST_GRID_STAT_INPUT_DIR = {OUTPUT_BASE}/qpf_grib/bucket
>> FCST_MAX_FORECAST = 36
>> FCST_INIT_INTERVAL = 6
>> FCST_LEVEL = 36
>> Here the goal was to see if I could get METplus to subtract the 00-
12h
>> accumulation from gfs.t00z.pgrb2.0p25.f012 from the 00-36h
accumulation
>> in gfs.t00z.pgrb2.0p25.f036. Thinking that FCST_LEVEL is the
forecast
>> bucket length, I figured setting it to '36' and artificially
specifying
>> FCST_MAX_FORECAST = 36 (instead of the full 384h) should work for
this
>> limited test where LEAD_SEQ is set to 36 (is there a way to specify
that
>> the bucket never gets emptied?). It failed with the follow err msg
>> (full message in
>>
>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test2
>> ):
>>
>> 04/16 20:01:42.754 jlog (master_metplus.py:217) CRITICAL:
>> master_metplus failed: ''
>> Traceback (most recent call last):
>> File "/meso/save/Ying.Lin/metplus/ush/master_metplus.py", line
213,
>> in <module>
>> main()
>> File "/meso/save/Ying.Lin/metplus/ush/master_metplus.py", line
170,
>> in main
>> process.run_at_time(-1, run_time)
>> File
>>
"/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/ush/reformat_gridded_wrapper.py",
>>
>> line 80, in run_at_time
>> self.run_at_time_once(task_info, var_info, rl)
>> File
>>
"/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/ush/pcp_combine_wrapper.py",
>> line 419, in run_at_time_once
>> ....
>>
>> 'test3' is an attempt from a different angle, thinking that if the
>> 'continuously running' bucket didn't work, maybe the 6-hourly
bucket
>> would. In
>>
>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test3,
>>
>> I changed LEAD_SEQ to 24, and set FCST_LEVEL = 6. It produced err
msgs
>> (
>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test3)
>>
>> that appeared to be the same as that from 'test2'.
>>
>> Earlier on I tried using the FCST_PCP_COMBINE feature with the NAM
>> 'CONUSNEST' output (with simple regular 6-hourly bucket) and got
the
>> same err msg, so I don't think FV3GFS's dual-bucket system is the
cause.
>>
>> Also posted on
>> https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/ are the
two
>> metplus_final.conf files from both 'test2' and 'test3'.
>>
>> I found in
>> /global/noscrub/Julie.Prestopnik/METplus/METplus-
2.0.4/parm/use_cases/qpf/examples/
>>
>> that hrefmean-vs-mrms-qpe.conf and nationalblend-vs-mrms-qpe.conf
had
>> "FCST_PCP_COMBINE_RUN = True", that's where I borrowed the
>> FCST_PCP_COMBINE_* set up in my fv3gfs_24h.conf.test2 and
>> fv3gfs_24h.conf.test3. Haven't been able to run the hrefmean and
>> nationalblend tests myself however - got err msg of " CRITICAL:
>> master_metplus failed: No option 'GEMPAKTOCF_CLASSPATH' in
section:
>> 'exe'" from both attempts.
>>
>> Thank you for looking into this -
>>
>> Ying
>>
>> --
>> Ying Lin
>> NCEP/EMC/Verification, Post-processing and Product Generation
Branch
>> NCWCP Cubicle No. 2015
>> Ying.Lin at noaa.gov
>>
>>
>>
>>
--
Ying Lin
NCEP/EMC/Verification, Post-processing and Product Generation Branch
NCWCP Cubicle No. 2015
Ying.Lin at noaa.gov
------------------------------------------------
Subject: Using FCST_PCP_COMBINE in METplus
From: George McCabe
Time: Thu Apr 18 13:51:50 2019
Hi Ying,
1) The config file I see has a valid time of 2018031000 and 48 hr
forecast
lead. You may be using an old version of METplus that had this set up
incorrectly. METplus version 2.0.4 should be correct.
2) pcp_combine -subtract does not actually use the FCST_LEVEL
variable, so
that should be removed from the use case. The code takes the valid
time and
subtracts the accumulation.
3) In the upcoming version of METplus (in a few weeks!) has a good
number
of config changes from the previous versions. For example, FCST_LEVEL
will
now be FCST_PCP_COMBINE_INPUT_LEVEL. These changes are designed to
make the
variables more clear to what they actually do. In the new version,
there is
now FCST_PCP_COMBINE_METHOD and OBS_PCP_COMBINE_METHOD, so you will be
able
to specify a different method for your forecast and analysis data.
FCST_PCP_COMBINE_RUN = True
FCST_PCP_COMBINE_METHOD = SUBTRACT
OBS_PCP_COMBINE_RUN = True
OBS_PCP_COMBINE_METHOD = SUM
This functionality has not been thoroughly tested as we haven't had
any
example use cases that need to do this so far, so please let me know
if it
does not work the way you expect and I will fix it.
Just to let you know, I will be out of the office tomorrow (Friday)
and I
wont have access to email. Feel free to send any follow-up emails to
me and
I will get back to you on Monday.
Thanks,
George
On Thu, Apr 18, 2019 at 7:38 PM Ying Lin via RT <met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789 >
>
> Thank you very much George, this is very helpful.
>
> About the example in
> parm/use_cases/grid_to_grid/examples/precip_continuous.conf
> I tried running it and it seems to have run through the pcp_combine
step
> fine: it did this:
> /global/noscrub/Julie.Prestopnik/met/8.0/bin/pcp_combine
-subtract
> /global/noscrub/Julie.Prestopnik/METplus/METplus-
2.0.4_sample_data/grid_to_grid/prfv3rt1/20180308/gfs.t00z.pgrb.1p00.f048
>
> 48
> /global/noscrub/Julie.Prestopnik/METplus/METplus-
2.0.4_sample_data/grid_to_grid/prfv3rt1/20180308/gfs.t00z.pgrb.1p00.f024
>
> 24
/meso/noscrub/Ying.Lin/metplus.out/gfs/bucket/gfs.2018030900_A024h
>
> Several questions about the example:
> 1) The precip_continuous.conf has
> LEAD_SEQ = 24
> I thought it meant forecast lead of 24h, e.g. look for 00-24h
forecast
> valid at 2018030900 (given in the config). If so, why doesn't it
> simply use gfs.t00z.pgrb.1p00.f024, which already have the 00-24h
> forecast? If it subtracts gfs.t00z.pgrb.1p00.f024 from
> gfs.t00z.pgrb.1p00.f048, the resulting file would have 24-48h
forecast
> valid at 2018031000. I did
> ncdump -h
> /meso/noscrub/Ying.Lin/metplus.out/gfs/bucket/gfs.2018030900_A024h
>
> and got
> APCP_24:init_time = "20180308_000000" ;
> APCP_24:init_time_ut = "1520467200" ;
> APCP_24:valid_time = "20180310_000000" ;
> APCP_24:valid_time_ut = "1520640000" ;
> APCP_24:accum_time = "240000" ;
> it seems to mean that the file contains APCP_24h with initial time
of
> 00z 03/08 and the valid time is 00Z 03/10 (that'll be APCP_48h
instead
> of 24h?)
>
> 2) The precip_continuous.conf has
> FCST_LEVEL = 6
> I thought FCST_LEVEL is the forecast bucket length. In the sample
GFS
> files, there isn't a 6-hourly bucket - gfs.t00z.pgrb.1p00.f024
contains
> 00-24h accumulation and gfs.t00z.pgrb.1p00.f048 contains 00-48h
> accumulation. Does FCST_LEVEL mean something other than the
forecast
> bucket length?
>
> 3) precip_continuous.conf has
> PCP_COMBINE_METHOD = SUBTRACT
> which should work for the example since the verifying analysis is
the
> 24h CCPA file. But what if I need to use 'SUBTRACT' for the
forecast
> field but 'SUM' for the analysis field (e.g. summing up 6h CCPAs
into a
> 24h total)? At one point I tried
> PCP_COMBINE_METHOD = SUBTRACT, SUM
> and got
> ERROR: Invalid PCP_COMBINE_METHOD specified
>
> After several failed attempts getting my own config file to work, I
> starting from precip_continuous.conf (which worked through the
> PcpCombine step; I think that's the only step it runs since the
config
> has only PcpCombine in the PROCESS_LIST) and tried to set a config
file
> as close to precip_continuous.conf as possible
>
(/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/parm/ylin_config/models/fv3gfs_24h.conf,
>
> also uploaded to
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf).
>
> The resulting err msg:
>
> 04/18 18:35:33.912 jlog (master_metplus.py:217) CRITICAL:
> master_metplus failed: ''
> Traceback (most recent call last):
> File "/meso/save/Ying.Lin/metplus/ush/master_metplus.py", line
213,
> in <module>
> main()
> File "/meso/save/Ying.Lin/metplus/ush/master_metplus.py", line
170,
> in main
> process.run_at_time(-1, run_time)
> File
>
"/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/ush/reformat_gridded_wrapper.py",
>
> line 80, in run_at_time
>
> Full output file:
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs.out.test5;
>
> metplus_final.conf:
>
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/metplus_final.conf
>
> The quoted err msg above appears to the the same as in my email
> yesterday, so there must still be something wrong with the format of
my
> config, though I think I eliminated the problems you pointed out in
your
> reply. There wouldn't be a ccpa_conus_1.0d_* of the valid day I
> specified, but I think the processing failed way before that point.
>
> Thank you again -
>
> Ying
>
>
> On 4/17/19 3:32 PM, George McCabe via RT wrote:
> > Hi Ying,
> >
> > You will need to set FCST_PCP_COMBINE_METHOD = SUBTRACT to config
METplus
> > to subtract accumulations files to get the accumulation you
desire. The
> > following use case has an example of this configuration:
> >
> > parm/use_cases/grid_to_grid/examples/precip_continous.conf
> >
> > I also noticed that your config file has the following items in
the
> > [config] section:
> >
> > FCST_PCP_COMBINE_INPUT_DIR = /gpfs/dell1/nco/ops/com/gfs/para
> > FCST_PCP_COMBINE_OUTPUT_DIR = {OUTPUT_BASE}/qpf_grib/bucket
> > FCST_GRID_STAT_INPUT_DIR = {OUTPUT_BASE}/qpf_grib/bucket
> >
> > You should move these to the [dir] section.
> >
> > Please let me know if you continue to have issues. The error
message you
> > provided has more information at the bottom that was not included.
The
> next
> > release of METplus contains more extensive error checking so the
error
> > message from that version should be much more helpful.
> >
> > Thanks,
> > George
> >
> > On Wed, Apr 17, 2019 at 12:43 PM Ying Lin via RT
<met_help at ucar.edu>
> wrote:
> >
> >> Wed Apr 17 12:43:02 2019: Request 89789 was acted upon.
> >> Transaction: Ticket created by ying.lin at noaa.gov
> >> Queue: met_help
> >> Subject: Using FCST_PCP_COMBINE in METplus
> >> Owner: Nobody
> >> Requestors: ying.lin at noaa.gov
> >> Status: new
> >> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789
> >
> >>
> >>
> >> Hello. I haven't been able to get the FCST_PCP_COMBINE feature
to work
> >> in METplus, running out of ideas and hope you can help.
> >>
> >> I've put some examples in
> >> https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/
(current
> test
> >> directory: gyre:/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/test).
> >> test_fv3gfs_24h.ksh is the simple test script. The goal is to
compute a
> >> 24h verification ending at 12Z 20190415.
> >>
> >> 'test1' did not need FCST_PCP_COMBINE - the config file
> >> fv3gfs_24h.conf.test1 simply had LEAD_SEQ set to '24',
> >> FCST_PCP_COMBINE_RUN was not set. The FV3GFS had two precip
buckets, 6h
> >> and continuously-accumulating, and METplus easily found the 24h
forecast
> >> accumulation and went on with the grid_stat.
> >>
> >> 'test2' had 'LEAD_SEQ = 36', and I added the following in the
config
> >> (
> >>
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test2
> >> ):
> >> FCST_PCP_COMBINE_RUN = True
> >> FCST_PCP_COMBINE_INPUT_DATATYPE = GRIB
> >> FCST_PCP_COMBINE_INPUT_DIR =
/gpfs/dell1/nco/ops/com/gfs/para
> >> FCST_PCP_COMBINE_OUTPUT_DIR = {OUTPUT_BASE}/qpf_grib/bucket
> >> FCST_GRID_STAT_INPUT_DIR = {OUTPUT_BASE}/qpf_grib/bucket
> >> FCST_MAX_FORECAST = 36
> >> FCST_INIT_INTERVAL = 6
> >> FCST_LEVEL = 36
> >> Here the goal was to see if I could get METplus to subtract the
00-12h
> >> accumulation from gfs.t00z.pgrb2.0p25.f012 from the 00-36h
accumulation
> >> in gfs.t00z.pgrb2.0p25.f036. Thinking that FCST_LEVEL is the
forecast
> >> bucket length, I figured setting it to '36' and artificially
specifying
> >> FCST_MAX_FORECAST = 36 (instead of the full 384h) should work for
this
> >> limited test where LEAD_SEQ is set to 36 (is there a way to
specify that
> >> the bucket never gets emptied?). It failed with the follow err
msg
> >> (full message in
> >>
> >>
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test2
> >> ):
> >>
> >> 04/16 20:01:42.754 jlog (master_metplus.py:217) CRITICAL:
> >> master_metplus failed: ''
> >> Traceback (most recent call last):
> >> File "/meso/save/Ying.Lin/metplus/ush/master_metplus.py",
line 213,
> >> in <module>
> >> main()
> >> File "/meso/save/Ying.Lin/metplus/ush/master_metplus.py",
line 170,
> >> in main
> >> process.run_at_time(-1, run_time)
> >> File
> >>
>
"/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/ush/reformat_gridded_wrapper.py",
> >>
> >> line 80, in run_at_time
> >> self.run_at_time_once(task_info, var_info, rl)
> >> File
> >>
"/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/ush/pcp_combine_wrapper.py",
> >> line 419, in run_at_time_once
> >> ....
> >>
> >> 'test3' is an attempt from a different angle, thinking that if
the
> >> 'continuously running' bucket didn't work, maybe the 6-hourly
bucket
> >> would. In
> >>
> >>
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test3
> ,
> >>
> >> I changed LEAD_SEQ to 24, and set FCST_LEVEL = 6. It produced
err msgs
> >> (
> >>
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test3
> )
> >>
> >> that appeared to be the same as that from 'test2'.
> >>
> >> Earlier on I tried using the FCST_PCP_COMBINE feature with the
NAM
> >> 'CONUSNEST' output (with simple regular 6-hourly bucket) and got
the
> >> same err msg, so I don't think FV3GFS's dual-bucket system is the
cause.
> >>
> >> Also posted on
> >> https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/ are
the two
> >> metplus_final.conf files from both 'test2' and 'test3'.
> >>
> >> I found in
> >>
> /global/noscrub/Julie.Prestopnik/METplus/METplus-
2.0.4/parm/use_cases/qpf/examples/
> >>
> >> that hrefmean-vs-mrms-qpe.conf and nationalblend-vs-mrms-qpe.conf
had
> >> "FCST_PCP_COMBINE_RUN = True", that's where I borrowed the
> >> FCST_PCP_COMBINE_* set up in my fv3gfs_24h.conf.test2 and
> >> fv3gfs_24h.conf.test3. Haven't been able to run the hrefmean and
> >> nationalblend tests myself however - got err msg of " CRITICAL:
> >> master_metplus failed: No option 'GEMPAKTOCF_CLASSPATH' in
section:
> >> 'exe'" from both attempts.
> >>
> >> Thank you for looking into this -
> >>
> >> Ying
> >>
> >> --
> >> Ying Lin
> >> NCEP/EMC/Verification, Post-processing and Product Generation
Branch
> >> NCWCP Cubicle No. 2015
> >> Ying.Lin at noaa.gov
> >>
> >>
> >>
> >>
>
> --
> Ying Lin
> NCEP/EMC/Verification, Post-processing and Product Generation Branch
> NCWCP Cubicle No. 2015
> Ying.Lin at noaa.gov
>
>
>
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #89789] Using FCST_PCP_COMBINE in METplus
From: Ying Lin
Time: Thu Apr 18 14:13:06 2019
Hi George,
Thank you for the quick reply. I was looking at
/gpfs/gd1/emc/global/noscrub/Julie.Prestopnik/METplus/METplus-
2.0.4/parm/use_cases/grid_to_grid/examples/precip_continuous.conf,
which has VALID_BEG = 2018030900, VALID_END = 2018030900. Date stamp
is
Jan 30. Is there a "newer version of v2.0.4" available on wcoss?
From
the above config it's not clear to me how METplus knows to subtract
the
gfs.*.f024 from the gfs.*.f048.
If you have time today, could you also take a look why my simple test
(conf set up to be as close to precip_continuous.conf as possible -
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf)
led to err msg in
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs.out.test5?
It seems that I corrected the problems you pointed out in your reply
this morning, but the err msg still seems to be the same as
yesterday's.
If you don't have time today, next Monday is fine too. Have a good
holiday weekend.
Ying
On 4/18/19 3:51 PM, George McCabe via RT wrote:
> Hi Ying,
>
> 1) The config file I see has a valid time of 2018031000 and 48 hr
forecast
> lead. You may be using an old version of METplus that had this set
up
> incorrectly. METplus version 2.0.4 should be correct.
>
> 2) pcp_combine -subtract does not actually use the FCST_LEVEL
variable, so
> that should be removed from the use case. The code takes the valid
time and
> subtracts the accumulation.
>
> 3) In the upcoming version of METplus (in a few weeks!) has a good
number
> of config changes from the previous versions. For example,
FCST_LEVEL will
> now be FCST_PCP_COMBINE_INPUT_LEVEL. These changes are designed to
make the
> variables more clear to what they actually do. In the new version,
there is
> now FCST_PCP_COMBINE_METHOD and OBS_PCP_COMBINE_METHOD, so you will
be able
> to specify a different method for your forecast and analysis data.
>
> FCST_PCP_COMBINE_RUN = True
> FCST_PCP_COMBINE_METHOD = SUBTRACT
>
> OBS_PCP_COMBINE_RUN = True
> OBS_PCP_COMBINE_METHOD = SUM
>
> This functionality has not been thoroughly tested as we haven't had
any
> example use cases that need to do this so far, so please let me know
if it
> does not work the way you expect and I will fix it.
>
> Just to let you know, I will be out of the office tomorrow (Friday)
and I
> wont have access to email. Feel free to send any follow-up emails to
me and
> I will get back to you on Monday.
>
> Thanks,
> George
>
> On Thu, Apr 18, 2019 at 7:38 PM Ying Lin via RT <met_help at ucar.edu>
wrote:
>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789 >
>>
>> Thank you very much George, this is very helpful.
>>
>> About the example in
>> parm/use_cases/grid_to_grid/examples/precip_continuous.conf
>> I tried running it and it seems to have run through the pcp_combine
step
>> fine: it did this:
>> /global/noscrub/Julie.Prestopnik/met/8.0/bin/pcp_combine
-subtract
>> /global/noscrub/Julie.Prestopnik/METplus/METplus-
2.0.4_sample_data/grid_to_grid/prfv3rt1/20180308/gfs.t00z.pgrb.1p00.f048
>>
>> 48
>> /global/noscrub/Julie.Prestopnik/METplus/METplus-
2.0.4_sample_data/grid_to_grid/prfv3rt1/20180308/gfs.t00z.pgrb.1p00.f024
>>
>> 24
/meso/noscrub/Ying.Lin/metplus.out/gfs/bucket/gfs.2018030900_A024h
>>
>> Several questions about the example:
>> 1) The precip_continuous.conf has
>> LEAD_SEQ = 24
>> I thought it meant forecast lead of 24h, e.g. look for 00-24h
forecast
>> valid at 2018030900 (given in the config). If so, why doesn't it
>> simply use gfs.t00z.pgrb.1p00.f024, which already have the 00-24h
>> forecast? If it subtracts gfs.t00z.pgrb.1p00.f024 from
>> gfs.t00z.pgrb.1p00.f048, the resulting file would have 24-48h
forecast
>> valid at 2018031000. I did
>> ncdump -h
>> /meso/noscrub/Ying.Lin/metplus.out/gfs/bucket/gfs.2018030900_A024h
>>
>> and got
>> APCP_24:init_time = "20180308_000000" ;
>> APCP_24:init_time_ut = "1520467200" ;
>> APCP_24:valid_time = "20180310_000000" ;
>> APCP_24:valid_time_ut = "1520640000" ;
>> APCP_24:accum_time = "240000" ;
>> it seems to mean that the file contains APCP_24h with initial time
of
>> 00z 03/08 and the valid time is 00Z 03/10 (that'll be APCP_48h
instead
>> of 24h?)
>>
>> 2) The precip_continuous.conf has
>> FCST_LEVEL = 6
>> I thought FCST_LEVEL is the forecast bucket length. In the sample
GFS
>> files, there isn't a 6-hourly bucket - gfs.t00z.pgrb.1p00.f024
contains
>> 00-24h accumulation and gfs.t00z.pgrb.1p00.f048 contains 00-48h
>> accumulation. Does FCST_LEVEL mean something other than the
forecast
>> bucket length?
>>
>> 3) precip_continuous.conf has
>> PCP_COMBINE_METHOD = SUBTRACT
>> which should work for the example since the verifying analysis is
the
>> 24h CCPA file. But what if I need to use 'SUBTRACT' for the
forecast
>> field but 'SUM' for the analysis field (e.g. summing up 6h CCPAs
into a
>> 24h total)? At one point I tried
>> PCP_COMBINE_METHOD = SUBTRACT, SUM
>> and got
>> ERROR: Invalid PCP_COMBINE_METHOD specified
>>
>> After several failed attempts getting my own config file to work, I
>> starting from precip_continuous.conf (which worked through the
>> PcpCombine step; I think that's the only step it runs since the
config
>> has only PcpCombine in the PROCESS_LIST) and tried to set a config
file
>> as close to precip_continuous.conf as possible
>>
(/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/parm/ylin_config/models/fv3gfs_24h.conf,
>>
>> also uploaded to
>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf).
>>
>> The resulting err msg:
>>
>> 04/18 18:35:33.912 jlog (master_metplus.py:217) CRITICAL:
>> master_metplus failed: ''
>> Traceback (most recent call last):
>> File "/meso/save/Ying.Lin/metplus/ush/master_metplus.py", line
213,
>> in <module>
>> main()
>> File "/meso/save/Ying.Lin/metplus/ush/master_metplus.py", line
170,
>> in main
>> process.run_at_time(-1, run_time)
>> File
>>
"/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/ush/reformat_gridded_wrapper.py",
>>
>> line 80, in run_at_time
>>
>> Full output file:
>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs.out.test5;
>>
>> metplus_final.conf:
>>
>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/metplus_final.conf
>>
>> The quoted err msg above appears to the the same as in my email
>> yesterday, so there must still be something wrong with the format
of my
>> config, though I think I eliminated the problems you pointed out in
your
>> reply. There wouldn't be a ccpa_conus_1.0d_* of the valid day I
>> specified, but I think the processing failed way before that point.
>>
>> Thank you again -
>>
>> Ying
>>
>>
>> On 4/17/19 3:32 PM, George McCabe via RT wrote:
>>> Hi Ying,
>>>
>>> You will need to set FCST_PCP_COMBINE_METHOD = SUBTRACT to config
METplus
>>> to subtract accumulations files to get the accumulation you
desire. The
>>> following use case has an example of this configuration:
>>>
>>> parm/use_cases/grid_to_grid/examples/precip_continous.conf
>>>
>>> I also noticed that your config file has the following items in
the
>>> [config] section:
>>>
>>> FCST_PCP_COMBINE_INPUT_DIR = /gpfs/dell1/nco/ops/com/gfs/para
>>> FCST_PCP_COMBINE_OUTPUT_DIR = {OUTPUT_BASE}/qpf_grib/bucket
>>> FCST_GRID_STAT_INPUT_DIR = {OUTPUT_BASE}/qpf_grib/bucket
>>>
>>> You should move these to the [dir] section.
>>>
>>> Please let me know if you continue to have issues. The error
message you
>>> provided has more information at the bottom that was not included.
The
>> next
>>> release of METplus contains more extensive error checking so the
error
>>> message from that version should be much more helpful.
>>>
>>> Thanks,
>>> George
>>>
>>> On Wed, Apr 17, 2019 at 12:43 PM Ying Lin via RT
<met_help at ucar.edu>
>> wrote:
>>>> Wed Apr 17 12:43:02 2019: Request 89789 was acted upon.
>>>> Transaction: Ticket created by ying.lin at noaa.gov
>>>> Queue: met_help
>>>> Subject: Using FCST_PCP_COMBINE in METplus
>>>> Owner: Nobody
>>>> Requestors: ying.lin at noaa.gov
>>>> Status: new
>>>> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789
>>>>
>>>> Hello. I haven't been able to get the FCST_PCP_COMBINE feature
to work
>>>> in METplus, running out of ideas and hope you can help.
>>>>
>>>> I've put some examples in
>>>> https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/
(current
>> test
>>>> directory: gyre:/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/test).
>>>> test_fv3gfs_24h.ksh is the simple test script. The goal is to
compute a
>>>> 24h verification ending at 12Z 20190415.
>>>>
>>>> 'test1' did not need FCST_PCP_COMBINE - the config file
>>>> fv3gfs_24h.conf.test1 simply had LEAD_SEQ set to '24',
>>>> FCST_PCP_COMBINE_RUN was not set. The FV3GFS had two precip
buckets, 6h
>>>> and continuously-accumulating, and METplus easily found the 24h
forecast
>>>> accumulation and went on with the grid_stat.
>>>>
>>>> 'test2' had 'LEAD_SEQ = 36', and I added the following in the
config
>>>> (
>>>>
>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test2
>>>> ):
>>>> FCST_PCP_COMBINE_RUN = True
>>>> FCST_PCP_COMBINE_INPUT_DATATYPE = GRIB
>>>> FCST_PCP_COMBINE_INPUT_DIR =
/gpfs/dell1/nco/ops/com/gfs/para
>>>> FCST_PCP_COMBINE_OUTPUT_DIR =
{OUTPUT_BASE}/qpf_grib/bucket
>>>> FCST_GRID_STAT_INPUT_DIR = {OUTPUT_BASE}/qpf_grib/bucket
>>>> FCST_MAX_FORECAST = 36
>>>> FCST_INIT_INTERVAL = 6
>>>> FCST_LEVEL = 36
>>>> Here the goal was to see if I could get METplus to subtract the
00-12h
>>>> accumulation from gfs.t00z.pgrb2.0p25.f012 from the 00-36h
accumulation
>>>> in gfs.t00z.pgrb2.0p25.f036. Thinking that FCST_LEVEL is the
forecast
>>>> bucket length, I figured setting it to '36' and artificially
specifying
>>>> FCST_MAX_FORECAST = 36 (instead of the full 384h) should work for
this
>>>> limited test where LEAD_SEQ is set to 36 (is there a way to
specify that
>>>> the bucket never gets emptied?). It failed with the follow err
msg
>>>> (full message in
>>>>
>>>>
>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test2
>>>> ):
>>>>
>>>> 04/16 20:01:42.754 jlog (master_metplus.py:217) CRITICAL:
>>>> master_metplus failed: ''
>>>> Traceback (most recent call last):
>>>> File "/meso/save/Ying.Lin/metplus/ush/master_metplus.py",
line 213,
>>>> in <module>
>>>> main()
>>>> File "/meso/save/Ying.Lin/metplus/ush/master_metplus.py",
line 170,
>>>> in main
>>>> process.run_at_time(-1, run_time)
>>>> File
>>>>
>>
"/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/ush/reformat_gridded_wrapper.py",
>>>> line 80, in run_at_time
>>>> self.run_at_time_once(task_info, var_info, rl)
>>>> File
>>>>
"/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/ush/pcp_combine_wrapper.py",
>>>> line 419, in run_at_time_once
>>>> ....
>>>>
>>>> 'test3' is an attempt from a different angle, thinking that if
the
>>>> 'continuously running' bucket didn't work, maybe the 6-hourly
bucket
>>>> would. In
>>>>
>>>>
>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test3
>> ,
>>>> I changed LEAD_SEQ to 24, and set FCST_LEVEL = 6. It produced
err msgs
>>>> (
>>>>
>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test3
>> )
>>>> that appeared to be the same as that from 'test2'.
>>>>
>>>> Earlier on I tried using the FCST_PCP_COMBINE feature with the
NAM
>>>> 'CONUSNEST' output (with simple regular 6-hourly bucket) and got
the
>>>> same err msg, so I don't think FV3GFS's dual-bucket system is the
cause.
>>>>
>>>> Also posted on
>>>> https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/ are
the two
>>>> metplus_final.conf files from both 'test2' and 'test3'.
>>>>
>>>> I found in
>>>>
>> /global/noscrub/Julie.Prestopnik/METplus/METplus-
2.0.4/parm/use_cases/qpf/examples/
>>>> that hrefmean-vs-mrms-qpe.conf and nationalblend-vs-mrms-qpe.conf
had
>>>> "FCST_PCP_COMBINE_RUN = True", that's where I borrowed the
>>>> FCST_PCP_COMBINE_* set up in my fv3gfs_24h.conf.test2 and
>>>> fv3gfs_24h.conf.test3. Haven't been able to run the hrefmean and
>>>> nationalblend tests myself however - got err msg of " CRITICAL:
>>>> master_metplus failed: No option 'GEMPAKTOCF_CLASSPATH' in
section:
>>>> 'exe'" from both attempts.
>>>>
>>>> Thank you for looking into this -
>>>>
>>>> Ying
>>>>
>>>> --
>>>> Ying Lin
>>>> NCEP/EMC/Verification, Post-processing and Product Generation
Branch
>>>> NCWCP Cubicle No. 2015
>>>> Ying.Lin at noaa.gov
>>>>
>>>>
>>>>
>>>>
>> --
>> Ying Lin
>> NCEP/EMC/Verification, Post-processing and Product Generation
Branch
>> NCWCP Cubicle No. 2015
>> Ying.Lin at noaa.gov
>>
>>
>>
>>
--
Ying Lin
NCEP/EMC/Verification, Post-processing and Product Generation Branch
NCWCP Cubicle No. 2015
Ying.Lin at noaa.gov
------------------------------------------------
Subject: Using FCST_PCP_COMBINE in METplus
From: George McCabe
Time: Thu Apr 18 14:25:50 2019
Hi Ying,
I am sorry, I was incorrect that the fixed precip_continuous config
file
was updated in 2.0.4. It looks like the fix was made after that
release. It
will be in the newest release. You can see the updated file on GitHub
in
the develop branch here:
https://github.com/NCAR/METplus/blob/develop/parm/use_cases/grid_to_grid/examples/precip_continuous.conf
For your use case, it looks like you don't have
FCST_PCP_COMBINE_INPUT_TEMPLATE set and you have 2 for grid_stat:
FCST_GRID_STAT_INPUT_TEMPLATE =
gfs.{init?fmt=%Y%m%d}/{init?fmt=%H}/gfs.t{init?fmt=%H}z.pgrb2.0p25.f{lead?fmt=%HHH}
FCST_PCP_COMBINE_OUTPUT_TEMPLATE =
gfs.{valid?fmt=%Y%m%d%H}_A{level?fmt=%HHH}h
FCST_GRID_STAT_INPUT_TEMPLATE =
gfs.{valid?fmt=%Y%m%d%H}_A{level?fmt=%HHH}h
Fixing this may solve your issue. Please let me know if it does not. I
have
to catch a flight this evening so I am going to stop working very
soon, but
if you reply very soon I may still be able to help you before the
weekend.
Thanks,
George
On Thu, Apr 18, 2019 at 8:13 PM Ying Lin via RT <met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789 >
>
> Hi George,
>
> Thank you for the quick reply. I was looking at
> /gpfs/gd1/emc/global/noscrub/Julie.Prestopnik/METplus/METplus-
2.0.4/parm/use_cases/grid_to_grid/examples/precip_continuous.conf,
>
> which has VALID_BEG = 2018030900, VALID_END = 2018030900. Date
stamp is
> Jan 30. Is there a "newer version of v2.0.4" available on wcoss?
From
> the above config it's not clear to me how METplus knows to subtract
the
> gfs.*.f024 from the gfs.*.f048.
>
> If you have time today, could you also take a look why my simple
test
> (conf set up to be as close to precip_continuous.conf as possible -
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf)
>
> led to err msg in
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs.out.test5?
>
> It seems that I corrected the problems you pointed out in your reply
> this morning, but the err msg still seems to be the same as
yesterday's.
>
> If you don't have time today, next Monday is fine too. Have a good
> holiday weekend.
>
> Ying
>
>
>
>
> On 4/18/19 3:51 PM, George McCabe via RT wrote:
> > Hi Ying,
> >
> > 1) The config file I see has a valid time of 2018031000 and 48 hr
> forecast
> > lead. You may be using an old version of METplus that had this set
up
> > incorrectly. METplus version 2.0.4 should be correct.
> >
> > 2) pcp_combine -subtract does not actually use the FCST_LEVEL
variable,
> so
> > that should be removed from the use case. The code takes the valid
time
> and
> > subtracts the accumulation.
> >
> > 3) In the upcoming version of METplus (in a few weeks!) has a good
number
> > of config changes from the previous versions. For example,
FCST_LEVEL
> will
> > now be FCST_PCP_COMBINE_INPUT_LEVEL. These changes are designed to
make
> the
> > variables more clear to what they actually do. In the new version,
there
> is
> > now FCST_PCP_COMBINE_METHOD and OBS_PCP_COMBINE_METHOD, so you
will be
> able
> > to specify a different method for your forecast and analysis data.
> >
> > FCST_PCP_COMBINE_RUN = True
> > FCST_PCP_COMBINE_METHOD = SUBTRACT
> >
> > OBS_PCP_COMBINE_RUN = True
> > OBS_PCP_COMBINE_METHOD = SUM
> >
> > This functionality has not been thoroughly tested as we haven't
had any
> > example use cases that need to do this so far, so please let me
know if
> it
> > does not work the way you expect and I will fix it.
> >
> > Just to let you know, I will be out of the office tomorrow
(Friday) and I
> > wont have access to email. Feel free to send any follow-up emails
to me
> and
> > I will get back to you on Monday.
> >
> > Thanks,
> > George
> >
> > On Thu, Apr 18, 2019 at 7:38 PM Ying Lin via RT
<met_help at ucar.edu>
> wrote:
> >
> >> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789 >
> >>
> >> Thank you very much George, this is very helpful.
> >>
> >> About the example in
> >> parm/use_cases/grid_to_grid/examples/precip_continuous.conf
> >> I tried running it and it seems to have run through the
pcp_combine step
> >> fine: it did this:
> >> /global/noscrub/Julie.Prestopnik/met/8.0/bin/pcp_combine
-subtract
> >>
> /global/noscrub/Julie.Prestopnik/METplus/METplus-
2.0.4_sample_data/grid_to_grid/prfv3rt1/20180308/gfs.t00z.pgrb.1p00.f048
> >>
> >> 48
> >>
> /global/noscrub/Julie.Prestopnik/METplus/METplus-
2.0.4_sample_data/grid_to_grid/prfv3rt1/20180308/gfs.t00z.pgrb.1p00.f024
> >>
> >> 24
/meso/noscrub/Ying.Lin/metplus.out/gfs/bucket/gfs.2018030900_A024h
> >>
> >> Several questions about the example:
> >> 1) The precip_continuous.conf has
> >> LEAD_SEQ = 24
> >> I thought it meant forecast lead of 24h, e.g. look for 00-24h
forecast
> >> valid at 2018030900 (given in the config). If so, why doesn't
it
> >> simply use gfs.t00z.pgrb.1p00.f024, which already have the 00-24h
> >> forecast? If it subtracts gfs.t00z.pgrb.1p00.f024 from
> >> gfs.t00z.pgrb.1p00.f048, the resulting file would have 24-48h
forecast
> >> valid at 2018031000. I did
> >> ncdump -h
> >>
/meso/noscrub/Ying.Lin/metplus.out/gfs/bucket/gfs.2018030900_A024h
> >>
> >> and got
> >> APCP_24:init_time = "20180308_000000" ;
> >> APCP_24:init_time_ut = "1520467200" ;
> >> APCP_24:valid_time = "20180310_000000" ;
> >> APCP_24:valid_time_ut = "1520640000" ;
> >> APCP_24:accum_time = "240000" ;
> >> it seems to mean that the file contains APCP_24h with initial
time of
> >> 00z 03/08 and the valid time is 00Z 03/10 (that'll be APCP_48h
instead
> >> of 24h?)
> >>
> >> 2) The precip_continuous.conf has
> >> FCST_LEVEL = 6
> >> I thought FCST_LEVEL is the forecast bucket length. In the
sample GFS
> >> files, there isn't a 6-hourly bucket - gfs.t00z.pgrb.1p00.f024
contains
> >> 00-24h accumulation and gfs.t00z.pgrb.1p00.f048 contains 00-48h
> >> accumulation. Does FCST_LEVEL mean something other than the
forecast
> >> bucket length?
> >>
> >> 3) precip_continuous.conf has
> >> PCP_COMBINE_METHOD = SUBTRACT
> >> which should work for the example since the verifying analysis is
the
> >> 24h CCPA file. But what if I need to use 'SUBTRACT' for the
forecast
> >> field but 'SUM' for the analysis field (e.g. summing up 6h CCPAs
into a
> >> 24h total)? At one point I tried
> >> PCP_COMBINE_METHOD = SUBTRACT, SUM
> >> and got
> >> ERROR: Invalid PCP_COMBINE_METHOD specified
> >>
> >> After several failed attempts getting my own config file to work,
I
> >> starting from precip_continuous.conf (which worked through the
> >> PcpCombine step; I think that's the only step it runs since the
config
> >> has only PcpCombine in the PROCESS_LIST) and tried to set a
config file
> >> as close to precip_continuous.conf as possible
> >>
>
(/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/parm/ylin_config/models/fv3gfs_24h.conf,
> >>
> >> also uploaded to
> >>
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf
> ).
> >>
> >> The resulting err msg:
> >>
> >> 04/18 18:35:33.912 jlog (master_metplus.py:217) CRITICAL:
> >> master_metplus failed: ''
> >> Traceback (most recent call last):
> >> File "/meso/save/Ying.Lin/metplus/ush/master_metplus.py",
line 213,
> >> in <module>
> >> main()
> >> File "/meso/save/Ying.Lin/metplus/ush/master_metplus.py",
line 170,
> >> in main
> >> process.run_at_time(-1, run_time)
> >> File
> >>
>
"/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/ush/reformat_gridded_wrapper.py",
> >>
> >> line 80, in run_at_time
> >>
> >> Full output file:
> >>
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs.out.test5
> ;
> >>
> >> metplus_final.conf:
> >>
> >>
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/metplus_final.conf
> >>
> >> The quoted err msg above appears to the the same as in my email
> >> yesterday, so there must still be something wrong with the format
of my
> >> config, though I think I eliminated the problems you pointed out
in your
> >> reply. There wouldn't be a ccpa_conus_1.0d_* of the valid day I
> >> specified, but I think the processing failed way before that
point.
> >>
> >> Thank you again -
> >>
> >> Ying
> >>
> >>
> >> On 4/17/19 3:32 PM, George McCabe via RT wrote:
> >>> Hi Ying,
> >>>
> >>> You will need to set FCST_PCP_COMBINE_METHOD = SUBTRACT to
config
> METplus
> >>> to subtract accumulations files to get the accumulation you
desire. The
> >>> following use case has an example of this configuration:
> >>>
> >>> parm/use_cases/grid_to_grid/examples/precip_continous.conf
> >>>
> >>> I also noticed that your config file has the following items in
the
> >>> [config] section:
> >>>
> >>> FCST_PCP_COMBINE_INPUT_DIR = /gpfs/dell1/nco/ops/com/gfs/para
> >>> FCST_PCP_COMBINE_OUTPUT_DIR = {OUTPUT_BASE}/qpf_grib/bucket
> >>> FCST_GRID_STAT_INPUT_DIR = {OUTPUT_BASE}/qpf_grib/bucket
> >>>
> >>> You should move these to the [dir] section.
> >>>
> >>> Please let me know if you continue to have issues. The error
message
> you
> >>> provided has more information at the bottom that was not
included. The
> >> next
> >>> release of METplus contains more extensive error checking so the
error
> >>> message from that version should be much more helpful.
> >>>
> >>> Thanks,
> >>> George
> >>>
> >>> On Wed, Apr 17, 2019 at 12:43 PM Ying Lin via RT
<met_help at ucar.edu>
> >> wrote:
> >>>> Wed Apr 17 12:43:02 2019: Request 89789 was acted upon.
> >>>> Transaction: Ticket created by ying.lin at noaa.gov
> >>>> Queue: met_help
> >>>> Subject: Using FCST_PCP_COMBINE in METplus
> >>>> Owner: Nobody
> >>>> Requestors: ying.lin at noaa.gov
> >>>> Status: new
> >>>> Ticket <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789
> >>>>
> >>>> Hello. I haven't been able to get the FCST_PCP_COMBINE feature
to
> work
> >>>> in METplus, running out of ideas and hope you can help.
> >>>>
> >>>> I've put some examples in
> >>>> https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/
(current
> >> test
> >>>> directory: gyre:/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/test).
> >>>> test_fv3gfs_24h.ksh is the simple test script. The goal is to
> compute a
> >>>> 24h verification ending at 12Z 20190415.
> >>>>
> >>>> 'test1' did not need FCST_PCP_COMBINE - the config file
> >>>> fv3gfs_24h.conf.test1 simply had LEAD_SEQ set to '24',
> >>>> FCST_PCP_COMBINE_RUN was not set. The FV3GFS had two precip
buckets,
> 6h
> >>>> and continuously-accumulating, and METplus easily found the 24h
> forecast
> >>>> accumulation and went on with the grid_stat.
> >>>>
> >>>> 'test2' had 'LEAD_SEQ = 36', and I added the following in the
config
> >>>> (
> >>>>
> >>
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test2
> >>>> ):
> >>>> FCST_PCP_COMBINE_RUN = True
> >>>> FCST_PCP_COMBINE_INPUT_DATATYPE = GRIB
> >>>> FCST_PCP_COMBINE_INPUT_DIR =
/gpfs/dell1/nco/ops/com/gfs/para
> >>>> FCST_PCP_COMBINE_OUTPUT_DIR =
{OUTPUT_BASE}/qpf_grib/bucket
> >>>> FCST_GRID_STAT_INPUT_DIR = {OUTPUT_BASE}/qpf_grib/bucket
> >>>> FCST_MAX_FORECAST = 36
> >>>> FCST_INIT_INTERVAL = 6
> >>>> FCST_LEVEL = 36
> >>>> Here the goal was to see if I could get METplus to subtract the
00-12h
> >>>> accumulation from gfs.t00z.pgrb2.0p25.f012 from the 00-36h
> accumulation
> >>>> in gfs.t00z.pgrb2.0p25.f036. Thinking that FCST_LEVEL is the
forecast
> >>>> bucket length, I figured setting it to '36' and artificially
> specifying
> >>>> FCST_MAX_FORECAST = 36 (instead of the full 384h) should work
for this
> >>>> limited test where LEAD_SEQ is set to 36 (is there a way to
specify
> that
> >>>> the bucket never gets emptied?). It failed with the follow err
msg
> >>>> (full message in
> >>>>
> >>>>
> >>
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test2
> >>>> ):
> >>>>
> >>>> 04/16 20:01:42.754 jlog (master_metplus.py:217) CRITICAL:
> >>>> master_metplus failed: ''
> >>>> Traceback (most recent call last):
> >>>> File "/meso/save/Ying.Lin/metplus/ush/master_metplus.py",
line
> 213,
> >>>> in <module>
> >>>> main()
> >>>> File "/meso/save/Ying.Lin/metplus/ush/master_metplus.py",
line
> 170,
> >>>> in main
> >>>> process.run_at_time(-1, run_time)
> >>>> File
> >>>>
> >>
>
"/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/ush/reformat_gridded_wrapper.py",
> >>>> line 80, in run_at_time
> >>>> self.run_at_time_once(task_info, var_info, rl)
> >>>> File
> >>>>
"/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/ush/pcp_combine_wrapper.py",
> >>>> line 419, in run_at_time_once
> >>>> ....
> >>>>
> >>>> 'test3' is an attempt from a different angle, thinking that if
the
> >>>> 'continuously running' bucket didn't work, maybe the 6-hourly
bucket
> >>>> would. In
> >>>>
> >>>>
> >>
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test3
> >> ,
> >>>> I changed LEAD_SEQ to 24, and set FCST_LEVEL = 6. It produced
err
> msgs
> >>>> (
> >>>>
> >>
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test3
> >> )
> >>>> that appeared to be the same as that from 'test2'.
> >>>>
> >>>> Earlier on I tried using the FCST_PCP_COMBINE feature with the
NAM
> >>>> 'CONUSNEST' output (with simple regular 6-hourly bucket) and
got the
> >>>> same err msg, so I don't think FV3GFS's dual-bucket system is
the
> cause.
> >>>>
> >>>> Also posted on
> >>>> https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/ are
the
> two
> >>>> metplus_final.conf files from both 'test2' and 'test3'.
> >>>>
> >>>> I found in
> >>>>
> >>
> /global/noscrub/Julie.Prestopnik/METplus/METplus-
2.0.4/parm/use_cases/qpf/examples/
> >>>> that hrefmean-vs-mrms-qpe.conf and nationalblend-vs-mrms-
qpe.conf had
> >>>> "FCST_PCP_COMBINE_RUN = True", that's where I borrowed the
> >>>> FCST_PCP_COMBINE_* set up in my fv3gfs_24h.conf.test2 and
> >>>> fv3gfs_24h.conf.test3. Haven't been able to run the hrefmean
and
> >>>> nationalblend tests myself however - got err msg of " CRITICAL:
> >>>> master_metplus failed: No option 'GEMPAKTOCF_CLASSPATH' in
section:
> >>>> 'exe'" from both attempts.
> >>>>
> >>>> Thank you for looking into this -
> >>>>
> >>>> Ying
> >>>>
> >>>> --
> >>>> Ying Lin
> >>>> NCEP/EMC/Verification, Post-processing and Product Generation
Branch
> >>>> NCWCP Cubicle No. 2015
> >>>> Ying.Lin at noaa.gov
> >>>>
> >>>>
> >>>>
> >>>>
> >> --
> >> Ying Lin
> >> NCEP/EMC/Verification, Post-processing and Product Generation
Branch
> >> NCWCP Cubicle No. 2015
> >> Ying.Lin at noaa.gov
> >>
> >>
> >>
> >>
>
> --
> Ying Lin
> NCEP/EMC/Verification, Post-processing and Product Generation Branch
> NCWCP Cubicle No. 2015
> Ying.Lin at noaa.gov
>
>
>
>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #89789] Using FCST_PCP_COMBINE in METplus
From: Ying Lin
Time: Thu Apr 18 17:09:50 2019
Hi George,
Thank you - you're right, replacing that first
FCST_GRID_STAT_INPUT_TEMPLATE with FCST_PCP_COMBINE_INPUT_TEMPLATE got
me past the err msg! There're still some old stuff with the way the
summing was done - similar to the odd result I mentioned in the last
email with the 2.0.4 precip_continuous.conf. I'll study the new
precip_continuous.conf in the github and follow up with you next week.
Thanks for your help and patience today. Have a great weekend -
Ying
On 4/18/19 4:25 PM, George McCabe via RT wrote:
> Hi Ying,
>
> I am sorry, I was incorrect that the fixed precip_continuous config
file
> was updated in 2.0.4. It looks like the fix was made after that
release. It
> will be in the newest release. You can see the updated file on
GitHub in
> the develop branch here:
>
>
https://github.com/NCAR/METplus/blob/develop/parm/use_cases/grid_to_grid/examples/precip_continuous.conf
>
> For your use case, it looks like you don't have
> FCST_PCP_COMBINE_INPUT_TEMPLATE set and you have 2 for grid_stat:
>
> FCST_GRID_STAT_INPUT_TEMPLATE =
>
gfs.{init?fmt=%Y%m%d}/{init?fmt=%H}/gfs.t{init?fmt=%H}z.pgrb2.0p25.f{lead?fmt=%HHH}
> FCST_PCP_COMBINE_OUTPUT_TEMPLATE =
> gfs.{valid?fmt=%Y%m%d%H}_A{level?fmt=%HHH}h
> FCST_GRID_STAT_INPUT_TEMPLATE =
gfs.{valid?fmt=%Y%m%d%H}_A{level?fmt=%HHH}h
>
> Fixing this may solve your issue. Please let me know if it does not.
I have
> to catch a flight this evening so I am going to stop working very
soon, but
> if you reply very soon I may still be able to help you before the
weekend.
>
> Thanks,
> George
>
> On Thu, Apr 18, 2019 at 8:13 PM Ying Lin via RT <met_help at ucar.edu>
wrote:
>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789 >
>>
>> Hi George,
>>
>> Thank you for the quick reply. I was looking at
>> /gpfs/gd1/emc/global/noscrub/Julie.Prestopnik/METplus/METplus-
2.0.4/parm/use_cases/grid_to_grid/examples/precip_continuous.conf,
>>
>> which has VALID_BEG = 2018030900, VALID_END = 2018030900. Date
stamp is
>> Jan 30. Is there a "newer version of v2.0.4" available on wcoss?
From
>> the above config it's not clear to me how METplus knows to subtract
the
>> gfs.*.f024 from the gfs.*.f048.
>>
>> If you have time today, could you also take a look why my simple
test
>> (conf set up to be as close to precip_continuous.conf as possible -
>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf)
>>
>> led to err msg in
>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs.out.test5?
>>
>> It seems that I corrected the problems you pointed out in your
reply
>> this morning, but the err msg still seems to be the same as
yesterday's.
>>
>> If you don't have time today, next Monday is fine too. Have a good
>> holiday weekend.
>>
>> Ying
>>
>>
>>
>>
>> On 4/18/19 3:51 PM, George McCabe via RT wrote:
>>> Hi Ying,
>>>
>>> 1) The config file I see has a valid time of 2018031000 and 48 hr
>> forecast
>>> lead. You may be using an old version of METplus that had this set
up
>>> incorrectly. METplus version 2.0.4 should be correct.
>>>
>>> 2) pcp_combine -subtract does not actually use the FCST_LEVEL
variable,
>> so
>>> that should be removed from the use case. The code takes the valid
time
>> and
>>> subtracts the accumulation.
>>>
>>> 3) In the upcoming version of METplus (in a few weeks!) has a good
number
>>> of config changes from the previous versions. For example,
FCST_LEVEL
>> will
>>> now be FCST_PCP_COMBINE_INPUT_LEVEL. These changes are designed to
make
>> the
>>> variables more clear to what they actually do. In the new version,
there
>> is
>>> now FCST_PCP_COMBINE_METHOD and OBS_PCP_COMBINE_METHOD, so you
will be
>> able
>>> to specify a different method for your forecast and analysis data.
>>>
>>> FCST_PCP_COMBINE_RUN = True
>>> FCST_PCP_COMBINE_METHOD = SUBTRACT
>>>
>>> OBS_PCP_COMBINE_RUN = True
>>> OBS_PCP_COMBINE_METHOD = SUM
>>>
>>> This functionality has not been thoroughly tested as we haven't
had any
>>> example use cases that need to do this so far, so please let me
know if
>> it
>>> does not work the way you expect and I will fix it.
>>>
>>> Just to let you know, I will be out of the office tomorrow
(Friday) and I
>>> wont have access to email. Feel free to send any follow-up emails
to me
>> and
>>> I will get back to you on Monday.
>>>
>>> Thanks,
>>> George
>>>
>>> On Thu, Apr 18, 2019 at 7:38 PM Ying Lin via RT
<met_help at ucar.edu>
>> wrote:
>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789 >
>>>>
>>>> Thank you very much George, this is very helpful.
>>>>
>>>> About the example in
>>>> parm/use_cases/grid_to_grid/examples/precip_continuous.conf
>>>> I tried running it and it seems to have run through the
pcp_combine step
>>>> fine: it did this:
>>>> /global/noscrub/Julie.Prestopnik/met/8.0/bin/pcp_combine
-subtract
>>>>
>> /global/noscrub/Julie.Prestopnik/METplus/METplus-
2.0.4_sample_data/grid_to_grid/prfv3rt1/20180308/gfs.t00z.pgrb.1p00.f048
>>>> 48
>>>>
>> /global/noscrub/Julie.Prestopnik/METplus/METplus-
2.0.4_sample_data/grid_to_grid/prfv3rt1/20180308/gfs.t00z.pgrb.1p00.f024
>>>> 24
/meso/noscrub/Ying.Lin/metplus.out/gfs/bucket/gfs.2018030900_A024h
>>>>
>>>> Several questions about the example:
>>>> 1) The precip_continuous.conf has
>>>> LEAD_SEQ = 24
>>>> I thought it meant forecast lead of 24h, e.g. look for 00-24h
forecast
>>>> valid at 2018030900 (given in the config). If so, why doesn't
it
>>>> simply use gfs.t00z.pgrb.1p00.f024, which already have the 00-24h
>>>> forecast? If it subtracts gfs.t00z.pgrb.1p00.f024 from
>>>> gfs.t00z.pgrb.1p00.f048, the resulting file would have 24-48h
forecast
>>>> valid at 2018031000. I did
>>>> ncdump -h
>>>>
/meso/noscrub/Ying.Lin/metplus.out/gfs/bucket/gfs.2018030900_A024h
>>>>
>>>> and got
>>>> APCP_24:init_time = "20180308_000000" ;
>>>> APCP_24:init_time_ut = "1520467200" ;
>>>> APCP_24:valid_time = "20180310_000000" ;
>>>> APCP_24:valid_time_ut = "1520640000" ;
>>>> APCP_24:accum_time = "240000" ;
>>>> it seems to mean that the file contains APCP_24h with initial
time of
>>>> 00z 03/08 and the valid time is 00Z 03/10 (that'll be APCP_48h
instead
>>>> of 24h?)
>>>>
>>>> 2) The precip_continuous.conf has
>>>> FCST_LEVEL = 6
>>>> I thought FCST_LEVEL is the forecast bucket length. In the
sample GFS
>>>> files, there isn't a 6-hourly bucket - gfs.t00z.pgrb.1p00.f024
contains
>>>> 00-24h accumulation and gfs.t00z.pgrb.1p00.f048 contains 00-48h
>>>> accumulation. Does FCST_LEVEL mean something other than the
forecast
>>>> bucket length?
>>>>
>>>> 3) precip_continuous.conf has
>>>> PCP_COMBINE_METHOD = SUBTRACT
>>>> which should work for the example since the verifying analysis is
the
>>>> 24h CCPA file. But what if I need to use 'SUBTRACT' for the
forecast
>>>> field but 'SUM' for the analysis field (e.g. summing up 6h CCPAs
into a
>>>> 24h total)? At one point I tried
>>>> PCP_COMBINE_METHOD = SUBTRACT, SUM
>>>> and got
>>>> ERROR: Invalid PCP_COMBINE_METHOD specified
>>>>
>>>> After several failed attempts getting my own config file to work,
I
>>>> starting from precip_continuous.conf (which worked through the
>>>> PcpCombine step; I think that's the only step it runs since the
config
>>>> has only PcpCombine in the PROCESS_LIST) and tried to set a
config file
>>>> as close to precip_continuous.conf as possible
>>>>
>>
(/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/parm/ylin_config/models/fv3gfs_24h.conf,
>>>> also uploaded to
>>>>
>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf
>> ).
>>>> The resulting err msg:
>>>>
>>>> 04/18 18:35:33.912 jlog (master_metplus.py:217) CRITICAL:
>>>> master_metplus failed: ''
>>>> Traceback (most recent call last):
>>>> File "/meso/save/Ying.Lin/metplus/ush/master_metplus.py",
line 213,
>>>> in <module>
>>>> main()
>>>> File "/meso/save/Ying.Lin/metplus/ush/master_metplus.py",
line 170,
>>>> in main
>>>> process.run_at_time(-1, run_time)
>>>> File
>>>>
>>
"/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/ush/reformat_gridded_wrapper.py",
>>>> line 80, in run_at_time
>>>>
>>>> Full output file:
>>>>
>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs.out.test5
>> ;
>>>> metplus_final.conf:
>>>>
>>>>
>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/metplus_final.conf
>>>> The quoted err msg above appears to the the same as in my email
>>>> yesterday, so there must still be something wrong with the format
of my
>>>> config, though I think I eliminated the problems you pointed out
in your
>>>> reply. There wouldn't be a ccpa_conus_1.0d_* of the valid day I
>>>> specified, but I think the processing failed way before that
point.
>>>>
>>>> Thank you again -
>>>>
>>>> Ying
>>>>
>>>>
>>>> On 4/17/19 3:32 PM, George McCabe via RT wrote:
>>>>> Hi Ying,
>>>>>
>>>>> You will need to set FCST_PCP_COMBINE_METHOD = SUBTRACT to
config
>> METplus
>>>>> to subtract accumulations files to get the accumulation you
desire. The
>>>>> following use case has an example of this configuration:
>>>>>
>>>>> parm/use_cases/grid_to_grid/examples/precip_continous.conf
>>>>>
>>>>> I also noticed that your config file has the following items in
the
>>>>> [config] section:
>>>>>
>>>>> FCST_PCP_COMBINE_INPUT_DIR = /gpfs/dell1/nco/ops/com/gfs/para
>>>>> FCST_PCP_COMBINE_OUTPUT_DIR = {OUTPUT_BASE}/qpf_grib/bucket
>>>>> FCST_GRID_STAT_INPUT_DIR = {OUTPUT_BASE}/qpf_grib/bucket
>>>>>
>>>>> You should move these to the [dir] section.
>>>>>
>>>>> Please let me know if you continue to have issues. The error
message
>> you
>>>>> provided has more information at the bottom that was not
included. The
>>>> next
>>>>> release of METplus contains more extensive error checking so the
error
>>>>> message from that version should be much more helpful.
>>>>>
>>>>> Thanks,
>>>>> George
>>>>>
>>>>> On Wed, Apr 17, 2019 at 12:43 PM Ying Lin via RT
<met_help at ucar.edu>
>>>> wrote:
>>>>>> Wed Apr 17 12:43:02 2019: Request 89789 was acted upon.
>>>>>> Transaction: Ticket created by ying.lin at noaa.gov
>>>>>> Queue: met_help
>>>>>> Subject: Using FCST_PCP_COMBINE in METplus
>>>>>> Owner: Nobody
>>>>>> Requestors: ying.lin at noaa.gov
>>>>>> Status: new
>>>>>> Ticket <URL:
>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789
>>>>>> Hello. I haven't been able to get the FCST_PCP_COMBINE feature
to
>> work
>>>>>> in METplus, running out of ideas and hope you can help.
>>>>>>
>>>>>> I've put some examples in
>>>>>> https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/
(current
>>>> test
>>>>>> directory: gyre:/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/test).
>>>>>> test_fv3gfs_24h.ksh is the simple test script. The goal is to
>> compute a
>>>>>> 24h verification ending at 12Z 20190415.
>>>>>>
>>>>>> 'test1' did not need FCST_PCP_COMBINE - the config file
>>>>>> fv3gfs_24h.conf.test1 simply had LEAD_SEQ set to '24',
>>>>>> FCST_PCP_COMBINE_RUN was not set. The FV3GFS had two precip
buckets,
>> 6h
>>>>>> and continuously-accumulating, and METplus easily found the 24h
>> forecast
>>>>>> accumulation and went on with the grid_stat.
>>>>>>
>>>>>> 'test2' had 'LEAD_SEQ = 36', and I added the following in the
config
>>>>>> (
>>>>>>
>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test2
>>>>>> ):
>>>>>> FCST_PCP_COMBINE_RUN = True
>>>>>> FCST_PCP_COMBINE_INPUT_DATATYPE = GRIB
>>>>>> FCST_PCP_COMBINE_INPUT_DIR =
/gpfs/dell1/nco/ops/com/gfs/para
>>>>>> FCST_PCP_COMBINE_OUTPUT_DIR =
{OUTPUT_BASE}/qpf_grib/bucket
>>>>>> FCST_GRID_STAT_INPUT_DIR =
{OUTPUT_BASE}/qpf_grib/bucket
>>>>>> FCST_MAX_FORECAST = 36
>>>>>> FCST_INIT_INTERVAL = 6
>>>>>> FCST_LEVEL = 36
>>>>>> Here the goal was to see if I could get METplus to subtract the
00-12h
>>>>>> accumulation from gfs.t00z.pgrb2.0p25.f012 from the 00-36h
>> accumulation
>>>>>> in gfs.t00z.pgrb2.0p25.f036. Thinking that FCST_LEVEL is the
forecast
>>>>>> bucket length, I figured setting it to '36' and artificially
>> specifying
>>>>>> FCST_MAX_FORECAST = 36 (instead of the full 384h) should work
for this
>>>>>> limited test where LEAD_SEQ is set to 36 (is there a way to
specify
>> that
>>>>>> the bucket never gets emptied?). It failed with the follow err
msg
>>>>>> (full message in
>>>>>>
>>>>>>
>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test2
>>>>>> ):
>>>>>>
>>>>>> 04/16 20:01:42.754 jlog (master_metplus.py:217) CRITICAL:
>>>>>> master_metplus failed: ''
>>>>>> Traceback (most recent call last):
>>>>>> File "/meso/save/Ying.Lin/metplus/ush/master_metplus.py",
line
>> 213,
>>>>>> in <module>
>>>>>> main()
>>>>>> File "/meso/save/Ying.Lin/metplus/ush/master_metplus.py",
line
>> 170,
>>>>>> in main
>>>>>> process.run_at_time(-1, run_time)
>>>>>> File
>>>>>>
>>
"/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/ush/reformat_gridded_wrapper.py",
>>>>>> line 80, in run_at_time
>>>>>> self.run_at_time_once(task_info, var_info, rl)
>>>>>> File
>>>>>>
"/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/ush/pcp_combine_wrapper.py",
>>>>>> line 419, in run_at_time_once
>>>>>> ....
>>>>>>
>>>>>> 'test3' is an attempt from a different angle, thinking that if
the
>>>>>> 'continuously running' bucket didn't work, maybe the 6-hourly
bucket
>>>>>> would. In
>>>>>>
>>>>>>
>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test3
>>>> ,
>>>>>> I changed LEAD_SEQ to 24, and set FCST_LEVEL = 6. It produced
err
>> msgs
>>>>>> (
>>>>>>
>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test3
>>>> )
>>>>>> that appeared to be the same as that from 'test2'.
>>>>>>
>>>>>> Earlier on I tried using the FCST_PCP_COMBINE feature with the
NAM
>>>>>> 'CONUSNEST' output (with simple regular 6-hourly bucket) and
got the
>>>>>> same err msg, so I don't think FV3GFS's dual-bucket system is
the
>> cause.
>>>>>> Also posted on
>>>>>> https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/ are
the
>> two
>>>>>> metplus_final.conf files from both 'test2' and 'test3'.
>>>>>>
>>>>>> I found in
>>>>>>
>> /global/noscrub/Julie.Prestopnik/METplus/METplus-
2.0.4/parm/use_cases/qpf/examples/
>>>>>> that hrefmean-vs-mrms-qpe.conf and nationalblend-vs-mrms-
qpe.conf had
>>>>>> "FCST_PCP_COMBINE_RUN = True", that's where I borrowed the
>>>>>> FCST_PCP_COMBINE_* set up in my fv3gfs_24h.conf.test2 and
>>>>>> fv3gfs_24h.conf.test3. Haven't been able to run the hrefmean
and
>>>>>> nationalblend tests myself however - got err msg of " CRITICAL:
>>>>>> master_metplus failed: No option 'GEMPAKTOCF_CLASSPATH' in
section:
>>>>>> 'exe'" from both attempts.
>>>>>>
>>>>>> Thank you for looking into this -
>>>>>>
>>>>>> Ying
>>>>>>
>>>>>> --
>>>>>> Ying Lin
>>>>>> NCEP/EMC/Verification, Post-processing and Product Generation
Branch
>>>>>> NCWCP Cubicle No. 2015
>>>>>> Ying.Lin at noaa.gov
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>> --
>>>> Ying Lin
>>>> NCEP/EMC/Verification, Post-processing and Product Generation
Branch
>>>> NCWCP Cubicle No. 2015
>>>> Ying.Lin at noaa.gov
>>>>
>>>>
>>>>
>>>>
>> --
>> Ying Lin
>> NCEP/EMC/Verification, Post-processing and Product Generation
Branch
>> NCWCP Cubicle No. 2015
>> Ying.Lin at noaa.gov
>>
>>
>>
>>
--
Ying Lin
NCEP/EMC/Verification, Post-processing and Product Generation Branch
NCWCP Cubicle No. 2015
Ying.Lin at noaa.gov
------------------------------------------------
Subject: Using FCST_PCP_COMBINE in METplus
From: George McCabe
Time: Thu Apr 18 19:15:58 2019
Hi Ying,
It looks like the bug with subtract mode is in v2.0.4. It will be
fixed in
the next release. You can look at the develop branch on GitHub to see
what
has changed in the subtract function. Have a good weekend!
On Thu, Apr 18, 2019, 5:09 PM Ying Lin via RT <met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789 >
>
> Hi George,
>
> Thank you - you're right, replacing that first
> FCST_GRID_STAT_INPUT_TEMPLATE with FCST_PCP_COMBINE_INPUT_TEMPLATE
got
> me past the err msg! There're still some old stuff with the way the
> summing was done - similar to the odd result I mentioned in the last
> email with the 2.0.4 precip_continuous.conf. I'll study the new
> precip_continuous.conf in the github and follow up with you next
week.
>
> Thanks for your help and patience today. Have a great weekend
-
>
> Ying
>
> On 4/18/19 4:25 PM, George McCabe via RT wrote:
> > Hi Ying,
> >
> > I am sorry, I was incorrect that the fixed precip_continuous
config file
> > was updated in 2.0.4. It looks like the fix was made after that
release.
> It
> > will be in the newest release. You can see the updated file on
GitHub in
> > the develop branch here:
> >
> >
>
https://github.com/NCAR/METplus/blob/develop/parm/use_cases/grid_to_grid/examples/precip_continuous.conf
> >
> > For your use case, it looks like you don't have
> > FCST_PCP_COMBINE_INPUT_TEMPLATE set and you have 2 for grid_stat:
> >
> > FCST_GRID_STAT_INPUT_TEMPLATE =
> >
>
gfs.{init?fmt=%Y%m%d}/{init?fmt=%H}/gfs.t{init?fmt=%H}z.pgrb2.0p25.f{lead?fmt=%HHH}
> > FCST_PCP_COMBINE_OUTPUT_TEMPLATE =
> > gfs.{valid?fmt=%Y%m%d%H}_A{level?fmt=%HHH}h
> > FCST_GRID_STAT_INPUT_TEMPLATE =
> gfs.{valid?fmt=%Y%m%d%H}_A{level?fmt=%HHH}h
> >
> > Fixing this may solve your issue. Please let me know if it does
not. I
> have
> > to catch a flight this evening so I am going to stop working very
soon,
> but
> > if you reply very soon I may still be able to help you before the
> weekend.
> >
> > Thanks,
> > George
> >
> > On Thu, Apr 18, 2019 at 8:13 PM Ying Lin via RT
<met_help at ucar.edu>
> wrote:
> >
> >> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789 >
> >>
> >> Hi George,
> >>
> >> Thank you for the quick reply. I was looking at
> >>
> /gpfs/gd1/emc/global/noscrub/Julie.Prestopnik/METplus/METplus-
2.0.4/parm/use_cases/grid_to_grid/examples/precip_continuous.conf,
> >>
> >> which has VALID_BEG = 2018030900, VALID_END = 2018030900. Date
stamp is
> >> Jan 30. Is there a "newer version of v2.0.4" available on wcoss?
From
> >> the above config it's not clear to me how METplus knows to
subtract the
> >> gfs.*.f024 from the gfs.*.f048.
> >>
> >> If you have time today, could you also take a look why my simple
test
> >> (conf set up to be as close to precip_continuous.conf as possible
-
> >>
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf)
> >>
> >> led to err msg in
> >>
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs.out.test5
> ?
> >>
> >> It seems that I corrected the problems you pointed out in your
reply
> >> this morning, but the err msg still seems to be the same as
yesterday's.
> >>
> >> If you don't have time today, next Monday is fine too. Have a
good
> >> holiday weekend.
> >>
> >> Ying
> >>
> >>
> >>
> >>
> >> On 4/18/19 3:51 PM, George McCabe via RT wrote:
> >>> Hi Ying,
> >>>
> >>> 1) The config file I see has a valid time of 2018031000 and 48
hr
> >> forecast
> >>> lead. You may be using an old version of METplus that had this
set up
> >>> incorrectly. METplus version 2.0.4 should be correct.
> >>>
> >>> 2) pcp_combine -subtract does not actually use the FCST_LEVEL
variable,
> >> so
> >>> that should be removed from the use case. The code takes the
valid time
> >> and
> >>> subtracts the accumulation.
> >>>
> >>> 3) In the upcoming version of METplus (in a few weeks!) has a
good
> number
> >>> of config changes from the previous versions. For example,
FCST_LEVEL
> >> will
> >>> now be FCST_PCP_COMBINE_INPUT_LEVEL. These changes are designed
to make
> >> the
> >>> variables more clear to what they actually do. In the new
version,
> there
> >> is
> >>> now FCST_PCP_COMBINE_METHOD and OBS_PCP_COMBINE_METHOD, so you
will be
> >> able
> >>> to specify a different method for your forecast and analysis
data.
> >>>
> >>> FCST_PCP_COMBINE_RUN = True
> >>> FCST_PCP_COMBINE_METHOD = SUBTRACT
> >>>
> >>> OBS_PCP_COMBINE_RUN = True
> >>> OBS_PCP_COMBINE_METHOD = SUM
> >>>
> >>> This functionality has not been thoroughly tested as we haven't
had any
> >>> example use cases that need to do this so far, so please let me
know if
> >> it
> >>> does not work the way you expect and I will fix it.
> >>>
> >>> Just to let you know, I will be out of the office tomorrow
(Friday)
> and I
> >>> wont have access to email. Feel free to send any follow-up
emails to me
> >> and
> >>> I will get back to you on Monday.
> >>>
> >>> Thanks,
> >>> George
> >>>
> >>> On Thu, Apr 18, 2019 at 7:38 PM Ying Lin via RT
<met_help at ucar.edu>
> >> wrote:
> >>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789 >
> >>>>
> >>>> Thank you very much George, this is very helpful.
> >>>>
> >>>> About the example in
> >>>>
parm/use_cases/grid_to_grid/examples/precip_continuous.conf
> >>>> I tried running it and it seems to have run through the
pcp_combine
> step
> >>>> fine: it did this:
> >>>> /global/noscrub/Julie.Prestopnik/met/8.0/bin/pcp_combine
> -subtract
> >>>>
> >>
> /global/noscrub/Julie.Prestopnik/METplus/METplus-
2.0.4_sample_data/grid_to_grid/prfv3rt1/20180308/gfs.t00z.pgrb.1p00.f048
> >>>> 48
> >>>>
> >>
> /global/noscrub/Julie.Prestopnik/METplus/METplus-
2.0.4_sample_data/grid_to_grid/prfv3rt1/20180308/gfs.t00z.pgrb.1p00.f024
> >>>> 24
/meso/noscrub/Ying.Lin/metplus.out/gfs/bucket/gfs.2018030900_A024h
> >>>>
> >>>> Several questions about the example:
> >>>> 1) The precip_continuous.conf has
> >>>> LEAD_SEQ = 24
> >>>> I thought it meant forecast lead of 24h, e.g. look for 00-24h
forecast
> >>>> valid at 2018030900 (given in the config). If so, why doesn't
it
> >>>> simply use gfs.t00z.pgrb.1p00.f024, which already have the 00-
24h
> >>>> forecast? If it subtracts gfs.t00z.pgrb.1p00.f024 from
> >>>> gfs.t00z.pgrb.1p00.f048, the resulting file would have 24-48h
forecast
> >>>> valid at 2018031000. I did
> >>>> ncdump -h
> >>>>
/meso/noscrub/Ying.Lin/metplus.out/gfs/bucket/gfs.2018030900_A024h
> >>>>
> >>>> and got
> >>>> APCP_24:init_time = "20180308_000000" ;
> >>>> APCP_24:init_time_ut = "1520467200" ;
> >>>> APCP_24:valid_time = "20180310_000000" ;
> >>>> APCP_24:valid_time_ut = "1520640000" ;
> >>>> APCP_24:accum_time = "240000" ;
> >>>> it seems to mean that the file contains APCP_24h with initial
time of
> >>>> 00z 03/08 and the valid time is 00Z 03/10 (that'll be APCP_48h
instead
> >>>> of 24h?)
> >>>>
> >>>> 2) The precip_continuous.conf has
> >>>> FCST_LEVEL = 6
> >>>> I thought FCST_LEVEL is the forecast bucket length. In the
sample GFS
> >>>> files, there isn't a 6-hourly bucket - gfs.t00z.pgrb.1p00.f024
> contains
> >>>> 00-24h accumulation and gfs.t00z.pgrb.1p00.f048 contains 00-48h
> >>>> accumulation. Does FCST_LEVEL mean something other than the
forecast
> >>>> bucket length?
> >>>>
> >>>> 3) precip_continuous.conf has
> >>>> PCP_COMBINE_METHOD = SUBTRACT
> >>>> which should work for the example since the verifying analysis
is the
> >>>> 24h CCPA file. But what if I need to use 'SUBTRACT' for the
forecast
> >>>> field but 'SUM' for the analysis field (e.g. summing up 6h
CCPAs into
> a
> >>>> 24h total)? At one point I tried
> >>>> PCP_COMBINE_METHOD = SUBTRACT, SUM
> >>>> and got
> >>>> ERROR: Invalid PCP_COMBINE_METHOD specified
> >>>>
> >>>> After several failed attempts getting my own config file to
work, I
> >>>> starting from precip_continuous.conf (which worked through the
> >>>> PcpCombine step; I think that's the only step it runs since the
config
> >>>> has only PcpCombine in the PROCESS_LIST) and tried to set a
config
> file
> >>>> as close to precip_continuous.conf as possible
> >>>>
> >>
>
(/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/parm/ylin_config/models/fv3gfs_24h.conf,
> >>>> also uploaded to
> >>>>
> >>
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf
> >> ).
> >>>> The resulting err msg:
> >>>>
> >>>> 04/18 18:35:33.912 jlog (master_metplus.py:217) CRITICAL:
> >>>> master_metplus failed: ''
> >>>> Traceback (most recent call last):
> >>>> File "/meso/save/Ying.Lin/metplus/ush/master_metplus.py",
line
> 213,
> >>>> in <module>
> >>>> main()
> >>>> File "/meso/save/Ying.Lin/metplus/ush/master_metplus.py",
line
> 170,
> >>>> in main
> >>>> process.run_at_time(-1, run_time)
> >>>> File
> >>>>
> >>
>
"/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/ush/reformat_gridded_wrapper.py",
> >>>> line 80, in run_at_time
> >>>>
> >>>> Full output file:
> >>>>
> >>
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs.out.test5
> >> ;
> >>>> metplus_final.conf:
> >>>>
> >>>>
> >>
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/metplus_final.conf
> >>>> The quoted err msg above appears to the the same as in my email
> >>>> yesterday, so there must still be something wrong with the
format of
> my
> >>>> config, though I think I eliminated the problems you pointed
out in
> your
> >>>> reply. There wouldn't be a ccpa_conus_1.0d_* of the valid day
I
> >>>> specified, but I think the processing failed way before that
point.
> >>>>
> >>>> Thank you again -
> >>>>
> >>>> Ying
> >>>>
> >>>>
> >>>> On 4/17/19 3:32 PM, George McCabe via RT wrote:
> >>>>> Hi Ying,
> >>>>>
> >>>>> You will need to set FCST_PCP_COMBINE_METHOD = SUBTRACT to
config
> >> METplus
> >>>>> to subtract accumulations files to get the accumulation you
desire.
> The
> >>>>> following use case has an example of this configuration:
> >>>>>
> >>>>> parm/use_cases/grid_to_grid/examples/precip_continous.conf
> >>>>>
> >>>>> I also noticed that your config file has the following items
in the
> >>>>> [config] section:
> >>>>>
> >>>>> FCST_PCP_COMBINE_INPUT_DIR = /gpfs/dell1/nco/ops/com/gfs/para
> >>>>> FCST_PCP_COMBINE_OUTPUT_DIR = {OUTPUT_BASE}/qpf_grib/bucket
> >>>>> FCST_GRID_STAT_INPUT_DIR = {OUTPUT_BASE}/qpf_grib/bucket
> >>>>>
> >>>>> You should move these to the [dir] section.
> >>>>>
> >>>>> Please let me know if you continue to have issues. The error
message
> >> you
> >>>>> provided has more information at the bottom that was not
included.
> The
> >>>> next
> >>>>> release of METplus contains more extensive error checking so
the
> error
> >>>>> message from that version should be much more helpful.
> >>>>>
> >>>>> Thanks,
> >>>>> George
> >>>>>
> >>>>> On Wed, Apr 17, 2019 at 12:43 PM Ying Lin via RT
<met_help at ucar.edu>
> >>>> wrote:
> >>>>>> Wed Apr 17 12:43:02 2019: Request 89789 was acted upon.
> >>>>>> Transaction: Ticket created by ying.lin at noaa.gov
> >>>>>> Queue: met_help
> >>>>>> Subject: Using FCST_PCP_COMBINE in METplus
> >>>>>> Owner: Nobody
> >>>>>> Requestors: ying.lin at noaa.gov
> >>>>>> Status: new
> >>>>>> Ticket <URL:
> >> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789
> >>>>>> Hello. I haven't been able to get the FCST_PCP_COMBINE
feature to
> >> work
> >>>>>> in METplus, running out of ideas and hope you can help.
> >>>>>>
> >>>>>> I've put some examples in
> >>>>>> https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/
(current
> >>>> test
> >>>>>> directory:
gyre:/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/test).
> >>>>>> test_fv3gfs_24h.ksh is the simple test script. The goal is
to
> >> compute a
> >>>>>> 24h verification ending at 12Z 20190415.
> >>>>>>
> >>>>>> 'test1' did not need FCST_PCP_COMBINE - the config file
> >>>>>> fv3gfs_24h.conf.test1 simply had LEAD_SEQ set to '24',
> >>>>>> FCST_PCP_COMBINE_RUN was not set. The FV3GFS had two precip
> buckets,
> >> 6h
> >>>>>> and continuously-accumulating, and METplus easily found the
24h
> >> forecast
> >>>>>> accumulation and went on with the grid_stat.
> >>>>>>
> >>>>>> 'test2' had 'LEAD_SEQ = 36', and I added the following in the
config
> >>>>>> (
> >>>>>>
> >>
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test2
> >>>>>> ):
> >>>>>> FCST_PCP_COMBINE_RUN = True
> >>>>>> FCST_PCP_COMBINE_INPUT_DATATYPE = GRIB
> >>>>>> FCST_PCP_COMBINE_INPUT_DIR =
> /gpfs/dell1/nco/ops/com/gfs/para
> >>>>>> FCST_PCP_COMBINE_OUTPUT_DIR =
{OUTPUT_BASE}/qpf_grib/bucket
> >>>>>> FCST_GRID_STAT_INPUT_DIR =
{OUTPUT_BASE}/qpf_grib/bucket
> >>>>>> FCST_MAX_FORECAST = 36
> >>>>>> FCST_INIT_INTERVAL = 6
> >>>>>> FCST_LEVEL = 36
> >>>>>> Here the goal was to see if I could get METplus to subtract
the
> 00-12h
> >>>>>> accumulation from gfs.t00z.pgrb2.0p25.f012 from the 00-36h
> >> accumulation
> >>>>>> in gfs.t00z.pgrb2.0p25.f036. Thinking that FCST_LEVEL is the
> forecast
> >>>>>> bucket length, I figured setting it to '36' and artificially
> >> specifying
> >>>>>> FCST_MAX_FORECAST = 36 (instead of the full 384h) should work
for
> this
> >>>>>> limited test where LEAD_SEQ is set to 36 (is there a way to
specify
> >> that
> >>>>>> the bucket never gets emptied?). It failed with the follow
err msg
> >>>>>> (full message in
> >>>>>>
> >>>>>>
> >>
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test2
> >>>>>> ):
> >>>>>>
> >>>>>> 04/16 20:01:42.754 jlog (master_metplus.py:217) CRITICAL:
> >>>>>> master_metplus failed: ''
> >>>>>> Traceback (most recent call last):
> >>>>>> File
"/meso/save/Ying.Lin/metplus/ush/master_metplus.py", line
> >> 213,
> >>>>>> in <module>
> >>>>>> main()
> >>>>>> File
"/meso/save/Ying.Lin/metplus/ush/master_metplus.py", line
> >> 170,
> >>>>>> in main
> >>>>>> process.run_at_time(-1, run_time)
> >>>>>> File
> >>>>>>
> >>
>
"/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/ush/reformat_gridded_wrapper.py",
> >>>>>> line 80, in run_at_time
> >>>>>> self.run_at_time_once(task_info, var_info, rl)
> >>>>>> File
> >>>>>>
>
"/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/ush/pcp_combine_wrapper.py",
> >>>>>> line 419, in run_at_time_once
> >>>>>> ....
> >>>>>>
> >>>>>> 'test3' is an attempt from a different angle, thinking that
if the
> >>>>>> 'continuously running' bucket didn't work, maybe the 6-hourly
bucket
> >>>>>> would. In
> >>>>>>
> >>>>>>
> >>
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test3
> >>>> ,
> >>>>>> I changed LEAD_SEQ to 24, and set FCST_LEVEL = 6. It
produced err
> >> msgs
> >>>>>> (
> >>>>>>
> >>
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test3
> >>>> )
> >>>>>> that appeared to be the same as that from 'test2'.
> >>>>>>
> >>>>>> Earlier on I tried using the FCST_PCP_COMBINE feature with
the NAM
> >>>>>> 'CONUSNEST' output (with simple regular 6-hourly bucket) and
got the
> >>>>>> same err msg, so I don't think FV3GFS's dual-bucket system is
the
> >> cause.
> >>>>>> Also posted on
> >>>>>> https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/
are the
> >> two
> >>>>>> metplus_final.conf files from both 'test2' and 'test3'.
> >>>>>>
> >>>>>> I found in
> >>>>>>
> >>
> /global/noscrub/Julie.Prestopnik/METplus/METplus-
2.0.4/parm/use_cases/qpf/examples/
> >>>>>> that hrefmean-vs-mrms-qpe.conf and nationalblend-vs-mrms-
qpe.conf
> had
> >>>>>> "FCST_PCP_COMBINE_RUN = True", that's where I borrowed the
> >>>>>> FCST_PCP_COMBINE_* set up in my fv3gfs_24h.conf.test2 and
> >>>>>> fv3gfs_24h.conf.test3. Haven't been able to run the hrefmean
and
> >>>>>> nationalblend tests myself however - got err msg of "
CRITICAL:
> >>>>>> master_metplus failed: No option 'GEMPAKTOCF_CLASSPATH' in
section:
> >>>>>> 'exe'" from both attempts.
> >>>>>>
> >>>>>> Thank you for looking into this -
> >>>>>>
> >>>>>> Ying
> >>>>>>
> >>>>>> --
> >>>>>> Ying Lin
> >>>>>> NCEP/EMC/Verification, Post-processing and Product Generation
Branch
> >>>>>> NCWCP Cubicle No. 2015
> >>>>>> Ying.Lin at noaa.gov
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>> --
> >>>> Ying Lin
> >>>> NCEP/EMC/Verification, Post-processing and Product Generation
Branch
> >>>> NCWCP Cubicle No. 2015
> >>>> Ying.Lin at noaa.gov
> >>>>
> >>>>
> >>>>
> >>>>
> >> --
> >> Ying Lin
> >> NCEP/EMC/Verification, Post-processing and Product Generation
Branch
> >> NCWCP Cubicle No. 2015
> >> Ying.Lin at noaa.gov
> >>
> >>
> >>
> >>
>
> --
> Ying Lin
> NCEP/EMC/Verification, Post-processing and Product Generation Branch
> NCWCP Cubicle No. 2015
> Ying.Lin at noaa.gov
>
>
>
>
------------------------------------------------
Subject: Using FCST_PCP_COMBINE in METplus
From: George McCabe
Time: Wed May 29 11:53:10 2019
Hi Ying,
I am just following up on this ticket. Have you tried running your
case on METplus v2.1? Let me know if you run into any other issues.
Thanks,
George
On Thu Apr 18 19:15:58 2019, mccabe wrote:
> Hi Ying,
>
> It looks like the bug with subtract mode is in v2.0.4. It will be
> fixed in
> the next release. You can look at the develop branch on GitHub to
see
> what
> has changed in the subtract function. Have a good weekend!
>
> On Thu, Apr 18, 2019, 5:09 PM Ying Lin via RT <met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789 >
> >
> > Hi George,
> >
> > Thank you - you're right, replacing that first
> > FCST_GRID_STAT_INPUT_TEMPLATE with FCST_PCP_COMBINE_INPUT_TEMPLATE
> > got
> > me past the err msg! There're still some old stuff with the way
the
> > summing was done - similar to the odd result I mentioned in the
last
> > email with the 2.0.4 precip_continuous.conf. I'll study the new
> > precip_continuous.conf in the github and follow up with you next
> > week.
> >
> > Thanks for your help and patience today. Have a great weekend -
> >
> > Ying
> >
> > On 4/18/19 4:25 PM, George McCabe via RT wrote:
> > > Hi Ying,
> > >
> > > I am sorry, I was incorrect that the fixed precip_continuous
config
> > > file
> > > was updated in 2.0.4. It looks like the fix was made after that
> > > release.
> > It
> > > will be in the newest release. You can see the updated file on
> > > GitHub in
> > > the develop branch here:
> > >
> > >
> >
https://github.com/NCAR/METplus/blob/develop/parm/use_cases/grid_to_grid/examples/precip_continuous.conf
> > >
> > > For your use case, it looks like you don't have
> > > FCST_PCP_COMBINE_INPUT_TEMPLATE set and you have 2 for
grid_stat:
> > >
> > > FCST_GRID_STAT_INPUT_TEMPLATE =
> > >
> >
gfs.{init?fmt=%Y%m%d}/{init?fmt=%H}/gfs.t{init?fmt=%H}z.pgrb2.0p25.f{lead?fmt=%HHH}
> > > FCST_PCP_COMBINE_OUTPUT_TEMPLATE =
> > > gfs.{valid?fmt=%Y%m%d%H}_A{level?fmt=%HHH}h
> > > FCST_GRID_STAT_INPUT_TEMPLATE =
> > gfs.{valid?fmt=%Y%m%d%H}_A{level?fmt=%HHH}h
> > >
> > > Fixing this may solve your issue. Please let me know if it does
> > > not. I
> > have
> > > to catch a flight this evening so I am going to stop working
very
> > > soon,
> > but
> > > if you reply very soon I may still be able to help you before
the
> > weekend.
> > >
> > > Thanks,
> > > George
> > >
> > > On Thu, Apr 18, 2019 at 8:13 PM Ying Lin via RT
<met_help at ucar.edu>
> > wrote:
> > >
> > >> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789 >
> > >>
> > >> Hi George,
> > >>
> > >> Thank you for the quick reply. I was looking at
> > >>
> > /gpfs/gd1/emc/global/noscrub/Julie.Prestopnik/METplus/METplus-
> > 2.0.4/parm/use_cases/grid_to_grid/examples/precip_continuous.conf,
> > >>
> > >> which has VALID_BEG = 2018030900, VALID_END = 2018030900. Date
> > >> stamp is
> > >> Jan 30. Is there a "newer version of v2.0.4" available on
wcoss?
> > >> From
> > >> the above config it's not clear to me how METplus knows to
> > >> subtract the
> > >> gfs.*.f024 from the gfs.*.f048.
> > >>
> > >> If you have time today, could you also take a look why my
simple
> > >> test
> > >> (conf set up to be as close to precip_continuous.conf as
possible
> > >> -
> > >>
> >
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf)
> > >>
> > >> led to err msg in
> > >>
> >
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs.out.test5
> > ?
> > >>
> > >> It seems that I corrected the problems you pointed out in your
> > >> reply
> > >> this morning, but the err msg still seems to be the same as
> > >> yesterday's.
> > >>
> > >> If you don't have time today, next Monday is fine too. Have a
> > >> good
> > >> holiday weekend.
> > >>
> > >> Ying
> > >>
> > >>
> > >>
> > >>
> > >> On 4/18/19 3:51 PM, George McCabe via RT wrote:
> > >>> Hi Ying,
> > >>>
> > >>> 1) The config file I see has a valid time of 2018031000 and 48
hr
> > >> forecast
> > >>> lead. You may be using an old version of METplus that had this
> > >>> set up
> > >>> incorrectly. METplus version 2.0.4 should be correct.
> > >>>
> > >>> 2) pcp_combine -subtract does not actually use the FCST_LEVEL
> > >>> variable,
> > >> so
> > >>> that should be removed from the use case. The code takes the
> > >>> valid time
> > >> and
> > >>> subtracts the accumulation.
> > >>>
> > >>> 3) In the upcoming version of METplus (in a few weeks!) has a
> > >>> good
> > number
> > >>> of config changes from the previous versions. For example,
> > >>> FCST_LEVEL
> > >> will
> > >>> now be FCST_PCP_COMBINE_INPUT_LEVEL. These changes are
designed
> > >>> to make
> > >> the
> > >>> variables more clear to what they actually do. In the new
> > >>> version,
> > there
> > >> is
> > >>> now FCST_PCP_COMBINE_METHOD and OBS_PCP_COMBINE_METHOD, so you
> > >>> will be
> > >> able
> > >>> to specify a different method for your forecast and analysis
> > >>> data.
> > >>>
> > >>> FCST_PCP_COMBINE_RUN = True
> > >>> FCST_PCP_COMBINE_METHOD = SUBTRACT
> > >>>
> > >>> OBS_PCP_COMBINE_RUN = True
> > >>> OBS_PCP_COMBINE_METHOD = SUM
> > >>>
> > >>> This functionality has not been thoroughly tested as we
haven't
> > >>> had any
> > >>> example use cases that need to do this so far, so please let
me
> > >>> know if
> > >> it
> > >>> does not work the way you expect and I will fix it.
> > >>>
> > >>> Just to let you know, I will be out of the office tomorrow
> > >>> (Friday)
> > and I
> > >>> wont have access to email. Feel free to send any follow-up
emails
> > >>> to me
> > >> and
> > >>> I will get back to you on Monday.
> > >>>
> > >>> Thanks,
> > >>> George
> > >>>
> > >>> On Thu, Apr 18, 2019 at 7:38 PM Ying Lin via RT
> > >>> <met_help at ucar.edu>
> > >> wrote:
> > >>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789
>
> > >>>>
> > >>>> Thank you very much George, this is very helpful.
> > >>>>
> > >>>> About the example in
> > >>>>
parm/use_cases/grid_to_grid/examples/precip_continuous.conf
> > >>>> I tried running it and it seems to have run through the
> > >>>> pcp_combine
> > step
> > >>>> fine: it did this:
> > >>>> /global/noscrub/Julie.Prestopnik/met/8.0/bin/pcp_combine
> > -subtract
> > >>>>
> > >>
> > /global/noscrub/Julie.Prestopnik/METplus/METplus-
> >
2.0.4_sample_data/grid_to_grid/prfv3rt1/20180308/gfs.t00z.pgrb.1p00.f048
> > >>>> 48
> > >>>>
> > >>
> > /global/noscrub/Julie.Prestopnik/METplus/METplus-
> >
2.0.4_sample_data/grid_to_grid/prfv3rt1/20180308/gfs.t00z.pgrb.1p00.f024
> > >>>> 24
> > >>>>
/meso/noscrub/Ying.Lin/metplus.out/gfs/bucket/gfs.2018030900_A024h
> > >>>>
> > >>>> Several questions about the example:
> > >>>> 1) The precip_continuous.conf has
> > >>>> LEAD_SEQ = 24
> > >>>> I thought it meant forecast lead of 24h, e.g. look for 00-24h
> > >>>> forecast
> > >>>> valid at 2018030900 (given in the config). If so, why
doesn't
> > >>>> it
> > >>>> simply use gfs.t00z.pgrb.1p00.f024, which already have the
00-
> > >>>> 24h
> > >>>> forecast? If it subtracts gfs.t00z.pgrb.1p00.f024 from
> > >>>> gfs.t00z.pgrb.1p00.f048, the resulting file would have 24-48h
> > >>>> forecast
> > >>>> valid at 2018031000. I did
> > >>>> ncdump -h
> > >>>>
/meso/noscrub/Ying.Lin/metplus.out/gfs/bucket/gfs.2018030900_A024h
> > >>>>
> > >>>> and got
> > >>>> APCP_24:init_time = "20180308_000000" ;
> > >>>> APCP_24:init_time_ut = "1520467200" ;
> > >>>> APCP_24:valid_time = "20180310_000000" ;
> > >>>> APCP_24:valid_time_ut = "1520640000" ;
> > >>>> APCP_24:accum_time = "240000" ;
> > >>>> it seems to mean that the file contains APCP_24h with initial
> > >>>> time of
> > >>>> 00z 03/08 and the valid time is 00Z 03/10 (that'll be
APCP_48h
> > >>>> instead
> > >>>> of 24h?)
> > >>>>
> > >>>> 2) The precip_continuous.conf has
> > >>>> FCST_LEVEL = 6
> > >>>> I thought FCST_LEVEL is the forecast bucket length. In the
> > >>>> sample GFS
> > >>>> files, there isn't a 6-hourly bucket -
gfs.t00z.pgrb.1p00.f024
> > contains
> > >>>> 00-24h accumulation and gfs.t00z.pgrb.1p00.f048 contains 00-
48h
> > >>>> accumulation. Does FCST_LEVEL mean something other than the
> > >>>> forecast
> > >>>> bucket length?
> > >>>>
> > >>>> 3) precip_continuous.conf has
> > >>>> PCP_COMBINE_METHOD = SUBTRACT
> > >>>> which should work for the example since the verifying
analysis
> > >>>> is the
> > >>>> 24h CCPA file. But what if I need to use 'SUBTRACT' for the
> > >>>> forecast
> > >>>> field but 'SUM' for the analysis field (e.g. summing up 6h
CCPAs
> > >>>> into
> > a
> > >>>> 24h total)? At one point I tried
> > >>>> PCP_COMBINE_METHOD = SUBTRACT, SUM
> > >>>> and got
> > >>>> ERROR: Invalid PCP_COMBINE_METHOD specified
> > >>>>
> > >>>> After several failed attempts getting my own config file to
> > >>>> work, I
> > >>>> starting from precip_continuous.conf (which worked through
the
> > >>>> PcpCombine step; I think that's the only step it runs since
the
> > >>>> config
> > >>>> has only PcpCombine in the PROCESS_LIST) and tried to set a
> > >>>> config
> > file
> > >>>> as close to precip_continuous.conf as possible
> > >>>>
> > >>
> >
(/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/parm/ylin_config/models/fv3gfs_24h.conf,
> > >>>> also uploaded to
> > >>>>
> > >>
> >
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf
> > >> ).
> > >>>> The resulting err msg:
> > >>>>
> > >>>> 04/18 18:35:33.912 jlog (master_metplus.py:217) CRITICAL:
> > >>>> master_metplus failed: ''
> > >>>> Traceback (most recent call last):
> > >>>> File "/meso/save/Ying.Lin/metplus/ush/master_metplus.py",
line
> > 213,
> > >>>> in <module>
> > >>>> main()
> > >>>> File
"/meso/save/Ying.Lin/metplus/ush/master_metplus.py",
> > >>>> line
> > 170,
> > >>>> in main
> > >>>> process.run_at_time(-1, run_time)
> > >>>> File
> > >>>>
> > >>
> >
"/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/ush/reformat_gridded_wrapper.py",
> > >>>> line 80, in run_at_time
> > >>>>
> > >>>> Full output file:
> > >>>>
> > >>
> >
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs.out.test5
> > >> ;
> > >>>> metplus_final.conf:
> > >>>>
> > >>>>
> > >>
> >
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/metplus_final.conf
> > >>>> The quoted err msg above appears to the the same as in my
email
> > >>>> yesterday, so there must still be something wrong with the
> > >>>> format of
> > my
> > >>>> config, though I think I eliminated the problems you pointed
out
> > >>>> in
> > your
> > >>>> reply. There wouldn't be a ccpa_conus_1.0d_* of the valid
day I
> > >>>> specified, but I think the processing failed way before that
> > >>>> point.
> > >>>>
> > >>>> Thank you again -
> > >>>>
> > >>>> Ying
> > >>>>
> > >>>>
> > >>>> On 4/17/19 3:32 PM, George McCabe via RT wrote:
> > >>>>> Hi Ying,
> > >>>>>
> > >>>>> You will need to set FCST_PCP_COMBINE_METHOD = SUBTRACT to
> > >>>>> config
> > >> METplus
> > >>>>> to subtract accumulations files to get the accumulation you
> > >>>>> desire.
> > The
> > >>>>> following use case has an example of this configuration:
> > >>>>>
> > >>>>> parm/use_cases/grid_to_grid/examples/precip_continous.conf
> > >>>>>
> > >>>>> I also noticed that your config file has the following items
in
> > >>>>> the
> > >>>>> [config] section:
> > >>>>>
> > >>>>> FCST_PCP_COMBINE_INPUT_DIR =
/gpfs/dell1/nco/ops/com/gfs/para
> > >>>>> FCST_PCP_COMBINE_OUTPUT_DIR = {OUTPUT_BASE}/qpf_grib/bucket
> > >>>>> FCST_GRID_STAT_INPUT_DIR = {OUTPUT_BASE}/qpf_grib/bucket
> > >>>>>
> > >>>>> You should move these to the [dir] section.
> > >>>>>
> > >>>>> Please let me know if you continue to have issues. The error
> > >>>>> message
> > >> you
> > >>>>> provided has more information at the bottom that was not
> > >>>>> included.
> > The
> > >>>> next
> > >>>>> release of METplus contains more extensive error checking so
> > >>>>> the
> > error
> > >>>>> message from that version should be much more helpful.
> > >>>>>
> > >>>>> Thanks,
> > >>>>> George
> > >>>>>
> > >>>>> On Wed, Apr 17, 2019 at 12:43 PM Ying Lin via RT
> > >>>>> <met_help at ucar.edu>
> > >>>> wrote:
> > >>>>>> Wed Apr 17 12:43:02 2019: Request 89789 was acted upon.
> > >>>>>> Transaction: Ticket created by ying.lin at noaa.gov
> > >>>>>> Queue: met_help
> > >>>>>> Subject: Using FCST_PCP_COMBINE in METplus
> > >>>>>> Owner: Nobody
> > >>>>>> Requestors: ying.lin at noaa.gov
> > >>>>>> Status: new
> > >>>>>> Ticket <URL:
> > >> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789
> > >>>>>> Hello. I haven't been able to get the FCST_PCP_COMBINE
> > >>>>>> feature to
> > >> work
> > >>>>>> in METplus, running out of ideas and hope you can help.
> > >>>>>>
> > >>>>>> I've put some examples in
> > >>>>>> https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/
> > >>>>>> (current
> > >>>> test
> > >>>>>> directory:
> > >>>>>> gyre:/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/test).
> > >>>>>> test_fv3gfs_24h.ksh is the simple test script. The goal is
to
> > >> compute a
> > >>>>>> 24h verification ending at 12Z 20190415.
> > >>>>>>
> > >>>>>> 'test1' did not need FCST_PCP_COMBINE - the config file
> > >>>>>> fv3gfs_24h.conf.test1 simply had LEAD_SEQ set to '24',
> > >>>>>> FCST_PCP_COMBINE_RUN was not set. The FV3GFS had two
precip
> > buckets,
> > >> 6h
> > >>>>>> and continuously-accumulating, and METplus easily found the
> > >>>>>> 24h
> > >> forecast
> > >>>>>> accumulation and went on with the grid_stat.
> > >>>>>>
> > >>>>>> 'test2' had 'LEAD_SEQ = 36', and I added the following in
the
> > >>>>>> config
> > >>>>>> (
> > >>>>>>
> > >>
> >
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test2
> > >>>>>> ):
> > >>>>>> FCST_PCP_COMBINE_RUN = True
> > >>>>>> FCST_PCP_COMBINE_INPUT_DATATYPE = GRIB
> > >>>>>> FCST_PCP_COMBINE_INPUT_DIR =
> > /gpfs/dell1/nco/ops/com/gfs/para
> > >>>>>> FCST_PCP_COMBINE_OUTPUT_DIR = {OUTPUT_BASE}/qpf_grib/bucket
> > >>>>>> FCST_GRID_STAT_INPUT_DIR = {OUTPUT_BASE}/qpf_grib/bucket
> > >>>>>> FCST_MAX_FORECAST = 36
> > >>>>>> FCST_INIT_INTERVAL = 6
> > >>>>>> FCST_LEVEL = 36
> > >>>>>> Here the goal was to see if I could get METplus to subtract
> > >>>>>> the
> > 00-12h
> > >>>>>> accumulation from gfs.t00z.pgrb2.0p25.f012 from the 00-36h
> > >> accumulation
> > >>>>>> in gfs.t00z.pgrb2.0p25.f036. Thinking that FCST_LEVEL is
the
> > forecast
> > >>>>>> bucket length, I figured setting it to '36' and
artificially
> > >> specifying
> > >>>>>> FCST_MAX_FORECAST = 36 (instead of the full 384h) should
work
> > >>>>>> for
> > this
> > >>>>>> limited test where LEAD_SEQ is set to 36 (is there a way to
> > >>>>>> specify
> > >> that
> > >>>>>> the bucket never gets emptied?). It failed with the follow
> > >>>>>> err msg
> > >>>>>> (full message in
> > >>>>>>
> > >>>>>>
> > >>
> >
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test2
> > >>>>>> ):
> > >>>>>>
> > >>>>>> 04/16 20:01:42.754 jlog (master_metplus.py:217) CRITICAL:
> > >>>>>> master_metplus failed: ''
> > >>>>>> Traceback (most recent call last):
> > >>>>>> File
> > >>>>>> "/meso/save/Ying.Lin/metplus/ush/master_metplus.py", line
> > >> 213,
> > >>>>>> in <module>
> > >>>>>> main()
> > >>>>>> File
> > >>>>>> "/meso/save/Ying.Lin/metplus/ush/master_metplus.py", line
> > >> 170,
> > >>>>>> in main
> > >>>>>> process.run_at_time(-1, run_time)
> > >>>>>> File
> > >>>>>>
> > >>
> >
"/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/ush/reformat_gridded_wrapper.py",
> > >>>>>> line 80, in run_at_time
> > >>>>>> self.run_at_time_once(task_info, var_info, rl)
> > >>>>>> File
> > >>>>>>
> >
"/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/ush/pcp_combine_wrapper.py",
> > >>>>>> line 419, in run_at_time_once
> > >>>>>> ....
> > >>>>>>
> > >>>>>> 'test3' is an attempt from a different angle, thinking that
if
> > >>>>>> the
> > >>>>>> 'continuously running' bucket didn't work, maybe the 6-
hourly
> > >>>>>> bucket
> > >>>>>> would. In
> > >>>>>>
> > >>>>>>
> > >>
> >
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test3
> > >>>> ,
> > >>>>>> I changed LEAD_SEQ to 24, and set FCST_LEVEL = 6. It
produced
> > >>>>>> err
> > >> msgs
> > >>>>>> (
> > >>>>>>
> > >>
> >
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test3
> > >>>> )
> > >>>>>> that appeared to be the same as that from 'test2'.
> > >>>>>>
> > >>>>>> Earlier on I tried using the FCST_PCP_COMBINE feature with
the
> > >>>>>> NAM
> > >>>>>> 'CONUSNEST' output (with simple regular 6-hourly bucket)
and
> > >>>>>> got the
> > >>>>>> same err msg, so I don't think FV3GFS's dual-bucket system
is
> > >>>>>> the
> > >> cause.
> > >>>>>> Also posted on
> > >>>>>> https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/
are
> > >>>>>> the
> > >> two
> > >>>>>> metplus_final.conf files from both 'test2' and 'test3'.
> > >>>>>>
> > >>>>>> I found in
> > >>>>>>
> > >>
> > /global/noscrub/Julie.Prestopnik/METplus/METplus-
> > 2.0.4/parm/use_cases/qpf/examples/
> > >>>>>> that hrefmean-vs-mrms-qpe.conf and nationalblend-vs-mrms-
> > >>>>>> qpe.conf
> > had
> > >>>>>> "FCST_PCP_COMBINE_RUN = True", that's where I borrowed the
> > >>>>>> FCST_PCP_COMBINE_* set up in my fv3gfs_24h.conf.test2 and
> > >>>>>> fv3gfs_24h.conf.test3. Haven't been able to run the
hrefmean
> > >>>>>> and
> > >>>>>> nationalblend tests myself however - got err msg of "
> > >>>>>> CRITICAL:
> > >>>>>> master_metplus failed: No option 'GEMPAKTOCF_CLASSPATH' in
> > >>>>>> section:
> > >>>>>> 'exe'" from both attempts.
> > >>>>>>
> > >>>>>> Thank you for looking into this -
> > >>>>>>
> > >>>>>> Ying
> > >>>>>>
> > >>>>>> --
> > >>>>>> Ying Lin
> > >>>>>> NCEP/EMC/Verification, Post-processing and Product
Generation
> > >>>>>> Branch
> > >>>>>> NCWCP Cubicle No. 2015
> > >>>>>> Ying.Lin at noaa.gov
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>> --
> > >>>> Ying Lin
> > >>>> NCEP/EMC/Verification, Post-processing and Product Generation
> > >>>> Branch
> > >>>> NCWCP Cubicle No. 2015
> > >>>> Ying.Lin at noaa.gov
> > >>>>
> > >>>>
> > >>>>
> > >>>>
> > >> --
> > >> Ying Lin
> > >> NCEP/EMC/Verification, Post-processing and Product Generation
> > >> Branch
> > >> NCWCP Cubicle No. 2015
> > >> Ying.Lin at noaa.gov
> > >>
> > >>
> > >>
> > >>
> >
> > --
> > Ying Lin
> > NCEP/EMC/Verification, Post-processing and Product Generation
Branch
> > NCWCP Cubicle No. 2015
> > Ying.Lin at noaa.gov
> >
> >
> >
> >
------------------------------------------------
Subject: Using FCST_PCP_COMBINE in METplus
From: Ying Lin
Time: Thu May 30 11:41:16 2019
Hi George,
Thanks for following up! The sample case in
parm/use_cases/grid_to_grid/examples/precip_continuous.conf seems to
"subtract" correctly now (it was off by 24h in v2.0.4). The
precip_continuous.conf only deals with the PcpCombine step though.
When I added the regrid_data_plane step to test a fv3gfs verif on
G212,
I got this err msg:
metplus.GridStat (config_wrapper.py:144) ERROR: Executable WGRIB2
does not exist at $WGRIB2
I thought there must have been a problem in my model config for the
v2.1
(found several actually) but when I run the use case at
parm/use_cases/qpf/examples/hrefmean-vs-qpe.conf the same thing
happened.
I'm attaching three files to this email:
1. test_hrefmean.ksh: script to run this use case
2. hrefmean.out: output for running the above script. You can see
that
after loading model grid_util (from ". ~/dot.for.metplus) there
was
"WGRIB2=/nwprod2/grib_util.v1.0.0/exec/wgrib2
export WGRIB2"; and at the end there is the err msg
"metplus.GridStat (config_wrapper.py:144) ERROR: Executable WGRIB2
does not exist at $WGRIB2"
3. metplus_final.conf from the above run
4. my "dot.for.metplus" file invoked by test_hrefmean.ksh. I
verified
manually - after "module load grib_util", $WGRIB2 does point to
/nwprod2/grib_util.v1.0.0/exec/wgrib2
5. my system.conf.wcoss invoked by test_hrefmean.ksh.
I looked back at run output from METplus-2.0.4, there was also
WGRIB2=/nwprod2/grib_util.v1.0.0/exec/wgrib2 and "export", and there
wasn't a complain about "Executable WGRIB2 does not exist at $WGRIB2".
Thanks again -
Ying
On 5/29/19 1:53 PM, George McCabe via RT wrote:
> Hi Ying,
>
> I am just following up on this ticket. Have you tried running your
case on METplus v2.1? Let me know if you run into any other issues.
>
> Thanks,
> George
>
> On Thu Apr 18 19:15:58 2019, mccabe wrote:
>> Hi Ying,
>>
>> It looks like the bug with subtract mode is in v2.0.4. It will be
>> fixed in
>> the next release. You can look at the develop branch on GitHub to
see
>> what
>> has changed in the subtract function. Have a good weekend!
>>
>> On Thu, Apr 18, 2019, 5:09 PM Ying Lin via RT <met_help at ucar.edu>
>> wrote:
>>
>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789 >
>>>
>>> Hi George,
>>>
>>> Thank you - you're right, replacing that first
>>> FCST_GRID_STAT_INPUT_TEMPLATE with FCST_PCP_COMBINE_INPUT_TEMPLATE
>>> got
>>> me past the err msg! There're still some old stuff with the way
the
>>> summing was done - similar to the odd result I mentioned in the
last
>>> email with the 2.0.4 precip_continuous.conf. I'll study the new
>>> precip_continuous.conf in the github and follow up with you next
>>> week.
>>>
>>> Thanks for your help and patience today. Have a great weekend -
>>>
>>> Ying
>>>
>>> On 4/18/19 4:25 PM, George McCabe via RT wrote:
>>>> Hi Ying,
>>>>
>>>> I am sorry, I was incorrect that the fixed precip_continuous
config
>>>> file
>>>> was updated in 2.0.4. It looks like the fix was made after that
>>>> release.
>>> It
>>>> will be in the newest release. You can see the updated file on
>>>> GitHub in
>>>> the develop branch here:
>>>>
>>>>
>>>
https://github.com/NCAR/METplus/blob/develop/parm/use_cases/grid_to_grid/examples/precip_continuous.conf
>>>> For your use case, it looks like you don't have
>>>> FCST_PCP_COMBINE_INPUT_TEMPLATE set and you have 2 for grid_stat:
>>>>
>>>> FCST_GRID_STAT_INPUT_TEMPLATE =
>>>>
>>>
gfs.{init?fmt=%Y%m%d}/{init?fmt=%H}/gfs.t{init?fmt=%H}z.pgrb2.0p25.f{lead?fmt=%HHH}
>>>> FCST_PCP_COMBINE_OUTPUT_TEMPLATE =
>>>> gfs.{valid?fmt=%Y%m%d%H}_A{level?fmt=%HHH}h
>>>> FCST_GRID_STAT_INPUT_TEMPLATE =
>>> gfs.{valid?fmt=%Y%m%d%H}_A{level?fmt=%HHH}h
>>>> Fixing this may solve your issue. Please let me know if it does
>>>> not. I
>>> have
>>>> to catch a flight this evening so I am going to stop working very
>>>> soon,
>>> but
>>>> if you reply very soon I may still be able to help you before the
>>> weekend.
>>>> Thanks,
>>>> George
>>>>
>>>> On Thu, Apr 18, 2019 at 8:13 PM Ying Lin via RT
<met_help at ucar.edu>
>>> wrote:
>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789 >
>>>>>
>>>>> Hi George,
>>>>>
>>>>> Thank you for the quick reply. I was looking at
>>>>>
>>> /gpfs/gd1/emc/global/noscrub/Julie.Prestopnik/METplus/METplus-
>>> 2.0.4/parm/use_cases/grid_to_grid/examples/precip_continuous.conf,
>>>>> which has VALID_BEG = 2018030900, VALID_END = 2018030900. Date
>>>>> stamp is
>>>>> Jan 30. Is there a "newer version of v2.0.4" available on
wcoss?
>>>>> From
>>>>> the above config it's not clear to me how METplus knows to
>>>>> subtract the
>>>>> gfs.*.f024 from the gfs.*.f048.
>>>>>
>>>>> If you have time today, could you also take a look why my simple
>>>>> test
>>>>> (conf set up to be as close to precip_continuous.conf as
possible
>>>>> -
>>>>>
>>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf)
>>>>> led to err msg in
>>>>>
>>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs.out.test5
>>> ?
>>>>> It seems that I corrected the problems you pointed out in your
>>>>> reply
>>>>> this morning, but the err msg still seems to be the same as
>>>>> yesterday's.
>>>>>
>>>>> If you don't have time today, next Monday is fine too. Have a
>>>>> good
>>>>> holiday weekend.
>>>>>
>>>>> Ying
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On 4/18/19 3:51 PM, George McCabe via RT wrote:
>>>>>> Hi Ying,
>>>>>>
>>>>>> 1) The config file I see has a valid time of 2018031000 and 48
hr
>>>>> forecast
>>>>>> lead. You may be using an old version of METplus that had this
>>>>>> set up
>>>>>> incorrectly. METplus version 2.0.4 should be correct.
>>>>>>
>>>>>> 2) pcp_combine -subtract does not actually use the FCST_LEVEL
>>>>>> variable,
>>>>> so
>>>>>> that should be removed from the use case. The code takes the
>>>>>> valid time
>>>>> and
>>>>>> subtracts the accumulation.
>>>>>>
>>>>>> 3) In the upcoming version of METplus (in a few weeks!) has a
>>>>>> good
>>> number
>>>>>> of config changes from the previous versions. For example,
>>>>>> FCST_LEVEL
>>>>> will
>>>>>> now be FCST_PCP_COMBINE_INPUT_LEVEL. These changes are designed
>>>>>> to make
>>>>> the
>>>>>> variables more clear to what they actually do. In the new
>>>>>> version,
>>> there
>>>>> is
>>>>>> now FCST_PCP_COMBINE_METHOD and OBS_PCP_COMBINE_METHOD, so you
>>>>>> will be
>>>>> able
>>>>>> to specify a different method for your forecast and analysis
>>>>>> data.
>>>>>>
>>>>>> FCST_PCP_COMBINE_RUN = True
>>>>>> FCST_PCP_COMBINE_METHOD = SUBTRACT
>>>>>>
>>>>>> OBS_PCP_COMBINE_RUN = True
>>>>>> OBS_PCP_COMBINE_METHOD = SUM
>>>>>>
>>>>>> This functionality has not been thoroughly tested as we haven't
>>>>>> had any
>>>>>> example use cases that need to do this so far, so please let me
>>>>>> know if
>>>>> it
>>>>>> does not work the way you expect and I will fix it.
>>>>>>
>>>>>> Just to let you know, I will be out of the office tomorrow
>>>>>> (Friday)
>>> and I
>>>>>> wont have access to email. Feel free to send any follow-up
emails
>>>>>> to me
>>>>> and
>>>>>> I will get back to you on Monday.
>>>>>>
>>>>>> Thanks,
>>>>>> George
>>>>>>
>>>>>> On Thu, Apr 18, 2019 at 7:38 PM Ying Lin via RT
>>>>>> <met_help at ucar.edu>
>>>>> wrote:
>>>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789
>
>>>>>>>
>>>>>>> Thank you very much George, this is very helpful.
>>>>>>>
>>>>>>> About the example in
>>>>>>>
parm/use_cases/grid_to_grid/examples/precip_continuous.conf
>>>>>>> I tried running it and it seems to have run through the
>>>>>>> pcp_combine
>>> step
>>>>>>> fine: it did this:
>>>>>>> /global/noscrub/Julie.Prestopnik/met/8.0/bin/pcp_combine
>>> -subtract
>>> /global/noscrub/Julie.Prestopnik/METplus/METplus-
>>>
2.0.4_sample_data/grid_to_grid/prfv3rt1/20180308/gfs.t00z.pgrb.1p00.f048
>>>>>>> 48
>>>>>>>
>>> /global/noscrub/Julie.Prestopnik/METplus/METplus-
>>>
2.0.4_sample_data/grid_to_grid/prfv3rt1/20180308/gfs.t00z.pgrb.1p00.f024
>>>>>>> 24
>>>>>>>
/meso/noscrub/Ying.Lin/metplus.out/gfs/bucket/gfs.2018030900_A024h
>>>>>>>
>>>>>>> Several questions about the example:
>>>>>>> 1) The precip_continuous.conf has
>>>>>>> LEAD_SEQ = 24
>>>>>>> I thought it meant forecast lead of 24h, e.g. look for 00-24h
>>>>>>> forecast
>>>>>>> valid at 2018030900 (given in the config). If so, why
doesn't
>>>>>>> it
>>>>>>> simply use gfs.t00z.pgrb.1p00.f024, which already have the 00-
>>>>>>> 24h
>>>>>>> forecast? If it subtracts gfs.t00z.pgrb.1p00.f024 from
>>>>>>> gfs.t00z.pgrb.1p00.f048, the resulting file would have 24-48h
>>>>>>> forecast
>>>>>>> valid at 2018031000. I did
>>>>>>> ncdump -h
>>>>>>>
/meso/noscrub/Ying.Lin/metplus.out/gfs/bucket/gfs.2018030900_A024h
>>>>>>>
>>>>>>> and got
>>>>>>> APCP_24:init_time = "20180308_000000" ;
>>>>>>> APCP_24:init_time_ut = "1520467200" ;
>>>>>>> APCP_24:valid_time = "20180310_000000" ;
>>>>>>> APCP_24:valid_time_ut = "1520640000" ;
>>>>>>> APCP_24:accum_time = "240000" ;
>>>>>>> it seems to mean that the file contains APCP_24h with initial
>>>>>>> time of
>>>>>>> 00z 03/08 and the valid time is 00Z 03/10 (that'll be APCP_48h
>>>>>>> instead
>>>>>>> of 24h?)
>>>>>>>
>>>>>>> 2) The precip_continuous.conf has
>>>>>>> FCST_LEVEL = 6
>>>>>>> I thought FCST_LEVEL is the forecast bucket length. In the
>>>>>>> sample GFS
>>>>>>> files, there isn't a 6-hourly bucket - gfs.t00z.pgrb.1p00.f024
>>> contains
>>>>>>> 00-24h accumulation and gfs.t00z.pgrb.1p00.f048 contains 00-
48h
>>>>>>> accumulation. Does FCST_LEVEL mean something other than the
>>>>>>> forecast
>>>>>>> bucket length?
>>>>>>>
>>>>>>> 3) precip_continuous.conf has
>>>>>>> PCP_COMBINE_METHOD = SUBTRACT
>>>>>>> which should work for the example since the verifying analysis
>>>>>>> is the
>>>>>>> 24h CCPA file. But what if I need to use 'SUBTRACT' for the
>>>>>>> forecast
>>>>>>> field but 'SUM' for the analysis field (e.g. summing up 6h
CCPAs
>>>>>>> into
>>> a
>>>>>>> 24h total)? At one point I tried
>>>>>>> PCP_COMBINE_METHOD = SUBTRACT, SUM
>>>>>>> and got
>>>>>>> ERROR: Invalid PCP_COMBINE_METHOD specified
>>>>>>>
>>>>>>> After several failed attempts getting my own config file to
>>>>>>> work, I
>>>>>>> starting from precip_continuous.conf (which worked through the
>>>>>>> PcpCombine step; I think that's the only step it runs since
the
>>>>>>> config
>>>>>>> has only PcpCombine in the PROCESS_LIST) and tried to set a
>>>>>>> config
>>> file
>>>>>>> as close to precip_continuous.conf as possible
>>>>>>>
>>>
(/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/parm/ylin_config/models/fv3gfs_24h.conf,
>>>>>>> also uploaded to
>>>>>>>
>>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf
>>>>> ).
>>>>>>> The resulting err msg:
>>>>>>>
>>>>>>> 04/18 18:35:33.912 jlog (master_metplus.py:217) CRITICAL:
>>>>>>> master_metplus failed: ''
>>>>>>> Traceback (most recent call last):
>>>>>>> File "/meso/save/Ying.Lin/metplus/ush/master_metplus.py", line
>>> 213,
>>>>>>> in <module>
>>>>>>> main()
>>>>>>> File
"/meso/save/Ying.Lin/metplus/ush/master_metplus.py",
>>>>>>> line
>>> 170,
>>>>>>> in main
>>>>>>> process.run_at_time(-1, run_time)
>>>>>>> File
>>>>>>>
>>>
"/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/ush/reformat_gridded_wrapper.py",
>>>>>>> line 80, in run_at_time
>>>>>>>
>>>>>>> Full output file:
>>>>>>>
>>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs.out.test5
>>>>> ;
>>>>>>> metplus_final.conf:
>>>>>>>
>>>>>>>
>>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/metplus_final.conf
>>>>>>> The quoted err msg above appears to the the same as in my
email
>>>>>>> yesterday, so there must still be something wrong with the
>>>>>>> format of
>>> my
>>>>>>> config, though I think I eliminated the problems you pointed
out
>>>>>>> in
>>> your
>>>>>>> reply. There wouldn't be a ccpa_conus_1.0d_* of the valid day
I
>>>>>>> specified, but I think the processing failed way before that
>>>>>>> point.
>>>>>>>
>>>>>>> Thank you again -
>>>>>>>
>>>>>>> Ying
>>>>>>>
>>>>>>>
>>>>>>> On 4/17/19 3:32 PM, George McCabe via RT wrote:
>>>>>>>> Hi Ying,
>>>>>>>>
>>>>>>>> You will need to set FCST_PCP_COMBINE_METHOD = SUBTRACT to
>>>>>>>> config
>>>>> METplus
>>>>>>>> to subtract accumulations files to get the accumulation you
>>>>>>>> desire.
>>> The
>>>>>>>> following use case has an example of this configuration:
>>>>>>>>
>>>>>>>> parm/use_cases/grid_to_grid/examples/precip_continous.conf
>>>>>>>>
>>>>>>>> I also noticed that your config file has the following items
in
>>>>>>>> the
>>>>>>>> [config] section:
>>>>>>>>
>>>>>>>> FCST_PCP_COMBINE_INPUT_DIR = /gpfs/dell1/nco/ops/com/gfs/para
>>>>>>>> FCST_PCP_COMBINE_OUTPUT_DIR = {OUTPUT_BASE}/qpf_grib/bucket
>>>>>>>> FCST_GRID_STAT_INPUT_DIR = {OUTPUT_BASE}/qpf_grib/bucket
>>>>>>>>
>>>>>>>> You should move these to the [dir] section.
>>>>>>>>
>>>>>>>> Please let me know if you continue to have issues. The error
>>>>>>>> message
>>>>> you
>>>>>>>> provided has more information at the bottom that was not
>>>>>>>> included.
>>> The
>>>>>>> next
>>>>>>>> release of METplus contains more extensive error checking so
>>>>>>>> the
>>> error
>>>>>>>> message from that version should be much more helpful.
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> George
>>>>>>>>
>>>>>>>> On Wed, Apr 17, 2019 at 12:43 PM Ying Lin via RT
>>>>>>>> <met_help at ucar.edu>
>>>>>>> wrote:
>>>>>>>>> Wed Apr 17 12:43:02 2019: Request 89789 was acted upon.
>>>>>>>>> Transaction: Ticket created by ying.lin at noaa.gov
>>>>>>>>> Queue: met_help
>>>>>>>>> Subject: Using FCST_PCP_COMBINE in METplus
>>>>>>>>> Owner: Nobody
>>>>>>>>> Requestors: ying.lin at noaa.gov
>>>>>>>>> Status: new
>>>>>>>>> Ticket <URL:
>>>>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789
>>>>>>>>> Hello. I haven't been able to get the FCST_PCP_COMBINE
>>>>>>>>> feature to
>>>>> work
>>>>>>>>> in METplus, running out of ideas and hope you can help.
>>>>>>>>>
>>>>>>>>> I've put some examples in
>>>>>>>>> https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/
>>>>>>>>> (current
>>>>>>> test
>>>>>>>>> directory:
>>>>>>>>> gyre:/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/test).
>>>>>>>>> test_fv3gfs_24h.ksh is the simple test script. The goal is
to
>>>>> compute a
>>>>>>>>> 24h verification ending at 12Z 20190415.
>>>>>>>>>
>>>>>>>>> 'test1' did not need FCST_PCP_COMBINE - the config file
>>>>>>>>> fv3gfs_24h.conf.test1 simply had LEAD_SEQ set to '24',
>>>>>>>>> FCST_PCP_COMBINE_RUN was not set. The FV3GFS had two precip
>>> buckets,
>>>>> 6h
>>>>>>>>> and continuously-accumulating, and METplus easily found the
>>>>>>>>> 24h
>>>>> forecast
>>>>>>>>> accumulation and went on with the grid_stat.
>>>>>>>>>
>>>>>>>>> 'test2' had 'LEAD_SEQ = 36', and I added the following in
the
>>>>>>>>> config
>>>>>>>>> (
>>>>>>>>>
>>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test2
>>>>>>>>> ):
>>>>>>>>> FCST_PCP_COMBINE_RUN = True
>>>>>>>>> FCST_PCP_COMBINE_INPUT_DATATYPE = GRIB
>>>>>>>>> FCST_PCP_COMBINE_INPUT_DIR =
>>> /gpfs/dell1/nco/ops/com/gfs/para
>>>>>>>>> FCST_PCP_COMBINE_OUTPUT_DIR = {OUTPUT_BASE}/qpf_grib/bucket
>>>>>>>>> FCST_GRID_STAT_INPUT_DIR = {OUTPUT_BASE}/qpf_grib/bucket
>>>>>>>>> FCST_MAX_FORECAST = 36
>>>>>>>>> FCST_INIT_INTERVAL = 6
>>>>>>>>> FCST_LEVEL = 36
>>>>>>>>> Here the goal was to see if I could get METplus to subtract
>>>>>>>>> the
>>> 00-12h
>>>>>>>>> accumulation from gfs.t00z.pgrb2.0p25.f012 from the 00-36h
>>>>> accumulation
>>>>>>>>> in gfs.t00z.pgrb2.0p25.f036. Thinking that FCST_LEVEL is
the
>>> forecast
>>>>>>>>> bucket length, I figured setting it to '36' and artificially
>>>>> specifying
>>>>>>>>> FCST_MAX_FORECAST = 36 (instead of the full 384h) should
work
>>>>>>>>> for
>>> this
>>>>>>>>> limited test where LEAD_SEQ is set to 36 (is there a way to
>>>>>>>>> specify
>>>>> that
>>>>>>>>> the bucket never gets emptied?). It failed with the follow
>>>>>>>>> err msg
>>>>>>>>> (full message in
>>>>>>>>>
>>>>>>>>>
>>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test2
>>>>>>>>> ):
>>>>>>>>>
>>>>>>>>> 04/16 20:01:42.754 jlog (master_metplus.py:217) CRITICAL:
>>>>>>>>> master_metplus failed: ''
>>>>>>>>> Traceback (most recent call last):
>>>>>>>>> File
>>>>>>>>> "/meso/save/Ying.Lin/metplus/ush/master_metplus.py", line
>>>>> 213,
>>>>>>>>> in <module>
>>>>>>>>> main()
>>>>>>>>> File
>>>>>>>>> "/meso/save/Ying.Lin/metplus/ush/master_metplus.py", line
>>>>> 170,
>>>>>>>>> in main
>>>>>>>>> process.run_at_time(-1, run_time)
>>>>>>>>> File
>>>>>>>>>
>>>
"/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/ush/reformat_gridded_wrapper.py",
>>>>>>>>> line 80, in run_at_time
>>>>>>>>> self.run_at_time_once(task_info, var_info, rl)
>>>>>>>>> File
>>>>>>>>>
>>>
"/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/ush/pcp_combine_wrapper.py",
>>>>>>>>> line 419, in run_at_time_once
>>>>>>>>> ....
>>>>>>>>>
>>>>>>>>> 'test3' is an attempt from a different angle, thinking that
if
>>>>>>>>> the
>>>>>>>>> 'continuously running' bucket didn't work, maybe the 6-
hourly
>>>>>>>>> bucket
>>>>>>>>> would. In
>>>>>>>>>
>>>>>>>>>
>>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test3
>>>>>>> ,
>>>>>>>>> I changed LEAD_SEQ to 24, and set FCST_LEVEL = 6. It
produced
>>>>>>>>> err
>>>>> msgs
>>>>>>>>> (
>>>>>>>>>
>>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test3
>>>>>>> )
>>>>>>>>> that appeared to be the same as that from 'test2'.
>>>>>>>>>
>>>>>>>>> Earlier on I tried using the FCST_PCP_COMBINE feature with
the
>>>>>>>>> NAM
>>>>>>>>> 'CONUSNEST' output (with simple regular 6-hourly bucket) and
>>>>>>>>> got the
>>>>>>>>> same err msg, so I don't think FV3GFS's dual-bucket system
is
>>>>>>>>> the
>>>>> cause.
>>>>>>>>> Also posted on
>>>>>>>>> https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/
are
>>>>>>>>> the
>>>>> two
>>>>>>>>> metplus_final.conf files from both 'test2' and 'test3'.
>>>>>>>>>
>>>>>>>>> I found in
>>>>>>>>>
>>> /global/noscrub/Julie.Prestopnik/METplus/METplus-
>>> 2.0.4/parm/use_cases/qpf/examples/
>>>>>>>>> that hrefmean-vs-mrms-qpe.conf and nationalblend-vs-mrms-
>>>>>>>>> qpe.conf
>>> had
>>>>>>>>> "FCST_PCP_COMBINE_RUN = True", that's where I borrowed the
>>>>>>>>> FCST_PCP_COMBINE_* set up in my fv3gfs_24h.conf.test2 and
>>>>>>>>> fv3gfs_24h.conf.test3. Haven't been able to run the
hrefmean
>>>>>>>>> and
>>>>>>>>> nationalblend tests myself however - got err msg of "
>>>>>>>>> CRITICAL:
>>>>>>>>> master_metplus failed: No option 'GEMPAKTOCF_CLASSPATH' in
>>>>>>>>> section:
>>>>>>>>> 'exe'" from both attempts.
>>>>>>>>>
>>>>>>>>> Thank you for looking into this -
>>>>>>>>>
>>>>>>>>> Ying
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Ying Lin
>>>>>>>>> NCEP/EMC/Verification, Post-processing and Product
Generation
>>>>>>>>> Branch
>>>>>>>>> NCWCP Cubicle No. 2015
>>>>>>>>> Ying.Lin at noaa.gov
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>> --
>>>>>>> Ying Lin
>>>>>>> NCEP/EMC/Verification, Post-processing and Product Generation
>>>>>>> Branch
>>>>>>> NCWCP Cubicle No. 2015
>>>>>>> Ying.Lin at noaa.gov
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>> --
>>>>> Ying Lin
>>>>> NCEP/EMC/Verification, Post-processing and Product Generation
>>>>> Branch
>>>>> NCWCP Cubicle No. 2015
>>>>> Ying.Lin at noaa.gov
>>>>>
>>>>>
>>>>>
>>>>>
>>> --
>>> Ying Lin
>>> NCEP/EMC/Verification, Post-processing and Product Generation
Branch
>>> NCWCP Cubicle No. 2015
>>> Ying.Lin at noaa.gov
>>>
>>>
>>>
>>>
>
>
--
Ying Lin
NCEP/EMC/Verification, Post-processing and Product Generation Branch
NCWCP Cubicle No. 2015
Ying.Lin at noaa.gov
------------------------------------------------
Subject: Using FCST_PCP_COMBINE in METplus
From: Ying Lin
Time: Thu May 30 11:41:16 2019
#!/bin/ksh
set -x
. ~/dot.for.metplus
echo 'Actual output starts here:'
master_metplus.py \
-c ${METPLUS_PATH}/parm/use_cases/qpf/examples/hrefmean-vs-qpe.conf \
-c ${METPLUS_PATH}/yl/parm/system.conf.wcoss
------------------------------------------------
Subject: Using FCST_PCP_COMBINE in METplus
From: Ying Lin
Time: Thu May 30 11:41:16 2019
+ . /u/Ying.Lin/dot.for.metplus
+ . /usrx/local/Modules/default/init/ksh
+ [ 3.2.10 '=' '' ]
+ MODULE_VERSION_STACK=3.2.10
+ export MODULE_VERSION_STACK
+ MODULESHOME=/usrx/local/Modules/3.2.10
+ export MODULESHOME
+ [
EnvVars/1.0.1:ibmpe/1.3.0.12:lsf/9.1:ics/12.1:anaconda2/latest:hpss/4.0.1.2:GrADS/2.0.2:HDF5/1.8.9/serial:NetCDF/4.2/serial
'=' '' ]
+ [
'/usrx/local/Modules/versions:/usrx/local/Modules/$MODULE_VERSION/modulefiles:/usrx/local/modulefiles:/nwprod2/modulefiles:/nwprod2/lib/modulefiles:/nwprod/lib/modulefiles:/u/Benjamin.Blake/modulefiles'
'=' '' ]
+ module purge
+ /usrx/local/Modules/3.2.10/bin/modulecmd ksh purge
+ eval MANPATH=/usr/share/man ';export'
'MANPATH;PATH=.:/nwprod/gempak/nawips/os/linux2.6.32_x86_64/bin:/usr/lib64/qt-
3.3/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/opt/ibutils/bin:/nwprod/util/exec::/nwprod/util/ush::/u/Wesley.Ebisuzaki/bin:/u/Ying.Lin/bin:/nwprod/gempak/nawips/bin:/nwprod/gempak/nawips/scripts/ez:/nwprod/gempak/nawips/scripts/nawips:/nwprod/gempak/nawips/scripts/prnt:/nwprod/gempak/nawips/scripts/decoder:/sss/emc/meso/shared/Eric.Rogers/ddapp/exe_lnx6:/sss/emc/meso/shared/Eric.Rogers/vsdb/scripts'
';export' 'PATH;unset' 'CLASSPATH;unset' 'CPATH;unset'
'CPRO_PATH;unset' 'FPATH;unset' 'GADDIR;unset' 'GrADS_ROOT;unset'
'HDF5;unset' 'HDF5_CFLAGS;unset' 'HDF5_CXXFLAGS;unset'
'HDF5_FFLAGS;unset' 'HDF5_INCLUDE;unset' 'HDF5_LDFLAGS;unset'
'HDF5_LDFLAGS_C;unset' 'HDF5_LDFLAGS_CXX;unset' 'HDF5_LDFLAGS_F;unset'
'IDB_HOME;unset' 'INCLUDE;unset' 'INSTALL_DIR;unset'
'INTEL_LICENSE_FILE;unset' 'IPPROOT;unset' 'I_MPI_INC;unset'
'I_MPI_ROOT;unset' 'LANG;unset' 'LAUNCH;unset' 'LD_LIBRARY_PATH;unset'
'LIBRARY_PATH;unset' 'LOADEDMODULES;unset' 'LSF_BINDIR;unset'
'LSF_ENVDIR;unset' 'LSF_LIBDIR;unset' 'LSF_SERVERDIR;unset'
'MKLROOT;unset' 'MKL_TARGET_ARCH;unset' 'MP_CLOCK_SOURCE;unset'
'MP_COMPILER;unset' 'MP_CONFIG;unset' 'MP_EUILIB;unset'
'MP_MPILIB;unset' 'NETCDF;unset' 'NETCDF_CFLAGS;unset'
'NETCDF_CXXFLAGS;unset' 'NETCDF_FFLAGS;unset' 'NETCDF_INCLUDE;unset'
'NETCDF_LDFLAGS;unset' 'NETCDF_LDFLAGS_C;unset'
'NETCDF_LDFLAGS_CXX;unset' 'NETCDF_LDFLAGS_F;unset' 'NLSPATH;unset'
'OMP_NUM_THREADS;unset' 'PROD_DIR;unset' 'PYTHONPATH;unset'
'SITE;unset' 'TBBROOT;unset' 'VT_LIB_DIR;unset' 'VT_ROOT;unset'
'VT_SLIB_DIR;unset' 'XLSF_UIDDIR;unset' '_LMFILES_;'
+ MANPATH=/usr/share/man
+ export MANPATH
+
PATH=.:/nwprod/gempak/nawips/os/linux2.6.32_x86_64/bin:/usr/lib64/qt-
3.3/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/opt/ibutils/bin:/nwprod/util/exec::/nwprod/util/ush::/u/Wesley.Ebisuzaki/bin:/u/Ying.Lin/bin:/nwprod/gempak/nawips/bin:/nwprod/gempak/nawips/scripts/ez:/nwprod/gempak/nawips/scripts/nawips:/nwprod/gempak/nawips/scripts/prnt:/nwprod/gempak/nawips/scripts/decoder:/sss/emc/meso/shared/Eric.Rogers/ddapp/exe_lnx6:/sss/emc/meso/shared/Eric.Rogers/vsdb/scripts
+ export PATH
+ unset CLASSPATH
+ unset CPATH
+ unset CPRO_PATH
+ unset FPATH
+ unset GADDIR
+ unset GrADS_ROOT
+ unset HDF5
+ unset HDF5_CFLAGS
+ unset HDF5_CXXFLAGS
+ unset HDF5_FFLAGS
+ unset HDF5_INCLUDE
+ unset HDF5_LDFLAGS
+ unset HDF5_LDFLAGS_C
+ unset HDF5_LDFLAGS_CXX
+ unset HDF5_LDFLAGS_F
+ unset IDB_HOME
+ unset INCLUDE
+ unset INSTALL_DIR
+ unset INTEL_LICENSE_FILE
+ unset IPPROOT
+ unset I_MPI_INC
+ unset I_MPI_ROOT
+ unset LANG
+ unset LAUNCH
+ unset LD_LIBRARY_PATH
+ unset LIBRARY_PATH
+ unset LOADEDMODULES
+ unset LSF_BINDIR
+ unset LSF_ENVDIR
+ unset LSF_LIBDIR
+ unset LSF_SERVERDIR
+ unset MKLROOT
+ unset MKL_TARGET_ARCH
+ unset MP_CLOCK_SOURCE
+ unset MP_COMPILER
+ unset MP_CONFIG
+ unset MP_EUILIB
+ unset MP_MPILIB
+ unset NETCDF
+ unset NETCDF_CFLAGS
+ unset NETCDF_CXXFLAGS
+ unset NETCDF_FFLAGS
+ unset NETCDF_INCLUDE
+ unset NETCDF_LDFLAGS
+ unset NETCDF_LDFLAGS_C
+ unset NETCDF_LDFLAGS_CXX
+ unset NETCDF_LDFLAGS_F
+ unset NLSPATH
+ unset OMP_NUM_THREADS
+ unset PROD_DIR
+ unset PYTHONPATH
+ unset SITE
+ unset TBBROOT
+ unset VT_LIB_DIR
+ unset VT_ROOT
+ unset VT_SLIB_DIR
+ unset XLSF_UIDDIR
+ unset _LMFILES_
+ module use /global/noscrub/Julie.Prestopnik/modulefiles
+ /usrx/local/Modules/3.2.10/bin/modulecmd ksh use
/global/noscrub/Julie.Prestopnik/modulefiles
+ eval
MODULEPATH='/global/noscrub/Julie.Prestopnik/modulefiles:/usrx/local/Modules/versions:/usrx/local/Modules/\$MODULE_VERSION/modulefiles:/usrx/local/modulefiles:/nwprod2/modulefiles:/nwprod2/lib/modulefiles:/nwprod/lib/modulefiles:/u/Benjamin.Blake/modulefiles'
';export' 'MODULEPATH;'
+
MODULEPATH='/global/noscrub/Julie.Prestopnik/modulefiles:/usrx/local/Modules/versions:/usrx/local/Modules/$MODULE_VERSION/modulefiles:/usrx/local/modulefiles:/nwprod2/modulefiles:/nwprod2/lib/modulefiles:/nwprod/lib/modulefiles:/u/Benjamin.Blake/modulefiles'
+ export MODULEPATH
+ module load met/8.1
+ /usrx/local/Modules/3.2.10/bin/modulecmd ksh load met/8.1
+ eval ADVISOR_XE_2015_DIR=/usrx/local/intel/2015/advisor_xe_2015
';export' 'ADVISOR_XE_2015_DIR;BIN_DIR=bin64' ';export'
'BIN_DIR;CLASSPATH=/usrx/local/intel/2015/itac_latest/lib' ';export'
'CLASSPATH;CPATH=/usrx/local/intel/2015/ipp/include:/usrx/local/intel/2015/mkl/include:/usrx/local/intel/2015/tbb/include'
';export'
'CPATH;CPRO_PATH=/usrx/local/intel/2015/composer_xe_2015.3.187'
';export'
'CPRO_PATH;GDBSERVER_MIC=/usrx/local/intel/2015/composerxe/debugger/gdb/target/mic/bin/gdbserver'
';export'
'GDBSERVER_MIC;GDB_CROSS=/usrx/local/intel/2015/composerxe/debugger/gdb/intel64_mic/bin/gdb-
mic' ';export' 'GDB_CROSS;INCLUDE=/usrx/local/intel/2015/mkl/include'
';export'
'INCLUDE;INFOPATH=/usrx/local/intel/2015/composerxe/debugger/gdb/intel64/share/info:/usrx/local/intel/2015/composerxe/debugger/gdb/intel64_mic/share/info'
';export'
'INFOPATH;INSPECTOR_2015_DIR=/usrx/local/intel/2015/inspector_xe_2015'
';export'
'INSPECTOR_2015_DIR;INTEL_LICENSE_FILE=/usrx/local/intel/licenses'
';export'
'INTEL_LICENSE_FILE;INTEL_PYTHONHOME=/usrx/local/intel/2015/composerxe/debugger/python/intel64'
';export' 'INTEL_PYTHONHOME;IPPROOT=/usrx/local/intel/2015/ipp'
';export' 'IPPROOT;I_MPI_HYDRA_IFACE=ib0' ';export'
'I_MPI_HYDRA_IFACE;I_MPI_INC=/usrx/local/intel/2015/impi_latest/include64'
';export' 'I_MPI_INC;I_MPI_ROOT=/usrx/local/intel/2015/impi_latest'
';export' 'I_MPI_ROOT;LANGUAGE_TERRITORY=en_US' ';export'
'LANGUAGE_TERRITORY;LD_LIBRARY_PATH=/usrx/local/python/2.7.13/lib:/usrx/local/intel/2015/impi_latest/intel64/lib:/usrx/local/intel/2015/itac_latest/mic/slib:/usrx/local/intel/2015/itac_latest/intel64/slib:/usrx/local/intel/2015/composerxe/lib/intel64:/usrx/local/intel/2015/mpirt/lib/intel64:/usrx/local/intel/2015/ipp/lib/intel64:/usrx/local/intel/2015/mkl/lib/intel64:/usrx/local/intel/2015/tbb/lib/intel64/gcc4.4'
';export'
'LD_LIBRARY_PATH;LIBRARY_PATH=/usrx/local/intel/2015/composerxe/lib/intel64:/usrx/local/intel/2015/ipp/lib/intel64:/usrx/local/intel/2015/mkl/lib/intel64:/usrx/local/intel/2015/tbb/lib/intel64/gcc4.4'
';export'
'LIBRARY_PATH;LOADEDMODULES=ics/15.0.3:python/2.7.13:met/8.1'
';export'
'LOADEDMODULES;MANPATH=/usrx/local/python/2.7.13/share/man:/usrx/local/intel/2015/impi_latest/man:/usrx/local/intel/2015/itac_latest/man:/usrx/local/intel/2015/composerxe/man/en_US:/usrx/local/intel/2015/composerxe/debugger/gdb/intel64/share/man:/usr/share/man:/usrx/local/intel/2015/composerxe/debugger/gdb/intel64_mic/share/man:/usrx/local/intel/2015/inspector_xe_2015/man:/usrx/local/intel/2015/vtune_amplifier_xe_2015/man:/usrx/local/intel/2015/advisor_xe_2015/man'
';export'
'MANPATH;MIC_LD_LIBRARY_PATH=/usrx/local/intel/2015/composerxe/lib/mic:/usrx/local/intel/2015/mpirt/lib/mic:/usrx/local/intel/2015/mkl/lib/mic:/usrx/local/intel/2015/tbb/lib/mic'
';export'
'MIC_LD_LIBRARY_PATH;MIC_LIBRARY_PATH=/usrx/local/intel/2015/composerxe/lib/mic:/usrx/local/intel/2015/mpirt/lib/mic:/usrx/local/intel/2015/tbb/lib/mic'
';export' 'MIC_LIBRARY_PATH;MKLROOT=/usrx/local/intel/2015/mkl'
';export'
'MKLROOT;MPM_LAUNCHER=/usrx/local/intel/2015/composerxe/debugger/mpm/bin/start_mpm.sh'
';export'
'MPM_LAUNCHER;NLSPATH=/usrx/local/intel/2015/composerxe/lib/intel64/locale/%l_%t/%N:/usrx/local/intel/2015/ipp/lib/intel64/locale/%l_%t/%N:/usrx/local/intel/2015/mkl/lib/intel64/locale/%l_%t/%N:/usrx/local/intel/2015/composer_xe_2015.3.187/debugger/gdb/intel64_mic/share/locale/%l_%t/%N:/usrx/local/intel/2015/composer_xe_2015.3.187/debugger/gdb/intel64/share/locale/%l_%t/%N'
';export'
'NLSPATH;PATH=/global/noscrub/Julie.Prestopnik/met/8.1//bin:/global/noscrub/Julie.Prestopnik/met/8.1//external_libs/bin:/usrx/local/python/2.7.13/bin:/usrx/local/intel/2015/advisor_xe_2015/bin64:/usrx/local/intel/2015/vtune_amplifier_xe_2015/bin64:/usrx/local/intel/2015/inspector_xe_2015/bin64:/usrx/local/intel/2015/impi_latest/intel64/bin:/usrx/local/intel/2015/itac_latest/intel64/bin:/usrx/local/intel/2015/composer_xe_2015.3.187/bin/intel64:/usrx/local/intel/2015/mpirt/bin/intel64:/usrx/local/intel/2015/composerxe/debugger/gdb/intel64_mic/bin:/usrx/local/intel/2015/composerxe/debugger/gdb/intel64/bin:.:/nwprod/gempak/nawips/os/linux2.6.32_x86_64/bin:/usr/lib64/qt-
3.3/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/opt/ibutils/bin:/nwprod/util/exec::/nwprod/util/ush::/u/Wesley.Ebisuzaki/bin:/u/Ying.Lin/bin:/nwprod/gempak/nawips/bin:/nwprod/gempak/nawips/scripts/ez:/nwprod/gempak/nawips/scripts/nawips:/nwprod/gempak/nawips/scripts/prnt:/nwprod/gempak/nawips/scripts/decoder:/sss/emc/meso/shared/Eric.Rogers/ddapp/exe_lnx6:/sss/emc/meso/shared/Eric.Rogers/vsdb/scripts'
';export' 'PATH;PROD_DIR=/usrx/local/intel/2015/composerxe' ';export'
'PROD_DIR;ROOTPATH=/usrx/local/intel/2015' ';export'
'ROOTPATH;SCRIPTPATH=/usrx/local/intel/2015' ';export'
'SCRIPTPATH;TBBROOT=/usrx/local/intel/2015/tbb' ';export'
'TBBROOT;VTUNE_AMPLIFIER_XE_2015_DIR=/usrx/local/intel/2015/vtune_amplifier_xe_2015'
';export' 'VTUNE_AMPLIFIER_XE_2015_DIR;VT_ADD_LIBS=-ldwarf\' '-lelf\'
'-lvtunwind\' '-lnsl\' '-lm\' '-ldl\' -lpthread ';export'
'VT_ADD_LIBS;VT_ARCH=intel64' ';export'
'VT_ARCH;VT_LIB_DIR=/usrx/local/intel/2015/itac_latest/lib' ';export'
'VT_LIB_DIR;VT_MIC_SLIB_DIR=/usrx/local/intel/2015/itac_latest/mic/slib'
';export' 'VT_MIC_SLIB_DIR;VT_MPI=impi4' ';export'
'VT_MPI;VT_ROOT=/usrx/local/intel/2015/itac_latest' ';export'
'VT_ROOT;VT_SLIB_DIR=/usrx/local/intel/2015/itac_latest/intel64/slib'
';export'
'VT_SLIB_DIR;_LMFILES_=/usrx/local/modulefiles/ics/15.0.3:/usrx/local/modulefiles/python/2.7.13:/global/noscrub/Julie.Prestopnik/modulefiles/met/8.1'
';export' '_LMFILES_;'
+ ADVISOR_XE_2015_DIR=/usrx/local/intel/2015/advisor_xe_2015
+ export ADVISOR_XE_2015_DIR
+ BIN_DIR=bin64
+ export BIN_DIR
+ CLASSPATH=/usrx/local/intel/2015/itac_latest/lib
+ export CLASSPATH
+
CPATH=/usrx/local/intel/2015/ipp/include:/usrx/local/intel/2015/mkl/include:/usrx/local/intel/2015/tbb/include
+ export CPATH
+ CPRO_PATH=/usrx/local/intel/2015/composer_xe_2015.3.187
+ export CPRO_PATH
+
GDBSERVER_MIC=/usrx/local/intel/2015/composerxe/debugger/gdb/target/mic/bin/gdbserver
+ export GDBSERVER_MIC
+
GDB_CROSS=/usrx/local/intel/2015/composerxe/debugger/gdb/intel64_mic/bin/gdb-
mic
+ export GDB_CROSS
+ INCLUDE=/usrx/local/intel/2015/mkl/include
+ export INCLUDE
+
INFOPATH=/usrx/local/intel/2015/composerxe/debugger/gdb/intel64/share/info:/usrx/local/intel/2015/composerxe/debugger/gdb/intel64_mic/share/info
+ export INFOPATH
+ INSPECTOR_2015_DIR=/usrx/local/intel/2015/inspector_xe_2015
+ export INSPECTOR_2015_DIR
+ INTEL_LICENSE_FILE=/usrx/local/intel/licenses
+ export INTEL_LICENSE_FILE
+
INTEL_PYTHONHOME=/usrx/local/intel/2015/composerxe/debugger/python/intel64
+ export INTEL_PYTHONHOME
+ IPPROOT=/usrx/local/intel/2015/ipp
+ export IPPROOT
+ I_MPI_HYDRA_IFACE=ib0
+ export I_MPI_HYDRA_IFACE
+ I_MPI_INC=/usrx/local/intel/2015/impi_latest/include64
+ export I_MPI_INC
+ I_MPI_ROOT=/usrx/local/intel/2015/impi_latest
+ export I_MPI_ROOT
+ LANGUAGE_TERRITORY=en_US
+ export LANGUAGE_TERRITORY
+
LD_LIBRARY_PATH=/usrx/local/python/2.7.13/lib:/usrx/local/intel/2015/impi_latest/intel64/lib:/usrx/local/intel/2015/itac_latest/mic/slib:/usrx/local/intel/2015/itac_latest/intel64/slib:/usrx/local/intel/2015/composerxe/lib/intel64:/usrx/local/intel/2015/mpirt/lib/intel64:/usrx/local/intel/2015/ipp/lib/intel64:/usrx/local/intel/2015/mkl/lib/intel64:/usrx/local/intel/2015/tbb/lib/intel64/gcc4.4
+ export LD_LIBRARY_PATH
+
LIBRARY_PATH=/usrx/local/intel/2015/composerxe/lib/intel64:/usrx/local/intel/2015/ipp/lib/intel64:/usrx/local/intel/2015/mkl/lib/intel64:/usrx/local/intel/2015/tbb/lib/intel64/gcc4.4
+ export LIBRARY_PATH
+ LOADEDMODULES=ics/15.0.3:python/2.7.13:met/8.1
+ export LOADEDMODULES
+
MANPATH=/usrx/local/python/2.7.13/share/man:/usrx/local/intel/2015/impi_latest/man:/usrx/local/intel/2015/itac_latest/man:/usrx/local/intel/2015/composerxe/man/en_US:/usrx/local/intel/2015/composerxe/debugger/gdb/intel64/share/man:/usr/share/man:/usrx/local/intel/2015/composerxe/debugger/gdb/intel64_mic/share/man:/usrx/local/intel/2015/inspector_xe_2015/man:/usrx/local/intel/2015/vtune_amplifier_xe_2015/man:/usrx/local/intel/2015/advisor_xe_2015/man
+ export MANPATH
+
MIC_LD_LIBRARY_PATH=/usrx/local/intel/2015/composerxe/lib/mic:/usrx/local/intel/2015/mpirt/lib/mic:/usrx/local/intel/2015/mkl/lib/mic:/usrx/local/intel/2015/tbb/lib/mic
+ export MIC_LD_LIBRARY_PATH
+
MIC_LIBRARY_PATH=/usrx/local/intel/2015/composerxe/lib/mic:/usrx/local/intel/2015/mpirt/lib/mic:/usrx/local/intel/2015/tbb/lib/mic
+ export MIC_LIBRARY_PATH
+ MKLROOT=/usrx/local/intel/2015/mkl
+ export MKLROOT
+
MPM_LAUNCHER=/usrx/local/intel/2015/composerxe/debugger/mpm/bin/start_mpm.sh
+ export MPM_LAUNCHER
+
NLSPATH=/usrx/local/intel/2015/composerxe/lib/intel64/locale/%l_%t/%N:/usrx/local/intel/2015/ipp/lib/intel64/locale/%l_%t/%N:/usrx/local/intel/2015/mkl/lib/intel64/locale/%l_%t/%N:/usrx/local/intel/2015/composer_xe_2015.3.187/debugger/gdb/intel64_mic/share/locale/%l_%t/%N:/usrx/local/intel/2015/composer_xe_2015.3.187/debugger/gdb/intel64/share/locale/%l_%t/%N
+ export NLSPATH
+
PATH=/global/noscrub/Julie.Prestopnik/met/8.1//bin:/global/noscrub/Julie.Prestopnik/met/8.1//external_libs/bin:/usrx/local/python/2.7.13/bin:/usrx/local/intel/2015/advisor_xe_2015/bin64:/usrx/local/intel/2015/vtune_amplifier_xe_2015/bin64:/usrx/local/intel/2015/inspector_xe_2015/bin64:/usrx/local/intel/2015/impi_latest/intel64/bin:/usrx/local/intel/2015/itac_latest/intel64/bin:/usrx/local/intel/2015/composer_xe_2015.3.187/bin/intel64:/usrx/local/intel/2015/mpirt/bin/intel64:/usrx/local/intel/2015/composerxe/debugger/gdb/intel64_mic/bin:/usrx/local/intel/2015/composerxe/debugger/gdb/intel64/bin:.:/nwprod/gempak/nawips/os/linux2.6.32_x86_64/bin:/usr/lib64/qt-
3.3/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/opt/ibutils/bin:/nwprod/util/exec::/nwprod/util/ush::/u/Wesley.Ebisuzaki/bin:/u/Ying.Lin/bin:/nwprod/gempak/nawips/bin:/nwprod/gempak/nawips/scripts/ez:/nwprod/gempak/nawips/scripts/nawips:/nwprod/gempak/nawips/scripts/prnt:/nwprod/gempak/nawips/scripts/decoder:/sss/emc/meso/shared/Eric.Rogers/ddapp/exe_lnx6:/sss/emc/meso/shared/Eric.Rogers/vsdb/scripts
+ export PATH
+ PROD_DIR=/usrx/local/intel/2015/composerxe
+ export PROD_DIR
+ ROOTPATH=/usrx/local/intel/2015
+ export ROOTPATH
+ SCRIPTPATH=/usrx/local/intel/2015
+ export SCRIPTPATH
+ TBBROOT=/usrx/local/intel/2015/tbb
+ export TBBROOT
+
VTUNE_AMPLIFIER_XE_2015_DIR=/usrx/local/intel/2015/vtune_amplifier_xe_2015
+ export VTUNE_AMPLIFIER_XE_2015_DIR
+ VT_ADD_LIBS='-ldwarf -lelf -lvtunwind -lnsl -lm -ldl -lpthread'
+ export VT_ADD_LIBS
+ VT_ARCH=intel64
+ export VT_ARCH
+ VT_LIB_DIR=/usrx/local/intel/2015/itac_latest/lib
+ export VT_LIB_DIR
+ VT_MIC_SLIB_DIR=/usrx/local/intel/2015/itac_latest/mic/slib
+ export VT_MIC_SLIB_DIR
+ VT_MPI=impi4
+ export VT_MPI
+ VT_ROOT=/usrx/local/intel/2015/itac_latest
+ export VT_ROOT
+ VT_SLIB_DIR=/usrx/local/intel/2015/itac_latest/intel64/slib
+ export VT_SLIB_DIR
+
_LMFILES_=/usrx/local/modulefiles/ics/15.0.3:/usrx/local/modulefiles/python/2.7.13:/global/noscrub/Julie.Prestopnik/modulefiles/met/8.1
+ export _LMFILES_
+ module load nco
+ /usrx/local/Modules/3.2.10/bin/modulecmd ksh load nco
+ eval LOADEDMODULES=ics/15.0.3:python/2.7.13:met/8.1:nco/4.2.4
';export'
'LOADEDMODULES;MANPATH=/usrx/local/nco/4.2.4/share/man:/usrx/local/python/2.7.13/share/man:/usrx/local/intel/2015/impi_latest/man:/usrx/local/intel/2015/itac_latest/man:/usrx/local/intel/2015/composerxe/man/en_US:/usrx/local/intel/2015/composerxe/debugger/gdb/intel64/share/man:/usr/share/man:/usrx/local/intel/2015/composerxe/debugger/gdb/intel64_mic/share/man:/usrx/local/intel/2015/inspector_xe_2015/man:/usrx/local/intel/2015/vtune_amplifier_xe_2015/man:/usrx/local/intel/2015/advisor_xe_2015/man'
';export' 'MANPATH;NCO_INCLUDE=-I/usrx/local/nco/4.2.4/include'
';export' 'NCO_INCLUDE;NCO_LDFLAGS_C=-L/usrx/local/nco/4.2.4/lib'
';export' 'NCO_LDFLAGS_C;NCO_LDFLAGS_CXX=-L/usrx/local/nco/4.2.4/lib'
';export'
'NCO_LDFLAGS_CXX;PATH=/usrx/local/nco/4.2.4/bin:/global/noscrub/Julie.Prestopnik/met/8.1//bin:/global/noscrub/Julie.Prestopnik/met/8.1//external_libs/bin:/usrx/local/python/2.7.13/bin:/usrx/local/intel/2015/advisor_xe_2015/bin64:/usrx/local/intel/2015/vtune_amplifier_xe_2015/bin64:/usrx/local/intel/2015/inspector_xe_2015/bin64:/usrx/local/intel/2015/impi_latest/intel64/bin:/usrx/local/intel/2015/itac_latest/intel64/bin:/usrx/local/intel/2015/composer_xe_2015.3.187/bin/intel64:/usrx/local/intel/2015/mpirt/bin/intel64:/usrx/local/intel/2015/composerxe/debugger/gdb/intel64_mic/bin:/usrx/local/intel/2015/composerxe/debugger/gdb/intel64/bin:.:/nwprod/gempak/nawips/os/linux2.6.32_x86_64/bin:/usr/lib64/qt-
3.3/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/opt/ibutils/bin:/nwprod/util/exec::/nwprod/util/ush::/u/Wesley.Ebisuzaki/bin:/u/Ying.Lin/bin:/nwprod/gempak/nawips/bin:/nwprod/gempak/nawips/scripts/ez:/nwprod/gempak/nawips/scripts/nawips:/nwprod/gempak/nawips/scripts/prnt:/nwprod/gempak/nawips/scripts/decoder:/sss/emc/meso/shared/Eric.Rogers/ddapp/exe_lnx6:/sss/emc/meso/shared/Eric.Rogers/vsdb/scripts'
';export'
'PATH;_LMFILES_=/usrx/local/modulefiles/ics/15.0.3:/usrx/local/modulefiles/python/2.7.13:/global/noscrub/Julie.Prestopnik/modulefiles/met/8.1:/usrx/local/modulefiles/nco/4.2.4'
';export' '_LMFILES_;'
+ LOADEDMODULES=ics/15.0.3:python/2.7.13:met/8.1:nco/4.2.4
+ export LOADEDMODULES
+
MANPATH=/usrx/local/nco/4.2.4/share/man:/usrx/local/python/2.7.13/share/man:/usrx/local/intel/2015/impi_latest/man:/usrx/local/intel/2015/itac_latest/man:/usrx/local/intel/2015/composerxe/man/en_US:/usrx/local/intel/2015/composerxe/debugger/gdb/intel64/share/man:/usr/share/man:/usrx/local/intel/2015/composerxe/debugger/gdb/intel64_mic/share/man:/usrx/local/intel/2015/inspector_xe_2015/man:/usrx/local/intel/2015/vtune_amplifier_xe_2015/man:/usrx/local/intel/2015/advisor_xe_2015/man
+ export MANPATH
+ NCO_INCLUDE=-I/usrx/local/nco/4.2.4/include
+ export NCO_INCLUDE
+ NCO_LDFLAGS_C=-L/usrx/local/nco/4.2.4/lib
+ export NCO_LDFLAGS_C
+ NCO_LDFLAGS_CXX=-L/usrx/local/nco/4.2.4/lib
+ export NCO_LDFLAGS_CXX
+
PATH=/usrx/local/nco/4.2.4/bin:/global/noscrub/Julie.Prestopnik/met/8.1//bin:/global/noscrub/Julie.Prestopnik/met/8.1//external_libs/bin:/usrx/local/python/2.7.13/bin:/usrx/local/intel/2015/advisor_xe_2015/bin64:/usrx/local/intel/2015/vtune_amplifier_xe_2015/bin64:/usrx/local/intel/2015/inspector_xe_2015/bin64:/usrx/local/intel/2015/impi_latest/intel64/bin:/usrx/local/intel/2015/itac_latest/intel64/bin:/usrx/local/intel/2015/composer_xe_2015.3.187/bin/intel64:/usrx/local/intel/2015/mpirt/bin/intel64:/usrx/local/intel/2015/composerxe/debugger/gdb/intel64_mic/bin:/usrx/local/intel/2015/composerxe/debugger/gdb/intel64/bin:.:/nwprod/gempak/nawips/os/linux2.6.32_x86_64/bin:/usr/lib64/qt-
3.3/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/opt/ibutils/bin:/nwprod/util/exec::/nwprod/util/ush::/u/Wesley.Ebisuzaki/bin:/u/Ying.Lin/bin:/nwprod/gempak/nawips/bin:/nwprod/gempak/nawips/scripts/ez:/nwprod/gempak/nawips/scripts/nawips:/nwprod/gempak/nawips/scripts/prnt:/nwprod/gempak/nawips/scripts/decoder:/sss/emc/meso/shared/Eric.Rogers/ddapp/exe_lnx6:/sss/emc/meso/shared/Eric.Rogers/vsdb/scripts
+ export PATH
+
_LMFILES_=/usrx/local/modulefiles/ics/15.0.3:/usrx/local/modulefiles/python/2.7.13:/global/noscrub/Julie.Prestopnik/modulefiles/met/8.1:/usrx/local/modulefiles/nco/4.2.4
+ export _LMFILES_
+ module load grib_util
+ /usrx/local/Modules/3.2.10/bin/modulecmd ksh load grib_util
+ eval CNVGRIB=/nwprod2/grib_util.v1.0.0/exec/cnvgrib ';export'
'CNVGRIB;COPYGB=/nwprod2/grib_util.v1.0.0/exec/copygb' ';export'
'COPYGB;COPYGB2=/nwprod2/grib_util.v1.0.0/exec/copygb2' ';export'
'COPYGB2;DEGRIB2=/nwprod2/grib_util.v1.0.0/exec/degrib2' ';export'
'DEGRIB2;GRB2INDEX=/nwprod2/grib_util.v1.0.0/exec/grb2index' ';export'
'GRB2INDEX;GRBINDEX=/nwprod2/grib_util.v1.0.0/exec/grbindex' ';export'
'GRBINDEX;GRIB2GRIB=/nwprod2/grib_util.v1.0.0/exec/grib2grib'
';export'
'GRIB2GRIB;LOADEDMODULES=ics/15.0.3:python/2.7.13:met/8.1:nco/4.2.4:grib_util/v1.0.0'
';export'
'LOADEDMODULES;TOCGRIB=/nwprod2/grib_util.v1.0.0/exec/tocgrib'
';export' 'TOCGRIB;TOCGRIB2=/nwprod2/grib_util.v1.0.0/exec/tocgrib2'
';export'
'TOCGRIB2;TOCGRIB2SUPER=/nwprod2/grib_util.v1.0.0/exec/tocgrib2super'
';export' 'TOCGRIB2SUPER;WGRIB=/nwprod2/grib_util.v1.0.0/exec/wgrib'
';export' 'WGRIB;WGRIB2=/nwprod2/grib_util.v1.0.0/exec/wgrib2'
';export'
'WGRIB2;_LMFILES_=/usrx/local/modulefiles/ics/15.0.3:/usrx/local/modulefiles/python/2.7.13:/global/noscrub/Julie.Prestopnik/modulefiles/met/8.1:/usrx/local/modulefiles/nco/4.2.4:/nwprod2/modulefiles/grib_util/v1.0.0'
';export' '_LMFILES_;'
+ CNVGRIB=/nwprod2/grib_util.v1.0.0/exec/cnvgrib
+ export CNVGRIB
+ COPYGB=/nwprod2/grib_util.v1.0.0/exec/copygb
+ export COPYGB
+ COPYGB2=/nwprod2/grib_util.v1.0.0/exec/copygb2
+ export COPYGB2
+ DEGRIB2=/nwprod2/grib_util.v1.0.0/exec/degrib2
+ export DEGRIB2
+ GRB2INDEX=/nwprod2/grib_util.v1.0.0/exec/grb2index
+ export GRB2INDEX
+ GRBINDEX=/nwprod2/grib_util.v1.0.0/exec/grbindex
+ export GRBINDEX
+ GRIB2GRIB=/nwprod2/grib_util.v1.0.0/exec/grib2grib
+ export GRIB2GRIB
+
LOADEDMODULES=ics/15.0.3:python/2.7.13:met/8.1:nco/4.2.4:grib_util/v1.0.0
+ export LOADEDMODULES
+ TOCGRIB=/nwprod2/grib_util.v1.0.0/exec/tocgrib
+ export TOCGRIB
+ TOCGRIB2=/nwprod2/grib_util.v1.0.0/exec/tocgrib2
+ export TOCGRIB2
+ TOCGRIB2SUPER=/nwprod2/grib_util.v1.0.0/exec/tocgrib2super
+ export TOCGRIB2SUPER
+ WGRIB=/nwprod2/grib_util.v1.0.0/exec/wgrib
+ export WGRIB
+ WGRIB2=/nwprod2/grib_util.v1.0.0/exec/wgrib2
+ export WGRIB2
+
_LMFILES_=/usrx/local/modulefiles/ics/15.0.3:/usrx/local/modulefiles/python/2.7.13:/global/noscrub/Julie.Prestopnik/modulefiles/met/8.1:/usrx/local/modulefiles/nco/4.2.4:/nwprod2/modulefiles/grib_util/v1.0.0
+ export _LMFILES_
+ METPLUS_PATH=/meso/save/Ying.Lin/metplus
+ export METPLUS_PATH
+ MET_PATH=/global/noscrub/Julie.Prestopnik/met/8.1
+ export MET_PATH
+ JLOGFILE=/meso/save/Ying.Lin/metplus/logs/metplus_jlogfile
+ export JLOGFILE
+
PYTHONPATH=/meso/save/Ying.Lin/metplus/ush:/meso/save/Ying.Lin/metplus/parm
+ export PYTHONPATH
+
PATH=/usrx/local/nco/4.2.4/bin:/global/noscrub/Julie.Prestopnik/met/8.1//bin:/global/noscrub/Julie.Prestopnik/met/8.1//external_libs/bin:/usrx/local/python/2.7.13/bin:/usrx/local/intel/2015/advisor_xe_2015/bin64:/usrx/local/intel/2015/vtune_amplifier_xe_2015/bin64:/usrx/local/intel/2015/inspector_xe_2015/bin64:/usrx/local/intel/2015/impi_latest/intel64/bin:/usrx/local/intel/2015/itac_latest/intel64/bin:/usrx/local/intel/2015/composer_xe_2015.3.187/bin/intel64:/usrx/local/intel/2015/mpirt/bin/intel64:/usrx/local/intel/2015/composerxe/debugger/gdb/intel64_mic/bin:/usrx/local/intel/2015/composerxe/debugger/gdb/intel64/bin:.:/nwprod/gempak/nawips/os/linux2.6.32_x86_64/bin:/usr/lib64/qt-
3.3/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/opt/ibutils/bin:/nwprod/util/exec::/nwprod/util/ush::/u/Wesley.Ebisuzaki/bin:/u/Ying.Lin/bin:/nwprod/gempak/nawips/bin:/nwprod/gempak/nawips/scripts/ez:/nwprod/gempak/nawips/scripts/nawips:/nwprod/gempak/nawips/scripts/prnt:/nwprod/gempak/nawips/scripts/decoder:/sss/emc/meso/shared/Eric.Rogers/ddapp/exe_lnx6:/sss/emc/meso/shared/Eric.Rogers/vsdb/scripts:/meso/save/Ying.Lin/metplus/ush:.
+ export PATH
+ echo 'Actual output starts here:'
Actual output starts here:
+ master_metplus.py -c
/meso/save/Ying.Lin/metplus/parm/use_cases/qpf/examples/hrefmean-vs-
qpe.conf -c /meso/save/Ying.Lin/metplus/yl/parm/system.conf.wcoss
05/30 17:13:06.037 master_metplus (master_metplus.py:64) INFO:
Starting METplus v2.1
05/30 17:13:06.037 metplus (config_metplus.py:73) INFO: Starting
METplus configuration setup.
05/30 17:13:06.058 metplus (config_launcher.py:214) INFO:
/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/parm/metplus_config/metplus_system.conf:
Parsed this file
05/30 17:13:06.074 metplus (config_launcher.py:214) INFO:
/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/parm/metplus_config/metplus_data.conf:
Parsed this file
05/30 17:13:06.075 metplus (config_launcher.py:214) INFO:
/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/parm/metplus_config/metplus_runtime.conf:
Parsed this file
05/30 17:13:06.083 metplus (config_launcher.py:214) INFO:
/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/parm/metplus_config/metplus_logging.conf:
Parsed this file
05/30 17:13:06.094 metplus (config_launcher.py:214) INFO:
/meso/save/Ying.Lin/metplus/parm/use_cases/qpf/examples/hrefmean-vs-
qpe.conf: Parsed this file
05/30 17:13:06.111 metplus (config_launcher.py:214) INFO:
/meso/save/Ying.Lin/metplus/yl/parm/system.conf.wcoss: Parsed this
file
05/30 17:13:06.112 metplus (met_util.py:523) INFO: Adding:
config.LOG_TIMESTAMP='20190530'
05/30 17:13:06.112 metplus (met_util.py:529) INFO: Replace [config]
LOG_METPLUS with
'/meso/noscrub/Ying.Lin/metplus.out/logs/master_metplus.log.20190530'
05/30 17:13:06.114 metplus (config_launcher.py:284) WARNING:
METPLUS_BASE from the conf files has no effect. Overriding to
/gpfs/gd1/emc/meso/save/Ying.Lin/metplus
05/30 17:13:06.114 metplus (config_launcher.py:297) INFO: Replace
[dir] OUTPUT_BASE with /meso/noscrub/Ying.Lin/metplus.out
05/30 17:13:06.114 metplus (config_launcher.py:297) INFO: Replace
[dir] METPLUS_BASE with /gpfs/gd1/emc/meso/save/Ying.Lin/metplus
05/30 17:13:06.114 metplus (config_launcher.py:307) INFO:
METPLUS_CONF: /meso/noscrub/Ying.Lin/metplus.out/metplus_final.conf
written here.
05/30 17:13:06.118 metplus (config_metplus.py:136) INFO: Completed
METplus configuration setup.
05/30 17:13:06.119 metplus (master_metplus.py:79) INFO: Running
METplus v2.1 called with command:
/meso/save/Ying.Lin/metplus/ush/master_metplus.py -c
/meso/save/Ying.Lin/metplus/parm/use_cases/qpf/examples/hrefmean-vs-
qpe.conf -c /meso/save/Ying.Lin/metplus/yl/parm/system.conf.wcoss
05/30 17:13:06.141 metplus.PcpCombine (config_wrapper.py:102) DEBUG:
Setting [config] PCP_COMBINE_SKIP_IF_OUTPUT_EXISTS to default value:
False.
05/30 17:13:06.141 metplus.PcpCombine (config_wrapper.py:102) DEBUG:
Setting [config] FCST_PCP_COMBINE_MIN_FORECAST to default value: 0.
05/30 17:13:06.141 metplus.PcpCombine (config_wrapper.py:102) DEBUG:
Setting [config] FCST_PCP_COMBINE_MAX_FORECAST to default value: 256.
05/30 17:13:06.141 metplus.PcpCombine (config_wrapper.py:102) DEBUG:
Setting [config] FCST_PCP_COMBINE_DATA_INTERVAL to default value: 1.
05/30 17:13:06.141 metplus.PcpCombine (config_wrapper.py:102) DEBUG:
Setting [config] FCST_PCP_COMBINE_TIMES_PER_FILE to default value: -1.
05/30 17:13:06.142 metplus.PcpCombine (config_wrapper.py:102) DEBUG:
Setting [config] FCST_PCP_COMBINE_INPUT_LEVEL to default value: -1.
05/30 17:13:06.142 metplus.PcpCombine (config_wrapper.py:102) DEBUG:
Setting [config] FCST_PCP_COMBINE_STAT_LIST to default value: .
05/30 17:13:06.142 metplus.PcpCombine (config_wrapper.py:102) DEBUG:
Setting [config] FCST_PCP_COMBINE_DERIVE_LOOKBACK to default value: 0.
05/30 17:13:06.142 metplus.PcpCombine (config_wrapper.py:102) DEBUG:
Setting [config] OBS_PCP_COMBINE_MIN_FORECAST to default value: 0.
05/30 17:13:06.143 metplus.PcpCombine (config_wrapper.py:102) DEBUG:
Setting [config] OBS_PCP_COMBINE_MAX_FORECAST to default value: 256.
05/30 17:13:06.143 metplus.PcpCombine (config_wrapper.py:102) DEBUG:
Setting [config] OBS_PCP_COMBINE_DATA_INTERVAL to default value: 1.
05/30 17:13:06.143 metplus.PcpCombine (config_wrapper.py:102) DEBUG:
Setting [config] OBS_PCP_COMBINE_TIMES_PER_FILE to default value: -1.
05/30 17:13:06.143 metplus.PcpCombine (config_wrapper.py:102) DEBUG:
Setting [config] OBS_PCP_COMBINE_IS_DAILY_FILE to default value:
False.
05/30 17:13:06.143 metplus.PcpCombine (config_wrapper.py:102) DEBUG:
Setting [config] OBS_PCP_COMBINE_INPUT_LEVEL to default value: -1.
05/30 17:13:06.144 metplus.PcpCombine (config_wrapper.py:102) DEBUG:
Setting [config] OBS_PCP_COMBINE_STAT_LIST to default value: .
05/30 17:13:06.144 metplus.PcpCombine (config_wrapper.py:102) DEBUG:
Setting [config] OBS_PCP_COMBINE_DERIVE_LOOKBACK to default value: 0.
05/30 17:13:06.144 metplus.RegridDataPlane (config_wrapper.py:102)
DEBUG: Setting [config] REGRID_DATA_PLANE_SKIP_IF_OUTPUT_EXISTS to
default value: False.
05/30 17:13:06.145 metplus.RegridDataPlane (config_wrapper.py:102)
DEBUG: Setting [config] FCST_REGRID_DATA_PLANE_RUN to default value:
False.
05/30 17:13:06.147 metplus.GridStat (config_wrapper.py:102) DEBUG:
Setting [config] OBS_IS_PROB to default value: False.
05/30 17:13:06.147 metplus.GridStat (config_wrapper.py:102) DEBUG:
Setting [config] FCST_WINDOW_BEGIN to default value: 0.
05/30 17:13:06.147 metplus.GridStat (config_wrapper.py:102) DEBUG:
Setting [config] FCST_WINDOW_END to default value: 0.
05/30 17:13:06.147 metplus.GridStat (config_wrapper.py:102) DEBUG:
Setting [config] OBS_WINDOW_BEGIN to default value: 0.
05/30 17:13:06.148 metplus.GridStat (config_wrapper.py:102) DEBUG:
Setting [config] OBS_WINDOW_END to default value: 0.
05/30 17:13:06.148 metplus.GridStat (config_wrapper.py:102) DEBUG:
Setting [config] FCST_FILE_WINDOW_BEGIN to default value: 0.
05/30 17:13:06.148 metplus.GridStat (config_wrapper.py:102) DEBUG:
Setting [config] FCST_FILE_WINDOW_END to default value: 0.
05/30 17:13:06.148 metplus.GridStat (config_wrapper.py:102) DEBUG:
Setting [config] OBS_FILE_WINDOW_BEGIN to default value: 0.
05/30 17:13:06.148 metplus.GridStat (config_wrapper.py:102) DEBUG:
Setting [config] OBS_FILE_WINDOW_END to default value: 0.
05/30 17:13:06.150 metplus.GridStat (config_wrapper.py:144) ERROR:
Executable WGRIB2 does not exist at $WGRIB2
------------------------------------------------
Subject: Using FCST_PCP_COMBINE in METplus
From: Ying Lin
Time: Thu May 30 11:41:16 2019
[config]
CLOCK_TIME = 20190530171306
LOOP_ORDER = times
PROCESS_LIST = PcpCombine, RegridDataPlane, GridStat
METPLUS_CONF = {OUTPUT_BASE}/metplus_final.conf
LOG_METPLUS =
/meso/noscrub/Ying.Lin/metplus.out/logs/master_metplus.log.20190530
LOG_TIMESTAMP_TEMPLATE = %Y%m%d
LOG_TIMESTAMP_USE_DATATIME = no
LOG_MET_OUTPUT_TO_METPLUS = yes
LOG_MET_VERBOSITY = 5
LOG_LEVEL = DEBUG
LOOP_BY = INIT
INIT_TIME_FMT = %Y%m%d%H
INIT_BEG = 2017050912
INIT_END = 2017050912
INIT_INCREMENT = 43200
LEAD_SEQ = 18
FCST_PCP_COMBINE_RUN = True
FCST_PCP_COMBINE_METHOD = ADD
OBS_PCP_COMBINE_RUN = True
OBS_PCP_COMBINE_METHOD = ADD
OBS_REGRID_DATA_PLANE_RUN = True
REGRID_DATA_PLANE_METHOD = BUDGET
REGRID_DATA_PLANE_WIDTH = 2
FCST_VAR1_NAME = APCP
FCST_VAR1_LEVELS = A06
FCST_VAR1_THRESH = gt12.7, gt25.4, gt50.8, gt76.2, gt152.4
MODEL = HREF_MEAN
OBTYPE = QPE
REGRID_DATA_PLANE_VERIF_GRID = {INPUT_BASE}/qpf/mask/CONUS_HRRRTLE.nc
GRID_STAT_CONFIG_FILE = {CONFIG_DIR}/GridStatConfig_MEAN
FCST_PCP_COMBINE_INPUT_DATATYPE = NETCDF
FCST_IS_PROB = false
FCST_PCP_COMBINE_IS_DAILY_FILE = false
FCST_PCP_COMBINE_1_FIELD_NAME = P01M_NONE
OBS_PCP_COMBINE_1_FIELD_NAME = P01M_NONE
OBS_PCP_COMBINE_3_FIELD_NAME = P03M_NONE
OBS_PCP_COMBINE_6_FIELD_NAME = P06M_NONE
OBS_PCP_COMBINE_12_FIELD_NAME = P12M_NONE
OBS_PCP_COMBINE_24_FIELD_NAME = P24M_NONE
OBS_PCP_COMBINE_INPUT_DATATYPE = NETCDF
LOG_DIR = {OUTPUT_BASE}/logs
LOG_TIMESTAMP = 20190530
METPLUS_VERSION = 2.1
[dir]
PARM_BASE = {METPLUS_BASE}/parm
OUTPUT_BASE = /meso/noscrub/Ying.Lin/metplus.out
STAGING_DIR = {OUTPUT_BASE}/stage
MET_INSTALL_DIR = /global/noscrub/Julie.Prestopnik/met/8.1
MET_BASE = {MET_INSTALL_DIR}/share/met
LOG_DIR = {OUTPUT_BASE}/logs
TMP_DIR = {OUTPUT_BASE}/tmp
PROJ_DIR = /global/noscrub/Julie.Prestopnik/METplus/METplus-
2.1_sample_data
INPUT_BASE = {PROJ_DIR}
MODEL_DATA_DIR = {PROJ_DIR}/model_data
CONFIG_DIR = {PARM_BASE}/use_cases/qpf/met_config
FCST_PCP_COMBINE_INPUT_DIR = {INPUT_BASE}/qpf/HREFv2_Mean
FCST_PCP_COMBINE_OUTPUT_DIR = {OUTPUT_BASE}/uswrp/HREFv2_Mean/bucket
FCST_GRID_STAT_INPUT_DIR = {OUTPUT_BASE}/uswrp/HREFv2_Mean/bucket
OBS_PCP_COMBINE_INPUT_DIR = {INPUT_BASE}/qpf/QPE_Data
OBS_PCP_COMBINE_OUTPUT_DIR = {OUTPUT_BASE}/uswrp/QPE_Data/bucket
OBS_REGRID_DATA_PLANE_INPUT_DIR = {OUTPUT_BASE}/uswrp/QPE_Data/bucket
OBS_REGRID_DATA_PLANE_OUTPUT_DIR = {OUTPUT_BASE}/uswrp/QPE_Data/regrid
OBS_GRID_STAT_INPUT_DIR = {OUTPUT_BASE}/uswrp/QPE_Data/regrid
GRID_STAT_OUTPUT_DIR = {OUTPUT_BASE}/uswrp/met_out/{MODEL}
METPLUS_BASE = /gpfs/gd1/emc/meso/save/Ying.Lin/metplus
MET_BUILD_BASE = {MET_INSTALL_DIR}
[exe]
WGRIB2 = $WGRIB2
RM_EXE = /bin/rm
CUT_EXE = /bin/cut
TR_EXE = /usr/bin/tr
NCAP2_EXE = ncap2
CONVERT_EXE = /usr/bin/convert
NCDUMP_EXE = {MET_INSTALL_DIR}/external_libs/bin/ncdump
EGREP_EXE = /bin/egrep
[filename_templates]
FCST_PCP_COMBINE_INPUT_TEMPLATE =
{init?fmt=%Y%m%d}/hrefmean_{init?fmt=%Y%m%d%H}f{lead?fmt=%HHH}.grd
FCST_PCP_COMBINE_OUTPUT_TEMPLATE =
{valid?fmt=%Y%m%d}/hrefmean_{valid?fmt=%Y%m%d%H}_A{level?fmt=%HH}.nc
FCST_GRID_STAT_INPUT_TEMPLATE =
{valid?fmt=%Y%m%d}/hrefmean_{valid?fmt=%Y%m%d%H}_A{level?fmt=%HH}.nc
OBS_PCP_COMBINE_INPUT_TEMPLATE =
{valid?fmt=%Y%m%d}/qpe_{valid?fmt=%Y%m%d%H}.grd
OBS_PCP_COMBINE_OUTPUT_TEMPLATE =
{valid?fmt=%Y%m%d}/qpe_{valid?fmt=%Y%m%d%H}_A{level?fmt=%HH}.nc
OBS_REGRID_DATA_PLANE_TEMPLATE =
{valid?fmt=%Y%m%d}/qpe_{valid?fmt=%Y%m%d%H}_A{level?fmt=%HH}.nc
OBS_GRID_STAT_INPUT_TEMPLATE =
{valid?fmt=%Y%m%d}/qpe_{valid?fmt=%Y%m%d%H}_A{level?fmt=%HH}.nc
GRID_STAT_OUTPUT_TEMPLATE = {init?fmt=%Y%m%d%H%M}/grid_stat
------------------------------------------------
Subject: Using FCST_PCP_COMBINE in METplus
From: Ying Lin
Time: Thu May 30 11:41:16 2019
. /usrx/local/Modules/default/init/ksh
module purge
module use /global/noscrub/Julie.Prestopnik/modulefiles
module load met/8.1
module load nco
module load grib_util
export METPLUS_PATH="/meso/save/Ying.Lin/metplus"
export MET_PATH="/global/noscrub/Julie.Prestopnik/met/8.1"
export JLOGFILE="${METPLUS_PATH}/logs/metplus_jlogfile"
export PYTHONPATH="${METPLUS_PATH}/ush:${METPLUS_PATH}/parm"
export PATH="${PATH}:${METPLUS_PATH}/ush:."
------------------------------------------------
Subject: Using FCST_PCP_COMBINE in METplus
From: Ying Lin
Time: Thu May 30 11:41:16 2019
[dir]
# PROJ_DIR points to the input data directory. In this case, I am
pointing to
# the sample data that came alongside the release
PROJ_DIR = /global/noscrub/Julie.Prestopnik/METplus/METplus-
2.1_sample_data
OUTPUT_BASE = /meso/noscrub/Ying.Lin/metplus.out
TMP_DIR = {OUTPUT_BASE}/tmp
METPLUS_BASE = /meso/save/Ying.Lin/metplus/
MET_INSTALL_DIR = /global/noscrub/Julie.Prestopnik/met/8.1
MET_BUILD_BASE = {MET_INSTALL_DIR}
PARM_BASE = {METPLUS_BASE}/parm
[config]
LOG_DIR = {OUTPUT_BASE}/logs
LOG_MET_VERBOSITY = 5
LOG_LEVEL = DEBUG
[exe]
# NCDUMP_EXE, DO NOT USE module load netcdf it will break
# the tc_pairs binary ... ldd tc_pairs and it cant find
lib.netcdf.so.6
WGRIB2 = $WGRIB2 ;; MUST run module load wgrib2
CUT_EXE = /bin/cut
TR_EXE = /usr/bin/tr
RM_EXE = /bin/rm
NCAP2_EXE = ncap2 ;; MUST run module load nco
CONVERT_EXE = /usr/bin/convert
NCDUMP_EXE = {MET_INSTALL_DIR}/external_libs/bin/ncdump ;; DO NOT run
module load netcdf
EGREP_EXE = /bin/egrep
#GEMPAKTOCF_BIN = /d1/mccabe/bin/GempakToCF
------------------------------------------------
Subject: Using FCST_PCP_COMBINE in METplus
From: George McCabe
Time: Thu May 30 13:36:54 2019
Hi Ying,
To reference an environment variable in the METplus config files, you
need
to use this syntax:
WGRIB2 = {ENV[WGRIB2]}
Since you have set WGRIB2 environment variable, this should set the
METplus
config variable correctly. You may also be able to set WGRIB2 = wgrib2
in
the METplus config file instead. If the exe is in the path of the user
running METplus, it will find the app properly. Let me know if that
doesn't
work.
On Thu, May 30, 2019 at 11:41 AM Ying Lin via RT <met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789 >
>
> Hi George,
>
> Thanks for following up! The sample case in
> parm/use_cases/grid_to_grid/examples/precip_continuous.conf seems to
> "subtract" correctly now (it was off by 24h in v2.0.4). The
> precip_continuous.conf only deals with the PcpCombine step though.
> When I added the regrid_data_plane step to test a fv3gfs verif on
G212,
> I got this err msg:
> metplus.GridStat (config_wrapper.py:144) ERROR: Executable
WGRIB2
> does not exist at $WGRIB2
> I thought there must have been a problem in my model config for the
v2.1
> (found several actually) but when I run the use case at
> parm/use_cases/qpf/examples/hrefmean-vs-qpe.conf the same thing
happened.
>
> I'm attaching three files to this email:
>
> 1. test_hrefmean.ksh: script to run this use case
> 2. hrefmean.out: output for running the above script. You can see
that
> after loading model grid_util (from ". ~/dot.for.metplus) there
was
> "WGRIB2=/nwprod2/grib_util.v1.0.0/exec/wgrib2
> export WGRIB2"; and at the end there is the err msg
> "metplus.GridStat (config_wrapper.py:144) ERROR: Executable
WGRIB2
> does not exist at $WGRIB2"
> 3. metplus_final.conf from the above run
> 4. my "dot.for.metplus" file invoked by test_hrefmean.ksh. I
verified
> manually - after "module load grib_util", $WGRIB2 does point to
> /nwprod2/grib_util.v1.0.0/exec/wgrib2
> 5. my system.conf.wcoss invoked by test_hrefmean.ksh.
>
> I looked back at run output from METplus-2.0.4, there was also
> WGRIB2=/nwprod2/grib_util.v1.0.0/exec/wgrib2 and "export", and there
> wasn't a complain about "Executable WGRIB2 does not exist at
$WGRIB2".
>
> Thanks again -
>
> Ying
>
>
> On 5/29/19 1:53 PM, George McCabe via RT wrote:
> > Hi Ying,
> >
> > I am just following up on this ticket. Have you tried running your
case
> on METplus v2.1? Let me know if you run into any other issues.
> >
> > Thanks,
> > George
> >
> > On Thu Apr 18 19:15:58 2019, mccabe wrote:
> >> Hi Ying,
> >>
> >> It looks like the bug with subtract mode is in v2.0.4. It will be
> >> fixed in
> >> the next release. You can look at the develop branch on GitHub to
see
> >> what
> >> has changed in the subtract function. Have a good weekend!
> >>
> >> On Thu, Apr 18, 2019, 5:09 PM Ying Lin via RT <met_help at ucar.edu>
> >> wrote:
> >>
> >>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789 >
> >>>
> >>> Hi George,
> >>>
> >>> Thank you - you're right, replacing that first
> >>> FCST_GRID_STAT_INPUT_TEMPLATE with
FCST_PCP_COMBINE_INPUT_TEMPLATE
> >>> got
> >>> me past the err msg! There're still some old stuff with the way
the
> >>> summing was done - similar to the odd result I mentioned in the
last
> >>> email with the 2.0.4 precip_continuous.conf. I'll study the new
> >>> precip_continuous.conf in the github and follow up with you next
> >>> week.
> >>>
> >>> Thanks for your help and patience today. Have a great weekend -
> >>>
> >>> Ying
> >>>
> >>> On 4/18/19 4:25 PM, George McCabe via RT wrote:
> >>>> Hi Ying,
> >>>>
> >>>> I am sorry, I was incorrect that the fixed precip_continuous
config
> >>>> file
> >>>> was updated in 2.0.4. It looks like the fix was made after that
> >>>> release.
> >>> It
> >>>> will be in the newest release. You can see the updated file on
> >>>> GitHub in
> >>>> the develop branch here:
> >>>>
> >>>>
> >>>
>
https://github.com/NCAR/METplus/blob/develop/parm/use_cases/grid_to_grid/examples/precip_continuous.conf
> >>>> For your use case, it looks like you don't have
> >>>> FCST_PCP_COMBINE_INPUT_TEMPLATE set and you have 2 for
grid_stat:
> >>>>
> >>>> FCST_GRID_STAT_INPUT_TEMPLATE =
> >>>>
> >>>
>
gfs.{init?fmt=%Y%m%d}/{init?fmt=%H}/gfs.t{init?fmt=%H}z.pgrb2.0p25.f{lead?fmt=%HHH}
> >>>> FCST_PCP_COMBINE_OUTPUT_TEMPLATE =
> >>>> gfs.{valid?fmt=%Y%m%d%H}_A{level?fmt=%HHH}h
> >>>> FCST_GRID_STAT_INPUT_TEMPLATE =
> >>> gfs.{valid?fmt=%Y%m%d%H}_A{level?fmt=%HHH}h
> >>>> Fixing this may solve your issue. Please let me know if it does
> >>>> not. I
> >>> have
> >>>> to catch a flight this evening so I am going to stop working
very
> >>>> soon,
> >>> but
> >>>> if you reply very soon I may still be able to help you before
the
> >>> weekend.
> >>>> Thanks,
> >>>> George
> >>>>
> >>>> On Thu, Apr 18, 2019 at 8:13 PM Ying Lin via RT
<met_help at ucar.edu>
> >>> wrote:
> >>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789
>
> >>>>>
> >>>>> Hi George,
> >>>>>
> >>>>> Thank you for the quick reply. I was looking at
> >>>>>
> >>> /gpfs/gd1/emc/global/noscrub/Julie.Prestopnik/METplus/METplus-
> >>>
2.0.4/parm/use_cases/grid_to_grid/examples/precip_continuous.conf,
> >>>>> which has VALID_BEG = 2018030900, VALID_END = 2018030900.
Date
> >>>>> stamp is
> >>>>> Jan 30. Is there a "newer version of v2.0.4" available on
wcoss?
> >>>>> From
> >>>>> the above config it's not clear to me how METplus knows to
> >>>>> subtract the
> >>>>> gfs.*.f024 from the gfs.*.f048.
> >>>>>
> >>>>> If you have time today, could you also take a look why my
simple
> >>>>> test
> >>>>> (conf set up to be as close to precip_continuous.conf as
possible
> >>>>> -
> >>>>>
> >>>
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf)
> >>>>> led to err msg in
> >>>>>
> >>>
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs.out.test5
> >>> ?
> >>>>> It seems that I corrected the problems you pointed out in your
> >>>>> reply
> >>>>> this morning, but the err msg still seems to be the same as
> >>>>> yesterday's.
> >>>>>
> >>>>> If you don't have time today, next Monday is fine too. Have a
> >>>>> good
> >>>>> holiday weekend.
> >>>>>
> >>>>> Ying
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> On 4/18/19 3:51 PM, George McCabe via RT wrote:
> >>>>>> Hi Ying,
> >>>>>>
> >>>>>> 1) The config file I see has a valid time of 2018031000 and
48 hr
> >>>>> forecast
> >>>>>> lead. You may be using an old version of METplus that had
this
> >>>>>> set up
> >>>>>> incorrectly. METplus version 2.0.4 should be correct.
> >>>>>>
> >>>>>> 2) pcp_combine -subtract does not actually use the FCST_LEVEL
> >>>>>> variable,
> >>>>> so
> >>>>>> that should be removed from the use case. The code takes the
> >>>>>> valid time
> >>>>> and
> >>>>>> subtracts the accumulation.
> >>>>>>
> >>>>>> 3) In the upcoming version of METplus (in a few weeks!) has a
> >>>>>> good
> >>> number
> >>>>>> of config changes from the previous versions. For example,
> >>>>>> FCST_LEVEL
> >>>>> will
> >>>>>> now be FCST_PCP_COMBINE_INPUT_LEVEL. These changes are
designed
> >>>>>> to make
> >>>>> the
> >>>>>> variables more clear to what they actually do. In the new
> >>>>>> version,
> >>> there
> >>>>> is
> >>>>>> now FCST_PCP_COMBINE_METHOD and OBS_PCP_COMBINE_METHOD, so
you
> >>>>>> will be
> >>>>> able
> >>>>>> to specify a different method for your forecast and analysis
> >>>>>> data.
> >>>>>>
> >>>>>> FCST_PCP_COMBINE_RUN = True
> >>>>>> FCST_PCP_COMBINE_METHOD = SUBTRACT
> >>>>>>
> >>>>>> OBS_PCP_COMBINE_RUN = True
> >>>>>> OBS_PCP_COMBINE_METHOD = SUM
> >>>>>>
> >>>>>> This functionality has not been thoroughly tested as we
haven't
> >>>>>> had any
> >>>>>> example use cases that need to do this so far, so please let
me
> >>>>>> know if
> >>>>> it
> >>>>>> does not work the way you expect and I will fix it.
> >>>>>>
> >>>>>> Just to let you know, I will be out of the office tomorrow
> >>>>>> (Friday)
> >>> and I
> >>>>>> wont have access to email. Feel free to send any follow-up
emails
> >>>>>> to me
> >>>>> and
> >>>>>> I will get back to you on Monday.
> >>>>>>
> >>>>>> Thanks,
> >>>>>> George
> >>>>>>
> >>>>>> On Thu, Apr 18, 2019 at 7:38 PM Ying Lin via RT
> >>>>>> <met_help at ucar.edu>
> >>>>> wrote:
> >>>>>>> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789 >
> >>>>>>>
> >>>>>>> Thank you very much George, this is very helpful.
> >>>>>>>
> >>>>>>> About the example in
> >>>>>>>
parm/use_cases/grid_to_grid/examples/precip_continuous.conf
> >>>>>>> I tried running it and it seems to have run through the
> >>>>>>> pcp_combine
> >>> step
> >>>>>>> fine: it did this:
> >>>>>>>
/global/noscrub/Julie.Prestopnik/met/8.0/bin/pcp_combine
> >>> -subtract
> >>> /global/noscrub/Julie.Prestopnik/METplus/METplus-
> >>>
>
2.0.4_sample_data/grid_to_grid/prfv3rt1/20180308/gfs.t00z.pgrb.1p00.f048
> >>>>>>> 48
> >>>>>>>
> >>> /global/noscrub/Julie.Prestopnik/METplus/METplus-
> >>>
>
2.0.4_sample_data/grid_to_grid/prfv3rt1/20180308/gfs.t00z.pgrb.1p00.f024
> >>>>>>> 24
> >>>>>>>
/meso/noscrub/Ying.Lin/metplus.out/gfs/bucket/gfs.2018030900_A024h
> >>>>>>>
> >>>>>>> Several questions about the example:
> >>>>>>> 1) The precip_continuous.conf has
> >>>>>>> LEAD_SEQ = 24
> >>>>>>> I thought it meant forecast lead of 24h, e.g. look for 00-
24h
> >>>>>>> forecast
> >>>>>>> valid at 2018030900 (given in the config). If so, why
doesn't
> >>>>>>> it
> >>>>>>> simply use gfs.t00z.pgrb.1p00.f024, which already have the
00-
> >>>>>>> 24h
> >>>>>>> forecast? If it subtracts gfs.t00z.pgrb.1p00.f024 from
> >>>>>>> gfs.t00z.pgrb.1p00.f048, the resulting file would have 24-
48h
> >>>>>>> forecast
> >>>>>>> valid at 2018031000. I did
> >>>>>>> ncdump -h
> >>>>>>>
/meso/noscrub/Ying.Lin/metplus.out/gfs/bucket/gfs.2018030900_A024h
> >>>>>>>
> >>>>>>> and got
> >>>>>>> APCP_24:init_time = "20180308_000000" ;
> >>>>>>> APCP_24:init_time_ut = "1520467200" ;
> >>>>>>> APCP_24:valid_time = "20180310_000000" ;
> >>>>>>> APCP_24:valid_time_ut = "1520640000" ;
> >>>>>>> APCP_24:accum_time = "240000" ;
> >>>>>>> it seems to mean that the file contains APCP_24h with
initial
> >>>>>>> time of
> >>>>>>> 00z 03/08 and the valid time is 00Z 03/10 (that'll be
APCP_48h
> >>>>>>> instead
> >>>>>>> of 24h?)
> >>>>>>>
> >>>>>>> 2) The precip_continuous.conf has
> >>>>>>> FCST_LEVEL = 6
> >>>>>>> I thought FCST_LEVEL is the forecast bucket length. In the
> >>>>>>> sample GFS
> >>>>>>> files, there isn't a 6-hourly bucket -
gfs.t00z.pgrb.1p00.f024
> >>> contains
> >>>>>>> 00-24h accumulation and gfs.t00z.pgrb.1p00.f048 contains 00-
48h
> >>>>>>> accumulation. Does FCST_LEVEL mean something other than the
> >>>>>>> forecast
> >>>>>>> bucket length?
> >>>>>>>
> >>>>>>> 3) precip_continuous.conf has
> >>>>>>> PCP_COMBINE_METHOD = SUBTRACT
> >>>>>>> which should work for the example since the verifying
analysis
> >>>>>>> is the
> >>>>>>> 24h CCPA file. But what if I need to use 'SUBTRACT' for the
> >>>>>>> forecast
> >>>>>>> field but 'SUM' for the analysis field (e.g. summing up 6h
CCPAs
> >>>>>>> into
> >>> a
> >>>>>>> 24h total)? At one point I tried
> >>>>>>> PCP_COMBINE_METHOD = SUBTRACT, SUM
> >>>>>>> and got
> >>>>>>> ERROR: Invalid PCP_COMBINE_METHOD specified
> >>>>>>>
> >>>>>>> After several failed attempts getting my own config file to
> >>>>>>> work, I
> >>>>>>> starting from precip_continuous.conf (which worked through
the
> >>>>>>> PcpCombine step; I think that's the only step it runs since
the
> >>>>>>> config
> >>>>>>> has only PcpCombine in the PROCESS_LIST) and tried to set a
> >>>>>>> config
> >>> file
> >>>>>>> as close to precip_continuous.conf as possible
> >>>>>>>
> >>>
>
(/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/parm/ylin_config/models/fv3gfs_24h.conf,
> >>>>>>> also uploaded to
> >>>>>>>
> >>>
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf
> >>>>> ).
> >>>>>>> The resulting err msg:
> >>>>>>>
> >>>>>>> 04/18 18:35:33.912 jlog (master_metplus.py:217) CRITICAL:
> >>>>>>> master_metplus failed: ''
> >>>>>>> Traceback (most recent call last):
> >>>>>>> File "/meso/save/Ying.Lin/metplus/ush/master_metplus.py",
line
> >>> 213,
> >>>>>>> in <module>
> >>>>>>> main()
> >>>>>>> File
"/meso/save/Ying.Lin/metplus/ush/master_metplus.py",
> >>>>>>> line
> >>> 170,
> >>>>>>> in main
> >>>>>>> process.run_at_time(-1, run_time)
> >>>>>>> File
> >>>>>>>
> >>>
>
"/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/ush/reformat_gridded_wrapper.py",
> >>>>>>> line 80, in run_at_time
> >>>>>>>
> >>>>>>> Full output file:
> >>>>>>>
> >>>
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs.out.test5
> >>>>> ;
> >>>>>>> metplus_final.conf:
> >>>>>>>
> >>>>>>>
> >>>
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/metplus_final.conf
> >>>>>>> The quoted err msg above appears to the the same as in my
email
> >>>>>>> yesterday, so there must still be something wrong with the
> >>>>>>> format of
> >>> my
> >>>>>>> config, though I think I eliminated the problems you pointed
out
> >>>>>>> in
> >>> your
> >>>>>>> reply. There wouldn't be a ccpa_conus_1.0d_* of the valid
day I
> >>>>>>> specified, but I think the processing failed way before that
> >>>>>>> point.
> >>>>>>>
> >>>>>>> Thank you again -
> >>>>>>>
> >>>>>>> Ying
> >>>>>>>
> >>>>>>>
> >>>>>>> On 4/17/19 3:32 PM, George McCabe via RT wrote:
> >>>>>>>> Hi Ying,
> >>>>>>>>
> >>>>>>>> You will need to set FCST_PCP_COMBINE_METHOD = SUBTRACT to
> >>>>>>>> config
> >>>>> METplus
> >>>>>>>> to subtract accumulations files to get the accumulation you
> >>>>>>>> desire.
> >>> The
> >>>>>>>> following use case has an example of this configuration:
> >>>>>>>>
> >>>>>>>> parm/use_cases/grid_to_grid/examples/precip_continous.conf
> >>>>>>>>
> >>>>>>>> I also noticed that your config file has the following
items in
> >>>>>>>> the
> >>>>>>>> [config] section:
> >>>>>>>>
> >>>>>>>> FCST_PCP_COMBINE_INPUT_DIR =
/gpfs/dell1/nco/ops/com/gfs/para
> >>>>>>>> FCST_PCP_COMBINE_OUTPUT_DIR = {OUTPUT_BASE}/qpf_grib/bucket
> >>>>>>>> FCST_GRID_STAT_INPUT_DIR = {OUTPUT_BASE}/qpf_grib/bucket
> >>>>>>>>
> >>>>>>>> You should move these to the [dir] section.
> >>>>>>>>
> >>>>>>>> Please let me know if you continue to have issues. The
error
> >>>>>>>> message
> >>>>> you
> >>>>>>>> provided has more information at the bottom that was not
> >>>>>>>> included.
> >>> The
> >>>>>>> next
> >>>>>>>> release of METplus contains more extensive error checking
so
> >>>>>>>> the
> >>> error
> >>>>>>>> message from that version should be much more helpful.
> >>>>>>>>
> >>>>>>>> Thanks,
> >>>>>>>> George
> >>>>>>>>
> >>>>>>>> On Wed, Apr 17, 2019 at 12:43 PM Ying Lin via RT
> >>>>>>>> <met_help at ucar.edu>
> >>>>>>> wrote:
> >>>>>>>>> Wed Apr 17 12:43:02 2019: Request 89789 was acted upon.
> >>>>>>>>> Transaction: Ticket created by ying.lin at noaa.gov
> >>>>>>>>> Queue: met_help
> >>>>>>>>> Subject: Using FCST_PCP_COMBINE in METplus
> >>>>>>>>> Owner: Nobody
> >>>>>>>>> Requestors: ying.lin at noaa.gov
> >>>>>>>>> Status: new
> >>>>>>>>> Ticket <URL:
> >>>>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789
> >>>>>>>>> Hello. I haven't been able to get the FCST_PCP_COMBINE
> >>>>>>>>> feature to
> >>>>> work
> >>>>>>>>> in METplus, running out of ideas and hope you can help.
> >>>>>>>>>
> >>>>>>>>> I've put some examples in
> >>>>>>>>> https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/
> >>>>>>>>> (current
> >>>>>>> test
> >>>>>>>>> directory:
> >>>>>>>>> gyre:/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/test).
> >>>>>>>>> test_fv3gfs_24h.ksh is the simple test script. The goal
is to
> >>>>> compute a
> >>>>>>>>> 24h verification ending at 12Z 20190415.
> >>>>>>>>>
> >>>>>>>>> 'test1' did not need FCST_PCP_COMBINE - the config file
> >>>>>>>>> fv3gfs_24h.conf.test1 simply had LEAD_SEQ set to '24',
> >>>>>>>>> FCST_PCP_COMBINE_RUN was not set. The FV3GFS had two
precip
> >>> buckets,
> >>>>> 6h
> >>>>>>>>> and continuously-accumulating, and METplus easily found
the
> >>>>>>>>> 24h
> >>>>> forecast
> >>>>>>>>> accumulation and went on with the grid_stat.
> >>>>>>>>>
> >>>>>>>>> 'test2' had 'LEAD_SEQ = 36', and I added the following in
the
> >>>>>>>>> config
> >>>>>>>>> (
> >>>>>>>>>
> >>>
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test2
> >>>>>>>>> ):
> >>>>>>>>> FCST_PCP_COMBINE_RUN = True
> >>>>>>>>> FCST_PCP_COMBINE_INPUT_DATATYPE = GRIB
> >>>>>>>>> FCST_PCP_COMBINE_INPUT_DIR =
> >>> /gpfs/dell1/nco/ops/com/gfs/para
> >>>>>>>>> FCST_PCP_COMBINE_OUTPUT_DIR =
{OUTPUT_BASE}/qpf_grib/bucket
> >>>>>>>>> FCST_GRID_STAT_INPUT_DIR = {OUTPUT_BASE}/qpf_grib/bucket
> >>>>>>>>> FCST_MAX_FORECAST = 36
> >>>>>>>>> FCST_INIT_INTERVAL = 6
> >>>>>>>>> FCST_LEVEL = 36
> >>>>>>>>> Here the goal was to see if I could get METplus to
subtract
> >>>>>>>>> the
> >>> 00-12h
> >>>>>>>>> accumulation from gfs.t00z.pgrb2.0p25.f012 from the 00-36h
> >>>>> accumulation
> >>>>>>>>> in gfs.t00z.pgrb2.0p25.f036. Thinking that FCST_LEVEL is
the
> >>> forecast
> >>>>>>>>> bucket length, I figured setting it to '36' and
artificially
> >>>>> specifying
> >>>>>>>>> FCST_MAX_FORECAST = 36 (instead of the full 384h) should
work
> >>>>>>>>> for
> >>> this
> >>>>>>>>> limited test where LEAD_SEQ is set to 36 (is there a way
to
> >>>>>>>>> specify
> >>>>> that
> >>>>>>>>> the bucket never gets emptied?). It failed with the
follow
> >>>>>>>>> err msg
> >>>>>>>>> (full message in
> >>>>>>>>>
> >>>>>>>>>
> >>>
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test2
> >>>>>>>>> ):
> >>>>>>>>>
> >>>>>>>>> 04/16 20:01:42.754 jlog (master_metplus.py:217) CRITICAL:
> >>>>>>>>> master_metplus failed: ''
> >>>>>>>>> Traceback (most recent call last):
> >>>>>>>>> File
> >>>>>>>>> "/meso/save/Ying.Lin/metplus/ush/master_metplus.py", line
> >>>>> 213,
> >>>>>>>>> in <module>
> >>>>>>>>> main()
> >>>>>>>>> File
> >>>>>>>>> "/meso/save/Ying.Lin/metplus/ush/master_metplus.py", line
> >>>>> 170,
> >>>>>>>>> in main
> >>>>>>>>> process.run_at_time(-1, run_time)
> >>>>>>>>> File
> >>>>>>>>>
> >>>
>
"/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/ush/reformat_gridded_wrapper.py",
> >>>>>>>>> line 80, in run_at_time
> >>>>>>>>> self.run_at_time_once(task_info, var_info, rl)
> >>>>>>>>> File
> >>>>>>>>>
> >>>
"/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/ush/pcp_combine_wrapper.py",
> >>>>>>>>> line 419, in run_at_time_once
> >>>>>>>>> ....
> >>>>>>>>>
> >>>>>>>>> 'test3' is an attempt from a different angle, thinking
that if
> >>>>>>>>> the
> >>>>>>>>> 'continuously running' bucket didn't work, maybe the 6-
hourly
> >>>>>>>>> bucket
> >>>>>>>>> would. In
> >>>>>>>>>
> >>>>>>>>>
> >>>
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test3
> >>>>>>> ,
> >>>>>>>>> I changed LEAD_SEQ to 24, and set FCST_LEVEL = 6. It
produced
> >>>>>>>>> err
> >>>>> msgs
> >>>>>>>>> (
> >>>>>>>>>
> >>>
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test3
> >>>>>>> )
> >>>>>>>>> that appeared to be the same as that from 'test2'.
> >>>>>>>>>
> >>>>>>>>> Earlier on I tried using the FCST_PCP_COMBINE feature with
the
> >>>>>>>>> NAM
> >>>>>>>>> 'CONUSNEST' output (with simple regular 6-hourly bucket)
and
> >>>>>>>>> got the
> >>>>>>>>> same err msg, so I don't think FV3GFS's dual-bucket system
is
> >>>>>>>>> the
> >>>>> cause.
> >>>>>>>>> Also posted on
> >>>>>>>>> https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/
are
> >>>>>>>>> the
> >>>>> two
> >>>>>>>>> metplus_final.conf files from both 'test2' and 'test3'.
> >>>>>>>>>
> >>>>>>>>> I found in
> >>>>>>>>>
> >>> /global/noscrub/Julie.Prestopnik/METplus/METplus-
> >>> 2.0.4/parm/use_cases/qpf/examples/
> >>>>>>>>> that hrefmean-vs-mrms-qpe.conf and nationalblend-vs-mrms-
> >>>>>>>>> qpe.conf
> >>> had
> >>>>>>>>> "FCST_PCP_COMBINE_RUN = True", that's where I borrowed the
> >>>>>>>>> FCST_PCP_COMBINE_* set up in my fv3gfs_24h.conf.test2 and
> >>>>>>>>> fv3gfs_24h.conf.test3. Haven't been able to run the
hrefmean
> >>>>>>>>> and
> >>>>>>>>> nationalblend tests myself however - got err msg of "
> >>>>>>>>> CRITICAL:
> >>>>>>>>> master_metplus failed: No option 'GEMPAKTOCF_CLASSPATH'
in
> >>>>>>>>> section:
> >>>>>>>>> 'exe'" from both attempts.
> >>>>>>>>>
> >>>>>>>>> Thank you for looking into this -
> >>>>>>>>>
> >>>>>>>>> Ying
> >>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> Ying Lin
> >>>>>>>>> NCEP/EMC/Verification, Post-processing and Product
Generation
> >>>>>>>>> Branch
> >>>>>>>>> NCWCP Cubicle No. 2015
> >>>>>>>>> Ying.Lin at noaa.gov
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>> --
> >>>>>>> Ying Lin
> >>>>>>> NCEP/EMC/Verification, Post-processing and Product
Generation
> >>>>>>> Branch
> >>>>>>> NCWCP Cubicle No. 2015
> >>>>>>> Ying.Lin at noaa.gov
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>> --
> >>>>> Ying Lin
> >>>>> NCEP/EMC/Verification, Post-processing and Product Generation
> >>>>> Branch
> >>>>> NCWCP Cubicle No. 2015
> >>>>> Ying.Lin at noaa.gov
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>> --
> >>> Ying Lin
> >>> NCEP/EMC/Verification, Post-processing and Product Generation
Branch
> >>> NCWCP Cubicle No. 2015
> >>> Ying.Lin at noaa.gov
> >>>
> >>>
> >>>
> >>>
> >
> >
>
> --
> Ying Lin
> NCEP/EMC/Verification, Post-processing and Product Generation Branch
> NCWCP Cubicle No. 2015
> Ying.Lin at noaa.gov
>
>
>
>
------------------------------------------------
Subject: Using FCST_PCP_COMBINE in METplus
From: George McCabe
Time: Thu May 30 14:32:08 2019
The reason you didn't have this issue previously is your use case
likely
didn't need to use any of those exes. I added a function to check and
add
all of the non-MET executables to all of the MET wrappers in case they
are
needed, but I realized later than they are not necessary for most of
the
cases. However, they may become required for other cases so we may
leave
the requirement to set them correctly to run METplus.
On Thu, May 30, 2019 at 1:36 PM George McCabe <mccabe at ucar.edu> wrote:
> Hi Ying,
>
> To reference an environment variable in the METplus config files,
you need
> to use this syntax:
>
> WGRIB2 = {ENV[WGRIB2]}
>
> Since you have set WGRIB2 environment variable, this should set the
> METplus config variable correctly. You may also be able to set
WGRIB2 =
> wgrib2 in the METplus config file instead. If the exe is in the path
of the
> user running METplus, it will find the app properly. Let me know if
that
> doesn't work.
>
> On Thu, May 30, 2019 at 11:41 AM Ying Lin via RT <met_help at ucar.edu>
> wrote:
>
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789 >
>>
>> Hi George,
>>
>> Thanks for following up! The sample case in
>> parm/use_cases/grid_to_grid/examples/precip_continuous.conf seems
to
>> "subtract" correctly now (it was off by 24h in v2.0.4). The
>> precip_continuous.conf only deals with the PcpCombine step though.
>> When I added the regrid_data_plane step to test a fv3gfs verif on
G212,
>> I got this err msg:
>> metplus.GridStat (config_wrapper.py:144) ERROR: Executable
WGRIB2
>> does not exist at $WGRIB2
>> I thought there must have been a problem in my model config for the
v2.1
>> (found several actually) but when I run the use case at
>> parm/use_cases/qpf/examples/hrefmean-vs-qpe.conf the same thing
happened.
>>
>> I'm attaching three files to this email:
>>
>> 1. test_hrefmean.ksh: script to run this use case
>> 2. hrefmean.out: output for running the above script. You can see
that
>> after loading model grid_util (from ". ~/dot.for.metplus) there
was
>> "WGRIB2=/nwprod2/grib_util.v1.0.0/exec/wgrib2
>> export WGRIB2"; and at the end there is the err msg
>> "metplus.GridStat (config_wrapper.py:144) ERROR: Executable
WGRIB2
>> does not exist at $WGRIB2"
>> 3. metplus_final.conf from the above run
>> 4. my "dot.for.metplus" file invoked by test_hrefmean.ksh. I
verified
>> manually - after "module load grib_util", $WGRIB2 does point to
>> /nwprod2/grib_util.v1.0.0/exec/wgrib2
>> 5. my system.conf.wcoss invoked by test_hrefmean.ksh.
>>
>> I looked back at run output from METplus-2.0.4, there was also
>> WGRIB2=/nwprod2/grib_util.v1.0.0/exec/wgrib2 and "export", and
there
>> wasn't a complain about "Executable WGRIB2 does not exist at
$WGRIB2".
>>
>> Thanks again -
>>
>> Ying
>>
>>
>> On 5/29/19 1:53 PM, George McCabe via RT wrote:
>> > Hi Ying,
>> >
>> > I am just following up on this ticket. Have you tried running
your case
>> on METplus v2.1? Let me know if you run into any other issues.
>> >
>> > Thanks,
>> > George
>> >
>> > On Thu Apr 18 19:15:58 2019, mccabe wrote:
>> >> Hi Ying,
>> >>
>> >> It looks like the bug with subtract mode is in v2.0.4. It will
be
>> >> fixed in
>> >> the next release. You can look at the develop branch on GitHub
to see
>> >> what
>> >> has changed in the subtract function. Have a good weekend!
>> >>
>> >> On Thu, Apr 18, 2019, 5:09 PM Ying Lin via RT
<met_help at ucar.edu>
>> >> wrote:
>> >>
>> >>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789 >
>> >>>
>> >>> Hi George,
>> >>>
>> >>> Thank you - you're right, replacing that first
>> >>> FCST_GRID_STAT_INPUT_TEMPLATE with
FCST_PCP_COMBINE_INPUT_TEMPLATE
>> >>> got
>> >>> me past the err msg! There're still some old stuff with the
way the
>> >>> summing was done - similar to the odd result I mentioned in the
last
>> >>> email with the 2.0.4 precip_continuous.conf. I'll study the
new
>> >>> precip_continuous.conf in the github and follow up with you
next
>> >>> week.
>> >>>
>> >>> Thanks for your help and patience today. Have a great weekend
-
>> >>>
>> >>> Ying
>> >>>
>> >>> On 4/18/19 4:25 PM, George McCabe via RT wrote:
>> >>>> Hi Ying,
>> >>>>
>> >>>> I am sorry, I was incorrect that the fixed precip_continuous
config
>> >>>> file
>> >>>> was updated in 2.0.4. It looks like the fix was made after
that
>> >>>> release.
>> >>> It
>> >>>> will be in the newest release. You can see the updated file on
>> >>>> GitHub in
>> >>>> the develop branch here:
>> >>>>
>> >>>>
>> >>>
>>
https://github.com/NCAR/METplus/blob/develop/parm/use_cases/grid_to_grid/examples/precip_continuous.conf
>> >>>> For your use case, it looks like you don't have
>> >>>> FCST_PCP_COMBINE_INPUT_TEMPLATE set and you have 2 for
grid_stat:
>> >>>>
>> >>>> FCST_GRID_STAT_INPUT_TEMPLATE =
>> >>>>
>> >>>
>>
gfs.{init?fmt=%Y%m%d}/{init?fmt=%H}/gfs.t{init?fmt=%H}z.pgrb2.0p25.f{lead?fmt=%HHH}
>> >>>> FCST_PCP_COMBINE_OUTPUT_TEMPLATE =
>> >>>> gfs.{valid?fmt=%Y%m%d%H}_A{level?fmt=%HHH}h
>> >>>> FCST_GRID_STAT_INPUT_TEMPLATE =
>> >>> gfs.{valid?fmt=%Y%m%d%H}_A{level?fmt=%HHH}h
>> >>>> Fixing this may solve your issue. Please let me know if it
does
>> >>>> not. I
>> >>> have
>> >>>> to catch a flight this evening so I am going to stop working
very
>> >>>> soon,
>> >>> but
>> >>>> if you reply very soon I may still be able to help you before
the
>> >>> weekend.
>> >>>> Thanks,
>> >>>> George
>> >>>>
>> >>>> On Thu, Apr 18, 2019 at 8:13 PM Ying Lin via RT
<met_help at ucar.edu>
>> >>> wrote:
>> >>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789
>
>> >>>>>
>> >>>>> Hi George,
>> >>>>>
>> >>>>> Thank you for the quick reply. I was looking at
>> >>>>>
>> >>> /gpfs/gd1/emc/global/noscrub/Julie.Prestopnik/METplus/METplus-
>> >>>
2.0.4/parm/use_cases/grid_to_grid/examples/precip_continuous.conf,
>> >>>>> which has VALID_BEG = 2018030900, VALID_END = 2018030900.
Date
>> >>>>> stamp is
>> >>>>> Jan 30. Is there a "newer version of v2.0.4" available on
wcoss?
>> >>>>> From
>> >>>>> the above config it's not clear to me how METplus knows to
>> >>>>> subtract the
>> >>>>> gfs.*.f024 from the gfs.*.f048.
>> >>>>>
>> >>>>> If you have time today, could you also take a look why my
simple
>> >>>>> test
>> >>>>> (conf set up to be as close to precip_continuous.conf as
possible
>> >>>>> -
>> >>>>>
>> >>>
>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf
>> )
>> >>>>> led to err msg in
>> >>>>>
>> >>>
>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs.out.test5
>> >>> ?
>> >>>>> It seems that I corrected the problems you pointed out in
your
>> >>>>> reply
>> >>>>> this morning, but the err msg still seems to be the same as
>> >>>>> yesterday's.
>> >>>>>
>> >>>>> If you don't have time today, next Monday is fine too. Have
a
>> >>>>> good
>> >>>>> holiday weekend.
>> >>>>>
>> >>>>> Ying
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> On 4/18/19 3:51 PM, George McCabe via RT wrote:
>> >>>>>> Hi Ying,
>> >>>>>>
>> >>>>>> 1) The config file I see has a valid time of 2018031000 and
48 hr
>> >>>>> forecast
>> >>>>>> lead. You may be using an old version of METplus that had
this
>> >>>>>> set up
>> >>>>>> incorrectly. METplus version 2.0.4 should be correct.
>> >>>>>>
>> >>>>>> 2) pcp_combine -subtract does not actually use the
FCST_LEVEL
>> >>>>>> variable,
>> >>>>> so
>> >>>>>> that should be removed from the use case. The code takes the
>> >>>>>> valid time
>> >>>>> and
>> >>>>>> subtracts the accumulation.
>> >>>>>>
>> >>>>>> 3) In the upcoming version of METplus (in a few weeks!) has
a
>> >>>>>> good
>> >>> number
>> >>>>>> of config changes from the previous versions. For example,
>> >>>>>> FCST_LEVEL
>> >>>>> will
>> >>>>>> now be FCST_PCP_COMBINE_INPUT_LEVEL. These changes are
designed
>> >>>>>> to make
>> >>>>> the
>> >>>>>> variables more clear to what they actually do. In the new
>> >>>>>> version,
>> >>> there
>> >>>>> is
>> >>>>>> now FCST_PCP_COMBINE_METHOD and OBS_PCP_COMBINE_METHOD, so
you
>> >>>>>> will be
>> >>>>> able
>> >>>>>> to specify a different method for your forecast and analysis
>> >>>>>> data.
>> >>>>>>
>> >>>>>> FCST_PCP_COMBINE_RUN = True
>> >>>>>> FCST_PCP_COMBINE_METHOD = SUBTRACT
>> >>>>>>
>> >>>>>> OBS_PCP_COMBINE_RUN = True
>> >>>>>> OBS_PCP_COMBINE_METHOD = SUM
>> >>>>>>
>> >>>>>> This functionality has not been thoroughly tested as we
haven't
>> >>>>>> had any
>> >>>>>> example use cases that need to do this so far, so please let
me
>> >>>>>> know if
>> >>>>> it
>> >>>>>> does not work the way you expect and I will fix it.
>> >>>>>>
>> >>>>>> Just to let you know, I will be out of the office tomorrow
>> >>>>>> (Friday)
>> >>> and I
>> >>>>>> wont have access to email. Feel free to send any follow-up
emails
>> >>>>>> to me
>> >>>>> and
>> >>>>>> I will get back to you on Monday.
>> >>>>>>
>> >>>>>> Thanks,
>> >>>>>> George
>> >>>>>>
>> >>>>>> On Thu, Apr 18, 2019 at 7:38 PM Ying Lin via RT
>> >>>>>> <met_help at ucar.edu>
>> >>>>> wrote:
>> >>>>>>> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789 >
>> >>>>>>>
>> >>>>>>> Thank you very much George, this is very helpful.
>> >>>>>>>
>> >>>>>>> About the example in
>> >>>>>>>
parm/use_cases/grid_to_grid/examples/precip_continuous.conf
>> >>>>>>> I tried running it and it seems to have run through the
>> >>>>>>> pcp_combine
>> >>> step
>> >>>>>>> fine: it did this:
>> >>>>>>>
/global/noscrub/Julie.Prestopnik/met/8.0/bin/pcp_combine
>> >>> -subtract
>> >>> /global/noscrub/Julie.Prestopnik/METplus/METplus-
>> >>>
>>
2.0.4_sample_data/grid_to_grid/prfv3rt1/20180308/gfs.t00z.pgrb.1p00.f048
>> >>>>>>> 48
>> >>>>>>>
>> >>> /global/noscrub/Julie.Prestopnik/METplus/METplus-
>> >>>
>>
2.0.4_sample_data/grid_to_grid/prfv3rt1/20180308/gfs.t00z.pgrb.1p00.f024
>> >>>>>>> 24
>> >>>>>>>
/meso/noscrub/Ying.Lin/metplus.out/gfs/bucket/gfs.2018030900_A024h
>> >>>>>>>
>> >>>>>>> Several questions about the example:
>> >>>>>>> 1) The precip_continuous.conf has
>> >>>>>>> LEAD_SEQ = 24
>> >>>>>>> I thought it meant forecast lead of 24h, e.g. look for 00-
24h
>> >>>>>>> forecast
>> >>>>>>> valid at 2018030900 (given in the config). If so, why
doesn't
>> >>>>>>> it
>> >>>>>>> simply use gfs.t00z.pgrb.1p00.f024, which already have the
00-
>> >>>>>>> 24h
>> >>>>>>> forecast? If it subtracts gfs.t00z.pgrb.1p00.f024 from
>> >>>>>>> gfs.t00z.pgrb.1p00.f048, the resulting file would have 24-
48h
>> >>>>>>> forecast
>> >>>>>>> valid at 2018031000. I did
>> >>>>>>> ncdump -h
>> >>>>>>>
/meso/noscrub/Ying.Lin/metplus.out/gfs/bucket/gfs.2018030900_A024h
>> >>>>>>>
>> >>>>>>> and got
>> >>>>>>> APCP_24:init_time = "20180308_000000" ;
>> >>>>>>> APCP_24:init_time_ut = "1520467200" ;
>> >>>>>>> APCP_24:valid_time = "20180310_000000" ;
>> >>>>>>> APCP_24:valid_time_ut = "1520640000" ;
>> >>>>>>> APCP_24:accum_time = "240000" ;
>> >>>>>>> it seems to mean that the file contains APCP_24h with
initial
>> >>>>>>> time of
>> >>>>>>> 00z 03/08 and the valid time is 00Z 03/10 (that'll be
APCP_48h
>> >>>>>>> instead
>> >>>>>>> of 24h?)
>> >>>>>>>
>> >>>>>>> 2) The precip_continuous.conf has
>> >>>>>>> FCST_LEVEL = 6
>> >>>>>>> I thought FCST_LEVEL is the forecast bucket length. In the
>> >>>>>>> sample GFS
>> >>>>>>> files, there isn't a 6-hourly bucket -
gfs.t00z.pgrb.1p00.f024
>> >>> contains
>> >>>>>>> 00-24h accumulation and gfs.t00z.pgrb.1p00.f048 contains
00-48h
>> >>>>>>> accumulation. Does FCST_LEVEL mean something other than
the
>> >>>>>>> forecast
>> >>>>>>> bucket length?
>> >>>>>>>
>> >>>>>>> 3) precip_continuous.conf has
>> >>>>>>> PCP_COMBINE_METHOD = SUBTRACT
>> >>>>>>> which should work for the example since the verifying
analysis
>> >>>>>>> is the
>> >>>>>>> 24h CCPA file. But what if I need to use 'SUBTRACT' for
the
>> >>>>>>> forecast
>> >>>>>>> field but 'SUM' for the analysis field (e.g. summing up 6h
CCPAs
>> >>>>>>> into
>> >>> a
>> >>>>>>> 24h total)? At one point I tried
>> >>>>>>> PCP_COMBINE_METHOD = SUBTRACT, SUM
>> >>>>>>> and got
>> >>>>>>> ERROR: Invalid PCP_COMBINE_METHOD specified
>> >>>>>>>
>> >>>>>>> After several failed attempts getting my own config file to
>> >>>>>>> work, I
>> >>>>>>> starting from precip_continuous.conf (which worked through
the
>> >>>>>>> PcpCombine step; I think that's the only step it runs since
the
>> >>>>>>> config
>> >>>>>>> has only PcpCombine in the PROCESS_LIST) and tried to set a
>> >>>>>>> config
>> >>> file
>> >>>>>>> as close to precip_continuous.conf as possible
>> >>>>>>>
>> >>>
>>
(/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/parm/ylin_config/models/fv3gfs_24h.conf,
>> >>>>>>> also uploaded to
>> >>>>>>>
>> >>>
>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf
>> >>>>> ).
>> >>>>>>> The resulting err msg:
>> >>>>>>>
>> >>>>>>> 04/18 18:35:33.912 jlog (master_metplus.py:217) CRITICAL:
>> >>>>>>> master_metplus failed: ''
>> >>>>>>> Traceback (most recent call last):
>> >>>>>>> File "/meso/save/Ying.Lin/metplus/ush/master_metplus.py",
line
>> >>> 213,
>> >>>>>>> in <module>
>> >>>>>>> main()
>> >>>>>>> File
"/meso/save/Ying.Lin/metplus/ush/master_metplus.py",
>> >>>>>>> line
>> >>> 170,
>> >>>>>>> in main
>> >>>>>>> process.run_at_time(-1, run_time)
>> >>>>>>> File
>> >>>>>>>
>> >>>
>>
"/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/ush/reformat_gridded_wrapper.py",
>> >>>>>>> line 80, in run_at_time
>> >>>>>>>
>> >>>>>>> Full output file:
>> >>>>>>>
>> >>>
>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs.out.test5
>> >>>>> ;
>> >>>>>>> metplus_final.conf:
>> >>>>>>>
>> >>>>>>>
>> >>>
>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/metplus_final.conf
>> >>>>>>> The quoted err msg above appears to the the same as in my
email
>> >>>>>>> yesterday, so there must still be something wrong with the
>> >>>>>>> format of
>> >>> my
>> >>>>>>> config, though I think I eliminated the problems you
pointed out
>> >>>>>>> in
>> >>> your
>> >>>>>>> reply. There wouldn't be a ccpa_conus_1.0d_* of the valid
day I
>> >>>>>>> specified, but I think the processing failed way before
that
>> >>>>>>> point.
>> >>>>>>>
>> >>>>>>> Thank you again -
>> >>>>>>>
>> >>>>>>> Ying
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> On 4/17/19 3:32 PM, George McCabe via RT wrote:
>> >>>>>>>> Hi Ying,
>> >>>>>>>>
>> >>>>>>>> You will need to set FCST_PCP_COMBINE_METHOD = SUBTRACT to
>> >>>>>>>> config
>> >>>>> METplus
>> >>>>>>>> to subtract accumulations files to get the accumulation
you
>> >>>>>>>> desire.
>> >>> The
>> >>>>>>>> following use case has an example of this configuration:
>> >>>>>>>>
>> >>>>>>>> parm/use_cases/grid_to_grid/examples/precip_continous.conf
>> >>>>>>>>
>> >>>>>>>> I also noticed that your config file has the following
items in
>> >>>>>>>> the
>> >>>>>>>> [config] section:
>> >>>>>>>>
>> >>>>>>>> FCST_PCP_COMBINE_INPUT_DIR =
/gpfs/dell1/nco/ops/com/gfs/para
>> >>>>>>>> FCST_PCP_COMBINE_OUTPUT_DIR =
{OUTPUT_BASE}/qpf_grib/bucket
>> >>>>>>>> FCST_GRID_STAT_INPUT_DIR = {OUTPUT_BASE}/qpf_grib/bucket
>> >>>>>>>>
>> >>>>>>>> You should move these to the [dir] section.
>> >>>>>>>>
>> >>>>>>>> Please let me know if you continue to have issues. The
error
>> >>>>>>>> message
>> >>>>> you
>> >>>>>>>> provided has more information at the bottom that was not
>> >>>>>>>> included.
>> >>> The
>> >>>>>>> next
>> >>>>>>>> release of METplus contains more extensive error checking
so
>> >>>>>>>> the
>> >>> error
>> >>>>>>>> message from that version should be much more helpful.
>> >>>>>>>>
>> >>>>>>>> Thanks,
>> >>>>>>>> George
>> >>>>>>>>
>> >>>>>>>> On Wed, Apr 17, 2019 at 12:43 PM Ying Lin via RT
>> >>>>>>>> <met_help at ucar.edu>
>> >>>>>>> wrote:
>> >>>>>>>>> Wed Apr 17 12:43:02 2019: Request 89789 was acted upon.
>> >>>>>>>>> Transaction: Ticket created by ying.lin at noaa.gov
>> >>>>>>>>> Queue: met_help
>> >>>>>>>>> Subject: Using FCST_PCP_COMBINE in METplus
>> >>>>>>>>> Owner: Nobody
>> >>>>>>>>> Requestors: ying.lin at noaa.gov
>> >>>>>>>>> Status: new
>> >>>>>>>>> Ticket <URL:
>> >>>>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789
>> >>>>>>>>> Hello. I haven't been able to get the FCST_PCP_COMBINE
>> >>>>>>>>> feature to
>> >>>>> work
>> >>>>>>>>> in METplus, running out of ideas and hope you can help.
>> >>>>>>>>>
>> >>>>>>>>> I've put some examples in
>> >>>>>>>>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/
>> >>>>>>>>> (current
>> >>>>>>> test
>> >>>>>>>>> directory:
>> >>>>>>>>> gyre:/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/test).
>> >>>>>>>>> test_fv3gfs_24h.ksh is the simple test script. The goal
is to
>> >>>>> compute a
>> >>>>>>>>> 24h verification ending at 12Z 20190415.
>> >>>>>>>>>
>> >>>>>>>>> 'test1' did not need FCST_PCP_COMBINE - the config file
>> >>>>>>>>> fv3gfs_24h.conf.test1 simply had LEAD_SEQ set to '24',
>> >>>>>>>>> FCST_PCP_COMBINE_RUN was not set. The FV3GFS had two
precip
>> >>> buckets,
>> >>>>> 6h
>> >>>>>>>>> and continuously-accumulating, and METplus easily found
the
>> >>>>>>>>> 24h
>> >>>>> forecast
>> >>>>>>>>> accumulation and went on with the grid_stat.
>> >>>>>>>>>
>> >>>>>>>>> 'test2' had 'LEAD_SEQ = 36', and I added the following in
the
>> >>>>>>>>> config
>> >>>>>>>>> (
>> >>>>>>>>>
>> >>>
>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test2
>> >>>>>>>>> ):
>> >>>>>>>>> FCST_PCP_COMBINE_RUN = True
>> >>>>>>>>> FCST_PCP_COMBINE_INPUT_DATATYPE = GRIB
>> >>>>>>>>> FCST_PCP_COMBINE_INPUT_DIR =
>> >>> /gpfs/dell1/nco/ops/com/gfs/para
>> >>>>>>>>> FCST_PCP_COMBINE_OUTPUT_DIR =
{OUTPUT_BASE}/qpf_grib/bucket
>> >>>>>>>>> FCST_GRID_STAT_INPUT_DIR = {OUTPUT_BASE}/qpf_grib/bucket
>> >>>>>>>>> FCST_MAX_FORECAST = 36
>> >>>>>>>>> FCST_INIT_INTERVAL = 6
>> >>>>>>>>> FCST_LEVEL = 36
>> >>>>>>>>> Here the goal was to see if I could get METplus to
subtract
>> >>>>>>>>> the
>> >>> 00-12h
>> >>>>>>>>> accumulation from gfs.t00z.pgrb2.0p25.f012 from the 00-
36h
>> >>>>> accumulation
>> >>>>>>>>> in gfs.t00z.pgrb2.0p25.f036. Thinking that FCST_LEVEL is
the
>> >>> forecast
>> >>>>>>>>> bucket length, I figured setting it to '36' and
artificially
>> >>>>> specifying
>> >>>>>>>>> FCST_MAX_FORECAST = 36 (instead of the full 384h) should
work
>> >>>>>>>>> for
>> >>> this
>> >>>>>>>>> limited test where LEAD_SEQ is set to 36 (is there a way
to
>> >>>>>>>>> specify
>> >>>>> that
>> >>>>>>>>> the bucket never gets emptied?). It failed with the
follow
>> >>>>>>>>> err msg
>> >>>>>>>>> (full message in
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>
>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test2
>> >>>>>>>>> ):
>> >>>>>>>>>
>> >>>>>>>>> 04/16 20:01:42.754 jlog (master_metplus.py:217) CRITICAL:
>> >>>>>>>>> master_metplus failed: ''
>> >>>>>>>>> Traceback (most recent call last):
>> >>>>>>>>> File
>> >>>>>>>>> "/meso/save/Ying.Lin/metplus/ush/master_metplus.py", line
>> >>>>> 213,
>> >>>>>>>>> in <module>
>> >>>>>>>>> main()
>> >>>>>>>>> File
>> >>>>>>>>> "/meso/save/Ying.Lin/metplus/ush/master_metplus.py", line
>> >>>>> 170,
>> >>>>>>>>> in main
>> >>>>>>>>> process.run_at_time(-1, run_time)
>> >>>>>>>>> File
>> >>>>>>>>>
>> >>>
>>
"/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/ush/reformat_gridded_wrapper.py",
>> >>>>>>>>> line 80, in run_at_time
>> >>>>>>>>> self.run_at_time_once(task_info, var_info, rl)
>> >>>>>>>>> File
>> >>>>>>>>>
>> >>>
"/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/ush/pcp_combine_wrapper.py",
>> >>>>>>>>> line 419, in run_at_time_once
>> >>>>>>>>> ....
>> >>>>>>>>>
>> >>>>>>>>> 'test3' is an attempt from a different angle, thinking
that if
>> >>>>>>>>> the
>> >>>>>>>>> 'continuously running' bucket didn't work, maybe the 6-
hourly
>> >>>>>>>>> bucket
>> >>>>>>>>> would. In
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>
>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test3
>> >>>>>>> ,
>> >>>>>>>>> I changed LEAD_SEQ to 24, and set FCST_LEVEL = 6. It
produced
>> >>>>>>>>> err
>> >>>>> msgs
>> >>>>>>>>> (
>> >>>>>>>>>
>> >>>
>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test3
>> >>>>>>> )
>> >>>>>>>>> that appeared to be the same as that from 'test2'.
>> >>>>>>>>>
>> >>>>>>>>> Earlier on I tried using the FCST_PCP_COMBINE feature
with the
>> >>>>>>>>> NAM
>> >>>>>>>>> 'CONUSNEST' output (with simple regular 6-hourly bucket)
and
>> >>>>>>>>> got the
>> >>>>>>>>> same err msg, so I don't think FV3GFS's dual-bucket
system is
>> >>>>>>>>> the
>> >>>>> cause.
>> >>>>>>>>> Also posted on
>> >>>>>>>>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/ are
>> >>>>>>>>> the
>> >>>>> two
>> >>>>>>>>> metplus_final.conf files from both 'test2' and 'test3'.
>> >>>>>>>>>
>> >>>>>>>>> I found in
>> >>>>>>>>>
>> >>> /global/noscrub/Julie.Prestopnik/METplus/METplus-
>> >>> 2.0.4/parm/use_cases/qpf/examples/
>> >>>>>>>>> that hrefmean-vs-mrms-qpe.conf and nationalblend-vs-mrms-
>> >>>>>>>>> qpe.conf
>> >>> had
>> >>>>>>>>> "FCST_PCP_COMBINE_RUN = True", that's where I borrowed
the
>> >>>>>>>>> FCST_PCP_COMBINE_* set up in my fv3gfs_24h.conf.test2 and
>> >>>>>>>>> fv3gfs_24h.conf.test3. Haven't been able to run the
hrefmean
>> >>>>>>>>> and
>> >>>>>>>>> nationalblend tests myself however - got err msg of "
>> >>>>>>>>> CRITICAL:
>> >>>>>>>>> master_metplus failed: No option 'GEMPAKTOCF_CLASSPATH'
in
>> >>>>>>>>> section:
>> >>>>>>>>> 'exe'" from both attempts.
>> >>>>>>>>>
>> >>>>>>>>> Thank you for looking into this -
>> >>>>>>>>>
>> >>>>>>>>> Ying
>> >>>>>>>>>
>> >>>>>>>>> --
>> >>>>>>>>> Ying Lin
>> >>>>>>>>> NCEP/EMC/Verification, Post-processing and Product
Generation
>> >>>>>>>>> Branch
>> >>>>>>>>> NCWCP Cubicle No. 2015
>> >>>>>>>>> Ying.Lin at noaa.gov
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>> --
>> >>>>>>> Ying Lin
>> >>>>>>> NCEP/EMC/Verification, Post-processing and Product
Generation
>> >>>>>>> Branch
>> >>>>>>> NCWCP Cubicle No. 2015
>> >>>>>>> Ying.Lin at noaa.gov
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>
>> >>>>> --
>> >>>>> Ying Lin
>> >>>>> NCEP/EMC/Verification, Post-processing and Product Generation
>> >>>>> Branch
>> >>>>> NCWCP Cubicle No. 2015
>> >>>>> Ying.Lin at noaa.gov
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>> --
>> >>> Ying Lin
>> >>> NCEP/EMC/Verification, Post-processing and Product Generation
Branch
>> >>> NCWCP Cubicle No. 2015
>> >>> Ying.Lin at noaa.gov
>> >>>
>> >>>
>> >>>
>> >>>
>> >
>> >
>>
>> --
>> Ying Lin
>> NCEP/EMC/Verification, Post-processing and Product Generation
Branch
>> NCWCP Cubicle No. 2015
>> Ying.Lin at noaa.gov
>>
>>
>>
>>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #89789] Using FCST_PCP_COMBINE in METplus
From: Ying Lin
Time: Thu May 30 14:37:29 2019
Hi George, i was writing to you when your email below came in ...
here
it is:
Thanks very much! In my system.conf.wcoss, I changed
WGRIB2 = $WGRIB2
to
WGRIB2 = {ENV[WGRIB2]}
and that seems to get rid of the $WGRIB2 complaint.
Now I'm getting a new err msg:
metplus.PcpCombine (config_wrapper.py:93) ERROR: Requested conf
[exe] GEMPAKTOCF_CLASSPATH was not set in config file
I see that in my system.conf.wcoss there was this commented out line:
#GEMPAKTOCF_BIN = /d1/mccabe/bin/GempakToCF
What should I set GEMPAKTOCF_BIN [and/or _CLASSPATH] to? Or is there
an
option to turn the GEMPAKTOCF off, If I won't be using this function?
Thanks again -
Ying
On 5/30/19 4:32 PM, George McCabe via RT wrote:
> The reason you didn't have this issue previously is your use case
likely
> didn't need to use any of those exes. I added a function to check
and add
> all of the non-MET executables to all of the MET wrappers in case
they are
> needed, but I realized later than they are not necessary for most of
the
> cases. However, they may become required for other cases so we may
leave
> the requirement to set them correctly to run METplus.
>
> On Thu, May 30, 2019 at 1:36 PM George McCabe <mccabe at ucar.edu>
wrote:
>
>> Hi Ying,
>>
>> To reference an environment variable in the METplus config files,
you need
>> to use this syntax:
>>
>> WGRIB2 = {ENV[WGRIB2]}
>>
>> Since you have set WGRIB2 environment variable, this should set the
>> METplus config variable correctly. You may also be able to set
WGRIB2 =
>> wgrib2 in the METplus config file instead. If the exe is in the
path of the
>> user running METplus, it will find the app properly. Let me know if
that
>> doesn't work.
>>
>> On Thu, May 30, 2019 at 11:41 AM Ying Lin via RT
<met_help at ucar.edu>
>> wrote:
>>
>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789 >
>>>
>>> Hi George,
>>>
>>> Thanks for following up! The sample case in
>>> parm/use_cases/grid_to_grid/examples/precip_continuous.conf seems
to
>>> "subtract" correctly now (it was off by 24h in v2.0.4). The
>>> precip_continuous.conf only deals with the PcpCombine step though.
>>> When I added the regrid_data_plane step to test a fv3gfs verif on
G212,
>>> I got this err msg:
>>> metplus.GridStat (config_wrapper.py:144) ERROR: Executable
WGRIB2
>>> does not exist at $WGRIB2
>>> I thought there must have been a problem in my model config for
the v2.1
>>> (found several actually) but when I run the use case at
>>> parm/use_cases/qpf/examples/hrefmean-vs-qpe.conf the same thing
happened.
>>>
>>> I'm attaching three files to this email:
>>>
>>> 1. test_hrefmean.ksh: script to run this use case
>>> 2. hrefmean.out: output for running the above script. You can
see that
>>> after loading model grid_util (from ". ~/dot.for.metplus)
there was
>>> "WGRIB2=/nwprod2/grib_util.v1.0.0/exec/wgrib2
>>> export WGRIB2"; and at the end there is the err msg
>>> "metplus.GridStat (config_wrapper.py:144) ERROR: Executable
WGRIB2
>>> does not exist at $WGRIB2"
>>> 3. metplus_final.conf from the above run
>>> 4. my "dot.for.metplus" file invoked by test_hrefmean.ksh. I
verified
>>> manually - after "module load grib_util", $WGRIB2 does point
to
>>> /nwprod2/grib_util.v1.0.0/exec/wgrib2
>>> 5. my system.conf.wcoss invoked by test_hrefmean.ksh.
>>>
>>> I looked back at run output from METplus-2.0.4, there was also
>>> WGRIB2=/nwprod2/grib_util.v1.0.0/exec/wgrib2 and "export", and
there
>>> wasn't a complain about "Executable WGRIB2 does not exist at
$WGRIB2".
>>>
>>> Thanks again -
>>>
>>> Ying
>>>
>>>
>>> On 5/29/19 1:53 PM, George McCabe via RT wrote:
>>>> Hi Ying,
>>>>
>>>> I am just following up on this ticket. Have you tried running
your case
>>> on METplus v2.1? Let me know if you run into any other issues.
>>>> Thanks,
>>>> George
>>>>
>>>> On Thu Apr 18 19:15:58 2019, mccabe wrote:
>>>>> Hi Ying,
>>>>>
>>>>> It looks like the bug with subtract mode is in v2.0.4. It will
be
>>>>> fixed in
>>>>> the next release. You can look at the develop branch on GitHub
to see
>>>>> what
>>>>> has changed in the subtract function. Have a good weekend!
>>>>>
>>>>> On Thu, Apr 18, 2019, 5:09 PM Ying Lin via RT
<met_help at ucar.edu>
>>>>> wrote:
>>>>>
>>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789 >
>>>>>>
>>>>>> Hi George,
>>>>>>
>>>>>> Thank you - you're right, replacing that first
>>>>>> FCST_GRID_STAT_INPUT_TEMPLATE with
FCST_PCP_COMBINE_INPUT_TEMPLATE
>>>>>> got
>>>>>> me past the err msg! There're still some old stuff with the
way the
>>>>>> summing was done - similar to the odd result I mentioned in the
last
>>>>>> email with the 2.0.4 precip_continuous.conf. I'll study the
new
>>>>>> precip_continuous.conf in the github and follow up with you
next
>>>>>> week.
>>>>>>
>>>>>> Thanks for your help and patience today. Have a great weekend
-
>>>>>>
>>>>>> Ying
>>>>>>
>>>>>> On 4/18/19 4:25 PM, George McCabe via RT wrote:
>>>>>>> Hi Ying,
>>>>>>>
>>>>>>> I am sorry, I was incorrect that the fixed precip_continuous
config
>>>>>>> file
>>>>>>> was updated in 2.0.4. It looks like the fix was made after
that
>>>>>>> release.
>>>>>> It
>>>>>>> will be in the newest release. You can see the updated file on
>>>>>>> GitHub in
>>>>>>> the develop branch here:
>>>>>>>
>>>>>>>
>>>
https://github.com/NCAR/METplus/blob/develop/parm/use_cases/grid_to_grid/examples/precip_continuous.conf
>>>>>>> For your use case, it looks like you don't have
>>>>>>> FCST_PCP_COMBINE_INPUT_TEMPLATE set and you have 2 for
grid_stat:
>>>>>>>
>>>>>>> FCST_GRID_STAT_INPUT_TEMPLATE =
>>>>>>>
>>>
gfs.{init?fmt=%Y%m%d}/{init?fmt=%H}/gfs.t{init?fmt=%H}z.pgrb2.0p25.f{lead?fmt=%HHH}
>>>>>>> FCST_PCP_COMBINE_OUTPUT_TEMPLATE =
>>>>>>> gfs.{valid?fmt=%Y%m%d%H}_A{level?fmt=%HHH}h
>>>>>>> FCST_GRID_STAT_INPUT_TEMPLATE =
>>>>>> gfs.{valid?fmt=%Y%m%d%H}_A{level?fmt=%HHH}h
>>>>>>> Fixing this may solve your issue. Please let me know if it
does
>>>>>>> not. I
>>>>>> have
>>>>>>> to catch a flight this evening so I am going to stop working
very
>>>>>>> soon,
>>>>>> but
>>>>>>> if you reply very soon I may still be able to help you before
the
>>>>>> weekend.
>>>>>>> Thanks,
>>>>>>> George
>>>>>>>
>>>>>>> On Thu, Apr 18, 2019 at 8:13 PM Ying Lin via RT
<met_help at ucar.edu>
>>>>>> wrote:
>>>>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789
>
>>>>>>>>
>>>>>>>> Hi George,
>>>>>>>>
>>>>>>>> Thank you for the quick reply. I was looking at
>>>>>>>>
>>>>>> /gpfs/gd1/emc/global/noscrub/Julie.Prestopnik/METplus/METplus-
>>>>>>
2.0.4/parm/use_cases/grid_to_grid/examples/precip_continuous.conf,
>>>>>>>> which has VALID_BEG = 2018030900, VALID_END = 2018030900.
Date
>>>>>>>> stamp is
>>>>>>>> Jan 30. Is there a "newer version of v2.0.4" available on
wcoss?
>>>>>>>> From
>>>>>>>> the above config it's not clear to me how METplus knows to
>>>>>>>> subtract the
>>>>>>>> gfs.*.f024 from the gfs.*.f048.
>>>>>>>>
>>>>>>>> If you have time today, could you also take a look why my
simple
>>>>>>>> test
>>>>>>>> (conf set up to be as close to precip_continuous.conf as
possible
>>>>>>>> -
>>>>>>>>
>>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf
>>> )
>>>>>>>> led to err msg in
>>>>>>>>
>>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs.out.test5
>>>>>> ?
>>>>>>>> It seems that I corrected the problems you pointed out in
your
>>>>>>>> reply
>>>>>>>> this morning, but the err msg still seems to be the same as
>>>>>>>> yesterday's.
>>>>>>>>
>>>>>>>> If you don't have time today, next Monday is fine too. Have
a
>>>>>>>> good
>>>>>>>> holiday weekend.
>>>>>>>>
>>>>>>>> Ying
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On 4/18/19 3:51 PM, George McCabe via RT wrote:
>>>>>>>>> Hi Ying,
>>>>>>>>>
>>>>>>>>> 1) The config file I see has a valid time of 2018031000 and
48 hr
>>>>>>>> forecast
>>>>>>>>> lead. You may be using an old version of METplus that had
this
>>>>>>>>> set up
>>>>>>>>> incorrectly. METplus version 2.0.4 should be correct.
>>>>>>>>>
>>>>>>>>> 2) pcp_combine -subtract does not actually use the
FCST_LEVEL
>>>>>>>>> variable,
>>>>>>>> so
>>>>>>>>> that should be removed from the use case. The code takes the
>>>>>>>>> valid time
>>>>>>>> and
>>>>>>>>> subtracts the accumulation.
>>>>>>>>>
>>>>>>>>> 3) In the upcoming version of METplus (in a few weeks!) has
a
>>>>>>>>> good
>>>>>> number
>>>>>>>>> of config changes from the previous versions. For example,
>>>>>>>>> FCST_LEVEL
>>>>>>>> will
>>>>>>>>> now be FCST_PCP_COMBINE_INPUT_LEVEL. These changes are
designed
>>>>>>>>> to make
>>>>>>>> the
>>>>>>>>> variables more clear to what they actually do. In the new
>>>>>>>>> version,
>>>>>> there
>>>>>>>> is
>>>>>>>>> now FCST_PCP_COMBINE_METHOD and OBS_PCP_COMBINE_METHOD, so
you
>>>>>>>>> will be
>>>>>>>> able
>>>>>>>>> to specify a different method for your forecast and analysis
>>>>>>>>> data.
>>>>>>>>>
>>>>>>>>> FCST_PCP_COMBINE_RUN = True
>>>>>>>>> FCST_PCP_COMBINE_METHOD = SUBTRACT
>>>>>>>>>
>>>>>>>>> OBS_PCP_COMBINE_RUN = True
>>>>>>>>> OBS_PCP_COMBINE_METHOD = SUM
>>>>>>>>>
>>>>>>>>> This functionality has not been thoroughly tested as we
haven't
>>>>>>>>> had any
>>>>>>>>> example use cases that need to do this so far, so please let
me
>>>>>>>>> know if
>>>>>>>> it
>>>>>>>>> does not work the way you expect and I will fix it.
>>>>>>>>>
>>>>>>>>> Just to let you know, I will be out of the office tomorrow
>>>>>>>>> (Friday)
>>>>>> and I
>>>>>>>>> wont have access to email. Feel free to send any follow-up
emails
>>>>>>>>> to me
>>>>>>>> and
>>>>>>>>> I will get back to you on Monday.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> George
>>>>>>>>>
>>>>>>>>> On Thu, Apr 18, 2019 at 7:38 PM Ying Lin via RT
>>>>>>>>> <met_help at ucar.edu>
>>>>>>>> wrote:
>>>>>>>>>> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789 >
>>>>>>>>>>
>>>>>>>>>> Thank you very much George, this is very helpful.
>>>>>>>>>>
>>>>>>>>>> About the example in
>>>>>>>>>>
parm/use_cases/grid_to_grid/examples/precip_continuous.conf
>>>>>>>>>> I tried running it and it seems to have run through the
>>>>>>>>>> pcp_combine
>>>>>> step
>>>>>>>>>> fine: it did this:
>>>>>>>>>>
/global/noscrub/Julie.Prestopnik/met/8.0/bin/pcp_combine
>>>>>> -subtract
>>>>>> /global/noscrub/Julie.Prestopnik/METplus/METplus-
>>>>>>
>>>
2.0.4_sample_data/grid_to_grid/prfv3rt1/20180308/gfs.t00z.pgrb.1p00.f048
>>>>>>>>>> 48
>>>>>>>>>>
>>>>>> /global/noscrub/Julie.Prestopnik/METplus/METplus-
>>>>>>
>>>
2.0.4_sample_data/grid_to_grid/prfv3rt1/20180308/gfs.t00z.pgrb.1p00.f024
>>>>>>>>>> 24
>>>>>>>>>>
/meso/noscrub/Ying.Lin/metplus.out/gfs/bucket/gfs.2018030900_A024h
>>>>>>>>>>
>>>>>>>>>> Several questions about the example:
>>>>>>>>>> 1) The precip_continuous.conf has
>>>>>>>>>> LEAD_SEQ = 24
>>>>>>>>>> I thought it meant forecast lead of 24h, e.g. look for 00-
24h
>>>>>>>>>> forecast
>>>>>>>>>> valid at 2018030900 (given in the config). If so, why
doesn't
>>>>>>>>>> it
>>>>>>>>>> simply use gfs.t00z.pgrb.1p00.f024, which already have the
00-
>>>>>>>>>> 24h
>>>>>>>>>> forecast? If it subtracts gfs.t00z.pgrb.1p00.f024 from
>>>>>>>>>> gfs.t00z.pgrb.1p00.f048, the resulting file would have 24-
48h
>>>>>>>>>> forecast
>>>>>>>>>> valid at 2018031000. I did
>>>>>>>>>> ncdump -h
>>>>>>>>>>
/meso/noscrub/Ying.Lin/metplus.out/gfs/bucket/gfs.2018030900_A024h
>>>>>>>>>>
>>>>>>>>>> and got
>>>>>>>>>> APCP_24:init_time = "20180308_000000" ;
>>>>>>>>>> APCP_24:init_time_ut = "1520467200" ;
>>>>>>>>>> APCP_24:valid_time = "20180310_000000" ;
>>>>>>>>>> APCP_24:valid_time_ut = "1520640000" ;
>>>>>>>>>> APCP_24:accum_time = "240000" ;
>>>>>>>>>> it seems to mean that the file contains APCP_24h with
initial
>>>>>>>>>> time of
>>>>>>>>>> 00z 03/08 and the valid time is 00Z 03/10 (that'll be
APCP_48h
>>>>>>>>>> instead
>>>>>>>>>> of 24h?)
>>>>>>>>>>
>>>>>>>>>> 2) The precip_continuous.conf has
>>>>>>>>>> FCST_LEVEL = 6
>>>>>>>>>> I thought FCST_LEVEL is the forecast bucket length. In the
>>>>>>>>>> sample GFS
>>>>>>>>>> files, there isn't a 6-hourly bucket -
gfs.t00z.pgrb.1p00.f024
>>>>>> contains
>>>>>>>>>> 00-24h accumulation and gfs.t00z.pgrb.1p00.f048 contains
00-48h
>>>>>>>>>> accumulation. Does FCST_LEVEL mean something other than
the
>>>>>>>>>> forecast
>>>>>>>>>> bucket length?
>>>>>>>>>>
>>>>>>>>>> 3) precip_continuous.conf has
>>>>>>>>>> PCP_COMBINE_METHOD = SUBTRACT
>>>>>>>>>> which should work for the example since the verifying
analysis
>>>>>>>>>> is the
>>>>>>>>>> 24h CCPA file. But what if I need to use 'SUBTRACT' for
the
>>>>>>>>>> forecast
>>>>>>>>>> field but 'SUM' for the analysis field (e.g. summing up 6h
CCPAs
>>>>>>>>>> into
>>>>>> a
>>>>>>>>>> 24h total)? At one point I tried
>>>>>>>>>> PCP_COMBINE_METHOD = SUBTRACT, SUM
>>>>>>>>>> and got
>>>>>>>>>> ERROR: Invalid PCP_COMBINE_METHOD specified
>>>>>>>>>>
>>>>>>>>>> After several failed attempts getting my own config file to
>>>>>>>>>> work, I
>>>>>>>>>> starting from precip_continuous.conf (which worked through
the
>>>>>>>>>> PcpCombine step; I think that's the only step it runs since
the
>>>>>>>>>> config
>>>>>>>>>> has only PcpCombine in the PROCESS_LIST) and tried to set a
>>>>>>>>>> config
>>>>>> file
>>>>>>>>>> as close to precip_continuous.conf as possible
>>>>>>>>>>
>>>
(/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/parm/ylin_config/models/fv3gfs_24h.conf,
>>>>>>>>>> also uploaded to
>>>>>>>>>>
>>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf
>>>>>>>> ).
>>>>>>>>>> The resulting err msg:
>>>>>>>>>>
>>>>>>>>>> 04/18 18:35:33.912 jlog (master_metplus.py:217) CRITICAL:
>>>>>>>>>> master_metplus failed: ''
>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>> File "/meso/save/Ying.Lin/metplus/ush/master_metplus.py",
line
>>>>>> 213,
>>>>>>>>>> in <module>
>>>>>>>>>> main()
>>>>>>>>>> File
"/meso/save/Ying.Lin/metplus/ush/master_metplus.py",
>>>>>>>>>> line
>>>>>> 170,
>>>>>>>>>> in main
>>>>>>>>>> process.run_at_time(-1, run_time)
>>>>>>>>>> File
>>>>>>>>>>
>>>
"/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/ush/reformat_gridded_wrapper.py",
>>>>>>>>>> line 80, in run_at_time
>>>>>>>>>>
>>>>>>>>>> Full output file:
>>>>>>>>>>
>>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs.out.test5
>>>>>>>> ;
>>>>>>>>>> metplus_final.conf:
>>>>>>>>>>
>>>>>>>>>>
>>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/metplus_final.conf
>>>>>>>>>> The quoted err msg above appears to the the same as in my
email
>>>>>>>>>> yesterday, so there must still be something wrong with the
>>>>>>>>>> format of
>>>>>> my
>>>>>>>>>> config, though I think I eliminated the problems you
pointed out
>>>>>>>>>> in
>>>>>> your
>>>>>>>>>> reply. There wouldn't be a ccpa_conus_1.0d_* of the valid
day I
>>>>>>>>>> specified, but I think the processing failed way before
that
>>>>>>>>>> point.
>>>>>>>>>>
>>>>>>>>>> Thank you again -
>>>>>>>>>>
>>>>>>>>>> Ying
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 4/17/19 3:32 PM, George McCabe via RT wrote:
>>>>>>>>>>> Hi Ying,
>>>>>>>>>>>
>>>>>>>>>>> You will need to set FCST_PCP_COMBINE_METHOD = SUBTRACT to
>>>>>>>>>>> config
>>>>>>>> METplus
>>>>>>>>>>> to subtract accumulations files to get the accumulation
you
>>>>>>>>>>> desire.
>>>>>> The
>>>>>>>>>>> following use case has an example of this configuration:
>>>>>>>>>>>
>>>>>>>>>>> parm/use_cases/grid_to_grid/examples/precip_continous.conf
>>>>>>>>>>>
>>>>>>>>>>> I also noticed that your config file has the following
items in
>>>>>>>>>>> the
>>>>>>>>>>> [config] section:
>>>>>>>>>>>
>>>>>>>>>>> FCST_PCP_COMBINE_INPUT_DIR =
/gpfs/dell1/nco/ops/com/gfs/para
>>>>>>>>>>> FCST_PCP_COMBINE_OUTPUT_DIR =
{OUTPUT_BASE}/qpf_grib/bucket
>>>>>>>>>>> FCST_GRID_STAT_INPUT_DIR = {OUTPUT_BASE}/qpf_grib/bucket
>>>>>>>>>>>
>>>>>>>>>>> You should move these to the [dir] section.
>>>>>>>>>>>
>>>>>>>>>>> Please let me know if you continue to have issues. The
error
>>>>>>>>>>> message
>>>>>>>> you
>>>>>>>>>>> provided has more information at the bottom that was not
>>>>>>>>>>> included.
>>>>>> The
>>>>>>>>>> next
>>>>>>>>>>> release of METplus contains more extensive error checking
so
>>>>>>>>>>> the
>>>>>> error
>>>>>>>>>>> message from that version should be much more helpful.
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> George
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Apr 17, 2019 at 12:43 PM Ying Lin via RT
>>>>>>>>>>> <met_help at ucar.edu>
>>>>>>>>>> wrote:
>>>>>>>>>>>> Wed Apr 17 12:43:02 2019: Request 89789 was acted upon.
>>>>>>>>>>>> Transaction: Ticket created by ying.lin at noaa.gov
>>>>>>>>>>>> Queue: met_help
>>>>>>>>>>>> Subject: Using FCST_PCP_COMBINE in METplus
>>>>>>>>>>>> Owner: Nobody
>>>>>>>>>>>> Requestors: ying.lin at noaa.gov
>>>>>>>>>>>> Status: new
>>>>>>>>>>>> Ticket <URL:
>>>>>>>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789
>>>>>>>>>>>> Hello. I haven't been able to get the FCST_PCP_COMBINE
>>>>>>>>>>>> feature to
>>>>>>>> work
>>>>>>>>>>>> in METplus, running out of ideas and hope you can help.
>>>>>>>>>>>>
>>>>>>>>>>>> I've put some examples in
>>>>>>>>>>>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/
>>>>>>>>>>>> (current
>>>>>>>>>> test
>>>>>>>>>>>> directory:
>>>>>>>>>>>> gyre:/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/test).
>>>>>>>>>>>> test_fv3gfs_24h.ksh is the simple test script. The goal
is to
>>>>>>>> compute a
>>>>>>>>>>>> 24h verification ending at 12Z 20190415.
>>>>>>>>>>>>
>>>>>>>>>>>> 'test1' did not need FCST_PCP_COMBINE - the config file
>>>>>>>>>>>> fv3gfs_24h.conf.test1 simply had LEAD_SEQ set to '24',
>>>>>>>>>>>> FCST_PCP_COMBINE_RUN was not set. The FV3GFS had two
precip
>>>>>> buckets,
>>>>>>>> 6h
>>>>>>>>>>>> and continuously-accumulating, and METplus easily found
the
>>>>>>>>>>>> 24h
>>>>>>>> forecast
>>>>>>>>>>>> accumulation and went on with the grid_stat.
>>>>>>>>>>>>
>>>>>>>>>>>> 'test2' had 'LEAD_SEQ = 36', and I added the following in
the
>>>>>>>>>>>> config
>>>>>>>>>>>> (
>>>>>>>>>>>>
>>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test2
>>>>>>>>>>>> ):
>>>>>>>>>>>> FCST_PCP_COMBINE_RUN = True
>>>>>>>>>>>> FCST_PCP_COMBINE_INPUT_DATATYPE = GRIB
>>>>>>>>>>>> FCST_PCP_COMBINE_INPUT_DIR =
>>>>>> /gpfs/dell1/nco/ops/com/gfs/para
>>>>>>>>>>>> FCST_PCP_COMBINE_OUTPUT_DIR =
{OUTPUT_BASE}/qpf_grib/bucket
>>>>>>>>>>>> FCST_GRID_STAT_INPUT_DIR = {OUTPUT_BASE}/qpf_grib/bucket
>>>>>>>>>>>> FCST_MAX_FORECAST = 36
>>>>>>>>>>>> FCST_INIT_INTERVAL = 6
>>>>>>>>>>>> FCST_LEVEL = 36
>>>>>>>>>>>> Here the goal was to see if I could get METplus to
subtract
>>>>>>>>>>>> the
>>>>>> 00-12h
>>>>>>>>>>>> accumulation from gfs.t00z.pgrb2.0p25.f012 from the 00-
36h
>>>>>>>> accumulation
>>>>>>>>>>>> in gfs.t00z.pgrb2.0p25.f036. Thinking that FCST_LEVEL is
the
>>>>>> forecast
>>>>>>>>>>>> bucket length, I figured setting it to '36' and
artificially
>>>>>>>> specifying
>>>>>>>>>>>> FCST_MAX_FORECAST = 36 (instead of the full 384h) should
work
>>>>>>>>>>>> for
>>>>>> this
>>>>>>>>>>>> limited test where LEAD_SEQ is set to 36 (is there a way
to
>>>>>>>>>>>> specify
>>>>>>>> that
>>>>>>>>>>>> the bucket never gets emptied?). It failed with the
follow
>>>>>>>>>>>> err msg
>>>>>>>>>>>> (full message in
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test2
>>>>>>>>>>>> ):
>>>>>>>>>>>>
>>>>>>>>>>>> 04/16 20:01:42.754 jlog (master_metplus.py:217) CRITICAL:
>>>>>>>>>>>> master_metplus failed: ''
>>>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>>>> File
>>>>>>>>>>>> "/meso/save/Ying.Lin/metplus/ush/master_metplus.py", line
>>>>>>>> 213,
>>>>>>>>>>>> in <module>
>>>>>>>>>>>> main()
>>>>>>>>>>>> File
>>>>>>>>>>>> "/meso/save/Ying.Lin/metplus/ush/master_metplus.py", line
>>>>>>>> 170,
>>>>>>>>>>>> in main
>>>>>>>>>>>> process.run_at_time(-1, run_time)
>>>>>>>>>>>> File
>>>>>>>>>>>>
>>>
"/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/ush/reformat_gridded_wrapper.py",
>>>>>>>>>>>> line 80, in run_at_time
>>>>>>>>>>>> self.run_at_time_once(task_info, var_info, rl)
>>>>>>>>>>>> File
>>>>>>>>>>>>
>>>>>>
"/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/ush/pcp_combine_wrapper.py",
>>>>>>>>>>>> line 419, in run_at_time_once
>>>>>>>>>>>> ....
>>>>>>>>>>>>
>>>>>>>>>>>> 'test3' is an attempt from a different angle, thinking
that if
>>>>>>>>>>>> the
>>>>>>>>>>>> 'continuously running' bucket didn't work, maybe the 6-
hourly
>>>>>>>>>>>> bucket
>>>>>>>>>>>> would. In
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test3
>>>>>>>>>> ,
>>>>>>>>>>>> I changed LEAD_SEQ to 24, and set FCST_LEVEL = 6. It
produced
>>>>>>>>>>>> err
>>>>>>>> msgs
>>>>>>>>>>>> (
>>>>>>>>>>>>
>>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test3
>>>>>>>>>> )
>>>>>>>>>>>> that appeared to be the same as that from 'test2'.
>>>>>>>>>>>>
>>>>>>>>>>>> Earlier on I tried using the FCST_PCP_COMBINE feature
with the
>>>>>>>>>>>> NAM
>>>>>>>>>>>> 'CONUSNEST' output (with simple regular 6-hourly bucket)
and
>>>>>>>>>>>> got the
>>>>>>>>>>>> same err msg, so I don't think FV3GFS's dual-bucket
system is
>>>>>>>>>>>> the
>>>>>>>> cause.
>>>>>>>>>>>> Also posted on
>>>>>>>>>>>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/ are
>>>>>>>>>>>> the
>>>>>>>> two
>>>>>>>>>>>> metplus_final.conf files from both 'test2' and 'test3'.
>>>>>>>>>>>>
>>>>>>>>>>>> I found in
>>>>>>>>>>>>
>>>>>> /global/noscrub/Julie.Prestopnik/METplus/METplus-
>>>>>> 2.0.4/parm/use_cases/qpf/examples/
>>>>>>>>>>>> that hrefmean-vs-mrms-qpe.conf and nationalblend-vs-mrms-
>>>>>>>>>>>> qpe.conf
>>>>>> had
>>>>>>>>>>>> "FCST_PCP_COMBINE_RUN = True", that's where I borrowed
the
>>>>>>>>>>>> FCST_PCP_COMBINE_* set up in my fv3gfs_24h.conf.test2 and
>>>>>>>>>>>> fv3gfs_24h.conf.test3. Haven't been able to run the
hrefmean
>>>>>>>>>>>> and
>>>>>>>>>>>> nationalblend tests myself however - got err msg of "
>>>>>>>>>>>> CRITICAL:
>>>>>>>>>>>> master_metplus failed: No option 'GEMPAKTOCF_CLASSPATH'
in
>>>>>>>>>>>> section:
>>>>>>>>>>>> 'exe'" from both attempts.
>>>>>>>>>>>>
>>>>>>>>>>>> Thank you for looking into this -
>>>>>>>>>>>>
>>>>>>>>>>>> Ying
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> Ying Lin
>>>>>>>>>>>> NCEP/EMC/Verification, Post-processing and Product
Generation
>>>>>>>>>>>> Branch
>>>>>>>>>>>> NCWCP Cubicle No. 2015
>>>>>>>>>>>> Ying.Lin at noaa.gov
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> Ying Lin
>>>>>>>>>> NCEP/EMC/Verification, Post-processing and Product
Generation
>>>>>>>>>> Branch
>>>>>>>>>> NCWCP Cubicle No. 2015
>>>>>>>>>> Ying.Lin at noaa.gov
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>> --
>>>>>>>> Ying Lin
>>>>>>>> NCEP/EMC/Verification, Post-processing and Product Generation
>>>>>>>> Branch
>>>>>>>> NCWCP Cubicle No. 2015
>>>>>>>> Ying.Lin at noaa.gov
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>> --
>>>>>> Ying Lin
>>>>>> NCEP/EMC/Verification, Post-processing and Product Generation
Branch
>>>>>> NCWCP Cubicle No. 2015
>>>>>> Ying.Lin at noaa.gov
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>
>>> --
>>> Ying Lin
>>> NCEP/EMC/Verification, Post-processing and Product Generation
Branch
>>> NCWCP Cubicle No. 2015
>>> Ying.Lin at noaa.gov
>>>
>>>
>>>
>>>
--
Ying Lin
NCEP/EMC/Verification, Post-processing and Product Generation Branch
NCWCP Cubicle No. 2015
Ying.Lin at noaa.gov
------------------------------------------------
Subject: Using FCST_PCP_COMBINE in METplus
From: George McCabe
Time: Thu May 30 15:10:33 2019
Hi Ying,
I see that the use case hrefmean-vs-qpe.conf reads Gempak data as
input. I
thought I had renamed all of the use cases that use Gempak to contain
'gempak' somewhere in the title, but it appears that I missed this
one. To
run it, you will need to have GempakToCF available on your computer.
If you
have it, you can set GEMPAKTOCF_BIN to the directory that contains the
files for the app. If you don't have it, then I would suggest picking
a
different use case to run. It is possible I apologize for the
confusion.
On Thu, May 30, 2019 at 2:38 PM Ying Lin via RT <met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789 >
>
> Hi George, i was writing to you when your email below came in ...
here
> it is:
>
> Thanks very much! In my system.conf.wcoss, I changed
> WGRIB2 = $WGRIB2
> to
> WGRIB2 = {ENV[WGRIB2]}
> and that seems to get rid of the $WGRIB2 complaint.
>
> Now I'm getting a new err msg:
> metplus.PcpCombine (config_wrapper.py:93) ERROR: Requested conf
> [exe] GEMPAKTOCF_CLASSPATH was not set in config file
>
> I see that in my system.conf.wcoss there was this commented out
line:
> #GEMPAKTOCF_BIN = /d1/mccabe/bin/GempakToCF
>
> What should I set GEMPAKTOCF_BIN [and/or _CLASSPATH] to? Or is
there an
> option to turn the GEMPAKTOCF off, If I won't be using this
function?
>
> Thanks again -
>
> Ying
>
> On 5/30/19 4:32 PM, George McCabe via RT wrote:
> > The reason you didn't have this issue previously is your use case
likely
> > didn't need to use any of those exes. I added a function to check
and add
> > all of the non-MET executables to all of the MET wrappers in case
they
> are
> > needed, but I realized later than they are not necessary for most
of the
> > cases. However, they may become required for other cases so we may
leave
> > the requirement to set them correctly to run METplus.
> >
> > On Thu, May 30, 2019 at 1:36 PM George McCabe <mccabe at ucar.edu>
wrote:
> >
> >> Hi Ying,
> >>
> >> To reference an environment variable in the METplus config files,
you
> need
> >> to use this syntax:
> >>
> >> WGRIB2 = {ENV[WGRIB2]}
> >>
> >> Since you have set WGRIB2 environment variable, this should set
the
> >> METplus config variable correctly. You may also be able to set
WGRIB2 =
> >> wgrib2 in the METplus config file instead. If the exe is in the
path of
> the
> >> user running METplus, it will find the app properly. Let me know
if that
> >> doesn't work.
> >>
> >> On Thu, May 30, 2019 at 11:41 AM Ying Lin via RT
<met_help at ucar.edu>
> >> wrote:
> >>
> >>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789 >
> >>>
> >>> Hi George,
> >>>
> >>> Thanks for following up! The sample case in
> >>> parm/use_cases/grid_to_grid/examples/precip_continuous.conf
seems to
> >>> "subtract" correctly now (it was off by 24h in v2.0.4). The
> >>> precip_continuous.conf only deals with the PcpCombine step
though.
> >>> When I added the regrid_data_plane step to test a fv3gfs verif
on G212,
> >>> I got this err msg:
> >>> metplus.GridStat (config_wrapper.py:144) ERROR: Executable
WGRIB2
> >>> does not exist at $WGRIB2
> >>> I thought there must have been a problem in my model config for
the
> v2.1
> >>> (found several actually) but when I run the use case at
> >>> parm/use_cases/qpf/examples/hrefmean-vs-qpe.conf the same thing
> happened.
> >>>
> >>> I'm attaching three files to this email:
> >>>
> >>> 1. test_hrefmean.ksh: script to run this use case
> >>> 2. hrefmean.out: output for running the above script. You can
see
> that
> >>> after loading model grid_util (from ". ~/dot.for.metplus)
there
> was
> >>> "WGRIB2=/nwprod2/grib_util.v1.0.0/exec/wgrib2
> >>> export WGRIB2"; and at the end there is the err msg
> >>> "metplus.GridStat (config_wrapper.py:144) ERROR: Executable
WGRIB2
> >>> does not exist at $WGRIB2"
> >>> 3. metplus_final.conf from the above run
> >>> 4. my "dot.for.metplus" file invoked by test_hrefmean.ksh. I
> verified
> >>> manually - after "module load grib_util", $WGRIB2 does
point to
> >>> /nwprod2/grib_util.v1.0.0/exec/wgrib2
> >>> 5. my system.conf.wcoss invoked by test_hrefmean.ksh.
> >>>
> >>> I looked back at run output from METplus-2.0.4, there was also
> >>> WGRIB2=/nwprod2/grib_util.v1.0.0/exec/wgrib2 and "export", and
there
> >>> wasn't a complain about "Executable WGRIB2 does not exist at
$WGRIB2".
> >>>
> >>> Thanks again -
> >>>
> >>> Ying
> >>>
> >>>
> >>> On 5/29/19 1:53 PM, George McCabe via RT wrote:
> >>>> Hi Ying,
> >>>>
> >>>> I am just following up on this ticket. Have you tried running
your
> case
> >>> on METplus v2.1? Let me know if you run into any other issues.
> >>>> Thanks,
> >>>> George
> >>>>
> >>>> On Thu Apr 18 19:15:58 2019, mccabe wrote:
> >>>>> Hi Ying,
> >>>>>
> >>>>> It looks like the bug with subtract mode is in v2.0.4. It will
be
> >>>>> fixed in
> >>>>> the next release. You can look at the develop branch on GitHub
to see
> >>>>> what
> >>>>> has changed in the subtract function. Have a good weekend!
> >>>>>
> >>>>> On Thu, Apr 18, 2019, 5:09 PM Ying Lin via RT
<met_help at ucar.edu>
> >>>>> wrote:
> >>>>>
> >>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789
>
> >>>>>>
> >>>>>> Hi George,
> >>>>>>
> >>>>>> Thank you - you're right, replacing that first
> >>>>>> FCST_GRID_STAT_INPUT_TEMPLATE with
FCST_PCP_COMBINE_INPUT_TEMPLATE
> >>>>>> got
> >>>>>> me past the err msg! There're still some old stuff with the
way the
> >>>>>> summing was done - similar to the odd result I mentioned in
the last
> >>>>>> email with the 2.0.4 precip_continuous.conf. I'll study the
new
> >>>>>> precip_continuous.conf in the github and follow up with you
next
> >>>>>> week.
> >>>>>>
> >>>>>> Thanks for your help and patience today. Have a great
weekend -
> >>>>>>
> >>>>>> Ying
> >>>>>>
> >>>>>> On 4/18/19 4:25 PM, George McCabe via RT wrote:
> >>>>>>> Hi Ying,
> >>>>>>>
> >>>>>>> I am sorry, I was incorrect that the fixed precip_continuous
config
> >>>>>>> file
> >>>>>>> was updated in 2.0.4. It looks like the fix was made after
that
> >>>>>>> release.
> >>>>>> It
> >>>>>>> will be in the newest release. You can see the updated file
on
> >>>>>>> GitHub in
> >>>>>>> the develop branch here:
> >>>>>>>
> >>>>>>>
> >>>
>
https://github.com/NCAR/METplus/blob/develop/parm/use_cases/grid_to_grid/examples/precip_continuous.conf
> >>>>>>> For your use case, it looks like you don't have
> >>>>>>> FCST_PCP_COMBINE_INPUT_TEMPLATE set and you have 2 for
grid_stat:
> >>>>>>>
> >>>>>>> FCST_GRID_STAT_INPUT_TEMPLATE =
> >>>>>>>
> >>>
>
gfs.{init?fmt=%Y%m%d}/{init?fmt=%H}/gfs.t{init?fmt=%H}z.pgrb2.0p25.f{lead?fmt=%HHH}
> >>>>>>> FCST_PCP_COMBINE_OUTPUT_TEMPLATE =
> >>>>>>> gfs.{valid?fmt=%Y%m%d%H}_A{level?fmt=%HHH}h
> >>>>>>> FCST_GRID_STAT_INPUT_TEMPLATE =
> >>>>>> gfs.{valid?fmt=%Y%m%d%H}_A{level?fmt=%HHH}h
> >>>>>>> Fixing this may solve your issue. Please let me know if it
does
> >>>>>>> not. I
> >>>>>> have
> >>>>>>> to catch a flight this evening so I am going to stop working
very
> >>>>>>> soon,
> >>>>>> but
> >>>>>>> if you reply very soon I may still be able to help you
before the
> >>>>>> weekend.
> >>>>>>> Thanks,
> >>>>>>> George
> >>>>>>>
> >>>>>>> On Thu, Apr 18, 2019 at 8:13 PM Ying Lin via RT
<met_help at ucar.edu
> >
> >>>>>> wrote:
> >>>>>>>> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789 >
> >>>>>>>>
> >>>>>>>> Hi George,
> >>>>>>>>
> >>>>>>>> Thank you for the quick reply. I was looking at
> >>>>>>>>
> >>>>>>
/gpfs/gd1/emc/global/noscrub/Julie.Prestopnik/METplus/METplus-
> >>>>>>
2.0.4/parm/use_cases/grid_to_grid/examples/precip_continuous.conf,
> >>>>>>>> which has VALID_BEG = 2018030900, VALID_END = 2018030900.
Date
> >>>>>>>> stamp is
> >>>>>>>> Jan 30. Is there a "newer version of v2.0.4" available on
wcoss?
> >>>>>>>> From
> >>>>>>>> the above config it's not clear to me how METplus knows to
> >>>>>>>> subtract the
> >>>>>>>> gfs.*.f024 from the gfs.*.f048.
> >>>>>>>>
> >>>>>>>> If you have time today, could you also take a look why my
simple
> >>>>>>>> test
> >>>>>>>> (conf set up to be as close to precip_continuous.conf as
possible
> >>>>>>>> -
> >>>>>>>>
> >>>
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf
> >>> )
> >>>>>>>> led to err msg in
> >>>>>>>>
> >>>
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs.out.test5
> >>>>>> ?
> >>>>>>>> It seems that I corrected the problems you pointed out in
your
> >>>>>>>> reply
> >>>>>>>> this morning, but the err msg still seems to be the same as
> >>>>>>>> yesterday's.
> >>>>>>>>
> >>>>>>>> If you don't have time today, next Monday is fine too.
Have a
> >>>>>>>> good
> >>>>>>>> holiday weekend.
> >>>>>>>>
> >>>>>>>> Ying
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On 4/18/19 3:51 PM, George McCabe via RT wrote:
> >>>>>>>>> Hi Ying,
> >>>>>>>>>
> >>>>>>>>> 1) The config file I see has a valid time of 2018031000
and 48 hr
> >>>>>>>> forecast
> >>>>>>>>> lead. You may be using an old version of METplus that had
this
> >>>>>>>>> set up
> >>>>>>>>> incorrectly. METplus version 2.0.4 should be correct.
> >>>>>>>>>
> >>>>>>>>> 2) pcp_combine -subtract does not actually use the
FCST_LEVEL
> >>>>>>>>> variable,
> >>>>>>>> so
> >>>>>>>>> that should be removed from the use case. The code takes
the
> >>>>>>>>> valid time
> >>>>>>>> and
> >>>>>>>>> subtracts the accumulation.
> >>>>>>>>>
> >>>>>>>>> 3) In the upcoming version of METplus (in a few weeks!)
has a
> >>>>>>>>> good
> >>>>>> number
> >>>>>>>>> of config changes from the previous versions. For example,
> >>>>>>>>> FCST_LEVEL
> >>>>>>>> will
> >>>>>>>>> now be FCST_PCP_COMBINE_INPUT_LEVEL. These changes are
designed
> >>>>>>>>> to make
> >>>>>>>> the
> >>>>>>>>> variables more clear to what they actually do. In the new
> >>>>>>>>> version,
> >>>>>> there
> >>>>>>>> is
> >>>>>>>>> now FCST_PCP_COMBINE_METHOD and OBS_PCP_COMBINE_METHOD, so
you
> >>>>>>>>> will be
> >>>>>>>> able
> >>>>>>>>> to specify a different method for your forecast and
analysis
> >>>>>>>>> data.
> >>>>>>>>>
> >>>>>>>>> FCST_PCP_COMBINE_RUN = True
> >>>>>>>>> FCST_PCP_COMBINE_METHOD = SUBTRACT
> >>>>>>>>>
> >>>>>>>>> OBS_PCP_COMBINE_RUN = True
> >>>>>>>>> OBS_PCP_COMBINE_METHOD = SUM
> >>>>>>>>>
> >>>>>>>>> This functionality has not been thoroughly tested as we
haven't
> >>>>>>>>> had any
> >>>>>>>>> example use cases that need to do this so far, so please
let me
> >>>>>>>>> know if
> >>>>>>>> it
> >>>>>>>>> does not work the way you expect and I will fix it.
> >>>>>>>>>
> >>>>>>>>> Just to let you know, I will be out of the office tomorrow
> >>>>>>>>> (Friday)
> >>>>>> and I
> >>>>>>>>> wont have access to email. Feel free to send any follow-up
emails
> >>>>>>>>> to me
> >>>>>>>> and
> >>>>>>>>> I will get back to you on Monday.
> >>>>>>>>>
> >>>>>>>>> Thanks,
> >>>>>>>>> George
> >>>>>>>>>
> >>>>>>>>> On Thu, Apr 18, 2019 at 7:38 PM Ying Lin via RT
> >>>>>>>>> <met_help at ucar.edu>
> >>>>>>>> wrote:
> >>>>>>>>>> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789 >
> >>>>>>>>>>
> >>>>>>>>>> Thank you very much George, this is very helpful.
> >>>>>>>>>>
> >>>>>>>>>> About the example in
> >>>>>>>>>>
> parm/use_cases/grid_to_grid/examples/precip_continuous.conf
> >>>>>>>>>> I tried running it and it seems to have run through the
> >>>>>>>>>> pcp_combine
> >>>>>> step
> >>>>>>>>>> fine: it did this:
> >>>>>>>>>>
/global/noscrub/Julie.Prestopnik/met/8.0/bin/pcp_combine
> >>>>>> -subtract
> >>>>>> /global/noscrub/Julie.Prestopnik/METplus/METplus-
> >>>>>>
> >>>
>
2.0.4_sample_data/grid_to_grid/prfv3rt1/20180308/gfs.t00z.pgrb.1p00.f048
> >>>>>>>>>> 48
> >>>>>>>>>>
> >>>>>> /global/noscrub/Julie.Prestopnik/METplus/METplus-
> >>>>>>
> >>>
>
2.0.4_sample_data/grid_to_grid/prfv3rt1/20180308/gfs.t00z.pgrb.1p00.f024
> >>>>>>>>>> 24
> >>>>>>>>>>
> /meso/noscrub/Ying.Lin/metplus.out/gfs/bucket/gfs.2018030900_A024h
> >>>>>>>>>>
> >>>>>>>>>> Several questions about the example:
> >>>>>>>>>> 1) The precip_continuous.conf has
> >>>>>>>>>> LEAD_SEQ = 24
> >>>>>>>>>> I thought it meant forecast lead of 24h, e.g. look for
00-24h
> >>>>>>>>>> forecast
> >>>>>>>>>> valid at 2018030900 (given in the config). If so, why
doesn't
> >>>>>>>>>> it
> >>>>>>>>>> simply use gfs.t00z.pgrb.1p00.f024, which already have
the 00-
> >>>>>>>>>> 24h
> >>>>>>>>>> forecast? If it subtracts gfs.t00z.pgrb.1p00.f024 from
> >>>>>>>>>> gfs.t00z.pgrb.1p00.f048, the resulting file would have
24-48h
> >>>>>>>>>> forecast
> >>>>>>>>>> valid at 2018031000. I did
> >>>>>>>>>> ncdump -h
> >>>>>>>>>>
> /meso/noscrub/Ying.Lin/metplus.out/gfs/bucket/gfs.2018030900_A024h
> >>>>>>>>>>
> >>>>>>>>>> and got
> >>>>>>>>>> APCP_24:init_time = "20180308_000000" ;
> >>>>>>>>>> APCP_24:init_time_ut = "1520467200" ;
> >>>>>>>>>> APCP_24:valid_time = "20180310_000000" ;
> >>>>>>>>>> APCP_24:valid_time_ut = "1520640000" ;
> >>>>>>>>>> APCP_24:accum_time = "240000" ;
> >>>>>>>>>> it seems to mean that the file contains APCP_24h with
initial
> >>>>>>>>>> time of
> >>>>>>>>>> 00z 03/08 and the valid time is 00Z 03/10 (that'll be
APCP_48h
> >>>>>>>>>> instead
> >>>>>>>>>> of 24h?)
> >>>>>>>>>>
> >>>>>>>>>> 2) The precip_continuous.conf has
> >>>>>>>>>> FCST_LEVEL = 6
> >>>>>>>>>> I thought FCST_LEVEL is the forecast bucket length. In
the
> >>>>>>>>>> sample GFS
> >>>>>>>>>> files, there isn't a 6-hourly bucket -
gfs.t00z.pgrb.1p00.f024
> >>>>>> contains
> >>>>>>>>>> 00-24h accumulation and gfs.t00z.pgrb.1p00.f048 contains
00-48h
> >>>>>>>>>> accumulation. Does FCST_LEVEL mean something other than
the
> >>>>>>>>>> forecast
> >>>>>>>>>> bucket length?
> >>>>>>>>>>
> >>>>>>>>>> 3) precip_continuous.conf has
> >>>>>>>>>> PCP_COMBINE_METHOD = SUBTRACT
> >>>>>>>>>> which should work for the example since the verifying
analysis
> >>>>>>>>>> is the
> >>>>>>>>>> 24h CCPA file. But what if I need to use 'SUBTRACT' for
the
> >>>>>>>>>> forecast
> >>>>>>>>>> field but 'SUM' for the analysis field (e.g. summing up
6h CCPAs
> >>>>>>>>>> into
> >>>>>> a
> >>>>>>>>>> 24h total)? At one point I tried
> >>>>>>>>>> PCP_COMBINE_METHOD = SUBTRACT, SUM
> >>>>>>>>>> and got
> >>>>>>>>>> ERROR: Invalid PCP_COMBINE_METHOD specified
> >>>>>>>>>>
> >>>>>>>>>> After several failed attempts getting my own config file
to
> >>>>>>>>>> work, I
> >>>>>>>>>> starting from precip_continuous.conf (which worked
through the
> >>>>>>>>>> PcpCombine step; I think that's the only step it runs
since the
> >>>>>>>>>> config
> >>>>>>>>>> has only PcpCombine in the PROCESS_LIST) and tried to set
a
> >>>>>>>>>> config
> >>>>>> file
> >>>>>>>>>> as close to precip_continuous.conf as possible
> >>>>>>>>>>
> >>>
>
(/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/parm/ylin_config/models/fv3gfs_24h.conf,
> >>>>>>>>>> also uploaded to
> >>>>>>>>>>
> >>>
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf
> >>>>>>>> ).
> >>>>>>>>>> The resulting err msg:
> >>>>>>>>>>
> >>>>>>>>>> 04/18 18:35:33.912 jlog (master_metplus.py:217) CRITICAL:
> >>>>>>>>>> master_metplus failed: ''
> >>>>>>>>>> Traceback (most recent call last):
> >>>>>>>>>> File "/meso/save/Ying.Lin/metplus/ush/master_metplus.py",
line
> >>>>>> 213,
> >>>>>>>>>> in <module>
> >>>>>>>>>> main()
> >>>>>>>>>> File
"/meso/save/Ying.Lin/metplus/ush/master_metplus.py",
> >>>>>>>>>> line
> >>>>>> 170,
> >>>>>>>>>> in main
> >>>>>>>>>> process.run_at_time(-1, run_time)
> >>>>>>>>>> File
> >>>>>>>>>>
> >>>
>
"/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/ush/reformat_gridded_wrapper.py",
> >>>>>>>>>> line 80, in run_at_time
> >>>>>>>>>>
> >>>>>>>>>> Full output file:
> >>>>>>>>>>
> >>>
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs.out.test5
> >>>>>>>> ;
> >>>>>>>>>> metplus_final.conf:
> >>>>>>>>>>
> >>>>>>>>>>
> >>>
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/metplus_final.conf
> >>>>>>>>>> The quoted err msg above appears to the the same as in my
email
> >>>>>>>>>> yesterday, so there must still be something wrong with
the
> >>>>>>>>>> format of
> >>>>>> my
> >>>>>>>>>> config, though I think I eliminated the problems you
pointed out
> >>>>>>>>>> in
> >>>>>> your
> >>>>>>>>>> reply. There wouldn't be a ccpa_conus_1.0d_* of the
valid day I
> >>>>>>>>>> specified, but I think the processing failed way before
that
> >>>>>>>>>> point.
> >>>>>>>>>>
> >>>>>>>>>> Thank you again -
> >>>>>>>>>>
> >>>>>>>>>> Ying
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On 4/17/19 3:32 PM, George McCabe via RT wrote:
> >>>>>>>>>>> Hi Ying,
> >>>>>>>>>>>
> >>>>>>>>>>> You will need to set FCST_PCP_COMBINE_METHOD = SUBTRACT
to
> >>>>>>>>>>> config
> >>>>>>>> METplus
> >>>>>>>>>>> to subtract accumulations files to get the accumulation
you
> >>>>>>>>>>> desire.
> >>>>>> The
> >>>>>>>>>>> following use case has an example of this configuration:
> >>>>>>>>>>>
> >>>>>>>>>>>
parm/use_cases/grid_to_grid/examples/precip_continous.conf
> >>>>>>>>>>>
> >>>>>>>>>>> I also noticed that your config file has the following
items in
> >>>>>>>>>>> the
> >>>>>>>>>>> [config] section:
> >>>>>>>>>>>
> >>>>>>>>>>> FCST_PCP_COMBINE_INPUT_DIR =
/gpfs/dell1/nco/ops/com/gfs/para
> >>>>>>>>>>> FCST_PCP_COMBINE_OUTPUT_DIR =
{OUTPUT_BASE}/qpf_grib/bucket
> >>>>>>>>>>> FCST_GRID_STAT_INPUT_DIR = {OUTPUT_BASE}/qpf_grib/bucket
> >>>>>>>>>>>
> >>>>>>>>>>> You should move these to the [dir] section.
> >>>>>>>>>>>
> >>>>>>>>>>> Please let me know if you continue to have issues. The
error
> >>>>>>>>>>> message
> >>>>>>>> you
> >>>>>>>>>>> provided has more information at the bottom that was not
> >>>>>>>>>>> included.
> >>>>>> The
> >>>>>>>>>> next
> >>>>>>>>>>> release of METplus contains more extensive error
checking so
> >>>>>>>>>>> the
> >>>>>> error
> >>>>>>>>>>> message from that version should be much more helpful.
> >>>>>>>>>>>
> >>>>>>>>>>> Thanks,
> >>>>>>>>>>> George
> >>>>>>>>>>>
> >>>>>>>>>>> On Wed, Apr 17, 2019 at 12:43 PM Ying Lin via RT
> >>>>>>>>>>> <met_help at ucar.edu>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>> Wed Apr 17 12:43:02 2019: Request 89789 was acted upon.
> >>>>>>>>>>>> Transaction: Ticket created by ying.lin at noaa.gov
> >>>>>>>>>>>> Queue: met_help
> >>>>>>>>>>>> Subject: Using FCST_PCP_COMBINE in METplus
> >>>>>>>>>>>> Owner: Nobody
> >>>>>>>>>>>> Requestors: ying.lin at noaa.gov
> >>>>>>>>>>>> Status: new
> >>>>>>>>>>>> Ticket <URL:
> >>>>>>>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789
> >>>>>>>>>>>> Hello. I haven't been able to get the FCST_PCP_COMBINE
> >>>>>>>>>>>> feature to
> >>>>>>>> work
> >>>>>>>>>>>> in METplus, running out of ideas and hope you can help.
> >>>>>>>>>>>>
> >>>>>>>>>>>> I've put some examples in
> >>>>>>>>>>>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/
> >>>>>>>>>>>> (current
> >>>>>>>>>> test
> >>>>>>>>>>>> directory:
> >>>>>>>>>>>> gyre:/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/test).
> >>>>>>>>>>>> test_fv3gfs_24h.ksh is the simple test script. The
goal is to
> >>>>>>>> compute a
> >>>>>>>>>>>> 24h verification ending at 12Z 20190415.
> >>>>>>>>>>>>
> >>>>>>>>>>>> 'test1' did not need FCST_PCP_COMBINE - the config file
> >>>>>>>>>>>> fv3gfs_24h.conf.test1 simply had LEAD_SEQ set to '24',
> >>>>>>>>>>>> FCST_PCP_COMBINE_RUN was not set. The FV3GFS had two
precip
> >>>>>> buckets,
> >>>>>>>> 6h
> >>>>>>>>>>>> and continuously-accumulating, and METplus easily found
the
> >>>>>>>>>>>> 24h
> >>>>>>>> forecast
> >>>>>>>>>>>> accumulation and went on with the grid_stat.
> >>>>>>>>>>>>
> >>>>>>>>>>>> 'test2' had 'LEAD_SEQ = 36', and I added the following
in the
> >>>>>>>>>>>> config
> >>>>>>>>>>>> (
> >>>>>>>>>>>>
> >>>
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test2
> >>>>>>>>>>>> ):
> >>>>>>>>>>>> FCST_PCP_COMBINE_RUN = True
> >>>>>>>>>>>> FCST_PCP_COMBINE_INPUT_DATATYPE = GRIB
> >>>>>>>>>>>> FCST_PCP_COMBINE_INPUT_DIR =
> >>>>>> /gpfs/dell1/nco/ops/com/gfs/para
> >>>>>>>>>>>> FCST_PCP_COMBINE_OUTPUT_DIR =
{OUTPUT_BASE}/qpf_grib/bucket
> >>>>>>>>>>>> FCST_GRID_STAT_INPUT_DIR =
{OUTPUT_BASE}/qpf_grib/bucket
> >>>>>>>>>>>> FCST_MAX_FORECAST = 36
> >>>>>>>>>>>> FCST_INIT_INTERVAL = 6
> >>>>>>>>>>>> FCST_LEVEL = 36
> >>>>>>>>>>>> Here the goal was to see if I could get METplus to
subtract
> >>>>>>>>>>>> the
> >>>>>> 00-12h
> >>>>>>>>>>>> accumulation from gfs.t00z.pgrb2.0p25.f012 from the 00-
36h
> >>>>>>>> accumulation
> >>>>>>>>>>>> in gfs.t00z.pgrb2.0p25.f036. Thinking that FCST_LEVEL
is the
> >>>>>> forecast
> >>>>>>>>>>>> bucket length, I figured setting it to '36' and
artificially
> >>>>>>>> specifying
> >>>>>>>>>>>> FCST_MAX_FORECAST = 36 (instead of the full 384h)
should work
> >>>>>>>>>>>> for
> >>>>>> this
> >>>>>>>>>>>> limited test where LEAD_SEQ is set to 36 (is there a
way to
> >>>>>>>>>>>> specify
> >>>>>>>> that
> >>>>>>>>>>>> the bucket never gets emptied?). It failed with the
follow
> >>>>>>>>>>>> err msg
> >>>>>>>>>>>> (full message in
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test2
> >>>>>>>>>>>> ):
> >>>>>>>>>>>>
> >>>>>>>>>>>> 04/16 20:01:42.754 jlog (master_metplus.py:217)
CRITICAL:
> >>>>>>>>>>>> master_metplus failed: ''
> >>>>>>>>>>>> Traceback (most recent call last):
> >>>>>>>>>>>> File
> >>>>>>>>>>>> "/meso/save/Ying.Lin/metplus/ush/master_metplus.py",
line
> >>>>>>>> 213,
> >>>>>>>>>>>> in <module>
> >>>>>>>>>>>> main()
> >>>>>>>>>>>> File
> >>>>>>>>>>>> "/meso/save/Ying.Lin/metplus/ush/master_metplus.py",
line
> >>>>>>>> 170,
> >>>>>>>>>>>> in main
> >>>>>>>>>>>> process.run_at_time(-1, run_time)
> >>>>>>>>>>>> File
> >>>>>>>>>>>>
> >>>
>
"/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/ush/reformat_gridded_wrapper.py",
> >>>>>>>>>>>> line 80, in run_at_time
> >>>>>>>>>>>> self.run_at_time_once(task_info, var_info,
rl)
> >>>>>>>>>>>> File
> >>>>>>>>>>>>
> >>>>>>
>
"/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/ush/pcp_combine_wrapper.py",
> >>>>>>>>>>>> line 419, in run_at_time_once
> >>>>>>>>>>>> ....
> >>>>>>>>>>>>
> >>>>>>>>>>>> 'test3' is an attempt from a different angle, thinking
that if
> >>>>>>>>>>>> the
> >>>>>>>>>>>> 'continuously running' bucket didn't work, maybe the 6-
hourly
> >>>>>>>>>>>> bucket
> >>>>>>>>>>>> would. In
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test3
> >>>>>>>>>> ,
> >>>>>>>>>>>> I changed LEAD_SEQ to 24, and set FCST_LEVEL = 6. It
produced
> >>>>>>>>>>>> err
> >>>>>>>> msgs
> >>>>>>>>>>>> (
> >>>>>>>>>>>>
> >>>
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test3
> >>>>>>>>>> )
> >>>>>>>>>>>> that appeared to be the same as that from 'test2'.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Earlier on I tried using the FCST_PCP_COMBINE feature
with the
> >>>>>>>>>>>> NAM
> >>>>>>>>>>>> 'CONUSNEST' output (with simple regular 6-hourly
bucket) and
> >>>>>>>>>>>> got the
> >>>>>>>>>>>> same err msg, so I don't think FV3GFS's dual-bucket
system is
> >>>>>>>>>>>> the
> >>>>>>>> cause.
> >>>>>>>>>>>> Also posted on
> >>>>>>>>>>>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/
> are
> >>>>>>>>>>>> the
> >>>>>>>> two
> >>>>>>>>>>>> metplus_final.conf files from both 'test2' and 'test3'.
> >>>>>>>>>>>>
> >>>>>>>>>>>> I found in
> >>>>>>>>>>>>
> >>>>>> /global/noscrub/Julie.Prestopnik/METplus/METplus-
> >>>>>> 2.0.4/parm/use_cases/qpf/examples/
> >>>>>>>>>>>> that hrefmean-vs-mrms-qpe.conf and nationalblend-vs-
mrms-
> >>>>>>>>>>>> qpe.conf
> >>>>>> had
> >>>>>>>>>>>> "FCST_PCP_COMBINE_RUN = True", that's where I borrowed
the
> >>>>>>>>>>>> FCST_PCP_COMBINE_* set up in my fv3gfs_24h.conf.test2
and
> >>>>>>>>>>>> fv3gfs_24h.conf.test3. Haven't been able to run the
hrefmean
> >>>>>>>>>>>> and
> >>>>>>>>>>>> nationalblend tests myself however - got err msg of "
> >>>>>>>>>>>> CRITICAL:
> >>>>>>>>>>>> master_metplus failed: No option
'GEMPAKTOCF_CLASSPATH' in
> >>>>>>>>>>>> section:
> >>>>>>>>>>>> 'exe'" from both attempts.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thank you for looking into this -
> >>>>>>>>>>>>
> >>>>>>>>>>>> Ying
> >>>>>>>>>>>>
> >>>>>>>>>>>> --
> >>>>>>>>>>>> Ying Lin
> >>>>>>>>>>>> NCEP/EMC/Verification, Post-processing and Product
Generation
> >>>>>>>>>>>> Branch
> >>>>>>>>>>>> NCWCP Cubicle No. 2015
> >>>>>>>>>>>> Ying.Lin at noaa.gov
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>> --
> >>>>>>>>>> Ying Lin
> >>>>>>>>>> NCEP/EMC/Verification, Post-processing and Product
Generation
> >>>>>>>>>> Branch
> >>>>>>>>>> NCWCP Cubicle No. 2015
> >>>>>>>>>> Ying.Lin at noaa.gov
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>> --
> >>>>>>>> Ying Lin
> >>>>>>>> NCEP/EMC/Verification, Post-processing and Product
Generation
> >>>>>>>> Branch
> >>>>>>>> NCWCP Cubicle No. 2015
> >>>>>>>> Ying.Lin at noaa.gov
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>> --
> >>>>>> Ying Lin
> >>>>>> NCEP/EMC/Verification, Post-processing and Product Generation
Branch
> >>>>>> NCWCP Cubicle No. 2015
> >>>>>> Ying.Lin at noaa.gov
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>
> >>> --
> >>> Ying Lin
> >>> NCEP/EMC/Verification, Post-processing and Product Generation
Branch
> >>> NCWCP Cubicle No. 2015
> >>> Ying.Lin at noaa.gov
> >>>
> >>>
> >>>
> >>>
>
> --
> Ying Lin
> NCEP/EMC/Verification, Post-processing and Product Generation Branch
> NCWCP Cubicle No. 2015
> Ying.Lin at noaa.gov
>
>
>
>
------------------------------------------------
Subject: Using FCST_PCP_COMBINE in METplus
From: George McCabe
Time: Thu May 30 15:11:29 2019
If you do use Gempak data in your work (not many users do) and you
don't
have GempakToCF available, let me know and I can send the files to
you.
On Thu, May 30, 2019 at 3:10 PM George McCabe <mccabe at ucar.edu> wrote:
> Hi Ying,
>
> I see that the use case hrefmean-vs-qpe.conf reads Gempak data as
input. I
> thought I had renamed all of the use cases that use Gempak to
contain
> 'gempak' somewhere in the title, but it appears that I missed this
one. To
> run it, you will need to have GempakToCF available on your computer.
If you
> have it, you can set GEMPAKTOCF_BIN to the directory that contains
the
> files for the app. If you don't have it, then I would suggest
picking a
> different use case to run. It is possible I apologize for the
confusion.
>
> On Thu, May 30, 2019 at 2:38 PM Ying Lin via RT <met_help at ucar.edu>
wrote:
>
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789 >
>>
>> Hi George, i was writing to you when your email below came in ...
here
>> it is:
>>
>> Thanks very much! In my system.conf.wcoss, I changed
>> WGRIB2 = $WGRIB2
>> to
>> WGRIB2 = {ENV[WGRIB2]}
>> and that seems to get rid of the $WGRIB2 complaint.
>>
>> Now I'm getting a new err msg:
>> metplus.PcpCombine (config_wrapper.py:93) ERROR: Requested
conf
>> [exe] GEMPAKTOCF_CLASSPATH was not set in config file
>>
>> I see that in my system.conf.wcoss there was this commented out
line:
>> #GEMPAKTOCF_BIN = /d1/mccabe/bin/GempakToCF
>>
>> What should I set GEMPAKTOCF_BIN [and/or _CLASSPATH] to? Or is
there an
>> option to turn the GEMPAKTOCF off, If I won't be using this
function?
>>
>> Thanks again -
>>
>> Ying
>>
>> On 5/30/19 4:32 PM, George McCabe via RT wrote:
>> > The reason you didn't have this issue previously is your use case
likely
>> > didn't need to use any of those exes. I added a function to check
and
>> add
>> > all of the non-MET executables to all of the MET wrappers in case
they
>> are
>> > needed, but I realized later than they are not necessary for most
of the
>> > cases. However, they may become required for other cases so we
may leave
>> > the requirement to set them correctly to run METplus.
>> >
>> > On Thu, May 30, 2019 at 1:36 PM George McCabe <mccabe at ucar.edu>
wrote:
>> >
>> >> Hi Ying,
>> >>
>> >> To reference an environment variable in the METplus config
files, you
>> need
>> >> to use this syntax:
>> >>
>> >> WGRIB2 = {ENV[WGRIB2]}
>> >>
>> >> Since you have set WGRIB2 environment variable, this should set
the
>> >> METplus config variable correctly. You may also be able to set
WGRIB2 =
>> >> wgrib2 in the METplus config file instead. If the exe is in the
path
>> of the
>> >> user running METplus, it will find the app properly. Let me know
if
>> that
>> >> doesn't work.
>> >>
>> >> On Thu, May 30, 2019 at 11:41 AM Ying Lin via RT
<met_help at ucar.edu>
>> >> wrote:
>> >>
>> >>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789 >
>> >>>
>> >>> Hi George,
>> >>>
>> >>> Thanks for following up! The sample case in
>> >>> parm/use_cases/grid_to_grid/examples/precip_continuous.conf
seems to
>> >>> "subtract" correctly now (it was off by 24h in v2.0.4). The
>> >>> precip_continuous.conf only deals with the PcpCombine step
though.
>> >>> When I added the regrid_data_plane step to test a fv3gfs verif
on
>> G212,
>> >>> I got this err msg:
>> >>> metplus.GridStat (config_wrapper.py:144) ERROR:
Executable
>> WGRIB2
>> >>> does not exist at $WGRIB2
>> >>> I thought there must have been a problem in my model config for
the
>> v2.1
>> >>> (found several actually) but when I run the use case at
>> >>> parm/use_cases/qpf/examples/hrefmean-vs-qpe.conf the same thing
>> happened.
>> >>>
>> >>> I'm attaching three files to this email:
>> >>>
>> >>> 1. test_hrefmean.ksh: script to run this use case
>> >>> 2. hrefmean.out: output for running the above script. You
can see
>> that
>> >>> after loading model grid_util (from ". ~/dot.for.metplus)
there
>> was
>> >>> "WGRIB2=/nwprod2/grib_util.v1.0.0/exec/wgrib2
>> >>> export WGRIB2"; and at the end there is the err msg
>> >>> "metplus.GridStat (config_wrapper.py:144) ERROR:
Executable
>> WGRIB2
>> >>> does not exist at $WGRIB2"
>> >>> 3. metplus_final.conf from the above run
>> >>> 4. my "dot.for.metplus" file invoked by test_hrefmean.ksh. I
>> verified
>> >>> manually - after "module load grib_util", $WGRIB2 does
point to
>> >>> /nwprod2/grib_util.v1.0.0/exec/wgrib2
>> >>> 5. my system.conf.wcoss invoked by test_hrefmean.ksh.
>> >>>
>> >>> I looked back at run output from METplus-2.0.4, there was also
>> >>> WGRIB2=/nwprod2/grib_util.v1.0.0/exec/wgrib2 and "export", and
there
>> >>> wasn't a complain about "Executable WGRIB2 does not exist at
$WGRIB2".
>> >>>
>> >>> Thanks again -
>> >>>
>> >>> Ying
>> >>>
>> >>>
>> >>> On 5/29/19 1:53 PM, George McCabe via RT wrote:
>> >>>> Hi Ying,
>> >>>>
>> >>>> I am just following up on this ticket. Have you tried running
your
>> case
>> >>> on METplus v2.1? Let me know if you run into any other issues.
>> >>>> Thanks,
>> >>>> George
>> >>>>
>> >>>> On Thu Apr 18 19:15:58 2019, mccabe wrote:
>> >>>>> Hi Ying,
>> >>>>>
>> >>>>> It looks like the bug with subtract mode is in v2.0.4. It
will be
>> >>>>> fixed in
>> >>>>> the next release. You can look at the develop branch on
GitHub to
>> see
>> >>>>> what
>> >>>>> has changed in the subtract function. Have a good weekend!
>> >>>>>
>> >>>>> On Thu, Apr 18, 2019, 5:09 PM Ying Lin via RT
<met_help at ucar.edu>
>> >>>>> wrote:
>> >>>>>
>> >>>>>> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789 >
>> >>>>>>
>> >>>>>> Hi George,
>> >>>>>>
>> >>>>>> Thank you - you're right, replacing that first
>> >>>>>> FCST_GRID_STAT_INPUT_TEMPLATE with
FCST_PCP_COMBINE_INPUT_TEMPLATE
>> >>>>>> got
>> >>>>>> me past the err msg! There're still some old stuff with the
way
>> the
>> >>>>>> summing was done - similar to the odd result I mentioned in
the
>> last
>> >>>>>> email with the 2.0.4 precip_continuous.conf. I'll study the
new
>> >>>>>> precip_continuous.conf in the github and follow up with you
next
>> >>>>>> week.
>> >>>>>>
>> >>>>>> Thanks for your help and patience today. Have a great
weekend -
>> >>>>>>
>> >>>>>> Ying
>> >>>>>>
>> >>>>>> On 4/18/19 4:25 PM, George McCabe via RT wrote:
>> >>>>>>> Hi Ying,
>> >>>>>>>
>> >>>>>>> I am sorry, I was incorrect that the fixed
precip_continuous
>> config
>> >>>>>>> file
>> >>>>>>> was updated in 2.0.4. It looks like the fix was made after
that
>> >>>>>>> release.
>> >>>>>> It
>> >>>>>>> will be in the newest release. You can see the updated file
on
>> >>>>>>> GitHub in
>> >>>>>>> the develop branch here:
>> >>>>>>>
>> >>>>>>>
>> >>>
>>
https://github.com/NCAR/METplus/blob/develop/parm/use_cases/grid_to_grid/examples/precip_continuous.conf
>> >>>>>>> For your use case, it looks like you don't have
>> >>>>>>> FCST_PCP_COMBINE_INPUT_TEMPLATE set and you have 2 for
grid_stat:
>> >>>>>>>
>> >>>>>>> FCST_GRID_STAT_INPUT_TEMPLATE =
>> >>>>>>>
>> >>>
>>
gfs.{init?fmt=%Y%m%d}/{init?fmt=%H}/gfs.t{init?fmt=%H}z.pgrb2.0p25.f{lead?fmt=%HHH}
>> >>>>>>> FCST_PCP_COMBINE_OUTPUT_TEMPLATE =
>> >>>>>>> gfs.{valid?fmt=%Y%m%d%H}_A{level?fmt=%HHH}h
>> >>>>>>> FCST_GRID_STAT_INPUT_TEMPLATE =
>> >>>>>> gfs.{valid?fmt=%Y%m%d%H}_A{level?fmt=%HHH}h
>> >>>>>>> Fixing this may solve your issue. Please let me know if it
does
>> >>>>>>> not. I
>> >>>>>> have
>> >>>>>>> to catch a flight this evening so I am going to stop
working very
>> >>>>>>> soon,
>> >>>>>> but
>> >>>>>>> if you reply very soon I may still be able to help you
before the
>> >>>>>> weekend.
>> >>>>>>> Thanks,
>> >>>>>>> George
>> >>>>>>>
>> >>>>>>> On Thu, Apr 18, 2019 at 8:13 PM Ying Lin via RT <
>> met_help at ucar.edu>
>> >>>>>> wrote:
>> >>>>>>>> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789 >
>> >>>>>>>>
>> >>>>>>>> Hi George,
>> >>>>>>>>
>> >>>>>>>> Thank you for the quick reply. I was looking at
>> >>>>>>>>
>> >>>>>>
/gpfs/gd1/emc/global/noscrub/Julie.Prestopnik/METplus/METplus-
>> >>>>>>
2.0.4/parm/use_cases/grid_to_grid/examples/precip_continuous.conf,
>> >>>>>>>> which has VALID_BEG = 2018030900, VALID_END = 2018030900.
Date
>> >>>>>>>> stamp is
>> >>>>>>>> Jan 30. Is there a "newer version of v2.0.4" available on
wcoss?
>> >>>>>>>> From
>> >>>>>>>> the above config it's not clear to me how METplus knows to
>> >>>>>>>> subtract the
>> >>>>>>>> gfs.*.f024 from the gfs.*.f048.
>> >>>>>>>>
>> >>>>>>>> If you have time today, could you also take a look why my
simple
>> >>>>>>>> test
>> >>>>>>>> (conf set up to be as close to precip_continuous.conf as
possible
>> >>>>>>>> -
>> >>>>>>>>
>> >>>
>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf
>> >>> )
>> >>>>>>>> led to err msg in
>> >>>>>>>>
>> >>>
>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs.out.test5
>> >>>>>> ?
>> >>>>>>>> It seems that I corrected the problems you pointed out in
your
>> >>>>>>>> reply
>> >>>>>>>> this morning, but the err msg still seems to be the same
as
>> >>>>>>>> yesterday's.
>> >>>>>>>>
>> >>>>>>>> If you don't have time today, next Monday is fine too.
Have a
>> >>>>>>>> good
>> >>>>>>>> holiday weekend.
>> >>>>>>>>
>> >>>>>>>> Ying
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>> On 4/18/19 3:51 PM, George McCabe via RT wrote:
>> >>>>>>>>> Hi Ying,
>> >>>>>>>>>
>> >>>>>>>>> 1) The config file I see has a valid time of 2018031000
and 48
>> hr
>> >>>>>>>> forecast
>> >>>>>>>>> lead. You may be using an old version of METplus that had
this
>> >>>>>>>>> set up
>> >>>>>>>>> incorrectly. METplus version 2.0.4 should be correct.
>> >>>>>>>>>
>> >>>>>>>>> 2) pcp_combine -subtract does not actually use the
FCST_LEVEL
>> >>>>>>>>> variable,
>> >>>>>>>> so
>> >>>>>>>>> that should be removed from the use case. The code takes
the
>> >>>>>>>>> valid time
>> >>>>>>>> and
>> >>>>>>>>> subtracts the accumulation.
>> >>>>>>>>>
>> >>>>>>>>> 3) In the upcoming version of METplus (in a few weeks!)
has a
>> >>>>>>>>> good
>> >>>>>> number
>> >>>>>>>>> of config changes from the previous versions. For
example,
>> >>>>>>>>> FCST_LEVEL
>> >>>>>>>> will
>> >>>>>>>>> now be FCST_PCP_COMBINE_INPUT_LEVEL. These changes are
designed
>> >>>>>>>>> to make
>> >>>>>>>> the
>> >>>>>>>>> variables more clear to what they actually do. In the new
>> >>>>>>>>> version,
>> >>>>>> there
>> >>>>>>>> is
>> >>>>>>>>> now FCST_PCP_COMBINE_METHOD and OBS_PCP_COMBINE_METHOD,
so you
>> >>>>>>>>> will be
>> >>>>>>>> able
>> >>>>>>>>> to specify a different method for your forecast and
analysis
>> >>>>>>>>> data.
>> >>>>>>>>>
>> >>>>>>>>> FCST_PCP_COMBINE_RUN = True
>> >>>>>>>>> FCST_PCP_COMBINE_METHOD = SUBTRACT
>> >>>>>>>>>
>> >>>>>>>>> OBS_PCP_COMBINE_RUN = True
>> >>>>>>>>> OBS_PCP_COMBINE_METHOD = SUM
>> >>>>>>>>>
>> >>>>>>>>> This functionality has not been thoroughly tested as we
haven't
>> >>>>>>>>> had any
>> >>>>>>>>> example use cases that need to do this so far, so please
let me
>> >>>>>>>>> know if
>> >>>>>>>> it
>> >>>>>>>>> does not work the way you expect and I will fix it.
>> >>>>>>>>>
>> >>>>>>>>> Just to let you know, I will be out of the office
tomorrow
>> >>>>>>>>> (Friday)
>> >>>>>> and I
>> >>>>>>>>> wont have access to email. Feel free to send any follow-
up
>> emails
>> >>>>>>>>> to me
>> >>>>>>>> and
>> >>>>>>>>> I will get back to you on Monday.
>> >>>>>>>>>
>> >>>>>>>>> Thanks,
>> >>>>>>>>> George
>> >>>>>>>>>
>> >>>>>>>>> On Thu, Apr 18, 2019 at 7:38 PM Ying Lin via RT
>> >>>>>>>>> <met_help at ucar.edu>
>> >>>>>>>> wrote:
>> >>>>>>>>>> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789
>> >
>> >>>>>>>>>>
>> >>>>>>>>>> Thank you very much George, this is very helpful.
>> >>>>>>>>>>
>> >>>>>>>>>> About the example in
>> >>>>>>>>>>
>> parm/use_cases/grid_to_grid/examples/precip_continuous.conf
>> >>>>>>>>>> I tried running it and it seems to have run through the
>> >>>>>>>>>> pcp_combine
>> >>>>>> step
>> >>>>>>>>>> fine: it did this:
>> >>>>>>>>>>
/global/noscrub/Julie.Prestopnik/met/8.0/bin/pcp_combine
>> >>>>>> -subtract
>> >>>>>> /global/noscrub/Julie.Prestopnik/METplus/METplus-
>> >>>>>>
>> >>>
>>
2.0.4_sample_data/grid_to_grid/prfv3rt1/20180308/gfs.t00z.pgrb.1p00.f048
>> >>>>>>>>>> 48
>> >>>>>>>>>>
>> >>>>>> /global/noscrub/Julie.Prestopnik/METplus/METplus-
>> >>>>>>
>> >>>
>>
2.0.4_sample_data/grid_to_grid/prfv3rt1/20180308/gfs.t00z.pgrb.1p00.f024
>> >>>>>>>>>> 24
>> >>>>>>>>>>
>> /meso/noscrub/Ying.Lin/metplus.out/gfs/bucket/gfs.2018030900_A024h
>> >>>>>>>>>>
>> >>>>>>>>>> Several questions about the example:
>> >>>>>>>>>> 1) The precip_continuous.conf has
>> >>>>>>>>>> LEAD_SEQ = 24
>> >>>>>>>>>> I thought it meant forecast lead of 24h, e.g. look for
00-24h
>> >>>>>>>>>> forecast
>> >>>>>>>>>> valid at 2018030900 (given in the config). If so, why
doesn't
>> >>>>>>>>>> it
>> >>>>>>>>>> simply use gfs.t00z.pgrb.1p00.f024, which already have
the 00-
>> >>>>>>>>>> 24h
>> >>>>>>>>>> forecast? If it subtracts gfs.t00z.pgrb.1p00.f024 from
>> >>>>>>>>>> gfs.t00z.pgrb.1p00.f048, the resulting file would have
24-48h
>> >>>>>>>>>> forecast
>> >>>>>>>>>> valid at 2018031000. I did
>> >>>>>>>>>> ncdump -h
>> >>>>>>>>>>
>> /meso/noscrub/Ying.Lin/metplus.out/gfs/bucket/gfs.2018030900_A024h
>> >>>>>>>>>>
>> >>>>>>>>>> and got
>> >>>>>>>>>> APCP_24:init_time = "20180308_000000" ;
>> >>>>>>>>>> APCP_24:init_time_ut = "1520467200" ;
>> >>>>>>>>>> APCP_24:valid_time = "20180310_000000" ;
>> >>>>>>>>>> APCP_24:valid_time_ut = "1520640000" ;
>> >>>>>>>>>> APCP_24:accum_time = "240000" ;
>> >>>>>>>>>> it seems to mean that the file contains APCP_24h with
initial
>> >>>>>>>>>> time of
>> >>>>>>>>>> 00z 03/08 and the valid time is 00Z 03/10 (that'll be
APCP_48h
>> >>>>>>>>>> instead
>> >>>>>>>>>> of 24h?)
>> >>>>>>>>>>
>> >>>>>>>>>> 2) The precip_continuous.conf has
>> >>>>>>>>>> FCST_LEVEL = 6
>> >>>>>>>>>> I thought FCST_LEVEL is the forecast bucket length. In
the
>> >>>>>>>>>> sample GFS
>> >>>>>>>>>> files, there isn't a 6-hourly bucket -
gfs.t00z.pgrb.1p00.f024
>> >>>>>> contains
>> >>>>>>>>>> 00-24h accumulation and gfs.t00z.pgrb.1p00.f048 contains
00-48h
>> >>>>>>>>>> accumulation. Does FCST_LEVEL mean something other than
the
>> >>>>>>>>>> forecast
>> >>>>>>>>>> bucket length?
>> >>>>>>>>>>
>> >>>>>>>>>> 3) precip_continuous.conf has
>> >>>>>>>>>> PCP_COMBINE_METHOD = SUBTRACT
>> >>>>>>>>>> which should work for the example since the verifying
analysis
>> >>>>>>>>>> is the
>> >>>>>>>>>> 24h CCPA file. But what if I need to use 'SUBTRACT' for
the
>> >>>>>>>>>> forecast
>> >>>>>>>>>> field but 'SUM' for the analysis field (e.g. summing up
6h
>> CCPAs
>> >>>>>>>>>> into
>> >>>>>> a
>> >>>>>>>>>> 24h total)? At one point I tried
>> >>>>>>>>>> PCP_COMBINE_METHOD = SUBTRACT, SUM
>> >>>>>>>>>> and got
>> >>>>>>>>>> ERROR: Invalid PCP_COMBINE_METHOD specified
>> >>>>>>>>>>
>> >>>>>>>>>> After several failed attempts getting my own config file
to
>> >>>>>>>>>> work, I
>> >>>>>>>>>> starting from precip_continuous.conf (which worked
through the
>> >>>>>>>>>> PcpCombine step; I think that's the only step it runs
since the
>> >>>>>>>>>> config
>> >>>>>>>>>> has only PcpCombine in the PROCESS_LIST) and tried to
set a
>> >>>>>>>>>> config
>> >>>>>> file
>> >>>>>>>>>> as close to precip_continuous.conf as possible
>> >>>>>>>>>>
>> >>>
>>
(/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/parm/ylin_config/models/fv3gfs_24h.conf,
>> >>>>>>>>>> also uploaded to
>> >>>>>>>>>>
>> >>>
>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf
>> >>>>>>>> ).
>> >>>>>>>>>> The resulting err msg:
>> >>>>>>>>>>
>> >>>>>>>>>> 04/18 18:35:33.912 jlog (master_metplus.py:217)
CRITICAL:
>> >>>>>>>>>> master_metplus failed: ''
>> >>>>>>>>>> Traceback (most recent call last):
>> >>>>>>>>>> File
"/meso/save/Ying.Lin/metplus/ush/master_metplus.py", line
>> >>>>>> 213,
>> >>>>>>>>>> in <module>
>> >>>>>>>>>> main()
>> >>>>>>>>>> File
>> "/meso/save/Ying.Lin/metplus/ush/master_metplus.py",
>> >>>>>>>>>> line
>> >>>>>> 170,
>> >>>>>>>>>> in main
>> >>>>>>>>>> process.run_at_time(-1, run_time)
>> >>>>>>>>>> File
>> >>>>>>>>>>
>> >>>
>>
"/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/ush/reformat_gridded_wrapper.py",
>> >>>>>>>>>> line 80, in run_at_time
>> >>>>>>>>>>
>> >>>>>>>>>> Full output file:
>> >>>>>>>>>>
>> >>>
>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs.out.test5
>> >>>>>>>> ;
>> >>>>>>>>>> metplus_final.conf:
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>
>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/metplus_final.conf
>> >>>>>>>>>> The quoted err msg above appears to the the same as in
my email
>> >>>>>>>>>> yesterday, so there must still be something wrong with
the
>> >>>>>>>>>> format of
>> >>>>>> my
>> >>>>>>>>>> config, though I think I eliminated the problems you
pointed
>> out
>> >>>>>>>>>> in
>> >>>>>> your
>> >>>>>>>>>> reply. There wouldn't be a ccpa_conus_1.0d_* of the
valid day
>> I
>> >>>>>>>>>> specified, but I think the processing failed way before
that
>> >>>>>>>>>> point.
>> >>>>>>>>>>
>> >>>>>>>>>> Thank you again -
>> >>>>>>>>>>
>> >>>>>>>>>> Ying
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>> On 4/17/19 3:32 PM, George McCabe via RT wrote:
>> >>>>>>>>>>> Hi Ying,
>> >>>>>>>>>>>
>> >>>>>>>>>>> You will need to set FCST_PCP_COMBINE_METHOD = SUBTRACT
to
>> >>>>>>>>>>> config
>> >>>>>>>> METplus
>> >>>>>>>>>>> to subtract accumulations files to get the accumulation
you
>> >>>>>>>>>>> desire.
>> >>>>>> The
>> >>>>>>>>>>> following use case has an example of this
configuration:
>> >>>>>>>>>>>
>> >>>>>>>>>>>
parm/use_cases/grid_to_grid/examples/precip_continous.conf
>> >>>>>>>>>>>
>> >>>>>>>>>>> I also noticed that your config file has the following
items
>> in
>> >>>>>>>>>>> the
>> >>>>>>>>>>> [config] section:
>> >>>>>>>>>>>
>> >>>>>>>>>>> FCST_PCP_COMBINE_INPUT_DIR =
/gpfs/dell1/nco/ops/com/gfs/para
>> >>>>>>>>>>> FCST_PCP_COMBINE_OUTPUT_DIR =
{OUTPUT_BASE}/qpf_grib/bucket
>> >>>>>>>>>>> FCST_GRID_STAT_INPUT_DIR =
{OUTPUT_BASE}/qpf_grib/bucket
>> >>>>>>>>>>>
>> >>>>>>>>>>> You should move these to the [dir] section.
>> >>>>>>>>>>>
>> >>>>>>>>>>> Please let me know if you continue to have issues. The
error
>> >>>>>>>>>>> message
>> >>>>>>>> you
>> >>>>>>>>>>> provided has more information at the bottom that was
not
>> >>>>>>>>>>> included.
>> >>>>>> The
>> >>>>>>>>>> next
>> >>>>>>>>>>> release of METplus contains more extensive error
checking so
>> >>>>>>>>>>> the
>> >>>>>> error
>> >>>>>>>>>>> message from that version should be much more helpful.
>> >>>>>>>>>>>
>> >>>>>>>>>>> Thanks,
>> >>>>>>>>>>> George
>> >>>>>>>>>>>
>> >>>>>>>>>>> On Wed, Apr 17, 2019 at 12:43 PM Ying Lin via RT
>> >>>>>>>>>>> <met_help at ucar.edu>
>> >>>>>>>>>> wrote:
>> >>>>>>>>>>>> Wed Apr 17 12:43:02 2019: Request 89789 was acted
upon.
>> >>>>>>>>>>>> Transaction: Ticket created by ying.lin at noaa.gov
>> >>>>>>>>>>>> Queue: met_help
>> >>>>>>>>>>>> Subject: Using FCST_PCP_COMBINE in METplus
>> >>>>>>>>>>>> Owner: Nobody
>> >>>>>>>>>>>> Requestors: ying.lin at noaa.gov
>> >>>>>>>>>>>> Status: new
>> >>>>>>>>>>>> Ticket <URL:
>> >>>>>>>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789
>> >>>>>>>>>>>> Hello. I haven't been able to get the
FCST_PCP_COMBINE
>> >>>>>>>>>>>> feature to
>> >>>>>>>> work
>> >>>>>>>>>>>> in METplus, running out of ideas and hope you can
help.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> I've put some examples in
>> >>>>>>>>>>>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/
>> >>>>>>>>>>>> (current
>> >>>>>>>>>> test
>> >>>>>>>>>>>> directory:
>> >>>>>>>>>>>> gyre:/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/test).
>> >>>>>>>>>>>> test_fv3gfs_24h.ksh is the simple test script. The
goal is
>> to
>> >>>>>>>> compute a
>> >>>>>>>>>>>> 24h verification ending at 12Z 20190415.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> 'test1' did not need FCST_PCP_COMBINE - the config
file
>> >>>>>>>>>>>> fv3gfs_24h.conf.test1 simply had LEAD_SEQ set to '24',
>> >>>>>>>>>>>> FCST_PCP_COMBINE_RUN was not set. The FV3GFS had two
precip
>> >>>>>> buckets,
>> >>>>>>>> 6h
>> >>>>>>>>>>>> and continuously-accumulating, and METplus easily
found the
>> >>>>>>>>>>>> 24h
>> >>>>>>>> forecast
>> >>>>>>>>>>>> accumulation and went on with the grid_stat.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> 'test2' had 'LEAD_SEQ = 36', and I added the following
in the
>> >>>>>>>>>>>> config
>> >>>>>>>>>>>> (
>> >>>>>>>>>>>>
>> >>>
>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test2
>> >>>>>>>>>>>> ):
>> >>>>>>>>>>>> FCST_PCP_COMBINE_RUN = True
>> >>>>>>>>>>>> FCST_PCP_COMBINE_INPUT_DATATYPE = GRIB
>> >>>>>>>>>>>> FCST_PCP_COMBINE_INPUT_DIR =
>> >>>>>> /gpfs/dell1/nco/ops/com/gfs/para
>> >>>>>>>>>>>> FCST_PCP_COMBINE_OUTPUT_DIR =
{OUTPUT_BASE}/qpf_grib/bucket
>> >>>>>>>>>>>> FCST_GRID_STAT_INPUT_DIR =
{OUTPUT_BASE}/qpf_grib/bucket
>> >>>>>>>>>>>> FCST_MAX_FORECAST = 36
>> >>>>>>>>>>>> FCST_INIT_INTERVAL = 6
>> >>>>>>>>>>>> FCST_LEVEL = 36
>> >>>>>>>>>>>> Here the goal was to see if I could get METplus to
subtract
>> >>>>>>>>>>>> the
>> >>>>>> 00-12h
>> >>>>>>>>>>>> accumulation from gfs.t00z.pgrb2.0p25.f012 from the
00-36h
>> >>>>>>>> accumulation
>> >>>>>>>>>>>> in gfs.t00z.pgrb2.0p25.f036. Thinking that FCST_LEVEL
is the
>> >>>>>> forecast
>> >>>>>>>>>>>> bucket length, I figured setting it to '36' and
artificially
>> >>>>>>>> specifying
>> >>>>>>>>>>>> FCST_MAX_FORECAST = 36 (instead of the full 384h)
should work
>> >>>>>>>>>>>> for
>> >>>>>> this
>> >>>>>>>>>>>> limited test where LEAD_SEQ is set to 36 (is there a
way to
>> >>>>>>>>>>>> specify
>> >>>>>>>> that
>> >>>>>>>>>>>> the bucket never gets emptied?). It failed with the
follow
>> >>>>>>>>>>>> err msg
>> >>>>>>>>>>>> (full message in
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>
>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test2
>> >>>>>>>>>>>> ):
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> 04/16 20:01:42.754 jlog (master_metplus.py:217)
CRITICAL:
>> >>>>>>>>>>>> master_metplus failed: ''
>> >>>>>>>>>>>> Traceback (most recent call last):
>> >>>>>>>>>>>> File
>> >>>>>>>>>>>> "/meso/save/Ying.Lin/metplus/ush/master_metplus.py",
line
>> >>>>>>>> 213,
>> >>>>>>>>>>>> in <module>
>> >>>>>>>>>>>> main()
>> >>>>>>>>>>>> File
>> >>>>>>>>>>>> "/meso/save/Ying.Lin/metplus/ush/master_metplus.py",
line
>> >>>>>>>> 170,
>> >>>>>>>>>>>> in main
>> >>>>>>>>>>>> process.run_at_time(-1, run_time)
>> >>>>>>>>>>>> File
>> >>>>>>>>>>>>
>> >>>
>>
"/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/ush/reformat_gridded_wrapper.py",
>> >>>>>>>>>>>> line 80, in run_at_time
>> >>>>>>>>>>>> self.run_at_time_once(task_info, var_info,
rl)
>> >>>>>>>>>>>> File
>> >>>>>>>>>>>>
>> >>>>>>
>>
"/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/ush/pcp_combine_wrapper.py",
>> >>>>>>>>>>>> line 419, in run_at_time_once
>> >>>>>>>>>>>> ....
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> 'test3' is an attempt from a different angle, thinking
that
>> if
>> >>>>>>>>>>>> the
>> >>>>>>>>>>>> 'continuously running' bucket didn't work, maybe the
6-hourly
>> >>>>>>>>>>>> bucket
>> >>>>>>>>>>>> would. In
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>
>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test3
>> >>>>>>>>>> ,
>> >>>>>>>>>>>> I changed LEAD_SEQ to 24, and set FCST_LEVEL = 6. It
>> produced
>> >>>>>>>>>>>> err
>> >>>>>>>> msgs
>> >>>>>>>>>>>> (
>> >>>>>>>>>>>>
>> >>>
>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test3
>> >>>>>>>>>> )
>> >>>>>>>>>>>> that appeared to be the same as that from 'test2'.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> Earlier on I tried using the FCST_PCP_COMBINE feature
with
>> the
>> >>>>>>>>>>>> NAM
>> >>>>>>>>>>>> 'CONUSNEST' output (with simple regular 6-hourly
bucket) and
>> >>>>>>>>>>>> got the
>> >>>>>>>>>>>> same err msg, so I don't think FV3GFS's dual-bucket
system is
>> >>>>>>>>>>>> the
>> >>>>>>>> cause.
>> >>>>>>>>>>>> Also posted on
>> >>>>>>>>>>>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/
>> are
>> >>>>>>>>>>>> the
>> >>>>>>>> two
>> >>>>>>>>>>>> metplus_final.conf files from both 'test2' and
'test3'.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> I found in
>> >>>>>>>>>>>>
>> >>>>>> /global/noscrub/Julie.Prestopnik/METplus/METplus-
>> >>>>>> 2.0.4/parm/use_cases/qpf/examples/
>> >>>>>>>>>>>> that hrefmean-vs-mrms-qpe.conf and nationalblend-vs-
mrms-
>> >>>>>>>>>>>> qpe.conf
>> >>>>>> had
>> >>>>>>>>>>>> "FCST_PCP_COMBINE_RUN = True", that's where I borrowed
the
>> >>>>>>>>>>>> FCST_PCP_COMBINE_* set up in my fv3gfs_24h.conf.test2
and
>> >>>>>>>>>>>> fv3gfs_24h.conf.test3. Haven't been able to run the
hrefmean
>> >>>>>>>>>>>> and
>> >>>>>>>>>>>> nationalblend tests myself however - got err msg of "
>> >>>>>>>>>>>> CRITICAL:
>> >>>>>>>>>>>> master_metplus failed: No option
'GEMPAKTOCF_CLASSPATH' in
>> >>>>>>>>>>>> section:
>> >>>>>>>>>>>> 'exe'" from both attempts.
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> Thank you for looking into this -
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> Ying
>> >>>>>>>>>>>>
>> >>>>>>>>>>>> --
>> >>>>>>>>>>>> Ying Lin
>> >>>>>>>>>>>> NCEP/EMC/Verification, Post-processing and Product
Generation
>> >>>>>>>>>>>> Branch
>> >>>>>>>>>>>> NCWCP Cubicle No. 2015
>> >>>>>>>>>>>> Ying.Lin at noaa.gov
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>>>>
>> >>>>>>>>>> --
>> >>>>>>>>>> Ying Lin
>> >>>>>>>>>> NCEP/EMC/Verification, Post-processing and Product
Generation
>> >>>>>>>>>> Branch
>> >>>>>>>>>> NCWCP Cubicle No. 2015
>> >>>>>>>>>> Ying.Lin at noaa.gov
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>>>>
>> >>>>>>>> --
>> >>>>>>>> Ying Lin
>> >>>>>>>> NCEP/EMC/Verification, Post-processing and Product
Generation
>> >>>>>>>> Branch
>> >>>>>>>> NCWCP Cubicle No. 2015
>> >>>>>>>> Ying.Lin at noaa.gov
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>> --
>> >>>>>> Ying Lin
>> >>>>>> NCEP/EMC/Verification, Post-processing and Product
Generation
>> Branch
>> >>>>>> NCWCP Cubicle No. 2015
>> >>>>>> Ying.Lin at noaa.gov
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>
>> >>> --
>> >>> Ying Lin
>> >>> NCEP/EMC/Verification, Post-processing and Product Generation
Branch
>> >>> NCWCP Cubicle No. 2015
>> >>> Ying.Lin at noaa.gov
>> >>>
>> >>>
>> >>>
>> >>>
>>
>> --
>> Ying Lin
>> NCEP/EMC/Verification, Post-processing and Product Generation
Branch
>> NCWCP Cubicle No. 2015
>> Ying.Lin at noaa.gov
>>
>>
>>
>>
------------------------------------------------
Subject: Re: [rt.rap.ucar.edu #89789] Using FCST_PCP_COMBINE in METplus
From: Ying Lin
Time: Thu May 30 15:13:24 2019
Hi George,
I don't have to run HREF - it was just something I've tried in
the
prev version. Will try the fv3gfs (grib2). Thanks!
Ying
On 5/30/19 5:11 PM, George McCabe via RT wrote:
> If you do use Gempak data in your work (not many users do) and you
don't
> have GempakToCF available, let me know and I can send the files to
you.
>
> On Thu, May 30, 2019 at 3:10 PM George McCabe <mccabe at ucar.edu>
wrote:
>
>> Hi Ying,
>>
>> I see that the use case hrefmean-vs-qpe.conf reads Gempak data as
input. I
>> thought I had renamed all of the use cases that use Gempak to
contain
>> 'gempak' somewhere in the title, but it appears that I missed this
one. To
>> run it, you will need to have GempakToCF available on your
computer. If you
>> have it, you can set GEMPAKTOCF_BIN to the directory that contains
the
>> files for the app. If you don't have it, then I would suggest
picking a
>> different use case to run. It is possible I apologize for the
confusion.
>>
>> On Thu, May 30, 2019 at 2:38 PM Ying Lin via RT <met_help at ucar.edu>
wrote:
>>
>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789 >
>>>
>>> Hi George, i was writing to you when your email below came in ...
here
>>> it is:
>>>
>>> Thanks very much! In my system.conf.wcoss, I changed
>>> WGRIB2 = $WGRIB2
>>> to
>>> WGRIB2 = {ENV[WGRIB2]}
>>> and that seems to get rid of the $WGRIB2 complaint.
>>>
>>> Now I'm getting a new err msg:
>>> metplus.PcpCombine (config_wrapper.py:93) ERROR: Requested
conf
>>> [exe] GEMPAKTOCF_CLASSPATH was not set in config file
>>>
>>> I see that in my system.conf.wcoss there was this commented out
line:
>>> #GEMPAKTOCF_BIN = /d1/mccabe/bin/GempakToCF
>>>
>>> What should I set GEMPAKTOCF_BIN [and/or _CLASSPATH] to? Or is
there an
>>> option to turn the GEMPAKTOCF off, If I won't be using this
function?
>>>
>>> Thanks again -
>>>
>>> Ying
>>>
>>> On 5/30/19 4:32 PM, George McCabe via RT wrote:
>>>> The reason you didn't have this issue previously is your use case
likely
>>>> didn't need to use any of those exes. I added a function to check
and
>>> add
>>>> all of the non-MET executables to all of the MET wrappers in case
they
>>> are
>>>> needed, but I realized later than they are not necessary for most
of the
>>>> cases. However, they may become required for other cases so we
may leave
>>>> the requirement to set them correctly to run METplus.
>>>>
>>>> On Thu, May 30, 2019 at 1:36 PM George McCabe <mccabe at ucar.edu>
wrote:
>>>>
>>>>> Hi Ying,
>>>>>
>>>>> To reference an environment variable in the METplus config
files, you
>>> need
>>>>> to use this syntax:
>>>>>
>>>>> WGRIB2 = {ENV[WGRIB2]}
>>>>>
>>>>> Since you have set WGRIB2 environment variable, this should set
the
>>>>> METplus config variable correctly. You may also be able to set
WGRIB2 =
>>>>> wgrib2 in the METplus config file instead. If the exe is in the
path
>>> of the
>>>>> user running METplus, it will find the app properly. Let me know
if
>>> that
>>>>> doesn't work.
>>>>>
>>>>> On Thu, May 30, 2019 at 11:41 AM Ying Lin via RT
<met_help at ucar.edu>
>>>>> wrote:
>>>>>
>>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789 >
>>>>>>
>>>>>> Hi George,
>>>>>>
>>>>>> Thanks for following up! The sample case in
>>>>>> parm/use_cases/grid_to_grid/examples/precip_continuous.conf
seems to
>>>>>> "subtract" correctly now (it was off by 24h in v2.0.4). The
>>>>>> precip_continuous.conf only deals with the PcpCombine step
though.
>>>>>> When I added the regrid_data_plane step to test a fv3gfs verif
on
>>> G212,
>>>>>> I got this err msg:
>>>>>> metplus.GridStat (config_wrapper.py:144) ERROR:
Executable
>>> WGRIB2
>>>>>> does not exist at $WGRIB2
>>>>>> I thought there must have been a problem in my model config for
the
>>> v2.1
>>>>>> (found several actually) but when I run the use case at
>>>>>> parm/use_cases/qpf/examples/hrefmean-vs-qpe.conf the same thing
>>> happened.
>>>>>> I'm attaching three files to this email:
>>>>>>
>>>>>> 1. test_hrefmean.ksh: script to run this use case
>>>>>> 2. hrefmean.out: output for running the above script. You
can see
>>> that
>>>>>> after loading model grid_util (from ". ~/dot.for.metplus)
there
>>> was
>>>>>> "WGRIB2=/nwprod2/grib_util.v1.0.0/exec/wgrib2
>>>>>> export WGRIB2"; and at the end there is the err msg
>>>>>> "metplus.GridStat (config_wrapper.py:144) ERROR:
Executable
>>> WGRIB2
>>>>>> does not exist at $WGRIB2"
>>>>>> 3. metplus_final.conf from the above run
>>>>>> 4. my "dot.for.metplus" file invoked by test_hrefmean.ksh.
I
>>> verified
>>>>>> manually - after "module load grib_util", $WGRIB2 does
point to
>>>>>> /nwprod2/grib_util.v1.0.0/exec/wgrib2
>>>>>> 5. my system.conf.wcoss invoked by test_hrefmean.ksh.
>>>>>>
>>>>>> I looked back at run output from METplus-2.0.4, there was also
>>>>>> WGRIB2=/nwprod2/grib_util.v1.0.0/exec/wgrib2 and "export", and
there
>>>>>> wasn't a complain about "Executable WGRIB2 does not exist at
$WGRIB2".
>>>>>>
>>>>>> Thanks again -
>>>>>>
>>>>>> Ying
>>>>>>
>>>>>>
>>>>>> On 5/29/19 1:53 PM, George McCabe via RT wrote:
>>>>>>> Hi Ying,
>>>>>>>
>>>>>>> I am just following up on this ticket. Have you tried running
your
>>> case
>>>>>> on METplus v2.1? Let me know if you run into any other issues.
>>>>>>> Thanks,
>>>>>>> George
>>>>>>>
>>>>>>> On Thu Apr 18 19:15:58 2019, mccabe wrote:
>>>>>>>> Hi Ying,
>>>>>>>>
>>>>>>>> It looks like the bug with subtract mode is in v2.0.4. It
will be
>>>>>>>> fixed in
>>>>>>>> the next release. You can look at the develop branch on
GitHub to
>>> see
>>>>>>>> what
>>>>>>>> has changed in the subtract function. Have a good weekend!
>>>>>>>>
>>>>>>>> On Thu, Apr 18, 2019, 5:09 PM Ying Lin via RT
<met_help at ucar.edu>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789 >
>>>>>>>>>
>>>>>>>>> Hi George,
>>>>>>>>>
>>>>>>>>> Thank you - you're right, replacing that first
>>>>>>>>> FCST_GRID_STAT_INPUT_TEMPLATE with
FCST_PCP_COMBINE_INPUT_TEMPLATE
>>>>>>>>> got
>>>>>>>>> me past the err msg! There're still some old stuff with the
way
>>> the
>>>>>>>>> summing was done - similar to the odd result I mentioned in
the
>>> last
>>>>>>>>> email with the 2.0.4 precip_continuous.conf. I'll study the
new
>>>>>>>>> precip_continuous.conf in the github and follow up with you
next
>>>>>>>>> week.
>>>>>>>>>
>>>>>>>>> Thanks for your help and patience today. Have a great
weekend -
>>>>>>>>>
>>>>>>>>> Ying
>>>>>>>>>
>>>>>>>>> On 4/18/19 4:25 PM, George McCabe via RT wrote:
>>>>>>>>>> Hi Ying,
>>>>>>>>>>
>>>>>>>>>> I am sorry, I was incorrect that the fixed
precip_continuous
>>> config
>>>>>>>>>> file
>>>>>>>>>> was updated in 2.0.4. It looks like the fix was made after
that
>>>>>>>>>> release.
>>>>>>>>> It
>>>>>>>>>> will be in the newest release. You can see the updated file
on
>>>>>>>>>> GitHub in
>>>>>>>>>> the develop branch here:
>>>>>>>>>>
>>>>>>>>>>
>>>
https://github.com/NCAR/METplus/blob/develop/parm/use_cases/grid_to_grid/examples/precip_continuous.conf
>>>>>>>>>> For your use case, it looks like you don't have
>>>>>>>>>> FCST_PCP_COMBINE_INPUT_TEMPLATE set and you have 2 for
grid_stat:
>>>>>>>>>>
>>>>>>>>>> FCST_GRID_STAT_INPUT_TEMPLATE =
>>>>>>>>>>
>>>
gfs.{init?fmt=%Y%m%d}/{init?fmt=%H}/gfs.t{init?fmt=%H}z.pgrb2.0p25.f{lead?fmt=%HHH}
>>>>>>>>>> FCST_PCP_COMBINE_OUTPUT_TEMPLATE =
>>>>>>>>>> gfs.{valid?fmt=%Y%m%d%H}_A{level?fmt=%HHH}h
>>>>>>>>>> FCST_GRID_STAT_INPUT_TEMPLATE =
>>>>>>>>> gfs.{valid?fmt=%Y%m%d%H}_A{level?fmt=%HHH}h
>>>>>>>>>> Fixing this may solve your issue. Please let me know if it
does
>>>>>>>>>> not. I
>>>>>>>>> have
>>>>>>>>>> to catch a flight this evening so I am going to stop
working very
>>>>>>>>>> soon,
>>>>>>>>> but
>>>>>>>>>> if you reply very soon I may still be able to help you
before the
>>>>>>>>> weekend.
>>>>>>>>>> Thanks,
>>>>>>>>>> George
>>>>>>>>>>
>>>>>>>>>> On Thu, Apr 18, 2019 at 8:13 PM Ying Lin via RT <
>>> met_help at ucar.edu>
>>>>>>>>> wrote:
>>>>>>>>>>> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789 >
>>>>>>>>>>>
>>>>>>>>>>> Hi George,
>>>>>>>>>>>
>>>>>>>>>>> Thank you for the quick reply. I was looking at
>>>>>>>>>>>
>>>>>>>>>
/gpfs/gd1/emc/global/noscrub/Julie.Prestopnik/METplus/METplus-
>>>>>>>>>
2.0.4/parm/use_cases/grid_to_grid/examples/precip_continuous.conf,
>>>>>>>>>>> which has VALID_BEG = 2018030900, VALID_END = 2018030900.
Date
>>>>>>>>>>> stamp is
>>>>>>>>>>> Jan 30. Is there a "newer version of v2.0.4" available on
wcoss?
>>>>>>>>>>> From
>>>>>>>>>>> the above config it's not clear to me how METplus knows to
>>>>>>>>>>> subtract the
>>>>>>>>>>> gfs.*.f024 from the gfs.*.f048.
>>>>>>>>>>>
>>>>>>>>>>> If you have time today, could you also take a look why my
simple
>>>>>>>>>>> test
>>>>>>>>>>> (conf set up to be as close to precip_continuous.conf as
possible
>>>>>>>>>>> -
>>>>>>>>>>>
>>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf
>>>>>> )
>>>>>>>>>>> led to err msg in
>>>>>>>>>>>
>>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs.out.test5
>>>>>>>>> ?
>>>>>>>>>>> It seems that I corrected the problems you pointed out in
your
>>>>>>>>>>> reply
>>>>>>>>>>> this morning, but the err msg still seems to be the same
as
>>>>>>>>>>> yesterday's.
>>>>>>>>>>>
>>>>>>>>>>> If you don't have time today, next Monday is fine too.
Have a
>>>>>>>>>>> good
>>>>>>>>>>> holiday weekend.
>>>>>>>>>>>
>>>>>>>>>>> Ying
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On 4/18/19 3:51 PM, George McCabe via RT wrote:
>>>>>>>>>>>> Hi Ying,
>>>>>>>>>>>>
>>>>>>>>>>>> 1) The config file I see has a valid time of 2018031000
and 48
>>> hr
>>>>>>>>>>> forecast
>>>>>>>>>>>> lead. You may be using an old version of METplus that had
this
>>>>>>>>>>>> set up
>>>>>>>>>>>> incorrectly. METplus version 2.0.4 should be correct.
>>>>>>>>>>>>
>>>>>>>>>>>> 2) pcp_combine -subtract does not actually use the
FCST_LEVEL
>>>>>>>>>>>> variable,
>>>>>>>>>>> so
>>>>>>>>>>>> that should be removed from the use case. The code takes
the
>>>>>>>>>>>> valid time
>>>>>>>>>>> and
>>>>>>>>>>>> subtracts the accumulation.
>>>>>>>>>>>>
>>>>>>>>>>>> 3) In the upcoming version of METplus (in a few weeks!)
has a
>>>>>>>>>>>> good
>>>>>>>>> number
>>>>>>>>>>>> of config changes from the previous versions. For
example,
>>>>>>>>>>>> FCST_LEVEL
>>>>>>>>>>> will
>>>>>>>>>>>> now be FCST_PCP_COMBINE_INPUT_LEVEL. These changes are
designed
>>>>>>>>>>>> to make
>>>>>>>>>>> the
>>>>>>>>>>>> variables more clear to what they actually do. In the new
>>>>>>>>>>>> version,
>>>>>>>>> there
>>>>>>>>>>> is
>>>>>>>>>>>> now FCST_PCP_COMBINE_METHOD and OBS_PCP_COMBINE_METHOD,
so you
>>>>>>>>>>>> will be
>>>>>>>>>>> able
>>>>>>>>>>>> to specify a different method for your forecast and
analysis
>>>>>>>>>>>> data.
>>>>>>>>>>>>
>>>>>>>>>>>> FCST_PCP_COMBINE_RUN = True
>>>>>>>>>>>> FCST_PCP_COMBINE_METHOD = SUBTRACT
>>>>>>>>>>>>
>>>>>>>>>>>> OBS_PCP_COMBINE_RUN = True
>>>>>>>>>>>> OBS_PCP_COMBINE_METHOD = SUM
>>>>>>>>>>>>
>>>>>>>>>>>> This functionality has not been thoroughly tested as we
haven't
>>>>>>>>>>>> had any
>>>>>>>>>>>> example use cases that need to do this so far, so please
let me
>>>>>>>>>>>> know if
>>>>>>>>>>> it
>>>>>>>>>>>> does not work the way you expect and I will fix it.
>>>>>>>>>>>>
>>>>>>>>>>>> Just to let you know, I will be out of the office
tomorrow
>>>>>>>>>>>> (Friday)
>>>>>>>>> and I
>>>>>>>>>>>> wont have access to email. Feel free to send any follow-
up
>>> emails
>>>>>>>>>>>> to me
>>>>>>>>>>> and
>>>>>>>>>>>> I will get back to you on Monday.
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> George
>>>>>>>>>>>>
>>>>>>>>>>>> On Thu, Apr 18, 2019 at 7:38 PM Ying Lin via RT
>>>>>>>>>>>> <met_help at ucar.edu>
>>>>>>>>>>> wrote:
>>>>>>>>>>>>> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789
>>>>>>>>>>>>> Thank you very much George, this is very helpful.
>>>>>>>>>>>>>
>>>>>>>>>>>>> About the example in
>>>>>>>>>>>>>
>>> parm/use_cases/grid_to_grid/examples/precip_continuous.conf
>>>>>>>>>>>>> I tried running it and it seems to have run through the
>>>>>>>>>>>>> pcp_combine
>>>>>>>>> step
>>>>>>>>>>>>> fine: it did this:
>>>>>>>>>>>>>
/global/noscrub/Julie.Prestopnik/met/8.0/bin/pcp_combine
>>>>>>>>> -subtract
>>>>>>>>> /global/noscrub/Julie.Prestopnik/METplus/METplus-
>>>>>>>>>
>>>
2.0.4_sample_data/grid_to_grid/prfv3rt1/20180308/gfs.t00z.pgrb.1p00.f048
>>>>>>>>>>>>> 48
>>>>>>>>>>>>>
>>>>>>>>> /global/noscrub/Julie.Prestopnik/METplus/METplus-
>>>>>>>>>
>>>
2.0.4_sample_data/grid_to_grid/prfv3rt1/20180308/gfs.t00z.pgrb.1p00.f024
>>>>>>>>>>>>> 24
>>>>>>>>>>>>>
>>> /meso/noscrub/Ying.Lin/metplus.out/gfs/bucket/gfs.2018030900_A024h
>>>>>>>>>>>>> Several questions about the example:
>>>>>>>>>>>>> 1) The precip_continuous.conf has
>>>>>>>>>>>>> LEAD_SEQ = 24
>>>>>>>>>>>>> I thought it meant forecast lead of 24h, e.g. look for
00-24h
>>>>>>>>>>>>> forecast
>>>>>>>>>>>>> valid at 2018030900 (given in the config). If so, why
doesn't
>>>>>>>>>>>>> it
>>>>>>>>>>>>> simply use gfs.t00z.pgrb.1p00.f024, which already have
the 00-
>>>>>>>>>>>>> 24h
>>>>>>>>>>>>> forecast? If it subtracts gfs.t00z.pgrb.1p00.f024 from
>>>>>>>>>>>>> gfs.t00z.pgrb.1p00.f048, the resulting file would have
24-48h
>>>>>>>>>>>>> forecast
>>>>>>>>>>>>> valid at 2018031000. I did
>>>>>>>>>>>>> ncdump -h
>>>>>>>>>>>>>
>>> /meso/noscrub/Ying.Lin/metplus.out/gfs/bucket/gfs.2018030900_A024h
>>>>>>>>>>>>> and got
>>>>>>>>>>>>> APCP_24:init_time = "20180308_000000" ;
>>>>>>>>>>>>> APCP_24:init_time_ut = "1520467200" ;
>>>>>>>>>>>>> APCP_24:valid_time = "20180310_000000" ;
>>>>>>>>>>>>> APCP_24:valid_time_ut = "1520640000" ;
>>>>>>>>>>>>> APCP_24:accum_time = "240000" ;
>>>>>>>>>>>>> it seems to mean that the file contains APCP_24h with
initial
>>>>>>>>>>>>> time of
>>>>>>>>>>>>> 00z 03/08 and the valid time is 00Z 03/10 (that'll be
APCP_48h
>>>>>>>>>>>>> instead
>>>>>>>>>>>>> of 24h?)
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2) The precip_continuous.conf has
>>>>>>>>>>>>> FCST_LEVEL = 6
>>>>>>>>>>>>> I thought FCST_LEVEL is the forecast bucket length. In
the
>>>>>>>>>>>>> sample GFS
>>>>>>>>>>>>> files, there isn't a 6-hourly bucket -
gfs.t00z.pgrb.1p00.f024
>>>>>>>>> contains
>>>>>>>>>>>>> 00-24h accumulation and gfs.t00z.pgrb.1p00.f048 contains
00-48h
>>>>>>>>>>>>> accumulation. Does FCST_LEVEL mean something other than
the
>>>>>>>>>>>>> forecast
>>>>>>>>>>>>> bucket length?
>>>>>>>>>>>>>
>>>>>>>>>>>>> 3) precip_continuous.conf has
>>>>>>>>>>>>> PCP_COMBINE_METHOD = SUBTRACT
>>>>>>>>>>>>> which should work for the example since the verifying
analysis
>>>>>>>>>>>>> is the
>>>>>>>>>>>>> 24h CCPA file. But what if I need to use 'SUBTRACT' for
the
>>>>>>>>>>>>> forecast
>>>>>>>>>>>>> field but 'SUM' for the analysis field (e.g. summing up
6h
>>> CCPAs
>>>>>>>>>>>>> into
>>>>>>>>> a
>>>>>>>>>>>>> 24h total)? At one point I tried
>>>>>>>>>>>>> PCP_COMBINE_METHOD = SUBTRACT, SUM
>>>>>>>>>>>>> and got
>>>>>>>>>>>>> ERROR: Invalid PCP_COMBINE_METHOD specified
>>>>>>>>>>>>>
>>>>>>>>>>>>> After several failed attempts getting my own config file
to
>>>>>>>>>>>>> work, I
>>>>>>>>>>>>> starting from precip_continuous.conf (which worked
through the
>>>>>>>>>>>>> PcpCombine step; I think that's the only step it runs
since the
>>>>>>>>>>>>> config
>>>>>>>>>>>>> has only PcpCombine in the PROCESS_LIST) and tried to
set a
>>>>>>>>>>>>> config
>>>>>>>>> file
>>>>>>>>>>>>> as close to precip_continuous.conf as possible
>>>>>>>>>>>>>
>>>
(/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/parm/ylin_config/models/fv3gfs_24h.conf,
>>>>>>>>>>>>> also uploaded to
>>>>>>>>>>>>>
>>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf
>>>>>>>>>>> ).
>>>>>>>>>>>>> The resulting err msg:
>>>>>>>>>>>>>
>>>>>>>>>>>>> 04/18 18:35:33.912 jlog (master_metplus.py:217)
CRITICAL:
>>>>>>>>>>>>> master_metplus failed: ''
>>>>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>>>>> File
"/meso/save/Ying.Lin/metplus/ush/master_metplus.py", line
>>>>>>>>> 213,
>>>>>>>>>>>>> in <module>
>>>>>>>>>>>>> main()
>>>>>>>>>>>>> File
>>> "/meso/save/Ying.Lin/metplus/ush/master_metplus.py",
>>>>>>>>>>>>> line
>>>>>>>>> 170,
>>>>>>>>>>>>> in main
>>>>>>>>>>>>> process.run_at_time(-1, run_time)
>>>>>>>>>>>>> File
>>>>>>>>>>>>>
>>>
"/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/ush/reformat_gridded_wrapper.py",
>>>>>>>>>>>>> line 80, in run_at_time
>>>>>>>>>>>>>
>>>>>>>>>>>>> Full output file:
>>>>>>>>>>>>>
>>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs.out.test5
>>>>>>>>>>> ;
>>>>>>>>>>>>> metplus_final.conf:
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/metplus_final.conf
>>>>>>>>>>>>> The quoted err msg above appears to the the same as in
my email
>>>>>>>>>>>>> yesterday, so there must still be something wrong with
the
>>>>>>>>>>>>> format of
>>>>>>>>> my
>>>>>>>>>>>>> config, though I think I eliminated the problems you
pointed
>>> out
>>>>>>>>>>>>> in
>>>>>>>>> your
>>>>>>>>>>>>> reply. There wouldn't be a ccpa_conus_1.0d_* of the
valid day
>>> I
>>>>>>>>>>>>> specified, but I think the processing failed way before
that
>>>>>>>>>>>>> point.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thank you again -
>>>>>>>>>>>>>
>>>>>>>>>>>>> Ying
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 4/17/19 3:32 PM, George McCabe via RT wrote:
>>>>>>>>>>>>>> Hi Ying,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> You will need to set FCST_PCP_COMBINE_METHOD = SUBTRACT
to
>>>>>>>>>>>>>> config
>>>>>>>>>>> METplus
>>>>>>>>>>>>>> to subtract accumulations files to get the accumulation
you
>>>>>>>>>>>>>> desire.
>>>>>>>>> The
>>>>>>>>>>>>>> following use case has an example of this
configuration:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
parm/use_cases/grid_to_grid/examples/precip_continous.conf
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I also noticed that your config file has the following
items
>>> in
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>> [config] section:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> FCST_PCP_COMBINE_INPUT_DIR =
/gpfs/dell1/nco/ops/com/gfs/para
>>>>>>>>>>>>>> FCST_PCP_COMBINE_OUTPUT_DIR =
{OUTPUT_BASE}/qpf_grib/bucket
>>>>>>>>>>>>>> FCST_GRID_STAT_INPUT_DIR =
{OUTPUT_BASE}/qpf_grib/bucket
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> You should move these to the [dir] section.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Please let me know if you continue to have issues. The
error
>>>>>>>>>>>>>> message
>>>>>>>>>>> you
>>>>>>>>>>>>>> provided has more information at the bottom that was
not
>>>>>>>>>>>>>> included.
>>>>>>>>> The
>>>>>>>>>>>>> next
>>>>>>>>>>>>>> release of METplus contains more extensive error
checking so
>>>>>>>>>>>>>> the
>>>>>>>>> error
>>>>>>>>>>>>>> message from that version should be much more helpful.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> George
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, Apr 17, 2019 at 12:43 PM Ying Lin via RT
>>>>>>>>>>>>>> <met_help at ucar.edu>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>> Wed Apr 17 12:43:02 2019: Request 89789 was acted
upon.
>>>>>>>>>>>>>>> Transaction: Ticket created by ying.lin at noaa.gov
>>>>>>>>>>>>>>> Queue: met_help
>>>>>>>>>>>>>>> Subject: Using FCST_PCP_COMBINE in METplus
>>>>>>>>>>>>>>> Owner: Nobody
>>>>>>>>>>>>>>> Requestors: ying.lin at noaa.gov
>>>>>>>>>>>>>>> Status: new
>>>>>>>>>>>>>>> Ticket <URL:
>>>>>>>>>>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789
>>>>>>>>>>>>>>> Hello. I haven't been able to get the
FCST_PCP_COMBINE
>>>>>>>>>>>>>>> feature to
>>>>>>>>>>> work
>>>>>>>>>>>>>>> in METplus, running out of ideas and hope you can
help.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I've put some examples in
>>>>>>>>>>>>>>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/
>>>>>>>>>>>>>>> (current
>>>>>>>>>>>>> test
>>>>>>>>>>>>>>> directory:
>>>>>>>>>>>>>>> gyre:/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/test).
>>>>>>>>>>>>>>> test_fv3gfs_24h.ksh is the simple test script. The
goal is
>>> to
>>>>>>>>>>> compute a
>>>>>>>>>>>>>>> 24h verification ending at 12Z 20190415.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 'test1' did not need FCST_PCP_COMBINE - the config
file
>>>>>>>>>>>>>>> fv3gfs_24h.conf.test1 simply had LEAD_SEQ set to '24',
>>>>>>>>>>>>>>> FCST_PCP_COMBINE_RUN was not set. The FV3GFS had two
precip
>>>>>>>>> buckets,
>>>>>>>>>>> 6h
>>>>>>>>>>>>>>> and continuously-accumulating, and METplus easily
found the
>>>>>>>>>>>>>>> 24h
>>>>>>>>>>> forecast
>>>>>>>>>>>>>>> accumulation and went on with the grid_stat.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 'test2' had 'LEAD_SEQ = 36', and I added the following
in the
>>>>>>>>>>>>>>> config
>>>>>>>>>>>>>>> (
>>>>>>>>>>>>>>>
>>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test2
>>>>>>>>>>>>>>> ):
>>>>>>>>>>>>>>> FCST_PCP_COMBINE_RUN = True
>>>>>>>>>>>>>>> FCST_PCP_COMBINE_INPUT_DATATYPE = GRIB
>>>>>>>>>>>>>>> FCST_PCP_COMBINE_INPUT_DIR =
>>>>>>>>> /gpfs/dell1/nco/ops/com/gfs/para
>>>>>>>>>>>>>>> FCST_PCP_COMBINE_OUTPUT_DIR =
{OUTPUT_BASE}/qpf_grib/bucket
>>>>>>>>>>>>>>> FCST_GRID_STAT_INPUT_DIR =
{OUTPUT_BASE}/qpf_grib/bucket
>>>>>>>>>>>>>>> FCST_MAX_FORECAST = 36
>>>>>>>>>>>>>>> FCST_INIT_INTERVAL = 6
>>>>>>>>>>>>>>> FCST_LEVEL = 36
>>>>>>>>>>>>>>> Here the goal was to see if I could get METplus to
subtract
>>>>>>>>>>>>>>> the
>>>>>>>>> 00-12h
>>>>>>>>>>>>>>> accumulation from gfs.t00z.pgrb2.0p25.f012 from the
00-36h
>>>>>>>>>>> accumulation
>>>>>>>>>>>>>>> in gfs.t00z.pgrb2.0p25.f036. Thinking that FCST_LEVEL
is the
>>>>>>>>> forecast
>>>>>>>>>>>>>>> bucket length, I figured setting it to '36' and
artificially
>>>>>>>>>>> specifying
>>>>>>>>>>>>>>> FCST_MAX_FORECAST = 36 (instead of the full 384h)
should work
>>>>>>>>>>>>>>> for
>>>>>>>>> this
>>>>>>>>>>>>>>> limited test where LEAD_SEQ is set to 36 (is there a
way to
>>>>>>>>>>>>>>> specify
>>>>>>>>>>> that
>>>>>>>>>>>>>>> the bucket never gets emptied?). It failed with the
follow
>>>>>>>>>>>>>>> err msg
>>>>>>>>>>>>>>> (full message in
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test2
>>>>>>>>>>>>>>> ):
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 04/16 20:01:42.754 jlog (master_metplus.py:217)
CRITICAL:
>>>>>>>>>>>>>>> master_metplus failed: ''
>>>>>>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>>>>>>> File
>>>>>>>>>>>>>>> "/meso/save/Ying.Lin/metplus/ush/master_metplus.py",
line
>>>>>>>>>>> 213,
>>>>>>>>>>>>>>> in <module>
>>>>>>>>>>>>>>> main()
>>>>>>>>>>>>>>> File
>>>>>>>>>>>>>>> "/meso/save/Ying.Lin/metplus/ush/master_metplus.py",
line
>>>>>>>>>>> 170,
>>>>>>>>>>>>>>> in main
>>>>>>>>>>>>>>> process.run_at_time(-1, run_time)
>>>>>>>>>>>>>>> File
>>>>>>>>>>>>>>>
>>>
"/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/ush/reformat_gridded_wrapper.py",
>>>>>>>>>>>>>>> line 80, in run_at_time
>>>>>>>>>>>>>>> self.run_at_time_once(task_info, var_info,
rl)
>>>>>>>>>>>>>>> File
>>>>>>>>>>>>>>>
>>>
"/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/ush/pcp_combine_wrapper.py",
>>>>>>>>>>>>>>> line 419, in run_at_time_once
>>>>>>>>>>>>>>> ....
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 'test3' is an attempt from a different angle, thinking
that
>>> if
>>>>>>>>>>>>>>> the
>>>>>>>>>>>>>>> 'continuously running' bucket didn't work, maybe the
6-hourly
>>>>>>>>>>>>>>> bucket
>>>>>>>>>>>>>>> would. In
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test3
>>>>>>>>>>>>> ,
>>>>>>>>>>>>>>> I changed LEAD_SEQ to 24, and set FCST_LEVEL = 6. It
>>> produced
>>>>>>>>>>>>>>> err
>>>>>>>>>>> msgs
>>>>>>>>>>>>>>> (
>>>>>>>>>>>>>>>
>>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test3
>>>>>>>>>>>>> )
>>>>>>>>>>>>>>> that appeared to be the same as that from 'test2'.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Earlier on I tried using the FCST_PCP_COMBINE feature
with
>>> the
>>>>>>>>>>>>>>> NAM
>>>>>>>>>>>>>>> 'CONUSNEST' output (with simple regular 6-hourly
bucket) and
>>>>>>>>>>>>>>> got the
>>>>>>>>>>>>>>> same err msg, so I don't think FV3GFS's dual-bucket
system is
>>>>>>>>>>>>>>> the
>>>>>>>>>>> cause.
>>>>>>>>>>>>>>> Also posted on
>>>>>>>>>>>>>>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/
>>> are
>>>>>>>>>>>>>>> the
>>>>>>>>>>> two
>>>>>>>>>>>>>>> metplus_final.conf files from both 'test2' and
'test3'.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I found in
>>>>>>>>>>>>>>>
>>>>>>>>> /global/noscrub/Julie.Prestopnik/METplus/METplus-
>>>>>>>>> 2.0.4/parm/use_cases/qpf/examples/
>>>>>>>>>>>>>>> that hrefmean-vs-mrms-qpe.conf and nationalblend-vs-
mrms-
>>>>>>>>>>>>>>> qpe.conf
>>>>>>>>> had
>>>>>>>>>>>>>>> "FCST_PCP_COMBINE_RUN = True", that's where I borrowed
the
>>>>>>>>>>>>>>> FCST_PCP_COMBINE_* set up in my fv3gfs_24h.conf.test2
and
>>>>>>>>>>>>>>> fv3gfs_24h.conf.test3. Haven't been able to run the
hrefmean
>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>> nationalblend tests myself however - got err msg of "
>>>>>>>>>>>>>>> CRITICAL:
>>>>>>>>>>>>>>> master_metplus failed: No option
'GEMPAKTOCF_CLASSPATH' in
>>>>>>>>>>>>>>> section:
>>>>>>>>>>>>>>> 'exe'" from both attempts.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thank you for looking into this -
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Ying
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> Ying Lin
>>>>>>>>>>>>>>> NCEP/EMC/Verification, Post-processing and Product
Generation
>>>>>>>>>>>>>>> Branch
>>>>>>>>>>>>>>> NCWCP Cubicle No. 2015
>>>>>>>>>>>>>>> Ying.Lin at noaa.gov
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> Ying Lin
>>>>>>>>>>>>> NCEP/EMC/Verification, Post-processing and Product
Generation
>>>>>>>>>>>>> Branch
>>>>>>>>>>>>> NCWCP Cubicle No. 2015
>>>>>>>>>>>>> Ying.Lin at noaa.gov
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Ying Lin
>>>>>>>>>>> NCEP/EMC/Verification, Post-processing and Product
Generation
>>>>>>>>>>> Branch
>>>>>>>>>>> NCWCP Cubicle No. 2015
>>>>>>>>>>> Ying.Lin at noaa.gov
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Ying Lin
>>>>>>>>> NCEP/EMC/Verification, Post-processing and Product
Generation
>>> Branch
>>>>>>>>> NCWCP Cubicle No. 2015
>>>>>>>>> Ying.Lin at noaa.gov
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>> --
>>>>>> Ying Lin
>>>>>> NCEP/EMC/Verification, Post-processing and Product Generation
Branch
>>>>>> NCWCP Cubicle No. 2015
>>>>>> Ying.Lin at noaa.gov
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>> --
>>> Ying Lin
>>> NCEP/EMC/Verification, Post-processing and Product Generation
Branch
>>> NCWCP Cubicle No. 2015
>>> Ying.Lin at noaa.gov
>>>
>>>
>>>
>>>
--
Ying Lin
NCEP/EMC/Verification, Post-processing and Product Generation Branch
NCWCP Cubicle No. 2015
Ying.Lin at noaa.gov
------------------------------------------------
Subject: Using FCST_PCP_COMBINE in METplus
From: George McCabe
Time: Thu May 30 15:17:57 2019
OK, sounds good. I am going to close this ticket. Feel free to create
a new
one if you have any other issues and I will help you again.
Thanks,
George
On Thu, May 30, 2019 at 3:13 PM Ying Lin via RT <met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789 >
>
> Hi George,
> I don't have to run HREF - it was just something I've tried in
the
> prev version. Will try the fv3gfs (grib2). Thanks!
> Ying
>
> On 5/30/19 5:11 PM, George McCabe via RT wrote:
> > If you do use Gempak data in your work (not many users do) and you
don't
> > have GempakToCF available, let me know and I can send the files to
you.
> >
> > On Thu, May 30, 2019 at 3:10 PM George McCabe <mccabe at ucar.edu>
wrote:
> >
> >> Hi Ying,
> >>
> >> I see that the use case hrefmean-vs-qpe.conf reads Gempak data as
> input. I
> >> thought I had renamed all of the use cases that use Gempak to
contain
> >> 'gempak' somewhere in the title, but it appears that I missed
this one.
> To
> >> run it, you will need to have GempakToCF available on your
computer. If
> you
> >> have it, you can set GEMPAKTOCF_BIN to the directory that
contains the
> >> files for the app. If you don't have it, then I would suggest
picking a
> >> different use case to run. It is possible I apologize for the
confusion.
> >>
> >> On Thu, May 30, 2019 at 2:38 PM Ying Lin via RT
<met_help at ucar.edu>
> wrote:
> >>
> >>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789 >
> >>>
> >>> Hi George, i was writing to you when your email below came in
... here
> >>> it is:
> >>>
> >>> Thanks very much! In my system.conf.wcoss, I changed
> >>> WGRIB2 = $WGRIB2
> >>> to
> >>> WGRIB2 = {ENV[WGRIB2]}
> >>> and that seems to get rid of the $WGRIB2 complaint.
> >>>
> >>> Now I'm getting a new err msg:
> >>> metplus.PcpCombine (config_wrapper.py:93) ERROR: Requested
conf
> >>> [exe] GEMPAKTOCF_CLASSPATH was not set in config file
> >>>
> >>> I see that in my system.conf.wcoss there was this commented out
line:
> >>> #GEMPAKTOCF_BIN = /d1/mccabe/bin/GempakToCF
> >>>
> >>> What should I set GEMPAKTOCF_BIN [and/or _CLASSPATH] to? Or is
there
> an
> >>> option to turn the GEMPAKTOCF off, If I won't be using this
function?
> >>>
> >>> Thanks again -
> >>>
> >>> Ying
> >>>
> >>> On 5/30/19 4:32 PM, George McCabe via RT wrote:
> >>>> The reason you didn't have this issue previously is your use
case
> likely
> >>>> didn't need to use any of those exes. I added a function to
check and
> >>> add
> >>>> all of the non-MET executables to all of the MET wrappers in
case they
> >>> are
> >>>> needed, but I realized later than they are not necessary for
most of
> the
> >>>> cases. However, they may become required for other cases so we
may
> leave
> >>>> the requirement to set them correctly to run METplus.
> >>>>
> >>>> On Thu, May 30, 2019 at 1:36 PM George McCabe <mccabe at ucar.edu>
> wrote:
> >>>>
> >>>>> Hi Ying,
> >>>>>
> >>>>> To reference an environment variable in the METplus config
files, you
> >>> need
> >>>>> to use this syntax:
> >>>>>
> >>>>> WGRIB2 = {ENV[WGRIB2]}
> >>>>>
> >>>>> Since you have set WGRIB2 environment variable, this should
set the
> >>>>> METplus config variable correctly. You may also be able to set
> WGRIB2 =
> >>>>> wgrib2 in the METplus config file instead. If the exe is in
the path
> >>> of the
> >>>>> user running METplus, it will find the app properly. Let me
know if
> >>> that
> >>>>> doesn't work.
> >>>>>
> >>>>> On Thu, May 30, 2019 at 11:41 AM Ying Lin via RT
<met_help at ucar.edu>
> >>>>> wrote:
> >>>>>
> >>>>>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789
>
> >>>>>>
> >>>>>> Hi George,
> >>>>>>
> >>>>>> Thanks for following up! The sample case in
> >>>>>> parm/use_cases/grid_to_grid/examples/precip_continuous.conf
seems to
> >>>>>> "subtract" correctly now (it was off by 24h in v2.0.4). The
> >>>>>> precip_continuous.conf only deals with the PcpCombine step
though.
> >>>>>> When I added the regrid_data_plane step to test a fv3gfs
verif on
> >>> G212,
> >>>>>> I got this err msg:
> >>>>>> metplus.GridStat (config_wrapper.py:144) ERROR:
Executable
> >>> WGRIB2
> >>>>>> does not exist at $WGRIB2
> >>>>>> I thought there must have been a problem in my model config
for the
> >>> v2.1
> >>>>>> (found several actually) but when I run the use case at
> >>>>>> parm/use_cases/qpf/examples/hrefmean-vs-qpe.conf the same
thing
> >>> happened.
> >>>>>> I'm attaching three files to this email:
> >>>>>>
> >>>>>> 1. test_hrefmean.ksh: script to run this use case
> >>>>>> 2. hrefmean.out: output for running the above script. You
can
> see
> >>> that
> >>>>>> after loading model grid_util (from ".
~/dot.for.metplus)
> there
> >>> was
> >>>>>> "WGRIB2=/nwprod2/grib_util.v1.0.0/exec/wgrib2
> >>>>>> export WGRIB2"; and at the end there is the err msg
> >>>>>> "metplus.GridStat (config_wrapper.py:144) ERROR:
Executable
> >>> WGRIB2
> >>>>>> does not exist at $WGRIB2"
> >>>>>> 3. metplus_final.conf from the above run
> >>>>>> 4. my "dot.for.metplus" file invoked by test_hrefmean.ksh.
I
> >>> verified
> >>>>>> manually - after "module load grib_util", $WGRIB2 does
point
> to
> >>>>>> /nwprod2/grib_util.v1.0.0/exec/wgrib2
> >>>>>> 5. my system.conf.wcoss invoked by test_hrefmean.ksh.
> >>>>>>
> >>>>>> I looked back at run output from METplus-2.0.4, there was
also
> >>>>>> WGRIB2=/nwprod2/grib_util.v1.0.0/exec/wgrib2 and "export",
and there
> >>>>>> wasn't a complain about "Executable WGRIB2 does not exist at
> $WGRIB2".
> >>>>>>
> >>>>>> Thanks again -
> >>>>>>
> >>>>>> Ying
> >>>>>>
> >>>>>>
> >>>>>> On 5/29/19 1:53 PM, George McCabe via RT wrote:
> >>>>>>> Hi Ying,
> >>>>>>>
> >>>>>>> I am just following up on this ticket. Have you tried
running your
> >>> case
> >>>>>> on METplus v2.1? Let me know if you run into any other
issues.
> >>>>>>> Thanks,
> >>>>>>> George
> >>>>>>>
> >>>>>>> On Thu Apr 18 19:15:58 2019, mccabe wrote:
> >>>>>>>> Hi Ying,
> >>>>>>>>
> >>>>>>>> It looks like the bug with subtract mode is in v2.0.4. It
will be
> >>>>>>>> fixed in
> >>>>>>>> the next release. You can look at the develop branch on
GitHub to
> >>> see
> >>>>>>>> what
> >>>>>>>> has changed in the subtract function. Have a good weekend!
> >>>>>>>>
> >>>>>>>> On Thu, Apr 18, 2019, 5:09 PM Ying Lin via RT
<met_help at ucar.edu>
> >>>>>>>> wrote:
> >>>>>>>>
> >>>>>>>>> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789 >
> >>>>>>>>>
> >>>>>>>>> Hi George,
> >>>>>>>>>
> >>>>>>>>> Thank you - you're right, replacing that first
> >>>>>>>>> FCST_GRID_STAT_INPUT_TEMPLATE with
> FCST_PCP_COMBINE_INPUT_TEMPLATE
> >>>>>>>>> got
> >>>>>>>>> me past the err msg! There're still some old stuff with
the way
> >>> the
> >>>>>>>>> summing was done - similar to the odd result I mentioned
in the
> >>> last
> >>>>>>>>> email with the 2.0.4 precip_continuous.conf. I'll study
the new
> >>>>>>>>> precip_continuous.conf in the github and follow up with
you next
> >>>>>>>>> week.
> >>>>>>>>>
> >>>>>>>>> Thanks for your help and patience today. Have a great
weekend -
> >>>>>>>>>
> >>>>>>>>> Ying
> >>>>>>>>>
> >>>>>>>>> On 4/18/19 4:25 PM, George McCabe via RT wrote:
> >>>>>>>>>> Hi Ying,
> >>>>>>>>>>
> >>>>>>>>>> I am sorry, I was incorrect that the fixed
precip_continuous
> >>> config
> >>>>>>>>>> file
> >>>>>>>>>> was updated in 2.0.4. It looks like the fix was made
after that
> >>>>>>>>>> release.
> >>>>>>>>> It
> >>>>>>>>>> will be in the newest release. You can see the updated
file on
> >>>>>>>>>> GitHub in
> >>>>>>>>>> the develop branch here:
> >>>>>>>>>>
> >>>>>>>>>>
> >>>
>
https://github.com/NCAR/METplus/blob/develop/parm/use_cases/grid_to_grid/examples/precip_continuous.conf
> >>>>>>>>>> For your use case, it looks like you don't have
> >>>>>>>>>> FCST_PCP_COMBINE_INPUT_TEMPLATE set and you have 2 for
> grid_stat:
> >>>>>>>>>>
> >>>>>>>>>> FCST_GRID_STAT_INPUT_TEMPLATE =
> >>>>>>>>>>
> >>>
>
gfs.{init?fmt=%Y%m%d}/{init?fmt=%H}/gfs.t{init?fmt=%H}z.pgrb2.0p25.f{lead?fmt=%HHH}
> >>>>>>>>>> FCST_PCP_COMBINE_OUTPUT_TEMPLATE =
> >>>>>>>>>> gfs.{valid?fmt=%Y%m%d%H}_A{level?fmt=%HHH}h
> >>>>>>>>>> FCST_GRID_STAT_INPUT_TEMPLATE =
> >>>>>>>>> gfs.{valid?fmt=%Y%m%d%H}_A{level?fmt=%HHH}h
> >>>>>>>>>> Fixing this may solve your issue. Please let me know if
it does
> >>>>>>>>>> not. I
> >>>>>>>>> have
> >>>>>>>>>> to catch a flight this evening so I am going to stop
working
> very
> >>>>>>>>>> soon,
> >>>>>>>>> but
> >>>>>>>>>> if you reply very soon I may still be able to help you
before
> the
> >>>>>>>>> weekend.
> >>>>>>>>>> Thanks,
> >>>>>>>>>> George
> >>>>>>>>>>
> >>>>>>>>>> On Thu, Apr 18, 2019 at 8:13 PM Ying Lin via RT <
> >>> met_help at ucar.edu>
> >>>>>>>>> wrote:
> >>>>>>>>>>> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789
> >
> >>>>>>>>>>>
> >>>>>>>>>>> Hi George,
> >>>>>>>>>>>
> >>>>>>>>>>> Thank you for the quick reply. I was looking at
> >>>>>>>>>>>
> >>>>>>>>>
/gpfs/gd1/emc/global/noscrub/Julie.Prestopnik/METplus/METplus-
> >>>>>>>>>
> 2.0.4/parm/use_cases/grid_to_grid/examples/precip_continuous.conf,
> >>>>>>>>>>> which has VALID_BEG = 2018030900, VALID_END =
2018030900. Date
> >>>>>>>>>>> stamp is
> >>>>>>>>>>> Jan 30. Is there a "newer version of v2.0.4" available
on
> wcoss?
> >>>>>>>>>>> From
> >>>>>>>>>>> the above config it's not clear to me how METplus knows
to
> >>>>>>>>>>> subtract the
> >>>>>>>>>>> gfs.*.f024 from the gfs.*.f048.
> >>>>>>>>>>>
> >>>>>>>>>>> If you have time today, could you also take a look why
my
> simple
> >>>>>>>>>>> test
> >>>>>>>>>>> (conf set up to be as close to precip_continuous.conf as
> possible
> >>>>>>>>>>> -
> >>>>>>>>>>>
> >>>
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf
> >>>>>> )
> >>>>>>>>>>> led to err msg in
> >>>>>>>>>>>
> >>>
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs.out.test5
> >>>>>>>>> ?
> >>>>>>>>>>> It seems that I corrected the problems you pointed out
in your
> >>>>>>>>>>> reply
> >>>>>>>>>>> this morning, but the err msg still seems to be the same
as
> >>>>>>>>>>> yesterday's.
> >>>>>>>>>>>
> >>>>>>>>>>> If you don't have time today, next Monday is fine too.
Have a
> >>>>>>>>>>> good
> >>>>>>>>>>> holiday weekend.
> >>>>>>>>>>>
> >>>>>>>>>>> Ying
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>> On 4/18/19 3:51 PM, George McCabe via RT wrote:
> >>>>>>>>>>>> Hi Ying,
> >>>>>>>>>>>>
> >>>>>>>>>>>> 1) The config file I see has a valid time of 2018031000
and 48
> >>> hr
> >>>>>>>>>>> forecast
> >>>>>>>>>>>> lead. You may be using an old version of METplus that
had this
> >>>>>>>>>>>> set up
> >>>>>>>>>>>> incorrectly. METplus version 2.0.4 should be correct.
> >>>>>>>>>>>>
> >>>>>>>>>>>> 2) pcp_combine -subtract does not actually use the
FCST_LEVEL
> >>>>>>>>>>>> variable,
> >>>>>>>>>>> so
> >>>>>>>>>>>> that should be removed from the use case. The code
takes the
> >>>>>>>>>>>> valid time
> >>>>>>>>>>> and
> >>>>>>>>>>>> subtracts the accumulation.
> >>>>>>>>>>>>
> >>>>>>>>>>>> 3) In the upcoming version of METplus (in a few weeks!)
has a
> >>>>>>>>>>>> good
> >>>>>>>>> number
> >>>>>>>>>>>> of config changes from the previous versions. For
example,
> >>>>>>>>>>>> FCST_LEVEL
> >>>>>>>>>>> will
> >>>>>>>>>>>> now be FCST_PCP_COMBINE_INPUT_LEVEL. These changes are
> designed
> >>>>>>>>>>>> to make
> >>>>>>>>>>> the
> >>>>>>>>>>>> variables more clear to what they actually do. In the
new
> >>>>>>>>>>>> version,
> >>>>>>>>> there
> >>>>>>>>>>> is
> >>>>>>>>>>>> now FCST_PCP_COMBINE_METHOD and OBS_PCP_COMBINE_METHOD,
so you
> >>>>>>>>>>>> will be
> >>>>>>>>>>> able
> >>>>>>>>>>>> to specify a different method for your forecast and
analysis
> >>>>>>>>>>>> data.
> >>>>>>>>>>>>
> >>>>>>>>>>>> FCST_PCP_COMBINE_RUN = True
> >>>>>>>>>>>> FCST_PCP_COMBINE_METHOD = SUBTRACT
> >>>>>>>>>>>>
> >>>>>>>>>>>> OBS_PCP_COMBINE_RUN = True
> >>>>>>>>>>>> OBS_PCP_COMBINE_METHOD = SUM
> >>>>>>>>>>>>
> >>>>>>>>>>>> This functionality has not been thoroughly tested as we
> haven't
> >>>>>>>>>>>> had any
> >>>>>>>>>>>> example use cases that need to do this so far, so
please let
> me
> >>>>>>>>>>>> know if
> >>>>>>>>>>> it
> >>>>>>>>>>>> does not work the way you expect and I will fix it.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Just to let you know, I will be out of the office
tomorrow
> >>>>>>>>>>>> (Friday)
> >>>>>>>>> and I
> >>>>>>>>>>>> wont have access to email. Feel free to send any
follow-up
> >>> emails
> >>>>>>>>>>>> to me
> >>>>>>>>>>> and
> >>>>>>>>>>>> I will get back to you on Monday.
> >>>>>>>>>>>>
> >>>>>>>>>>>> Thanks,
> >>>>>>>>>>>> George
> >>>>>>>>>>>>
> >>>>>>>>>>>> On Thu, Apr 18, 2019 at 7:38 PM Ying Lin via RT
> >>>>>>>>>>>> <met_help at ucar.edu>
> >>>>>>>>>>> wrote:
> >>>>>>>>>>>>> <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789
> >>>>>>>>>>>>> Thank you very much George, this is very helpful.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> About the example in
> >>>>>>>>>>>>>
> >>> parm/use_cases/grid_to_grid/examples/precip_continuous.conf
> >>>>>>>>>>>>> I tried running it and it seems to have run through
the
> >>>>>>>>>>>>> pcp_combine
> >>>>>>>>> step
> >>>>>>>>>>>>> fine: it did this:
> >>>>>>>>>>>>>
> /global/noscrub/Julie.Prestopnik/met/8.0/bin/pcp_combine
> >>>>>>>>> -subtract
> >>>>>>>>> /global/noscrub/Julie.Prestopnik/METplus/METplus-
> >>>>>>>>>
> >>>
>
2.0.4_sample_data/grid_to_grid/prfv3rt1/20180308/gfs.t00z.pgrb.1p00.f048
> >>>>>>>>>>>>> 48
> >>>>>>>>>>>>>
> >>>>>>>>> /global/noscrub/Julie.Prestopnik/METplus/METplus-
> >>>>>>>>>
> >>>
>
2.0.4_sample_data/grid_to_grid/prfv3rt1/20180308/gfs.t00z.pgrb.1p00.f024
> >>>>>>>>>>>>> 24
> >>>>>>>>>>>>>
> >>>
/meso/noscrub/Ying.Lin/metplus.out/gfs/bucket/gfs.2018030900_A024h
> >>>>>>>>>>>>> Several questions about the example:
> >>>>>>>>>>>>> 1) The precip_continuous.conf has
> >>>>>>>>>>>>> LEAD_SEQ = 24
> >>>>>>>>>>>>> I thought it meant forecast lead of 24h, e.g. look for
00-24h
> >>>>>>>>>>>>> forecast
> >>>>>>>>>>>>> valid at 2018030900 (given in the config). If so,
why
> doesn't
> >>>>>>>>>>>>> it
> >>>>>>>>>>>>> simply use gfs.t00z.pgrb.1p00.f024, which already have
the
> 00-
> >>>>>>>>>>>>> 24h
> >>>>>>>>>>>>> forecast? If it subtracts gfs.t00z.pgrb.1p00.f024
from
> >>>>>>>>>>>>> gfs.t00z.pgrb.1p00.f048, the resulting file would have
24-48h
> >>>>>>>>>>>>> forecast
> >>>>>>>>>>>>> valid at 2018031000. I did
> >>>>>>>>>>>>> ncdump -h
> >>>>>>>>>>>>>
> >>>
/meso/noscrub/Ying.Lin/metplus.out/gfs/bucket/gfs.2018030900_A024h
> >>>>>>>>>>>>> and got
> >>>>>>>>>>>>> APCP_24:init_time = "20180308_000000" ;
> >>>>>>>>>>>>> APCP_24:init_time_ut = "1520467200" ;
> >>>>>>>>>>>>> APCP_24:valid_time = "20180310_000000" ;
> >>>>>>>>>>>>> APCP_24:valid_time_ut = "1520640000" ;
> >>>>>>>>>>>>> APCP_24:accum_time = "240000" ;
> >>>>>>>>>>>>> it seems to mean that the file contains APCP_24h with
initial
> >>>>>>>>>>>>> time of
> >>>>>>>>>>>>> 00z 03/08 and the valid time is 00Z 03/10 (that'll be
> APCP_48h
> >>>>>>>>>>>>> instead
> >>>>>>>>>>>>> of 24h?)
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 2) The precip_continuous.conf has
> >>>>>>>>>>>>> FCST_LEVEL = 6
> >>>>>>>>>>>>> I thought FCST_LEVEL is the forecast bucket length.
In the
> >>>>>>>>>>>>> sample GFS
> >>>>>>>>>>>>> files, there isn't a 6-hourly bucket -
> gfs.t00z.pgrb.1p00.f024
> >>>>>>>>> contains
> >>>>>>>>>>>>> 00-24h accumulation and gfs.t00z.pgrb.1p00.f048
contains
> 00-48h
> >>>>>>>>>>>>> accumulation. Does FCST_LEVEL mean something other
than the
> >>>>>>>>>>>>> forecast
> >>>>>>>>>>>>> bucket length?
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 3) precip_continuous.conf has
> >>>>>>>>>>>>> PCP_COMBINE_METHOD = SUBTRACT
> >>>>>>>>>>>>> which should work for the example since the verifying
> analysis
> >>>>>>>>>>>>> is the
> >>>>>>>>>>>>> 24h CCPA file. But what if I need to use 'SUBTRACT'
for the
> >>>>>>>>>>>>> forecast
> >>>>>>>>>>>>> field but 'SUM' for the analysis field (e.g. summing
up 6h
> >>> CCPAs
> >>>>>>>>>>>>> into
> >>>>>>>>> a
> >>>>>>>>>>>>> 24h total)? At one point I tried
> >>>>>>>>>>>>> PCP_COMBINE_METHOD = SUBTRACT, SUM
> >>>>>>>>>>>>> and got
> >>>>>>>>>>>>> ERROR: Invalid PCP_COMBINE_METHOD specified
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> After several failed attempts getting my own config
file to
> >>>>>>>>>>>>> work, I
> >>>>>>>>>>>>> starting from precip_continuous.conf (which worked
through
> the
> >>>>>>>>>>>>> PcpCombine step; I think that's the only step it runs
since
> the
> >>>>>>>>>>>>> config
> >>>>>>>>>>>>> has only PcpCombine in the PROCESS_LIST) and tried to
set a
> >>>>>>>>>>>>> config
> >>>>>>>>> file
> >>>>>>>>>>>>> as close to precip_continuous.conf as possible
> >>>>>>>>>>>>>
> >>>
>
(/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/parm/ylin_config/models/fv3gfs_24h.conf,
> >>>>>>>>>>>>> also uploaded to
> >>>>>>>>>>>>>
> >>>
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf
> >>>>>>>>>>> ).
> >>>>>>>>>>>>> The resulting err msg:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> 04/18 18:35:33.912 jlog (master_metplus.py:217)
CRITICAL:
> >>>>>>>>>>>>> master_metplus failed: ''
> >>>>>>>>>>>>> Traceback (most recent call last):
> >>>>>>>>>>>>> File
"/meso/save/Ying.Lin/metplus/ush/master_metplus.py",
> line
> >>>>>>>>> 213,
> >>>>>>>>>>>>> in <module>
> >>>>>>>>>>>>> main()
> >>>>>>>>>>>>> File
> >>> "/meso/save/Ying.Lin/metplus/ush/master_metplus.py",
> >>>>>>>>>>>>> line
> >>>>>>>>> 170,
> >>>>>>>>>>>>> in main
> >>>>>>>>>>>>> process.run_at_time(-1, run_time)
> >>>>>>>>>>>>> File
> >>>>>>>>>>>>>
> >>>
>
"/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/ush/reformat_gridded_wrapper.py",
> >>>>>>>>>>>>> line 80, in run_at_time
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Full output file:
> >>>>>>>>>>>>>
> >>>
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs.out.test5
> >>>>>>>>>>> ;
> >>>>>>>>>>>>> metplus_final.conf:
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/metplus_final.conf
> >>>>>>>>>>>>> The quoted err msg above appears to the the same as in
my
> email
> >>>>>>>>>>>>> yesterday, so there must still be something wrong with
the
> >>>>>>>>>>>>> format of
> >>>>>>>>> my
> >>>>>>>>>>>>> config, though I think I eliminated the problems you
pointed
> >>> out
> >>>>>>>>>>>>> in
> >>>>>>>>> your
> >>>>>>>>>>>>> reply. There wouldn't be a ccpa_conus_1.0d_* of the
valid
> day
> >>> I
> >>>>>>>>>>>>> specified, but I think the processing failed way
before that
> >>>>>>>>>>>>> point.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Thank you again -
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> Ying
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>> On 4/17/19 3:32 PM, George McCabe via RT wrote:
> >>>>>>>>>>>>>> Hi Ying,
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> You will need to set FCST_PCP_COMBINE_METHOD =
SUBTRACT to
> >>>>>>>>>>>>>> config
> >>>>>>>>>>> METplus
> >>>>>>>>>>>>>> to subtract accumulations files to get the
accumulation you
> >>>>>>>>>>>>>> desire.
> >>>>>>>>> The
> >>>>>>>>>>>>>> following use case has an example of this
configuration:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>>
parm/use_cases/grid_to_grid/examples/precip_continous.conf
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> I also noticed that your config file has the
following items
> >>> in
> >>>>>>>>>>>>>> the
> >>>>>>>>>>>>>> [config] section:
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> FCST_PCP_COMBINE_INPUT_DIR =
> /gpfs/dell1/nco/ops/com/gfs/para
> >>>>>>>>>>>>>> FCST_PCP_COMBINE_OUTPUT_DIR =
{OUTPUT_BASE}/qpf_grib/bucket
> >>>>>>>>>>>>>> FCST_GRID_STAT_INPUT_DIR =
{OUTPUT_BASE}/qpf_grib/bucket
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> You should move these to the [dir] section.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Please let me know if you continue to have issues.
The error
> >>>>>>>>>>>>>> message
> >>>>>>>>>>> you
> >>>>>>>>>>>>>> provided has more information at the bottom that was
not
> >>>>>>>>>>>>>> included.
> >>>>>>>>> The
> >>>>>>>>>>>>> next
> >>>>>>>>>>>>>> release of METplus contains more extensive error
checking so
> >>>>>>>>>>>>>> the
> >>>>>>>>> error
> >>>>>>>>>>>>>> message from that version should be much more
helpful.
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> Thanks,
> >>>>>>>>>>>>>> George
> >>>>>>>>>>>>>>
> >>>>>>>>>>>>>> On Wed, Apr 17, 2019 at 12:43 PM Ying Lin via RT
> >>>>>>>>>>>>>> <met_help at ucar.edu>
> >>>>>>>>>>>>> wrote:
> >>>>>>>>>>>>>>> Wed Apr 17 12:43:02 2019: Request 89789 was acted
upon.
> >>>>>>>>>>>>>>> Transaction: Ticket created by ying.lin at noaa.gov
> >>>>>>>>>>>>>>> Queue: met_help
> >>>>>>>>>>>>>>> Subject: Using FCST_PCP_COMBINE in
METplus
> >>>>>>>>>>>>>>> Owner: Nobody
> >>>>>>>>>>>>>>> Requestors: ying.lin at noaa.gov
> >>>>>>>>>>>>>>> Status: new
> >>>>>>>>>>>>>>> Ticket <URL:
> >>>>>>>>>>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=89789
> >>>>>>>>>>>>>>> Hello. I haven't been able to get the
FCST_PCP_COMBINE
> >>>>>>>>>>>>>>> feature to
> >>>>>>>>>>> work
> >>>>>>>>>>>>>>> in METplus, running out of ideas and hope you can
help.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I've put some examples in
> >>>>>>>>>>>>>>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/
> >>>>>>>>>>>>>>> (current
> >>>>>>>>>>>>> test
> >>>>>>>>>>>>>>> directory:
> >>>>>>>>>>>>>>> gyre:/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/test).
> >>>>>>>>>>>>>>> test_fv3gfs_24h.ksh is the simple test script. The
goal is
> >>> to
> >>>>>>>>>>> compute a
> >>>>>>>>>>>>>>> 24h verification ending at 12Z 20190415.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> 'test1' did not need FCST_PCP_COMBINE - the config
file
> >>>>>>>>>>>>>>> fv3gfs_24h.conf.test1 simply had LEAD_SEQ set to
'24',
> >>>>>>>>>>>>>>> FCST_PCP_COMBINE_RUN was not set. The FV3GFS had
two
> precip
> >>>>>>>>> buckets,
> >>>>>>>>>>> 6h
> >>>>>>>>>>>>>>> and continuously-accumulating, and METplus easily
found the
> >>>>>>>>>>>>>>> 24h
> >>>>>>>>>>> forecast
> >>>>>>>>>>>>>>> accumulation and went on with the grid_stat.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> 'test2' had 'LEAD_SEQ = 36', and I added the
following in
> the
> >>>>>>>>>>>>>>> config
> >>>>>>>>>>>>>>> (
> >>>>>>>>>>>>>>>
> >>>
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test2
> >>>>>>>>>>>>>>> ):
> >>>>>>>>>>>>>>> FCST_PCP_COMBINE_RUN = True
> >>>>>>>>>>>>>>> FCST_PCP_COMBINE_INPUT_DATATYPE = GRIB
> >>>>>>>>>>>>>>> FCST_PCP_COMBINE_INPUT_DIR =
> >>>>>>>>> /gpfs/dell1/nco/ops/com/gfs/para
> >>>>>>>>>>>>>>> FCST_PCP_COMBINE_OUTPUT_DIR =
{OUTPUT_BASE}/qpf_grib/bucket
> >>>>>>>>>>>>>>> FCST_GRID_STAT_INPUT_DIR =
{OUTPUT_BASE}/qpf_grib/bucket
> >>>>>>>>>>>>>>> FCST_MAX_FORECAST = 36
> >>>>>>>>>>>>>>> FCST_INIT_INTERVAL = 6
> >>>>>>>>>>>>>>> FCST_LEVEL = 36
> >>>>>>>>>>>>>>> Here the goal was to see if I could get METplus to
subtract
> >>>>>>>>>>>>>>> the
> >>>>>>>>> 00-12h
> >>>>>>>>>>>>>>> accumulation from gfs.t00z.pgrb2.0p25.f012 from the
00-36h
> >>>>>>>>>>> accumulation
> >>>>>>>>>>>>>>> in gfs.t00z.pgrb2.0p25.f036. Thinking that
FCST_LEVEL is
> the
> >>>>>>>>> forecast
> >>>>>>>>>>>>>>> bucket length, I figured setting it to '36' and
> artificially
> >>>>>>>>>>> specifying
> >>>>>>>>>>>>>>> FCST_MAX_FORECAST = 36 (instead of the full 384h)
should
> work
> >>>>>>>>>>>>>>> for
> >>>>>>>>> this
> >>>>>>>>>>>>>>> limited test where LEAD_SEQ is set to 36 (is there a
way to
> >>>>>>>>>>>>>>> specify
> >>>>>>>>>>> that
> >>>>>>>>>>>>>>> the bucket never gets emptied?). It failed with the
follow
> >>>>>>>>>>>>>>> err msg
> >>>>>>>>>>>>>>> (full message in
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test2
> >>>>>>>>>>>>>>> ):
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> 04/16 20:01:42.754 jlog (master_metplus.py:217)
CRITICAL:
> >>>>>>>>>>>>>>> master_metplus failed: ''
> >>>>>>>>>>>>>>> Traceback (most recent call last):
> >>>>>>>>>>>>>>> File
> >>>>>>>>>>>>>>> "/meso/save/Ying.Lin/metplus/ush/master_metplus.py",
line
> >>>>>>>>>>> 213,
> >>>>>>>>>>>>>>> in <module>
> >>>>>>>>>>>>>>> main()
> >>>>>>>>>>>>>>> File
> >>>>>>>>>>>>>>> "/meso/save/Ying.Lin/metplus/ush/master_metplus.py",
line
> >>>>>>>>>>> 170,
> >>>>>>>>>>>>>>> in main
> >>>>>>>>>>>>>>> process.run_at_time(-1, run_time)
> >>>>>>>>>>>>>>> File
> >>>>>>>>>>>>>>>
> >>>
>
"/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/ush/reformat_gridded_wrapper.py",
> >>>>>>>>>>>>>>> line 80, in run_at_time
> >>>>>>>>>>>>>>> self.run_at_time_once(task_info,
var_info, rl)
> >>>>>>>>>>>>>>> File
> >>>>>>>>>>>>>>>
> >>>
"/gpfs/gd1/emc/meso/save/Ying.Lin/metplus/ush/pcp_combine_wrapper.py",
> >>>>>>>>>>>>>>> line 419, in run_at_time_once
> >>>>>>>>>>>>>>> ....
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> 'test3' is an attempt from a different angle,
thinking that
> >>> if
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>>>>>> 'continuously running' bucket didn't work, maybe the
> 6-hourly
> >>>>>>>>>>>>>>> bucket
> >>>>>>>>>>>>>>> would. In
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test3
> >>>>>>>>>>>>> ,
> >>>>>>>>>>>>>>> I changed LEAD_SEQ to 24, and set FCST_LEVEL = 6.
It
> >>> produced
> >>>>>>>>>>>>>>> err
> >>>>>>>>>>> msgs
> >>>>>>>>>>>>>>> (
> >>>>>>>>>>>>>>>
> >>>
>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/fv3gfs_24h.conf.test3
> >>>>>>>>>>>>> )
> >>>>>>>>>>>>>>> that appeared to be the same as that from 'test2'.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Earlier on I tried using the FCST_PCP_COMBINE
feature with
> >>> the
> >>>>>>>>>>>>>>> NAM
> >>>>>>>>>>>>>>> 'CONUSNEST' output (with simple regular 6-hourly
bucket)
> and
> >>>>>>>>>>>>>>> got the
> >>>>>>>>>>>>>>> same err msg, so I don't think FV3GFS's dual-bucket
system
> is
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>> cause.
> >>>>>>>>>>>>>>> Also posted on
> >>>>>>>>>>>>>>>
https://ftp.emc.ncep.noaa.gov/mmb/precip/METplus.question/
> >>> are
> >>>>>>>>>>>>>>> the
> >>>>>>>>>>> two
> >>>>>>>>>>>>>>> metplus_final.conf files from both 'test2' and
'test3'.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> I found in
> >>>>>>>>>>>>>>>
> >>>>>>>>> /global/noscrub/Julie.Prestopnik/METplus/METplus-
> >>>>>>>>> 2.0.4/parm/use_cases/qpf/examples/
> >>>>>>>>>>>>>>> that hrefmean-vs-mrms-qpe.conf and nationalblend-vs-
mrms-
> >>>>>>>>>>>>>>> qpe.conf
> >>>>>>>>> had
> >>>>>>>>>>>>>>> "FCST_PCP_COMBINE_RUN = True", that's where I
borrowed the
> >>>>>>>>>>>>>>> FCST_PCP_COMBINE_* set up in my
fv3gfs_24h.conf.test2 and
> >>>>>>>>>>>>>>> fv3gfs_24h.conf.test3. Haven't been able to run the
> hrefmean
> >>>>>>>>>>>>>>> and
> >>>>>>>>>>>>>>> nationalblend tests myself however - got err msg of
"
> >>>>>>>>>>>>>>> CRITICAL:
> >>>>>>>>>>>>>>> master_metplus failed: No option
'GEMPAKTOCF_CLASSPATH' in
> >>>>>>>>>>>>>>> section:
> >>>>>>>>>>>>>>> 'exe'" from both attempts.
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Thank you for looking into this -
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> Ying
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>> --
> >>>>>>>>>>>>>>> Ying Lin
> >>>>>>>>>>>>>>> NCEP/EMC/Verification, Post-processing and Product
> Generation
> >>>>>>>>>>>>>>> Branch
> >>>>>>>>>>>>>>> NCWCP Cubicle No. 2015
> >>>>>>>>>>>>>>> Ying.Lin at noaa.gov
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>>>>
> >>>>>>>>>>>>> --
> >>>>>>>>>>>>> Ying Lin
> >>>>>>>>>>>>> NCEP/EMC/Verification, Post-processing and Product
Generation
> >>>>>>>>>>>>> Branch
> >>>>>>>>>>>>> NCWCP Cubicle No. 2015
> >>>>>>>>>>>>> Ying.Lin at noaa.gov
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>> --
> >>>>>>>>>>> Ying Lin
> >>>>>>>>>>> NCEP/EMC/Verification, Post-processing and Product
Generation
> >>>>>>>>>>> Branch
> >>>>>>>>>>> NCWCP Cubicle No. 2015
> >>>>>>>>>>> Ying.Lin at noaa.gov
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>> --
> >>>>>>>>> Ying Lin
> >>>>>>>>> NCEP/EMC/Verification, Post-processing and Product
Generation
> >>> Branch
> >>>>>>>>> NCWCP Cubicle No. 2015
> >>>>>>>>> Ying.Lin at noaa.gov
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>> --
> >>>>>> Ying Lin
> >>>>>> NCEP/EMC/Verification, Post-processing and Product Generation
Branch
> >>>>>> NCWCP Cubicle No. 2015
> >>>>>> Ying.Lin at noaa.gov
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>> --
> >>> Ying Lin
> >>> NCEP/EMC/Verification, Post-processing and Product Generation
Branch
> >>> NCWCP Cubicle No. 2015
> >>> Ying.Lin at noaa.gov
> >>>
> >>>
> >>>
> >>>
>
> --
> Ying Lin
> NCEP/EMC/Verification, Post-processing and Product Generation Branch
> NCWCP Cubicle No. 2015
> Ying.Lin at noaa.gov
>
>
>
>
------------------------------------------------
More information about the Met_help
mailing list