[Met_help] [rt.rap.ucar.edu #99429] History for How to set time for hourly 12-24 hours data ?
George McCabe via RT
met_help at ucar.edu
Wed Jun 2 16:06:37 MDT 2021
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hello,
Could you let me know how to set time on processing hourly Obs. data for
lasting 12 -24hours using "PB2NC", and do hourly verification for lasting
12-24 hours using Point-stat?
such as:
VALID_TIME_FMT = %Y%m%d%H
VALID_BEG = 2015071500
VALID_END = 2015071500
VALID_INCREMENT = 1M
LEAD_SEQ = 0
Thanks!
Zhuming
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: How to set time for hourly 12-24 hours data ?
From: George McCabe
Time: Tue Apr 06 16:23:00 2021
Hi Zhuming,
To run the tools hourly you would set the VALID_INCREMENT value for
1H.
Here is an example that runs once per hour for a day from 2015-07-15
0Z to
2017-07-16 0Z:
VALID_TIME_FMT = %Y%m%d%H
VALID_BEG = 2015071500
VALID_END = 2015071600
VALID_INCREMENT = 1H
LEAD_SEQ = 0
Each run time will use the PB2NC_INPUT_TEMPLATE to find files whose
filenames match the time you need.
To read prepBUFR observations for 12 hours, you will want to set
PB2NC_VALID_BEGIN and PB2NC_VALID_END to specify the range around the
valid
time to process. These values depend on what is contained inside your
prepBUFR files. If you want to use data from the valid time of the
filename
backwards 12 hours, you would set:
PB2NC_VALID_BEGIN = {valid?fmt=%Y%m%d_%H?shift=-12H }
PB2NC_VALID_END = {valid?fmt=%Y%m%d_%H}
For a runtime of 2015071512, this would be substituted as:
valid_beg: 20150715_00
valid_end: 20150715_12
If you want to create an output file for each hour in PB2NC you would
need
to make sure that the hour is specified in the PB2NC_OUTPUT_TEMPLATE.
PointStat has similar config variables to set the valid beg and end
range
to use (POINT_STAT_OBS_VALID_BEG and POINT_STAT_OBS_VALID_END).
If your prepBUFR files are separated out into multiple files that need
to
be read into PB2NC, you can set a file window around the valid time to
allow files to be passed into the app. More information on that
functionality is here:
https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#using-
windows-to-find-valid-files
If you are still having trouble getting this configured, you could
send me
the paths of the files you are trying to read and what data is
contained in
them so I can give you suggestions on what to set for the
configuration.
Thanks,
George
On Tue, Apr 6, 2021 at 2:17 PM Julie Prestopnik via RT
<met_help at ucar.edu>
wrote:
>
> Tue Apr 06 14:16:42 2021: Request 99429 was acted upon.
> Transaction: Given to mccabe (George McCabe) by jpresto
> Queue: met_help
> Subject: How to set time for hourly 12-24 hours data ?
> Owner: mccabe
> Requestors: zhuming at ucar.edu
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429 >
>
>
> This transaction appears to have no content
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: How to set time for hourly 12-24 hours data ?
From: zhuming at ucar.edu
Time: Tue Apr 06 17:16:50 2021
Hi George,
Thanks for your reply!
I'm still confused, I need to output hourly obs data to do hourly
verification.
The original GDAS Prepbufr data has 4 files one day, should with 6
hours
obs data in one file.
After "PB2NC", I can see pbs.2015071500.nc (obs time is 00? or 6 times
of
00,01,02,03,04,05?), pbs.2015071506.nc...
my config file : cheyenne:/gpfs/fs1/work/zhuming/METplus4/PB2NC1.conf
and /gpfs/fs1/work/zhuming/METplus4/PointStat1.conf
Thanks,
Zhuming
On Tue, Apr 6, 2021 at 4:23 PM George McCabe via RT
<met_help at ucar.edu>
wrote:
> Hi Zhuming,
>
> To run the tools hourly you would set the VALID_INCREMENT value for
1H.
> Here is an example that runs once per hour for a day from 2015-07-15
0Z to
> 2017-07-16 0Z:
>
> VALID_TIME_FMT = %Y%m%d%H
>
> VALID_BEG = 2015071500
>
> VALID_END = 2015071600
>
> VALID_INCREMENT = 1H
>
> LEAD_SEQ = 0
>
> Each run time will use the PB2NC_INPUT_TEMPLATE to find files whose
> filenames match the time you need.
>
> To read prepBUFR observations for 12 hours, you will want to set
> PB2NC_VALID_BEGIN and PB2NC_VALID_END to specify the range around
the valid
> time to process. These values depend on what is contained inside
your
> prepBUFR files. If you want to use data from the valid time of the
filename
> backwards 12 hours, you would set:
>
> PB2NC_VALID_BEGIN = {valid?fmt=%Y%m%d_%H?shift=-12H }
> PB2NC_VALID_END = {valid?fmt=%Y%m%d_%H}
>
> For a runtime of 2015071512, this would be substituted as:
>
> valid_beg: 20150715_00
> valid_end: 20150715_12
>
> If you want to create an output file for each hour in PB2NC you
would need
> to make sure that the hour is specified in the
PB2NC_OUTPUT_TEMPLATE.
>
> PointStat has similar config variables to set the valid beg and end
range
> to use (POINT_STAT_OBS_VALID_BEG and POINT_STAT_OBS_VALID_END).
>
> If your prepBUFR files are separated out into multiple files that
need to
> be read into PB2NC, you can set a file window around the valid time
to
> allow files to be passed into the app. More information on that
> functionality is here:
>
>
https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#using-
windows-to-find-valid-files
>
> If you are still having trouble getting this configured, you could
send me
> the paths of the files you are trying to read and what data is
contained in
> them so I can give you suggestions on what to set for the
configuration.
>
> Thanks,
> George
>
>
>
> On Tue, Apr 6, 2021 at 2:17 PM Julie Prestopnik via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > Tue Apr 06 14:16:42 2021: Request 99429 was acted upon.
> > Transaction: Given to mccabe (George McCabe) by jpresto
> > Queue: met_help
> > Subject: How to set time for hourly 12-24 hours data ?
> > Owner: mccabe
> > Requestors: zhuming at ucar.edu
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429 >
> >
> >
> > This transaction appears to have no content
> >
>
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>
------------------------------------------------
Subject: How to set time for hourly 12-24 hours data ?
From: George McCabe
Time: Tue Apr 06 18:34:09 2021
Hi Zhuming,
Thanks for the information. You will need to use the truncate keyword
to
find the files, then set the valid beg/end range to 1 hour. This is
what I
changed from your PB2NC1.conf config file:
*PB2NC_VALID_BEGIN = {valid?fmt=%Y%m%d_%H}PB2NC_VALID_END =
{valid?fmt=%Y%m%d_%H?shift=1H}*
*PB2NC_INPUT_TEMPLATE =
{valid?fmt=%Y}/prepbufr.gdas.{valid?fmt=%Y%m%d%H?truncate=21600}.nr*
My config file is on cheyenne here:
/glade/u/home/mccabe/PB2NC1.conf
I just started running it and the output is being written here:
/glade/u/home/mccabe/data/out/pb2nc1
For PointStat, you will want to loop over the same times. I noticed
you
specify message type in the VAR<n>_OPTIONS, but you do not need to do
this
since there is a POINT_STAT_MESSAGE_TYPE variable that defines this.
This
is what I changed in your PointStat1.conf file:
*VALID_END = 2015071600*
*VALID_INCREMENT = 1HPOINT_STAT_OBS_VALID_BEG
=POINT_STAT_OBS_VALID_BEG =*
*POINT_STAT_MESSAGE_TYPE = ADPSFC*
*#OBS_VAR1_OPTIONS = message_type = ["ADPSFC"];#OBS_VAR2_OPTIONS =
message_type = ["ADPSFC"];#OBS_VAR3_OPTIONS = message_type =
["ADPSFC"];#OBS_POINT_STAT_INPUT_DIR =
/glade/scratch/zhuming/metplus4/pb2nc1*
*OBS_POINT_STAT_INPUT_DIR = {OUTPUT_BASE}/pb2nc1*
Or you can view my file in the same directory. I usually try to test
with a
single run time to make sure it is working properly, so I set
VALID_END
back to 2015071500. I ran it and it produced an output file, but there
were
no values besides the header. I checked the log file and saw this:
*DEBUG 2: Processing T_sfc(0,*,*) versus TMP/Z2, for observation type
ADPSFC, over region FULL, for interpolation method BILIN(4), using 0
matched pairs.DEBUG 3: Number of matched pairs = 0DEBUG 3:
Observations
processed = 393373DEBUG 3: Rejected: station id = 0DEBUG 3:
Rejected: obs type = 390528DEBUG 3: Rejected: valid time =
0DEBUG 3: Rejected: bad obs value = 0DEBUG 3: Rejected: off the grid
=
2586DEBUG 3: Rejected: topography = 0DEBUG 3: Rejected: level
mismatch
= 259DEBUG 3: Rejected: quality marker = 0DEBUG 3: Rejected: message
type
= 0DEBUG 3: Rejected: masking region = 0DEBUG 3: Rejected: bad
fcst
value = 0DEBUG 3: Rejected: bad climo mean = 0DEBUG 3: Rejected: bad
climo stdev = 0DEBUG 3: Rejected: duplicates = 0DEBUG 2:DEBUG 2:
--------------------------------------------------------------------------------DEBUG
2:DEBUG 2: Processing u_10m_gr(0,*,*) versus UGRD/Z10, for observation
type
ADPSFC, over region FULL, for interpolation method BILIN(4), using 0
matched pairs.DEBUG 3: Number of matched pairs = 0DEBUG 3:
Observations
processed = 393373*
*DEBUG 3: Rejected: station id = 0DEBUG 3: Rejected: obs type
=
382965DEBUG 3: Rejected: valid time = 0DEBUG 3: Rejected: bad obs
value = 0DEBUG 3: Rejected: off the grid = 9240DEBUG 3: Rejected:
topography = 0DEBUG 3: Rejected: level mismatch = 1168DEBUG 3:
Rejected: quality marker = 0DEBUG 3: Rejected: message type =
0DEBUG 3:
Rejected: masking region = 0DEBUG 3: Rejected: bad fcst value =
0DEBUG 3:
Rejected: bad climo mean = 0DEBUG 3: Rejected: bad climo stdev =
0DEBUG 3:
Rejected: duplicates = 0DEBUG 2:DEBUG 2:
--------------------------------------------------------------------------------DEBUG
2:DEBUG 2: Processing v_10m_gr(0,*,*) versus VGRD/Z10, for observation
type
ADPSFC, over region FULL, for interpolation method BILIN(4), using 0
matched pairs.DEBUG 3: Number of matched pairs = 0DEBUG 3:
Observations
processed = 393373DEBUG 3: Rejected: station id = 0DEBUG 3:
Rejected: obs type = 382965DEBUG 3: Rejected: valid time =
0DEBUG 3: Rejected: bad obs value = 0DEBUG 3: Rejected: off the grid
=
9240DEBUG 3: Rejected: topography = 0DEBUG 3: Rejected: level
mismatch
= 1168DEBUG 3: Rejected: quality marker = 0DEBUG 3: Rejected:
message
type = 0DEBUG 3: Rejected: masking region = 0DEBUG 3: Rejected:
bad
fcst value = 0DEBUG 3: Rejected: bad climo mean = 0DEBUG 3:
Rejected: bad
climo stdev = 0*
*DEBUG 3: Rejected: duplicates = 0DEBUG 2:DEBUG 2:
--------------------------------------------------------------------------------DEBUG
2:DEBUG 1: Output file:
/glade/u/home/mccabe/data/out/point_stat1/point_stat_000000L_20150715_000000V.stat*
This describes the reason why each observation was rejected. Some
observations didn't match the message type you requested, some were
not
within the forecast grid, and some did not match the vertical level.
You
will have to make adjustments to your settings so that the data
matches up
properly. Maybe the grid you are using in the PB2NC config file does
not
match the forecast grid and you are missing values. If that is the
case,
you will have to reconfigure and rerun PB2NC to get output on the
correct
gr5id. Maybe the observation points don't line up with the vertical
level
of the model fields. You can use a range of level values for the
observation data to allow data within that range, i.e. Z0-10.
Let me know if you are still having issues getting this to work and I
can
help out.
Thanks,
George
On Tue, Apr 6, 2021 at 5:17 PM zhuming at ucar.edu via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429 >
>
> Hi George,
>
> Thanks for your reply!
>
> I'm still confused, I need to output hourly obs data to do hourly
> verification.
> The original GDAS Prepbufr data has 4 files one day, should with 6
hours
> obs data in one file.
>
> After "PB2NC", I can see pbs.2015071500.nc (obs time is 00? or 6
times of
> 00,01,02,03,04,05?), pbs.2015071506.nc...
>
> my config file :
cheyenne:/gpfs/fs1/work/zhuming/METplus4/PB2NC1.conf
> and /gpfs/fs1/work/zhuming/METplus4/PointStat1.conf
>
> Thanks,
> Zhuming
>
> On Tue, Apr 6, 2021 at 4:23 PM George McCabe via RT
<met_help at ucar.edu>
> wrote:
>
> > Hi Zhuming,
> >
> > To run the tools hourly you would set the VALID_INCREMENT value
for 1H.
> > Here is an example that runs once per hour for a day from 2015-07-
15 0Z
> to
> > 2017-07-16 0Z:
> >
> > VALID_TIME_FMT = %Y%m%d%H
> >
> > VALID_BEG = 2015071500
> >
> > VALID_END = 2015071600
> >
> > VALID_INCREMENT = 1H
> >
> > LEAD_SEQ = 0
> >
> > Each run time will use the PB2NC_INPUT_TEMPLATE to find files
whose
> > filenames match the time you need.
> >
> > To read prepBUFR observations for 12 hours, you will want to set
> > PB2NC_VALID_BEGIN and PB2NC_VALID_END to specify the range around
the
> valid
> > time to process. These values depend on what is contained inside
your
> > prepBUFR files. If you want to use data from the valid time of the
> filename
> > backwards 12 hours, you would set:
> >
> > PB2NC_VALID_BEGIN = {valid?fmt=%Y%m%d_%H?shift=-12H }
> > PB2NC_VALID_END = {valid?fmt=%Y%m%d_%H}
> >
> > For a runtime of 2015071512, this would be substituted as:
> >
> > valid_beg: 20150715_00
> > valid_end: 20150715_12
> >
> > If you want to create an output file for each hour in PB2NC you
would
> need
> > to make sure that the hour is specified in the
PB2NC_OUTPUT_TEMPLATE.
> >
> > PointStat has similar config variables to set the valid beg and
end range
> > to use (POINT_STAT_OBS_VALID_BEG and POINT_STAT_OBS_VALID_END).
> >
> > If your prepBUFR files are separated out into multiple files that
need to
> > be read into PB2NC, you can set a file window around the valid
time to
> > allow files to be passed into the app. More information on that
> > functionality is here:
> >
> >
>
https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#using-
windows-to-find-valid-files
> >
> > If you are still having trouble getting this configured, you could
send
> me
> > the paths of the files you are trying to read and what data is
contained
> in
> > them so I can give you suggestions on what to set for the
configuration.
> >
> > Thanks,
> > George
> >
> >
> >
> > On Tue, Apr 6, 2021 at 2:17 PM Julie Prestopnik via RT <
> met_help at ucar.edu>
> > wrote:
> >
> > >
> > > Tue Apr 06 14:16:42 2021: Request 99429 was acted upon.
> > > Transaction: Given to mccabe (George McCabe) by jpresto
> > > Queue: met_help
> > > Subject: How to set time for hourly 12-24 hours data ?
> > > Owner: mccabe
> > > Requestors: zhuming at ucar.edu
> > > Status: new
> > > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429
> >
> > >
> > >
> > > This transaction appears to have no content
> > >
> >
> >
> > --
> > George McCabe - Software Engineer III
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > 303-497-2768
> > ---
> > My working day may not be your working day. Please do not feel
obliged to
> > reply to this email outside of your normal working hours.
> >
> >
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: How to set time for hourly 12-24 hours data ?
From: zhuming at ucar.edu
Time: Wed Apr 07 17:31:37 2021
Hi George,
Now makes me confused is why processed obs of several times are
missing.
There has files of time at 00,01,02, but no 03,04,05; has 06,07,08,
but no
09,10,11, has 12,13,14, no 15,16,17; has 18,19,20, but with 21,no
22,23.
>From yours and mines:
ls -l /glade/u/home/mccabe/data/out/pb2nc1/
-rw-r--r-- 1 mccabe ral 10141339 Apr 6 17:52 pbs.2015071500.nc
-rw-r--r-- 1 mccabe ral 5413907 Apr 6 17:56 pbs.2015071501.nc
-rw-r--r-- 1 mccabe ral 4923711 Apr 6 18:00 pbs.2015071502.nc
-rw-r--r-- 1 mccabe ral 5724447 Apr 6 18:14 pbs.2015071506.nc
-rw-r--r-- 1 mccabe ral 5280463 Apr 6 18:18 pbs.2015071507.nc
-rw-r--r-- 1 mccabe ral 4706119 Apr 6 18:22 pbs.2015071508.nc
-rw-r--r-- 1 mccabe ral 9878263 Apr 6 18:36 pbs.2015071512.nc
-rw-r--r-- 1 mccabe ral 5392527 Apr 6 18:40 pbs.2015071513.nc
-rw-r--r-- 1 mccabe ral 96 Apr 6 18:40 pbs.2015071514.nc
-rw-r--r-- 1 mccabe ral 96 Apr 6 18:42 pbs.2015071515.nc
cheyenne3:/glade/scratch/zhuming/metplus4/pb2nc1>l
total 96112
-rw-r--r-- 1 zhuming ncar 11550172 Apr 7 15:11 pbs.2015071500.nc
-rw-r--r-- 1 zhuming ncar 6615956 Apr 7 15:14 pbs.2015071501.nc
-rw-r--r-- 1 zhuming ncar 5984328 Apr 7 15:18 pbs.2015071502.nc
-rw-r--r-- 1 zhuming ncar 6980712 Apr 7 15:33 pbs.2015071506.nc
-rw-r--r-- 1 zhuming ncar 6455824 Apr 7 15:36 pbs.2015071507.nc
-rw-r--r-- 1 zhuming ncar 5749432 Apr 7 15:40 pbs.2015071508.nc
-rw-r--r-- 1 zhuming ncar 11284936 Apr 7 15:55 pbs.2015071512.nc
-rw-r--r-- 1 zhuming ncar 6600888 Apr 7 15:59 pbs.2015071513.nc
-rw-r--r-- 1 zhuming ncar 5868168 Apr 7 16:03 pbs.2015071514.nc
-rw-r--r-- 1 zhuming ncar 7273092 Apr 7 16:18 pbs.2015071518.nc
-rw-r--r-- 1 zhuming ncar 6656344 Apr 7 16:22 pbs.2015071519.nc
-rw-r--r-- 1 zhuming ncar 6109788 Apr 7 16:27 pbs.2015071520.nc
-rw-r--r-- 1 zhuming ncar 58808 Apr 7 16:31 pbs.2015071521.nc
-rw-r--r-- 1 zhuming ncar 11132440 Apr 7 16:43 pbs.2015071600.nc
Thanks,
Zhuming
On Tue, Apr 6, 2021 at 6:34 PM George McCabe via RT
<met_help at ucar.edu>
wrote:
> Hi Zhuming,
>
> Thanks for the information. You will need to use the truncate
keyword to
> find the files, then set the valid beg/end range to 1 hour. This is
what I
> changed from your PB2NC1.conf config file:
>
>
> *PB2NC_VALID_BEGIN = {valid?fmt=%Y%m%d_%H}PB2NC_VALID_END =
> {valid?fmt=%Y%m%d_%H?shift=1H}*
>
> *PB2NC_INPUT_TEMPLATE =
> {valid?fmt=%Y}/prepbufr.gdas.{valid?fmt=%Y%m%d%H?truncate=21600}.nr*
>
> My config file is on cheyenne here:
> /glade/u/home/mccabe/PB2NC1.conf
>
> I just started running it and the output is being written here:
> /glade/u/home/mccabe/data/out/pb2nc1
>
>
> For PointStat, you will want to loop over the same times. I noticed
you
> specify message type in the VAR<n>_OPTIONS, but you do not need to
do this
> since there is a POINT_STAT_MESSAGE_TYPE variable that defines this.
This
> is what I changed in your PointStat1.conf file:
>
> *VALID_END = 2015071600*
>
>
>
> *VALID_INCREMENT = 1HPOINT_STAT_OBS_VALID_BEG
=POINT_STAT_OBS_VALID_BEG =*
>
>
> *POINT_STAT_MESSAGE_TYPE = ADPSFC*
>
>
>
>
>
> *#OBS_VAR1_OPTIONS = message_type = ["ADPSFC"];#OBS_VAR2_OPTIONS =
> message_type = ["ADPSFC"];#OBS_VAR3_OPTIONS = message_type =
> ["ADPSFC"];#OBS_POINT_STAT_INPUT_DIR =
> /glade/scratch/zhuming/metplus4/pb2nc1*
>
> *OBS_POINT_STAT_INPUT_DIR = {OUTPUT_BASE}/pb2nc1*
>
> Or you can view my file in the same directory. I usually try to test
with a
> single run time to make sure it is working properly, so I set
VALID_END
> back to 2015071500. I ran it and it produced an output file, but
there were
> no values besides the header. I checked the log file and saw this:
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> *DEBUG 2: Processing T_sfc(0,*,*) versus TMP/Z2, for observation
type
> ADPSFC, over region FULL, for interpolation method BILIN(4), using 0
> matched pairs.DEBUG 3: Number of matched pairs = 0DEBUG 3:
Observations
> processed = 393373DEBUG 3: Rejected: station id = 0DEBUG 3:
> Rejected: obs type = 390528DEBUG 3: Rejected: valid time
=
> 0DEBUG 3: Rejected: bad obs value = 0DEBUG 3: Rejected: off the
grid =
> 2586DEBUG 3: Rejected: topography = 0DEBUG 3: Rejected: level
mismatch
> = 259DEBUG 3: Rejected: quality marker = 0DEBUG 3: Rejected:
message type
> = 0DEBUG 3: Rejected: masking region = 0DEBUG 3: Rejected: bad
fcst
> value = 0DEBUG 3: Rejected: bad climo mean = 0DEBUG 3: Rejected:
bad
> climo stdev = 0DEBUG 3: Rejected: duplicates = 0DEBUG 2:DEBUG
2:
>
>
--------------------------------------------------------------------------------DEBUG
> 2:DEBUG 2: Processing u_10m_gr(0,*,*) versus UGRD/Z10, for
observation type
> ADPSFC, over region FULL, for interpolation method BILIN(4), using 0
> matched pairs.DEBUG 3: Number of matched pairs = 0DEBUG 3:
Observations
> processed = 393373*
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> *DEBUG 3: Rejected: station id = 0DEBUG 3: Rejected: obs type
=
> 382965DEBUG 3: Rejected: valid time = 0DEBUG 3: Rejected: bad
obs
> value = 0DEBUG 3: Rejected: off the grid = 9240DEBUG 3:
Rejected:
> topography = 0DEBUG 3: Rejected: level mismatch = 1168DEBUG 3:
> Rejected: quality marker = 0DEBUG 3: Rejected: message type =
0DEBUG 3:
> Rejected: masking region = 0DEBUG 3: Rejected: bad fcst value =
0DEBUG 3:
> Rejected: bad climo mean = 0DEBUG 3: Rejected: bad climo stdev =
0DEBUG 3:
> Rejected: duplicates = 0DEBUG 2:DEBUG 2:
>
>
--------------------------------------------------------------------------------DEBUG
> 2:DEBUG 2: Processing v_10m_gr(0,*,*) versus VGRD/Z10, for
observation type
> ADPSFC, over region FULL, for interpolation method BILIN(4), using 0
> matched pairs.DEBUG 3: Number of matched pairs = 0DEBUG 3:
Observations
> processed = 393373DEBUG 3: Rejected: station id = 0DEBUG 3:
> Rejected: obs type = 382965DEBUG 3: Rejected: valid time
=
> 0DEBUG 3: Rejected: bad obs value = 0DEBUG 3: Rejected: off the
grid =
> 9240DEBUG 3: Rejected: topography = 0DEBUG 3: Rejected: level
mismatch
> = 1168DEBUG 3: Rejected: quality marker = 0DEBUG 3: Rejected:
message
> type = 0DEBUG 3: Rejected: masking region = 0DEBUG 3: Rejected:
bad
> fcst value = 0DEBUG 3: Rejected: bad climo mean = 0DEBUG 3:
Rejected: bad
> climo stdev = 0*
>
>
>
>
> *DEBUG 3: Rejected: duplicates = 0DEBUG 2:DEBUG 2:
>
>
--------------------------------------------------------------------------------DEBUG
> 2:DEBUG 1: Output file:
>
>
/glade/u/home/mccabe/data/out/point_stat1/point_stat_000000L_20150715_000000V.stat*
>
>
>
> This describes the reason why each observation was rejected. Some
> observations didn't match the message type you requested, some were
not
> within the forecast grid, and some did not match the vertical level.
You
> will have to make adjustments to your settings so that the data
matches up
> properly. Maybe the grid you are using in the PB2NC config file does
not
> match the forecast grid and you are missing values. If that is the
case,
> you will have to reconfigure and rerun PB2NC to get output on the
correct
> gr5id. Maybe the observation points don't line up with the vertical
level
> of the model fields. You can use a range of level values for the
> observation data to allow data within that range, i.e. Z0-10.
>
> Let me know if you are still having issues getting this to work and
I can
> help out.
>
> Thanks,
> George
>
>
> On Tue, Apr 6, 2021 at 5:17 PM zhuming at ucar.edu via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429 >
> >
> > Hi George,
> >
> > Thanks for your reply!
> >
> > I'm still confused, I need to output hourly obs data to do hourly
> > verification.
> > The original GDAS Prepbufr data has 4 files one day, should with 6
hours
> > obs data in one file.
> >
> > After "PB2NC", I can see pbs.2015071500.nc (obs time is 00? or 6
times
> of
> > 00,01,02,03,04,05?), pbs.2015071506.nc...
> >
> > my config file :
cheyenne:/gpfs/fs1/work/zhuming/METplus4/PB2NC1.conf
> > and /gpfs/fs1/work/zhuming/METplus4/PointStat1.conf
> >
> > Thanks,
> > Zhuming
> >
> > On Tue, Apr 6, 2021 at 4:23 PM George McCabe via RT
<met_help at ucar.edu>
> > wrote:
> >
> > > Hi Zhuming,
> > >
> > > To run the tools hourly you would set the VALID_INCREMENT value
for 1H.
> > > Here is an example that runs once per hour for a day from 2015-
07-15 0Z
> > to
> > > 2017-07-16 0Z:
> > >
> > > VALID_TIME_FMT = %Y%m%d%H
> > >
> > > VALID_BEG = 2015071500
> > >
> > > VALID_END = 2015071600
> > >
> > > VALID_INCREMENT = 1H
> > >
> > > LEAD_SEQ = 0
> > >
> > > Each run time will use the PB2NC_INPUT_TEMPLATE to find files
whose
> > > filenames match the time you need.
> > >
> > > To read prepBUFR observations for 12 hours, you will want to set
> > > PB2NC_VALID_BEGIN and PB2NC_VALID_END to specify the range
around the
> > valid
> > > time to process. These values depend on what is contained inside
your
> > > prepBUFR files. If you want to use data from the valid time of
the
> > filename
> > > backwards 12 hours, you would set:
> > >
> > > PB2NC_VALID_BEGIN = {valid?fmt=%Y%m%d_%H?shift=-12H }
> > > PB2NC_VALID_END = {valid?fmt=%Y%m%d_%H}
> > >
> > > For a runtime of 2015071512, this would be substituted as:
> > >
> > > valid_beg: 20150715_00
> > > valid_end: 20150715_12
> > >
> > > If you want to create an output file for each hour in PB2NC you
would
> > need
> > > to make sure that the hour is specified in the
PB2NC_OUTPUT_TEMPLATE.
> > >
> > > PointStat has similar config variables to set the valid beg and
end
> range
> > > to use (POINT_STAT_OBS_VALID_BEG and POINT_STAT_OBS_VALID_END).
> > >
> > > If your prepBUFR files are separated out into multiple files
that need
> to
> > > be read into PB2NC, you can set a file window around the valid
time to
> > > allow files to be passed into the app. More information on that
> > > functionality is here:
> > >
> > >
> >
>
https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#using-
windows-to-find-valid-files
> > >
> > > If you are still having trouble getting this configured, you
could send
> > me
> > > the paths of the files you are trying to read and what data is
> contained
> > in
> > > them so I can give you suggestions on what to set for the
> configuration.
> > >
> > > Thanks,
> > > George
> > >
> > >
> > >
> > > On Tue, Apr 6, 2021 at 2:17 PM Julie Prestopnik via RT <
> > met_help at ucar.edu>
> > > wrote:
> > >
> > > >
> > > > Tue Apr 06 14:16:42 2021: Request 99429 was acted upon.
> > > > Transaction: Given to mccabe (George McCabe) by jpresto
> > > > Queue: met_help
> > > > Subject: How to set time for hourly 12-24 hours data ?
> > > > Owner: mccabe
> > > > Requestors: zhuming at ucar.edu
> > > > Status: new
> > > > Ticket <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429
> > >
> > > >
> > > >
> > > > This transaction appears to have no content
> > > >
> > >
> > >
> > > --
> > > George McCabe - Software Engineer III
> > > National Center for Atmospheric Research
> > > Research Applications Laboratory
> > > 303-497-2768
> > > ---
> > > My working day may not be your working day. Please do not feel
obliged
> to
> > > reply to this email outside of your normal working hours.
> > >
> > >
> >
> >
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>
------------------------------------------------
Subject: How to set time for hourly 12-24 hours data ?
From: George McCabe
Time: Wed Apr 07 17:42:33 2021
Hi Zhuming,
Looking at the log output, it looks like the data is stored in the
prepBUFR
files differently that you may have thought:
DEBUG 2: PrepBufr Time Center: 20150715_000000
DEBUG 2: Searching Time Window: 20150715_030000 to
20150715_040000
5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90%
95%
100%
DEBUG 2: Total Messages processed = 590836
DEBUG 2: Rejected based on message type = 0
DEBUG 2: Rejected based on station id = 0
DEBUG 2: Rejected based on valid time = 590815
DEBUG 2: Rejected based on masking grid = 20
DEBUG 2: Rejected based on masking polygon = 0
DEBUG 2: Rejected based on elevation = 0
DEBUG 2: Rejected based on pb report type = 1
DEBUG 2: Rejected based on input report type = 0
DEBUG 2: Rejected based on instrument type = 0
DEBUG 2: Rejected based on zero observations = 0
DEBUG 2: Total Messages retained = 0
DEBUG 2: Total observations retained or derived = 0
DEBUG 1: Obs time between *20150714_210000 and 20150715_030000*
WARNING:
WARNING: process_pbfile() -> No PrepBufr messages retained from file:
/glade/collections/rda/data/ds337.0/prepnr/2015/prepbufr.gdas.2015071500.nr
WARNING:
ERROR :
ERROR : write_netcdf_hdr_data() -> No PrepBufr messages retained.
Nothing
to write.
ERROR :
The prepbufr.gdas.2015071500.nr file actually contains data from
20150714_210000 to 20150715_030000, not 20150715_000000 to
20150715_060000.
To get the correct file for each run, you will have to shift the valid
time
before truncating it so that it gets the correct file for each run.
Instead
of setting:
PB2NC_INPUT_TEMPLATE =
{valid?fmt=%Y}/prepbufr.gdas.{valid?fmt=%Y%m%d%H?truncate=21600}.nr
You will have to also shift forward 3 hours, i.e.:
PB2NC_INPUT_TEMPLATE =
{valid?fmt=%Y}/prepbufr.gdas.{valid?fmt=%Y%m%d%H?shift=3H?truncate=21600}.nr
This says to add 3 hours to the current valid time before truncating
to the
nearest 6 hours.
Valid: 20150715 0Z + 3 = 3Z truncate 6H = 0Z
File: prepbufr.gdas.2015071500.nr
Valid: 20150715 1Z + 3 = 4Z truncate 6H = 0Z
File: prepbufr.gdas.2015071500.nr
Valid: 20150715 2Z + 3 = 5Z truncate 6H = 0Z
File: prepbufr.gdas.2015071500.nr
Valid: 20150715 3Z + 3 = 6Z truncate 6H = 6Z
File: prepbufr.gdas.2015071506.nr
Let me know if that doesn't work.
Thanks,
George
On Wed, Apr 7, 2021 at 5:32 PM zhuming at ucar.edu via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429 >
>
> Hi George,
>
> Now makes me confused is why processed obs of several times are
missing.
> There has files of time at 00,01,02, but no 03,04,05; has 06,07,08,
but no
> 09,10,11, has 12,13,14, no 15,16,17; has 18,19,20, but with 21,no
22,23.
> From yours and mines:
>
> ls -l /glade/u/home/mccabe/data/out/pb2nc1/
>
>
> -rw-r--r-- 1 mccabe ral 10141339 Apr 6 17:52 pbs.2015071500.nc
>
> -rw-r--r-- 1 mccabe ral 5413907 Apr 6 17:56 pbs.2015071501.nc
>
> -rw-r--r-- 1 mccabe ral 4923711 Apr 6 18:00 pbs.2015071502.nc
>
> -rw-r--r-- 1 mccabe ral 5724447 Apr 6 18:14 pbs.2015071506.nc
>
> -rw-r--r-- 1 mccabe ral 5280463 Apr 6 18:18 pbs.2015071507.nc
>
> -rw-r--r-- 1 mccabe ral 4706119 Apr 6 18:22 pbs.2015071508.nc
>
> -rw-r--r-- 1 mccabe ral 9878263 Apr 6 18:36 pbs.2015071512.nc
>
> -rw-r--r-- 1 mccabe ral 5392527 Apr 6 18:40 pbs.2015071513.nc
>
> -rw-r--r-- 1 mccabe ral 96 Apr 6 18:40 pbs.2015071514.nc
>
> -rw-r--r-- 1 mccabe ral 96 Apr 6 18:42 pbs.2015071515.nc
>
>
> cheyenne3:/glade/scratch/zhuming/metplus4/pb2nc1>l
>
> total 96112
>
> -rw-r--r-- 1 zhuming ncar 11550172 Apr 7 15:11 pbs.2015071500.nc
>
> -rw-r--r-- 1 zhuming ncar 6615956 Apr 7 15:14 pbs.2015071501.nc
>
> -rw-r--r-- 1 zhuming ncar 5984328 Apr 7 15:18 pbs.2015071502.nc
>
> -rw-r--r-- 1 zhuming ncar 6980712 Apr 7 15:33 pbs.2015071506.nc
>
> -rw-r--r-- 1 zhuming ncar 6455824 Apr 7 15:36 pbs.2015071507.nc
>
> -rw-r--r-- 1 zhuming ncar 5749432 Apr 7 15:40 pbs.2015071508.nc
>
> -rw-r--r-- 1 zhuming ncar 11284936 Apr 7 15:55 pbs.2015071512.nc
>
> -rw-r--r-- 1 zhuming ncar 6600888 Apr 7 15:59 pbs.2015071513.nc
>
> -rw-r--r-- 1 zhuming ncar 5868168 Apr 7 16:03 pbs.2015071514.nc
>
> -rw-r--r-- 1 zhuming ncar 7273092 Apr 7 16:18 pbs.2015071518.nc
>
> -rw-r--r-- 1 zhuming ncar 6656344 Apr 7 16:22 pbs.2015071519.nc
>
> -rw-r--r-- 1 zhuming ncar 6109788 Apr 7 16:27 pbs.2015071520.nc
>
> -rw-r--r-- 1 zhuming ncar 58808 Apr 7 16:31 pbs.2015071521.nc
>
> -rw-r--r-- 1 zhuming ncar 11132440 Apr 7 16:43 pbs.2015071600.nc
>
>
>
> Thanks,
>
> Zhuming
>
> On Tue, Apr 6, 2021 at 6:34 PM George McCabe via RT
<met_help at ucar.edu>
> wrote:
>
> > Hi Zhuming,
> >
> > Thanks for the information. You will need to use the truncate
keyword to
> > find the files, then set the valid beg/end range to 1 hour. This
is what
> I
> > changed from your PB2NC1.conf config file:
> >
> >
> > *PB2NC_VALID_BEGIN = {valid?fmt=%Y%m%d_%H}PB2NC_VALID_END =
> > {valid?fmt=%Y%m%d_%H?shift=1H}*
> >
> > *PB2NC_INPUT_TEMPLATE =
> >
{valid?fmt=%Y}/prepbufr.gdas.{valid?fmt=%Y%m%d%H?truncate=21600}.nr*
> >
> > My config file is on cheyenne here:
> > /glade/u/home/mccabe/PB2NC1.conf
> >
> > I just started running it and the output is being written here:
> > /glade/u/home/mccabe/data/out/pb2nc1
> >
> >
> > For PointStat, you will want to loop over the same times. I
noticed you
> > specify message type in the VAR<n>_OPTIONS, but you do not need to
do
> this
> > since there is a POINT_STAT_MESSAGE_TYPE variable that defines
this. This
> > is what I changed in your PointStat1.conf file:
> >
> > *VALID_END = 2015071600*
> >
> >
> >
> > *VALID_INCREMENT = 1HPOINT_STAT_OBS_VALID_BEG
=POINT_STAT_OBS_VALID_BEG
> =*
> >
> >
> > *POINT_STAT_MESSAGE_TYPE = ADPSFC*
> >
> >
> >
> >
> >
> > *#OBS_VAR1_OPTIONS = message_type = ["ADPSFC"];#OBS_VAR2_OPTIONS =
> > message_type = ["ADPSFC"];#OBS_VAR3_OPTIONS = message_type =
> > ["ADPSFC"];#OBS_POINT_STAT_INPUT_DIR =
> > /glade/scratch/zhuming/metplus4/pb2nc1*
> >
> > *OBS_POINT_STAT_INPUT_DIR = {OUTPUT_BASE}/pb2nc1*
> >
> > Or you can view my file in the same directory. I usually try to
test
> with a
> > single run time to make sure it is working properly, so I set
VALID_END
> > back to 2015071500. I ran it and it produced an output file, but
there
> were
> > no values besides the header. I checked the log file and saw this:
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > *DEBUG 2: Processing T_sfc(0,*,*) versus TMP/Z2, for observation
type
> > ADPSFC, over region FULL, for interpolation method BILIN(4), using
0
> > matched pairs.DEBUG 3: Number of matched pairs = 0DEBUG 3:
Observations
> > processed = 393373DEBUG 3: Rejected: station id = 0DEBUG
3:
> > Rejected: obs type = 390528DEBUG 3: Rejected: valid time
=
> > 0DEBUG 3: Rejected: bad obs value = 0DEBUG 3: Rejected: off the
grid
> =
> > 2586DEBUG 3: Rejected: topography = 0DEBUG 3: Rejected: level
> mismatch
> > = 259DEBUG 3: Rejected: quality marker = 0DEBUG 3: Rejected:
message
> type
> > = 0DEBUG 3: Rejected: masking region = 0DEBUG 3: Rejected: bad
fcst
> > value = 0DEBUG 3: Rejected: bad climo mean = 0DEBUG 3: Rejected:
bad
> > climo stdev = 0DEBUG 3: Rejected: duplicates = 0DEBUG 2:DEBUG
2:
> >
> >
>
--------------------------------------------------------------------------------DEBUG
> > 2:DEBUG 2: Processing u_10m_gr(0,*,*) versus UGRD/Z10, for
observation
> type
> > ADPSFC, over region FULL, for interpolation method BILIN(4), using
0
> > matched pairs.DEBUG 3: Number of matched pairs = 0DEBUG 3:
Observations
> > processed = 393373*
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > *DEBUG 3: Rejected: station id = 0DEBUG 3: Rejected: obs type
> =
> > 382965DEBUG 3: Rejected: valid time = 0DEBUG 3: Rejected: bad
obs
> > value = 0DEBUG 3: Rejected: off the grid = 9240DEBUG 3:
Rejected:
> > topography = 0DEBUG 3: Rejected: level mismatch = 1168DEBUG
3:
> > Rejected: quality marker = 0DEBUG 3: Rejected: message type =
0DEBUG
> 3:
> > Rejected: masking region = 0DEBUG 3: Rejected: bad fcst value =
0DEBUG
> 3:
> > Rejected: bad climo mean = 0DEBUG 3: Rejected: bad climo stdev =
0DEBUG
> 3:
> > Rejected: duplicates = 0DEBUG 2:DEBUG 2:
> >
> >
>
--------------------------------------------------------------------------------DEBUG
> > 2:DEBUG 2: Processing v_10m_gr(0,*,*) versus VGRD/Z10, for
observation
> type
> > ADPSFC, over region FULL, for interpolation method BILIN(4), using
0
> > matched pairs.DEBUG 3: Number of matched pairs = 0DEBUG 3:
Observations
> > processed = 393373DEBUG 3: Rejected: station id = 0DEBUG
3:
> > Rejected: obs type = 382965DEBUG 3: Rejected: valid time
=
> > 0DEBUG 3: Rejected: bad obs value = 0DEBUG 3: Rejected: off the
grid
> =
> > 9240DEBUG 3: Rejected: topography = 0DEBUG 3: Rejected: level
> mismatch
> > = 1168DEBUG 3: Rejected: quality marker = 0DEBUG 3: Rejected:
message
> > type = 0DEBUG 3: Rejected: masking region = 0DEBUG 3:
Rejected: bad
> > fcst value = 0DEBUG 3: Rejected: bad climo mean = 0DEBUG 3:
Rejected:
> bad
> > climo stdev = 0*
> >
> >
> >
> >
> > *DEBUG 3: Rejected: duplicates = 0DEBUG 2:DEBUG 2:
> >
> >
>
--------------------------------------------------------------------------------DEBUG
> > 2:DEBUG 1: Output file:
> >
> >
>
/glade/u/home/mccabe/data/out/point_stat1/point_stat_000000L_20150715_000000V.stat*
> >
> >
> >
> > This describes the reason why each observation was rejected. Some
> > observations didn't match the message type you requested, some
were not
> > within the forecast grid, and some did not match the vertical
level. You
> > will have to make adjustments to your settings so that the data
matches
> up
> > properly. Maybe the grid you are using in the PB2NC config file
does not
> > match the forecast grid and you are missing values. If that is the
case,
> > you will have to reconfigure and rerun PB2NC to get output on the
correct
> > gr5id. Maybe the observation points don't line up with the
vertical level
> > of the model fields. You can use a range of level values for the
> > observation data to allow data within that range, i.e. Z0-10.
> >
> > Let me know if you are still having issues getting this to work
and I can
> > help out.
> >
> > Thanks,
> > George
> >
> >
> > On Tue, Apr 6, 2021 at 5:17 PM zhuming at ucar.edu via RT <
> met_help at ucar.edu>
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429 >
> > >
> > > Hi George,
> > >
> > > Thanks for your reply!
> > >
> > > I'm still confused, I need to output hourly obs data to do
hourly
> > > verification.
> > > The original GDAS Prepbufr data has 4 files one day, should with
6
> hours
> > > obs data in one file.
> > >
> > > After "PB2NC", I can see pbs.2015071500.nc (obs time is 00? or 6
times
> > of
> > > 00,01,02,03,04,05?), pbs.2015071506.nc...
> > >
> > > my config file :
cheyenne:/gpfs/fs1/work/zhuming/METplus4/PB2NC1.conf
> > > and /gpfs/fs1/work/zhuming/METplus4/PointStat1.conf
> > >
> > > Thanks,
> > > Zhuming
> > >
> > > On Tue, Apr 6, 2021 at 4:23 PM George McCabe via RT
<met_help at ucar.edu
> >
> > > wrote:
> > >
> > > > Hi Zhuming,
> > > >
> > > > To run the tools hourly you would set the VALID_INCREMENT
value for
> 1H.
> > > > Here is an example that runs once per hour for a day from
2015-07-15
> 0Z
> > > to
> > > > 2017-07-16 0Z:
> > > >
> > > > VALID_TIME_FMT = %Y%m%d%H
> > > >
> > > > VALID_BEG = 2015071500
> > > >
> > > > VALID_END = 2015071600
> > > >
> > > > VALID_INCREMENT = 1H
> > > >
> > > > LEAD_SEQ = 0
> > > >
> > > > Each run time will use the PB2NC_INPUT_TEMPLATE to find files
whose
> > > > filenames match the time you need.
> > > >
> > > > To read prepBUFR observations for 12 hours, you will want to
set
> > > > PB2NC_VALID_BEGIN and PB2NC_VALID_END to specify the range
around the
> > > valid
> > > > time to process. These values depend on what is contained
inside your
> > > > prepBUFR files. If you want to use data from the valid time of
the
> > > filename
> > > > backwards 12 hours, you would set:
> > > >
> > > > PB2NC_VALID_BEGIN = {valid?fmt=%Y%m%d_%H?shift=-12H }
> > > > PB2NC_VALID_END = {valid?fmt=%Y%m%d_%H}
> > > >
> > > > For a runtime of 2015071512, this would be substituted as:
> > > >
> > > > valid_beg: 20150715_00
> > > > valid_end: 20150715_12
> > > >
> > > > If you want to create an output file for each hour in PB2NC
you would
> > > need
> > > > to make sure that the hour is specified in the
PB2NC_OUTPUT_TEMPLATE.
> > > >
> > > > PointStat has similar config variables to set the valid beg
and end
> > range
> > > > to use (POINT_STAT_OBS_VALID_BEG and
POINT_STAT_OBS_VALID_END).
> > > >
> > > > If your prepBUFR files are separated out into multiple files
that
> need
> > to
> > > > be read into PB2NC, you can set a file window around the valid
time
> to
> > > > allow files to be passed into the app. More information on
that
> > > > functionality is here:
> > > >
> > > >
> > >
> >
>
https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#using-
windows-to-find-valid-files
> > > >
> > > > If you are still having trouble getting this configured, you
could
> send
> > > me
> > > > the paths of the files you are trying to read and what data is
> > contained
> > > in
> > > > them so I can give you suggestions on what to set for the
> > configuration.
> > > >
> > > > Thanks,
> > > > George
> > > >
> > > >
> > > >
> > > > On Tue, Apr 6, 2021 at 2:17 PM Julie Prestopnik via RT <
> > > met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > >
> > > > > Tue Apr 06 14:16:42 2021: Request 99429 was acted upon.
> > > > > Transaction: Given to mccabe (George McCabe) by jpresto
> > > > > Queue: met_help
> > > > > Subject: How to set time for hourly 12-24 hours data ?
> > > > > Owner: mccabe
> > > > > Requestors: zhuming at ucar.edu
> > > > > Status: new
> > > > > Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429
> > > >
> > > > >
> > > > >
> > > > > This transaction appears to have no content
> > > > >
> > > >
> > > >
> > > > --
> > > > George McCabe - Software Engineer III
> > > > National Center for Atmospheric Research
> > > > Research Applications Laboratory
> > > > 303-497-2768
> > > > ---
> > > > My working day may not be your working day. Please do not feel
> obliged
> > to
> > > > reply to this email outside of your normal working hours.
> > > >
> > > >
> > >
> > >
> >
> > --
> > George McCabe - Software Engineer III
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > 303-497-2768
> > ---
> > My working day may not be your working day. Please do not feel
obliged to
> > reply to this email outside of your normal working hours.
> >
> >
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: How to set time for hourly 12-24 hours data ?
From: zhuming at ucar.edu
Time: Thu Apr 08 09:45:31 2021
Hi George,
Processing Obs data using "PB2NC" are right now.
Now I'm using "PointStat" , I set VALID_BEG = 2015071500, VALID_END =
2015071501. It looks for the forecast data 2015071500/
wrfout_d02_2015-07-15_00:00:00_cf.nc, and 2015071501/
wrfout_d02_2015-07-15_01:00:00_cf.nc.
But my forecast data are all put in dir of 2015071500/.
So how could I set FCST_POINT_STAT_INPUT_TEMPLATE to get the right
fcst
data?
FCST_POINT_STAT_INPUT_TEMPLATE =
{valid?fmt=%Y%m%d%H}/wrfout_d02_{valid?fmt=%Y-%m-%d_%H}:00:00_cf.nc
config file: /gpfs/fs1/work/zhuming/METplus4/PointStat1.conf
log file:
/glade/scratch/zhuming/metplus4/logs/master_metplus.log.20210408085450
Thanks,
Zhuming
On Wed, Apr 7, 2021 at 5:43 PM George McCabe via RT
<met_help at ucar.edu>
wrote:
> Hi Zhuming,
>
> Looking at the log output, it looks like the data is stored in the
prepBUFR
> files differently that you may have thought:
>
> DEBUG 2: PrepBufr Time Center: 20150715_000000
> DEBUG 2: Searching Time Window: 20150715_030000 to
20150715_040000
> 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85%
90% 95%
> 100%
>
> DEBUG 2: Total Messages processed = 590836
> DEBUG 2: Rejected based on message type = 0
> DEBUG 2: Rejected based on station id = 0
> DEBUG 2: Rejected based on valid time = 590815
> DEBUG 2: Rejected based on masking grid = 20
> DEBUG 2: Rejected based on masking polygon = 0
> DEBUG 2: Rejected based on elevation = 0
> DEBUG 2: Rejected based on pb report type = 1
> DEBUG 2: Rejected based on input report type = 0
> DEBUG 2: Rejected based on instrument type = 0
> DEBUG 2: Rejected based on zero observations = 0
> DEBUG 2: Total Messages retained = 0
> DEBUG 2: Total observations retained or derived = 0
> DEBUG 1: Obs time between *20150714_210000 and 20150715_030000*
> WARNING:
> WARNING: process_pbfile() -> No PrepBufr messages retained from
file:
> /glade/collections/rda/data/ds337.0/prepnr/2015/
> prepbufr.gdas.2015071500.nr
> WARNING:
> ERROR :
> ERROR : write_netcdf_hdr_data() -> No PrepBufr messages retained.
Nothing
> to write.
> ERROR :
>
> The prepbufr.gdas.2015071500.nr file actually contains data from
> 20150714_210000 to 20150715_030000, not 20150715_000000 to
20150715_060000.
>
> To get the correct file for each run, you will have to shift the
valid time
> before truncating it so that it gets the correct file for each run.
Instead
> of setting:
>
> PB2NC_INPUT_TEMPLATE =
> {valid?fmt=%Y}/prepbufr.gdas.{valid?fmt=%Y%m%d%H?truncate=21600}.nr
>
> You will have to also shift forward 3 hours, i.e.:
>
> PB2NC_INPUT_TEMPLATE =
>
>
{valid?fmt=%Y}/prepbufr.gdas.{valid?fmt=%Y%m%d%H?shift=3H?truncate=21600}.nr
>
> This says to add 3 hours to the current valid time before truncating
to the
> nearest 6 hours.
>
> Valid: 20150715 0Z + 3 = 3Z truncate 6H = 0Z
> File: prepbufr.gdas.2015071500.nr
>
> Valid: 20150715 1Z + 3 = 4Z truncate 6H = 0Z
> File: prepbufr.gdas.2015071500.nr
>
> Valid: 20150715 2Z + 3 = 5Z truncate 6H = 0Z
> File: prepbufr.gdas.2015071500.nr
>
> Valid: 20150715 3Z + 3 = 6Z truncate 6H = 6Z
> File: prepbufr.gdas.2015071506.nr
>
> Let me know if that doesn't work.
>
> Thanks,
> George
>
>
> On Wed, Apr 7, 2021 at 5:32 PM zhuming at ucar.edu via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429 >
> >
> > Hi George,
> >
> > Now makes me confused is why processed obs of several times are
missing.
> > There has files of time at 00,01,02, but no 03,04,05; has
06,07,08, but
> no
> > 09,10,11, has 12,13,14, no 15,16,17; has 18,19,20, but with 21,no
22,23.
> > From yours and mines:
> >
> > ls -l /glade/u/home/mccabe/data/out/pb2nc1/
> >
> >
> > -rw-r--r-- 1 mccabe ral 10141339 Apr 6 17:52 pbs.2015071500.nc
> >
> > -rw-r--r-- 1 mccabe ral 5413907 Apr 6 17:56 pbs.2015071501.nc
> >
> > -rw-r--r-- 1 mccabe ral 4923711 Apr 6 18:00 pbs.2015071502.nc
> >
> > -rw-r--r-- 1 mccabe ral 5724447 Apr 6 18:14 pbs.2015071506.nc
> >
> > -rw-r--r-- 1 mccabe ral 5280463 Apr 6 18:18 pbs.2015071507.nc
> >
> > -rw-r--r-- 1 mccabe ral 4706119 Apr 6 18:22 pbs.2015071508.nc
> >
> > -rw-r--r-- 1 mccabe ral 9878263 Apr 6 18:36 pbs.2015071512.nc
> >
> > -rw-r--r-- 1 mccabe ral 5392527 Apr 6 18:40 pbs.2015071513.nc
> >
> > -rw-r--r-- 1 mccabe ral 96 Apr 6 18:40 pbs.2015071514.nc
> >
> > -rw-r--r-- 1 mccabe ral 96 Apr 6 18:42 pbs.2015071515.nc
> >
> >
> > cheyenne3:/glade/scratch/zhuming/metplus4/pb2nc1>l
> >
> > total 96112
> >
> > -rw-r--r-- 1 zhuming ncar 11550172 Apr 7 15:11 pbs.2015071500.nc
> >
> > -rw-r--r-- 1 zhuming ncar 6615956 Apr 7 15:14 pbs.2015071501.nc
> >
> > -rw-r--r-- 1 zhuming ncar 5984328 Apr 7 15:18 pbs.2015071502.nc
> >
> > -rw-r--r-- 1 zhuming ncar 6980712 Apr 7 15:33 pbs.2015071506.nc
> >
> > -rw-r--r-- 1 zhuming ncar 6455824 Apr 7 15:36 pbs.2015071507.nc
> >
> > -rw-r--r-- 1 zhuming ncar 5749432 Apr 7 15:40 pbs.2015071508.nc
> >
> > -rw-r--r-- 1 zhuming ncar 11284936 Apr 7 15:55 pbs.2015071512.nc
> >
> > -rw-r--r-- 1 zhuming ncar 6600888 Apr 7 15:59 pbs.2015071513.nc
> >
> > -rw-r--r-- 1 zhuming ncar 5868168 Apr 7 16:03 pbs.2015071514.nc
> >
> > -rw-r--r-- 1 zhuming ncar 7273092 Apr 7 16:18 pbs.2015071518.nc
> >
> > -rw-r--r-- 1 zhuming ncar 6656344 Apr 7 16:22 pbs.2015071519.nc
> >
> > -rw-r--r-- 1 zhuming ncar 6109788 Apr 7 16:27 pbs.2015071520.nc
> >
> > -rw-r--r-- 1 zhuming ncar 58808 Apr 7 16:31 pbs.2015071521.nc
> >
> > -rw-r--r-- 1 zhuming ncar 11132440 Apr 7 16:43 pbs.2015071600.nc
> >
> >
> >
> > Thanks,
> >
> > Zhuming
> >
> > On Tue, Apr 6, 2021 at 6:34 PM George McCabe via RT
<met_help at ucar.edu>
> > wrote:
> >
> > > Hi Zhuming,
> > >
> > > Thanks for the information. You will need to use the truncate
keyword
> to
> > > find the files, then set the valid beg/end range to 1 hour. This
is
> what
> > I
> > > changed from your PB2NC1.conf config file:
> > >
> > >
> > > *PB2NC_VALID_BEGIN = {valid?fmt=%Y%m%d_%H}PB2NC_VALID_END =
> > > {valid?fmt=%Y%m%d_%H?shift=1H}*
> > >
> > > *PB2NC_INPUT_TEMPLATE =
> > >
{valid?fmt=%Y}/prepbufr.gdas.{valid?fmt=%Y%m%d%H?truncate=21600}.nr*
> > >
> > > My config file is on cheyenne here:
> > > /glade/u/home/mccabe/PB2NC1.conf
> > >
> > > I just started running it and the output is being written here:
> > > /glade/u/home/mccabe/data/out/pb2nc1
> > >
> > >
> > > For PointStat, you will want to loop over the same times. I
noticed you
> > > specify message type in the VAR<n>_OPTIONS, but you do not need
to do
> > this
> > > since there is a POINT_STAT_MESSAGE_TYPE variable that defines
this.
> This
> > > is what I changed in your PointStat1.conf file:
> > >
> > > *VALID_END = 2015071600*
> > >
> > >
> > >
> > > *VALID_INCREMENT = 1HPOINT_STAT_OBS_VALID_BEG
=POINT_STAT_OBS_VALID_BEG
> > =*
> > >
> > >
> > > *POINT_STAT_MESSAGE_TYPE = ADPSFC*
> > >
> > >
> > >
> > >
> > >
> > > *#OBS_VAR1_OPTIONS = message_type = ["ADPSFC"];#OBS_VAR2_OPTIONS
=
> > > message_type = ["ADPSFC"];#OBS_VAR3_OPTIONS = message_type =
> > > ["ADPSFC"];#OBS_POINT_STAT_INPUT_DIR =
> > > /glade/scratch/zhuming/metplus4/pb2nc1*
> > >
> > > *OBS_POINT_STAT_INPUT_DIR = {OUTPUT_BASE}/pb2nc1*
> > >
> > > Or you can view my file in the same directory. I usually try to
test
> > with a
> > > single run time to make sure it is working properly, so I set
VALID_END
> > > back to 2015071500. I ran it and it produced an output file, but
there
> > were
> > > no values besides the header. I checked the log file and saw
this:
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > *DEBUG 2: Processing T_sfc(0,*,*) versus TMP/Z2, for observation
type
> > > ADPSFC, over region FULL, for interpolation method BILIN(4),
using 0
> > > matched pairs.DEBUG 3: Number of matched pairs = 0DEBUG 3:
> Observations
> > > processed = 393373DEBUG 3: Rejected: station id = 0DEBUG
3:
> > > Rejected: obs type = 390528DEBUG 3: Rejected: valid time
=
> > > 0DEBUG 3: Rejected: bad obs value = 0DEBUG 3: Rejected: off
the grid
> > =
> > > 2586DEBUG 3: Rejected: topography = 0DEBUG 3: Rejected:
level
> > mismatch
> > > = 259DEBUG 3: Rejected: quality marker = 0DEBUG 3: Rejected:
message
> > type
> > > = 0DEBUG 3: Rejected: masking region = 0DEBUG 3: Rejected:
bad fcst
> > > value = 0DEBUG 3: Rejected: bad climo mean = 0DEBUG 3:
Rejected: bad
> > > climo stdev = 0DEBUG 3: Rejected: duplicates = 0DEBUG
2:DEBUG 2:
> > >
> > >
> >
>
--------------------------------------------------------------------------------DEBUG
> > > 2:DEBUG 2: Processing u_10m_gr(0,*,*) versus UGRD/Z10, for
observation
> > type
> > > ADPSFC, over region FULL, for interpolation method BILIN(4),
using 0
> > > matched pairs.DEBUG 3: Number of matched pairs = 0DEBUG 3:
> Observations
> > > processed = 393373*
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > *DEBUG 3: Rejected: station id = 0DEBUG 3: Rejected: obs
type
> > =
> > > 382965DEBUG 3: Rejected: valid time = 0DEBUG 3: Rejected:
bad obs
> > > value = 0DEBUG 3: Rejected: off the grid = 9240DEBUG 3:
Rejected:
> > > topography = 0DEBUG 3: Rejected: level mismatch =
1168DEBUG 3:
> > > Rejected: quality marker = 0DEBUG 3: Rejected: message type
=
> 0DEBUG
> > 3:
> > > Rejected: masking region = 0DEBUG 3: Rejected: bad fcst value
=
> 0DEBUG
> > 3:
> > > Rejected: bad climo mean = 0DEBUG 3: Rejected: bad climo stdev
=
> 0DEBUG
> > 3:
> > > Rejected: duplicates = 0DEBUG 2:DEBUG 2:
> > >
> > >
> >
>
--------------------------------------------------------------------------------DEBUG
> > > 2:DEBUG 2: Processing v_10m_gr(0,*,*) versus VGRD/Z10, for
observation
> > type
> > > ADPSFC, over region FULL, for interpolation method BILIN(4),
using 0
> > > matched pairs.DEBUG 3: Number of matched pairs = 0DEBUG 3:
> Observations
> > > processed = 393373DEBUG 3: Rejected: station id = 0DEBUG
3:
> > > Rejected: obs type = 382965DEBUG 3: Rejected: valid time
=
> > > 0DEBUG 3: Rejected: bad obs value = 0DEBUG 3: Rejected: off
the grid
> > =
> > > 9240DEBUG 3: Rejected: topography = 0DEBUG 3: Rejected:
level
> > mismatch
> > > = 1168DEBUG 3: Rejected: quality marker = 0DEBUG 3: Rejected:
message
> > > type = 0DEBUG 3: Rejected: masking region = 0DEBUG 3:
Rejected: bad
> > > fcst value = 0DEBUG 3: Rejected: bad climo mean = 0DEBUG 3:
Rejected:
> > bad
> > > climo stdev = 0*
> > >
> > >
> > >
> > >
> > > *DEBUG 3: Rejected: duplicates = 0DEBUG 2:DEBUG 2:
> > >
> > >
> >
>
--------------------------------------------------------------------------------DEBUG
> > > 2:DEBUG 1: Output file:
> > >
> > >
> >
>
/glade/u/home/mccabe/data/out/point_stat1/point_stat_000000L_20150715_000000V.stat*
> > >
> > >
> > >
> > > This describes the reason why each observation was rejected.
Some
> > > observations didn't match the message type you requested, some
were not
> > > within the forecast grid, and some did not match the vertical
level.
> You
> > > will have to make adjustments to your settings so that the data
matches
> > up
> > > properly. Maybe the grid you are using in the PB2NC config file
does
> not
> > > match the forecast grid and you are missing values. If that is
the
> case,
> > > you will have to reconfigure and rerun PB2NC to get output on
the
> correct
> > > gr5id. Maybe the observation points don't line up with the
vertical
> level
> > > of the model fields. You can use a range of level values for the
> > > observation data to allow data within that range, i.e. Z0-10.
> > >
> > > Let me know if you are still having issues getting this to work
and I
> can
> > > help out.
> > >
> > > Thanks,
> > > George
> > >
> > >
> > > On Tue, Apr 6, 2021 at 5:17 PM zhuming at ucar.edu via RT <
> > met_help at ucar.edu>
> > > wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429
>
> > > >
> > > > Hi George,
> > > >
> > > > Thanks for your reply!
> > > >
> > > > I'm still confused, I need to output hourly obs data to do
hourly
> > > > verification.
> > > > The original GDAS Prepbufr data has 4 files one day, should
with 6
> > hours
> > > > obs data in one file.
> > > >
> > > > After "PB2NC", I can see pbs.2015071500.nc (obs time is 00? or
6
> times
> > > of
> > > > 00,01,02,03,04,05?), pbs.2015071506.nc...
> > > >
> > > > my config file :
cheyenne:/gpfs/fs1/work/zhuming/METplus4/PB2NC1.conf
> > > > and /gpfs/fs1/work/zhuming/METplus4/PointStat1.conf
> > > >
> > > > Thanks,
> > > > Zhuming
> > > >
> > > > On Tue, Apr 6, 2021 at 4:23 PM George McCabe via RT <
> met_help at ucar.edu
> > >
> > > > wrote:
> > > >
> > > > > Hi Zhuming,
> > > > >
> > > > > To run the tools hourly you would set the VALID_INCREMENT
value for
> > 1H.
> > > > > Here is an example that runs once per hour for a day from
> 2015-07-15
> > 0Z
> > > > to
> > > > > 2017-07-16 0Z:
> > > > >
> > > > > VALID_TIME_FMT = %Y%m%d%H
> > > > >
> > > > > VALID_BEG = 2015071500
> > > > >
> > > > > VALID_END = 2015071600
> > > > >
> > > > > VALID_INCREMENT = 1H
> > > > >
> > > > > LEAD_SEQ = 0
> > > > >
> > > > > Each run time will use the PB2NC_INPUT_TEMPLATE to find
files whose
> > > > > filenames match the time you need.
> > > > >
> > > > > To read prepBUFR observations for 12 hours, you will want to
set
> > > > > PB2NC_VALID_BEGIN and PB2NC_VALID_END to specify the range
around
> the
> > > > valid
> > > > > time to process. These values depend on what is contained
inside
> your
> > > > > prepBUFR files. If you want to use data from the valid time
of the
> > > > filename
> > > > > backwards 12 hours, you would set:
> > > > >
> > > > > PB2NC_VALID_BEGIN = {valid?fmt=%Y%m%d_%H?shift=-12H }
> > > > > PB2NC_VALID_END = {valid?fmt=%Y%m%d_%H}
> > > > >
> > > > > For a runtime of 2015071512, this would be substituted as:
> > > > >
> > > > > valid_beg: 20150715_00
> > > > > valid_end: 20150715_12
> > > > >
> > > > > If you want to create an output file for each hour in PB2NC
you
> would
> > > > need
> > > > > to make sure that the hour is specified in the
> PB2NC_OUTPUT_TEMPLATE.
> > > > >
> > > > > PointStat has similar config variables to set the valid beg
and end
> > > range
> > > > > to use (POINT_STAT_OBS_VALID_BEG and
POINT_STAT_OBS_VALID_END).
> > > > >
> > > > > If your prepBUFR files are separated out into multiple files
that
> > need
> > > to
> > > > > be read into PB2NC, you can set a file window around the
valid time
> > to
> > > > > allow files to be passed into the app. More information on
that
> > > > > functionality is here:
> > > > >
> > > > >
> > > >
> > >
> >
>
https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#using-
windows-to-find-valid-files
> > > > >
> > > > > If you are still having trouble getting this configured, you
could
> > send
> > > > me
> > > > > the paths of the files you are trying to read and what data
is
> > > contained
> > > > in
> > > > > them so I can give you suggestions on what to set for the
> > > configuration.
> > > > >
> > > > > Thanks,
> > > > > George
> > > > >
> > > > >
> > > > >
> > > > > On Tue, Apr 6, 2021 at 2:17 PM Julie Prestopnik via RT <
> > > > met_help at ucar.edu>
> > > > > wrote:
> > > > >
> > > > > >
> > > > > > Tue Apr 06 14:16:42 2021: Request 99429 was acted upon.
> > > > > > Transaction: Given to mccabe (George McCabe) by jpresto
> > > > > > Queue: met_help
> > > > > > Subject: How to set time for hourly 12-24 hours data
?
> > > > > > Owner: mccabe
> > > > > > Requestors: zhuming at ucar.edu
> > > > > > Status: new
> > > > > > Ticket <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429
> > > > >
> > > > > >
> > > > > >
> > > > > > This transaction appears to have no content
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > George McCabe - Software Engineer III
> > > > > National Center for Atmospheric Research
> > > > > Research Applications Laboratory
> > > > > 303-497-2768
> > > > > ---
> > > > > My working day may not be your working day. Please do not
feel
> > obliged
> > > to
> > > > > reply to this email outside of your normal working hours.
> > > > >
> > > > >
> > > >
> > > >
> > >
> > > --
> > > George McCabe - Software Engineer III
> > > National Center for Atmospheric Research
> > > Research Applications Laboratory
> > > 303-497-2768
> > > ---
> > > My working day may not be your working day. Please do not feel
obliged
> to
> > > reply to this email outside of your normal working hours.
> > >
> > >
> >
> >
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>
------------------------------------------------
Subject: How to set time for hourly 12-24 hours data ?
From: George McCabe
Time: Thu Apr 08 10:04:44 2021
Hi Zhuming,
Typically directories that contain data for an entire day are named
with
YYYYMMDD, so your filename template tag would be {valid?fmt=%Y%m%d},
however that may not always be the case, as you are seeing here. There
are
a couple ways to handle this.
If you know that the directory will always be YYYYMMDD with 00 at the
end,
you can simply put 00 in the template instead of adding %H in the tag,
i.e.
FCST_POINT_STAT_INPUT_TEMPLATE =
{valid?fmt=%Y%m%d}00/wrfout_d02_{valid?fmt=%Y-%m-%d_%H}:00:00_cf.nc
If the HH value can vary, for instance if there are 2 directories
created
per day (i.e. 2015071400 and 2015071412), then you can use the
truncate
keyword to handle it.
If directories are created every 24 hours:
FCST_POINT_STAT_INPUT_TEMPLATE =
{valid?fmt=%Y%m%d%H?truncate=24H}/wrfout_d02_{valid?fmt=%Y-%m-
%d_%H}:00:
00_cf.nc
If directories are created every 12 hours:
FCST_POINT_STAT_INPUT_TEMPLATE =
{valid?fmt=%Y%m%d%H?truncate=12H}/wrfout_d02_{valid?fmt=%Y-%m-
%d_%H}:00:
00_cf.nc
Let me know if that doesn't work for you.
Thanks,
George
On Thu, Apr 8, 2021 at 9:46 AM zhuming at ucar.edu via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429 >
>
> Hi George,
>
> Processing Obs data using "PB2NC" are right now.
>
> Now I'm using "PointStat" , I set VALID_BEG = 2015071500, VALID_END
=
> 2015071501. It looks for the forecast data 2015071500/
> wrfout_d02_2015-07-15_00:00:00_cf.nc, and 2015071501/
> wrfout_d02_2015-07-15_01:00:00_cf.nc.
> But my forecast data are all put in dir of 2015071500/.
> So how could I set FCST_POINT_STAT_INPUT_TEMPLATE to get the right
fcst
> data?
>
> FCST_POINT_STAT_INPUT_TEMPLATE =
> {valid?fmt=%Y%m%d%H}/wrfout_d02_{valid?fmt=%Y-%m-%d_%H}:00:00_cf.nc
>
>
> config file: /gpfs/fs1/work/zhuming/METplus4/PointStat1.conf
>
> log file:
>
/glade/scratch/zhuming/metplus4/logs/master_metplus.log.20210408085450
>
>
> Thanks,
>
> Zhuming
>
> On Wed, Apr 7, 2021 at 5:43 PM George McCabe via RT
<met_help at ucar.edu>
> wrote:
>
> > Hi Zhuming,
> >
> > Looking at the log output, it looks like the data is stored in the
> prepBUFR
> > files differently that you may have thought:
> >
> > DEBUG 2: PrepBufr Time Center: 20150715_000000
> > DEBUG 2: Searching Time Window: 20150715_030000 to
> 20150715_040000
> > 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85%
90%
> 95%
> > 100%
> >
> > DEBUG 2: Total Messages processed = 590836
> > DEBUG 2: Rejected based on message type = 0
> > DEBUG 2: Rejected based on station id = 0
> > DEBUG 2: Rejected based on valid time = 590815
> > DEBUG 2: Rejected based on masking grid = 20
> > DEBUG 2: Rejected based on masking polygon = 0
> > DEBUG 2: Rejected based on elevation = 0
> > DEBUG 2: Rejected based on pb report type = 1
> > DEBUG 2: Rejected based on input report type = 0
> > DEBUG 2: Rejected based on instrument type = 0
> > DEBUG 2: Rejected based on zero observations = 0
> > DEBUG 2: Total Messages retained = 0
> > DEBUG 2: Total observations retained or derived = 0
> > DEBUG 1: Obs time between *20150714_210000 and 20150715_030000*
> > WARNING:
> > WARNING: process_pbfile() -> No PrepBufr messages retained from
file:
> > /glade/collections/rda/data/ds337.0/prepnr/2015/
> > prepbufr.gdas.2015071500.nr
> > WARNING:
> > ERROR :
> > ERROR : write_netcdf_hdr_data() -> No PrepBufr messages retained.
> Nothing
> > to write.
> > ERROR :
> >
> > The prepbufr.gdas.2015071500.nr file actually contains data from
> > 20150714_210000 to 20150715_030000, not 20150715_000000 to
> 20150715_060000.
> >
> > To get the correct file for each run, you will have to shift the
valid
> time
> > before truncating it so that it gets the correct file for each
run.
> Instead
> > of setting:
> >
> > PB2NC_INPUT_TEMPLATE =
> >
{valid?fmt=%Y}/prepbufr.gdas.{valid?fmt=%Y%m%d%H?truncate=21600}.nr
> >
> > You will have to also shift forward 3 hours, i.e.:
> >
> > PB2NC_INPUT_TEMPLATE =
> >
> >
>
{valid?fmt=%Y}/prepbufr.gdas.{valid?fmt=%Y%m%d%H?shift=3H?truncate=21600}.nr
> >
> > This says to add 3 hours to the current valid time before
truncating to
> the
> > nearest 6 hours.
> >
> > Valid: 20150715 0Z + 3 = 3Z truncate 6H = 0Z
> > File: prepbufr.gdas.2015071500.nr
> >
> > Valid: 20150715 1Z + 3 = 4Z truncate 6H = 0Z
> > File: prepbufr.gdas.2015071500.nr
> >
> > Valid: 20150715 2Z + 3 = 5Z truncate 6H = 0Z
> > File: prepbufr.gdas.2015071500.nr
> >
> > Valid: 20150715 3Z + 3 = 6Z truncate 6H = 6Z
> > File: prepbufr.gdas.2015071506.nr
> >
> > Let me know if that doesn't work.
> >
> > Thanks,
> > George
> >
> >
> > On Wed, Apr 7, 2021 at 5:32 PM zhuming at ucar.edu via RT <
> met_help at ucar.edu>
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429 >
> > >
> > > Hi George,
> > >
> > > Now makes me confused is why processed obs of several times are
> missing.
> > > There has files of time at 00,01,02, but no 03,04,05; has
06,07,08, but
> > no
> > > 09,10,11, has 12,13,14, no 15,16,17; has 18,19,20, but with
21,no
> 22,23.
> > > From yours and mines:
> > >
> > > ls -l /glade/u/home/mccabe/data/out/pb2nc1/
> > >
> > >
> > > -rw-r--r-- 1 mccabe ral 10141339 Apr 6 17:52 pbs.2015071500.nc
> > >
> > > -rw-r--r-- 1 mccabe ral 5413907 Apr 6 17:56 pbs.2015071501.nc
> > >
> > > -rw-r--r-- 1 mccabe ral 4923711 Apr 6 18:00 pbs.2015071502.nc
> > >
> > > -rw-r--r-- 1 mccabe ral 5724447 Apr 6 18:14 pbs.2015071506.nc
> > >
> > > -rw-r--r-- 1 mccabe ral 5280463 Apr 6 18:18 pbs.2015071507.nc
> > >
> > > -rw-r--r-- 1 mccabe ral 4706119 Apr 6 18:22 pbs.2015071508.nc
> > >
> > > -rw-r--r-- 1 mccabe ral 9878263 Apr 6 18:36 pbs.2015071512.nc
> > >
> > > -rw-r--r-- 1 mccabe ral 5392527 Apr 6 18:40 pbs.2015071513.nc
> > >
> > > -rw-r--r-- 1 mccabe ral 96 Apr 6 18:40 pbs.2015071514.nc
> > >
> > > -rw-r--r-- 1 mccabe ral 96 Apr 6 18:42 pbs.2015071515.nc
> > >
> > >
> > > cheyenne3:/glade/scratch/zhuming/metplus4/pb2nc1>l
> > >
> > > total 96112
> > >
> > > -rw-r--r-- 1 zhuming ncar 11550172 Apr 7 15:11
pbs.2015071500.nc
> > >
> > > -rw-r--r-- 1 zhuming ncar 6615956 Apr 7 15:14
pbs.2015071501.nc
> > >
> > > -rw-r--r-- 1 zhuming ncar 5984328 Apr 7 15:18
pbs.2015071502.nc
> > >
> > > -rw-r--r-- 1 zhuming ncar 6980712 Apr 7 15:33
pbs.2015071506.nc
> > >
> > > -rw-r--r-- 1 zhuming ncar 6455824 Apr 7 15:36
pbs.2015071507.nc
> > >
> > > -rw-r--r-- 1 zhuming ncar 5749432 Apr 7 15:40
pbs.2015071508.nc
> > >
> > > -rw-r--r-- 1 zhuming ncar 11284936 Apr 7 15:55
pbs.2015071512.nc
> > >
> > > -rw-r--r-- 1 zhuming ncar 6600888 Apr 7 15:59
pbs.2015071513.nc
> > >
> > > -rw-r--r-- 1 zhuming ncar 5868168 Apr 7 16:03
pbs.2015071514.nc
> > >
> > > -rw-r--r-- 1 zhuming ncar 7273092 Apr 7 16:18
pbs.2015071518.nc
> > >
> > > -rw-r--r-- 1 zhuming ncar 6656344 Apr 7 16:22
pbs.2015071519.nc
> > >
> > > -rw-r--r-- 1 zhuming ncar 6109788 Apr 7 16:27
pbs.2015071520.nc
> > >
> > > -rw-r--r-- 1 zhuming ncar 58808 Apr 7 16:31
pbs.2015071521.nc
> > >
> > > -rw-r--r-- 1 zhuming ncar 11132440 Apr 7 16:43
pbs.2015071600.nc
> > >
> > >
> > >
> > > Thanks,
> > >
> > > Zhuming
> > >
> > > On Tue, Apr 6, 2021 at 6:34 PM George McCabe via RT
<met_help at ucar.edu
> >
> > > wrote:
> > >
> > > > Hi Zhuming,
> > > >
> > > > Thanks for the information. You will need to use the truncate
keyword
> > to
> > > > find the files, then set the valid beg/end range to 1 hour.
This is
> > what
> > > I
> > > > changed from your PB2NC1.conf config file:
> > > >
> > > >
> > > > *PB2NC_VALID_BEGIN = {valid?fmt=%Y%m%d_%H}PB2NC_VALID_END =
> > > > {valid?fmt=%Y%m%d_%H?shift=1H}*
> > > >
> > > > *PB2NC_INPUT_TEMPLATE =
> > > >
{valid?fmt=%Y}/prepbufr.gdas.{valid?fmt=%Y%m%d%H?truncate=21600}.nr*
> > > >
> > > > My config file is on cheyenne here:
> > > > /glade/u/home/mccabe/PB2NC1.conf
> > > >
> > > > I just started running it and the output is being written
here:
> > > > /glade/u/home/mccabe/data/out/pb2nc1
> > > >
> > > >
> > > > For PointStat, you will want to loop over the same times. I
noticed
> you
> > > > specify message type in the VAR<n>_OPTIONS, but you do not
need to do
> > > this
> > > > since there is a POINT_STAT_MESSAGE_TYPE variable that defines
this.
> > This
> > > > is what I changed in your PointStat1.conf file:
> > > >
> > > > *VALID_END = 2015071600*
> > > >
> > > >
> > > >
> > > > *VALID_INCREMENT = 1HPOINT_STAT_OBS_VALID_BEG
> =POINT_STAT_OBS_VALID_BEG
> > > =*
> > > >
> > > >
> > > > *POINT_STAT_MESSAGE_TYPE = ADPSFC*
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > *#OBS_VAR1_OPTIONS = message_type =
["ADPSFC"];#OBS_VAR2_OPTIONS =
> > > > message_type = ["ADPSFC"];#OBS_VAR3_OPTIONS = message_type =
> > > > ["ADPSFC"];#OBS_POINT_STAT_INPUT_DIR =
> > > > /glade/scratch/zhuming/metplus4/pb2nc1*
> > > >
> > > > *OBS_POINT_STAT_INPUT_DIR = {OUTPUT_BASE}/pb2nc1*
> > > >
> > > > Or you can view my file in the same directory. I usually try
to test
> > > with a
> > > > single run time to make sure it is working properly, so I set
> VALID_END
> > > > back to 2015071500. I ran it and it produced an output file,
but
> there
> > > were
> > > > no values besides the header. I checked the log file and saw
this:
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > *DEBUG 2: Processing T_sfc(0,*,*) versus TMP/Z2, for
observation type
> > > > ADPSFC, over region FULL, for interpolation method BILIN(4),
using 0
> > > > matched pairs.DEBUG 3: Number of matched pairs = 0DEBUG 3:
> > Observations
> > > > processed = 393373DEBUG 3: Rejected: station id =
0DEBUG 3:
> > > > Rejected: obs type = 390528DEBUG 3: Rejected: valid
time
> =
> > > > 0DEBUG 3: Rejected: bad obs value = 0DEBUG 3: Rejected: off
the
> grid
> > > =
> > > > 2586DEBUG 3: Rejected: topography = 0DEBUG 3: Rejected:
level
> > > mismatch
> > > > = 259DEBUG 3: Rejected: quality marker = 0DEBUG 3: Rejected:
> message
> > > type
> > > > = 0DEBUG 3: Rejected: masking region = 0DEBUG 3: Rejected:
bad
> fcst
> > > > value = 0DEBUG 3: Rejected: bad climo mean = 0DEBUG 3:
Rejected:
> bad
> > > > climo stdev = 0DEBUG 3: Rejected: duplicates = 0DEBUG
2:DEBUG 2:
> > > >
> > > >
> > >
> >
>
--------------------------------------------------------------------------------DEBUG
> > > > 2:DEBUG 2: Processing u_10m_gr(0,*,*) versus UGRD/Z10, for
> observation
> > > type
> > > > ADPSFC, over region FULL, for interpolation method BILIN(4),
using 0
> > > > matched pairs.DEBUG 3: Number of matched pairs = 0DEBUG 3:
> > Observations
> > > > processed = 393373*
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > *DEBUG 3: Rejected: station id = 0DEBUG 3: Rejected: obs
type
> > > =
> > > > 382965DEBUG 3: Rejected: valid time = 0DEBUG 3: Rejected:
bad
> obs
> > > > value = 0DEBUG 3: Rejected: off the grid = 9240DEBUG 3:
> Rejected:
> > > > topography = 0DEBUG 3: Rejected: level mismatch =
1168DEBUG 3:
> > > > Rejected: quality marker = 0DEBUG 3: Rejected: message type
=
> > 0DEBUG
> > > 3:
> > > > Rejected: masking region = 0DEBUG 3: Rejected: bad fcst value
=
> > 0DEBUG
> > > 3:
> > > > Rejected: bad climo mean = 0DEBUG 3: Rejected: bad climo
stdev =
> > 0DEBUG
> > > 3:
> > > > Rejected: duplicates = 0DEBUG 2:DEBUG 2:
> > > >
> > > >
> > >
> >
>
--------------------------------------------------------------------------------DEBUG
> > > > 2:DEBUG 2: Processing v_10m_gr(0,*,*) versus VGRD/Z10, for
> observation
> > > type
> > > > ADPSFC, over region FULL, for interpolation method BILIN(4),
using 0
> > > > matched pairs.DEBUG 3: Number of matched pairs = 0DEBUG 3:
> > Observations
> > > > processed = 393373DEBUG 3: Rejected: station id =
0DEBUG 3:
> > > > Rejected: obs type = 382965DEBUG 3: Rejected: valid
time
> =
> > > > 0DEBUG 3: Rejected: bad obs value = 0DEBUG 3: Rejected: off
the
> grid
> > > =
> > > > 9240DEBUG 3: Rejected: topography = 0DEBUG 3: Rejected:
level
> > > mismatch
> > > > = 1168DEBUG 3: Rejected: quality marker = 0DEBUG 3:
Rejected:
> message
> > > > type = 0DEBUG 3: Rejected: masking region = 0DEBUG 3:
Rejected:
> bad
> > > > fcst value = 0DEBUG 3: Rejected: bad climo mean = 0DEBUG 3:
> Rejected:
> > > bad
> > > > climo stdev = 0*
> > > >
> > > >
> > > >
> > > >
> > > > *DEBUG 3: Rejected: duplicates = 0DEBUG 2:DEBUG 2:
> > > >
> > > >
> > >
> >
>
--------------------------------------------------------------------------------DEBUG
> > > > 2:DEBUG 1: Output file:
> > > >
> > > >
> > >
> >
>
/glade/u/home/mccabe/data/out/point_stat1/point_stat_000000L_20150715_000000V.stat*
> > > >
> > > >
> > > >
> > > > This describes the reason why each observation was rejected.
Some
> > > > observations didn't match the message type you requested, some
were
> not
> > > > within the forecast grid, and some did not match the vertical
level.
> > You
> > > > will have to make adjustments to your settings so that the
data
> matches
> > > up
> > > > properly. Maybe the grid you are using in the PB2NC config
file does
> > not
> > > > match the forecast grid and you are missing values. If that is
the
> > case,
> > > > you will have to reconfigure and rerun PB2NC to get output on
the
> > correct
> > > > gr5id. Maybe the observation points don't line up with the
vertical
> > level
> > > > of the model fields. You can use a range of level values for
the
> > > > observation data to allow data within that range, i.e. Z0-10.
> > > >
> > > > Let me know if you are still having issues getting this to
work and I
> > can
> > > > help out.
> > > >
> > > > Thanks,
> > > > George
> > > >
> > > >
> > > > On Tue, Apr 6, 2021 at 5:17 PM zhuming at ucar.edu via RT <
> > > met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > >
> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429 >
> > > > >
> > > > > Hi George,
> > > > >
> > > > > Thanks for your reply!
> > > > >
> > > > > I'm still confused, I need to output hourly obs data to do
hourly
> > > > > verification.
> > > > > The original GDAS Prepbufr data has 4 files one day, should
with 6
> > > hours
> > > > > obs data in one file.
> > > > >
> > > > > After "PB2NC", I can see pbs.2015071500.nc (obs time is 00?
or 6
> > times
> > > > of
> > > > > 00,01,02,03,04,05?), pbs.2015071506.nc...
> > > > >
> > > > > my config file :
> cheyenne:/gpfs/fs1/work/zhuming/METplus4/PB2NC1.conf
> > > > > and /gpfs/fs1/work/zhuming/METplus4/PointStat1.conf
> > > > >
> > > > > Thanks,
> > > > > Zhuming
> > > > >
> > > > > On Tue, Apr 6, 2021 at 4:23 PM George McCabe via RT <
> > met_help at ucar.edu
> > > >
> > > > > wrote:
> > > > >
> > > > > > Hi Zhuming,
> > > > > >
> > > > > > To run the tools hourly you would set the VALID_INCREMENT
value
> for
> > > 1H.
> > > > > > Here is an example that runs once per hour for a day from
> > 2015-07-15
> > > 0Z
> > > > > to
> > > > > > 2017-07-16 0Z:
> > > > > >
> > > > > > VALID_TIME_FMT = %Y%m%d%H
> > > > > >
> > > > > > VALID_BEG = 2015071500
> > > > > >
> > > > > > VALID_END = 2015071600
> > > > > >
> > > > > > VALID_INCREMENT = 1H
> > > > > >
> > > > > > LEAD_SEQ = 0
> > > > > >
> > > > > > Each run time will use the PB2NC_INPUT_TEMPLATE to find
files
> whose
> > > > > > filenames match the time you need.
> > > > > >
> > > > > > To read prepBUFR observations for 12 hours, you will want
to set
> > > > > > PB2NC_VALID_BEGIN and PB2NC_VALID_END to specify the range
around
> > the
> > > > > valid
> > > > > > time to process. These values depend on what is contained
inside
> > your
> > > > > > prepBUFR files. If you want to use data from the valid
time of
> the
> > > > > filename
> > > > > > backwards 12 hours, you would set:
> > > > > >
> > > > > > PB2NC_VALID_BEGIN = {valid?fmt=%Y%m%d_%H?shift=-12H }
> > > > > > PB2NC_VALID_END = {valid?fmt=%Y%m%d_%H}
> > > > > >
> > > > > > For a runtime of 2015071512, this would be substituted as:
> > > > > >
> > > > > > valid_beg: 20150715_00
> > > > > > valid_end: 20150715_12
> > > > > >
> > > > > > If you want to create an output file for each hour in
PB2NC you
> > would
> > > > > need
> > > > > > to make sure that the hour is specified in the
> > PB2NC_OUTPUT_TEMPLATE.
> > > > > >
> > > > > > PointStat has similar config variables to set the valid
beg and
> end
> > > > range
> > > > > > to use (POINT_STAT_OBS_VALID_BEG and
POINT_STAT_OBS_VALID_END).
> > > > > >
> > > > > > If your prepBUFR files are separated out into multiple
files that
> > > need
> > > > to
> > > > > > be read into PB2NC, you can set a file window around the
valid
> time
> > > to
> > > > > > allow files to be passed into the app. More information on
that
> > > > > > functionality is here:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#using-
windows-to-find-valid-files
> > > > > >
> > > > > > If you are still having trouble getting this configured,
you
> could
> > > send
> > > > > me
> > > > > > the paths of the files you are trying to read and what
data is
> > > > contained
> > > > > in
> > > > > > them so I can give you suggestions on what to set for the
> > > > configuration.
> > > > > >
> > > > > > Thanks,
> > > > > > George
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Tue, Apr 6, 2021 at 2:17 PM Julie Prestopnik via RT <
> > > > > met_help at ucar.edu>
> > > > > > wrote:
> > > > > >
> > > > > > >
> > > > > > > Tue Apr 06 14:16:42 2021: Request 99429 was acted upon.
> > > > > > > Transaction: Given to mccabe (George McCabe) by jpresto
> > > > > > > Queue: met_help
> > > > > > > Subject: How to set time for hourly 12-24 hours
data ?
> > > > > > > Owner: mccabe
> > > > > > > Requestors: zhuming at ucar.edu
> > > > > > > Status: new
> > > > > > > Ticket <URL:
> > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429
> > > > > >
> > > > > > >
> > > > > > >
> > > > > > > This transaction appears to have no content
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > George McCabe - Software Engineer III
> > > > > > National Center for Atmospheric Research
> > > > > > Research Applications Laboratory
> > > > > > 303-497-2768
> > > > > > ---
> > > > > > My working day may not be your working day. Please do not
feel
> > > obliged
> > > > to
> > > > > > reply to this email outside of your normal working hours.
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > > --
> > > > George McCabe - Software Engineer III
> > > > National Center for Atmospheric Research
> > > > Research Applications Laboratory
> > > > 303-497-2768
> > > > ---
> > > > My working day may not be your working day. Please do not feel
> obliged
> > to
> > > > reply to this email outside of your normal working hours.
> > > >
> > > >
> > >
> > >
> >
> > --
> > George McCabe - Software Engineer III
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > 303-497-2768
> > ---
> > My working day may not be your working day. Please do not feel
obliged to
> > reply to this email outside of your normal working hours.
> >
> >
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: How to set time for hourly 12-24 hours data ?
From: George McCabe
Time: Wed Apr 14 13:56:25 2021
Hi Zhuming,
Just following up on this ticket. Were you able to get this resolved?
Let me know if you need more assistance.
Thanks,
George
On Thu Apr 08 10:04:44 2021, mccabe wrote:
> Hi Zhuming,
>
> Typically directories that contain data for an entire day are named
> with
> YYYYMMDD, so your filename template tag would be
{valid?fmt=%Y%m%d},
> however that may not always be the case, as you are seeing here.
There
> are
> a couple ways to handle this.
>
> If you know that the directory will always be YYYYMMDD with 00 at
the
> end,
> you can simply put 00 in the template instead of adding %H in the
tag,
> i.e.
>
> FCST_POINT_STAT_INPUT_TEMPLATE =
> {valid?fmt=%Y%m%d}00/wrfout_d02_{valid?fmt=%Y-%m-%d_%H}:00:00_cf.nc
>
> If the HH value can vary, for instance if there are 2 directories
> created
> per day (i.e. 2015071400 and 2015071412), then you can use the
> truncate
> keyword to handle it.
>
> If directories are created every 24 hours:
>
> FCST_POINT_STAT_INPUT_TEMPLATE =
> {valid?fmt=%Y%m%d%H?truncate=24H}/wrfout_d02_{valid?fmt=%Y-%m-
> %d_%H}:00:
> 00_cf.nc
>
> If directories are created every 12 hours:
>
> FCST_POINT_STAT_INPUT_TEMPLATE =
> {valid?fmt=%Y%m%d%H?truncate=12H}/wrfout_d02_{valid?fmt=%Y-%m-
> %d_%H}:00:
> 00_cf.nc
>
> Let me know if that doesn't work for you.
>
> Thanks,
> George
>
> On Thu, Apr 8, 2021 at 9:46 AM zhuming at ucar.edu via RT
> <met_help at ucar.edu>
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429 >
> >
> > Hi George,
> >
> > Processing Obs data using "PB2NC" are right now.
> >
> > Now I'm using "PointStat" , I set VALID_BEG = 2015071500,
VALID_END =
> > 2015071501. It looks for the forecast data 2015071500/
> > wrfout_d02_2015-07-15_00:00:00_cf.nc, and 2015071501/
> > wrfout_d02_2015-07-15_01:00:00_cf.nc.
> > But my forecast data are all put in dir of 2015071500/.
> > So how could I set FCST_POINT_STAT_INPUT_TEMPLATE to get the
right
> > fcst
> > data?
> >
> > FCST_POINT_STAT_INPUT_TEMPLATE =
> > {valid?fmt=%Y%m%d%H}/wrfout_d02_{valid?fmt=%Y-%m-
%d_%H}:00:00_cf.nc
> >
> >
> > config file: /gpfs/fs1/work/zhuming/METplus4/PointStat1.conf
> >
> > log file:
> >
/glade/scratch/zhuming/metplus4/logs/master_metplus.log.20210408085450
> >
> >
> > Thanks,
> >
> > Zhuming
> >
> > On Wed, Apr 7, 2021 at 5:43 PM George McCabe via RT
> > <met_help at ucar.edu>
> > wrote:
> >
> > > Hi Zhuming,
> > >
> > > Looking at the log output, it looks like the data is stored in
the
> > prepBUFR
> > > files differently that you may have thought:
> > >
> > > DEBUG 2: PrepBufr Time Center: 20150715_000000
> > > DEBUG 2: Searching Time Window: 20150715_030000 to
> > 20150715_040000
> > > 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80%
85%
> > > 90%
> > 95%
> > > 100%
> > >
> > > DEBUG 2: Total Messages processed = 590836
> > > DEBUG 2: Rejected based on message type = 0
> > > DEBUG 2: Rejected based on station id = 0
> > > DEBUG 2: Rejected based on valid time = 590815
> > > DEBUG 2: Rejected based on masking grid = 20
> > > DEBUG 2: Rejected based on masking polygon = 0
> > > DEBUG 2: Rejected based on elevation = 0
> > > DEBUG 2: Rejected based on pb report type = 1
> > > DEBUG 2: Rejected based on input report type = 0
> > > DEBUG 2: Rejected based on instrument type = 0
> > > DEBUG 2: Rejected based on zero observations = 0
> > > DEBUG 2: Total Messages retained = 0
> > > DEBUG 2: Total observations retained or derived = 0
> > > DEBUG 1: Obs time between *20150714_210000 and 20150715_030000*
> > > WARNING:
> > > WARNING: process_pbfile() -> No PrepBufr messages retained from
> > > file:
> > > /glade/collections/rda/data/ds337.0/prepnr/2015/
> > > prepbufr.gdas.2015071500.nr
> > > WARNING:
> > > ERROR :
> > > ERROR : write_netcdf_hdr_data() -> No PrepBufr messages
retained.
> > Nothing
> > > to write.
> > > ERROR :
> > >
> > > The prepbufr.gdas.2015071500.nr file actually contains data from
> > > 20150714_210000 to 20150715_030000, not 20150715_000000 to
> > 20150715_060000.
> > >
> > > To get the correct file for each run, you will have to shift the
> > > valid
> > time
> > > before truncating it so that it gets the correct file for each
run.
> > Instead
> > > of setting:
> > >
> > > PB2NC_INPUT_TEMPLATE =
> > >
{valid?fmt=%Y}/prepbufr.gdas.{valid?fmt=%Y%m%d%H?truncate=21600}.nr
> > >
> > > You will have to also shift forward 3 hours, i.e.:
> > >
> > > PB2NC_INPUT_TEMPLATE =
> > >
> > >
> >
{valid?fmt=%Y}/prepbufr.gdas.{valid?fmt=%Y%m%d%H?shift=3H?truncate=21600}.nr
> > >
> > > This says to add 3 hours to the current valid time before
> > > truncating to
> > the
> > > nearest 6 hours.
> > >
> > > Valid: 20150715 0Z + 3 = 3Z truncate 6H = 0Z
> > > File: prepbufr.gdas.2015071500.nr
> > >
> > > Valid: 20150715 1Z + 3 = 4Z truncate 6H = 0Z
> > > File: prepbufr.gdas.2015071500.nr
> > >
> > > Valid: 20150715 2Z + 3 = 5Z truncate 6H = 0Z
> > > File: prepbufr.gdas.2015071500.nr
> > >
> > > Valid: 20150715 3Z + 3 = 6Z truncate 6H = 6Z
> > > File: prepbufr.gdas.2015071506.nr
> > >
> > > Let me know if that doesn't work.
> > >
> > > Thanks,
> > > George
> > >
> > >
> > > On Wed, Apr 7, 2021 at 5:32 PM zhuming at ucar.edu via RT <
> > met_help at ucar.edu>
> > > wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429
>
> > > >
> > > > Hi George,
> > > >
> > > > Now makes me confused is why processed obs of several times
are
> > missing.
> > > > There has files of time at 00,01,02, but no 03,04,05; has
> > > > 06,07,08, but
> > > no
> > > > 09,10,11, has 12,13,14, no 15,16,17; has 18,19,20, but with
> > > > 21,no
> > 22,23.
> > > > From yours and mines:
> > > >
> > > > ls -l /glade/u/home/mccabe/data/out/pb2nc1/
> > > >
> > > >
> > > > -rw-r--r-- 1 mccabe ral 10141339 Apr 6 17:52
pbs.2015071500.nc
> > > >
> > > > -rw-r--r-- 1 mccabe ral 5413907 Apr 6 17:56
pbs.2015071501.nc
> > > >
> > > > -rw-r--r-- 1 mccabe ral 4923711 Apr 6 18:00
pbs.2015071502.nc
> > > >
> > > > -rw-r--r-- 1 mccabe ral 5724447 Apr 6 18:14
pbs.2015071506.nc
> > > >
> > > > -rw-r--r-- 1 mccabe ral 5280463 Apr 6 18:18
pbs.2015071507.nc
> > > >
> > > > -rw-r--r-- 1 mccabe ral 4706119 Apr 6 18:22
pbs.2015071508.nc
> > > >
> > > > -rw-r--r-- 1 mccabe ral 9878263 Apr 6 18:36
pbs.2015071512.nc
> > > >
> > > > -rw-r--r-- 1 mccabe ral 5392527 Apr 6 18:40
pbs.2015071513.nc
> > > >
> > > > -rw-r--r-- 1 mccabe ral 96 Apr 6 18:40
pbs.2015071514.nc
> > > >
> > > > -rw-r--r-- 1 mccabe ral 96 Apr 6 18:42
pbs.2015071515.nc
> > > >
> > > >
> > > > cheyenne3:/glade/scratch/zhuming/metplus4/pb2nc1>l
> > > >
> > > > total 96112
> > > >
> > > > -rw-r--r-- 1 zhuming ncar 11550172 Apr 7 15:11
pbs.2015071500.nc
> > > >
> > > > -rw-r--r-- 1 zhuming ncar 6615956 Apr 7 15:14
pbs.2015071501.nc
> > > >
> > > > -rw-r--r-- 1 zhuming ncar 5984328 Apr 7 15:18
pbs.2015071502.nc
> > > >
> > > > -rw-r--r-- 1 zhuming ncar 6980712 Apr 7 15:33
pbs.2015071506.nc
> > > >
> > > > -rw-r--r-- 1 zhuming ncar 6455824 Apr 7 15:36
pbs.2015071507.nc
> > > >
> > > > -rw-r--r-- 1 zhuming ncar 5749432 Apr 7 15:40
pbs.2015071508.nc
> > > >
> > > > -rw-r--r-- 1 zhuming ncar 11284936 Apr 7 15:55
pbs.2015071512.nc
> > > >
> > > > -rw-r--r-- 1 zhuming ncar 6600888 Apr 7 15:59
pbs.2015071513.nc
> > > >
> > > > -rw-r--r-- 1 zhuming ncar 5868168 Apr 7 16:03
pbs.2015071514.nc
> > > >
> > > > -rw-r--r-- 1 zhuming ncar 7273092 Apr 7 16:18
pbs.2015071518.nc
> > > >
> > > > -rw-r--r-- 1 zhuming ncar 6656344 Apr 7 16:22
pbs.2015071519.nc
> > > >
> > > > -rw-r--r-- 1 zhuming ncar 6109788 Apr 7 16:27
pbs.2015071520.nc
> > > >
> > > > -rw-r--r-- 1 zhuming ncar 58808 Apr 7 16:31
pbs.2015071521.nc
> > > >
> > > > -rw-r--r-- 1 zhuming ncar 11132440 Apr 7 16:43
pbs.2015071600.nc
> > > >
> > > >
> > > >
> > > > Thanks,
> > > >
> > > > Zhuming
> > > >
> > > > On Tue, Apr 6, 2021 at 6:34 PM George McCabe via RT
> > > > <met_help at ucar.edu
> > >
> > > > wrote:
> > > >
> > > > > Hi Zhuming,
> > > > >
> > > > > Thanks for the information. You will need to use the
truncate
> > > > > keyword
> > > to
> > > > > find the files, then set the valid beg/end range to 1 hour.
> > > > > This is
> > > what
> > > > I
> > > > > changed from your PB2NC1.conf config file:
> > > > >
> > > > >
> > > > > *PB2NC_VALID_BEGIN = {valid?fmt=%Y%m%d_%H}PB2NC_VALID_END =
> > > > > {valid?fmt=%Y%m%d_%H?shift=1H}*
> > > > >
> > > > > *PB2NC_INPUT_TEMPLATE =
> > > > >
{valid?fmt=%Y}/prepbufr.gdas.{valid?fmt=%Y%m%d%H?truncate=21600}.nr*
> > > > >
> > > > > My config file is on cheyenne here:
> > > > > /glade/u/home/mccabe/PB2NC1.conf
> > > > >
> > > > > I just started running it and the output is being written
here:
> > > > > /glade/u/home/mccabe/data/out/pb2nc1
> > > > >
> > > > >
> > > > > For PointStat, you will want to loop over the same times. I
> > > > > noticed
> > you
> > > > > specify message type in the VAR<n>_OPTIONS, but you do not
need
> > > > > to do
> > > > this
> > > > > since there is a POINT_STAT_MESSAGE_TYPE variable that
defines
> > > > > this.
> > > This
> > > > > is what I changed in your PointStat1.conf file:
> > > > >
> > > > > *VALID_END = 2015071600*
> > > > >
> > > > >
> > > > >
> > > > > *VALID_INCREMENT = 1HPOINT_STAT_OBS_VALID_BEG
> > =POINT_STAT_OBS_VALID_BEG
> > > > =*
> > > > >
> > > > >
> > > > > *POINT_STAT_MESSAGE_TYPE = ADPSFC*
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > *#OBS_VAR1_OPTIONS = message_type =
> > > > > ["ADPSFC"];#OBS_VAR2_OPTIONS =
> > > > > message_type = ["ADPSFC"];#OBS_VAR3_OPTIONS = message_type =
> > > > > ["ADPSFC"];#OBS_POINT_STAT_INPUT_DIR =
> > > > > /glade/scratch/zhuming/metplus4/pb2nc1*
> > > > >
> > > > > *OBS_POINT_STAT_INPUT_DIR = {OUTPUT_BASE}/pb2nc1*
> > > > >
> > > > > Or you can view my file in the same directory. I usually try
to
> > > > > test
> > > > with a
> > > > > single run time to make sure it is working properly, so I
set
> > VALID_END
> > > > > back to 2015071500. I ran it and it produced an output file,
> > > > > but
> > there
> > > > were
> > > > > no values besides the header. I checked the log file and saw
> > > > > this:
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > *DEBUG 2: Processing T_sfc(0,*,*) versus TMP/Z2, for
> > > > > observation type
> > > > > ADPSFC, over region FULL, for interpolation method BILIN(4),
> > > > > using 0
> > > > > matched pairs.DEBUG 3: Number of matched pairs = 0DEBUG 3:
> > > Observations
> > > > > processed = 393373DEBUG 3: Rejected: station id =
> > > > > 0DEBUG 3:
> > > > > Rejected: obs type = 390528DEBUG 3: Rejected: valid
time
> > =
> > > > > 0DEBUG 3: Rejected: bad obs value = 0DEBUG 3: Rejected:
off
> > > > > the
> > grid
> > > > =
> > > > > 2586DEBUG 3: Rejected: topography = 0DEBUG 3: Rejected:
> > > > > level
> > > > mismatch
> > > > > = 259DEBUG 3: Rejected: quality marker = 0DEBUG 3:
Rejected:
> > message
> > > > type
> > > > > = 0DEBUG 3: Rejected: masking region = 0DEBUG 3: Rejected:
bad
> > fcst
> > > > > value = 0DEBUG 3: Rejected: bad climo mean = 0DEBUG 3:
> > > > > Rejected:
> > bad
> > > > > climo stdev = 0DEBUG 3: Rejected: duplicates = 0DEBUG
> > > > > 2:DEBUG 2:
> > > > >
> > > > >
> > > >
> > >
> >
--------------------------------------------------------------------------------DEBUG
> > > > > 2:DEBUG 2: Processing u_10m_gr(0,*,*) versus UGRD/Z10, for
> > observation
> > > > type
> > > > > ADPSFC, over region FULL, for interpolation method BILIN(4),
> > > > > using 0
> > > > > matched pairs.DEBUG 3: Number of matched pairs = 0DEBUG 3:
> > > Observations
> > > > > processed = 393373*
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > *DEBUG 3: Rejected: station id = 0DEBUG 3: Rejected:
obs
> > > > > type
> > > > =
> > > > > 382965DEBUG 3: Rejected: valid time = 0DEBUG 3:
Rejected:
> > > > > bad
> > obs
> > > > > value = 0DEBUG 3: Rejected: off the grid = 9240DEBUG 3:
> > Rejected:
> > > > > topography = 0DEBUG 3: Rejected: level mismatch =
> > > > > 1168DEBUG 3:
> > > > > Rejected: quality marker = 0DEBUG 3: Rejected: message type
> > > > > =
> > > 0DEBUG
> > > > 3:
> > > > > Rejected: masking region = 0DEBUG 3: Rejected: bad fcst
value
> > > > > =
> > > 0DEBUG
> > > > 3:
> > > > > Rejected: bad climo mean = 0DEBUG 3: Rejected: bad climo
stdev
> > > > > =
> > > 0DEBUG
> > > > 3:
> > > > > Rejected: duplicates = 0DEBUG 2:DEBUG 2:
> > > > >
> > > > >
> > > >
> > >
> >
--------------------------------------------------------------------------------DEBUG
> > > > > 2:DEBUG 2: Processing v_10m_gr(0,*,*) versus VGRD/Z10, for
> > observation
> > > > type
> > > > > ADPSFC, over region FULL, for interpolation method BILIN(4),
> > > > > using 0
> > > > > matched pairs.DEBUG 3: Number of matched pairs = 0DEBUG 3:
> > > Observations
> > > > > processed = 393373DEBUG 3: Rejected: station id =
> > > > > 0DEBUG 3:
> > > > > Rejected: obs type = 382965DEBUG 3: Rejected: valid
time
> > =
> > > > > 0DEBUG 3: Rejected: bad obs value = 0DEBUG 3: Rejected:
off
> > > > > the
> > grid
> > > > =
> > > > > 9240DEBUG 3: Rejected: topography = 0DEBUG 3: Rejected:
> > > > > level
> > > > mismatch
> > > > > = 1168DEBUG 3: Rejected: quality marker = 0DEBUG 3:
Rejected:
> > message
> > > > > type = 0DEBUG 3: Rejected: masking region = 0DEBUG 3:
> > > > > Rejected:
> > bad
> > > > > fcst value = 0DEBUG 3: Rejected: bad climo mean = 0DEBUG
3:
> > Rejected:
> > > > bad
> > > > > climo stdev = 0*
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > *DEBUG 3: Rejected: duplicates = 0DEBUG 2:DEBUG 2:
> > > > >
> > > > >
> > > >
> > >
> >
--------------------------------------------------------------------------------DEBUG
> > > > > 2:DEBUG 1: Output file:
> > > > >
> > > > >
> > > >
> > >
> >
/glade/u/home/mccabe/data/out/point_stat1/point_stat_000000L_20150715_000000V.stat*
> > > > >
> > > > >
> > > > >
> > > > > This describes the reason why each observation was rejected.
> > > > > Some
> > > > > observations didn't match the message type you requested,
some
> > > > > were
> > not
> > > > > within the forecast grid, and some did not match the
vertical
> > > > > level.
> > > You
> > > > > will have to make adjustments to your settings so that the
data
> > matches
> > > > up
> > > > > properly. Maybe the grid you are using in the PB2NC config
file
> > > > > does
> > > not
> > > > > match the forecast grid and you are missing values. If that
is
> > > > > the
> > > case,
> > > > > you will have to reconfigure and rerun PB2NC to get output
on
> > > > > the
> > > correct
> > > > > gr5id. Maybe the observation points don't line up with the
> > > > > vertical
> > > level
> > > > > of the model fields. You can use a range of level values for
> > > > > the
> > > > > observation data to allow data within that range, i.e. Z0-
10.
> > > > >
> > > > > Let me know if you are still having issues getting this to
work
> > > > > and I
> > > can
> > > > > help out.
> > > > >
> > > > > Thanks,
> > > > > George
> > > > >
> > > > >
> > > > > On Tue, Apr 6, 2021 at 5:17 PM zhuming at ucar.edu via RT <
> > > > met_help at ucar.edu>
> > > > > wrote:
> > > > >
> > > > > >
> > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429
> > > > > > >
> > > > > >
> > > > > > Hi George,
> > > > > >
> > > > > > Thanks for your reply!
> > > > > >
> > > > > > I'm still confused, I need to output hourly obs data to do
> > > > > > hourly
> > > > > > verification.
> > > > > > The original GDAS Prepbufr data has 4 files one day,
should
> > > > > > with 6
> > > > hours
> > > > > > obs data in one file.
> > > > > >
> > > > > > After "PB2NC", I can see pbs.2015071500.nc (obs time is
00?
> > > > > > or 6
> > > times
> > > > > of
> > > > > > 00,01,02,03,04,05?), pbs.2015071506.nc...
> > > > > >
> > > > > > my config file :
> > cheyenne:/gpfs/fs1/work/zhuming/METplus4/PB2NC1.conf
> > > > > > and /gpfs/fs1/work/zhuming/METplus4/PointStat1.conf
> > > > > >
> > > > > > Thanks,
> > > > > > Zhuming
> > > > > >
> > > > > > On Tue, Apr 6, 2021 at 4:23 PM George McCabe via RT <
> > > met_help at ucar.edu
> > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Hi Zhuming,
> > > > > > >
> > > > > > > To run the tools hourly you would set the
VALID_INCREMENT
> > > > > > > value
> > for
> > > > 1H.
> > > > > > > Here is an example that runs once per hour for a day
from
> > > 2015-07-15
> > > > 0Z
> > > > > > to
> > > > > > > 2017-07-16 0Z:
> > > > > > >
> > > > > > > VALID_TIME_FMT = %Y%m%d%H
> > > > > > >
> > > > > > > VALID_BEG = 2015071500
> > > > > > >
> > > > > > > VALID_END = 2015071600
> > > > > > >
> > > > > > > VALID_INCREMENT = 1H
> > > > > > >
> > > > > > > LEAD_SEQ = 0
> > > > > > >
> > > > > > > Each run time will use the PB2NC_INPUT_TEMPLATE to find
> > > > > > > files
> > whose
> > > > > > > filenames match the time you need.
> > > > > > >
> > > > > > > To read prepBUFR observations for 12 hours, you will
want
> > > > > > > to set
> > > > > > > PB2NC_VALID_BEGIN and PB2NC_VALID_END to specify the
range
> > > > > > > around
> > > the
> > > > > > valid
> > > > > > > time to process. These values depend on what is
contained
> > > > > > > inside
> > > your
> > > > > > > prepBUFR files. If you want to use data from the valid
time
> > > > > > > of
> > the
> > > > > > filename
> > > > > > > backwards 12 hours, you would set:
> > > > > > >
> > > > > > > PB2NC_VALID_BEGIN = {valid?fmt=%Y%m%d_%H?shift=-12H }
> > > > > > > PB2NC_VALID_END = {valid?fmt=%Y%m%d_%H}
> > > > > > >
> > > > > > > For a runtime of 2015071512, this would be substituted
as:
> > > > > > >
> > > > > > > valid_beg: 20150715_00
> > > > > > > valid_end: 20150715_12
> > > > > > >
> > > > > > > If you want to create an output file for each hour in
PB2NC
> > > > > > > you
> > > would
> > > > > > need
> > > > > > > to make sure that the hour is specified in the
> > > PB2NC_OUTPUT_TEMPLATE.
> > > > > > >
> > > > > > > PointStat has similar config variables to set the valid
beg
> > > > > > > and
> > end
> > > > > range
> > > > > > > to use (POINT_STAT_OBS_VALID_BEG and
> > > > > > > POINT_STAT_OBS_VALID_END).
> > > > > > >
> > > > > > > If your prepBUFR files are separated out into multiple
> > > > > > > files that
> > > > need
> > > > > to
> > > > > > > be read into PB2NC, you can set a file window around the
> > > > > > > valid
> > time
> > > > to
> > > > > > > allow files to be passed into the app. More information
on
> > > > > > > that
> > > > > > > functionality is here:
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#using-
> > windows-to-find-valid-files
> > > > > > >
> > > > > > > If you are still having trouble getting this configured,
> > > > > > > you
> > could
> > > > send
> > > > > > me
> > > > > > > the paths of the files you are trying to read and what
data
> > > > > > > is
> > > > > contained
> > > > > > in
> > > > > > > them so I can give you suggestions on what to set for
the
> > > > > configuration.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > George
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > On Tue, Apr 6, 2021 at 2:17 PM Julie Prestopnik via RT <
> > > > > > met_help at ucar.edu>
> > > > > > > wrote:
> > > > > > >
> > > > > > > >
> > > > > > > > Tue Apr 06 14:16:42 2021: Request 99429 was acted
upon.
> > > > > > > > Transaction: Given to mccabe (George McCabe) by
jpresto
> > > > > > > > Queue: met_help
> > > > > > > > Subject: How to set time for hourly 12-24 hours
data
> > > > > > > > ?
> > > > > > > > Owner: mccabe
> > > > > > > > Requestors: zhuming at ucar.edu
> > > > > > > > Status: new
> > > > > > > > Ticket <URL:
> > > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429
> > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > This transaction appears to have no content
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > George McCabe - Software Engineer III
> > > > > > > National Center for Atmospheric Research
> > > > > > > Research Applications Laboratory
> > > > > > > 303-497-2768
> > > > > > > ---
> > > > > > > My working day may not be your working day. Please do
not
> > > > > > > feel
> > > > obliged
> > > > > to
> > > > > > > reply to this email outside of your normal working
hours.
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > > --
> > > > > George McCabe - Software Engineer III
> > > > > National Center for Atmospheric Research
> > > > > Research Applications Laboratory
> > > > > 303-497-2768
> > > > > ---
> > > > > My working day may not be your working day. Please do not
feel
> > obliged
> > > to
> > > > > reply to this email outside of your normal working hours.
> > > > >
> > > > >
> > > >
> > > >
> > >
> > > --
> > > George McCabe - Software Engineer III
> > > National Center for Atmospheric Research
> > > Research Applications Laboratory
> > > 303-497-2768
> > > ---
> > > My working day may not be your working day. Please do not feel
> > > obliged to
> > > reply to this email outside of your normal working hours.
> > >
> > >
> >
> >
------------------------------------------------
Subject: How to set time for hourly 12-24 hours data ?
From: zhuming at ucar.edu
Time: Wed Apr 14 15:17:49 2021
Hi George,
Thanks for your email, I just would like to send you for help.
I already resolved the last problem with the WRF forecast, thank you
for
your help.
I also need to verify the MPAS model forecast. These 12 hour
MAPS forecasts are stored in a one file, such as
2015071500/mpas_wrf_cf_2015071500.nc, has 12h forecasts
2015071512/mpas_wrf_cf_2015071512.nc
It seems MET is doing fine on the first 2 hours, but no matched pairs
for
later hours.
I'm not sure what's wrong with it? the time is not matched?
see the log file
/glade/scratch/zhuming/metplus4/logs/master_metplus.log.20210414145101
config file /gpfs/fs1/work/zhuming/METplus4/PointStat2.conf
Thanks,
Zhuming
On Wed, Apr 14, 2021 at 1:56 PM George McCabe via RT
<met_help at ucar.edu>
wrote:
> Hi Zhuming,
>
> Just following up on this ticket. Were you able to get this
resolved? Let
> me know if you need more assistance.
>
> Thanks,
> George
>
> On Thu Apr 08 10:04:44 2021, mccabe wrote:
> > Hi Zhuming,
> >
> > Typically directories that contain data for an entire day are
named
> > with
> > YYYYMMDD, so your filename template tag would be
{valid?fmt=%Y%m%d},
> > however that may not always be the case, as you are seeing here.
There
> > are
> > a couple ways to handle this.
> >
> > If you know that the directory will always be YYYYMMDD with 00 at
the
> > end,
> > you can simply put 00 in the template instead of adding %H in the
tag,
> > i.e.
> >
> > FCST_POINT_STAT_INPUT_TEMPLATE =
> > {valid?fmt=%Y%m%d}00/wrfout_d02_{valid?fmt=%Y-%m-
%d_%H}:00:00_cf.nc
> >
> > If the HH value can vary, for instance if there are 2 directories
> > created
> > per day (i.e. 2015071400 and 2015071412), then you can use the
> > truncate
> > keyword to handle it.
> >
> > If directories are created every 24 hours:
> >
> > FCST_POINT_STAT_INPUT_TEMPLATE =
> > {valid?fmt=%Y%m%d%H?truncate=24H}/wrfout_d02_{valid?fmt=%Y-%m-
> > %d_%H}:00:
> > 00_cf.nc
> >
> > If directories are created every 12 hours:
> >
> > FCST_POINT_STAT_INPUT_TEMPLATE =
> > {valid?fmt=%Y%m%d%H?truncate=12H}/wrfout_d02_{valid?fmt=%Y-%m-
> > %d_%H}:00:
> > 00_cf.nc
> >
> > Let me know if that doesn't work for you.
> >
> > Thanks,
> > George
> >
> > On Thu, Apr 8, 2021 at 9:46 AM zhuming at ucar.edu via RT
> > <met_help at ucar.edu>
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429 >
> > >
> > > Hi George,
> > >
> > > Processing Obs data using "PB2NC" are right now.
> > >
> > > Now I'm using "PointStat" , I set VALID_BEG = 2015071500,
VALID_END =
> > > 2015071501. It looks for the forecast data 2015071500/
> > > wrfout_d02_2015-07-15_00:00:00_cf.nc, and 2015071501/
> > > wrfout_d02_2015-07-15_01:00:00_cf.nc.
> > > But my forecast data are all put in dir of 2015071500/.
> > > So how could I set FCST_POINT_STAT_INPUT_TEMPLATE to get the
right
> > > fcst
> > > data?
> > >
> > > FCST_POINT_STAT_INPUT_TEMPLATE =
> > > {valid?fmt=%Y%m%d%H}/wrfout_d02_{valid?fmt=%Y-%m-
%d_%H}:00:00_cf.nc
> > >
> > >
> > > config file: /gpfs/fs1/work/zhuming/METplus4/PointStat1.conf
> > >
> > > log file:
> > >
/glade/scratch/zhuming/metplus4/logs/master_metplus.log.20210408085450
> > >
> > >
> > > Thanks,
> > >
> > > Zhuming
> > >
> > > On Wed, Apr 7, 2021 at 5:43 PM George McCabe via RT
> > > <met_help at ucar.edu>
> > > wrote:
> > >
> > > > Hi Zhuming,
> > > >
> > > > Looking at the log output, it looks like the data is stored in
the
> > > prepBUFR
> > > > files differently that you may have thought:
> > > >
> > > > DEBUG 2: PrepBufr Time Center: 20150715_000000
> > > > DEBUG 2: Searching Time Window: 20150715_030000 to
> > > 20150715_040000
> > > > 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80%
85%
> > > > 90%
> > > 95%
> > > > 100%
> > > >
> > > > DEBUG 2: Total Messages processed = 590836
> > > > DEBUG 2: Rejected based on message type = 0
> > > > DEBUG 2: Rejected based on station id = 0
> > > > DEBUG 2: Rejected based on valid time = 590815
> > > > DEBUG 2: Rejected based on masking grid = 20
> > > > DEBUG 2: Rejected based on masking polygon = 0
> > > > DEBUG 2: Rejected based on elevation = 0
> > > > DEBUG 2: Rejected based on pb report type = 1
> > > > DEBUG 2: Rejected based on input report type = 0
> > > > DEBUG 2: Rejected based on instrument type = 0
> > > > DEBUG 2: Rejected based on zero observations = 0
> > > > DEBUG 2: Total Messages retained = 0
> > > > DEBUG 2: Total observations retained or derived = 0
> > > > DEBUG 1: Obs time between *20150714_210000 and
20150715_030000*
> > > > WARNING:
> > > > WARNING: process_pbfile() -> No PrepBufr messages retained
from
> > > > file:
> > > > /glade/collections/rda/data/ds337.0/prepnr/2015/
> > > > prepbufr.gdas.2015071500.nr
> > > > WARNING:
> > > > ERROR :
> > > > ERROR : write_netcdf_hdr_data() -> No PrepBufr messages
retained.
> > > Nothing
> > > > to write.
> > > > ERROR :
> > > >
> > > > The prepbufr.gdas.2015071500.nr file actually contains data
from
> > > > 20150714_210000 to 20150715_030000, not 20150715_000000 to
> > > 20150715_060000.
> > > >
> > > > To get the correct file for each run, you will have to shift
the
> > > > valid
> > > time
> > > > before truncating it so that it gets the correct file for each
run.
> > > Instead
> > > > of setting:
> > > >
> > > > PB2NC_INPUT_TEMPLATE =
> > > >
{valid?fmt=%Y}/prepbufr.gdas.{valid?fmt=%Y%m%d%H?truncate=21600}.nr
> > > >
> > > > You will have to also shift forward 3 hours, i.e.:
> > > >
> > > > PB2NC_INPUT_TEMPLATE =
> > > >
> > > >
> > >
>
{valid?fmt=%Y}/prepbufr.gdas.{valid?fmt=%Y%m%d%H?shift=3H?truncate=21600}.nr
> > > >
> > > > This says to add 3 hours to the current valid time before
> > > > truncating to
> > > the
> > > > nearest 6 hours.
> > > >
> > > > Valid: 20150715 0Z + 3 = 3Z truncate 6H = 0Z
> > > > File: prepbufr.gdas.2015071500.nr
> > > >
> > > > Valid: 20150715 1Z + 3 = 4Z truncate 6H = 0Z
> > > > File: prepbufr.gdas.2015071500.nr
> > > >
> > > > Valid: 20150715 2Z + 3 = 5Z truncate 6H = 0Z
> > > > File: prepbufr.gdas.2015071500.nr
> > > >
> > > > Valid: 20150715 3Z + 3 = 6Z truncate 6H = 6Z
> > > > File: prepbufr.gdas.2015071506.nr
> > > >
> > > > Let me know if that doesn't work.
> > > >
> > > > Thanks,
> > > > George
> > > >
> > > >
> > > > On Wed, Apr 7, 2021 at 5:32 PM zhuming at ucar.edu via RT <
> > > met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > >
> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429 >
> > > > >
> > > > > Hi George,
> > > > >
> > > > > Now makes me confused is why processed obs of several times
are
> > > missing.
> > > > > There has files of time at 00,01,02, but no 03,04,05; has
> > > > > 06,07,08, but
> > > > no
> > > > > 09,10,11, has 12,13,14, no 15,16,17; has 18,19,20, but with
> > > > > 21,no
> > > 22,23.
> > > > > From yours and mines:
> > > > >
> > > > > ls -l /glade/u/home/mccabe/data/out/pb2nc1/
> > > > >
> > > > >
> > > > > -rw-r--r-- 1 mccabe ral 10141339 Apr 6 17:52
pbs.2015071500.nc
> > > > >
> > > > > -rw-r--r-- 1 mccabe ral 5413907 Apr 6 17:56
pbs.2015071501.nc
> > > > >
> > > > > -rw-r--r-- 1 mccabe ral 4923711 Apr 6 18:00
pbs.2015071502.nc
> > > > >
> > > > > -rw-r--r-- 1 mccabe ral 5724447 Apr 6 18:14
pbs.2015071506.nc
> > > > >
> > > > > -rw-r--r-- 1 mccabe ral 5280463 Apr 6 18:18
pbs.2015071507.nc
> > > > >
> > > > > -rw-r--r-- 1 mccabe ral 4706119 Apr 6 18:22
pbs.2015071508.nc
> > > > >
> > > > > -rw-r--r-- 1 mccabe ral 9878263 Apr 6 18:36
pbs.2015071512.nc
> > > > >
> > > > > -rw-r--r-- 1 mccabe ral 5392527 Apr 6 18:40
pbs.2015071513.nc
> > > > >
> > > > > -rw-r--r-- 1 mccabe ral 96 Apr 6 18:40
pbs.2015071514.nc
> > > > >
> > > > > -rw-r--r-- 1 mccabe ral 96 Apr 6 18:42
pbs.2015071515.nc
> > > > >
> > > > >
> > > > > cheyenne3:/glade/scratch/zhuming/metplus4/pb2nc1>l
> > > > >
> > > > > total 96112
> > > > >
> > > > > -rw-r--r-- 1 zhuming ncar 11550172 Apr 7 15:11
pbs.2015071500.nc
> > > > >
> > > > > -rw-r--r-- 1 zhuming ncar 6615956 Apr 7 15:14
pbs.2015071501.nc
> > > > >
> > > > > -rw-r--r-- 1 zhuming ncar 5984328 Apr 7 15:18
pbs.2015071502.nc
> > > > >
> > > > > -rw-r--r-- 1 zhuming ncar 6980712 Apr 7 15:33
pbs.2015071506.nc
> > > > >
> > > > > -rw-r--r-- 1 zhuming ncar 6455824 Apr 7 15:36
pbs.2015071507.nc
> > > > >
> > > > > -rw-r--r-- 1 zhuming ncar 5749432 Apr 7 15:40
pbs.2015071508.nc
> > > > >
> > > > > -rw-r--r-- 1 zhuming ncar 11284936 Apr 7 15:55
pbs.2015071512.nc
> > > > >
> > > > > -rw-r--r-- 1 zhuming ncar 6600888 Apr 7 15:59
pbs.2015071513.nc
> > > > >
> > > > > -rw-r--r-- 1 zhuming ncar 5868168 Apr 7 16:03
pbs.2015071514.nc
> > > > >
> > > > > -rw-r--r-- 1 zhuming ncar 7273092 Apr 7 16:18
pbs.2015071518.nc
> > > > >
> > > > > -rw-r--r-- 1 zhuming ncar 6656344 Apr 7 16:22
pbs.2015071519.nc
> > > > >
> > > > > -rw-r--r-- 1 zhuming ncar 6109788 Apr 7 16:27
pbs.2015071520.nc
> > > > >
> > > > > -rw-r--r-- 1 zhuming ncar 58808 Apr 7 16:31
pbs.2015071521.nc
> > > > >
> > > > > -rw-r--r-- 1 zhuming ncar 11132440 Apr 7 16:43
pbs.2015071600.nc
> > > > >
> > > > >
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Zhuming
> > > > >
> > > > > On Tue, Apr 6, 2021 at 6:34 PM George McCabe via RT
> > > > > <met_help at ucar.edu
> > > >
> > > > > wrote:
> > > > >
> > > > > > Hi Zhuming,
> > > > > >
> > > > > > Thanks for the information. You will need to use the
truncate
> > > > > > keyword
> > > > to
> > > > > > find the files, then set the valid beg/end range to 1
hour.
> > > > > > This is
> > > > what
> > > > > I
> > > > > > changed from your PB2NC1.conf config file:
> > > > > >
> > > > > >
> > > > > > *PB2NC_VALID_BEGIN = {valid?fmt=%Y%m%d_%H}PB2NC_VALID_END
=
> > > > > > {valid?fmt=%Y%m%d_%H?shift=1H}*
> > > > > >
> > > > > > *PB2NC_INPUT_TEMPLATE =
> > > > > >
> {valid?fmt=%Y}/prepbufr.gdas.{valid?fmt=%Y%m%d%H?truncate=21600}.nr*
> > > > > >
> > > > > > My config file is on cheyenne here:
> > > > > > /glade/u/home/mccabe/PB2NC1.conf
> > > > > >
> > > > > > I just started running it and the output is being written
here:
> > > > > > /glade/u/home/mccabe/data/out/pb2nc1
> > > > > >
> > > > > >
> > > > > > For PointStat, you will want to loop over the same times.
I
> > > > > > noticed
> > > you
> > > > > > specify message type in the VAR<n>_OPTIONS, but you do not
need
> > > > > > to do
> > > > > this
> > > > > > since there is a POINT_STAT_MESSAGE_TYPE variable that
defines
> > > > > > this.
> > > > This
> > > > > > is what I changed in your PointStat1.conf file:
> > > > > >
> > > > > > *VALID_END = 2015071600*
> > > > > >
> > > > > >
> > > > > >
> > > > > > *VALID_INCREMENT = 1HPOINT_STAT_OBS_VALID_BEG
> > > =POINT_STAT_OBS_VALID_BEG
> > > > > =*
> > > > > >
> > > > > >
> > > > > > *POINT_STAT_MESSAGE_TYPE = ADPSFC*
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > *#OBS_VAR1_OPTIONS = message_type =
> > > > > > ["ADPSFC"];#OBS_VAR2_OPTIONS =
> > > > > > message_type = ["ADPSFC"];#OBS_VAR3_OPTIONS = message_type
=
> > > > > > ["ADPSFC"];#OBS_POINT_STAT_INPUT_DIR =
> > > > > > /glade/scratch/zhuming/metplus4/pb2nc1*
> > > > > >
> > > > > > *OBS_POINT_STAT_INPUT_DIR = {OUTPUT_BASE}/pb2nc1*
> > > > > >
> > > > > > Or you can view my file in the same directory. I usually
try to
> > > > > > test
> > > > > with a
> > > > > > single run time to make sure it is working properly, so I
set
> > > VALID_END
> > > > > > back to 2015071500. I ran it and it produced an output
file,
> > > > > > but
> > > there
> > > > > were
> > > > > > no values besides the header. I checked the log file and
saw
> > > > > > this:
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > *DEBUG 2: Processing T_sfc(0,*,*) versus TMP/Z2, for
> > > > > > observation type
> > > > > > ADPSFC, over region FULL, for interpolation method
BILIN(4),
> > > > > > using 0
> > > > > > matched pairs.DEBUG 3: Number of matched pairs = 0DEBUG
3:
> > > > Observations
> > > > > > processed = 393373DEBUG 3: Rejected: station id =
> > > > > > 0DEBUG 3:
> > > > > > Rejected: obs type = 390528DEBUG 3: Rejected: valid
time
> > > =
> > > > > > 0DEBUG 3: Rejected: bad obs value = 0DEBUG 3: Rejected:
off
> > > > > > the
> > > grid
> > > > > =
> > > > > > 2586DEBUG 3: Rejected: topography = 0DEBUG 3:
Rejected:
> > > > > > level
> > > > > mismatch
> > > > > > = 259DEBUG 3: Rejected: quality marker = 0DEBUG 3:
Rejected:
> > > message
> > > > > type
> > > > > > = 0DEBUG 3: Rejected: masking region = 0DEBUG 3:
Rejected: bad
> > > fcst
> > > > > > value = 0DEBUG 3: Rejected: bad climo mean = 0DEBUG 3:
> > > > > > Rejected:
> > > bad
> > > > > > climo stdev = 0DEBUG 3: Rejected: duplicates = 0DEBUG
> > > > > > 2:DEBUG 2:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
>
--------------------------------------------------------------------------------DEBUG
> > > > > > 2:DEBUG 2: Processing u_10m_gr(0,*,*) versus UGRD/Z10, for
> > > observation
> > > > > type
> > > > > > ADPSFC, over region FULL, for interpolation method
BILIN(4),
> > > > > > using 0
> > > > > > matched pairs.DEBUG 3: Number of matched pairs = 0DEBUG
3:
> > > > Observations
> > > > > > processed = 393373*
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > *DEBUG 3: Rejected: station id = 0DEBUG 3: Rejected:
obs
> > > > > > type
> > > > > =
> > > > > > 382965DEBUG 3: Rejected: valid time = 0DEBUG 3:
Rejected:
> > > > > > bad
> > > obs
> > > > > > value = 0DEBUG 3: Rejected: off the grid = 9240DEBUG
3:
> > > Rejected:
> > > > > > topography = 0DEBUG 3: Rejected: level mismatch =
> > > > > > 1168DEBUG 3:
> > > > > > Rejected: quality marker = 0DEBUG 3: Rejected: message
type
> > > > > > =
> > > > 0DEBUG
> > > > > 3:
> > > > > > Rejected: masking region = 0DEBUG 3: Rejected: bad fcst
value
> > > > > > =
> > > > 0DEBUG
> > > > > 3:
> > > > > > Rejected: bad climo mean = 0DEBUG 3: Rejected: bad climo
stdev
> > > > > > =
> > > > 0DEBUG
> > > > > 3:
> > > > > > Rejected: duplicates = 0DEBUG 2:DEBUG 2:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
>
--------------------------------------------------------------------------------DEBUG
> > > > > > 2:DEBUG 2: Processing v_10m_gr(0,*,*) versus VGRD/Z10, for
> > > observation
> > > > > type
> > > > > > ADPSFC, over region FULL, for interpolation method
BILIN(4),
> > > > > > using 0
> > > > > > matched pairs.DEBUG 3: Number of matched pairs = 0DEBUG
3:
> > > > Observations
> > > > > > processed = 393373DEBUG 3: Rejected: station id =
> > > > > > 0DEBUG 3:
> > > > > > Rejected: obs type = 382965DEBUG 3: Rejected: valid
time
> > > =
> > > > > > 0DEBUG 3: Rejected: bad obs value = 0DEBUG 3: Rejected:
off
> > > > > > the
> > > grid
> > > > > =
> > > > > > 9240DEBUG 3: Rejected: topography = 0DEBUG 3:
Rejected:
> > > > > > level
> > > > > mismatch
> > > > > > = 1168DEBUG 3: Rejected: quality marker = 0DEBUG 3:
Rejected:
> > > message
> > > > > > type = 0DEBUG 3: Rejected: masking region = 0DEBUG 3:
> > > > > > Rejected:
> > > bad
> > > > > > fcst value = 0DEBUG 3: Rejected: bad climo mean = 0DEBUG
3:
> > > Rejected:
> > > > > bad
> > > > > > climo stdev = 0*
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > *DEBUG 3: Rejected: duplicates = 0DEBUG 2:DEBUG 2:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
>
--------------------------------------------------------------------------------DEBUG
> > > > > > 2:DEBUG 1: Output file:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
>
/glade/u/home/mccabe/data/out/point_stat1/point_stat_000000L_20150715_000000V.stat*
> > > > > >
> > > > > >
> > > > > >
> > > > > > This describes the reason why each observation was
rejected.
> > > > > > Some
> > > > > > observations didn't match the message type you requested,
some
> > > > > > were
> > > not
> > > > > > within the forecast grid, and some did not match the
vertical
> > > > > > level.
> > > > You
> > > > > > will have to make adjustments to your settings so that the
data
> > > matches
> > > > > up
> > > > > > properly. Maybe the grid you are using in the PB2NC config
file
> > > > > > does
> > > > not
> > > > > > match the forecast grid and you are missing values. If
that is
> > > > > > the
> > > > case,
> > > > > > you will have to reconfigure and rerun PB2NC to get output
on
> > > > > > the
> > > > correct
> > > > > > gr5id. Maybe the observation points don't line up with the
> > > > > > vertical
> > > > level
> > > > > > of the model fields. You can use a range of level values
for
> > > > > > the
> > > > > > observation data to allow data within that range, i.e. Z0-
10.
> > > > > >
> > > > > > Let me know if you are still having issues getting this to
work
> > > > > > and I
> > > > can
> > > > > > help out.
> > > > > >
> > > > > > Thanks,
> > > > > > George
> > > > > >
> > > > > >
> > > > > > On Tue, Apr 6, 2021 at 5:17 PM zhuming at ucar.edu via RT <
> > > > > met_help at ucar.edu>
> > > > > > wrote:
> > > > > >
> > > > > > >
> > > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429
> > > > > > > >
> > > > > > >
> > > > > > > Hi George,
> > > > > > >
> > > > > > > Thanks for your reply!
> > > > > > >
> > > > > > > I'm still confused, I need to output hourly obs data to
do
> > > > > > > hourly
> > > > > > > verification.
> > > > > > > The original GDAS Prepbufr data has 4 files one day,
should
> > > > > > > with 6
> > > > > hours
> > > > > > > obs data in one file.
> > > > > > >
> > > > > > > After "PB2NC", I can see pbs.2015071500.nc (obs time is
00?
> > > > > > > or 6
> > > > times
> > > > > > of
> > > > > > > 00,01,02,03,04,05?), pbs.2015071506.nc...
> > > > > > >
> > > > > > > my config file :
> > > cheyenne:/gpfs/fs1/work/zhuming/METplus4/PB2NC1.conf
> > > > > > > and /gpfs/fs1/work/zhuming/METplus4/PointStat1.conf
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Zhuming
> > > > > > >
> > > > > > > On Tue, Apr 6, 2021 at 4:23 PM George McCabe via RT <
> > > > met_help at ucar.edu
> > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hi Zhuming,
> > > > > > > >
> > > > > > > > To run the tools hourly you would set the
VALID_INCREMENT
> > > > > > > > value
> > > for
> > > > > 1H.
> > > > > > > > Here is an example that runs once per hour for a day
from
> > > > 2015-07-15
> > > > > 0Z
> > > > > > > to
> > > > > > > > 2017-07-16 0Z:
> > > > > > > >
> > > > > > > > VALID_TIME_FMT = %Y%m%d%H
> > > > > > > >
> > > > > > > > VALID_BEG = 2015071500
> > > > > > > >
> > > > > > > > VALID_END = 2015071600
> > > > > > > >
> > > > > > > > VALID_INCREMENT = 1H
> > > > > > > >
> > > > > > > > LEAD_SEQ = 0
> > > > > > > >
> > > > > > > > Each run time will use the PB2NC_INPUT_TEMPLATE to
find
> > > > > > > > files
> > > whose
> > > > > > > > filenames match the time you need.
> > > > > > > >
> > > > > > > > To read prepBUFR observations for 12 hours, you will
want
> > > > > > > > to set
> > > > > > > > PB2NC_VALID_BEGIN and PB2NC_VALID_END to specify the
range
> > > > > > > > around
> > > > the
> > > > > > > valid
> > > > > > > > time to process. These values depend on what is
contained
> > > > > > > > inside
> > > > your
> > > > > > > > prepBUFR files. If you want to use data from the valid
time
> > > > > > > > of
> > > the
> > > > > > > filename
> > > > > > > > backwards 12 hours, you would set:
> > > > > > > >
> > > > > > > > PB2NC_VALID_BEGIN = {valid?fmt=%Y%m%d_%H?shift=-12H }
> > > > > > > > PB2NC_VALID_END = {valid?fmt=%Y%m%d_%H}
> > > > > > > >
> > > > > > > > For a runtime of 2015071512, this would be substituted
as:
> > > > > > > >
> > > > > > > > valid_beg: 20150715_00
> > > > > > > > valid_end: 20150715_12
> > > > > > > >
> > > > > > > > If you want to create an output file for each hour in
PB2NC
> > > > > > > > you
> > > > would
> > > > > > > need
> > > > > > > > to make sure that the hour is specified in the
> > > > PB2NC_OUTPUT_TEMPLATE.
> > > > > > > >
> > > > > > > > PointStat has similar config variables to set the
valid beg
> > > > > > > > and
> > > end
> > > > > > range
> > > > > > > > to use (POINT_STAT_OBS_VALID_BEG and
> > > > > > > > POINT_STAT_OBS_VALID_END).
> > > > > > > >
> > > > > > > > If your prepBUFR files are separated out into multiple
> > > > > > > > files that
> > > > > need
> > > > > > to
> > > > > > > > be read into PB2NC, you can set a file window around
the
> > > > > > > > valid
> > > time
> > > > > to
> > > > > > > > allow files to be passed into the app. More
information on
> > > > > > > > that
> > > > > > > > functionality is here:
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
>
https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#using-
> > > windows-to-find-valid-files
> > > > > > > >
> > > > > > > > If you are still having trouble getting this
configured,
> > > > > > > > you
> > > could
> > > > > send
> > > > > > > me
> > > > > > > > the paths of the files you are trying to read and what
data
> > > > > > > > is
> > > > > > contained
> > > > > > > in
> > > > > > > > them so I can give you suggestions on what to set for
the
> > > > > > configuration.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > George
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On Tue, Apr 6, 2021 at 2:17 PM Julie Prestopnik via RT
<
> > > > > > > met_help at ucar.edu>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > >
> > > > > > > > > Tue Apr 06 14:16:42 2021: Request 99429 was acted
upon.
> > > > > > > > > Transaction: Given to mccabe (George McCabe) by
jpresto
> > > > > > > > > Queue: met_help
> > > > > > > > > Subject: How to set time for hourly 12-24 hours
data
> > > > > > > > > ?
> > > > > > > > > Owner: mccabe
> > > > > > > > > Requestors: zhuming at ucar.edu
> > > > > > > > > Status: new
> > > > > > > > > Ticket <URL:
> > > > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429
> > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > This transaction appears to have no content
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > George McCabe - Software Engineer III
> > > > > > > > National Center for Atmospheric Research
> > > > > > > > Research Applications Laboratory
> > > > > > > > 303-497-2768
> > > > > > > > ---
> > > > > > > > My working day may not be your working day. Please do
not
> > > > > > > > feel
> > > > > obliged
> > > > > > to
> > > > > > > > reply to this email outside of your normal working
hours.
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > > --
> > > > > > George McCabe - Software Engineer III
> > > > > > National Center for Atmospheric Research
> > > > > > Research Applications Laboratory
> > > > > > 303-497-2768
> > > > > > ---
> > > > > > My working day may not be your working day. Please do not
feel
> > > obliged
> > > > to
> > > > > > reply to this email outside of your normal working hours.
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > >
> > > > --
> > > > George McCabe - Software Engineer III
> > > > National Center for Atmospheric Research
> > > > Research Applications Laboratory
> > > > 303-497-2768
> > > > ---
> > > > My working day may not be your working day. Please do not feel
> > > > obliged to
> > > > reply to this email outside of your normal working hours.
> > > >
> > > >
> > >
> > >
>
>
>
>
------------------------------------------------
Subject: How to set time for hourly 12-24 hours data ?
From: George McCabe
Time: Wed Apr 14 15:41:29 2021
Hi Zhuming,
It looks like you are requesting the first time in the 12 hour file
for
each run:
FCST_VAR1_LEVELS = (0,*,*)
...
FCST_VAR2_LEVELS = (0,*,*)
The first dimension in these files is the time offset. You will need
the
correct index that corresponds to each valid time. Fortunately, MET
makes
this easy by allowing you to specify the valid time for a time
dimension
explicitly and it will find the correct data. You can set:
FCST_VAR1_LEVELS = ({valid?fmt=%Y%m%d_%H%M%S},*,*)
...
FCST_VAR2_LEVELS = ({valid?fmt=%Y%m%d_%H%M%S},*,*)
And it will substitute the valid time for each run. Let me know if you
have
any questions.
Thanks,
George
On Wed, Apr 14, 2021 at 3:18 PM zhuming at ucar.edu via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429 >
>
> Hi George,
>
> Thanks for your email, I just would like to send you for help.
>
> I already resolved the last problem with the WRF forecast, thank you
for
> your help.
>
> I also need to verify the MPAS model forecast. These 12 hour
> MAPS forecasts are stored in a one file, such as
> 2015071500/mpas_wrf_cf_2015071500.nc, has 12h forecasts
> 2015071512/mpas_wrf_cf_2015071512.nc
>
> It seems MET is doing fine on the first 2 hours, but no matched
pairs for
> later hours.
> I'm not sure what's wrong with it? the time is not matched?
>
> see the log file
>
/glade/scratch/zhuming/metplus4/logs/master_metplus.log.20210414145101
> config file /gpfs/fs1/work/zhuming/METplus4/PointStat2.conf
>
> Thanks,
> Zhuming
>
> On Wed, Apr 14, 2021 at 1:56 PM George McCabe via RT
<met_help at ucar.edu>
> wrote:
>
> > Hi Zhuming,
> >
> > Just following up on this ticket. Were you able to get this
resolved? Let
> > me know if you need more assistance.
> >
> > Thanks,
> > George
> >
> > On Thu Apr 08 10:04:44 2021, mccabe wrote:
> > > Hi Zhuming,
> > >
> > > Typically directories that contain data for an entire day are
named
> > > with
> > > YYYYMMDD, so your filename template tag would be
{valid?fmt=%Y%m%d},
> > > however that may not always be the case, as you are seeing here.
There
> > > are
> > > a couple ways to handle this.
> > >
> > > If you know that the directory will always be YYYYMMDD with 00
at the
> > > end,
> > > you can simply put 00 in the template instead of adding %H in
the tag,
> > > i.e.
> > >
> > > FCST_POINT_STAT_INPUT_TEMPLATE =
> > > {valid?fmt=%Y%m%d}00/wrfout_d02_{valid?fmt=%Y-%m-
%d_%H}:00:00_cf.nc
> > >
> > > If the HH value can vary, for instance if there are 2
directories
> > > created
> > > per day (i.e. 2015071400 and 2015071412), then you can use the
> > > truncate
> > > keyword to handle it.
> > >
> > > If directories are created every 24 hours:
> > >
> > > FCST_POINT_STAT_INPUT_TEMPLATE =
> > > {valid?fmt=%Y%m%d%H?truncate=24H}/wrfout_d02_{valid?fmt=%Y-%m-
> > > %d_%H}:00:
> > > 00_cf.nc
> > >
> > > If directories are created every 12 hours:
> > >
> > > FCST_POINT_STAT_INPUT_TEMPLATE =
> > > {valid?fmt=%Y%m%d%H?truncate=12H}/wrfout_d02_{valid?fmt=%Y-%m-
> > > %d_%H}:00:
> > > 00_cf.nc
> > >
> > > Let me know if that doesn't work for you.
> > >
> > > Thanks,
> > > George
> > >
> > > On Thu, Apr 8, 2021 at 9:46 AM zhuming at ucar.edu via RT
> > > <met_help at ucar.edu>
> > > wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429
>
> > > >
> > > > Hi George,
> > > >
> > > > Processing Obs data using "PB2NC" are right now.
> > > >
> > > > Now I'm using "PointStat" , I set VALID_BEG = 2015071500,
VALID_END =
> > > > 2015071501. It looks for the forecast data 2015071500/
> > > > wrfout_d02_2015-07-15_00:00:00_cf.nc, and 2015071501/
> > > > wrfout_d02_2015-07-15_01:00:00_cf.nc.
> > > > But my forecast data are all put in dir of 2015071500/.
> > > > So how could I set FCST_POINT_STAT_INPUT_TEMPLATE to get the
right
> > > > fcst
> > > > data?
> > > >
> > > > FCST_POINT_STAT_INPUT_TEMPLATE =
> > > > {valid?fmt=%Y%m%d%H}/wrfout_d02_{valid?fmt=%Y-%m-
%d_%H}:00:00_cf.nc
> > > >
> > > >
> > > > config file: /gpfs/fs1/work/zhuming/METplus4/PointStat1.conf
> > > >
> > > > log file:
> > > >
>
/glade/scratch/zhuming/metplus4/logs/master_metplus.log.20210408085450
> > > >
> > > >
> > > > Thanks,
> > > >
> > > > Zhuming
> > > >
> > > > On Wed, Apr 7, 2021 at 5:43 PM George McCabe via RT
> > > > <met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > > Hi Zhuming,
> > > > >
> > > > > Looking at the log output, it looks like the data is stored
in the
> > > > prepBUFR
> > > > > files differently that you may have thought:
> > > > >
> > > > > DEBUG 2: PrepBufr Time Center: 20150715_000000
> > > > > DEBUG 2: Searching Time Window: 20150715_030000 to
> > > > 20150715_040000
> > > > > 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75%
80% 85%
> > > > > 90%
> > > > 95%
> > > > > 100%
> > > > >
> > > > > DEBUG 2: Total Messages processed = 590836
> > > > > DEBUG 2: Rejected based on message type = 0
> > > > > DEBUG 2: Rejected based on station id = 0
> > > > > DEBUG 2: Rejected based on valid time = 590815
> > > > > DEBUG 2: Rejected based on masking grid = 20
> > > > > DEBUG 2: Rejected based on masking polygon = 0
> > > > > DEBUG 2: Rejected based on elevation = 0
> > > > > DEBUG 2: Rejected based on pb report type = 1
> > > > > DEBUG 2: Rejected based on input report type = 0
> > > > > DEBUG 2: Rejected based on instrument type = 0
> > > > > DEBUG 2: Rejected based on zero observations = 0
> > > > > DEBUG 2: Total Messages retained = 0
> > > > > DEBUG 2: Total observations retained or derived = 0
> > > > > DEBUG 1: Obs time between *20150714_210000 and
20150715_030000*
> > > > > WARNING:
> > > > > WARNING: process_pbfile() -> No PrepBufr messages retained
from
> > > > > file:
> > > > > /glade/collections/rda/data/ds337.0/prepnr/2015/
> > > > > prepbufr.gdas.2015071500.nr
> > > > > WARNING:
> > > > > ERROR :
> > > > > ERROR : write_netcdf_hdr_data() -> No PrepBufr messages
retained.
> > > > Nothing
> > > > > to write.
> > > > > ERROR :
> > > > >
> > > > > The prepbufr.gdas.2015071500.nr file actually contains data
from
> > > > > 20150714_210000 to 20150715_030000, not 20150715_000000 to
> > > > 20150715_060000.
> > > > >
> > > > > To get the correct file for each run, you will have to shift
the
> > > > > valid
> > > > time
> > > > > before truncating it so that it gets the correct file for
each run.
> > > > Instead
> > > > > of setting:
> > > > >
> > > > > PB2NC_INPUT_TEMPLATE =
> > > > >
{valid?fmt=%Y}/prepbufr.gdas.{valid?fmt=%Y%m%d%H?truncate=21600}.nr
> > > > >
> > > > > You will have to also shift forward 3 hours, i.e.:
> > > > >
> > > > > PB2NC_INPUT_TEMPLATE =
> > > > >
> > > > >
> > > >
> >
>
{valid?fmt=%Y}/prepbufr.gdas.{valid?fmt=%Y%m%d%H?shift=3H?truncate=21600}.nr
> > > > >
> > > > > This says to add 3 hours to the current valid time before
> > > > > truncating to
> > > > the
> > > > > nearest 6 hours.
> > > > >
> > > > > Valid: 20150715 0Z + 3 = 3Z truncate 6H = 0Z
> > > > > File: prepbufr.gdas.2015071500.nr
> > > > >
> > > > > Valid: 20150715 1Z + 3 = 4Z truncate 6H = 0Z
> > > > > File: prepbufr.gdas.2015071500.nr
> > > > >
> > > > > Valid: 20150715 2Z + 3 = 5Z truncate 6H = 0Z
> > > > > File: prepbufr.gdas.2015071500.nr
> > > > >
> > > > > Valid: 20150715 3Z + 3 = 6Z truncate 6H = 6Z
> > > > > File: prepbufr.gdas.2015071506.nr
> > > > >
> > > > > Let me know if that doesn't work.
> > > > >
> > > > > Thanks,
> > > > > George
> > > > >
> > > > >
> > > > > On Wed, Apr 7, 2021 at 5:32 PM zhuming at ucar.edu via RT <
> > > > met_help at ucar.edu>
> > > > > wrote:
> > > > >
> > > > > >
> > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429 >
> > > > > >
> > > > > > Hi George,
> > > > > >
> > > > > > Now makes me confused is why processed obs of several
times are
> > > > missing.
> > > > > > There has files of time at 00,01,02, but no 03,04,05; has
> > > > > > 06,07,08, but
> > > > > no
> > > > > > 09,10,11, has 12,13,14, no 15,16,17; has 18,19,20, but
with
> > > > > > 21,no
> > > > 22,23.
> > > > > > From yours and mines:
> > > > > >
> > > > > > ls -l /glade/u/home/mccabe/data/out/pb2nc1/
> > > > > >
> > > > > >
> > > > > > -rw-r--r-- 1 mccabe ral 10141339 Apr 6 17:52
pbs.2015071500.nc
> > > > > >
> > > > > > -rw-r--r-- 1 mccabe ral 5413907 Apr 6 17:56
pbs.2015071501.nc
> > > > > >
> > > > > > -rw-r--r-- 1 mccabe ral 4923711 Apr 6 18:00
pbs.2015071502.nc
> > > > > >
> > > > > > -rw-r--r-- 1 mccabe ral 5724447 Apr 6 18:14
pbs.2015071506.nc
> > > > > >
> > > > > > -rw-r--r-- 1 mccabe ral 5280463 Apr 6 18:18
pbs.2015071507.nc
> > > > > >
> > > > > > -rw-r--r-- 1 mccabe ral 4706119 Apr 6 18:22
pbs.2015071508.nc
> > > > > >
> > > > > > -rw-r--r-- 1 mccabe ral 9878263 Apr 6 18:36
pbs.2015071512.nc
> > > > > >
> > > > > > -rw-r--r-- 1 mccabe ral 5392527 Apr 6 18:40
pbs.2015071513.nc
> > > > > >
> > > > > > -rw-r--r-- 1 mccabe ral 96 Apr 6 18:40
pbs.2015071514.nc
> > > > > >
> > > > > > -rw-r--r-- 1 mccabe ral 96 Apr 6 18:42
pbs.2015071515.nc
> > > > > >
> > > > > >
> > > > > > cheyenne3:/glade/scratch/zhuming/metplus4/pb2nc1>l
> > > > > >
> > > > > > total 96112
> > > > > >
> > > > > > -rw-r--r-- 1 zhuming ncar 11550172 Apr 7 15:11
> pbs.2015071500.nc
> > > > > >
> > > > > > -rw-r--r-- 1 zhuming ncar 6615956 Apr 7 15:14
> pbs.2015071501.nc
> > > > > >
> > > > > > -rw-r--r-- 1 zhuming ncar 5984328 Apr 7 15:18
> pbs.2015071502.nc
> > > > > >
> > > > > > -rw-r--r-- 1 zhuming ncar 6980712 Apr 7 15:33
> pbs.2015071506.nc
> > > > > >
> > > > > > -rw-r--r-- 1 zhuming ncar 6455824 Apr 7 15:36
> pbs.2015071507.nc
> > > > > >
> > > > > > -rw-r--r-- 1 zhuming ncar 5749432 Apr 7 15:40
> pbs.2015071508.nc
> > > > > >
> > > > > > -rw-r--r-- 1 zhuming ncar 11284936 Apr 7 15:55
> pbs.2015071512.nc
> > > > > >
> > > > > > -rw-r--r-- 1 zhuming ncar 6600888 Apr 7 15:59
> pbs.2015071513.nc
> > > > > >
> > > > > > -rw-r--r-- 1 zhuming ncar 5868168 Apr 7 16:03
> pbs.2015071514.nc
> > > > > >
> > > > > > -rw-r--r-- 1 zhuming ncar 7273092 Apr 7 16:18
> pbs.2015071518.nc
> > > > > >
> > > > > > -rw-r--r-- 1 zhuming ncar 6656344 Apr 7 16:22
> pbs.2015071519.nc
> > > > > >
> > > > > > -rw-r--r-- 1 zhuming ncar 6109788 Apr 7 16:27
> pbs.2015071520.nc
> > > > > >
> > > > > > -rw-r--r-- 1 zhuming ncar 58808 Apr 7 16:31
> pbs.2015071521.nc
> > > > > >
> > > > > > -rw-r--r-- 1 zhuming ncar 11132440 Apr 7 16:43
> pbs.2015071600.nc
> > > > > >
> > > > > >
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Zhuming
> > > > > >
> > > > > > On Tue, Apr 6, 2021 at 6:34 PM George McCabe via RT
> > > > > > <met_help at ucar.edu
> > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Hi Zhuming,
> > > > > > >
> > > > > > > Thanks for the information. You will need to use the
truncate
> > > > > > > keyword
> > > > > to
> > > > > > > find the files, then set the valid beg/end range to 1
hour.
> > > > > > > This is
> > > > > what
> > > > > > I
> > > > > > > changed from your PB2NC1.conf config file:
> > > > > > >
> > > > > > >
> > > > > > > *PB2NC_VALID_BEGIN =
{valid?fmt=%Y%m%d_%H}PB2NC_VALID_END =
> > > > > > > {valid?fmt=%Y%m%d_%H?shift=1H}*
> > > > > > >
> > > > > > > *PB2NC_INPUT_TEMPLATE =
> > > > > > >
> >
{valid?fmt=%Y}/prepbufr.gdas.{valid?fmt=%Y%m%d%H?truncate=21600}.nr*
> > > > > > >
> > > > > > > My config file is on cheyenne here:
> > > > > > > /glade/u/home/mccabe/PB2NC1.conf
> > > > > > >
> > > > > > > I just started running it and the output is being
written here:
> > > > > > > /glade/u/home/mccabe/data/out/pb2nc1
> > > > > > >
> > > > > > >
> > > > > > > For PointStat, you will want to loop over the same
times. I
> > > > > > > noticed
> > > > you
> > > > > > > specify message type in the VAR<n>_OPTIONS, but you do
not need
> > > > > > > to do
> > > > > > this
> > > > > > > since there is a POINT_STAT_MESSAGE_TYPE variable that
defines
> > > > > > > this.
> > > > > This
> > > > > > > is what I changed in your PointStat1.conf file:
> > > > > > >
> > > > > > > *VALID_END = 2015071600*
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > *VALID_INCREMENT = 1HPOINT_STAT_OBS_VALID_BEG
> > > > =POINT_STAT_OBS_VALID_BEG
> > > > > > =*
> > > > > > >
> > > > > > >
> > > > > > > *POINT_STAT_MESSAGE_TYPE = ADPSFC*
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > *#OBS_VAR1_OPTIONS = message_type =
> > > > > > > ["ADPSFC"];#OBS_VAR2_OPTIONS =
> > > > > > > message_type = ["ADPSFC"];#OBS_VAR3_OPTIONS =
message_type =
> > > > > > > ["ADPSFC"];#OBS_POINT_STAT_INPUT_DIR =
> > > > > > > /glade/scratch/zhuming/metplus4/pb2nc1*
> > > > > > >
> > > > > > > *OBS_POINT_STAT_INPUT_DIR = {OUTPUT_BASE}/pb2nc1*
> > > > > > >
> > > > > > > Or you can view my file in the same directory. I usually
try to
> > > > > > > test
> > > > > > with a
> > > > > > > single run time to make sure it is working properly, so
I set
> > > > VALID_END
> > > > > > > back to 2015071500. I ran it and it produced an output
file,
> > > > > > > but
> > > > there
> > > > > > were
> > > > > > > no values besides the header. I checked the log file and
saw
> > > > > > > this:
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > *DEBUG 2: Processing T_sfc(0,*,*) versus TMP/Z2, for
> > > > > > > observation type
> > > > > > > ADPSFC, over region FULL, for interpolation method
BILIN(4),
> > > > > > > using 0
> > > > > > > matched pairs.DEBUG 3: Number of matched pairs =
0DEBUG 3:
> > > > > Observations
> > > > > > > processed = 393373DEBUG 3: Rejected: station id
=
> > > > > > > 0DEBUG 3:
> > > > > > > Rejected: obs type = 390528DEBUG 3: Rejected:
valid time
> > > > =
> > > > > > > 0DEBUG 3: Rejected: bad obs value = 0DEBUG 3:
Rejected: off
> > > > > > > the
> > > > grid
> > > > > > =
> > > > > > > 2586DEBUG 3: Rejected: topography = 0DEBUG 3:
Rejected:
> > > > > > > level
> > > > > > mismatch
> > > > > > > = 259DEBUG 3: Rejected: quality marker = 0DEBUG 3:
Rejected:
> > > > message
> > > > > > type
> > > > > > > = 0DEBUG 3: Rejected: masking region = 0DEBUG 3:
Rejected: bad
> > > > fcst
> > > > > > > value = 0DEBUG 3: Rejected: bad climo mean = 0DEBUG 3:
> > > > > > > Rejected:
> > > > bad
> > > > > > > climo stdev = 0DEBUG 3: Rejected: duplicates =
0DEBUG
> > > > > > > 2:DEBUG 2:
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> >
>
--------------------------------------------------------------------------------DEBUG
> > > > > > > 2:DEBUG 2: Processing u_10m_gr(0,*,*) versus UGRD/Z10,
for
> > > > observation
> > > > > > type
> > > > > > > ADPSFC, over region FULL, for interpolation method
BILIN(4),
> > > > > > > using 0
> > > > > > > matched pairs.DEBUG 3: Number of matched pairs =
0DEBUG 3:
> > > > > Observations
> > > > > > > processed = 393373*
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > *DEBUG 3: Rejected: station id = 0DEBUG 3:
Rejected: obs
> > > > > > > type
> > > > > > =
> > > > > > > 382965DEBUG 3: Rejected: valid time = 0DEBUG 3:
Rejected:
> > > > > > > bad
> > > > obs
> > > > > > > value = 0DEBUG 3: Rejected: off the grid =
9240DEBUG 3:
> > > > Rejected:
> > > > > > > topography = 0DEBUG 3: Rejected: level mismatch =
> > > > > > > 1168DEBUG 3:
> > > > > > > Rejected: quality marker = 0DEBUG 3: Rejected: message
type
> > > > > > > =
> > > > > 0DEBUG
> > > > > > 3:
> > > > > > > Rejected: masking region = 0DEBUG 3: Rejected: bad fcst
value
> > > > > > > =
> > > > > 0DEBUG
> > > > > > 3:
> > > > > > > Rejected: bad climo mean = 0DEBUG 3: Rejected: bad
climo stdev
> > > > > > > =
> > > > > 0DEBUG
> > > > > > 3:
> > > > > > > Rejected: duplicates = 0DEBUG 2:DEBUG 2:
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> >
>
--------------------------------------------------------------------------------DEBUG
> > > > > > > 2:DEBUG 2: Processing v_10m_gr(0,*,*) versus VGRD/Z10,
for
> > > > observation
> > > > > > type
> > > > > > > ADPSFC, over region FULL, for interpolation method
BILIN(4),
> > > > > > > using 0
> > > > > > > matched pairs.DEBUG 3: Number of matched pairs =
0DEBUG 3:
> > > > > Observations
> > > > > > > processed = 393373DEBUG 3: Rejected: station id
=
> > > > > > > 0DEBUG 3:
> > > > > > > Rejected: obs type = 382965DEBUG 3: Rejected:
valid time
> > > > =
> > > > > > > 0DEBUG 3: Rejected: bad obs value = 0DEBUG 3:
Rejected: off
> > > > > > > the
> > > > grid
> > > > > > =
> > > > > > > 9240DEBUG 3: Rejected: topography = 0DEBUG 3:
Rejected:
> > > > > > > level
> > > > > > mismatch
> > > > > > > = 1168DEBUG 3: Rejected: quality marker = 0DEBUG 3:
Rejected:
> > > > message
> > > > > > > type = 0DEBUG 3: Rejected: masking region = 0DEBUG
3:
> > > > > > > Rejected:
> > > > bad
> > > > > > > fcst value = 0DEBUG 3: Rejected: bad climo mean =
0DEBUG 3:
> > > > Rejected:
> > > > > > bad
> > > > > > > climo stdev = 0*
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > *DEBUG 3: Rejected: duplicates = 0DEBUG 2:DEBUG 2:
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> >
>
--------------------------------------------------------------------------------DEBUG
> > > > > > > 2:DEBUG 1: Output file:
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> >
>
/glade/u/home/mccabe/data/out/point_stat1/point_stat_000000L_20150715_000000V.stat*
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > This describes the reason why each observation was
rejected.
> > > > > > > Some
> > > > > > > observations didn't match the message type you
requested, some
> > > > > > > were
> > > > not
> > > > > > > within the forecast grid, and some did not match the
vertical
> > > > > > > level.
> > > > > You
> > > > > > > will have to make adjustments to your settings so that
the data
> > > > matches
> > > > > > up
> > > > > > > properly. Maybe the grid you are using in the PB2NC
config file
> > > > > > > does
> > > > > not
> > > > > > > match the forecast grid and you are missing values. If
that is
> > > > > > > the
> > > > > case,
> > > > > > > you will have to reconfigure and rerun PB2NC to get
output on
> > > > > > > the
> > > > > correct
> > > > > > > gr5id. Maybe the observation points don't line up with
the
> > > > > > > vertical
> > > > > level
> > > > > > > of the model fields. You can use a range of level values
for
> > > > > > > the
> > > > > > > observation data to allow data within that range, i.e.
Z0-10.
> > > > > > >
> > > > > > > Let me know if you are still having issues getting this
to work
> > > > > > > and I
> > > > > can
> > > > > > > help out.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > George
> > > > > > >
> > > > > > >
> > > > > > > On Tue, Apr 6, 2021 at 5:17 PM zhuming at ucar.edu via RT <
> > > > > > met_help at ucar.edu>
> > > > > > > wrote:
> > > > > > >
> > > > > > > >
> > > > > > > > <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429
> > > > > > > > >
> > > > > > > >
> > > > > > > > Hi George,
> > > > > > > >
> > > > > > > > Thanks for your reply!
> > > > > > > >
> > > > > > > > I'm still confused, I need to output hourly obs data
to do
> > > > > > > > hourly
> > > > > > > > verification.
> > > > > > > > The original GDAS Prepbufr data has 4 files one day,
should
> > > > > > > > with 6
> > > > > > hours
> > > > > > > > obs data in one file.
> > > > > > > >
> > > > > > > > After "PB2NC", I can see pbs.2015071500.nc (obs time
is 00?
> > > > > > > > or 6
> > > > > times
> > > > > > > of
> > > > > > > > 00,01,02,03,04,05?), pbs.2015071506.nc...
> > > > > > > >
> > > > > > > > my config file :
> > > > cheyenne:/gpfs/fs1/work/zhuming/METplus4/PB2NC1.conf
> > > > > > > > and /gpfs/fs1/work/zhuming/METplus4/PointStat1.conf
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Zhuming
> > > > > > > >
> > > > > > > > On Tue, Apr 6, 2021 at 4:23 PM George McCabe via RT <
> > > > > met_help at ucar.edu
> > > > > > >
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi Zhuming,
> > > > > > > > >
> > > > > > > > > To run the tools hourly you would set the
VALID_INCREMENT
> > > > > > > > > value
> > > > for
> > > > > > 1H.
> > > > > > > > > Here is an example that runs once per hour for a day
from
> > > > > 2015-07-15
> > > > > > 0Z
> > > > > > > > to
> > > > > > > > > 2017-07-16 0Z:
> > > > > > > > >
> > > > > > > > > VALID_TIME_FMT = %Y%m%d%H
> > > > > > > > >
> > > > > > > > > VALID_BEG = 2015071500
> > > > > > > > >
> > > > > > > > > VALID_END = 2015071600
> > > > > > > > >
> > > > > > > > > VALID_INCREMENT = 1H
> > > > > > > > >
> > > > > > > > > LEAD_SEQ = 0
> > > > > > > > >
> > > > > > > > > Each run time will use the PB2NC_INPUT_TEMPLATE to
find
> > > > > > > > > files
> > > > whose
> > > > > > > > > filenames match the time you need.
> > > > > > > > >
> > > > > > > > > To read prepBUFR observations for 12 hours, you will
want
> > > > > > > > > to set
> > > > > > > > > PB2NC_VALID_BEGIN and PB2NC_VALID_END to specify the
range
> > > > > > > > > around
> > > > > the
> > > > > > > > valid
> > > > > > > > > time to process. These values depend on what is
contained
> > > > > > > > > inside
> > > > > your
> > > > > > > > > prepBUFR files. If you want to use data from the
valid time
> > > > > > > > > of
> > > > the
> > > > > > > > filename
> > > > > > > > > backwards 12 hours, you would set:
> > > > > > > > >
> > > > > > > > > PB2NC_VALID_BEGIN = {valid?fmt=%Y%m%d_%H?shift=-12H
}
> > > > > > > > > PB2NC_VALID_END = {valid?fmt=%Y%m%d_%H}
> > > > > > > > >
> > > > > > > > > For a runtime of 2015071512, this would be
substituted as:
> > > > > > > > >
> > > > > > > > > valid_beg: 20150715_00
> > > > > > > > > valid_end: 20150715_12
> > > > > > > > >
> > > > > > > > > If you want to create an output file for each hour
in PB2NC
> > > > > > > > > you
> > > > > would
> > > > > > > > need
> > > > > > > > > to make sure that the hour is specified in the
> > > > > PB2NC_OUTPUT_TEMPLATE.
> > > > > > > > >
> > > > > > > > > PointStat has similar config variables to set the
valid beg
> > > > > > > > > and
> > > > end
> > > > > > > range
> > > > > > > > > to use (POINT_STAT_OBS_VALID_BEG and
> > > > > > > > > POINT_STAT_OBS_VALID_END).
> > > > > > > > >
> > > > > > > > > If your prepBUFR files are separated out into
multiple
> > > > > > > > > files that
> > > > > > need
> > > > > > > to
> > > > > > > > > be read into PB2NC, you can set a file window around
the
> > > > > > > > > valid
> > > > time
> > > > > > to
> > > > > > > > > allow files to be passed into the app. More
information on
> > > > > > > > > that
> > > > > > > > > functionality is here:
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> >
>
https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#using-
> > > > windows-to-find-valid-files
> > > > > > > > >
> > > > > > > > > If you are still having trouble getting this
configured,
> > > > > > > > > you
> > > > could
> > > > > > send
> > > > > > > > me
> > > > > > > > > the paths of the files you are trying to read and
what data
> > > > > > > > > is
> > > > > > > contained
> > > > > > > > in
> > > > > > > > > them so I can give you suggestions on what to set
for the
> > > > > > > configuration.
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > George
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Tue, Apr 6, 2021 at 2:17 PM Julie Prestopnik via
RT <
> > > > > > > > met_help at ucar.edu>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Tue Apr 06 14:16:42 2021: Request 99429 was acted
upon.
> > > > > > > > > > Transaction: Given to mccabe (George McCabe) by
jpresto
> > > > > > > > > > Queue: met_help
> > > > > > > > > > Subject: How to set time for hourly 12-24
hours data
> > > > > > > > > > ?
> > > > > > > > > > Owner: mccabe
> > > > > > > > > > Requestors: zhuming at ucar.edu
> > > > > > > > > > Status: new
> > > > > > > > > > Ticket <URL:
> > > > > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > This transaction appears to have no content
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > George McCabe - Software Engineer III
> > > > > > > > > National Center for Atmospheric Research
> > > > > > > > > Research Applications Laboratory
> > > > > > > > > 303-497-2768
> > > > > > > > > ---
> > > > > > > > > My working day may not be your working day. Please
do not
> > > > > > > > > feel
> > > > > > obliged
> > > > > > > to
> > > > > > > > > reply to this email outside of your normal working
hours.
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > George McCabe - Software Engineer III
> > > > > > > National Center for Atmospheric Research
> > > > > > > Research Applications Laboratory
> > > > > > > 303-497-2768
> > > > > > > ---
> > > > > > > My working day may not be your working day. Please do
not feel
> > > > obliged
> > > > > to
> > > > > > > reply to this email outside of your normal working
hours.
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > >
> > > > > --
> > > > > George McCabe - Software Engineer III
> > > > > National Center for Atmospheric Research
> > > > > Research Applications Laboratory
> > > > > 303-497-2768
> > > > > ---
> > > > > My working day may not be your working day. Please do not
feel
> > > > > obliged to
> > > > > reply to this email outside of your normal working hours.
> > > > >
> > > > >
> > > >
> > > >
> >
> >
> >
> >
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: How to set time for hourly 12-24 hours data ?
From: zhuming at ucar.edu
Time: Wed Apr 14 16:13:51 2021
Hi George,
It still can't find the correct time. I tried
FCST_VAR1_LEVELS = (*,*,*)
FCST_VAR1_LEVELS = ({valid?fmt=%Y%m%d_%H%M%S},*,*)
FCST_VAR1_LEVELS = ({valid?fmt=%Y%m%d_%H%M},*,*)
FCST_VAR1_LEVELS = ({valid?fmt=%Y%m%d%H%M},*,*)
Thanks!
Zhuming
On Wed, Apr 14, 2021 at 3:41 PM George McCabe via RT
<met_help at ucar.edu>
wrote:
> Hi Zhuming,
>
> It looks like you are requesting the first time in the 12 hour file
for
> each run:
>
> FCST_VAR1_LEVELS = (0,*,*)
> ...
> FCST_VAR2_LEVELS = (0,*,*)
>
> The first dimension in these files is the time offset. You will need
the
> correct index that corresponds to each valid time. Fortunately, MET
makes
> this easy by allowing you to specify the valid time for a time
dimension
> explicitly and it will find the correct data. You can set:
>
> FCST_VAR1_LEVELS = ({valid?fmt=%Y%m%d_%H%M%S},*,*)
> ...
> FCST_VAR2_LEVELS = ({valid?fmt=%Y%m%d_%H%M%S},*,*)
>
> And it will substitute the valid time for each run. Let me know if
you have
> any questions.
>
> Thanks,
> George
>
> On Wed, Apr 14, 2021 at 3:18 PM zhuming at ucar.edu via RT
<met_help at ucar.edu
> >
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429 >
> >
> > Hi George,
> >
> > Thanks for your email, I just would like to send you for help.
> >
> > I already resolved the last problem with the WRF forecast, thank
you for
> > your help.
> >
> > I also need to verify the MPAS model forecast. These 12 hour
> > MAPS forecasts are stored in a one file, such as
> > 2015071500/mpas_wrf_cf_2015071500.nc, has 12h forecasts
> > 2015071512/mpas_wrf_cf_2015071512.nc
> >
> > It seems MET is doing fine on the first 2 hours, but no matched
pairs for
> > later hours.
> > I'm not sure what's wrong with it? the time is not matched?
> >
> > see the log file
> >
/glade/scratch/zhuming/metplus4/logs/master_metplus.log.20210414145101
> > config file /gpfs/fs1/work/zhuming/METplus4/PointStat2.conf
> >
> > Thanks,
> > Zhuming
> >
> > On Wed, Apr 14, 2021 at 1:56 PM George McCabe via RT
<met_help at ucar.edu>
> > wrote:
> >
> > > Hi Zhuming,
> > >
> > > Just following up on this ticket. Were you able to get this
resolved?
> Let
> > > me know if you need more assistance.
> > >
> > > Thanks,
> > > George
> > >
> > > On Thu Apr 08 10:04:44 2021, mccabe wrote:
> > > > Hi Zhuming,
> > > >
> > > > Typically directories that contain data for an entire day are
named
> > > > with
> > > > YYYYMMDD, so your filename template tag would be
{valid?fmt=%Y%m%d},
> > > > however that may not always be the case, as you are seeing
here.
> There
> > > > are
> > > > a couple ways to handle this.
> > > >
> > > > If you know that the directory will always be YYYYMMDD with 00
at the
> > > > end,
> > > > you can simply put 00 in the template instead of adding %H in
the
> tag,
> > > > i.e.
> > > >
> > > > FCST_POINT_STAT_INPUT_TEMPLATE =
> > > > {valid?fmt=%Y%m%d}00/wrfout_d02_{valid?fmt=%Y-%m-
%d_%H}:00:00_cf.nc
> > > >
> > > > If the HH value can vary, for instance if there are 2
directories
> > > > created
> > > > per day (i.e. 2015071400 and 2015071412), then you can use
the
> > > > truncate
> > > > keyword to handle it.
> > > >
> > > > If directories are created every 24 hours:
> > > >
> > > > FCST_POINT_STAT_INPUT_TEMPLATE =
> > > > {valid?fmt=%Y%m%d%H?truncate=24H}/wrfout_d02_{valid?fmt=%Y-%m-
> > > > %d_%H}:00:
> > > > 00_cf.nc
> > > >
> > > > If directories are created every 12 hours:
> > > >
> > > > FCST_POINT_STAT_INPUT_TEMPLATE =
> > > > {valid?fmt=%Y%m%d%H?truncate=12H}/wrfout_d02_{valid?fmt=%Y-%m-
> > > > %d_%H}:00:
> > > > 00_cf.nc
> > > >
> > > > Let me know if that doesn't work for you.
> > > >
> > > > Thanks,
> > > > George
> > > >
> > > > On Thu, Apr 8, 2021 at 9:46 AM zhuming at ucar.edu via RT
> > > > <met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > >
> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429 >
> > > > >
> > > > > Hi George,
> > > > >
> > > > > Processing Obs data using "PB2NC" are right now.
> > > > >
> > > > > Now I'm using "PointStat" , I set VALID_BEG = 2015071500,
> VALID_END =
> > > > > 2015071501. It looks for the forecast data 2015071500/
> > > > > wrfout_d02_2015-07-15_00:00:00_cf.nc, and 2015071501/
> > > > > wrfout_d02_2015-07-15_01:00:00_cf.nc.
> > > > > But my forecast data are all put in dir of 2015071500/.
> > > > > So how could I set FCST_POINT_STAT_INPUT_TEMPLATE to get
the right
> > > > > fcst
> > > > > data?
> > > > >
> > > > > FCST_POINT_STAT_INPUT_TEMPLATE =
> > > > > {valid?fmt=%Y%m%d%H}/wrfout_d02_{valid?fmt=%Y-%m-%d_%H}:00:
> 00_cf.nc
> > > > >
> > > > >
> > > > > config file:
/gpfs/fs1/work/zhuming/METplus4/PointStat1.conf
> > > > >
> > > > > log file:
> > > > >
> >
/glade/scratch/zhuming/metplus4/logs/master_metplus.log.20210408085450
> > > > >
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Zhuming
> > > > >
> > > > > On Wed, Apr 7, 2021 at 5:43 PM George McCabe via RT
> > > > > <met_help at ucar.edu>
> > > > > wrote:
> > > > >
> > > > > > Hi Zhuming,
> > > > > >
> > > > > > Looking at the log output, it looks like the data is
stored in
> the
> > > > > prepBUFR
> > > > > > files differently that you may have thought:
> > > > > >
> > > > > > DEBUG 2: PrepBufr Time Center: 20150715_000000
> > > > > > DEBUG 2: Searching Time Window: 20150715_030000 to
> > > > > 20150715_040000
> > > > > > 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75%
80%
> 85%
> > > > > > 90%
> > > > > 95%
> > > > > > 100%
> > > > > >
> > > > > > DEBUG 2: Total Messages processed = 590836
> > > > > > DEBUG 2: Rejected based on message type = 0
> > > > > > DEBUG 2: Rejected based on station id = 0
> > > > > > DEBUG 2: Rejected based on valid time = 590815
> > > > > > DEBUG 2: Rejected based on masking grid = 20
> > > > > > DEBUG 2: Rejected based on masking polygon = 0
> > > > > > DEBUG 2: Rejected based on elevation = 0
> > > > > > DEBUG 2: Rejected based on pb report type = 1
> > > > > > DEBUG 2: Rejected based on input report type = 0
> > > > > > DEBUG 2: Rejected based on instrument type = 0
> > > > > > DEBUG 2: Rejected based on zero observations = 0
> > > > > > DEBUG 2: Total Messages retained = 0
> > > > > > DEBUG 2: Total observations retained or derived = 0
> > > > > > DEBUG 1: Obs time between *20150714_210000 and
20150715_030000*
> > > > > > WARNING:
> > > > > > WARNING: process_pbfile() -> No PrepBufr messages retained
from
> > > > > > file:
> > > > > > /glade/collections/rda/data/ds337.0/prepnr/2015/
> > > > > > prepbufr.gdas.2015071500.nr
> > > > > > WARNING:
> > > > > > ERROR :
> > > > > > ERROR : write_netcdf_hdr_data() -> No PrepBufr messages
> retained.
> > > > > Nothing
> > > > > > to write.
> > > > > > ERROR :
> > > > > >
> > > > > > The prepbufr.gdas.2015071500.nr file actually contains
data from
> > > > > > 20150714_210000 to 20150715_030000, not 20150715_000000 to
> > > > > 20150715_060000.
> > > > > >
> > > > > > To get the correct file for each run, you will have to
shift the
> > > > > > valid
> > > > > time
> > > > > > before truncating it so that it gets the correct file for
each
> run.
> > > > > Instead
> > > > > > of setting:
> > > > > >
> > > > > > PB2NC_INPUT_TEMPLATE =
> > > > > >
> {valid?fmt=%Y}/prepbufr.gdas.{valid?fmt=%Y%m%d%H?truncate=21600}.nr
> > > > > >
> > > > > > You will have to also shift forward 3 hours, i.e.:
> > > > > >
> > > > > > PB2NC_INPUT_TEMPLATE =
> > > > > >
> > > > > >
> > > > >
> > >
> >
>
{valid?fmt=%Y}/prepbufr.gdas.{valid?fmt=%Y%m%d%H?shift=3H?truncate=21600}.nr
> > > > > >
> > > > > > This says to add 3 hours to the current valid time before
> > > > > > truncating to
> > > > > the
> > > > > > nearest 6 hours.
> > > > > >
> > > > > > Valid: 20150715 0Z + 3 = 3Z truncate 6H = 0Z
> > > > > > File: prepbufr.gdas.2015071500.nr
> > > > > >
> > > > > > Valid: 20150715 1Z + 3 = 4Z truncate 6H = 0Z
> > > > > > File: prepbufr.gdas.2015071500.nr
> > > > > >
> > > > > > Valid: 20150715 2Z + 3 = 5Z truncate 6H = 0Z
> > > > > > File: prepbufr.gdas.2015071500.nr
> > > > > >
> > > > > > Valid: 20150715 3Z + 3 = 6Z truncate 6H = 6Z
> > > > > > File: prepbufr.gdas.2015071506.nr
> > > > > >
> > > > > > Let me know if that doesn't work.
> > > > > >
> > > > > > Thanks,
> > > > > > George
> > > > > >
> > > > > >
> > > > > > On Wed, Apr 7, 2021 at 5:32 PM zhuming at ucar.edu via RT <
> > > > > met_help at ucar.edu>
> > > > > > wrote:
> > > > > >
> > > > > > >
> > > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429
> >
> > > > > > >
> > > > > > > Hi George,
> > > > > > >
> > > > > > > Now makes me confused is why processed obs of several
times are
> > > > > missing.
> > > > > > > There has files of time at 00,01,02, but no 03,04,05;
has
> > > > > > > 06,07,08, but
> > > > > > no
> > > > > > > 09,10,11, has 12,13,14, no 15,16,17; has 18,19,20, but
with
> > > > > > > 21,no
> > > > > 22,23.
> > > > > > > From yours and mines:
> > > > > > >
> > > > > > > ls -l /glade/u/home/mccabe/data/out/pb2nc1/
> > > > > > >
> > > > > > >
> > > > > > > -rw-r--r-- 1 mccabe ral 10141339 Apr 6 17:52
> pbs.2015071500.nc
> > > > > > >
> > > > > > > -rw-r--r-- 1 mccabe ral 5413907 Apr 6 17:56
> pbs.2015071501.nc
> > > > > > >
> > > > > > > -rw-r--r-- 1 mccabe ral 4923711 Apr 6 18:00
> pbs.2015071502.nc
> > > > > > >
> > > > > > > -rw-r--r-- 1 mccabe ral 5724447 Apr 6 18:14
> pbs.2015071506.nc
> > > > > > >
> > > > > > > -rw-r--r-- 1 mccabe ral 5280463 Apr 6 18:18
> pbs.2015071507.nc
> > > > > > >
> > > > > > > -rw-r--r-- 1 mccabe ral 4706119 Apr 6 18:22
> pbs.2015071508.nc
> > > > > > >
> > > > > > > -rw-r--r-- 1 mccabe ral 9878263 Apr 6 18:36
> pbs.2015071512.nc
> > > > > > >
> > > > > > > -rw-r--r-- 1 mccabe ral 5392527 Apr 6 18:40
> pbs.2015071513.nc
> > > > > > >
> > > > > > > -rw-r--r-- 1 mccabe ral 96 Apr 6 18:40
> pbs.2015071514.nc
> > > > > > >
> > > > > > > -rw-r--r-- 1 mccabe ral 96 Apr 6 18:42
> pbs.2015071515.nc
> > > > > > >
> > > > > > >
> > > > > > > cheyenne3:/glade/scratch/zhuming/metplus4/pb2nc1>l
> > > > > > >
> > > > > > > total 96112
> > > > > > >
> > > > > > > -rw-r--r-- 1 zhuming ncar 11550172 Apr 7 15:11
> > pbs.2015071500.nc
> > > > > > >
> > > > > > > -rw-r--r-- 1 zhuming ncar 6615956 Apr 7 15:14
> > pbs.2015071501.nc
> > > > > > >
> > > > > > > -rw-r--r-- 1 zhuming ncar 5984328 Apr 7 15:18
> > pbs.2015071502.nc
> > > > > > >
> > > > > > > -rw-r--r-- 1 zhuming ncar 6980712 Apr 7 15:33
> > pbs.2015071506.nc
> > > > > > >
> > > > > > > -rw-r--r-- 1 zhuming ncar 6455824 Apr 7 15:36
> > pbs.2015071507.nc
> > > > > > >
> > > > > > > -rw-r--r-- 1 zhuming ncar 5749432 Apr 7 15:40
> > pbs.2015071508.nc
> > > > > > >
> > > > > > > -rw-r--r-- 1 zhuming ncar 11284936 Apr 7 15:55
> > pbs.2015071512.nc
> > > > > > >
> > > > > > > -rw-r--r-- 1 zhuming ncar 6600888 Apr 7 15:59
> > pbs.2015071513.nc
> > > > > > >
> > > > > > > -rw-r--r-- 1 zhuming ncar 5868168 Apr 7 16:03
> > pbs.2015071514.nc
> > > > > > >
> > > > > > > -rw-r--r-- 1 zhuming ncar 7273092 Apr 7 16:18
> > pbs.2015071518.nc
> > > > > > >
> > > > > > > -rw-r--r-- 1 zhuming ncar 6656344 Apr 7 16:22
> > pbs.2015071519.nc
> > > > > > >
> > > > > > > -rw-r--r-- 1 zhuming ncar 6109788 Apr 7 16:27
> > pbs.2015071520.nc
> > > > > > >
> > > > > > > -rw-r--r-- 1 zhuming ncar 58808 Apr 7 16:31
> > pbs.2015071521.nc
> > > > > > >
> > > > > > > -rw-r--r-- 1 zhuming ncar 11132440 Apr 7 16:43
> > pbs.2015071600.nc
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Thanks,
> > > > > > >
> > > > > > > Zhuming
> > > > > > >
> > > > > > > On Tue, Apr 6, 2021 at 6:34 PM George McCabe via RT
> > > > > > > <met_help at ucar.edu
> > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > Hi Zhuming,
> > > > > > > >
> > > > > > > > Thanks for the information. You will need to use the
truncate
> > > > > > > > keyword
> > > > > > to
> > > > > > > > find the files, then set the valid beg/end range to 1
hour.
> > > > > > > > This is
> > > > > > what
> > > > > > > I
> > > > > > > > changed from your PB2NC1.conf config file:
> > > > > > > >
> > > > > > > >
> > > > > > > > *PB2NC_VALID_BEGIN =
{valid?fmt=%Y%m%d_%H}PB2NC_VALID_END =
> > > > > > > > {valid?fmt=%Y%m%d_%H?shift=1H}*
> > > > > > > >
> > > > > > > > *PB2NC_INPUT_TEMPLATE =
> > > > > > > >
> > >
{valid?fmt=%Y}/prepbufr.gdas.{valid?fmt=%Y%m%d%H?truncate=21600}.nr*
> > > > > > > >
> > > > > > > > My config file is on cheyenne here:
> > > > > > > > /glade/u/home/mccabe/PB2NC1.conf
> > > > > > > >
> > > > > > > > I just started running it and the output is being
written
> here:
> > > > > > > > /glade/u/home/mccabe/data/out/pb2nc1
> > > > > > > >
> > > > > > > >
> > > > > > > > For PointStat, you will want to loop over the same
times. I
> > > > > > > > noticed
> > > > > you
> > > > > > > > specify message type in the VAR<n>_OPTIONS, but you do
not
> need
> > > > > > > > to do
> > > > > > > this
> > > > > > > > since there is a POINT_STAT_MESSAGE_TYPE variable that
> defines
> > > > > > > > this.
> > > > > > This
> > > > > > > > is what I changed in your PointStat1.conf file:
> > > > > > > >
> > > > > > > > *VALID_END = 2015071600*
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > *VALID_INCREMENT = 1HPOINT_STAT_OBS_VALID_BEG
> > > > > =POINT_STAT_OBS_VALID_BEG
> > > > > > > =*
> > > > > > > >
> > > > > > > >
> > > > > > > > *POINT_STAT_MESSAGE_TYPE = ADPSFC*
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > *#OBS_VAR1_OPTIONS = message_type =
> > > > > > > > ["ADPSFC"];#OBS_VAR2_OPTIONS =
> > > > > > > > message_type = ["ADPSFC"];#OBS_VAR3_OPTIONS =
message_type =
> > > > > > > > ["ADPSFC"];#OBS_POINT_STAT_INPUT_DIR =
> > > > > > > > /glade/scratch/zhuming/metplus4/pb2nc1*
> > > > > > > >
> > > > > > > > *OBS_POINT_STAT_INPUT_DIR = {OUTPUT_BASE}/pb2nc1*
> > > > > > > >
> > > > > > > > Or you can view my file in the same directory. I
usually try
> to
> > > > > > > > test
> > > > > > > with a
> > > > > > > > single run time to make sure it is working properly,
so I set
> > > > > VALID_END
> > > > > > > > back to 2015071500. I ran it and it produced an output
file,
> > > > > > > > but
> > > > > there
> > > > > > > were
> > > > > > > > no values besides the header. I checked the log file
and saw
> > > > > > > > this:
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > *DEBUG 2: Processing T_sfc(0,*,*) versus TMP/Z2, for
> > > > > > > > observation type
> > > > > > > > ADPSFC, over region FULL, for interpolation method
BILIN(4),
> > > > > > > > using 0
> > > > > > > > matched pairs.DEBUG 3: Number of matched pairs =
0DEBUG 3:
> > > > > > Observations
> > > > > > > > processed = 393373DEBUG 3: Rejected: station id
=
> > > > > > > > 0DEBUG 3:
> > > > > > > > Rejected: obs type = 390528DEBUG 3: Rejected:
valid
> time
> > > > > =
> > > > > > > > 0DEBUG 3: Rejected: bad obs value = 0DEBUG 3:
Rejected: off
> > > > > > > > the
> > > > > grid
> > > > > > > =
> > > > > > > > 2586DEBUG 3: Rejected: topography = 0DEBUG 3:
Rejected:
> > > > > > > > level
> > > > > > > mismatch
> > > > > > > > = 259DEBUG 3: Rejected: quality marker = 0DEBUG 3:
Rejected:
> > > > > message
> > > > > > > type
> > > > > > > > = 0DEBUG 3: Rejected: masking region = 0DEBUG 3:
Rejected:
> bad
> > > > > fcst
> > > > > > > > value = 0DEBUG 3: Rejected: bad climo mean = 0DEBUG
3:
> > > > > > > > Rejected:
> > > > > bad
> > > > > > > > climo stdev = 0DEBUG 3: Rejected: duplicates =
0DEBUG
> > > > > > > > 2:DEBUG 2:
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > >
> >
>
--------------------------------------------------------------------------------DEBUG
> > > > > > > > 2:DEBUG 2: Processing u_10m_gr(0,*,*) versus UGRD/Z10,
for
> > > > > observation
> > > > > > > type
> > > > > > > > ADPSFC, over region FULL, for interpolation method
BILIN(4),
> > > > > > > > using 0
> > > > > > > > matched pairs.DEBUG 3: Number of matched pairs =
0DEBUG 3:
> > > > > > Observations
> > > > > > > > processed = 393373*
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > *DEBUG 3: Rejected: station id = 0DEBUG 3:
Rejected: obs
> > > > > > > > type
> > > > > > > =
> > > > > > > > 382965DEBUG 3: Rejected: valid time = 0DEBUG 3:
> Rejected:
> > > > > > > > bad
> > > > > obs
> > > > > > > > value = 0DEBUG 3: Rejected: off the grid =
9240DEBUG 3:
> > > > > Rejected:
> > > > > > > > topography = 0DEBUG 3: Rejected: level mismatch
=
> > > > > > > > 1168DEBUG 3:
> > > > > > > > Rejected: quality marker = 0DEBUG 3: Rejected:
message type
> > > > > > > > =
> > > > > > 0DEBUG
> > > > > > > 3:
> > > > > > > > Rejected: masking region = 0DEBUG 3: Rejected: bad
fcst
> value
> > > > > > > > =
> > > > > > 0DEBUG
> > > > > > > 3:
> > > > > > > > Rejected: bad climo mean = 0DEBUG 3: Rejected: bad
climo
> stdev
> > > > > > > > =
> > > > > > 0DEBUG
> > > > > > > 3:
> > > > > > > > Rejected: duplicates = 0DEBUG 2:DEBUG 2:
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > >
> >
>
--------------------------------------------------------------------------------DEBUG
> > > > > > > > 2:DEBUG 2: Processing v_10m_gr(0,*,*) versus VGRD/Z10,
for
> > > > > observation
> > > > > > > type
> > > > > > > > ADPSFC, over region FULL, for interpolation method
BILIN(4),
> > > > > > > > using 0
> > > > > > > > matched pairs.DEBUG 3: Number of matched pairs =
0DEBUG 3:
> > > > > > Observations
> > > > > > > > processed = 393373DEBUG 3: Rejected: station id
=
> > > > > > > > 0DEBUG 3:
> > > > > > > > Rejected: obs type = 382965DEBUG 3: Rejected:
valid
> time
> > > > > =
> > > > > > > > 0DEBUG 3: Rejected: bad obs value = 0DEBUG 3:
Rejected: off
> > > > > > > > the
> > > > > grid
> > > > > > > =
> > > > > > > > 9240DEBUG 3: Rejected: topography = 0DEBUG 3:
Rejected:
> > > > > > > > level
> > > > > > > mismatch
> > > > > > > > = 1168DEBUG 3: Rejected: quality marker = 0DEBUG 3:
> Rejected:
> > > > > message
> > > > > > > > type = 0DEBUG 3: Rejected: masking region = 0DEBUG
3:
> > > > > > > > Rejected:
> > > > > bad
> > > > > > > > fcst value = 0DEBUG 3: Rejected: bad climo mean =
0DEBUG 3:
> > > > > Rejected:
> > > > > > > bad
> > > > > > > > climo stdev = 0*
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > *DEBUG 3: Rejected: duplicates = 0DEBUG 2:DEBUG
2:
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > >
> >
>
--------------------------------------------------------------------------------DEBUG
> > > > > > > > 2:DEBUG 1: Output file:
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > >
> >
>
/glade/u/home/mccabe/data/out/point_stat1/point_stat_000000L_20150715_000000V.stat*
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > This describes the reason why each observation was
rejected.
> > > > > > > > Some
> > > > > > > > observations didn't match the message type you
requested,
> some
> > > > > > > > were
> > > > > not
> > > > > > > > within the forecast grid, and some did not match the
vertical
> > > > > > > > level.
> > > > > > You
> > > > > > > > will have to make adjustments to your settings so that
the
> data
> > > > > matches
> > > > > > > up
> > > > > > > > properly. Maybe the grid you are using in the PB2NC
config
> file
> > > > > > > > does
> > > > > > not
> > > > > > > > match the forecast grid and you are missing values. If
that
> is
> > > > > > > > the
> > > > > > case,
> > > > > > > > you will have to reconfigure and rerun PB2NC to get
output on
> > > > > > > > the
> > > > > > correct
> > > > > > > > gr5id. Maybe the observation points don't line up with
the
> > > > > > > > vertical
> > > > > > level
> > > > > > > > of the model fields. You can use a range of level
values for
> > > > > > > > the
> > > > > > > > observation data to allow data within that range, i.e.
Z0-10.
> > > > > > > >
> > > > > > > > Let me know if you are still having issues getting
this to
> work
> > > > > > > > and I
> > > > > > can
> > > > > > > > help out.
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > George
> > > > > > > >
> > > > > > > >
> > > > > > > > On Tue, Apr 6, 2021 at 5:17 PM zhuming at ucar.edu via RT
<
> > > > > > > met_help at ucar.edu>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > >
> > > > > > > > > <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > Hi George,
> > > > > > > > >
> > > > > > > > > Thanks for your reply!
> > > > > > > > >
> > > > > > > > > I'm still confused, I need to output hourly obs data
to do
> > > > > > > > > hourly
> > > > > > > > > verification.
> > > > > > > > > The original GDAS Prepbufr data has 4 files one day,
should
> > > > > > > > > with 6
> > > > > > > hours
> > > > > > > > > obs data in one file.
> > > > > > > > >
> > > > > > > > > After "PB2NC", I can see pbs.2015071500.nc (obs time
is
> 00?
> > > > > > > > > or 6
> > > > > > times
> > > > > > > > of
> > > > > > > > > 00,01,02,03,04,05?), pbs.2015071506.nc...
> > > > > > > > >
> > > > > > > > > my config file :
> > > > > cheyenne:/gpfs/fs1/work/zhuming/METplus4/PB2NC1.conf
> > > > > > > > > and /gpfs/fs1/work/zhuming/METplus4/PointStat1.conf
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > Zhuming
> > > > > > > > >
> > > > > > > > > On Tue, Apr 6, 2021 at 4:23 PM George McCabe via RT
<
> > > > > > met_help at ucar.edu
> > > > > > > >
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > Hi Zhuming,
> > > > > > > > > >
> > > > > > > > > > To run the tools hourly you would set the
VALID_INCREMENT
> > > > > > > > > > value
> > > > > for
> > > > > > > 1H.
> > > > > > > > > > Here is an example that runs once per hour for a
day from
> > > > > > 2015-07-15
> > > > > > > 0Z
> > > > > > > > > to
> > > > > > > > > > 2017-07-16 0Z:
> > > > > > > > > >
> > > > > > > > > > VALID_TIME_FMT = %Y%m%d%H
> > > > > > > > > >
> > > > > > > > > > VALID_BEG = 2015071500
> > > > > > > > > >
> > > > > > > > > > VALID_END = 2015071600
> > > > > > > > > >
> > > > > > > > > > VALID_INCREMENT = 1H
> > > > > > > > > >
> > > > > > > > > > LEAD_SEQ = 0
> > > > > > > > > >
> > > > > > > > > > Each run time will use the PB2NC_INPUT_TEMPLATE to
find
> > > > > > > > > > files
> > > > > whose
> > > > > > > > > > filenames match the time you need.
> > > > > > > > > >
> > > > > > > > > > To read prepBUFR observations for 12 hours, you
will want
> > > > > > > > > > to set
> > > > > > > > > > PB2NC_VALID_BEGIN and PB2NC_VALID_END to specify
the
> range
> > > > > > > > > > around
> > > > > > the
> > > > > > > > > valid
> > > > > > > > > > time to process. These values depend on what is
contained
> > > > > > > > > > inside
> > > > > > your
> > > > > > > > > > prepBUFR files. If you want to use data from the
valid
> time
> > > > > > > > > > of
> > > > > the
> > > > > > > > > filename
> > > > > > > > > > backwards 12 hours, you would set:
> > > > > > > > > >
> > > > > > > > > > PB2NC_VALID_BEGIN = {valid?fmt=%Y%m%d_%H?shift=-
12H }
> > > > > > > > > > PB2NC_VALID_END = {valid?fmt=%Y%m%d_%H}
> > > > > > > > > >
> > > > > > > > > > For a runtime of 2015071512, this would be
substituted
> as:
> > > > > > > > > >
> > > > > > > > > > valid_beg: 20150715_00
> > > > > > > > > > valid_end: 20150715_12
> > > > > > > > > >
> > > > > > > > > > If you want to create an output file for each hour
in
> PB2NC
> > > > > > > > > > you
> > > > > > would
> > > > > > > > > need
> > > > > > > > > > to make sure that the hour is specified in the
> > > > > > PB2NC_OUTPUT_TEMPLATE.
> > > > > > > > > >
> > > > > > > > > > PointStat has similar config variables to set the
valid
> beg
> > > > > > > > > > and
> > > > > end
> > > > > > > > range
> > > > > > > > > > to use (POINT_STAT_OBS_VALID_BEG and
> > > > > > > > > > POINT_STAT_OBS_VALID_END).
> > > > > > > > > >
> > > > > > > > > > If your prepBUFR files are separated out into
multiple
> > > > > > > > > > files that
> > > > > > > need
> > > > > > > > to
> > > > > > > > > > be read into PB2NC, you can set a file window
around the
> > > > > > > > > > valid
> > > > > time
> > > > > > > to
> > > > > > > > > > allow files to be passed into the app. More
information
> on
> > > > > > > > > > that
> > > > > > > > > > functionality is here:
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > >
> >
>
https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#using-
> > > > > windows-to-find-valid-files
> > > > > > > > > >
> > > > > > > > > > If you are still having trouble getting this
configured,
> > > > > > > > > > you
> > > > > could
> > > > > > > send
> > > > > > > > > me
> > > > > > > > > > the paths of the files you are trying to read and
what
> data
> > > > > > > > > > is
> > > > > > > > contained
> > > > > > > > > in
> > > > > > > > > > them so I can give you suggestions on what to set
for the
> > > > > > > > configuration.
> > > > > > > > > >
> > > > > > > > > > Thanks,
> > > > > > > > > > George
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On Tue, Apr 6, 2021 at 2:17 PM Julie Prestopnik
via RT <
> > > > > > > > > met_help at ucar.edu>
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > Tue Apr 06 14:16:42 2021: Request 99429 was
acted upon.
> > > > > > > > > > > Transaction: Given to mccabe (George McCabe) by
jpresto
> > > > > > > > > > > Queue: met_help
> > > > > > > > > > > Subject: How to set time for hourly 12-24
hours
> data
> > > > > > > > > > > ?
> > > > > > > > > > > Owner: mccabe
> > > > > > > > > > > Requestors: zhuming at ucar.edu
> > > > > > > > > > > Status: new
> > > > > > > > > > > Ticket <URL:
> > > > > > > >
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429
> > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > This transaction appears to have no content
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > --
> > > > > > > > > > George McCabe - Software Engineer III
> > > > > > > > > > National Center for Atmospheric Research
> > > > > > > > > > Research Applications Laboratory
> > > > > > > > > > 303-497-2768
> > > > > > > > > > ---
> > > > > > > > > > My working day may not be your working day. Please
do not
> > > > > > > > > > feel
> > > > > > > obliged
> > > > > > > > to
> > > > > > > > > > reply to this email outside of your normal working
hours.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > George McCabe - Software Engineer III
> > > > > > > > National Center for Atmospheric Research
> > > > > > > > Research Applications Laboratory
> > > > > > > > 303-497-2768
> > > > > > > > ---
> > > > > > > > My working day may not be your working day. Please do
not
> feel
> > > > > obliged
> > > > > > to
> > > > > > > > reply to this email outside of your normal working
hours.
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > >
> > > > > > --
> > > > > > George McCabe - Software Engineer III
> > > > > > National Center for Atmospheric Research
> > > > > > Research Applications Laboratory
> > > > > > 303-497-2768
> > > > > > ---
> > > > > > My working day may not be your working day. Please do not
feel
> > > > > > obliged to
> > > > > > reply to this email outside of your normal working hours.
> > > > > >
> > > > > >
> > > > >
> > > > >
> > >
> > >
> > >
> > >
> >
> >
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>
------------------------------------------------
Subject: How to set time for hourly 12-24 hours data ?
From: George McCabe
Time: Wed Apr 14 17:29:02 2021
Hi Zhuming,
I tried to get this to work on cheyenne with your data, but was not
successful. I am able to read the data with plot_data_plane and
generate an
image, so I think there may be a bug in the code. I have forwarded
info to
one of our C++ developers to look into it. I will report back when I
know
more.
Thanks,
George
On Wed, Apr 14, 2021 at 4:13 PM zhuming at ucar.edu via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429 >
>
> Hi George,
>
> It still can't find the correct time. I tried
>
> FCST_VAR1_LEVELS = (*,*,*)
> FCST_VAR1_LEVELS = ({valid?fmt=%Y%m%d_%H%M%S},*,*)
> FCST_VAR1_LEVELS = ({valid?fmt=%Y%m%d_%H%M},*,*)
> FCST_VAR1_LEVELS = ({valid?fmt=%Y%m%d%H%M},*,*)
>
> Thanks!
>
> Zhuming
>
> On Wed, Apr 14, 2021 at 3:41 PM George McCabe via RT
<met_help at ucar.edu>
> wrote:
>
> > Hi Zhuming,
> >
> > It looks like you are requesting the first time in the 12 hour
file for
> > each run:
> >
> > FCST_VAR1_LEVELS = (0,*,*)
> > ...
> > FCST_VAR2_LEVELS = (0,*,*)
> >
> > The first dimension in these files is the time offset. You will
need the
> > correct index that corresponds to each valid time. Fortunately,
MET makes
> > this easy by allowing you to specify the valid time for a time
dimension
> > explicitly and it will find the correct data. You can set:
> >
> > FCST_VAR1_LEVELS = ({valid?fmt=%Y%m%d_%H%M%S},*,*)
> > ...
> > FCST_VAR2_LEVELS = ({valid?fmt=%Y%m%d_%H%M%S},*,*)
> >
> > And it will substitute the valid time for each run. Let me know if
you
> have
> > any questions.
> >
> > Thanks,
> > George
> >
> > On Wed, Apr 14, 2021 at 3:18 PM zhuming at ucar.edu via RT <
> met_help at ucar.edu
> > >
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429 >
> > >
> > > Hi George,
> > >
> > > Thanks for your email, I just would like to send you for help.
> > >
> > > I already resolved the last problem with the WRF forecast, thank
you
> for
> > > your help.
> > >
> > > I also need to verify the MPAS model forecast. These 12 hour
> > > MAPS forecasts are stored in a one file, such as
> > > 2015071500/mpas_wrf_cf_2015071500.nc, has 12h forecasts
> > > 2015071512/mpas_wrf_cf_2015071512.nc
> > >
> > > It seems MET is doing fine on the first 2 hours, but no matched
pairs
> for
> > > later hours.
> > > I'm not sure what's wrong with it? the time is not matched?
> > >
> > > see the log file
> > >
/glade/scratch/zhuming/metplus4/logs/master_metplus.log.20210414145101
> > > config file /gpfs/fs1/work/zhuming/METplus4/PointStat2.conf
> > >
> > > Thanks,
> > > Zhuming
> > >
> > > On Wed, Apr 14, 2021 at 1:56 PM George McCabe via RT <
> met_help at ucar.edu>
> > > wrote:
> > >
> > > > Hi Zhuming,
> > > >
> > > > Just following up on this ticket. Were you able to get this
resolved?
> > Let
> > > > me know if you need more assistance.
> > > >
> > > > Thanks,
> > > > George
> > > >
> > > > On Thu Apr 08 10:04:44 2021, mccabe wrote:
> > > > > Hi Zhuming,
> > > > >
> > > > > Typically directories that contain data for an entire day
are named
> > > > > with
> > > > > YYYYMMDD, so your filename template tag would be
> {valid?fmt=%Y%m%d},
> > > > > however that may not always be the case, as you are seeing
here.
> > There
> > > > > are
> > > > > a couple ways to handle this.
> > > > >
> > > > > If you know that the directory will always be YYYYMMDD with
00 at
> the
> > > > > end,
> > > > > you can simply put 00 in the template instead of adding %H
in the
> > tag,
> > > > > i.e.
> > > > >
> > > > > FCST_POINT_STAT_INPUT_TEMPLATE =
> > > > > {valid?fmt=%Y%m%d}00/wrfout_d02_{valid?fmt=%Y-%m-%d_%H}:00:
> 00_cf.nc
> > > > >
> > > > > If the HH value can vary, for instance if there are 2
directories
> > > > > created
> > > > > per day (i.e. 2015071400 and 2015071412), then you can use
the
> > > > > truncate
> > > > > keyword to handle it.
> > > > >
> > > > > If directories are created every 24 hours:
> > > > >
> > > > > FCST_POINT_STAT_INPUT_TEMPLATE =
> > > > > {valid?fmt=%Y%m%d%H?truncate=24H}/wrfout_d02_{valid?fmt=%Y-
%m-
> > > > > %d_%H}:00:
> > > > > 00_cf.nc
> > > > >
> > > > > If directories are created every 12 hours:
> > > > >
> > > > > FCST_POINT_STAT_INPUT_TEMPLATE =
> > > > > {valid?fmt=%Y%m%d%H?truncate=12H}/wrfout_d02_{valid?fmt=%Y-
%m-
> > > > > %d_%H}:00:
> > > > > 00_cf.nc
> > > > >
> > > > > Let me know if that doesn't work for you.
> > > > >
> > > > > Thanks,
> > > > > George
> > > > >
> > > > > On Thu, Apr 8, 2021 at 9:46 AM zhuming at ucar.edu via RT
> > > > > <met_help at ucar.edu>
> > > > > wrote:
> > > > >
> > > > > >
> > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429 >
> > > > > >
> > > > > > Hi George,
> > > > > >
> > > > > > Processing Obs data using "PB2NC" are right now.
> > > > > >
> > > > > > Now I'm using "PointStat" , I set VALID_BEG = 2015071500,
> > VALID_END =
> > > > > > 2015071501. It looks for the forecast data 2015071500/
> > > > > > wrfout_d02_2015-07-15_00:00:00_cf.nc, and 2015071501/
> > > > > > wrfout_d02_2015-07-15_01:00:00_cf.nc.
> > > > > > But my forecast data are all put in dir of 2015071500/.
> > > > > > So how could I set FCST_POINT_STAT_INPUT_TEMPLATE to get
the
> right
> > > > > > fcst
> > > > > > data?
> > > > > >
> > > > > > FCST_POINT_STAT_INPUT_TEMPLATE =
> > > > > > {valid?fmt=%Y%m%d%H}/wrfout_d02_{valid?fmt=%Y-%m-
%d_%H}:00:
> > 00_cf.nc
> > > > > >
> > > > > >
> > > > > > config file:
/gpfs/fs1/work/zhuming/METplus4/PointStat1.conf
> > > > > >
> > > > > > log file:
> > > > > >
> > >
/glade/scratch/zhuming/metplus4/logs/master_metplus.log.20210408085450
> > > > > >
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > Zhuming
> > > > > >
> > > > > > On Wed, Apr 7, 2021 at 5:43 PM George McCabe via RT
> > > > > > <met_help at ucar.edu>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi Zhuming,
> > > > > > >
> > > > > > > Looking at the log output, it looks like the data is
stored in
> > the
> > > > > > prepBUFR
> > > > > > > files differently that you may have thought:
> > > > > > >
> > > > > > > DEBUG 2: PrepBufr Time Center: 20150715_000000
> > > > > > > DEBUG 2: Searching Time Window: 20150715_030000
to
> > > > > > 20150715_040000
> > > > > > > 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70%
75% 80%
> > 85%
> > > > > > > 90%
> > > > > > 95%
> > > > > > > 100%
> > > > > > >
> > > > > > > DEBUG 2: Total Messages processed = 590836
> > > > > > > DEBUG 2: Rejected based on message type = 0
> > > > > > > DEBUG 2: Rejected based on station id = 0
> > > > > > > DEBUG 2: Rejected based on valid time = 590815
> > > > > > > DEBUG 2: Rejected based on masking grid = 20
> > > > > > > DEBUG 2: Rejected based on masking polygon = 0
> > > > > > > DEBUG 2: Rejected based on elevation = 0
> > > > > > > DEBUG 2: Rejected based on pb report type = 1
> > > > > > > DEBUG 2: Rejected based on input report type = 0
> > > > > > > DEBUG 2: Rejected based on instrument type = 0
> > > > > > > DEBUG 2: Rejected based on zero observations = 0
> > > > > > > DEBUG 2: Total Messages retained = 0
> > > > > > > DEBUG 2: Total observations retained or derived = 0
> > > > > > > DEBUG 1: Obs time between *20150714_210000 and
20150715_030000*
> > > > > > > WARNING:
> > > > > > > WARNING: process_pbfile() -> No PrepBufr messages
retained from
> > > > > > > file:
> > > > > > > /glade/collections/rda/data/ds337.0/prepnr/2015/
> > > > > > > prepbufr.gdas.2015071500.nr
> > > > > > > WARNING:
> > > > > > > ERROR :
> > > > > > > ERROR : write_netcdf_hdr_data() -> No PrepBufr messages
> > retained.
> > > > > > Nothing
> > > > > > > to write.
> > > > > > > ERROR :
> > > > > > >
> > > > > > > The prepbufr.gdas.2015071500.nr file actually contains
data
> from
> > > > > > > 20150714_210000 to 20150715_030000, not 20150715_000000
to
> > > > > > 20150715_060000.
> > > > > > >
> > > > > > > To get the correct file for each run, you will have to
shift
> the
> > > > > > > valid
> > > > > > time
> > > > > > > before truncating it so that it gets the correct file
for each
> > run.
> > > > > > Instead
> > > > > > > of setting:
> > > > > > >
> > > > > > > PB2NC_INPUT_TEMPLATE =
> > > > > > >
> >
{valid?fmt=%Y}/prepbufr.gdas.{valid?fmt=%Y%m%d%H?truncate=21600}.nr
> > > > > > >
> > > > > > > You will have to also shift forward 3 hours, i.e.:
> > > > > > >
> > > > > > > PB2NC_INPUT_TEMPLATE =
> > > > > > >
> > > > > > >
> > > > > >
> > > >
> > >
> >
>
{valid?fmt=%Y}/prepbufr.gdas.{valid?fmt=%Y%m%d%H?shift=3H?truncate=21600}.nr
> > > > > > >
> > > > > > > This says to add 3 hours to the current valid time
before
> > > > > > > truncating to
> > > > > > the
> > > > > > > nearest 6 hours.
> > > > > > >
> > > > > > > Valid: 20150715 0Z + 3 = 3Z truncate 6H = 0Z
> > > > > > > File: prepbufr.gdas.2015071500.nr
> > > > > > >
> > > > > > > Valid: 20150715 1Z + 3 = 4Z truncate 6H = 0Z
> > > > > > > File: prepbufr.gdas.2015071500.nr
> > > > > > >
> > > > > > > Valid: 20150715 2Z + 3 = 5Z truncate 6H = 0Z
> > > > > > > File: prepbufr.gdas.2015071500.nr
> > > > > > >
> > > > > > > Valid: 20150715 3Z + 3 = 6Z truncate 6H = 6Z
> > > > > > > File: prepbufr.gdas.2015071506.nr
> > > > > > >
> > > > > > > Let me know if that doesn't work.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > George
> > > > > > >
> > > > > > >
> > > > > > > On Wed, Apr 7, 2021 at 5:32 PM zhuming at ucar.edu via RT <
> > > > > > met_help at ucar.edu>
> > > > > > > wrote:
> > > > > > >
> > > > > > > >
> > > > > > > > <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429
> > >
> > > > > > > >
> > > > > > > > Hi George,
> > > > > > > >
> > > > > > > > Now makes me confused is why processed obs of several
times
> are
> > > > > > missing.
> > > > > > > > There has files of time at 00,01,02, but no 03,04,05;
has
> > > > > > > > 06,07,08, but
> > > > > > > no
> > > > > > > > 09,10,11, has 12,13,14, no 15,16,17; has 18,19,20,
but with
> > > > > > > > 21,no
> > > > > > 22,23.
> > > > > > > > From yours and mines:
> > > > > > > >
> > > > > > > > ls -l /glade/u/home/mccabe/data/out/pb2nc1/
> > > > > > > >
> > > > > > > >
> > > > > > > > -rw-r--r-- 1 mccabe ral 10141339 Apr 6 17:52
> > pbs.2015071500.nc
> > > > > > > >
> > > > > > > > -rw-r--r-- 1 mccabe ral 5413907 Apr 6 17:56
> > pbs.2015071501.nc
> > > > > > > >
> > > > > > > > -rw-r--r-- 1 mccabe ral 4923711 Apr 6 18:00
> > pbs.2015071502.nc
> > > > > > > >
> > > > > > > > -rw-r--r-- 1 mccabe ral 5724447 Apr 6 18:14
> > pbs.2015071506.nc
> > > > > > > >
> > > > > > > > -rw-r--r-- 1 mccabe ral 5280463 Apr 6 18:18
> > pbs.2015071507.nc
> > > > > > > >
> > > > > > > > -rw-r--r-- 1 mccabe ral 4706119 Apr 6 18:22
> > pbs.2015071508.nc
> > > > > > > >
> > > > > > > > -rw-r--r-- 1 mccabe ral 9878263 Apr 6 18:36
> > pbs.2015071512.nc
> > > > > > > >
> > > > > > > > -rw-r--r-- 1 mccabe ral 5392527 Apr 6 18:40
> > pbs.2015071513.nc
> > > > > > > >
> > > > > > > > -rw-r--r-- 1 mccabe ral 96 Apr 6 18:40
> > pbs.2015071514.nc
> > > > > > > >
> > > > > > > > -rw-r--r-- 1 mccabe ral 96 Apr 6 18:42
> > pbs.2015071515.nc
> > > > > > > >
> > > > > > > >
> > > > > > > > cheyenne3:/glade/scratch/zhuming/metplus4/pb2nc1>l
> > > > > > > >
> > > > > > > > total 96112
> > > > > > > >
> > > > > > > > -rw-r--r-- 1 zhuming ncar 11550172 Apr 7 15:11
> > > pbs.2015071500.nc
> > > > > > > >
> > > > > > > > -rw-r--r-- 1 zhuming ncar 6615956 Apr 7 15:14
> > > pbs.2015071501.nc
> > > > > > > >
> > > > > > > > -rw-r--r-- 1 zhuming ncar 5984328 Apr 7 15:18
> > > pbs.2015071502.nc
> > > > > > > >
> > > > > > > > -rw-r--r-- 1 zhuming ncar 6980712 Apr 7 15:33
> > > pbs.2015071506.nc
> > > > > > > >
> > > > > > > > -rw-r--r-- 1 zhuming ncar 6455824 Apr 7 15:36
> > > pbs.2015071507.nc
> > > > > > > >
> > > > > > > > -rw-r--r-- 1 zhuming ncar 5749432 Apr 7 15:40
> > > pbs.2015071508.nc
> > > > > > > >
> > > > > > > > -rw-r--r-- 1 zhuming ncar 11284936 Apr 7 15:55
> > > pbs.2015071512.nc
> > > > > > > >
> > > > > > > > -rw-r--r-- 1 zhuming ncar 6600888 Apr 7 15:59
> > > pbs.2015071513.nc
> > > > > > > >
> > > > > > > > -rw-r--r-- 1 zhuming ncar 5868168 Apr 7 16:03
> > > pbs.2015071514.nc
> > > > > > > >
> > > > > > > > -rw-r--r-- 1 zhuming ncar 7273092 Apr 7 16:18
> > > pbs.2015071518.nc
> > > > > > > >
> > > > > > > > -rw-r--r-- 1 zhuming ncar 6656344 Apr 7 16:22
> > > pbs.2015071519.nc
> > > > > > > >
> > > > > > > > -rw-r--r-- 1 zhuming ncar 6109788 Apr 7 16:27
> > > pbs.2015071520.nc
> > > > > > > >
> > > > > > > > -rw-r--r-- 1 zhuming ncar 58808 Apr 7 16:31
> > > pbs.2015071521.nc
> > > > > > > >
> > > > > > > > -rw-r--r-- 1 zhuming ncar 11132440 Apr 7 16:43
> > > pbs.2015071600.nc
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > >
> > > > > > > > Zhuming
> > > > > > > >
> > > > > > > > On Tue, Apr 6, 2021 at 6:34 PM George McCabe via RT
> > > > > > > > <met_help at ucar.edu
> > > > > > >
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Hi Zhuming,
> > > > > > > > >
> > > > > > > > > Thanks for the information. You will need to use the
> truncate
> > > > > > > > > keyword
> > > > > > > to
> > > > > > > > > find the files, then set the valid beg/end range to
1 hour.
> > > > > > > > > This is
> > > > > > > what
> > > > > > > > I
> > > > > > > > > changed from your PB2NC1.conf config file:
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > *PB2NC_VALID_BEGIN =
{valid?fmt=%Y%m%d_%H}PB2NC_VALID_END =
> > > > > > > > > {valid?fmt=%Y%m%d_%H?shift=1H}*
> > > > > > > > >
> > > > > > > > > *PB2NC_INPUT_TEMPLATE =
> > > > > > > > >
> > > >
{valid?fmt=%Y}/prepbufr.gdas.{valid?fmt=%Y%m%d%H?truncate=21600}.nr*
> > > > > > > > >
> > > > > > > > > My config file is on cheyenne here:
> > > > > > > > > /glade/u/home/mccabe/PB2NC1.conf
> > > > > > > > >
> > > > > > > > > I just started running it and the output is being
written
> > here:
> > > > > > > > > /glade/u/home/mccabe/data/out/pb2nc1
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > For PointStat, you will want to loop over the same
times. I
> > > > > > > > > noticed
> > > > > > you
> > > > > > > > > specify message type in the VAR<n>_OPTIONS, but you
do not
> > need
> > > > > > > > > to do
> > > > > > > > this
> > > > > > > > > since there is a POINT_STAT_MESSAGE_TYPE variable
that
> > defines
> > > > > > > > > this.
> > > > > > > This
> > > > > > > > > is what I changed in your PointStat1.conf file:
> > > > > > > > >
> > > > > > > > > *VALID_END = 2015071600*
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > *VALID_INCREMENT = 1HPOINT_STAT_OBS_VALID_BEG
> > > > > > =POINT_STAT_OBS_VALID_BEG
> > > > > > > > =*
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > *POINT_STAT_MESSAGE_TYPE = ADPSFC*
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > *#OBS_VAR1_OPTIONS = message_type =
> > > > > > > > > ["ADPSFC"];#OBS_VAR2_OPTIONS =
> > > > > > > > > message_type = ["ADPSFC"];#OBS_VAR3_OPTIONS =
message_type
> =
> > > > > > > > > ["ADPSFC"];#OBS_POINT_STAT_INPUT_DIR =
> > > > > > > > > /glade/scratch/zhuming/metplus4/pb2nc1*
> > > > > > > > >
> > > > > > > > > *OBS_POINT_STAT_INPUT_DIR = {OUTPUT_BASE}/pb2nc1*
> > > > > > > > >
> > > > > > > > > Or you can view my file in the same directory. I
usually
> try
> > to
> > > > > > > > > test
> > > > > > > > with a
> > > > > > > > > single run time to make sure it is working properly,
so I
> set
> > > > > > VALID_END
> > > > > > > > > back to 2015071500. I ran it and it produced an
output
> file,
> > > > > > > > > but
> > > > > > there
> > > > > > > > were
> > > > > > > > > no values besides the header. I checked the log file
and
> saw
> > > > > > > > > this:
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > *DEBUG 2: Processing T_sfc(0,*,*) versus TMP/Z2, for
> > > > > > > > > observation type
> > > > > > > > > ADPSFC, over region FULL, for interpolation method
> BILIN(4),
> > > > > > > > > using 0
> > > > > > > > > matched pairs.DEBUG 3: Number of matched pairs =
0DEBUG
> 3:
> > > > > > > Observations
> > > > > > > > > processed = 393373DEBUG 3: Rejected: station id
=
> > > > > > > > > 0DEBUG 3:
> > > > > > > > > Rejected: obs type = 390528DEBUG 3: Rejected:
valid
> > time
> > > > > > =
> > > > > > > > > 0DEBUG 3: Rejected: bad obs value = 0DEBUG 3:
Rejected:
> off
> > > > > > > > > the
> > > > > > grid
> > > > > > > > =
> > > > > > > > > 2586DEBUG 3: Rejected: topography = 0DEBUG 3:
> Rejected:
> > > > > > > > > level
> > > > > > > > mismatch
> > > > > > > > > = 259DEBUG 3: Rejected: quality marker = 0DEBUG 3:
> Rejected:
> > > > > > message
> > > > > > > > type
> > > > > > > > > = 0DEBUG 3: Rejected: masking region = 0DEBUG 3:
Rejected:
> > bad
> > > > > > fcst
> > > > > > > > > value = 0DEBUG 3: Rejected: bad climo mean =
0DEBUG 3:
> > > > > > > > > Rejected:
> > > > > > bad
> > > > > > > > > climo stdev = 0DEBUG 3: Rejected: duplicates =
0DEBUG
> > > > > > > > > 2:DEBUG 2:
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > >
> > >
> >
>
--------------------------------------------------------------------------------DEBUG
> > > > > > > > > 2:DEBUG 2: Processing u_10m_gr(0,*,*) versus
UGRD/Z10, for
> > > > > > observation
> > > > > > > > type
> > > > > > > > > ADPSFC, over region FULL, for interpolation method
> BILIN(4),
> > > > > > > > > using 0
> > > > > > > > > matched pairs.DEBUG 3: Number of matched pairs =
0DEBUG
> 3:
> > > > > > > Observations
> > > > > > > > > processed = 393373*
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > *DEBUG 3: Rejected: station id = 0DEBUG 3:
Rejected:
> obs
> > > > > > > > > type
> > > > > > > > =
> > > > > > > > > 382965DEBUG 3: Rejected: valid time = 0DEBUG 3:
> > Rejected:
> > > > > > > > > bad
> > > > > > obs
> > > > > > > > > value = 0DEBUG 3: Rejected: off the grid =
9240DEBUG
> 3:
> > > > > > Rejected:
> > > > > > > > > topography = 0DEBUG 3: Rejected: level mismatch
=
> > > > > > > > > 1168DEBUG 3:
> > > > > > > > > Rejected: quality marker = 0DEBUG 3: Rejected:
message
> type
> > > > > > > > > =
> > > > > > > 0DEBUG
> > > > > > > > 3:
> > > > > > > > > Rejected: masking region = 0DEBUG 3: Rejected: bad
fcst
> > value
> > > > > > > > > =
> > > > > > > 0DEBUG
> > > > > > > > 3:
> > > > > > > > > Rejected: bad climo mean = 0DEBUG 3: Rejected: bad
climo
> > stdev
> > > > > > > > > =
> > > > > > > 0DEBUG
> > > > > > > > 3:
> > > > > > > > > Rejected: duplicates = 0DEBUG 2:DEBUG 2:
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > >
> > >
> >
>
--------------------------------------------------------------------------------DEBUG
> > > > > > > > > 2:DEBUG 2: Processing v_10m_gr(0,*,*) versus
VGRD/Z10, for
> > > > > > observation
> > > > > > > > type
> > > > > > > > > ADPSFC, over region FULL, for interpolation method
> BILIN(4),
> > > > > > > > > using 0
> > > > > > > > > matched pairs.DEBUG 3: Number of matched pairs =
0DEBUG
> 3:
> > > > > > > Observations
> > > > > > > > > processed = 393373DEBUG 3: Rejected: station id
=
> > > > > > > > > 0DEBUG 3:
> > > > > > > > > Rejected: obs type = 382965DEBUG 3: Rejected:
valid
> > time
> > > > > > =
> > > > > > > > > 0DEBUG 3: Rejected: bad obs value = 0DEBUG 3:
Rejected:
> off
> > > > > > > > > the
> > > > > > grid
> > > > > > > > =
> > > > > > > > > 9240DEBUG 3: Rejected: topography = 0DEBUG 3:
> Rejected:
> > > > > > > > > level
> > > > > > > > mismatch
> > > > > > > > > = 1168DEBUG 3: Rejected: quality marker = 0DEBUG 3:
> > Rejected:
> > > > > > message
> > > > > > > > > type = 0DEBUG 3: Rejected: masking region =
0DEBUG 3:
> > > > > > > > > Rejected:
> > > > > > bad
> > > > > > > > > fcst value = 0DEBUG 3: Rejected: bad climo mean =
0DEBUG
> 3:
> > > > > > Rejected:
> > > > > > > > bad
> > > > > > > > > climo stdev = 0*
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > *DEBUG 3: Rejected: duplicates = 0DEBUG 2:DEBUG
2:
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > >
> > >
> >
>
--------------------------------------------------------------------------------DEBUG
> > > > > > > > > 2:DEBUG 1: Output file:
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > >
> > >
> >
>
/glade/u/home/mccabe/data/out/point_stat1/point_stat_000000L_20150715_000000V.stat*
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > This describes the reason why each observation was
> rejected.
> > > > > > > > > Some
> > > > > > > > > observations didn't match the message type you
requested,
> > some
> > > > > > > > > were
> > > > > > not
> > > > > > > > > within the forecast grid, and some did not match the
> vertical
> > > > > > > > > level.
> > > > > > > You
> > > > > > > > > will have to make adjustments to your settings so
that the
> > data
> > > > > > matches
> > > > > > > > up
> > > > > > > > > properly. Maybe the grid you are using in the PB2NC
config
> > file
> > > > > > > > > does
> > > > > > > not
> > > > > > > > > match the forecast grid and you are missing values.
If that
> > is
> > > > > > > > > the
> > > > > > > case,
> > > > > > > > > you will have to reconfigure and rerun PB2NC to get
output
> on
> > > > > > > > > the
> > > > > > > correct
> > > > > > > > > gr5id. Maybe the observation points don't line up
with the
> > > > > > > > > vertical
> > > > > > > level
> > > > > > > > > of the model fields. You can use a range of level
values
> for
> > > > > > > > > the
> > > > > > > > > observation data to allow data within that range,
i.e.
> Z0-10.
> > > > > > > > >
> > > > > > > > > Let me know if you are still having issues getting
this to
> > work
> > > > > > > > > and I
> > > > > > > can
> > > > > > > > > help out.
> > > > > > > > >
> > > > > > > > > Thanks,
> > > > > > > > > George
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On Tue, Apr 6, 2021 at 5:17 PM zhuming at ucar.edu via
RT <
> > > > > > > > met_help at ucar.edu>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Hi George,
> > > > > > > > > >
> > > > > > > > > > Thanks for your reply!
> > > > > > > > > >
> > > > > > > > > > I'm still confused, I need to output hourly obs
data to
> do
> > > > > > > > > > hourly
> > > > > > > > > > verification.
> > > > > > > > > > The original GDAS Prepbufr data has 4 files one
day,
> should
> > > > > > > > > > with 6
> > > > > > > > hours
> > > > > > > > > > obs data in one file.
> > > > > > > > > >
> > > > > > > > > > After "PB2NC", I can see pbs.2015071500.nc (obs
time is
> > 00?
> > > > > > > > > > or 6
> > > > > > > times
> > > > > > > > > of
> > > > > > > > > > 00,01,02,03,04,05?), pbs.2015071506.nc...
> > > > > > > > > >
> > > > > > > > > > my config file :
> > > > > > cheyenne:/gpfs/fs1/work/zhuming/METplus4/PB2NC1.conf
> > > > > > > > > > and
/gpfs/fs1/work/zhuming/METplus4/PointStat1.conf
> > > > > > > > > >
> > > > > > > > > > Thanks,
> > > > > > > > > > Zhuming
> > > > > > > > > >
> > > > > > > > > > On Tue, Apr 6, 2021 at 4:23 PM George McCabe via
RT <
> > > > > > > met_help at ucar.edu
> > > > > > > > >
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Hi Zhuming,
> > > > > > > > > > >
> > > > > > > > > > > To run the tools hourly you would set the
> VALID_INCREMENT
> > > > > > > > > > > value
> > > > > > for
> > > > > > > > 1H.
> > > > > > > > > > > Here is an example that runs once per hour for a
day
> from
> > > > > > > 2015-07-15
> > > > > > > > 0Z
> > > > > > > > > > to
> > > > > > > > > > > 2017-07-16 0Z:
> > > > > > > > > > >
> > > > > > > > > > > VALID_TIME_FMT = %Y%m%d%H
> > > > > > > > > > >
> > > > > > > > > > > VALID_BEG = 2015071500
> > > > > > > > > > >
> > > > > > > > > > > VALID_END = 2015071600
> > > > > > > > > > >
> > > > > > > > > > > VALID_INCREMENT = 1H
> > > > > > > > > > >
> > > > > > > > > > > LEAD_SEQ = 0
> > > > > > > > > > >
> > > > > > > > > > > Each run time will use the PB2NC_INPUT_TEMPLATE
to find
> > > > > > > > > > > files
> > > > > > whose
> > > > > > > > > > > filenames match the time you need.
> > > > > > > > > > >
> > > > > > > > > > > To read prepBUFR observations for 12 hours, you
will
> want
> > > > > > > > > > > to set
> > > > > > > > > > > PB2NC_VALID_BEGIN and PB2NC_VALID_END to specify
the
> > range
> > > > > > > > > > > around
> > > > > > > the
> > > > > > > > > > valid
> > > > > > > > > > > time to process. These values depend on what is
> contained
> > > > > > > > > > > inside
> > > > > > > your
> > > > > > > > > > > prepBUFR files. If you want to use data from the
valid
> > time
> > > > > > > > > > > of
> > > > > > the
> > > > > > > > > > filename
> > > > > > > > > > > backwards 12 hours, you would set:
> > > > > > > > > > >
> > > > > > > > > > > PB2NC_VALID_BEGIN = {valid?fmt=%Y%m%d_%H?shift=-
12H }
> > > > > > > > > > > PB2NC_VALID_END = {valid?fmt=%Y%m%d_%H}
> > > > > > > > > > >
> > > > > > > > > > > For a runtime of 2015071512, this would be
substituted
> > as:
> > > > > > > > > > >
> > > > > > > > > > > valid_beg: 20150715_00
> > > > > > > > > > > valid_end: 20150715_12
> > > > > > > > > > >
> > > > > > > > > > > If you want to create an output file for each
hour in
> > PB2NC
> > > > > > > > > > > you
> > > > > > > would
> > > > > > > > > > need
> > > > > > > > > > > to make sure that the hour is specified in the
> > > > > > > PB2NC_OUTPUT_TEMPLATE.
> > > > > > > > > > >
> > > > > > > > > > > PointStat has similar config variables to set
the valid
> > beg
> > > > > > > > > > > and
> > > > > > end
> > > > > > > > > range
> > > > > > > > > > > to use (POINT_STAT_OBS_VALID_BEG and
> > > > > > > > > > > POINT_STAT_OBS_VALID_END).
> > > > > > > > > > >
> > > > > > > > > > > If your prepBUFR files are separated out into
multiple
> > > > > > > > > > > files that
> > > > > > > > need
> > > > > > > > > to
> > > > > > > > > > > be read into PB2NC, you can set a file window
around
> the
> > > > > > > > > > > valid
> > > > > > time
> > > > > > > > to
> > > > > > > > > > > allow files to be passed into the app. More
information
> > on
> > > > > > > > > > > that
> > > > > > > > > > > functionality is here:
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > >
> > >
> >
>
https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#using-
> > > > > > windows-to-find-valid-files
> > > > > > > > > > >
> > > > > > > > > > > If you are still having trouble getting this
> configured,
> > > > > > > > > > > you
> > > > > > could
> > > > > > > > send
> > > > > > > > > > me
> > > > > > > > > > > the paths of the files you are trying to read
and what
> > data
> > > > > > > > > > > is
> > > > > > > > > contained
> > > > > > > > > > in
> > > > > > > > > > > them so I can give you suggestions on what to
set for
> the
> > > > > > > > > configuration.
> > > > > > > > > > >
> > > > > > > > > > > Thanks,
> > > > > > > > > > > George
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > On Tue, Apr 6, 2021 at 2:17 PM Julie Prestopnik
via RT
> <
> > > > > > > > > > met_help at ucar.edu>
> > > > > > > > > > > wrote:
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > Tue Apr 06 14:16:42 2021: Request 99429 was
acted
> upon.
> > > > > > > > > > > > Transaction: Given to mccabe (George McCabe)
by
> jpresto
> > > > > > > > > > > > Queue: met_help
> > > > > > > > > > > > Subject: How to set time for hourly 12-24
hours
> > data
> > > > > > > > > > > > ?
> > > > > > > > > > > > Owner: mccabe
> > > > > > > > > > > > Requestors: zhuming at ucar.edu
> > > > > > > > > > > > Status: new
> > > > > > > > > > > > Ticket <URL:
> > > > > > > > >
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429
> > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > This transaction appears to have no content
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > --
> > > > > > > > > > > George McCabe - Software Engineer III
> > > > > > > > > > > National Center for Atmospheric Research
> > > > > > > > > > > Research Applications Laboratory
> > > > > > > > > > > 303-497-2768
> > > > > > > > > > > ---
> > > > > > > > > > > My working day may not be your working day.
Please do
> not
> > > > > > > > > > > feel
> > > > > > > > obliged
> > > > > > > > > to
> > > > > > > > > > > reply to this email outside of your normal
working
> hours.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > George McCabe - Software Engineer III
> > > > > > > > > National Center for Atmospheric Research
> > > > > > > > > Research Applications Laboratory
> > > > > > > > > 303-497-2768
> > > > > > > > > ---
> > > > > > > > > My working day may not be your working day. Please
do not
> > feel
> > > > > > obliged
> > > > > > > to
> > > > > > > > > reply to this email outside of your normal working
hours.
> > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > George McCabe - Software Engineer III
> > > > > > > National Center for Atmospheric Research
> > > > > > > Research Applications Laboratory
> > > > > > > 303-497-2768
> > > > > > > ---
> > > > > > > My working day may not be your working day. Please do
not feel
> > > > > > > obliged to
> > > > > > > reply to this email outside of your normal working
hours.
> > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > >
> > > >
> > > >
> > > >
> > >
> > >
> >
> > --
> > George McCabe - Software Engineer III
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > 303-497-2768
> > ---
> > My working day may not be your working day. Please do not feel
obliged to
> > reply to this email outside of your normal working hours.
> >
> >
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: How to set time for hourly 12-24 hours data ?
From: George McCabe
Time: Thu Apr 15 13:52:17 2021
Hi Zhuming,
There was indeed a bug in the code. Howard has fixed the issue with
this
pull request: https://github.com/dtcenter/MET/pull/1756
This fix will be included in the MET 10.0.0-beta5 release that is
tentatively scheduled for next week. This beta version can be
installed on
cheyenne so you can test that the update fixes your issue. I will
follow up
and let you know when the beta release is available on cheyenne.
Thanks,
George
On Wed, Apr 14, 2021 at 5:28 PM George McCabe <mccabe at ucar.edu> wrote:
> Hi Zhuming,
>
> I tried to get this to work on cheyenne with your data, but was not
> successful. I am able to read the data with plot_data_plane and
generate an
> image, so I think there may be a bug in the code. I have forwarded
info to
> one of our C++ developers to look into it. I will report back when I
know
> more.
>
> Thanks,
> George
>
> On Wed, Apr 14, 2021 at 4:13 PM zhuming at ucar.edu via RT
<met_help at ucar.edu>
> wrote:
>
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429 >
>>
>> Hi George,
>>
>> It still can't find the correct time. I tried
>>
>> FCST_VAR1_LEVELS = (*,*,*)
>> FCST_VAR1_LEVELS = ({valid?fmt=%Y%m%d_%H%M%S},*,*)
>> FCST_VAR1_LEVELS = ({valid?fmt=%Y%m%d_%H%M},*,*)
>> FCST_VAR1_LEVELS = ({valid?fmt=%Y%m%d%H%M},*,*)
>>
>> Thanks!
>>
>> Zhuming
>>
>> On Wed, Apr 14, 2021 at 3:41 PM George McCabe via RT
<met_help at ucar.edu>
>> wrote:
>>
>> > Hi Zhuming,
>> >
>> > It looks like you are requesting the first time in the 12 hour
file for
>> > each run:
>> >
>> > FCST_VAR1_LEVELS = (0,*,*)
>> > ...
>> > FCST_VAR2_LEVELS = (0,*,*)
>> >
>> > The first dimension in these files is the time offset. You will
need the
>> > correct index that corresponds to each valid time. Fortunately,
MET
>> makes
>> > this easy by allowing you to specify the valid time for a time
dimension
>> > explicitly and it will find the correct data. You can set:
>> >
>> > FCST_VAR1_LEVELS = ({valid?fmt=%Y%m%d_%H%M%S},*,*)
>> > ...
>> > FCST_VAR2_LEVELS = ({valid?fmt=%Y%m%d_%H%M%S},*,*)
>> >
>> > And it will substitute the valid time for each run. Let me know
if you
>> have
>> > any questions.
>> >
>> > Thanks,
>> > George
>> >
>> > On Wed, Apr 14, 2021 at 3:18 PM zhuming at ucar.edu via RT <
>> met_help at ucar.edu
>> > >
>> > wrote:
>> >
>> > >
>> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429 >
>> > >
>> > > Hi George,
>> > >
>> > > Thanks for your email, I just would like to send you for help.
>> > >
>> > > I already resolved the last problem with the WRF forecast,
thank you
>> for
>> > > your help.
>> > >
>> > > I also need to verify the MPAS model forecast. These 12 hour
>> > > MAPS forecasts are stored in a one file, such as
>> > > 2015071500/mpas_wrf_cf_2015071500.nc, has 12h forecasts
>> > > 2015071512/mpas_wrf_cf_2015071512.nc
>> > >
>> > > It seems MET is doing fine on the first 2 hours, but no matched
pairs
>> for
>> > > later hours.
>> > > I'm not sure what's wrong with it? the time is not matched?
>> > >
>> > > see the log file
>> > >
/glade/scratch/zhuming/metplus4/logs/master_metplus.log.20210414145101
>> > > config file /gpfs/fs1/work/zhuming/METplus4/PointStat2.conf
>> > >
>> > > Thanks,
>> > > Zhuming
>> > >
>> > > On Wed, Apr 14, 2021 at 1:56 PM George McCabe via RT <
>> met_help at ucar.edu>
>> > > wrote:
>> > >
>> > > > Hi Zhuming,
>> > > >
>> > > > Just following up on this ticket. Were you able to get this
>> resolved?
>> > Let
>> > > > me know if you need more assistance.
>> > > >
>> > > > Thanks,
>> > > > George
>> > > >
>> > > > On Thu Apr 08 10:04:44 2021, mccabe wrote:
>> > > > > Hi Zhuming,
>> > > > >
>> > > > > Typically directories that contain data for an entire day
are
>> named
>> > > > > with
>> > > > > YYYYMMDD, so your filename template tag would be
>> {valid?fmt=%Y%m%d},
>> > > > > however that may not always be the case, as you are seeing
here.
>> > There
>> > > > > are
>> > > > > a couple ways to handle this.
>> > > > >
>> > > > > If you know that the directory will always be YYYYMMDD with
00 at
>> the
>> > > > > end,
>> > > > > you can simply put 00 in the template instead of adding %H
in the
>> > tag,
>> > > > > i.e.
>> > > > >
>> > > > > FCST_POINT_STAT_INPUT_TEMPLATE =
>> > > > > {valid?fmt=%Y%m%d}00/wrfout_d02_{valid?fmt=%Y-%m-%d_%H}:00:
>> 00_cf.nc
>> > > > >
>> > > > > If the HH value can vary, for instance if there are 2
directories
>> > > > > created
>> > > > > per day (i.e. 2015071400 and 2015071412), then you can use
the
>> > > > > truncate
>> > > > > keyword to handle it.
>> > > > >
>> > > > > If directories are created every 24 hours:
>> > > > >
>> > > > > FCST_POINT_STAT_INPUT_TEMPLATE =
>> > > > > {valid?fmt=%Y%m%d%H?truncate=24H}/wrfout_d02_{valid?fmt=%Y-
%m-
>> > > > > %d_%H}:00:
>> > > > > 00_cf.nc
>> > > > >
>> > > > > If directories are created every 12 hours:
>> > > > >
>> > > > > FCST_POINT_STAT_INPUT_TEMPLATE =
>> > > > > {valid?fmt=%Y%m%d%H?truncate=12H}/wrfout_d02_{valid?fmt=%Y-
%m-
>> > > > > %d_%H}:00:
>> > > > > 00_cf.nc
>> > > > >
>> > > > > Let me know if that doesn't work for you.
>> > > > >
>> > > > > Thanks,
>> > > > > George
>> > > > >
>> > > > > On Thu, Apr 8, 2021 at 9:46 AM zhuming at ucar.edu via RT
>> > > > > <met_help at ucar.edu>
>> > > > > wrote:
>> > > > >
>> > > > > >
>> > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429 >
>> > > > > >
>> > > > > > Hi George,
>> > > > > >
>> > > > > > Processing Obs data using "PB2NC" are right now.
>> > > > > >
>> > > > > > Now I'm using "PointStat" , I set VALID_BEG = 2015071500,
>> > VALID_END =
>> > > > > > 2015071501. It looks for the forecast data 2015071500/
>> > > > > > wrfout_d02_2015-07-15_00:00:00_cf.nc, and 2015071501/
>> > > > > > wrfout_d02_2015-07-15_01:00:00_cf.nc.
>> > > > > > But my forecast data are all put in dir of 2015071500/.
>> > > > > > So how could I set FCST_POINT_STAT_INPUT_TEMPLATE to get
the
>> right
>> > > > > > fcst
>> > > > > > data?
>> > > > > >
>> > > > > > FCST_POINT_STAT_INPUT_TEMPLATE =
>> > > > > > {valid?fmt=%Y%m%d%H}/wrfout_d02_{valid?fmt=%Y-%m-
%d_%H}:00:
>> > 00_cf.nc
>> > > > > >
>> > > > > >
>> > > > > > config file:
/gpfs/fs1/work/zhuming/METplus4/PointStat1.conf
>> > > > > >
>> > > > > > log file:
>> > > > > >
>> > >
/glade/scratch/zhuming/metplus4/logs/master_metplus.log.20210408085450
>> > > > > >
>> > > > > >
>> > > > > > Thanks,
>> > > > > >
>> > > > > > Zhuming
>> > > > > >
>> > > > > > On Wed, Apr 7, 2021 at 5:43 PM George McCabe via RT
>> > > > > > <met_help at ucar.edu>
>> > > > > > wrote:
>> > > > > >
>> > > > > > > Hi Zhuming,
>> > > > > > >
>> > > > > > > Looking at the log output, it looks like the data is
stored in
>> > the
>> > > > > > prepBUFR
>> > > > > > > files differently that you may have thought:
>> > > > > > >
>> > > > > > > DEBUG 2: PrepBufr Time Center: 20150715_000000
>> > > > > > > DEBUG 2: Searching Time Window: 20150715_030000
to
>> > > > > > 20150715_040000
>> > > > > > > 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70%
75% 80%
>> > 85%
>> > > > > > > 90%
>> > > > > > 95%
>> > > > > > > 100%
>> > > > > > >
>> > > > > > > DEBUG 2: Total Messages processed =
590836
>> > > > > > > DEBUG 2: Rejected based on message type = 0
>> > > > > > > DEBUG 2: Rejected based on station id = 0
>> > > > > > > DEBUG 2: Rejected based on valid time =
590815
>> > > > > > > DEBUG 2: Rejected based on masking grid = 20
>> > > > > > > DEBUG 2: Rejected based on masking polygon = 0
>> > > > > > > DEBUG 2: Rejected based on elevation = 0
>> > > > > > > DEBUG 2: Rejected based on pb report type = 1
>> > > > > > > DEBUG 2: Rejected based on input report type = 0
>> > > > > > > DEBUG 2: Rejected based on instrument type = 0
>> > > > > > > DEBUG 2: Rejected based on zero observations = 0
>> > > > > > > DEBUG 2: Total Messages retained = 0
>> > > > > > > DEBUG 2: Total observations retained or derived = 0
>> > > > > > > DEBUG 1: Obs time between *20150714_210000 and
>> 20150715_030000*
>> > > > > > > WARNING:
>> > > > > > > WARNING: process_pbfile() -> No PrepBufr messages
retained
>> from
>> > > > > > > file:
>> > > > > > > /glade/collections/rda/data/ds337.0/prepnr/2015/
>> > > > > > > prepbufr.gdas.2015071500.nr
>> > > > > > > WARNING:
>> > > > > > > ERROR :
>> > > > > > > ERROR : write_netcdf_hdr_data() -> No PrepBufr
messages
>> > retained.
>> > > > > > Nothing
>> > > > > > > to write.
>> > > > > > > ERROR :
>> > > > > > >
>> > > > > > > The prepbufr.gdas.2015071500.nr file actually contains
data
>> from
>> > > > > > > 20150714_210000 to 20150715_030000, not 20150715_000000
to
>> > > > > > 20150715_060000.
>> > > > > > >
>> > > > > > > To get the correct file for each run, you will have to
shift
>> the
>> > > > > > > valid
>> > > > > > time
>> > > > > > > before truncating it so that it gets the correct file
for each
>> > run.
>> > > > > > Instead
>> > > > > > > of setting:
>> > > > > > >
>> > > > > > > PB2NC_INPUT_TEMPLATE =
>> > > > > > >
>> >
{valid?fmt=%Y}/prepbufr.gdas.{valid?fmt=%Y%m%d%H?truncate=21600}.nr
>> > > > > > >
>> > > > > > > You will have to also shift forward 3 hours, i.e.:
>> > > > > > >
>> > > > > > > PB2NC_INPUT_TEMPLATE =
>> > > > > > >
>> > > > > > >
>> > > > > >
>> > > >
>> > >
>> >
>>
{valid?fmt=%Y}/prepbufr.gdas.{valid?fmt=%Y%m%d%H?shift=3H?truncate=21600}.nr
>> > > > > > >
>> > > > > > > This says to add 3 hours to the current valid time
before
>> > > > > > > truncating to
>> > > > > > the
>> > > > > > > nearest 6 hours.
>> > > > > > >
>> > > > > > > Valid: 20150715 0Z + 3 = 3Z truncate 6H = 0Z
>> > > > > > > File: prepbufr.gdas.2015071500.nr
>> > > > > > >
>> > > > > > > Valid: 20150715 1Z + 3 = 4Z truncate 6H = 0Z
>> > > > > > > File: prepbufr.gdas.2015071500.nr
>> > > > > > >
>> > > > > > > Valid: 20150715 2Z + 3 = 5Z truncate 6H = 0Z
>> > > > > > > File: prepbufr.gdas.2015071500.nr
>> > > > > > >
>> > > > > > > Valid: 20150715 3Z + 3 = 6Z truncate 6H = 6Z
>> > > > > > > File: prepbufr.gdas.2015071506.nr
>> > > > > > >
>> > > > > > > Let me know if that doesn't work.
>> > > > > > >
>> > > > > > > Thanks,
>> > > > > > > George
>> > > > > > >
>> > > > > > >
>> > > > > > > On Wed, Apr 7, 2021 at 5:32 PM zhuming at ucar.edu via RT
<
>> > > > > > met_help at ucar.edu>
>> > > > > > > wrote:
>> > > > > > >
>> > > > > > > >
>> > > > > > > > <URL:
>> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429
>> > >
>> > > > > > > >
>> > > > > > > > Hi George,
>> > > > > > > >
>> > > > > > > > Now makes me confused is why processed obs of several
times
>> are
>> > > > > > missing.
>> > > > > > > > There has files of time at 00,01,02, but no 03,04,05;
has
>> > > > > > > > 06,07,08, but
>> > > > > > > no
>> > > > > > > > 09,10,11, has 12,13,14, no 15,16,17; has 18,19,20,
but with
>> > > > > > > > 21,no
>> > > > > > 22,23.
>> > > > > > > > From yours and mines:
>> > > > > > > >
>> > > > > > > > ls -l /glade/u/home/mccabe/data/out/pb2nc1/
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > -rw-r--r-- 1 mccabe ral 10141339 Apr 6 17:52
>> > pbs.2015071500.nc
>> > > > > > > >
>> > > > > > > > -rw-r--r-- 1 mccabe ral 5413907 Apr 6 17:56
>> > pbs.2015071501.nc
>> > > > > > > >
>> > > > > > > > -rw-r--r-- 1 mccabe ral 4923711 Apr 6 18:00
>> > pbs.2015071502.nc
>> > > > > > > >
>> > > > > > > > -rw-r--r-- 1 mccabe ral 5724447 Apr 6 18:14
>> > pbs.2015071506.nc
>> > > > > > > >
>> > > > > > > > -rw-r--r-- 1 mccabe ral 5280463 Apr 6 18:18
>> > pbs.2015071507.nc
>> > > > > > > >
>> > > > > > > > -rw-r--r-- 1 mccabe ral 4706119 Apr 6 18:22
>> > pbs.2015071508.nc
>> > > > > > > >
>> > > > > > > > -rw-r--r-- 1 mccabe ral 9878263 Apr 6 18:36
>> > pbs.2015071512.nc
>> > > > > > > >
>> > > > > > > > -rw-r--r-- 1 mccabe ral 5392527 Apr 6 18:40
>> > pbs.2015071513.nc
>> > > > > > > >
>> > > > > > > > -rw-r--r-- 1 mccabe ral 96 Apr 6 18:40
>> > pbs.2015071514.nc
>> > > > > > > >
>> > > > > > > > -rw-r--r-- 1 mccabe ral 96 Apr 6 18:42
>> > pbs.2015071515.nc
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > cheyenne3:/glade/scratch/zhuming/metplus4/pb2nc1>l
>> > > > > > > >
>> > > > > > > > total 96112
>> > > > > > > >
>> > > > > > > > -rw-r--r-- 1 zhuming ncar 11550172 Apr 7 15:11
>> > > pbs.2015071500.nc
>> > > > > > > >
>> > > > > > > > -rw-r--r-- 1 zhuming ncar 6615956 Apr 7 15:14
>> > > pbs.2015071501.nc
>> > > > > > > >
>> > > > > > > > -rw-r--r-- 1 zhuming ncar 5984328 Apr 7 15:18
>> > > pbs.2015071502.nc
>> > > > > > > >
>> > > > > > > > -rw-r--r-- 1 zhuming ncar 6980712 Apr 7 15:33
>> > > pbs.2015071506.nc
>> > > > > > > >
>> > > > > > > > -rw-r--r-- 1 zhuming ncar 6455824 Apr 7 15:36
>> > > pbs.2015071507.nc
>> > > > > > > >
>> > > > > > > > -rw-r--r-- 1 zhuming ncar 5749432 Apr 7 15:40
>> > > pbs.2015071508.nc
>> > > > > > > >
>> > > > > > > > -rw-r--r-- 1 zhuming ncar 11284936 Apr 7 15:55
>> > > pbs.2015071512.nc
>> > > > > > > >
>> > > > > > > > -rw-r--r-- 1 zhuming ncar 6600888 Apr 7 15:59
>> > > pbs.2015071513.nc
>> > > > > > > >
>> > > > > > > > -rw-r--r-- 1 zhuming ncar 5868168 Apr 7 16:03
>> > > pbs.2015071514.nc
>> > > > > > > >
>> > > > > > > > -rw-r--r-- 1 zhuming ncar 7273092 Apr 7 16:18
>> > > pbs.2015071518.nc
>> > > > > > > >
>> > > > > > > > -rw-r--r-- 1 zhuming ncar 6656344 Apr 7 16:22
>> > > pbs.2015071519.nc
>> > > > > > > >
>> > > > > > > > -rw-r--r-- 1 zhuming ncar 6109788 Apr 7 16:27
>> > > pbs.2015071520.nc
>> > > > > > > >
>> > > > > > > > -rw-r--r-- 1 zhuming ncar 58808 Apr 7 16:31
>> > > pbs.2015071521.nc
>> > > > > > > >
>> > > > > > > > -rw-r--r-- 1 zhuming ncar 11132440 Apr 7 16:43
>> > > pbs.2015071600.nc
>> > > > > > > >
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > Thanks,
>> > > > > > > >
>> > > > > > > > Zhuming
>> > > > > > > >
>> > > > > > > > On Tue, Apr 6, 2021 at 6:34 PM George McCabe via RT
>> > > > > > > > <met_help at ucar.edu
>> > > > > > >
>> > > > > > > > wrote:
>> > > > > > > >
>> > > > > > > > > Hi Zhuming,
>> > > > > > > > >
>> > > > > > > > > Thanks for the information. You will need to use
the
>> truncate
>> > > > > > > > > keyword
>> > > > > > > to
>> > > > > > > > > find the files, then set the valid beg/end range to
1
>> hour.
>> > > > > > > > > This is
>> > > > > > > what
>> > > > > > > > I
>> > > > > > > > > changed from your PB2NC1.conf config file:
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > *PB2NC_VALID_BEGIN =
{valid?fmt=%Y%m%d_%H}PB2NC_VALID_END
>> =
>> > > > > > > > > {valid?fmt=%Y%m%d_%H?shift=1H}*
>> > > > > > > > >
>> > > > > > > > > *PB2NC_INPUT_TEMPLATE =
>> > > > > > > > >
>> > > >
{valid?fmt=%Y}/prepbufr.gdas.{valid?fmt=%Y%m%d%H?truncate=21600}.nr*
>> > > > > > > > >
>> > > > > > > > > My config file is on cheyenne here:
>> > > > > > > > > /glade/u/home/mccabe/PB2NC1.conf
>> > > > > > > > >
>> > > > > > > > > I just started running it and the output is being
written
>> > here:
>> > > > > > > > > /glade/u/home/mccabe/data/out/pb2nc1
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > For PointStat, you will want to loop over the same
times.
>> I
>> > > > > > > > > noticed
>> > > > > > you
>> > > > > > > > > specify message type in the VAR<n>_OPTIONS, but you
do not
>> > need
>> > > > > > > > > to do
>> > > > > > > > this
>> > > > > > > > > since there is a POINT_STAT_MESSAGE_TYPE variable
that
>> > defines
>> > > > > > > > > this.
>> > > > > > > This
>> > > > > > > > > is what I changed in your PointStat1.conf file:
>> > > > > > > > >
>> > > > > > > > > *VALID_END = 2015071600*
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > *VALID_INCREMENT = 1HPOINT_STAT_OBS_VALID_BEG
>> > > > > > =POINT_STAT_OBS_VALID_BEG
>> > > > > > > > =*
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > *POINT_STAT_MESSAGE_TYPE = ADPSFC*
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > *#OBS_VAR1_OPTIONS = message_type =
>> > > > > > > > > ["ADPSFC"];#OBS_VAR2_OPTIONS =
>> > > > > > > > > message_type = ["ADPSFC"];#OBS_VAR3_OPTIONS =
>> message_type =
>> > > > > > > > > ["ADPSFC"];#OBS_POINT_STAT_INPUT_DIR =
>> > > > > > > > > /glade/scratch/zhuming/metplus4/pb2nc1*
>> > > > > > > > >
>> > > > > > > > > *OBS_POINT_STAT_INPUT_DIR = {OUTPUT_BASE}/pb2nc1*
>> > > > > > > > >
>> > > > > > > > > Or you can view my file in the same directory. I
usually
>> try
>> > to
>> > > > > > > > > test
>> > > > > > > > with a
>> > > > > > > > > single run time to make sure it is working
properly, so I
>> set
>> > > > > > VALID_END
>> > > > > > > > > back to 2015071500. I ran it and it produced an
output
>> file,
>> > > > > > > > > but
>> > > > > > there
>> > > > > > > > were
>> > > > > > > > > no values besides the header. I checked the log
file and
>> saw
>> > > > > > > > > this:
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > *DEBUG 2: Processing T_sfc(0,*,*) versus TMP/Z2,
for
>> > > > > > > > > observation type
>> > > > > > > > > ADPSFC, over region FULL, for interpolation method
>> BILIN(4),
>> > > > > > > > > using 0
>> > > > > > > > > matched pairs.DEBUG 3: Number of matched pairs =
0DEBUG
>> 3:
>> > > > > > > Observations
>> > > > > > > > > processed = 393373DEBUG 3: Rejected: station id
=
>> > > > > > > > > 0DEBUG 3:
>> > > > > > > > > Rejected: obs type = 390528DEBUG 3:
Rejected: valid
>> > time
>> > > > > > =
>> > > > > > > > > 0DEBUG 3: Rejected: bad obs value = 0DEBUG 3:
Rejected:
>> off
>> > > > > > > > > the
>> > > > > > grid
>> > > > > > > > =
>> > > > > > > > > 2586DEBUG 3: Rejected: topography = 0DEBUG 3:
>> Rejected:
>> > > > > > > > > level
>> > > > > > > > mismatch
>> > > > > > > > > = 259DEBUG 3: Rejected: quality marker = 0DEBUG 3:
>> Rejected:
>> > > > > > message
>> > > > > > > > type
>> > > > > > > > > = 0DEBUG 3: Rejected: masking region = 0DEBUG 3:
>> Rejected:
>> > bad
>> > > > > > fcst
>> > > > > > > > > value = 0DEBUG 3: Rejected: bad climo mean =
0DEBUG 3:
>> > > > > > > > > Rejected:
>> > > > > > bad
>> > > > > > > > > climo stdev = 0DEBUG 3: Rejected: duplicates =
0DEBUG
>> > > > > > > > > 2:DEBUG 2:
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > >
>> > >
>> >
>>
--------------------------------------------------------------------------------DEBUG
>> > > > > > > > > 2:DEBUG 2: Processing u_10m_gr(0,*,*) versus
UGRD/Z10, for
>> > > > > > observation
>> > > > > > > > type
>> > > > > > > > > ADPSFC, over region FULL, for interpolation method
>> BILIN(4),
>> > > > > > > > > using 0
>> > > > > > > > > matched pairs.DEBUG 3: Number of matched pairs =
0DEBUG
>> 3:
>> > > > > > > Observations
>> > > > > > > > > processed = 393373*
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > *DEBUG 3: Rejected: station id = 0DEBUG 3:
Rejected:
>> obs
>> > > > > > > > > type
>> > > > > > > > =
>> > > > > > > > > 382965DEBUG 3: Rejected: valid time = 0DEBUG
3:
>> > Rejected:
>> > > > > > > > > bad
>> > > > > > obs
>> > > > > > > > > value = 0DEBUG 3: Rejected: off the grid =
9240DEBUG
>> 3:
>> > > > > > Rejected:
>> > > > > > > > > topography = 0DEBUG 3: Rejected: level
mismatch =
>> > > > > > > > > 1168DEBUG 3:
>> > > > > > > > > Rejected: quality marker = 0DEBUG 3: Rejected:
message
>> type
>> > > > > > > > > =
>> > > > > > > 0DEBUG
>> > > > > > > > 3:
>> > > > > > > > > Rejected: masking region = 0DEBUG 3: Rejected: bad
fcst
>> > value
>> > > > > > > > > =
>> > > > > > > 0DEBUG
>> > > > > > > > 3:
>> > > > > > > > > Rejected: bad climo mean = 0DEBUG 3: Rejected: bad
climo
>> > stdev
>> > > > > > > > > =
>> > > > > > > 0DEBUG
>> > > > > > > > 3:
>> > > > > > > > > Rejected: duplicates = 0DEBUG 2:DEBUG 2:
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > >
>> > >
>> >
>>
--------------------------------------------------------------------------------DEBUG
>> > > > > > > > > 2:DEBUG 2: Processing v_10m_gr(0,*,*) versus
VGRD/Z10, for
>> > > > > > observation
>> > > > > > > > type
>> > > > > > > > > ADPSFC, over region FULL, for interpolation method
>> BILIN(4),
>> > > > > > > > > using 0
>> > > > > > > > > matched pairs.DEBUG 3: Number of matched pairs =
0DEBUG
>> 3:
>> > > > > > > Observations
>> > > > > > > > > processed = 393373DEBUG 3: Rejected: station id
=
>> > > > > > > > > 0DEBUG 3:
>> > > > > > > > > Rejected: obs type = 382965DEBUG 3:
Rejected: valid
>> > time
>> > > > > > =
>> > > > > > > > > 0DEBUG 3: Rejected: bad obs value = 0DEBUG 3:
Rejected:
>> off
>> > > > > > > > > the
>> > > > > > grid
>> > > > > > > > =
>> > > > > > > > > 9240DEBUG 3: Rejected: topography = 0DEBUG 3:
>> Rejected:
>> > > > > > > > > level
>> > > > > > > > mismatch
>> > > > > > > > > = 1168DEBUG 3: Rejected: quality marker = 0DEBUG
3:
>> > Rejected:
>> > > > > > message
>> > > > > > > > > type = 0DEBUG 3: Rejected: masking region =
0DEBUG 3:
>> > > > > > > > > Rejected:
>> > > > > > bad
>> > > > > > > > > fcst value = 0DEBUG 3: Rejected: bad climo mean =
>> 0DEBUG 3:
>> > > > > > Rejected:
>> > > > > > > > bad
>> > > > > > > > > climo stdev = 0*
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > *DEBUG 3: Rejected: duplicates = 0DEBUG
2:DEBUG 2:
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > >
>> > >
>> >
>>
--------------------------------------------------------------------------------DEBUG
>> > > > > > > > > 2:DEBUG 1: Output file:
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > >
>> > >
>> >
>>
/glade/u/home/mccabe/data/out/point_stat1/point_stat_000000L_20150715_000000V.stat*
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > This describes the reason why each observation was
>> rejected.
>> > > > > > > > > Some
>> > > > > > > > > observations didn't match the message type you
requested,
>> > some
>> > > > > > > > > were
>> > > > > > not
>> > > > > > > > > within the forecast grid, and some did not match
the
>> vertical
>> > > > > > > > > level.
>> > > > > > > You
>> > > > > > > > > will have to make adjustments to your settings so
that the
>> > data
>> > > > > > matches
>> > > > > > > > up
>> > > > > > > > > properly. Maybe the grid you are using in the PB2NC
config
>> > file
>> > > > > > > > > does
>> > > > > > > not
>> > > > > > > > > match the forecast grid and you are missing values.
If
>> that
>> > is
>> > > > > > > > > the
>> > > > > > > case,
>> > > > > > > > > you will have to reconfigure and rerun PB2NC to get
>> output on
>> > > > > > > > > the
>> > > > > > > correct
>> > > > > > > > > gr5id. Maybe the observation points don't line up
with the
>> > > > > > > > > vertical
>> > > > > > > level
>> > > > > > > > > of the model fields. You can use a range of level
values
>> for
>> > > > > > > > > the
>> > > > > > > > > observation data to allow data within that range,
i.e.
>> Z0-10.
>> > > > > > > > >
>> > > > > > > > > Let me know if you are still having issues getting
this to
>> > work
>> > > > > > > > > and I
>> > > > > > > can
>> > > > > > > > > help out.
>> > > > > > > > >
>> > > > > > > > > Thanks,
>> > > > > > > > > George
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > On Tue, Apr 6, 2021 at 5:17 PM zhuming at ucar.edu via
RT <
>> > > > > > > > met_help at ucar.edu>
>> > > > > > > > > wrote:
>> > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > <URL:
>> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429
>> > > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > > Hi George,
>> > > > > > > > > >
>> > > > > > > > > > Thanks for your reply!
>> > > > > > > > > >
>> > > > > > > > > > I'm still confused, I need to output hourly obs
data to
>> do
>> > > > > > > > > > hourly
>> > > > > > > > > > verification.
>> > > > > > > > > > The original GDAS Prepbufr data has 4 files one
day,
>> should
>> > > > > > > > > > with 6
>> > > > > > > > hours
>> > > > > > > > > > obs data in one file.
>> > > > > > > > > >
>> > > > > > > > > > After "PB2NC", I can see pbs.2015071500.nc (obs
time is
>> > 00?
>> > > > > > > > > > or 6
>> > > > > > > times
>> > > > > > > > > of
>> > > > > > > > > > 00,01,02,03,04,05?), pbs.2015071506.nc...
>> > > > > > > > > >
>> > > > > > > > > > my config file :
>> > > > > > cheyenne:/gpfs/fs1/work/zhuming/METplus4/PB2NC1.conf
>> > > > > > > > > > and
/gpfs/fs1/work/zhuming/METplus4/PointStat1.conf
>> > > > > > > > > >
>> > > > > > > > > > Thanks,
>> > > > > > > > > > Zhuming
>> > > > > > > > > >
>> > > > > > > > > > On Tue, Apr 6, 2021 at 4:23 PM George McCabe via
RT <
>> > > > > > > met_help at ucar.edu
>> > > > > > > > >
>> > > > > > > > > > wrote:
>> > > > > > > > > >
>> > > > > > > > > > > Hi Zhuming,
>> > > > > > > > > > >
>> > > > > > > > > > > To run the tools hourly you would set the
>> VALID_INCREMENT
>> > > > > > > > > > > value
>> > > > > > for
>> > > > > > > > 1H.
>> > > > > > > > > > > Here is an example that runs once per hour for
a day
>> from
>> > > > > > > 2015-07-15
>> > > > > > > > 0Z
>> > > > > > > > > > to
>> > > > > > > > > > > 2017-07-16 0Z:
>> > > > > > > > > > >
>> > > > > > > > > > > VALID_TIME_FMT = %Y%m%d%H
>> > > > > > > > > > >
>> > > > > > > > > > > VALID_BEG = 2015071500
>> > > > > > > > > > >
>> > > > > > > > > > > VALID_END = 2015071600
>> > > > > > > > > > >
>> > > > > > > > > > > VALID_INCREMENT = 1H
>> > > > > > > > > > >
>> > > > > > > > > > > LEAD_SEQ = 0
>> > > > > > > > > > >
>> > > > > > > > > > > Each run time will use the PB2NC_INPUT_TEMPLATE
to
>> find
>> > > > > > > > > > > files
>> > > > > > whose
>> > > > > > > > > > > filenames match the time you need.
>> > > > > > > > > > >
>> > > > > > > > > > > To read prepBUFR observations for 12 hours, you
will
>> want
>> > > > > > > > > > > to set
>> > > > > > > > > > > PB2NC_VALID_BEGIN and PB2NC_VALID_END to
specify the
>> > range
>> > > > > > > > > > > around
>> > > > > > > the
>> > > > > > > > > > valid
>> > > > > > > > > > > time to process. These values depend on what is
>> contained
>> > > > > > > > > > > inside
>> > > > > > > your
>> > > > > > > > > > > prepBUFR files. If you want to use data from
the valid
>> > time
>> > > > > > > > > > > of
>> > > > > > the
>> > > > > > > > > > filename
>> > > > > > > > > > > backwards 12 hours, you would set:
>> > > > > > > > > > >
>> > > > > > > > > > > PB2NC_VALID_BEGIN =
{valid?fmt=%Y%m%d_%H?shift=-12H }
>> > > > > > > > > > > PB2NC_VALID_END = {valid?fmt=%Y%m%d_%H}
>> > > > > > > > > > >
>> > > > > > > > > > > For a runtime of 2015071512, this would be
substituted
>> > as:
>> > > > > > > > > > >
>> > > > > > > > > > > valid_beg: 20150715_00
>> > > > > > > > > > > valid_end: 20150715_12
>> > > > > > > > > > >
>> > > > > > > > > > > If you want to create an output file for each
hour in
>> > PB2NC
>> > > > > > > > > > > you
>> > > > > > > would
>> > > > > > > > > > need
>> > > > > > > > > > > to make sure that the hour is specified in the
>> > > > > > > PB2NC_OUTPUT_TEMPLATE.
>> > > > > > > > > > >
>> > > > > > > > > > > PointStat has similar config variables to set
the
>> valid
>> > beg
>> > > > > > > > > > > and
>> > > > > > end
>> > > > > > > > > range
>> > > > > > > > > > > to use (POINT_STAT_OBS_VALID_BEG and
>> > > > > > > > > > > POINT_STAT_OBS_VALID_END).
>> > > > > > > > > > >
>> > > > > > > > > > > If your prepBUFR files are separated out into
multiple
>> > > > > > > > > > > files that
>> > > > > > > > need
>> > > > > > > > > to
>> > > > > > > > > > > be read into PB2NC, you can set a file window
around
>> the
>> > > > > > > > > > > valid
>> > > > > > time
>> > > > > > > > to
>> > > > > > > > > > > allow files to be passed into the app. More
>> information
>> > on
>> > > > > > > > > > > that
>> > > > > > > > > > > functionality is here:
>> > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > >
>> > >
>> >
>>
https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#using-
>> > > > > > windows-to-find-valid-files
>> > > > > > > > > > >
>> > > > > > > > > > > If you are still having trouble getting this
>> configured,
>> > > > > > > > > > > you
>> > > > > > could
>> > > > > > > > send
>> > > > > > > > > > me
>> > > > > > > > > > > the paths of the files you are trying to read
and what
>> > data
>> > > > > > > > > > > is
>> > > > > > > > > contained
>> > > > > > > > > > in
>> > > > > > > > > > > them so I can give you suggestions on what to
set for
>> the
>> > > > > > > > > configuration.
>> > > > > > > > > > >
>> > > > > > > > > > > Thanks,
>> > > > > > > > > > > George
>> > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > > On Tue, Apr 6, 2021 at 2:17 PM Julie Prestopnik
via
>> RT <
>> > > > > > > > > > met_help at ucar.edu>
>> > > > > > > > > > > wrote:
>> > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > > > Tue Apr 06 14:16:42 2021: Request 99429 was
acted
>> upon.
>> > > > > > > > > > > > Transaction: Given to mccabe (George McCabe)
by
>> jpresto
>> > > > > > > > > > > > Queue: met_help
>> > > > > > > > > > > > Subject: How to set time for hourly 12-
24 hours
>> > data
>> > > > > > > > > > > > ?
>> > > > > > > > > > > > Owner: mccabe
>> > > > > > > > > > > > Requestors: zhuming at ucar.edu
>> > > > > > > > > > > > Status: new
>> > > > > > > > > > > > Ticket <URL:
>> > > > > > > > >
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429
>> > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > > >
>> > > > > > > > > > > > This transaction appears to have no content
>> > > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > > > --
>> > > > > > > > > > > George McCabe - Software Engineer III
>> > > > > > > > > > > National Center for Atmospheric Research
>> > > > > > > > > > > Research Applications Laboratory
>> > > > > > > > > > > 303-497-2768
>> > > > > > > > > > > ---
>> > > > > > > > > > > My working day may not be your working day.
Please do
>> not
>> > > > > > > > > > > feel
>> > > > > > > > obliged
>> > > > > > > > > to
>> > > > > > > > > > > reply to this email outside of your normal
working
>> hours.
>> > > > > > > > > > >
>> > > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > > >
>> > > > > > > > >
>> > > > > > > > > --
>> > > > > > > > > George McCabe - Software Engineer III
>> > > > > > > > > National Center for Atmospheric Research
>> > > > > > > > > Research Applications Laboratory
>> > > > > > > > > 303-497-2768
>> > > > > > > > > ---
>> > > > > > > > > My working day may not be your working day. Please
do not
>> > feel
>> > > > > > obliged
>> > > > > > > to
>> > > > > > > > > reply to this email outside of your normal working
hours.
>> > > > > > > > >
>> > > > > > > > >
>> > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > > > --
>> > > > > > > George McCabe - Software Engineer III
>> > > > > > > National Center for Atmospheric Research
>> > > > > > > Research Applications Laboratory
>> > > > > > > 303-497-2768
>> > > > > > > ---
>> > > > > > > My working day may not be your working day. Please do
not feel
>> > > > > > > obliged to
>> > > > > > > reply to this email outside of your normal working
hours.
>> > > > > > >
>> > > > > > >
>> > > > > >
>> > > > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > >
>> > >
>> >
>> > --
>> > George McCabe - Software Engineer III
>> > National Center for Atmospheric Research
>> > Research Applications Laboratory
>> > 303-497-2768
>> > ---
>> > My working day may not be your working day. Please do not feel
obliged
>> to
>> > reply to this email outside of your normal working hours.
>> >
>> >
>>
>>
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: How to set time for hourly 12-24 hours data ?
From: zhuming at ucar.edu
Time: Thu Apr 22 09:52:34 2021
Hi George,
My verification result shows that there is a big error on a non-
standard
level, so my question is:
Is the forecast grid match to Obs, or obs station match to Forecast? I
need
to do is the forecast grid interpolated to the obs location and
vertical
levels. So how to set the levels?
Thanks,
Zhuming
FCST_VAR3_NAME = v_10m_tr
FCST_VAR3_LEVELS = (0,*,*)
FCST_VAR3_THRESH = >=5, <5
OBS_VAR3_NAME = VGRD
OBS_VAR3_LEVELS = Z10
OBS_VAR3_THRESH = >=5, <5
OBS_VAR3_OPTIONS = message_type = ["ADPSFC"];
FCST_VAR3_NAME = v_tr_p
FCST_VAR3_LEVELS =
(0,0,*,*),(0,1,*,*),(0,2,*,*),(0,3,*,*),(0,4,*,*),(0,5,*,*),(0,6,*,*),(0,7,*,*),(0,8,*,*),(0,9,*,*)
FCST_VAR3_THRESH = >=5, <5
OBS_VAR3_NAME = VGRD
OBS_VAR3_LEVELS =
P900-950,P825-875,P675-725,P575-625,P475-525,P375-425,P275-325,P225-
275,P175-225,P75-125
OBS_VAR3_THRESH = >=5, <5
OBS_VAR3_OPTIONS = message_type = ["ADPUPA"];
On Thu, Apr 15, 2021 at 1:52 PM George McCabe via RT
<met_help at ucar.edu>
wrote:
> Hi Zhuming,
>
> There was indeed a bug in the code. Howard has fixed the issue with
this
> pull request: https://github.com/dtcenter/MET/pull/1756
> This fix will be included in the MET 10.0.0-beta5 release that is
> tentatively scheduled for next week. This beta version can be
installed on
> cheyenne so you can test that the update fixes your issue. I will
follow up
> and let you know when the beta release is available on cheyenne.
>
> Thanks,
> George
>
> On Wed, Apr 14, 2021 at 5:28 PM George McCabe <mccabe at ucar.edu>
wrote:
>
> > Hi Zhuming,
> >
> > I tried to get this to work on cheyenne with your data, but was
not
> > successful. I am able to read the data with plot_data_plane and
generate
> an
> > image, so I think there may be a bug in the code. I have forwarded
info
> to
> > one of our C++ developers to look into it. I will report back when
I know
> > more.
> >
> > Thanks,
> > George
> >
> > On Wed, Apr 14, 2021 at 4:13 PM zhuming at ucar.edu via RT <
> met_help at ucar.edu>
> > wrote:
> >
> >>
> >> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429 >
> >>
> >> Hi George,
> >>
> >> It still can't find the correct time. I tried
> >>
> >> FCST_VAR1_LEVELS = (*,*,*)
> >> FCST_VAR1_LEVELS = ({valid?fmt=%Y%m%d_%H%M%S},*,*)
> >> FCST_VAR1_LEVELS = ({valid?fmt=%Y%m%d_%H%M},*,*)
> >> FCST_VAR1_LEVELS = ({valid?fmt=%Y%m%d%H%M},*,*)
> >>
> >> Thanks!
> >>
> >> Zhuming
> >>
> >> On Wed, Apr 14, 2021 at 3:41 PM George McCabe via RT
<met_help at ucar.edu
> >
> >> wrote:
> >>
> >> > Hi Zhuming,
> >> >
> >> > It looks like you are requesting the first time in the 12 hour
file
> for
> >> > each run:
> >> >
> >> > FCST_VAR1_LEVELS = (0,*,*)
> >> > ...
> >> > FCST_VAR2_LEVELS = (0,*,*)
> >> >
> >> > The first dimension in these files is the time offset. You will
need
> the
> >> > correct index that corresponds to each valid time. Fortunately,
MET
> >> makes
> >> > this easy by allowing you to specify the valid time for a time
> dimension
> >> > explicitly and it will find the correct data. You can set:
> >> >
> >> > FCST_VAR1_LEVELS = ({valid?fmt=%Y%m%d_%H%M%S},*,*)
> >> > ...
> >> > FCST_VAR2_LEVELS = ({valid?fmt=%Y%m%d_%H%M%S},*,*)
> >> >
> >> > And it will substitute the valid time for each run. Let me know
if you
> >> have
> >> > any questions.
> >> >
> >> > Thanks,
> >> > George
> >> >
> >> > On Wed, Apr 14, 2021 at 3:18 PM zhuming at ucar.edu via RT <
> >> met_help at ucar.edu
> >> > >
> >> > wrote:
> >> >
> >> > >
> >> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429
>
> >> > >
> >> > > Hi George,
> >> > >
> >> > > Thanks for your email, I just would like to send you for
help.
> >> > >
> >> > > I already resolved the last problem with the WRF forecast,
thank you
> >> for
> >> > > your help.
> >> > >
> >> > > I also need to verify the MPAS model forecast. These 12 hour
> >> > > MAPS forecasts are stored in a one file, such as
> >> > > 2015071500/mpas_wrf_cf_2015071500.nc, has 12h forecasts
> >> > > 2015071512/mpas_wrf_cf_2015071512.nc
> >> > >
> >> > > It seems MET is doing fine on the first 2 hours, but no
matched
> pairs
> >> for
> >> > > later hours.
> >> > > I'm not sure what's wrong with it? the time is not matched?
> >> > >
> >> > > see the log file
> >> > >
>
/glade/scratch/zhuming/metplus4/logs/master_metplus.log.20210414145101
> >> > > config file /gpfs/fs1/work/zhuming/METplus4/PointStat2.conf
> >> > >
> >> > > Thanks,
> >> > > Zhuming
> >> > >
> >> > > On Wed, Apr 14, 2021 at 1:56 PM George McCabe via RT <
> >> met_help at ucar.edu>
> >> > > wrote:
> >> > >
> >> > > > Hi Zhuming,
> >> > > >
> >> > > > Just following up on this ticket. Were you able to get this
> >> resolved?
> >> > Let
> >> > > > me know if you need more assistance.
> >> > > >
> >> > > > Thanks,
> >> > > > George
> >> > > >
> >> > > > On Thu Apr 08 10:04:44 2021, mccabe wrote:
> >> > > > > Hi Zhuming,
> >> > > > >
> >> > > > > Typically directories that contain data for an entire day
are
> >> named
> >> > > > > with
> >> > > > > YYYYMMDD, so your filename template tag would be
> >> {valid?fmt=%Y%m%d},
> >> > > > > however that may not always be the case, as you are
seeing here.
> >> > There
> >> > > > > are
> >> > > > > a couple ways to handle this.
> >> > > > >
> >> > > > > If you know that the directory will always be YYYYMMDD
with 00
> at
> >> the
> >> > > > > end,
> >> > > > > you can simply put 00 in the template instead of adding
%H in
> the
> >> > tag,
> >> > > > > i.e.
> >> > > > >
> >> > > > > FCST_POINT_STAT_INPUT_TEMPLATE =
> >> > > > > {valid?fmt=%Y%m%d}00/wrfout_d02_{valid?fmt=%Y-%m-
%d_%H}:00:
> >> 00_cf.nc
> >> > > > >
> >> > > > > If the HH value can vary, for instance if there are 2
> directories
> >> > > > > created
> >> > > > > per day (i.e. 2015071400 and 2015071412), then you can
use the
> >> > > > > truncate
> >> > > > > keyword to handle it.
> >> > > > >
> >> > > > > If directories are created every 24 hours:
> >> > > > >
> >> > > > > FCST_POINT_STAT_INPUT_TEMPLATE =
> >> > > > >
{valid?fmt=%Y%m%d%H?truncate=24H}/wrfout_d02_{valid?fmt=%Y-%m-
> >> > > > > %d_%H}:00:
> >> > > > > 00_cf.nc
> >> > > > >
> >> > > > > If directories are created every 12 hours:
> >> > > > >
> >> > > > > FCST_POINT_STAT_INPUT_TEMPLATE =
> >> > > > >
{valid?fmt=%Y%m%d%H?truncate=12H}/wrfout_d02_{valid?fmt=%Y-%m-
> >> > > > > %d_%H}:00:
> >> > > > > 00_cf.nc
> >> > > > >
> >> > > > > Let me know if that doesn't work for you.
> >> > > > >
> >> > > > > Thanks,
> >> > > > > George
> >> > > > >
> >> > > > > On Thu, Apr 8, 2021 at 9:46 AM zhuming at ucar.edu via RT
> >> > > > > <met_help at ucar.edu>
> >> > > > > wrote:
> >> > > > >
> >> > > > > >
> >> > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429
> >
> >> > > > > >
> >> > > > > > Hi George,
> >> > > > > >
> >> > > > > > Processing Obs data using "PB2NC" are right now.
> >> > > > > >
> >> > > > > > Now I'm using "PointStat" , I set VALID_BEG =
2015071500,
> >> > VALID_END =
> >> > > > > > 2015071501. It looks for the forecast data 2015071500/
> >> > > > > > wrfout_d02_2015-07-15_00:00:00_cf.nc, and 2015071501/
> >> > > > > > wrfout_d02_2015-07-15_01:00:00_cf.nc.
> >> > > > > > But my forecast data are all put in dir of 2015071500/.
> >> > > > > > So how could I set FCST_POINT_STAT_INPUT_TEMPLATE to
get the
> >> right
> >> > > > > > fcst
> >> > > > > > data?
> >> > > > > >
> >> > > > > > FCST_POINT_STAT_INPUT_TEMPLATE =
> >> > > > > > {valid?fmt=%Y%m%d%H}/wrfout_d02_{valid?fmt=%Y-%m-
%d_%H}:00:
> >> > 00_cf.nc
> >> > > > > >
> >> > > > > >
> >> > > > > > config file:
/gpfs/fs1/work/zhuming/METplus4/PointStat1.conf
> >> > > > > >
> >> > > > > > log file:
> >> > > > > >
> >> > >
>
/glade/scratch/zhuming/metplus4/logs/master_metplus.log.20210408085450
> >> > > > > >
> >> > > > > >
> >> > > > > > Thanks,
> >> > > > > >
> >> > > > > > Zhuming
> >> > > > > >
> >> > > > > > On Wed, Apr 7, 2021 at 5:43 PM George McCabe via RT
> >> > > > > > <met_help at ucar.edu>
> >> > > > > > wrote:
> >> > > > > >
> >> > > > > > > Hi Zhuming,
> >> > > > > > >
> >> > > > > > > Looking at the log output, it looks like the data is
stored
> in
> >> > the
> >> > > > > > prepBUFR
> >> > > > > > > files differently that you may have thought:
> >> > > > > > >
> >> > > > > > > DEBUG 2: PrepBufr Time Center:
20150715_000000
> >> > > > > > > DEBUG 2: Searching Time Window:
20150715_030000 to
> >> > > > > > 20150715_040000
> >> > > > > > > 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65%
70% 75%
> 80%
> >> > 85%
> >> > > > > > > 90%
> >> > > > > > 95%
> >> > > > > > > 100%
> >> > > > > > >
> >> > > > > > > DEBUG 2: Total Messages processed =
590836
> >> > > > > > > DEBUG 2: Rejected based on message type = 0
> >> > > > > > > DEBUG 2: Rejected based on station id = 0
> >> > > > > > > DEBUG 2: Rejected based on valid time =
590815
> >> > > > > > > DEBUG 2: Rejected based on masking grid = 20
> >> > > > > > > DEBUG 2: Rejected based on masking polygon = 0
> >> > > > > > > DEBUG 2: Rejected based on elevation = 0
> >> > > > > > > DEBUG 2: Rejected based on pb report type = 1
> >> > > > > > > DEBUG 2: Rejected based on input report type = 0
> >> > > > > > > DEBUG 2: Rejected based on instrument type = 0
> >> > > > > > > DEBUG 2: Rejected based on zero observations = 0
> >> > > > > > > DEBUG 2: Total Messages retained = 0
> >> > > > > > > DEBUG 2: Total observations retained or derived = 0
> >> > > > > > > DEBUG 1: Obs time between *20150714_210000 and
> >> 20150715_030000*
> >> > > > > > > WARNING:
> >> > > > > > > WARNING: process_pbfile() -> No PrepBufr messages
retained
> >> from
> >> > > > > > > file:
> >> > > > > > > /glade/collections/rda/data/ds337.0/prepnr/2015/
> >> > > > > > > prepbufr.gdas.2015071500.nr
> >> > > > > > > WARNING:
> >> > > > > > > ERROR :
> >> > > > > > > ERROR : write_netcdf_hdr_data() -> No PrepBufr
messages
> >> > retained.
> >> > > > > > Nothing
> >> > > > > > > to write.
> >> > > > > > > ERROR :
> >> > > > > > >
> >> > > > > > > The prepbufr.gdas.2015071500.nr file actually
contains data
> >> from
> >> > > > > > > 20150714_210000 to 20150715_030000, not
20150715_000000 to
> >> > > > > > 20150715_060000.
> >> > > > > > >
> >> > > > > > > To get the correct file for each run, you will have
to shift
> >> the
> >> > > > > > > valid
> >> > > > > > time
> >> > > > > > > before truncating it so that it gets the correct file
for
> each
> >> > run.
> >> > > > > > Instead
> >> > > > > > > of setting:
> >> > > > > > >
> >> > > > > > > PB2NC_INPUT_TEMPLATE =
> >> > > > > > >
> >> >
{valid?fmt=%Y}/prepbufr.gdas.{valid?fmt=%Y%m%d%H?truncate=21600}.nr
> >> > > > > > >
> >> > > > > > > You will have to also shift forward 3 hours, i.e.:
> >> > > > > > >
> >> > > > > > > PB2NC_INPUT_TEMPLATE =
> >> > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > >
> >> > >
> >> >
> >>
>
{valid?fmt=%Y}/prepbufr.gdas.{valid?fmt=%Y%m%d%H?shift=3H?truncate=21600}.nr
> >> > > > > > >
> >> > > > > > > This says to add 3 hours to the current valid time
before
> >> > > > > > > truncating to
> >> > > > > > the
> >> > > > > > > nearest 6 hours.
> >> > > > > > >
> >> > > > > > > Valid: 20150715 0Z + 3 = 3Z truncate 6H = 0Z
> >> > > > > > > File: prepbufr.gdas.2015071500.nr
> >> > > > > > >
> >> > > > > > > Valid: 20150715 1Z + 3 = 4Z truncate 6H = 0Z
> >> > > > > > > File: prepbufr.gdas.2015071500.nr
> >> > > > > > >
> >> > > > > > > Valid: 20150715 2Z + 3 = 5Z truncate 6H = 0Z
> >> > > > > > > File: prepbufr.gdas.2015071500.nr
> >> > > > > > >
> >> > > > > > > Valid: 20150715 3Z + 3 = 6Z truncate 6H = 6Z
> >> > > > > > > File: prepbufr.gdas.2015071506.nr
> >> > > > > > >
> >> > > > > > > Let me know if that doesn't work.
> >> > > > > > >
> >> > > > > > > Thanks,
> >> > > > > > > George
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > On Wed, Apr 7, 2021 at 5:32 PM zhuming at ucar.edu via
RT <
> >> > > > > > met_help at ucar.edu>
> >> > > > > > > wrote:
> >> > > > > > >
> >> > > > > > > >
> >> > > > > > > > <URL:
> >> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429
> >> > >
> >> > > > > > > >
> >> > > > > > > > Hi George,
> >> > > > > > > >
> >> > > > > > > > Now makes me confused is why processed obs of
several
> times
> >> are
> >> > > > > > missing.
> >> > > > > > > > There has files of time at 00,01,02, but no
03,04,05; has
> >> > > > > > > > 06,07,08, but
> >> > > > > > > no
> >> > > > > > > > 09,10,11, has 12,13,14, no 15,16,17; has 18,19,20,
but
> with
> >> > > > > > > > 21,no
> >> > > > > > 22,23.
> >> > > > > > > > From yours and mines:
> >> > > > > > > >
> >> > > > > > > > ls -l /glade/u/home/mccabe/data/out/pb2nc1/
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > > > -rw-r--r-- 1 mccabe ral 10141339 Apr 6 17:52
> >> > pbs.2015071500.nc
> >> > > > > > > >
> >> > > > > > > > -rw-r--r-- 1 mccabe ral 5413907 Apr 6 17:56
> >> > pbs.2015071501.nc
> >> > > > > > > >
> >> > > > > > > > -rw-r--r-- 1 mccabe ral 4923711 Apr 6 18:00
> >> > pbs.2015071502.nc
> >> > > > > > > >
> >> > > > > > > > -rw-r--r-- 1 mccabe ral 5724447 Apr 6 18:14
> >> > pbs.2015071506.nc
> >> > > > > > > >
> >> > > > > > > > -rw-r--r-- 1 mccabe ral 5280463 Apr 6 18:18
> >> > pbs.2015071507.nc
> >> > > > > > > >
> >> > > > > > > > -rw-r--r-- 1 mccabe ral 4706119 Apr 6 18:22
> >> > pbs.2015071508.nc
> >> > > > > > > >
> >> > > > > > > > -rw-r--r-- 1 mccabe ral 9878263 Apr 6 18:36
> >> > pbs.2015071512.nc
> >> > > > > > > >
> >> > > > > > > > -rw-r--r-- 1 mccabe ral 5392527 Apr 6 18:40
> >> > pbs.2015071513.nc
> >> > > > > > > >
> >> > > > > > > > -rw-r--r-- 1 mccabe ral 96 Apr 6 18:40
> >> > pbs.2015071514.nc
> >> > > > > > > >
> >> > > > > > > > -rw-r--r-- 1 mccabe ral 96 Apr 6 18:42
> >> > pbs.2015071515.nc
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > > > cheyenne3:/glade/scratch/zhuming/metplus4/pb2nc1>l
> >> > > > > > > >
> >> > > > > > > > total 96112
> >> > > > > > > >
> >> > > > > > > > -rw-r--r-- 1 zhuming ncar 11550172 Apr 7 15:11
> >> > > pbs.2015071500.nc
> >> > > > > > > >
> >> > > > > > > > -rw-r--r-- 1 zhuming ncar 6615956 Apr 7 15:14
> >> > > pbs.2015071501.nc
> >> > > > > > > >
> >> > > > > > > > -rw-r--r-- 1 zhuming ncar 5984328 Apr 7 15:18
> >> > > pbs.2015071502.nc
> >> > > > > > > >
> >> > > > > > > > -rw-r--r-- 1 zhuming ncar 6980712 Apr 7 15:33
> >> > > pbs.2015071506.nc
> >> > > > > > > >
> >> > > > > > > > -rw-r--r-- 1 zhuming ncar 6455824 Apr 7 15:36
> >> > > pbs.2015071507.nc
> >> > > > > > > >
> >> > > > > > > > -rw-r--r-- 1 zhuming ncar 5749432 Apr 7 15:40
> >> > > pbs.2015071508.nc
> >> > > > > > > >
> >> > > > > > > > -rw-r--r-- 1 zhuming ncar 11284936 Apr 7 15:55
> >> > > pbs.2015071512.nc
> >> > > > > > > >
> >> > > > > > > > -rw-r--r-- 1 zhuming ncar 6600888 Apr 7 15:59
> >> > > pbs.2015071513.nc
> >> > > > > > > >
> >> > > > > > > > -rw-r--r-- 1 zhuming ncar 5868168 Apr 7 16:03
> >> > > pbs.2015071514.nc
> >> > > > > > > >
> >> > > > > > > > -rw-r--r-- 1 zhuming ncar 7273092 Apr 7 16:18
> >> > > pbs.2015071518.nc
> >> > > > > > > >
> >> > > > > > > > -rw-r--r-- 1 zhuming ncar 6656344 Apr 7 16:22
> >> > > pbs.2015071519.nc
> >> > > > > > > >
> >> > > > > > > > -rw-r--r-- 1 zhuming ncar 6109788 Apr 7 16:27
> >> > > pbs.2015071520.nc
> >> > > > > > > >
> >> > > > > > > > -rw-r--r-- 1 zhuming ncar 58808 Apr 7 16:31
> >> > > pbs.2015071521.nc
> >> > > > > > > >
> >> > > > > > > > -rw-r--r-- 1 zhuming ncar 11132440 Apr 7 16:43
> >> > > pbs.2015071600.nc
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > > > Thanks,
> >> > > > > > > >
> >> > > > > > > > Zhuming
> >> > > > > > > >
> >> > > > > > > > On Tue, Apr 6, 2021 at 6:34 PM George McCabe via RT
> >> > > > > > > > <met_help at ucar.edu
> >> > > > > > >
> >> > > > > > > > wrote:
> >> > > > > > > >
> >> > > > > > > > > Hi Zhuming,
> >> > > > > > > > >
> >> > > > > > > > > Thanks for the information. You will need to use
the
> >> truncate
> >> > > > > > > > > keyword
> >> > > > > > > to
> >> > > > > > > > > find the files, then set the valid beg/end range
to 1
> >> hour.
> >> > > > > > > > > This is
> >> > > > > > > what
> >> > > > > > > > I
> >> > > > > > > > > changed from your PB2NC1.conf config file:
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > > *PB2NC_VALID_BEGIN =
> {valid?fmt=%Y%m%d_%H}PB2NC_VALID_END
> >> =
> >> > > > > > > > > {valid?fmt=%Y%m%d_%H?shift=1H}*
> >> > > > > > > > >
> >> > > > > > > > > *PB2NC_INPUT_TEMPLATE =
> >> > > > > > > > >
> >> > > >
> {valid?fmt=%Y}/prepbufr.gdas.{valid?fmt=%Y%m%d%H?truncate=21600}.nr*
> >> > > > > > > > >
> >> > > > > > > > > My config file is on cheyenne here:
> >> > > > > > > > > /glade/u/home/mccabe/PB2NC1.conf
> >> > > > > > > > >
> >> > > > > > > > > I just started running it and the output is being
> written
> >> > here:
> >> > > > > > > > > /glade/u/home/mccabe/data/out/pb2nc1
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > > For PointStat, you will want to loop over the
same
> times.
> >> I
> >> > > > > > > > > noticed
> >> > > > > > you
> >> > > > > > > > > specify message type in the VAR<n>_OPTIONS, but
you do
> not
> >> > need
> >> > > > > > > > > to do
> >> > > > > > > > this
> >> > > > > > > > > since there is a POINT_STAT_MESSAGE_TYPE variable
that
> >> > defines
> >> > > > > > > > > this.
> >> > > > > > > This
> >> > > > > > > > > is what I changed in your PointStat1.conf file:
> >> > > > > > > > >
> >> > > > > > > > > *VALID_END = 2015071600*
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > > *VALID_INCREMENT = 1HPOINT_STAT_OBS_VALID_BEG
> >> > > > > > =POINT_STAT_OBS_VALID_BEG
> >> > > > > > > > =*
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > > *POINT_STAT_MESSAGE_TYPE = ADPSFC*
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > > *#OBS_VAR1_OPTIONS = message_type =
> >> > > > > > > > > ["ADPSFC"];#OBS_VAR2_OPTIONS =
> >> > > > > > > > > message_type = ["ADPSFC"];#OBS_VAR3_OPTIONS =
> >> message_type =
> >> > > > > > > > > ["ADPSFC"];#OBS_POINT_STAT_INPUT_DIR =
> >> > > > > > > > > /glade/scratch/zhuming/metplus4/pb2nc1*
> >> > > > > > > > >
> >> > > > > > > > > *OBS_POINT_STAT_INPUT_DIR = {OUTPUT_BASE}/pb2nc1*
> >> > > > > > > > >
> >> > > > > > > > > Or you can view my file in the same directory. I
usually
> >> try
> >> > to
> >> > > > > > > > > test
> >> > > > > > > > with a
> >> > > > > > > > > single run time to make sure it is working
properly, so
> I
> >> set
> >> > > > > > VALID_END
> >> > > > > > > > > back to 2015071500. I ran it and it produced an
output
> >> file,
> >> > > > > > > > > but
> >> > > > > > there
> >> > > > > > > > were
> >> > > > > > > > > no values besides the header. I checked the log
file and
> >> saw
> >> > > > > > > > > this:
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > > *DEBUG 2: Processing T_sfc(0,*,*) versus TMP/Z2,
for
> >> > > > > > > > > observation type
> >> > > > > > > > > ADPSFC, over region FULL, for interpolation
method
> >> BILIN(4),
> >> > > > > > > > > using 0
> >> > > > > > > > > matched pairs.DEBUG 3: Number of matched pairs
=
> 0DEBUG
> >> 3:
> >> > > > > > > Observations
> >> > > > > > > > > processed = 393373DEBUG 3: Rejected: station
id
> =
> >> > > > > > > > > 0DEBUG 3:
> >> > > > > > > > > Rejected: obs type = 390528DEBUG 3:
Rejected:
> valid
> >> > time
> >> > > > > > =
> >> > > > > > > > > 0DEBUG 3: Rejected: bad obs value = 0DEBUG 3:
> Rejected:
> >> off
> >> > > > > > > > > the
> >> > > > > > grid
> >> > > > > > > > =
> >> > > > > > > > > 2586DEBUG 3: Rejected: topography = 0DEBUG
3:
> >> Rejected:
> >> > > > > > > > > level
> >> > > > > > > > mismatch
> >> > > > > > > > > = 259DEBUG 3: Rejected: quality marker = 0DEBUG
3:
> >> Rejected:
> >> > > > > > message
> >> > > > > > > > type
> >> > > > > > > > > = 0DEBUG 3: Rejected: masking region = 0DEBUG 3:
> >> Rejected:
> >> > bad
> >> > > > > > fcst
> >> > > > > > > > > value = 0DEBUG 3: Rejected: bad climo mean =
0DEBUG 3:
> >> > > > > > > > > Rejected:
> >> > > > > > bad
> >> > > > > > > > > climo stdev = 0DEBUG 3: Rejected: duplicates
=
> 0DEBUG
> >> > > > > > > > > 2:DEBUG 2:
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > >
> >> > >
> >> >
> >>
>
--------------------------------------------------------------------------------DEBUG
> >> > > > > > > > > 2:DEBUG 2: Processing u_10m_gr(0,*,*) versus
UGRD/Z10,
> for
> >> > > > > > observation
> >> > > > > > > > type
> >> > > > > > > > > ADPSFC, over region FULL, for interpolation
method
> >> BILIN(4),
> >> > > > > > > > > using 0
> >> > > > > > > > > matched pairs.DEBUG 3: Number of matched pairs
=
> 0DEBUG
> >> 3:
> >> > > > > > > Observations
> >> > > > > > > > > processed = 393373*
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > > *DEBUG 3: Rejected: station id = 0DEBUG 3:
> Rejected:
> >> obs
> >> > > > > > > > > type
> >> > > > > > > > =
> >> > > > > > > > > 382965DEBUG 3: Rejected: valid time = 0DEBUG
3:
> >> > Rejected:
> >> > > > > > > > > bad
> >> > > > > > obs
> >> > > > > > > > > value = 0DEBUG 3: Rejected: off the grid =
> 9240DEBUG
> >> 3:
> >> > > > > > Rejected:
> >> > > > > > > > > topography = 0DEBUG 3: Rejected: level
mismatch =
> >> > > > > > > > > 1168DEBUG 3:
> >> > > > > > > > > Rejected: quality marker = 0DEBUG 3: Rejected:
message
> >> type
> >> > > > > > > > > =
> >> > > > > > > 0DEBUG
> >> > > > > > > > 3:
> >> > > > > > > > > Rejected: masking region = 0DEBUG 3: Rejected:
bad fcst
> >> > value
> >> > > > > > > > > =
> >> > > > > > > 0DEBUG
> >> > > > > > > > 3:
> >> > > > > > > > > Rejected: bad climo mean = 0DEBUG 3: Rejected:
bad
> climo
> >> > stdev
> >> > > > > > > > > =
> >> > > > > > > 0DEBUG
> >> > > > > > > > 3:
> >> > > > > > > > > Rejected: duplicates = 0DEBUG 2:DEBUG 2:
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > >
> >> > >
> >> >
> >>
>
--------------------------------------------------------------------------------DEBUG
> >> > > > > > > > > 2:DEBUG 2: Processing v_10m_gr(0,*,*) versus
VGRD/Z10,
> for
> >> > > > > > observation
> >> > > > > > > > type
> >> > > > > > > > > ADPSFC, over region FULL, for interpolation
method
> >> BILIN(4),
> >> > > > > > > > > using 0
> >> > > > > > > > > matched pairs.DEBUG 3: Number of matched pairs
=
> 0DEBUG
> >> 3:
> >> > > > > > > Observations
> >> > > > > > > > > processed = 393373DEBUG 3: Rejected: station
id
> =
> >> > > > > > > > > 0DEBUG 3:
> >> > > > > > > > > Rejected: obs type = 382965DEBUG 3:
Rejected:
> valid
> >> > time
> >> > > > > > =
> >> > > > > > > > > 0DEBUG 3: Rejected: bad obs value = 0DEBUG 3:
> Rejected:
> >> off
> >> > > > > > > > > the
> >> > > > > > grid
> >> > > > > > > > =
> >> > > > > > > > > 9240DEBUG 3: Rejected: topography = 0DEBUG
3:
> >> Rejected:
> >> > > > > > > > > level
> >> > > > > > > > mismatch
> >> > > > > > > > > = 1168DEBUG 3: Rejected: quality marker = 0DEBUG
3:
> >> > Rejected:
> >> > > > > > message
> >> > > > > > > > > type = 0DEBUG 3: Rejected: masking region =
0DEBUG
> 3:
> >> > > > > > > > > Rejected:
> >> > > > > > bad
> >> > > > > > > > > fcst value = 0DEBUG 3: Rejected: bad climo mean
=
> >> 0DEBUG 3:
> >> > > > > > Rejected:
> >> > > > > > > > bad
> >> > > > > > > > > climo stdev = 0*
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > > *DEBUG 3: Rejected: duplicates = 0DEBUG
2:DEBUG 2:
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > >
> >> > >
> >> >
> >>
>
--------------------------------------------------------------------------------DEBUG
> >> > > > > > > > > 2:DEBUG 1: Output file:
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > >
> >> > >
> >> >
> >>
>
/glade/u/home/mccabe/data/out/point_stat1/point_stat_000000L_20150715_000000V.stat*
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > > This describes the reason why each observation
was
> >> rejected.
> >> > > > > > > > > Some
> >> > > > > > > > > observations didn't match the message type you
> requested,
> >> > some
> >> > > > > > > > > were
> >> > > > > > not
> >> > > > > > > > > within the forecast grid, and some did not match
the
> >> vertical
> >> > > > > > > > > level.
> >> > > > > > > You
> >> > > > > > > > > will have to make adjustments to your settings so
that
> the
> >> > data
> >> > > > > > matches
> >> > > > > > > > up
> >> > > > > > > > > properly. Maybe the grid you are using in the
PB2NC
> config
> >> > file
> >> > > > > > > > > does
> >> > > > > > > not
> >> > > > > > > > > match the forecast grid and you are missing
values. If
> >> that
> >> > is
> >> > > > > > > > > the
> >> > > > > > > case,
> >> > > > > > > > > you will have to reconfigure and rerun PB2NC to
get
> >> output on
> >> > > > > > > > > the
> >> > > > > > > correct
> >> > > > > > > > > gr5id. Maybe the observation points don't line up
with
> the
> >> > > > > > > > > vertical
> >> > > > > > > level
> >> > > > > > > > > of the model fields. You can use a range of level
values
> >> for
> >> > > > > > > > > the
> >> > > > > > > > > observation data to allow data within that range,
i.e.
> >> Z0-10.
> >> > > > > > > > >
> >> > > > > > > > > Let me know if you are still having issues
getting this
> to
> >> > work
> >> > > > > > > > > and I
> >> > > > > > > can
> >> > > > > > > > > help out.
> >> > > > > > > > >
> >> > > > > > > > > Thanks,
> >> > > > > > > > > George
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > > On Tue, Apr 6, 2021 at 5:17 PM zhuming at ucar.edu
via RT
> <
> >> > > > > > > > met_help at ucar.edu>
> >> > > > > > > > > wrote:
> >> > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > <URL:
> >> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429
> >> > > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > > Hi George,
> >> > > > > > > > > >
> >> > > > > > > > > > Thanks for your reply!
> >> > > > > > > > > >
> >> > > > > > > > > > I'm still confused, I need to output hourly obs
data
> to
> >> do
> >> > > > > > > > > > hourly
> >> > > > > > > > > > verification.
> >> > > > > > > > > > The original GDAS Prepbufr data has 4 files one
day,
> >> should
> >> > > > > > > > > > with 6
> >> > > > > > > > hours
> >> > > > > > > > > > obs data in one file.
> >> > > > > > > > > >
> >> > > > > > > > > > After "PB2NC", I can see pbs.2015071500.nc (obs
time
> is
> >> > 00?
> >> > > > > > > > > > or 6
> >> > > > > > > times
> >> > > > > > > > > of
> >> > > > > > > > > > 00,01,02,03,04,05?), pbs.2015071506.nc...
> >> > > > > > > > > >
> >> > > > > > > > > > my config file :
> >> > > > > > cheyenne:/gpfs/fs1/work/zhuming/METplus4/PB2NC1.conf
> >> > > > > > > > > > and
/gpfs/fs1/work/zhuming/METplus4/PointStat1.conf
> >> > > > > > > > > >
> >> > > > > > > > > > Thanks,
> >> > > > > > > > > > Zhuming
> >> > > > > > > > > >
> >> > > > > > > > > > On Tue, Apr 6, 2021 at 4:23 PM George McCabe
via RT <
> >> > > > > > > met_help at ucar.edu
> >> > > > > > > > >
> >> > > > > > > > > > wrote:
> >> > > > > > > > > >
> >> > > > > > > > > > > Hi Zhuming,
> >> > > > > > > > > > >
> >> > > > > > > > > > > To run the tools hourly you would set the
> >> VALID_INCREMENT
> >> > > > > > > > > > > value
> >> > > > > > for
> >> > > > > > > > 1H.
> >> > > > > > > > > > > Here is an example that runs once per hour
for a day
> >> from
> >> > > > > > > 2015-07-15
> >> > > > > > > > 0Z
> >> > > > > > > > > > to
> >> > > > > > > > > > > 2017-07-16 0Z:
> >> > > > > > > > > > >
> >> > > > > > > > > > > VALID_TIME_FMT = %Y%m%d%H
> >> > > > > > > > > > >
> >> > > > > > > > > > > VALID_BEG = 2015071500
> >> > > > > > > > > > >
> >> > > > > > > > > > > VALID_END = 2015071600
> >> > > > > > > > > > >
> >> > > > > > > > > > > VALID_INCREMENT = 1H
> >> > > > > > > > > > >
> >> > > > > > > > > > > LEAD_SEQ = 0
> >> > > > > > > > > > >
> >> > > > > > > > > > > Each run time will use the
PB2NC_INPUT_TEMPLATE to
> >> find
> >> > > > > > > > > > > files
> >> > > > > > whose
> >> > > > > > > > > > > filenames match the time you need.
> >> > > > > > > > > > >
> >> > > > > > > > > > > To read prepBUFR observations for 12 hours,
you will
> >> want
> >> > > > > > > > > > > to set
> >> > > > > > > > > > > PB2NC_VALID_BEGIN and PB2NC_VALID_END to
specify the
> >> > range
> >> > > > > > > > > > > around
> >> > > > > > > the
> >> > > > > > > > > > valid
> >> > > > > > > > > > > time to process. These values depend on what
is
> >> contained
> >> > > > > > > > > > > inside
> >> > > > > > > your
> >> > > > > > > > > > > prepBUFR files. If you want to use data from
the
> valid
> >> > time
> >> > > > > > > > > > > of
> >> > > > > > the
> >> > > > > > > > > > filename
> >> > > > > > > > > > > backwards 12 hours, you would set:
> >> > > > > > > > > > >
> >> > > > > > > > > > > PB2NC_VALID_BEGIN =
{valid?fmt=%Y%m%d_%H?shift=-12H
> }
> >> > > > > > > > > > > PB2NC_VALID_END = {valid?fmt=%Y%m%d_%H}
> >> > > > > > > > > > >
> >> > > > > > > > > > > For a runtime of 2015071512, this would be
> substituted
> >> > as:
> >> > > > > > > > > > >
> >> > > > > > > > > > > valid_beg: 20150715_00
> >> > > > > > > > > > > valid_end: 20150715_12
> >> > > > > > > > > > >
> >> > > > > > > > > > > If you want to create an output file for each
hour
> in
> >> > PB2NC
> >> > > > > > > > > > > you
> >> > > > > > > would
> >> > > > > > > > > > need
> >> > > > > > > > > > > to make sure that the hour is specified in
the
> >> > > > > > > PB2NC_OUTPUT_TEMPLATE.
> >> > > > > > > > > > >
> >> > > > > > > > > > > PointStat has similar config variables to set
the
> >> valid
> >> > beg
> >> > > > > > > > > > > and
> >> > > > > > end
> >> > > > > > > > > range
> >> > > > > > > > > > > to use (POINT_STAT_OBS_VALID_BEG and
> >> > > > > > > > > > > POINT_STAT_OBS_VALID_END).
> >> > > > > > > > > > >
> >> > > > > > > > > > > If your prepBUFR files are separated out into
> multiple
> >> > > > > > > > > > > files that
> >> > > > > > > > need
> >> > > > > > > > > to
> >> > > > > > > > > > > be read into PB2NC, you can set a file window
around
> >> the
> >> > > > > > > > > > > valid
> >> > > > > > time
> >> > > > > > > > to
> >> > > > > > > > > > > allow files to be passed into the app. More
> >> information
> >> > on
> >> > > > > > > > > > > that
> >> > > > > > > > > > > functionality is here:
> >> > > > > > > > > > >
> >> > > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > >
> >> > >
> >> >
> >>
>
https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#using-
> >> > > > > > windows-to-find-valid-files
> >> > > > > > > > > > >
> >> > > > > > > > > > > If you are still having trouble getting this
> >> configured,
> >> > > > > > > > > > > you
> >> > > > > > could
> >> > > > > > > > send
> >> > > > > > > > > > me
> >> > > > > > > > > > > the paths of the files you are trying to read
and
> what
> >> > data
> >> > > > > > > > > > > is
> >> > > > > > > > > contained
> >> > > > > > > > > > in
> >> > > > > > > > > > > them so I can give you suggestions on what to
set
> for
> >> the
> >> > > > > > > > > configuration.
> >> > > > > > > > > > >
> >> > > > > > > > > > > Thanks,
> >> > > > > > > > > > > George
> >> > > > > > > > > > >
> >> > > > > > > > > > >
> >> > > > > > > > > > >
> >> > > > > > > > > > > On Tue, Apr 6, 2021 at 2:17 PM Julie
Prestopnik via
> >> RT <
> >> > > > > > > > > > met_help at ucar.edu>
> >> > > > > > > > > > > wrote:
> >> > > > > > > > > > >
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > Tue Apr 06 14:16:42 2021: Request 99429 was
acted
> >> upon.
> >> > > > > > > > > > > > Transaction: Given to mccabe (George
McCabe) by
> >> jpresto
> >> > > > > > > > > > > > Queue: met_help
> >> > > > > > > > > > > > Subject: How to set time for hourly
12-24
> hours
> >> > data
> >> > > > > > > > > > > > ?
> >> > > > > > > > > > > > Owner: mccabe
> >> > > > > > > > > > > > Requestors: zhuming at ucar.edu
> >> > > > > > > > > > > > Status: new
> >> > > > > > > > > > > > Ticket <URL:
> >> > > > > > > > >
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429
> >> > > > > > > > > > >
> >> > > > > > > > > > > >
> >> > > > > > > > > > > >
> >> > > > > > > > > > > > This transaction appears to have no content
> >> > > > > > > > > > > >
> >> > > > > > > > > > >
> >> > > > > > > > > > >
> >> > > > > > > > > > > --
> >> > > > > > > > > > > George McCabe - Software Engineer III
> >> > > > > > > > > > > National Center for Atmospheric Research
> >> > > > > > > > > > > Research Applications Laboratory
> >> > > > > > > > > > > 303-497-2768
> >> > > > > > > > > > > ---
> >> > > > > > > > > > > My working day may not be your working day.
Please
> do
> >> not
> >> > > > > > > > > > > feel
> >> > > > > > > > obliged
> >> > > > > > > > > to
> >> > > > > > > > > > > reply to this email outside of your normal
working
> >> hours.
> >> > > > > > > > > > >
> >> > > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > > > --
> >> > > > > > > > > George McCabe - Software Engineer III
> >> > > > > > > > > National Center for Atmospheric Research
> >> > > > > > > > > Research Applications Laboratory
> >> > > > > > > > > 303-497-2768
> >> > > > > > > > > ---
> >> > > > > > > > > My working day may not be your working day.
Please do
> not
> >> > feel
> >> > > > > > obliged
> >> > > > > > > to
> >> > > > > > > > > reply to this email outside of your normal
working
> hours.
> >> > > > > > > > >
> >> > > > > > > > >
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > > > --
> >> > > > > > > George McCabe - Software Engineer III
> >> > > > > > > National Center for Atmospheric Research
> >> > > > > > > Research Applications Laboratory
> >> > > > > > > 303-497-2768
> >> > > > > > > ---
> >> > > > > > > My working day may not be your working day. Please do
not
> feel
> >> > > > > > > obliged to
> >> > > > > > > reply to this email outside of your normal working
hours.
> >> > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > >
> >> > >
> >> >
> >> > --
> >> > George McCabe - Software Engineer III
> >> > National Center for Atmospheric Research
> >> > Research Applications Laboratory
> >> > 303-497-2768
> >> > ---
> >> > My working day may not be your working day. Please do not feel
obliged
> >> to
> >> > reply to this email outside of your normal working hours.
> >> >
> >> >
> >>
> >>
> >
> > --
> > George McCabe - Software Engineer III
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > 303-497-2768
> > ---
> > My working day may not be your working day. Please do not feel
obliged to
> > reply to this email outside of your normal working hours.
> >
>
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>
------------------------------------------------
Subject: How to set time for hourly 12-24 hours data ?
From: John Halley Gotway
Time: Thu Apr 22 11:40:23 2021
Zhuming,
This is John HG. George asked me to take a look at this question.
Since
you're asking about "non-standard" levels, I assume you're asking
about the
verification of the pressure levels against the ADPUPA message type.
In general, the logic is this...
For each point observation, loop through all the verification tasks
and see
if it should be used.
For example, let's talk about the 4 level entry for VAR3:
FCST_VAR3_NAME = v_tr_p
FCST_VAR3_LEVELS = ...(0,3,*,*)...
OBS_VAR3_NAME = VGRD
OBS_VAR3_LEVELS = ...P575-625...
The point observation will be used here if it's a "VGRD" observation
and
has a level value between 575 and 625. This verification task has been
configured to only verify a single forecast level of data:
v_tr_p(0,3,*,*).
So no vertical interpolation is done. MET interpolates horizontally
from
the forecast grid points to the observation lat/lon location (using
user-configurable interpolation options).
To put it simply, all obs between 575 and 625 mb are compared directly
to
the forecast values from v_tr_p(0,3,*,*). If the model output is
actually
on pressure levels, then that's a reasonable thing to do. If the
model's
vertical coordinate is something other than pressure levels, then
that's a
potential source for large errors.
Note that MET does do vertical interpolation when processing the GRIB
output from the Unified Post Processor tool. But all the details of
that
depend on how the configuration file is setup. I'll explain below.
To date we have avoided including direct knowledge about the vertical
coordinate of each modelling system in the verification code. The goal
is
to create verification tools applicable to many models, rather than
customizing them for each. We do realize that desire for the latter
and are
exploring ways to pull data from the IODA data assimilation system
instead,
which necessarily knows a lot about each model.
So let's say you were using GRIB data, and configured it like this:
VAR3_NAME = VGRD
VAR3_LEVELS = P500-1000
Since we haven't prefixed these options with FCST_ and OBS_, the same
settings are applied to both.
MET would read ALL forecast GRIB records from the model file with
pressures
between 500 and 1000. Perhaps, that's every 100mb or perhaps every
50mb? It
reads whatever data falls in that range. For each point observation
with a
pressure level between 500 and 1000, it'll first find the actual model
pressure level above/below it. It'll interpolate the forecast
data horizontally to the obs lat/lon location for both. And then it'll
interpolate those values above/below vertically to the actual
observation
level, linear in the log of pressure.
This issue is that the processing of pressure LAYERS is not currently
supported for NetCDF data I believe. And the code could be enhanced to
support that. But again, this only makes sense if the model's vertical
coordinate is actually pressure levels and not something else.
Hope that helps clarify.
Thanks,
John
On Thu, Apr 22, 2021 at 9:52 AM zhuming at ucar.edu via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429 >
>
> Hi George,
>
> My verification result shows that there is a big error on a non-
standard
> level, so my question is:
> Is the forecast grid match to Obs, or obs station match to Forecast?
I need
> to do is the forecast grid interpolated to the obs location and
vertical
> levels. So how to set the levels?
>
> Thanks,
> Zhuming
>
> FCST_VAR3_NAME = v_10m_tr
>
> FCST_VAR3_LEVELS = (0,*,*)
>
> FCST_VAR3_THRESH = >=5, <5
>
> OBS_VAR3_NAME = VGRD
>
> OBS_VAR3_LEVELS = Z10
>
> OBS_VAR3_THRESH = >=5, <5
>
> OBS_VAR3_OPTIONS = message_type = ["ADPSFC"];
>
>
> FCST_VAR3_NAME = v_tr_p
>
> FCST_VAR3_LEVELS =
>
>
(0,0,*,*),(0,1,*,*),(0,2,*,*),(0,3,*,*),(0,4,*,*),(0,5,*,*),(0,6,*,*),(0,7,*,*),(0,8,*,*),(0,9,*,*)
>
> FCST_VAR3_THRESH = >=5, <5
>
> OBS_VAR3_NAME = VGRD
>
> OBS_VAR3_LEVELS =
>
> P900-950,P825-875,P675-725,P575-625,P475-525,P375-425,P275-325,P225-
275,P175-225,P75-125
>
> OBS_VAR3_THRESH = >=5, <5
>
> OBS_VAR3_OPTIONS = message_type = ["ADPUPA"];
>
>
>
> On Thu, Apr 15, 2021 at 1:52 PM George McCabe via RT
<met_help at ucar.edu>
> wrote:
>
> > Hi Zhuming,
> >
> > There was indeed a bug in the code. Howard has fixed the issue
with this
> > pull request: https://github.com/dtcenter/MET/pull/1756
> > This fix will be included in the MET 10.0.0-beta5 release that is
> > tentatively scheduled for next week. This beta version can be
installed
> on
> > cheyenne so you can test that the update fixes your issue. I will
follow
> up
> > and let you know when the beta release is available on cheyenne.
> >
> > Thanks,
> > George
> >
> > On Wed, Apr 14, 2021 at 5:28 PM George McCabe <mccabe at ucar.edu>
wrote:
> >
> > > Hi Zhuming,
> > >
> > > I tried to get this to work on cheyenne with your data, but was
not
> > > successful. I am able to read the data with plot_data_plane and
> generate
> > an
> > > image, so I think there may be a bug in the code. I have
forwarded info
> > to
> > > one of our C++ developers to look into it. I will report back
when I
> know
> > > more.
> > >
> > > Thanks,
> > > George
> > >
> > > On Wed, Apr 14, 2021 at 4:13 PM zhuming at ucar.edu via RT <
> > met_help at ucar.edu>
> > > wrote:
> > >
> > >>
> > >> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429 >
> > >>
> > >> Hi George,
> > >>
> > >> It still can't find the correct time. I tried
> > >>
> > >> FCST_VAR1_LEVELS = (*,*,*)
> > >> FCST_VAR1_LEVELS = ({valid?fmt=%Y%m%d_%H%M%S},*,*)
> > >> FCST_VAR1_LEVELS = ({valid?fmt=%Y%m%d_%H%M},*,*)
> > >> FCST_VAR1_LEVELS = ({valid?fmt=%Y%m%d%H%M},*,*)
> > >>
> > >> Thanks!
> > >>
> > >> Zhuming
> > >>
> > >> On Wed, Apr 14, 2021 at 3:41 PM George McCabe via RT <
> met_help at ucar.edu
> > >
> > >> wrote:
> > >>
> > >> > Hi Zhuming,
> > >> >
> > >> > It looks like you are requesting the first time in the 12
hour file
> > for
> > >> > each run:
> > >> >
> > >> > FCST_VAR1_LEVELS = (0,*,*)
> > >> > ...
> > >> > FCST_VAR2_LEVELS = (0,*,*)
> > >> >
> > >> > The first dimension in these files is the time offset. You
will need
> > the
> > >> > correct index that corresponds to each valid time.
Fortunately, MET
> > >> makes
> > >> > this easy by allowing you to specify the valid time for a
time
> > dimension
> > >> > explicitly and it will find the correct data. You can set:
> > >> >
> > >> > FCST_VAR1_LEVELS = ({valid?fmt=%Y%m%d_%H%M%S},*,*)
> > >> > ...
> > >> > FCST_VAR2_LEVELS = ({valid?fmt=%Y%m%d_%H%M%S},*,*)
> > >> >
> > >> > And it will substitute the valid time for each run. Let me
know if
> you
> > >> have
> > >> > any questions.
> > >> >
> > >> > Thanks,
> > >> > George
> > >> >
> > >> > On Wed, Apr 14, 2021 at 3:18 PM zhuming at ucar.edu via RT <
> > >> met_help at ucar.edu
> > >> > >
> > >> > wrote:
> > >> >
> > >> > >
> > >> > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429 >
> > >> > >
> > >> > > Hi George,
> > >> > >
> > >> > > Thanks for your email, I just would like to send you for
help.
> > >> > >
> > >> > > I already resolved the last problem with the WRF forecast,
thank
> you
> > >> for
> > >> > > your help.
> > >> > >
> > >> > > I also need to verify the MPAS model forecast. These 12
hour
> > >> > > MAPS forecasts are stored in a one file, such as
> > >> > > 2015071500/mpas_wrf_cf_2015071500.nc, has 12h forecasts
> > >> > > 2015071512/mpas_wrf_cf_2015071512.nc
> > >> > >
> > >> > > It seems MET is doing fine on the first 2 hours, but no
matched
> > pairs
> > >> for
> > >> > > later hours.
> > >> > > I'm not sure what's wrong with it? the time is not matched?
> > >> > >
> > >> > > see the log file
> > >> > >
> >
/glade/scratch/zhuming/metplus4/logs/master_metplus.log.20210414145101
> > >> > > config file /gpfs/fs1/work/zhuming/METplus4/PointStat2.conf
> > >> > >
> > >> > > Thanks,
> > >> > > Zhuming
> > >> > >
> > >> > > On Wed, Apr 14, 2021 at 1:56 PM George McCabe via RT <
> > >> met_help at ucar.edu>
> > >> > > wrote:
> > >> > >
> > >> > > > Hi Zhuming,
> > >> > > >
> > >> > > > Just following up on this ticket. Were you able to get
this
> > >> resolved?
> > >> > Let
> > >> > > > me know if you need more assistance.
> > >> > > >
> > >> > > > Thanks,
> > >> > > > George
> > >> > > >
> > >> > > > On Thu Apr 08 10:04:44 2021, mccabe wrote:
> > >> > > > > Hi Zhuming,
> > >> > > > >
> > >> > > > > Typically directories that contain data for an entire
day are
> > >> named
> > >> > > > > with
> > >> > > > > YYYYMMDD, so your filename template tag would be
> > >> {valid?fmt=%Y%m%d},
> > >> > > > > however that may not always be the case, as you are
seeing
> here.
> > >> > There
> > >> > > > > are
> > >> > > > > a couple ways to handle this.
> > >> > > > >
> > >> > > > > If you know that the directory will always be YYYYMMDD
with 00
> > at
> > >> the
> > >> > > > > end,
> > >> > > > > you can simply put 00 in the template instead of adding
%H in
> > the
> > >> > tag,
> > >> > > > > i.e.
> > >> > > > >
> > >> > > > > FCST_POINT_STAT_INPUT_TEMPLATE =
> > >> > > > > {valid?fmt=%Y%m%d}00/wrfout_d02_{valid?fmt=%Y-%m-
%d_%H}:00:
> > >> 00_cf.nc
> > >> > > > >
> > >> > > > > If the HH value can vary, for instance if there are 2
> > directories
> > >> > > > > created
> > >> > > > > per day (i.e. 2015071400 and 2015071412), then you can
use
> the
> > >> > > > > truncate
> > >> > > > > keyword to handle it.
> > >> > > > >
> > >> > > > > If directories are created every 24 hours:
> > >> > > > >
> > >> > > > > FCST_POINT_STAT_INPUT_TEMPLATE =
> > >> > > > >
{valid?fmt=%Y%m%d%H?truncate=24H}/wrfout_d02_{valid?fmt=%Y-%m-
> > >> > > > > %d_%H}:00:
> > >> > > > > 00_cf.nc
> > >> > > > >
> > >> > > > > If directories are created every 12 hours:
> > >> > > > >
> > >> > > > > FCST_POINT_STAT_INPUT_TEMPLATE =
> > >> > > > >
{valid?fmt=%Y%m%d%H?truncate=12H}/wrfout_d02_{valid?fmt=%Y-%m-
> > >> > > > > %d_%H}:00:
> > >> > > > > 00_cf.nc
> > >> > > > >
> > >> > > > > Let me know if that doesn't work for you.
> > >> > > > >
> > >> > > > > Thanks,
> > >> > > > > George
> > >> > > > >
> > >> > > > > On Thu, Apr 8, 2021 at 9:46 AM zhuming at ucar.edu via RT
> > >> > > > > <met_help at ucar.edu>
> > >> > > > > wrote:
> > >> > > > >
> > >> > > > > >
> > >> > > > > > <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429
> > >
> > >> > > > > >
> > >> > > > > > Hi George,
> > >> > > > > >
> > >> > > > > > Processing Obs data using "PB2NC" are right now.
> > >> > > > > >
> > >> > > > > > Now I'm using "PointStat" , I set VALID_BEG =
2015071500,
> > >> > VALID_END =
> > >> > > > > > 2015071501. It looks for the forecast data
2015071500/
> > >> > > > > > wrfout_d02_2015-07-15_00:00:00_cf.nc, and 2015071501/
> > >> > > > > > wrfout_d02_2015-07-15_01:00:00_cf.nc.
> > >> > > > > > But my forecast data are all put in dir of
2015071500/.
> > >> > > > > > So how could I set FCST_POINT_STAT_INPUT_TEMPLATE to
get
> the
> > >> right
> > >> > > > > > fcst
> > >> > > > > > data?
> > >> > > > > >
> > >> > > > > > FCST_POINT_STAT_INPUT_TEMPLATE =
> > >> > > > > > {valid?fmt=%Y%m%d%H}/wrfout_d02_{valid?fmt=%Y-%m-
%d_%H}:00:
> > >> > 00_cf.nc
> > >> > > > > >
> > >> > > > > >
> > >> > > > > > config file:
> /gpfs/fs1/work/zhuming/METplus4/PointStat1.conf
> > >> > > > > >
> > >> > > > > > log file:
> > >> > > > > >
> > >> > >
> >
/glade/scratch/zhuming/metplus4/logs/master_metplus.log.20210408085450
> > >> > > > > >
> > >> > > > > >
> > >> > > > > > Thanks,
> > >> > > > > >
> > >> > > > > > Zhuming
> > >> > > > > >
> > >> > > > > > On Wed, Apr 7, 2021 at 5:43 PM George McCabe via RT
> > >> > > > > > <met_help at ucar.edu>
> > >> > > > > > wrote:
> > >> > > > > >
> > >> > > > > > > Hi Zhuming,
> > >> > > > > > >
> > >> > > > > > > Looking at the log output, it looks like the data
is
> stored
> > in
> > >> > the
> > >> > > > > > prepBUFR
> > >> > > > > > > files differently that you may have thought:
> > >> > > > > > >
> > >> > > > > > > DEBUG 2: PrepBufr Time Center:
20150715_000000
> > >> > > > > > > DEBUG 2: Searching Time Window:
20150715_030000 to
> > >> > > > > > 20150715_040000
> > >> > > > > > > 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65%
70% 75%
> > 80%
> > >> > 85%
> > >> > > > > > > 90%
> > >> > > > > > 95%
> > >> > > > > > > 100%
> > >> > > > > > >
> > >> > > > > > > DEBUG 2: Total Messages processed =
590836
> > >> > > > > > > DEBUG 2: Rejected based on message type = 0
> > >> > > > > > > DEBUG 2: Rejected based on station id = 0
> > >> > > > > > > DEBUG 2: Rejected based on valid time =
590815
> > >> > > > > > > DEBUG 2: Rejected based on masking grid =
20
> > >> > > > > > > DEBUG 2: Rejected based on masking polygon = 0
> > >> > > > > > > DEBUG 2: Rejected based on elevation = 0
> > >> > > > > > > DEBUG 2: Rejected based on pb report type = 1
> > >> > > > > > > DEBUG 2: Rejected based on input report type = 0
> > >> > > > > > > DEBUG 2: Rejected based on instrument type = 0
> > >> > > > > > > DEBUG 2: Rejected based on zero observations = 0
> > >> > > > > > > DEBUG 2: Total Messages retained = 0
> > >> > > > > > > DEBUG 2: Total observations retained or derived = 0
> > >> > > > > > > DEBUG 1: Obs time between *20150714_210000 and
> > >> 20150715_030000*
> > >> > > > > > > WARNING:
> > >> > > > > > > WARNING: process_pbfile() -> No PrepBufr messages
retained
> > >> from
> > >> > > > > > > file:
> > >> > > > > > > /glade/collections/rda/data/ds337.0/prepnr/2015/
> > >> > > > > > > prepbufr.gdas.2015071500.nr
> > >> > > > > > > WARNING:
> > >> > > > > > > ERROR :
> > >> > > > > > > ERROR : write_netcdf_hdr_data() -> No PrepBufr
messages
> > >> > retained.
> > >> > > > > > Nothing
> > >> > > > > > > to write.
> > >> > > > > > > ERROR :
> > >> > > > > > >
> > >> > > > > > > The prepbufr.gdas.2015071500.nr file actually
contains
> data
> > >> from
> > >> > > > > > > 20150714_210000 to 20150715_030000, not
20150715_000000 to
> > >> > > > > > 20150715_060000.
> > >> > > > > > >
> > >> > > > > > > To get the correct file for each run, you will have
to
> shift
> > >> the
> > >> > > > > > > valid
> > >> > > > > > time
> > >> > > > > > > before truncating it so that it gets the correct
file for
> > each
> > >> > run.
> > >> > > > > > Instead
> > >> > > > > > > of setting:
> > >> > > > > > >
> > >> > > > > > > PB2NC_INPUT_TEMPLATE =
> > >> > > > > > >
> > >> >
{valid?fmt=%Y}/prepbufr.gdas.{valid?fmt=%Y%m%d%H?truncate=21600}.nr
> > >> > > > > > >
> > >> > > > > > > You will have to also shift forward 3 hours, i.e.:
> > >> > > > > > >
> > >> > > > > > > PB2NC_INPUT_TEMPLATE =
> > >> > > > > > >
> > >> > > > > > >
> > >> > > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
>
{valid?fmt=%Y}/prepbufr.gdas.{valid?fmt=%Y%m%d%H?shift=3H?truncate=21600}.nr
> > >> > > > > > >
> > >> > > > > > > This says to add 3 hours to the current valid time
before
> > >> > > > > > > truncating to
> > >> > > > > > the
> > >> > > > > > > nearest 6 hours.
> > >> > > > > > >
> > >> > > > > > > Valid: 20150715 0Z + 3 = 3Z truncate 6H = 0Z
> > >> > > > > > > File: prepbufr.gdas.2015071500.nr
> > >> > > > > > >
> > >> > > > > > > Valid: 20150715 1Z + 3 = 4Z truncate 6H = 0Z
> > >> > > > > > > File: prepbufr.gdas.2015071500.nr
> > >> > > > > > >
> > >> > > > > > > Valid: 20150715 2Z + 3 = 5Z truncate 6H = 0Z
> > >> > > > > > > File: prepbufr.gdas.2015071500.nr
> > >> > > > > > >
> > >> > > > > > > Valid: 20150715 3Z + 3 = 6Z truncate 6H = 6Z
> > >> > > > > > > File: prepbufr.gdas.2015071506.nr
> > >> > > > > > >
> > >> > > > > > > Let me know if that doesn't work.
> > >> > > > > > >
> > >> > > > > > > Thanks,
> > >> > > > > > > George
> > >> > > > > > >
> > >> > > > > > >
> > >> > > > > > > On Wed, Apr 7, 2021 at 5:32 PM zhuming at ucar.edu via
RT <
> > >> > > > > > met_help at ucar.edu>
> > >> > > > > > > wrote:
> > >> > > > > > >
> > >> > > > > > > >
> > >> > > > > > > > <URL:
> > >> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429
> > >> > >
> > >> > > > > > > >
> > >> > > > > > > > Hi George,
> > >> > > > > > > >
> > >> > > > > > > > Now makes me confused is why processed obs of
several
> > times
> > >> are
> > >> > > > > > missing.
> > >> > > > > > > > There has files of time at 00,01,02, but no
03,04,05;
> has
> > >> > > > > > > > 06,07,08, but
> > >> > > > > > > no
> > >> > > > > > > > 09,10,11, has 12,13,14, no 15,16,17; has
18,19,20, but
> > with
> > >> > > > > > > > 21,no
> > >> > > > > > 22,23.
> > >> > > > > > > > From yours and mines:
> > >> > > > > > > >
> > >> > > > > > > > ls -l /glade/u/home/mccabe/data/out/pb2nc1/
> > >> > > > > > > >
> > >> > > > > > > >
> > >> > > > > > > > -rw-r--r-- 1 mccabe ral 10141339 Apr 6 17:52
> > >> > pbs.2015071500.nc
> > >> > > > > > > >
> > >> > > > > > > > -rw-r--r-- 1 mccabe ral 5413907 Apr 6 17:56
> > >> > pbs.2015071501.nc
> > >> > > > > > > >
> > >> > > > > > > > -rw-r--r-- 1 mccabe ral 4923711 Apr 6 18:00
> > >> > pbs.2015071502.nc
> > >> > > > > > > >
> > >> > > > > > > > -rw-r--r-- 1 mccabe ral 5724447 Apr 6 18:14
> > >> > pbs.2015071506.nc
> > >> > > > > > > >
> > >> > > > > > > > -rw-r--r-- 1 mccabe ral 5280463 Apr 6 18:18
> > >> > pbs.2015071507.nc
> > >> > > > > > > >
> > >> > > > > > > > -rw-r--r-- 1 mccabe ral 4706119 Apr 6 18:22
> > >> > pbs.2015071508.nc
> > >> > > > > > > >
> > >> > > > > > > > -rw-r--r-- 1 mccabe ral 9878263 Apr 6 18:36
> > >> > pbs.2015071512.nc
> > >> > > > > > > >
> > >> > > > > > > > -rw-r--r-- 1 mccabe ral 5392527 Apr 6 18:40
> > >> > pbs.2015071513.nc
> > >> > > > > > > >
> > >> > > > > > > > -rw-r--r-- 1 mccabe ral 96 Apr 6 18:40
> > >> > pbs.2015071514.nc
> > >> > > > > > > >
> > >> > > > > > > > -rw-r--r-- 1 mccabe ral 96 Apr 6 18:42
> > >> > pbs.2015071515.nc
> > >> > > > > > > >
> > >> > > > > > > >
> > >> > > > > > > >
cheyenne3:/glade/scratch/zhuming/metplus4/pb2nc1>l
> > >> > > > > > > >
> > >> > > > > > > > total 96112
> > >> > > > > > > >
> > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 11550172 Apr 7 15:11
> > >> > > pbs.2015071500.nc
> > >> > > > > > > >
> > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 6615956 Apr 7 15:14
> > >> > > pbs.2015071501.nc
> > >> > > > > > > >
> > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 5984328 Apr 7 15:18
> > >> > > pbs.2015071502.nc
> > >> > > > > > > >
> > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 6980712 Apr 7 15:33
> > >> > > pbs.2015071506.nc
> > >> > > > > > > >
> > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 6455824 Apr 7 15:36
> > >> > > pbs.2015071507.nc
> > >> > > > > > > >
> > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 5749432 Apr 7 15:40
> > >> > > pbs.2015071508.nc
> > >> > > > > > > >
> > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 11284936 Apr 7 15:55
> > >> > > pbs.2015071512.nc
> > >> > > > > > > >
> > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 6600888 Apr 7 15:59
> > >> > > pbs.2015071513.nc
> > >> > > > > > > >
> > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 5868168 Apr 7 16:03
> > >> > > pbs.2015071514.nc
> > >> > > > > > > >
> > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 7273092 Apr 7 16:18
> > >> > > pbs.2015071518.nc
> > >> > > > > > > >
> > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 6656344 Apr 7 16:22
> > >> > > pbs.2015071519.nc
> > >> > > > > > > >
> > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 6109788 Apr 7 16:27
> > >> > > pbs.2015071520.nc
> > >> > > > > > > >
> > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 58808 Apr 7 16:31
> > >> > > pbs.2015071521.nc
> > >> > > > > > > >
> > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 11132440 Apr 7 16:43
> > >> > > pbs.2015071600.nc
> > >> > > > > > > >
> > >> > > > > > > >
> > >> > > > > > > >
> > >> > > > > > > > Thanks,
> > >> > > > > > > >
> > >> > > > > > > > Zhuming
> > >> > > > > > > >
> > >> > > > > > > > On Tue, Apr 6, 2021 at 6:34 PM George McCabe via
RT
> > >> > > > > > > > <met_help at ucar.edu
> > >> > > > > > >
> > >> > > > > > > > wrote:
> > >> > > > > > > >
> > >> > > > > > > > > Hi Zhuming,
> > >> > > > > > > > >
> > >> > > > > > > > > Thanks for the information. You will need to
use the
> > >> truncate
> > >> > > > > > > > > keyword
> > >> > > > > > > to
> > >> > > > > > > > > find the files, then set the valid beg/end
range to 1
> > >> hour.
> > >> > > > > > > > > This is
> > >> > > > > > > what
> > >> > > > > > > > I
> > >> > > > > > > > > changed from your PB2NC1.conf config file:
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > > *PB2NC_VALID_BEGIN =
> > {valid?fmt=%Y%m%d_%H}PB2NC_VALID_END
> > >> =
> > >> > > > > > > > > {valid?fmt=%Y%m%d_%H?shift=1H}*
> > >> > > > > > > > >
> > >> > > > > > > > > *PB2NC_INPUT_TEMPLATE =
> > >> > > > > > > > >
> > >> > > >
> >
{valid?fmt=%Y}/prepbufr.gdas.{valid?fmt=%Y%m%d%H?truncate=21600}.nr*
> > >> > > > > > > > >
> > >> > > > > > > > > My config file is on cheyenne here:
> > >> > > > > > > > > /glade/u/home/mccabe/PB2NC1.conf
> > >> > > > > > > > >
> > >> > > > > > > > > I just started running it and the output is
being
> > written
> > >> > here:
> > >> > > > > > > > > /glade/u/home/mccabe/data/out/pb2nc1
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > > For PointStat, you will want to loop over the
same
> > times.
> > >> I
> > >> > > > > > > > > noticed
> > >> > > > > > you
> > >> > > > > > > > > specify message type in the VAR<n>_OPTIONS, but
you do
> > not
> > >> > need
> > >> > > > > > > > > to do
> > >> > > > > > > > this
> > >> > > > > > > > > since there is a POINT_STAT_MESSAGE_TYPE
variable that
> > >> > defines
> > >> > > > > > > > > this.
> > >> > > > > > > This
> > >> > > > > > > > > is what I changed in your PointStat1.conf file:
> > >> > > > > > > > >
> > >> > > > > > > > > *VALID_END = 2015071600*
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > > *VALID_INCREMENT = 1HPOINT_STAT_OBS_VALID_BEG
> > >> > > > > > =POINT_STAT_OBS_VALID_BEG
> > >> > > > > > > > =*
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > > *POINT_STAT_MESSAGE_TYPE = ADPSFC*
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > > *#OBS_VAR1_OPTIONS = message_type =
> > >> > > > > > > > > ["ADPSFC"];#OBS_VAR2_OPTIONS =
> > >> > > > > > > > > message_type = ["ADPSFC"];#OBS_VAR3_OPTIONS =
> > >> message_type =
> > >> > > > > > > > > ["ADPSFC"];#OBS_POINT_STAT_INPUT_DIR =
> > >> > > > > > > > > /glade/scratch/zhuming/metplus4/pb2nc1*
> > >> > > > > > > > >
> > >> > > > > > > > > *OBS_POINT_STAT_INPUT_DIR =
{OUTPUT_BASE}/pb2nc1*
> > >> > > > > > > > >
> > >> > > > > > > > > Or you can view my file in the same directory.
I
> usually
> > >> try
> > >> > to
> > >> > > > > > > > > test
> > >> > > > > > > > with a
> > >> > > > > > > > > single run time to make sure it is working
properly,
> so
> > I
> > >> set
> > >> > > > > > VALID_END
> > >> > > > > > > > > back to 2015071500. I ran it and it produced an
output
> > >> file,
> > >> > > > > > > > > but
> > >> > > > > > there
> > >> > > > > > > > were
> > >> > > > > > > > > no values besides the header. I checked the log
file
> and
> > >> saw
> > >> > > > > > > > > this:
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > > *DEBUG 2: Processing T_sfc(0,*,*) versus
TMP/Z2, for
> > >> > > > > > > > > observation type
> > >> > > > > > > > > ADPSFC, over region FULL, for interpolation
method
> > >> BILIN(4),
> > >> > > > > > > > > using 0
> > >> > > > > > > > > matched pairs.DEBUG 3: Number of matched pairs
=
> > 0DEBUG
> > >> 3:
> > >> > > > > > > Observations
> > >> > > > > > > > > processed = 393373DEBUG 3: Rejected: station
id
> > =
> > >> > > > > > > > > 0DEBUG 3:
> > >> > > > > > > > > Rejected: obs type = 390528DEBUG 3:
Rejected:
> > valid
> > >> > time
> > >> > > > > > =
> > >> > > > > > > > > 0DEBUG 3: Rejected: bad obs value = 0DEBUG 3:
> > Rejected:
> > >> off
> > >> > > > > > > > > the
> > >> > > > > > grid
> > >> > > > > > > > =
> > >> > > > > > > > > 2586DEBUG 3: Rejected: topography = 0DEBUG
3:
> > >> Rejected:
> > >> > > > > > > > > level
> > >> > > > > > > > mismatch
> > >> > > > > > > > > = 259DEBUG 3: Rejected: quality marker =
0DEBUG 3:
> > >> Rejected:
> > >> > > > > > message
> > >> > > > > > > > type
> > >> > > > > > > > > = 0DEBUG 3: Rejected: masking region = 0DEBUG
3:
> > >> Rejected:
> > >> > bad
> > >> > > > > > fcst
> > >> > > > > > > > > value = 0DEBUG 3: Rejected: bad climo mean =
0DEBUG
> 3:
> > >> > > > > > > > > Rejected:
> > >> > > > > > bad
> > >> > > > > > > > > climo stdev = 0DEBUG 3: Rejected: duplicates
=
> > 0DEBUG
> > >> > > > > > > > > 2:DEBUG 2:
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > >
> > >> > > > > > >
> > >> > > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
>
--------------------------------------------------------------------------------DEBUG
> > >> > > > > > > > > 2:DEBUG 2: Processing u_10m_gr(0,*,*) versus
UGRD/Z10,
> > for
> > >> > > > > > observation
> > >> > > > > > > > type
> > >> > > > > > > > > ADPSFC, over region FULL, for interpolation
method
> > >> BILIN(4),
> > >> > > > > > > > > using 0
> > >> > > > > > > > > matched pairs.DEBUG 3: Number of matched pairs
=
> > 0DEBUG
> > >> 3:
> > >> > > > > > > Observations
> > >> > > > > > > > > processed = 393373*
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > > *DEBUG 3: Rejected: station id = 0DEBUG 3:
> > Rejected:
> > >> obs
> > >> > > > > > > > > type
> > >> > > > > > > > =
> > >> > > > > > > > > 382965DEBUG 3: Rejected: valid time =
0DEBUG 3:
> > >> > Rejected:
> > >> > > > > > > > > bad
> > >> > > > > > obs
> > >> > > > > > > > > value = 0DEBUG 3: Rejected: off the grid =
> > 9240DEBUG
> > >> 3:
> > >> > > > > > Rejected:
> > >> > > > > > > > > topography = 0DEBUG 3: Rejected: level
mismatch
> =
> > >> > > > > > > > > 1168DEBUG 3:
> > >> > > > > > > > > Rejected: quality marker = 0DEBUG 3: Rejected:
> message
> > >> type
> > >> > > > > > > > > =
> > >> > > > > > > 0DEBUG
> > >> > > > > > > > 3:
> > >> > > > > > > > > Rejected: masking region = 0DEBUG 3: Rejected:
bad
> fcst
> > >> > value
> > >> > > > > > > > > =
> > >> > > > > > > 0DEBUG
> > >> > > > > > > > 3:
> > >> > > > > > > > > Rejected: bad climo mean = 0DEBUG 3: Rejected:
bad
> > climo
> > >> > stdev
> > >> > > > > > > > > =
> > >> > > > > > > 0DEBUG
> > >> > > > > > > > 3:
> > >> > > > > > > > > Rejected: duplicates = 0DEBUG 2:DEBUG 2:
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > >
> > >> > > > > > >
> > >> > > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
>
--------------------------------------------------------------------------------DEBUG
> > >> > > > > > > > > 2:DEBUG 2: Processing v_10m_gr(0,*,*) versus
VGRD/Z10,
> > for
> > >> > > > > > observation
> > >> > > > > > > > type
> > >> > > > > > > > > ADPSFC, over region FULL, for interpolation
method
> > >> BILIN(4),
> > >> > > > > > > > > using 0
> > >> > > > > > > > > matched pairs.DEBUG 3: Number of matched pairs
=
> > 0DEBUG
> > >> 3:
> > >> > > > > > > Observations
> > >> > > > > > > > > processed = 393373DEBUG 3: Rejected: station
id
> > =
> > >> > > > > > > > > 0DEBUG 3:
> > >> > > > > > > > > Rejected: obs type = 382965DEBUG 3:
Rejected:
> > valid
> > >> > time
> > >> > > > > > =
> > >> > > > > > > > > 0DEBUG 3: Rejected: bad obs value = 0DEBUG 3:
> > Rejected:
> > >> off
> > >> > > > > > > > > the
> > >> > > > > > grid
> > >> > > > > > > > =
> > >> > > > > > > > > 9240DEBUG 3: Rejected: topography = 0DEBUG
3:
> > >> Rejected:
> > >> > > > > > > > > level
> > >> > > > > > > > mismatch
> > >> > > > > > > > > = 1168DEBUG 3: Rejected: quality marker =
0DEBUG 3:
> > >> > Rejected:
> > >> > > > > > message
> > >> > > > > > > > > type = 0DEBUG 3: Rejected: masking region =
0DEBUG
> > 3:
> > >> > > > > > > > > Rejected:
> > >> > > > > > bad
> > >> > > > > > > > > fcst value = 0DEBUG 3: Rejected: bad climo
mean =
> > >> 0DEBUG 3:
> > >> > > > > > Rejected:
> > >> > > > > > > > bad
> > >> > > > > > > > > climo stdev = 0*
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > > *DEBUG 3: Rejected: duplicates = 0DEBUG
2:DEBUG
> 2:
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > >
> > >> > > > > > >
> > >> > > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
>
--------------------------------------------------------------------------------DEBUG
> > >> > > > > > > > > 2:DEBUG 1: Output file:
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > >
> > >> > > > > > >
> > >> > > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
>
/glade/u/home/mccabe/data/out/point_stat1/point_stat_000000L_20150715_000000V.stat*
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > > This describes the reason why each observation
was
> > >> rejected.
> > >> > > > > > > > > Some
> > >> > > > > > > > > observations didn't match the message type you
> > requested,
> > >> > some
> > >> > > > > > > > > were
> > >> > > > > > not
> > >> > > > > > > > > within the forecast grid, and some did not
match the
> > >> vertical
> > >> > > > > > > > > level.
> > >> > > > > > > You
> > >> > > > > > > > > will have to make adjustments to your settings
so that
> > the
> > >> > data
> > >> > > > > > matches
> > >> > > > > > > > up
> > >> > > > > > > > > properly. Maybe the grid you are using in the
PB2NC
> > config
> > >> > file
> > >> > > > > > > > > does
> > >> > > > > > > not
> > >> > > > > > > > > match the forecast grid and you are missing
values. If
> > >> that
> > >> > is
> > >> > > > > > > > > the
> > >> > > > > > > case,
> > >> > > > > > > > > you will have to reconfigure and rerun PB2NC to
get
> > >> output on
> > >> > > > > > > > > the
> > >> > > > > > > correct
> > >> > > > > > > > > gr5id. Maybe the observation points don't line
up with
> > the
> > >> > > > > > > > > vertical
> > >> > > > > > > level
> > >> > > > > > > > > of the model fields. You can use a range of
level
> values
> > >> for
> > >> > > > > > > > > the
> > >> > > > > > > > > observation data to allow data within that
range, i.e.
> > >> Z0-10.
> > >> > > > > > > > >
> > >> > > > > > > > > Let me know if you are still having issues
getting
> this
> > to
> > >> > work
> > >> > > > > > > > > and I
> > >> > > > > > > can
> > >> > > > > > > > > help out.
> > >> > > > > > > > >
> > >> > > > > > > > > Thanks,
> > >> > > > > > > > > George
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > > On Tue, Apr 6, 2021 at 5:17 PM zhuming at ucar.edu
via
> RT
> > <
> > >> > > > > > > > met_help at ucar.edu>
> > >> > > > > > > > > wrote:
> > >> > > > > > > > >
> > >> > > > > > > > > >
> > >> > > > > > > > > > <URL:
> > >> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429
> > >> > > > > > > > > > >
> > >> > > > > > > > > >
> > >> > > > > > > > > > Hi George,
> > >> > > > > > > > > >
> > >> > > > > > > > > > Thanks for your reply!
> > >> > > > > > > > > >
> > >> > > > > > > > > > I'm still confused, I need to output hourly
obs data
> > to
> > >> do
> > >> > > > > > > > > > hourly
> > >> > > > > > > > > > verification.
> > >> > > > > > > > > > The original GDAS Prepbufr data has 4 files
one day,
> > >> should
> > >> > > > > > > > > > with 6
> > >> > > > > > > > hours
> > >> > > > > > > > > > obs data in one file.
> > >> > > > > > > > > >
> > >> > > > > > > > > > After "PB2NC", I can see pbs.2015071500.nc
(obs
> time
> > is
> > >> > 00?
> > >> > > > > > > > > > or 6
> > >> > > > > > > times
> > >> > > > > > > > > of
> > >> > > > > > > > > > 00,01,02,03,04,05?), pbs.2015071506.nc...
> > >> > > > > > > > > >
> > >> > > > > > > > > > my config file :
> > >> > > > > > cheyenne:/gpfs/fs1/work/zhuming/METplus4/PB2NC1.conf
> > >> > > > > > > > > > and
/gpfs/fs1/work/zhuming/METplus4/PointStat1.conf
> > >> > > > > > > > > >
> > >> > > > > > > > > > Thanks,
> > >> > > > > > > > > > Zhuming
> > >> > > > > > > > > >
> > >> > > > > > > > > > On Tue, Apr 6, 2021 at 4:23 PM George McCabe
via RT
> <
> > >> > > > > > > met_help at ucar.edu
> > >> > > > > > > > >
> > >> > > > > > > > > > wrote:
> > >> > > > > > > > > >
> > >> > > > > > > > > > > Hi Zhuming,
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > To run the tools hourly you would set the
> > >> VALID_INCREMENT
> > >> > > > > > > > > > > value
> > >> > > > > > for
> > >> > > > > > > > 1H.
> > >> > > > > > > > > > > Here is an example that runs once per hour
for a
> day
> > >> from
> > >> > > > > > > 2015-07-15
> > >> > > > > > > > 0Z
> > >> > > > > > > > > > to
> > >> > > > > > > > > > > 2017-07-16 0Z:
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > VALID_TIME_FMT = %Y%m%d%H
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > VALID_BEG = 2015071500
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > VALID_END = 2015071600
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > VALID_INCREMENT = 1H
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > LEAD_SEQ = 0
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > Each run time will use the
PB2NC_INPUT_TEMPLATE to
> > >> find
> > >> > > > > > > > > > > files
> > >> > > > > > whose
> > >> > > > > > > > > > > filenames match the time you need.
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > To read prepBUFR observations for 12 hours,
you
> will
> > >> want
> > >> > > > > > > > > > > to set
> > >> > > > > > > > > > > PB2NC_VALID_BEGIN and PB2NC_VALID_END to
specify
> the
> > >> > range
> > >> > > > > > > > > > > around
> > >> > > > > > > the
> > >> > > > > > > > > > valid
> > >> > > > > > > > > > > time to process. These values depend on
what is
> > >> contained
> > >> > > > > > > > > > > inside
> > >> > > > > > > your
> > >> > > > > > > > > > > prepBUFR files. If you want to use data
from the
> > valid
> > >> > time
> > >> > > > > > > > > > > of
> > >> > > > > > the
> > >> > > > > > > > > > filename
> > >> > > > > > > > > > > backwards 12 hours, you would set:
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > PB2NC_VALID_BEGIN =
> {valid?fmt=%Y%m%d_%H?shift=-12H
> > }
> > >> > > > > > > > > > > PB2NC_VALID_END = {valid?fmt=%Y%m%d_%H}
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > For a runtime of 2015071512, this would be
> > substituted
> > >> > as:
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > valid_beg: 20150715_00
> > >> > > > > > > > > > > valid_end: 20150715_12
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > If you want to create an output file for
each hour
> > in
> > >> > PB2NC
> > >> > > > > > > > > > > you
> > >> > > > > > > would
> > >> > > > > > > > > > need
> > >> > > > > > > > > > > to make sure that the hour is specified in
the
> > >> > > > > > > PB2NC_OUTPUT_TEMPLATE.
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > PointStat has similar config variables to
set the
> > >> valid
> > >> > beg
> > >> > > > > > > > > > > and
> > >> > > > > > end
> > >> > > > > > > > > range
> > >> > > > > > > > > > > to use (POINT_STAT_OBS_VALID_BEG and
> > >> > > > > > > > > > > POINT_STAT_OBS_VALID_END).
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > If your prepBUFR files are separated out
into
> > multiple
> > >> > > > > > > > > > > files that
> > >> > > > > > > > need
> > >> > > > > > > > > to
> > >> > > > > > > > > > > be read into PB2NC, you can set a file
window
> around
> > >> the
> > >> > > > > > > > > > > valid
> > >> > > > > > time
> > >> > > > > > > > to
> > >> > > > > > > > > > > allow files to be passed into the app. More
> > >> information
> > >> > on
> > >> > > > > > > > > > > that
> > >> > > > > > > > > > > functionality is here:
> > >> > > > > > > > > > >
> > >> > > > > > > > > > >
> > >> > > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > >
> > >> > > > > > >
> > >> > > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
>
https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#using-
> > >> > > > > > windows-to-find-valid-files
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > If you are still having trouble getting
this
> > >> configured,
> > >> > > > > > > > > > > you
> > >> > > > > > could
> > >> > > > > > > > send
> > >> > > > > > > > > > me
> > >> > > > > > > > > > > the paths of the files you are trying to
read and
> > what
> > >> > data
> > >> > > > > > > > > > > is
> > >> > > > > > > > > contained
> > >> > > > > > > > > > in
> > >> > > > > > > > > > > them so I can give you suggestions on what
to set
> > for
> > >> the
> > >> > > > > > > > > configuration.
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > Thanks,
> > >> > > > > > > > > > > George
> > >> > > > > > > > > > >
> > >> > > > > > > > > > >
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > On Tue, Apr 6, 2021 at 2:17 PM Julie
Prestopnik
> via
> > >> RT <
> > >> > > > > > > > > > met_help at ucar.edu>
> > >> > > > > > > > > > > wrote:
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > >
> > >> > > > > > > > > > > > Tue Apr 06 14:16:42 2021: Request 99429
was
> acted
> > >> upon.
> > >> > > > > > > > > > > > Transaction: Given to mccabe (George
McCabe) by
> > >> jpresto
> > >> > > > > > > > > > > > Queue: met_help
> > >> > > > > > > > > > > > Subject: How to set time for hourly
12-24
> > hours
> > >> > data
> > >> > > > > > > > > > > > ?
> > >> > > > > > > > > > > > Owner: mccabe
> > >> > > > > > > > > > > > Requestors: zhuming at ucar.edu
> > >> > > > > > > > > > > > Status: new
> > >> > > > > > > > > > > > Ticket <URL:
> > >> > > > > > > > >
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > >
> > >> > > > > > > > > > > >
> > >> > > > > > > > > > > > This transaction appears to have no
content
> > >> > > > > > > > > > > >
> > >> > > > > > > > > > >
> > >> > > > > > > > > > >
> > >> > > > > > > > > > > --
> > >> > > > > > > > > > > George McCabe - Software Engineer III
> > >> > > > > > > > > > > National Center for Atmospheric Research
> > >> > > > > > > > > > > Research Applications Laboratory
> > >> > > > > > > > > > > 303-497-2768
> > >> > > > > > > > > > > ---
> > >> > > > > > > > > > > My working day may not be your working day.
Please
> > do
> > >> not
> > >> > > > > > > > > > > feel
> > >> > > > > > > > obliged
> > >> > > > > > > > > to
> > >> > > > > > > > > > > reply to this email outside of your normal
working
> > >> hours.
> > >> > > > > > > > > > >
> > >> > > > > > > > > > >
> > >> > > > > > > > > >
> > >> > > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > > > --
> > >> > > > > > > > > George McCabe - Software Engineer III
> > >> > > > > > > > > National Center for Atmospheric Research
> > >> > > > > > > > > Research Applications Laboratory
> > >> > > > > > > > > 303-497-2768
> > >> > > > > > > > > ---
> > >> > > > > > > > > My working day may not be your working day.
Please do
> > not
> > >> > feel
> > >> > > > > > obliged
> > >> > > > > > > to
> > >> > > > > > > > > reply to this email outside of your normal
working
> > hours.
> > >> > > > > > > > >
> > >> > > > > > > > >
> > >> > > > > > > >
> > >> > > > > > > >
> > >> > > > > > >
> > >> > > > > > > --
> > >> > > > > > > George McCabe - Software Engineer III
> > >> > > > > > > National Center for Atmospheric Research
> > >> > > > > > > Research Applications Laboratory
> > >> > > > > > > 303-497-2768
> > >> > > > > > > ---
> > >> > > > > > > My working day may not be your working day. Please
do not
> > feel
> > >> > > > > > > obliged to
> > >> > > > > > > reply to this email outside of your normal working
hours.
> > >> > > > > > >
> > >> > > > > > >
> > >> > > > > >
> > >> > > > > >
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > >
> > >> > >
> > >> >
> > >> > --
> > >> > George McCabe - Software Engineer III
> > >> > National Center for Atmospheric Research
> > >> > Research Applications Laboratory
> > >> > 303-497-2768
> > >> > ---
> > >> > My working day may not be your working day. Please do not
feel
> obliged
> > >> to
> > >> > reply to this email outside of your normal working hours.
> > >> >
> > >> >
> > >>
> > >>
> > >
> > > --
> > > George McCabe - Software Engineer III
> > > National Center for Atmospheric Research
> > > Research Applications Laboratory
> > > 303-497-2768
> > > ---
> > > My working day may not be your working day. Please do not feel
obliged
> to
> > > reply to this email outside of your normal working hours.
> > >
> >
> >
> > --
> > George McCabe - Software Engineer III
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > 303-497-2768
> > ---
> > My working day may not be your working day. Please do not feel
obliged to
> > reply to this email outside of your normal working hours.
> >
> >
>
>
------------------------------------------------
Subject: How to set time for hourly 12-24 hours data ?
From: George McCabe
Time: Mon May 03 15:47:14 2021
Hi Zhuming,
The METplus 4.0.0-beta5 release is now available on Cheyenne. If you
are
able to test that this version works and fixes some of the issues you
mentioned in this ticket before the 4.0.0 release next week, that
would be
great! Information on how to use the new version can be found here:
https://dtcenter.org/community-code/metplus/metplus-4-0-existing-
builds
(Under NCAR > CHEYENNE > METplus-4.0.0-beta5)
Here are the commands to set it up:
module use /glade/p/ral/jntp/MET/METplus/modulefiles
module load metplus/4.0.0-beta5
ncar_pylib
Please let me know if you have any issues or questions.
Thanks,
George
On Thu, Apr 22, 2021 at 11:40 AM John Halley Gotway via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429 >
>
> Zhuming,
>
> This is John HG. George asked me to take a look at this question.
Since
> you're asking about "non-standard" levels, I assume you're asking
about the
> verification of the pressure levels against the ADPUPA message type.
>
> In general, the logic is this...
> For each point observation, loop through all the verification tasks
and see
> if it should be used.
> For example, let's talk about the 4 level entry for VAR3:
>
> FCST_VAR3_NAME = v_tr_p
> FCST_VAR3_LEVELS = ...(0,3,*,*)...
> OBS_VAR3_NAME = VGRD
> OBS_VAR3_LEVELS = ...P575-625...
>
> The point observation will be used here if it's a "VGRD" observation
and
> has a level value between 575 and 625. This verification task has
been
> configured to only verify a single forecast level of data:
v_tr_p(0,3,*,*).
> So no vertical interpolation is done. MET interpolates horizontally
from
> the forecast grid points to the observation lat/lon location (using
> user-configurable interpolation options).
>
> To put it simply, all obs between 575 and 625 mb are compared
directly to
> the forecast values from v_tr_p(0,3,*,*). If the model output is
actually
> on pressure levels, then that's a reasonable thing to do. If the
model's
> vertical coordinate is something other than pressure levels, then
that's a
> potential source for large errors.
>
> Note that MET does do vertical interpolation when processing the
GRIB
> output from the Unified Post Processor tool. But all the details of
that
> depend on how the configuration file is setup. I'll explain below.
>
> To date we have avoided including direct knowledge about the
vertical
> coordinate of each modelling system in the verification code. The
goal is
> to create verification tools applicable to many models, rather than
> customizing them for each. We do realize that desire for the latter
and are
> exploring ways to pull data from the IODA data assimilation system
instead,
> which necessarily knows a lot about each model.
>
> So let's say you were using GRIB data, and configured it like this:
>
> VAR3_NAME = VGRD
> VAR3_LEVELS = P500-1000
>
> Since we haven't prefixed these options with FCST_ and OBS_, the
same
> settings are applied to both.
>
> MET would read ALL forecast GRIB records from the model file with
pressures
> between 500 and 1000. Perhaps, that's every 100mb or perhaps every
50mb? It
> reads whatever data falls in that range. For each point observation
with a
> pressure level between 500 and 1000, it'll first find the actual
model
> pressure level above/below it. It'll interpolate the forecast
> data horizontally to the obs lat/lon location for both. And then
it'll
> interpolate those values above/below vertically to the actual
observation
> level, linear in the log of pressure.
>
> This issue is that the processing of pressure LAYERS is not
currently
> supported for NetCDF data I believe. And the code could be enhanced
to
> support that. But again, this only makes sense if the model's
vertical
> coordinate is actually pressure levels and not something else.
>
> Hope that helps clarify.
>
> Thanks,
> John
>
> On Thu, Apr 22, 2021 at 9:52 AM zhuming at ucar.edu via RT
<met_help at ucar.edu
> >
> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429 >
> >
> > Hi George,
> >
> > My verification result shows that there is a big error on a non-
standard
> > level, so my question is:
> > Is the forecast grid match to Obs, or obs station match to
Forecast? I
> need
> > to do is the forecast grid interpolated to the obs location and
vertical
> > levels. So how to set the levels?
> >
> > Thanks,
> > Zhuming
> >
> > FCST_VAR3_NAME = v_10m_tr
> >
> > FCST_VAR3_LEVELS = (0,*,*)
> >
> > FCST_VAR3_THRESH = >=5, <5
> >
> > OBS_VAR3_NAME = VGRD
> >
> > OBS_VAR3_LEVELS = Z10
> >
> > OBS_VAR3_THRESH = >=5, <5
> >
> > OBS_VAR3_OPTIONS = message_type = ["ADPSFC"];
> >
> >
> > FCST_VAR3_NAME = v_tr_p
> >
> > FCST_VAR3_LEVELS =
> >
> >
>
(0,0,*,*),(0,1,*,*),(0,2,*,*),(0,3,*,*),(0,4,*,*),(0,5,*,*),(0,6,*,*),(0,7,*,*),(0,8,*,*),(0,9,*,*)
> >
> > FCST_VAR3_THRESH = >=5, <5
> >
> > OBS_VAR3_NAME = VGRD
> >
> > OBS_VAR3_LEVELS =
> >
> >
> P900-950,P825-875,P675-725,P575-625,P475-525,P375-425,P275-325,P225-
275,P175-225,P75-125
> >
> > OBS_VAR3_THRESH = >=5, <5
> >
> > OBS_VAR3_OPTIONS = message_type = ["ADPUPA"];
> >
> >
> >
> > On Thu, Apr 15, 2021 at 1:52 PM George McCabe via RT
<met_help at ucar.edu>
> > wrote:
> >
> > > Hi Zhuming,
> > >
> > > There was indeed a bug in the code. Howard has fixed the issue
with
> this
> > > pull request: https://github.com/dtcenter/MET/pull/1756
> > > This fix will be included in the MET 10.0.0-beta5 release that
is
> > > tentatively scheduled for next week. This beta version can be
installed
> > on
> > > cheyenne so you can test that the update fixes your issue. I
will
> follow
> > up
> > > and let you know when the beta release is available on cheyenne.
> > >
> > > Thanks,
> > > George
> > >
> > > On Wed, Apr 14, 2021 at 5:28 PM George McCabe <mccabe at ucar.edu>
wrote:
> > >
> > > > Hi Zhuming,
> > > >
> > > > I tried to get this to work on cheyenne with your data, but
was not
> > > > successful. I am able to read the data with plot_data_plane
and
> > generate
> > > an
> > > > image, so I think there may be a bug in the code. I have
forwarded
> info
> > > to
> > > > one of our C++ developers to look into it. I will report back
when I
> > know
> > > > more.
> > > >
> > > > Thanks,
> > > > George
> > > >
> > > > On Wed, Apr 14, 2021 at 4:13 PM zhuming at ucar.edu via RT <
> > > met_help at ucar.edu>
> > > > wrote:
> > > >
> > > >>
> > > >> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429
>
> > > >>
> > > >> Hi George,
> > > >>
> > > >> It still can't find the correct time. I tried
> > > >>
> > > >> FCST_VAR1_LEVELS = (*,*,*)
> > > >> FCST_VAR1_LEVELS = ({valid?fmt=%Y%m%d_%H%M%S},*,*)
> > > >> FCST_VAR1_LEVELS = ({valid?fmt=%Y%m%d_%H%M},*,*)
> > > >> FCST_VAR1_LEVELS = ({valid?fmt=%Y%m%d%H%M},*,*)
> > > >>
> > > >> Thanks!
> > > >>
> > > >> Zhuming
> > > >>
> > > >> On Wed, Apr 14, 2021 at 3:41 PM George McCabe via RT <
> > met_help at ucar.edu
> > > >
> > > >> wrote:
> > > >>
> > > >> > Hi Zhuming,
> > > >> >
> > > >> > It looks like you are requesting the first time in the 12
hour
> file
> > > for
> > > >> > each run:
> > > >> >
> > > >> > FCST_VAR1_LEVELS = (0,*,*)
> > > >> > ...
> > > >> > FCST_VAR2_LEVELS = (0,*,*)
> > > >> >
> > > >> > The first dimension in these files is the time offset. You
will
> need
> > > the
> > > >> > correct index that corresponds to each valid time.
Fortunately,
> MET
> > > >> makes
> > > >> > this easy by allowing you to specify the valid time for a
time
> > > dimension
> > > >> > explicitly and it will find the correct data. You can set:
> > > >> >
> > > >> > FCST_VAR1_LEVELS = ({valid?fmt=%Y%m%d_%H%M%S},*,*)
> > > >> > ...
> > > >> > FCST_VAR2_LEVELS = ({valid?fmt=%Y%m%d_%H%M%S},*,*)
> > > >> >
> > > >> > And it will substitute the valid time for each run. Let me
know if
> > you
> > > >> have
> > > >> > any questions.
> > > >> >
> > > >> > Thanks,
> > > >> > George
> > > >> >
> > > >> > On Wed, Apr 14, 2021 at 3:18 PM zhuming at ucar.edu via RT <
> > > >> met_help at ucar.edu
> > > >> > >
> > > >> > wrote:
> > > >> >
> > > >> > >
> > > >> > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429 >
> > > >> > >
> > > >> > > Hi George,
> > > >> > >
> > > >> > > Thanks for your email, I just would like to send you for
help.
> > > >> > >
> > > >> > > I already resolved the last problem with the WRF
forecast, thank
> > you
> > > >> for
> > > >> > > your help.
> > > >> > >
> > > >> > > I also need to verify the MPAS model forecast. These 12
hour
> > > >> > > MAPS forecasts are stored in a one file, such as
> > > >> > > 2015071500/mpas_wrf_cf_2015071500.nc, has 12h forecasts
> > > >> > > 2015071512/mpas_wrf_cf_2015071512.nc
> > > >> > >
> > > >> > > It seems MET is doing fine on the first 2 hours, but no
matched
> > > pairs
> > > >> for
> > > >> > > later hours.
> > > >> > > I'm not sure what's wrong with it? the time is not
matched?
> > > >> > >
> > > >> > > see the log file
> > > >> > >
> > >
/glade/scratch/zhuming/metplus4/logs/master_metplus.log.20210414145101
> > > >> > > config file
/gpfs/fs1/work/zhuming/METplus4/PointStat2.conf
> > > >> > >
> > > >> > > Thanks,
> > > >> > > Zhuming
> > > >> > >
> > > >> > > On Wed, Apr 14, 2021 at 1:56 PM George McCabe via RT <
> > > >> met_help at ucar.edu>
> > > >> > > wrote:
> > > >> > >
> > > >> > > > Hi Zhuming,
> > > >> > > >
> > > >> > > > Just following up on this ticket. Were you able to get
this
> > > >> resolved?
> > > >> > Let
> > > >> > > > me know if you need more assistance.
> > > >> > > >
> > > >> > > > Thanks,
> > > >> > > > George
> > > >> > > >
> > > >> > > > On Thu Apr 08 10:04:44 2021, mccabe wrote:
> > > >> > > > > Hi Zhuming,
> > > >> > > > >
> > > >> > > > > Typically directories that contain data for an entire
day
> are
> > > >> named
> > > >> > > > > with
> > > >> > > > > YYYYMMDD, so your filename template tag would be
> > > >> {valid?fmt=%Y%m%d},
> > > >> > > > > however that may not always be the case, as you are
seeing
> > here.
> > > >> > There
> > > >> > > > > are
> > > >> > > > > a couple ways to handle this.
> > > >> > > > >
> > > >> > > > > If you know that the directory will always be
YYYYMMDD with
> 00
> > > at
> > > >> the
> > > >> > > > > end,
> > > >> > > > > you can simply put 00 in the template instead of
adding %H
> in
> > > the
> > > >> > tag,
> > > >> > > > > i.e.
> > > >> > > > >
> > > >> > > > > FCST_POINT_STAT_INPUT_TEMPLATE =
> > > >> > > > > {valid?fmt=%Y%m%d}00/wrfout_d02_{valid?fmt=%Y-%m-
%d_%H}:00:
> > > >> 00_cf.nc
> > > >> > > > >
> > > >> > > > > If the HH value can vary, for instance if there are 2
> > > directories
> > > >> > > > > created
> > > >> > > > > per day (i.e. 2015071400 and 2015071412), then you
can use
> > the
> > > >> > > > > truncate
> > > >> > > > > keyword to handle it.
> > > >> > > > >
> > > >> > > > > If directories are created every 24 hours:
> > > >> > > > >
> > > >> > > > > FCST_POINT_STAT_INPUT_TEMPLATE =
> > > >> > > > >
> {valid?fmt=%Y%m%d%H?truncate=24H}/wrfout_d02_{valid?fmt=%Y-%m-
> > > >> > > > > %d_%H}:00:
> > > >> > > > > 00_cf.nc
> > > >> > > > >
> > > >> > > > > If directories are created every 12 hours:
> > > >> > > > >
> > > >> > > > > FCST_POINT_STAT_INPUT_TEMPLATE =
> > > >> > > > >
> {valid?fmt=%Y%m%d%H?truncate=12H}/wrfout_d02_{valid?fmt=%Y-%m-
> > > >> > > > > %d_%H}:00:
> > > >> > > > > 00_cf.nc
> > > >> > > > >
> > > >> > > > > Let me know if that doesn't work for you.
> > > >> > > > >
> > > >> > > > > Thanks,
> > > >> > > > > George
> > > >> > > > >
> > > >> > > > > On Thu, Apr 8, 2021 at 9:46 AM zhuming at ucar.edu via
RT
> > > >> > > > > <met_help at ucar.edu>
> > > >> > > > > wrote:
> > > >> > > > >
> > > >> > > > > >
> > > >> > > > > > <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429
> > > >
> > > >> > > > > >
> > > >> > > > > > Hi George,
> > > >> > > > > >
> > > >> > > > > > Processing Obs data using "PB2NC" are right now.
> > > >> > > > > >
> > > >> > > > > > Now I'm using "PointStat" , I set VALID_BEG =
2015071500,
> > > >> > VALID_END =
> > > >> > > > > > 2015071501. It looks for the forecast data
2015071500/
> > > >> > > > > > wrfout_d02_2015-07-15_00:00:00_cf.nc, and
2015071501/
> > > >> > > > > > wrfout_d02_2015-07-15_01:00:00_cf.nc.
> > > >> > > > > > But my forecast data are all put in dir of
2015071500/.
> > > >> > > > > > So how could I set FCST_POINT_STAT_INPUT_TEMPLATE
to get
> > the
> > > >> right
> > > >> > > > > > fcst
> > > >> > > > > > data?
> > > >> > > > > >
> > > >> > > > > > FCST_POINT_STAT_INPUT_TEMPLATE =
> > > >> > > > > >
> {valid?fmt=%Y%m%d%H}/wrfout_d02_{valid?fmt=%Y-%m-%d_%H}:00:
> > > >> > 00_cf.nc
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > > config file:
> > /gpfs/fs1/work/zhuming/METplus4/PointStat1.conf
> > > >> > > > > >
> > > >> > > > > > log file:
> > > >> > > > > >
> > > >> > >
> > >
/glade/scratch/zhuming/metplus4/logs/master_metplus.log.20210408085450
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > > > Thanks,
> > > >> > > > > >
> > > >> > > > > > Zhuming
> > > >> > > > > >
> > > >> > > > > > On Wed, Apr 7, 2021 at 5:43 PM George McCabe via RT
> > > >> > > > > > <met_help at ucar.edu>
> > > >> > > > > > wrote:
> > > >> > > > > >
> > > >> > > > > > > Hi Zhuming,
> > > >> > > > > > >
> > > >> > > > > > > Looking at the log output, it looks like the data
is
> > stored
> > > in
> > > >> > the
> > > >> > > > > > prepBUFR
> > > >> > > > > > > files differently that you may have thought:
> > > >> > > > > > >
> > > >> > > > > > > DEBUG 2: PrepBufr Time Center:
20150715_000000
> > > >> > > > > > > DEBUG 2: Searching Time Window:
20150715_030000
> to
> > > >> > > > > > 20150715_040000
> > > >> > > > > > > 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60%
65% 70%
> 75%
> > > 80%
> > > >> > 85%
> > > >> > > > > > > 90%
> > > >> > > > > > 95%
> > > >> > > > > > > 100%
> > > >> > > > > > >
> > > >> > > > > > > DEBUG 2: Total Messages processed =
590836
> > > >> > > > > > > DEBUG 2: Rejected based on message type =
0
> > > >> > > > > > > DEBUG 2: Rejected based on station id =
0
> > > >> > > > > > > DEBUG 2: Rejected based on valid time =
590815
> > > >> > > > > > > DEBUG 2: Rejected based on masking grid =
20
> > > >> > > > > > > DEBUG 2: Rejected based on masking polygon =
0
> > > >> > > > > > > DEBUG 2: Rejected based on elevation =
0
> > > >> > > > > > > DEBUG 2: Rejected based on pb report type =
1
> > > >> > > > > > > DEBUG 2: Rejected based on input report type =
0
> > > >> > > > > > > DEBUG 2: Rejected based on instrument type =
0
> > > >> > > > > > > DEBUG 2: Rejected based on zero observations =
0
> > > >> > > > > > > DEBUG 2: Total Messages retained =
0
> > > >> > > > > > > DEBUG 2: Total observations retained or derived =
0
> > > >> > > > > > > DEBUG 1: Obs time between *20150714_210000 and
> > > >> 20150715_030000*
> > > >> > > > > > > WARNING:
> > > >> > > > > > > WARNING: process_pbfile() -> No PrepBufr messages
> retained
> > > >> from
> > > >> > > > > > > file:
> > > >> > > > > > > /glade/collections/rda/data/ds337.0/prepnr/2015/
> > > >> > > > > > > prepbufr.gdas.2015071500.nr
> > > >> > > > > > > WARNING:
> > > >> > > > > > > ERROR :
> > > >> > > > > > > ERROR : write_netcdf_hdr_data() -> No PrepBufr
messages
> > > >> > retained.
> > > >> > > > > > Nothing
> > > >> > > > > > > to write.
> > > >> > > > > > > ERROR :
> > > >> > > > > > >
> > > >> > > > > > > The prepbufr.gdas.2015071500.nr file actually
contains
> > data
> > > >> from
> > > >> > > > > > > 20150714_210000 to 20150715_030000, not
20150715_000000
> to
> > > >> > > > > > 20150715_060000.
> > > >> > > > > > >
> > > >> > > > > > > To get the correct file for each run, you will
have to
> > shift
> > > >> the
> > > >> > > > > > > valid
> > > >> > > > > > time
> > > >> > > > > > > before truncating it so that it gets the correct
file
> for
> > > each
> > > >> > run.
> > > >> > > > > > Instead
> > > >> > > > > > > of setting:
> > > >> > > > > > >
> > > >> > > > > > > PB2NC_INPUT_TEMPLATE =
> > > >> > > > > > >
> > > >> >
> {valid?fmt=%Y}/prepbufr.gdas.{valid?fmt=%Y%m%d%H?truncate=21600}.nr
> > > >> > > > > > >
> > > >> > > > > > > You will have to also shift forward 3 hours,
i.e.:
> > > >> > > > > > >
> > > >> > > > > > > PB2NC_INPUT_TEMPLATE =
> > > >> > > > > > >
> > > >> > > > > > >
> > > >> > > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
>
{valid?fmt=%Y}/prepbufr.gdas.{valid?fmt=%Y%m%d%H?shift=3H?truncate=21600}.nr
> > > >> > > > > > >
> > > >> > > > > > > This says to add 3 hours to the current valid
time
> before
> > > >> > > > > > > truncating to
> > > >> > > > > > the
> > > >> > > > > > > nearest 6 hours.
> > > >> > > > > > >
> > > >> > > > > > > Valid: 20150715 0Z + 3 = 3Z truncate 6H = 0Z
> > > >> > > > > > > File: prepbufr.gdas.2015071500.nr
> > > >> > > > > > >
> > > >> > > > > > > Valid: 20150715 1Z + 3 = 4Z truncate 6H = 0Z
> > > >> > > > > > > File: prepbufr.gdas.2015071500.nr
> > > >> > > > > > >
> > > >> > > > > > > Valid: 20150715 2Z + 3 = 5Z truncate 6H = 0Z
> > > >> > > > > > > File: prepbufr.gdas.2015071500.nr
> > > >> > > > > > >
> > > >> > > > > > > Valid: 20150715 3Z + 3 = 6Z truncate 6H = 6Z
> > > >> > > > > > > File: prepbufr.gdas.2015071506.nr
> > > >> > > > > > >
> > > >> > > > > > > Let me know if that doesn't work.
> > > >> > > > > > >
> > > >> > > > > > > Thanks,
> > > >> > > > > > > George
> > > >> > > > > > >
> > > >> > > > > > >
> > > >> > > > > > > On Wed, Apr 7, 2021 at 5:32 PM zhuming at ucar.edu
via RT
> <
> > > >> > > > > > met_help at ucar.edu>
> > > >> > > > > > > wrote:
> > > >> > > > > > >
> > > >> > > > > > > >
> > > >> > > > > > > > <URL:
> > > >> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429
> > > >> > >
> > > >> > > > > > > >
> > > >> > > > > > > > Hi George,
> > > >> > > > > > > >
> > > >> > > > > > > > Now makes me confused is why processed obs of
several
> > > times
> > > >> are
> > > >> > > > > > missing.
> > > >> > > > > > > > There has files of time at 00,01,02, but no
03,04,05;
> > has
> > > >> > > > > > > > 06,07,08, but
> > > >> > > > > > > no
> > > >> > > > > > > > 09,10,11, has 12,13,14, no 15,16,17; has
18,19,20,
> but
> > > with
> > > >> > > > > > > > 21,no
> > > >> > > > > > 22,23.
> > > >> > > > > > > > From yours and mines:
> > > >> > > > > > > >
> > > >> > > > > > > > ls -l /glade/u/home/mccabe/data/out/pb2nc1/
> > > >> > > > > > > >
> > > >> > > > > > > >
> > > >> > > > > > > > -rw-r--r-- 1 mccabe ral 10141339 Apr 6 17:52
> > > >> > pbs.2015071500.nc
> > > >> > > > > > > >
> > > >> > > > > > > > -rw-r--r-- 1 mccabe ral 5413907 Apr 6 17:56
> > > >> > pbs.2015071501.nc
> > > >> > > > > > > >
> > > >> > > > > > > > -rw-r--r-- 1 mccabe ral 4923711 Apr 6 18:00
> > > >> > pbs.2015071502.nc
> > > >> > > > > > > >
> > > >> > > > > > > > -rw-r--r-- 1 mccabe ral 5724447 Apr 6 18:14
> > > >> > pbs.2015071506.nc
> > > >> > > > > > > >
> > > >> > > > > > > > -rw-r--r-- 1 mccabe ral 5280463 Apr 6 18:18
> > > >> > pbs.2015071507.nc
> > > >> > > > > > > >
> > > >> > > > > > > > -rw-r--r-- 1 mccabe ral 4706119 Apr 6 18:22
> > > >> > pbs.2015071508.nc
> > > >> > > > > > > >
> > > >> > > > > > > > -rw-r--r-- 1 mccabe ral 9878263 Apr 6 18:36
> > > >> > pbs.2015071512.nc
> > > >> > > > > > > >
> > > >> > > > > > > > -rw-r--r-- 1 mccabe ral 5392527 Apr 6 18:40
> > > >> > pbs.2015071513.nc
> > > >> > > > > > > >
> > > >> > > > > > > > -rw-r--r-- 1 mccabe ral 96 Apr 6 18:40
> > > >> > pbs.2015071514.nc
> > > >> > > > > > > >
> > > >> > > > > > > > -rw-r--r-- 1 mccabe ral 96 Apr 6 18:42
> > > >> > pbs.2015071515.nc
> > > >> > > > > > > >
> > > >> > > > > > > >
> > > >> > > > > > > >
cheyenne3:/glade/scratch/zhuming/metplus4/pb2nc1>l
> > > >> > > > > > > >
> > > >> > > > > > > > total 96112
> > > >> > > > > > > >
> > > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 11550172 Apr 7 15:11
> > > >> > > pbs.2015071500.nc
> > > >> > > > > > > >
> > > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 6615956 Apr 7 15:14
> > > >> > > pbs.2015071501.nc
> > > >> > > > > > > >
> > > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 5984328 Apr 7 15:18
> > > >> > > pbs.2015071502.nc
> > > >> > > > > > > >
> > > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 6980712 Apr 7 15:33
> > > >> > > pbs.2015071506.nc
> > > >> > > > > > > >
> > > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 6455824 Apr 7 15:36
> > > >> > > pbs.2015071507.nc
> > > >> > > > > > > >
> > > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 5749432 Apr 7 15:40
> > > >> > > pbs.2015071508.nc
> > > >> > > > > > > >
> > > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 11284936 Apr 7 15:55
> > > >> > > pbs.2015071512.nc
> > > >> > > > > > > >
> > > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 6600888 Apr 7 15:59
> > > >> > > pbs.2015071513.nc
> > > >> > > > > > > >
> > > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 5868168 Apr 7 16:03
> > > >> > > pbs.2015071514.nc
> > > >> > > > > > > >
> > > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 7273092 Apr 7 16:18
> > > >> > > pbs.2015071518.nc
> > > >> > > > > > > >
> > > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 6656344 Apr 7 16:22
> > > >> > > pbs.2015071519.nc
> > > >> > > > > > > >
> > > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 6109788 Apr 7 16:27
> > > >> > > pbs.2015071520.nc
> > > >> > > > > > > >
> > > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 58808 Apr 7 16:31
> > > >> > > pbs.2015071521.nc
> > > >> > > > > > > >
> > > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 11132440 Apr 7 16:43
> > > >> > > pbs.2015071600.nc
> > > >> > > > > > > >
> > > >> > > > > > > >
> > > >> > > > > > > >
> > > >> > > > > > > > Thanks,
> > > >> > > > > > > >
> > > >> > > > > > > > Zhuming
> > > >> > > > > > > >
> > > >> > > > > > > > On Tue, Apr 6, 2021 at 6:34 PM George McCabe
via RT
> > > >> > > > > > > > <met_help at ucar.edu
> > > >> > > > > > >
> > > >> > > > > > > > wrote:
> > > >> > > > > > > >
> > > >> > > > > > > > > Hi Zhuming,
> > > >> > > > > > > > >
> > > >> > > > > > > > > Thanks for the information. You will need to
use the
> > > >> truncate
> > > >> > > > > > > > > keyword
> > > >> > > > > > > to
> > > >> > > > > > > > > find the files, then set the valid beg/end
range to
> 1
> > > >> hour.
> > > >> > > > > > > > > This is
> > > >> > > > > > > what
> > > >> > > > > > > > I
> > > >> > > > > > > > > changed from your PB2NC1.conf config file:
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > > *PB2NC_VALID_BEGIN =
> > > {valid?fmt=%Y%m%d_%H}PB2NC_VALID_END
> > > >> =
> > > >> > > > > > > > > {valid?fmt=%Y%m%d_%H?shift=1H}*
> > > >> > > > > > > > >
> > > >> > > > > > > > > *PB2NC_INPUT_TEMPLATE =
> > > >> > > > > > > > >
> > > >> > > >
> > >
{valid?fmt=%Y}/prepbufr.gdas.{valid?fmt=%Y%m%d%H?truncate=21600}.nr*
> > > >> > > > > > > > >
> > > >> > > > > > > > > My config file is on cheyenne here:
> > > >> > > > > > > > > /glade/u/home/mccabe/PB2NC1.conf
> > > >> > > > > > > > >
> > > >> > > > > > > > > I just started running it and the output is
being
> > > written
> > > >> > here:
> > > >> > > > > > > > > /glade/u/home/mccabe/data/out/pb2nc1
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > > For PointStat, you will want to loop over the
same
> > > times.
> > > >> I
> > > >> > > > > > > > > noticed
> > > >> > > > > > you
> > > >> > > > > > > > > specify message type in the VAR<n>_OPTIONS,
but you
> do
> > > not
> > > >> > need
> > > >> > > > > > > > > to do
> > > >> > > > > > > > this
> > > >> > > > > > > > > since there is a POINT_STAT_MESSAGE_TYPE
variable
> that
> > > >> > defines
> > > >> > > > > > > > > this.
> > > >> > > > > > > This
> > > >> > > > > > > > > is what I changed in your PointStat1.conf
file:
> > > >> > > > > > > > >
> > > >> > > > > > > > > *VALID_END = 2015071600*
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > > *VALID_INCREMENT = 1HPOINT_STAT_OBS_VALID_BEG
> > > >> > > > > > =POINT_STAT_OBS_VALID_BEG
> > > >> > > > > > > > =*
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > > *POINT_STAT_MESSAGE_TYPE = ADPSFC*
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > > *#OBS_VAR1_OPTIONS = message_type =
> > > >> > > > > > > > > ["ADPSFC"];#OBS_VAR2_OPTIONS =
> > > >> > > > > > > > > message_type = ["ADPSFC"];#OBS_VAR3_OPTIONS =
> > > >> message_type =
> > > >> > > > > > > > > ["ADPSFC"];#OBS_POINT_STAT_INPUT_DIR =
> > > >> > > > > > > > > /glade/scratch/zhuming/metplus4/pb2nc1*
> > > >> > > > > > > > >
> > > >> > > > > > > > > *OBS_POINT_STAT_INPUT_DIR =
{OUTPUT_BASE}/pb2nc1*
> > > >> > > > > > > > >
> > > >> > > > > > > > > Or you can view my file in the same
directory. I
> > usually
> > > >> try
> > > >> > to
> > > >> > > > > > > > > test
> > > >> > > > > > > > with a
> > > >> > > > > > > > > single run time to make sure it is working
properly,
> > so
> > > I
> > > >> set
> > > >> > > > > > VALID_END
> > > >> > > > > > > > > back to 2015071500. I ran it and it produced
an
> output
> > > >> file,
> > > >> > > > > > > > > but
> > > >> > > > > > there
> > > >> > > > > > > > were
> > > >> > > > > > > > > no values besides the header. I checked the
log file
> > and
> > > >> saw
> > > >> > > > > > > > > this:
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > > *DEBUG 2: Processing T_sfc(0,*,*) versus
TMP/Z2, for
> > > >> > > > > > > > > observation type
> > > >> > > > > > > > > ADPSFC, over region FULL, for interpolation
method
> > > >> BILIN(4),
> > > >> > > > > > > > > using 0
> > > >> > > > > > > > > matched pairs.DEBUG 3: Number of matched
pairs =
> > > 0DEBUG
> > > >> 3:
> > > >> > > > > > > Observations
> > > >> > > > > > > > > processed = 393373DEBUG 3: Rejected:
station id
> > > =
> > > >> > > > > > > > > 0DEBUG 3:
> > > >> > > > > > > > > Rejected: obs type = 390528DEBUG 3:
Rejected:
> > > valid
> > > >> > time
> > > >> > > > > > =
> > > >> > > > > > > > > 0DEBUG 3: Rejected: bad obs value = 0DEBUG
3:
> > > Rejected:
> > > >> off
> > > >> > > > > > > > > the
> > > >> > > > > > grid
> > > >> > > > > > > > =
> > > >> > > > > > > > > 2586DEBUG 3: Rejected: topography =
0DEBUG 3:
> > > >> Rejected:
> > > >> > > > > > > > > level
> > > >> > > > > > > > mismatch
> > > >> > > > > > > > > = 259DEBUG 3: Rejected: quality marker =
0DEBUG 3:
> > > >> Rejected:
> > > >> > > > > > message
> > > >> > > > > > > > type
> > > >> > > > > > > > > = 0DEBUG 3: Rejected: masking region =
0DEBUG 3:
> > > >> Rejected:
> > > >> > bad
> > > >> > > > > > fcst
> > > >> > > > > > > > > value = 0DEBUG 3: Rejected: bad climo mean
=
> 0DEBUG
> > 3:
> > > >> > > > > > > > > Rejected:
> > > >> > > > > > bad
> > > >> > > > > > > > > climo stdev = 0DEBUG 3: Rejected: duplicates
=
> > > 0DEBUG
> > > >> > > > > > > > > 2:DEBUG 2:
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > >
> > > >> > > > > > >
> > > >> > > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
>
--------------------------------------------------------------------------------DEBUG
> > > >> > > > > > > > > 2:DEBUG 2: Processing u_10m_gr(0,*,*) versus
> UGRD/Z10,
> > > for
> > > >> > > > > > observation
> > > >> > > > > > > > type
> > > >> > > > > > > > > ADPSFC, over region FULL, for interpolation
method
> > > >> BILIN(4),
> > > >> > > > > > > > > using 0
> > > >> > > > > > > > > matched pairs.DEBUG 3: Number of matched
pairs =
> > > 0DEBUG
> > > >> 3:
> > > >> > > > > > > Observations
> > > >> > > > > > > > > processed = 393373*
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > > *DEBUG 3: Rejected: station id = 0DEBUG
3:
> > > Rejected:
> > > >> obs
> > > >> > > > > > > > > type
> > > >> > > > > > > > =
> > > >> > > > > > > > > 382965DEBUG 3: Rejected: valid time =
0DEBUG 3:
> > > >> > Rejected:
> > > >> > > > > > > > > bad
> > > >> > > > > > obs
> > > >> > > > > > > > > value = 0DEBUG 3: Rejected: off the grid
=
> > > 9240DEBUG
> > > >> 3:
> > > >> > > > > > Rejected:
> > > >> > > > > > > > > topography = 0DEBUG 3: Rejected: level
mismatch
> > =
> > > >> > > > > > > > > 1168DEBUG 3:
> > > >> > > > > > > > > Rejected: quality marker = 0DEBUG 3:
Rejected:
> > message
> > > >> type
> > > >> > > > > > > > > =
> > > >> > > > > > > 0DEBUG
> > > >> > > > > > > > 3:
> > > >> > > > > > > > > Rejected: masking region = 0DEBUG 3:
Rejected: bad
> > fcst
> > > >> > value
> > > >> > > > > > > > > =
> > > >> > > > > > > 0DEBUG
> > > >> > > > > > > > 3:
> > > >> > > > > > > > > Rejected: bad climo mean = 0DEBUG 3:
Rejected: bad
> > > climo
> > > >> > stdev
> > > >> > > > > > > > > =
> > > >> > > > > > > 0DEBUG
> > > >> > > > > > > > 3:
> > > >> > > > > > > > > Rejected: duplicates = 0DEBUG 2:DEBUG 2:
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > >
> > > >> > > > > > >
> > > >> > > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
>
--------------------------------------------------------------------------------DEBUG
> > > >> > > > > > > > > 2:DEBUG 2: Processing v_10m_gr(0,*,*) versus
> VGRD/Z10,
> > > for
> > > >> > > > > > observation
> > > >> > > > > > > > type
> > > >> > > > > > > > > ADPSFC, over region FULL, for interpolation
method
> > > >> BILIN(4),
> > > >> > > > > > > > > using 0
> > > >> > > > > > > > > matched pairs.DEBUG 3: Number of matched
pairs =
> > > 0DEBUG
> > > >> 3:
> > > >> > > > > > > Observations
> > > >> > > > > > > > > processed = 393373DEBUG 3: Rejected:
station id
> > > =
> > > >> > > > > > > > > 0DEBUG 3:
> > > >> > > > > > > > > Rejected: obs type = 382965DEBUG 3:
Rejected:
> > > valid
> > > >> > time
> > > >> > > > > > =
> > > >> > > > > > > > > 0DEBUG 3: Rejected: bad obs value = 0DEBUG
3:
> > > Rejected:
> > > >> off
> > > >> > > > > > > > > the
> > > >> > > > > > grid
> > > >> > > > > > > > =
> > > >> > > > > > > > > 9240DEBUG 3: Rejected: topography =
0DEBUG 3:
> > > >> Rejected:
> > > >> > > > > > > > > level
> > > >> > > > > > > > mismatch
> > > >> > > > > > > > > = 1168DEBUG 3: Rejected: quality marker =
0DEBUG 3:
> > > >> > Rejected:
> > > >> > > > > > message
> > > >> > > > > > > > > type = 0DEBUG 3: Rejected: masking region
=
> 0DEBUG
> > > 3:
> > > >> > > > > > > > > Rejected:
> > > >> > > > > > bad
> > > >> > > > > > > > > fcst value = 0DEBUG 3: Rejected: bad climo
mean =
> > > >> 0DEBUG 3:
> > > >> > > > > > Rejected:
> > > >> > > > > > > > bad
> > > >> > > > > > > > > climo stdev = 0*
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > > *DEBUG 3: Rejected: duplicates = 0DEBUG
2:DEBUG
> > 2:
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > >
> > > >> > > > > > >
> > > >> > > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
>
--------------------------------------------------------------------------------DEBUG
> > > >> > > > > > > > > 2:DEBUG 1: Output file:
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > >
> > > >> > > > > > >
> > > >> > > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
>
/glade/u/home/mccabe/data/out/point_stat1/point_stat_000000L_20150715_000000V.stat*
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > > This describes the reason why each
observation was
> > > >> rejected.
> > > >> > > > > > > > > Some
> > > >> > > > > > > > > observations didn't match the message type
you
> > > requested,
> > > >> > some
> > > >> > > > > > > > > were
> > > >> > > > > > not
> > > >> > > > > > > > > within the forecast grid, and some did not
match the
> > > >> vertical
> > > >> > > > > > > > > level.
> > > >> > > > > > > You
> > > >> > > > > > > > > will have to make adjustments to your
settings so
> that
> > > the
> > > >> > data
> > > >> > > > > > matches
> > > >> > > > > > > > up
> > > >> > > > > > > > > properly. Maybe the grid you are using in the
PB2NC
> > > config
> > > >> > file
> > > >> > > > > > > > > does
> > > >> > > > > > > not
> > > >> > > > > > > > > match the forecast grid and you are missing
values.
> If
> > > >> that
> > > >> > is
> > > >> > > > > > > > > the
> > > >> > > > > > > case,
> > > >> > > > > > > > > you will have to reconfigure and rerun PB2NC
to get
> > > >> output on
> > > >> > > > > > > > > the
> > > >> > > > > > > correct
> > > >> > > > > > > > > gr5id. Maybe the observation points don't
line up
> with
> > > the
> > > >> > > > > > > > > vertical
> > > >> > > > > > > level
> > > >> > > > > > > > > of the model fields. You can use a range of
level
> > values
> > > >> for
> > > >> > > > > > > > > the
> > > >> > > > > > > > > observation data to allow data within that
range,
> i.e.
> > > >> Z0-10.
> > > >> > > > > > > > >
> > > >> > > > > > > > > Let me know if you are still having issues
getting
> > this
> > > to
> > > >> > work
> > > >> > > > > > > > > and I
> > > >> > > > > > > can
> > > >> > > > > > > > > help out.
> > > >> > > > > > > > >
> > > >> > > > > > > > > Thanks,
> > > >> > > > > > > > > George
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > > On Tue, Apr 6, 2021 at 5:17 PM
zhuming at ucar.edu via
> > RT
> > > <
> > > >> > > > > > > > met_help at ucar.edu>
> > > >> > > > > > > > > wrote:
> > > >> > > > > > > > >
> > > >> > > > > > > > > >
> > > >> > > > > > > > > > <URL:
> > > >> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > >
> > > >> > > > > > > > > > Hi George,
> > > >> > > > > > > > > >
> > > >> > > > > > > > > > Thanks for your reply!
> > > >> > > > > > > > > >
> > > >> > > > > > > > > > I'm still confused, I need to output hourly
obs
> data
> > > to
> > > >> do
> > > >> > > > > > > > > > hourly
> > > >> > > > > > > > > > verification.
> > > >> > > > > > > > > > The original GDAS Prepbufr data has 4 files
one
> day,
> > > >> should
> > > >> > > > > > > > > > with 6
> > > >> > > > > > > > hours
> > > >> > > > > > > > > > obs data in one file.
> > > >> > > > > > > > > >
> > > >> > > > > > > > > > After "PB2NC", I can see pbs.2015071500.nc
(obs
> > time
> > > is
> > > >> > 00?
> > > >> > > > > > > > > > or 6
> > > >> > > > > > > times
> > > >> > > > > > > > > of
> > > >> > > > > > > > > > 00,01,02,03,04,05?), pbs.2015071506.nc...
> > > >> > > > > > > > > >
> > > >> > > > > > > > > > my config file :
> > > >> > > > > >
cheyenne:/gpfs/fs1/work/zhuming/METplus4/PB2NC1.conf
> > > >> > > > > > > > > > and
> /gpfs/fs1/work/zhuming/METplus4/PointStat1.conf
> > > >> > > > > > > > > >
> > > >> > > > > > > > > > Thanks,
> > > >> > > > > > > > > > Zhuming
> > > >> > > > > > > > > >
> > > >> > > > > > > > > > On Tue, Apr 6, 2021 at 4:23 PM George
McCabe via
> RT
> > <
> > > >> > > > > > > met_help at ucar.edu
> > > >> > > > > > > > >
> > > >> > > > > > > > > > wrote:
> > > >> > > > > > > > > >
> > > >> > > > > > > > > > > Hi Zhuming,
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > > > To run the tools hourly you would set the
> > > >> VALID_INCREMENT
> > > >> > > > > > > > > > > value
> > > >> > > > > > for
> > > >> > > > > > > > 1H.
> > > >> > > > > > > > > > > Here is an example that runs once per
hour for a
> > day
> > > >> from
> > > >> > > > > > > 2015-07-15
> > > >> > > > > > > > 0Z
> > > >> > > > > > > > > > to
> > > >> > > > > > > > > > > 2017-07-16 0Z:
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > > > VALID_TIME_FMT = %Y%m%d%H
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > > > VALID_BEG = 2015071500
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > > > VALID_END = 2015071600
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > > > VALID_INCREMENT = 1H
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > > > LEAD_SEQ = 0
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > > > Each run time will use the
PB2NC_INPUT_TEMPLATE
> to
> > > >> find
> > > >> > > > > > > > > > > files
> > > >> > > > > > whose
> > > >> > > > > > > > > > > filenames match the time you need.
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > > > To read prepBUFR observations for 12
hours, you
> > will
> > > >> want
> > > >> > > > > > > > > > > to set
> > > >> > > > > > > > > > > PB2NC_VALID_BEGIN and PB2NC_VALID_END to
specify
> > the
> > > >> > range
> > > >> > > > > > > > > > > around
> > > >> > > > > > > the
> > > >> > > > > > > > > > valid
> > > >> > > > > > > > > > > time to process. These values depend on
what is
> > > >> contained
> > > >> > > > > > > > > > > inside
> > > >> > > > > > > your
> > > >> > > > > > > > > > > prepBUFR files. If you want to use data
from the
> > > valid
> > > >> > time
> > > >> > > > > > > > > > > of
> > > >> > > > > > the
> > > >> > > > > > > > > > filename
> > > >> > > > > > > > > > > backwards 12 hours, you would set:
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > > > PB2NC_VALID_BEGIN =
> > {valid?fmt=%Y%m%d_%H?shift=-12H
> > > }
> > > >> > > > > > > > > > > PB2NC_VALID_END = {valid?fmt=%Y%m%d_%H}
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > > > For a runtime of 2015071512, this would
be
> > > substituted
> > > >> > as:
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > > > valid_beg: 20150715_00
> > > >> > > > > > > > > > > valid_end: 20150715_12
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > > > If you want to create an output file for
each
> hour
> > > in
> > > >> > PB2NC
> > > >> > > > > > > > > > > you
> > > >> > > > > > > would
> > > >> > > > > > > > > > need
> > > >> > > > > > > > > > > to make sure that the hour is specified
in the
> > > >> > > > > > > PB2NC_OUTPUT_TEMPLATE.
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > > > PointStat has similar config variables to
set
> the
> > > >> valid
> > > >> > beg
> > > >> > > > > > > > > > > and
> > > >> > > > > > end
> > > >> > > > > > > > > range
> > > >> > > > > > > > > > > to use (POINT_STAT_OBS_VALID_BEG and
> > > >> > > > > > > > > > > POINT_STAT_OBS_VALID_END).
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > > > If your prepBUFR files are separated out
into
> > > multiple
> > > >> > > > > > > > > > > files that
> > > >> > > > > > > > need
> > > >> > > > > > > > > to
> > > >> > > > > > > > > > > be read into PB2NC, you can set a file
window
> > around
> > > >> the
> > > >> > > > > > > > > > > valid
> > > >> > > > > > time
> > > >> > > > > > > > to
> > > >> > > > > > > > > > > allow files to be passed into the app.
More
> > > >> information
> > > >> > on
> > > >> > > > > > > > > > > that
> > > >> > > > > > > > > > > functionality is here:
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > >
> > > >> > > > > > >
> > > >> > > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
>
https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#using-
> > > >> > > > > > windows-to-find-valid-files
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > > > If you are still having trouble getting
this
> > > >> configured,
> > > >> > > > > > > > > > > you
> > > >> > > > > > could
> > > >> > > > > > > > send
> > > >> > > > > > > > > > me
> > > >> > > > > > > > > > > the paths of the files you are trying to
read
> and
> > > what
> > > >> > data
> > > >> > > > > > > > > > > is
> > > >> > > > > > > > > contained
> > > >> > > > > > > > > > in
> > > >> > > > > > > > > > > them so I can give you suggestions on
what to
> set
> > > for
> > > >> the
> > > >> > > > > > > > > configuration.
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > > > Thanks,
> > > >> > > > > > > > > > > George
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > > > On Tue, Apr 6, 2021 at 2:17 PM Julie
Prestopnik
> > via
> > > >> RT <
> > > >> > > > > > > > > > met_help at ucar.edu>
> > > >> > > > > > > > > > > wrote:
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > > > >
> > > >> > > > > > > > > > > > Tue Apr 06 14:16:42 2021: Request 99429
was
> > acted
> > > >> upon.
> > > >> > > > > > > > > > > > Transaction: Given to mccabe (George
McCabe)
> by
> > > >> jpresto
> > > >> > > > > > > > > > > > Queue: met_help
> > > >> > > > > > > > > > > > Subject: How to set time for
hourly 12-24
> > > hours
> > > >> > data
> > > >> > > > > > > > > > > > ?
> > > >> > > > > > > > > > > > Owner: mccabe
> > > >> > > > > > > > > > > > Requestors: zhuming at ucar.edu
> > > >> > > > > > > > > > > > Status: new
> > > >> > > > > > > > > > > > Ticket <URL:
> > > >> > > > > > > > >
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > > > >
> > > >> > > > > > > > > > > >
> > > >> > > > > > > > > > > > This transaction appears to have no
content
> > > >> > > > > > > > > > > >
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > > > --
> > > >> > > > > > > > > > > George McCabe - Software Engineer III
> > > >> > > > > > > > > > > National Center for Atmospheric Research
> > > >> > > > > > > > > > > Research Applications Laboratory
> > > >> > > > > > > > > > > 303-497-2768
> > > >> > > > > > > > > > > ---
> > > >> > > > > > > > > > > My working day may not be your working
day.
> Please
> > > do
> > > >> not
> > > >> > > > > > > > > > > feel
> > > >> > > > > > > > obliged
> > > >> > > > > > > > > to
> > > >> > > > > > > > > > > reply to this email outside of your
normal
> working
> > > >> hours.
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > > >
> > > >> > > > > > > > > >
> > > >> > > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > > > --
> > > >> > > > > > > > > George McCabe - Software Engineer III
> > > >> > > > > > > > > National Center for Atmospheric Research
> > > >> > > > > > > > > Research Applications Laboratory
> > > >> > > > > > > > > 303-497-2768
> > > >> > > > > > > > > ---
> > > >> > > > > > > > > My working day may not be your working day.
Please
> do
> > > not
> > > >> > feel
> > > >> > > > > > obliged
> > > >> > > > > > > to
> > > >> > > > > > > > > reply to this email outside of your normal
working
> > > hours.
> > > >> > > > > > > > >
> > > >> > > > > > > > >
> > > >> > > > > > > >
> > > >> > > > > > > >
> > > >> > > > > > >
> > > >> > > > > > > --
> > > >> > > > > > > George McCabe - Software Engineer III
> > > >> > > > > > > National Center for Atmospheric Research
> > > >> > > > > > > Research Applications Laboratory
> > > >> > > > > > > 303-497-2768
> > > >> > > > > > > ---
> > > >> > > > > > > My working day may not be your working day.
Please do
> not
> > > feel
> > > >> > > > > > > obliged to
> > > >> > > > > > > reply to this email outside of your normal
working
> hours.
> > > >> > > > > > >
> > > >> > > > > > >
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > >
> > > >> > > >
> > > >> > > >
> > > >> > > >
> > > >> > >
> > > >> > >
> > > >> >
> > > >> > --
> > > >> > George McCabe - Software Engineer III
> > > >> > National Center for Atmospheric Research
> > > >> > Research Applications Laboratory
> > > >> > 303-497-2768
> > > >> > ---
> > > >> > My working day may not be your working day. Please do not
feel
> > obliged
> > > >> to
> > > >> > reply to this email outside of your normal working hours.
> > > >> >
> > > >> >
> > > >>
> > > >>
> > > >
> > > > --
> > > > George McCabe - Software Engineer III
> > > > National Center for Atmospheric Research
> > > > Research Applications Laboratory
> > > > 303-497-2768
> > > > ---
> > > > My working day may not be your working day. Please do not feel
> obliged
> > to
> > > > reply to this email outside of your normal working hours.
> > > >
> > >
> > >
> > > --
> > > George McCabe - Software Engineer III
> > > National Center for Atmospheric Research
> > > Research Applications Laboratory
> > > 303-497-2768
> > > ---
> > > My working day may not be your working day. Please do not feel
obliged
> to
> > > reply to this email outside of your normal working hours.
> > >
> > >
> >
> >
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: How to set time for hourly 12-24 hours data ?
From: zhuming at ucar.edu
Time: Wed May 19 14:43:44 2021
Hi George,
I tried running MET new version beta5. It seems something is wrong.
see log file
/glade/scratch/zhuming/metplus4/logs/metplus.log.20210519143436
config file in /glade/work/zhuming/METplus4
Thanks!
Zhuming
(NPL) cheyenne1:/glade/work/zhuming/METplus4>run_metplus.py -c
./PointStat.conf -c ./defaults.conf
Running METplus 4.0.0-beta5
Starting METplus v4.0.0-beta5
05/19 14:34:36.341 metplus (config_metplus.py:117) INFO: Starting
METplus
configuration setup.
05/19 14:34:36.343 metplus (config_metplus.py:247) INFO:
/gpfs/fs1/work/zhuming/METplus4/parm/metplus_config/defaults.conf:
Parsed
this file
05/19 14:34:36.344 metplus (config_metplus.py:247) INFO:
./PointStat.conf:
Parsed this file
05/19 14:34:36.345 metplus (config_metplus.py:247) INFO:
./defaults.conf:
Parsed this file
05/19 14:34:36.346 metplus (config_metplus.py:436) INFO: Adding:
config.LOG_TIMESTAMP='20210519143436'
05/19 14:34:36.346 metplus (config_metplus.py:442) INFO: Replace
[config]
LOG_METPLUS with
'/glade/scratch/zhuming/metplus4/logs/metplus.log.20210519143436'
05/19 14:34:36.348 metplus (config_metplus.py:314) INFO: Setting [dir]
METPLUS_BASE to /glade/p/ral/jntp/MET/METplus/METplus-4.0.0-beta5
05/19 14:34:36.348 metplus (config_metplus.py:314) INFO: Setting [dir]
PARM_BASE to /gpfs/fs1/work/zhuming/METplus4/parm
05/19 14:34:36.348 metplus (config_metplus.py:321) INFO: METPLUS_CONF:
/glade/scratch/zhuming/metplus4/metplus_final.conf written here.
05/19 14:34:36.349 metplus (config_metplus.py:137) INFO: Completed
METplus
configuration setup.
05/19 14:34:36.349 metplus (met_util.py:68) INFO: Running METplus
v4.0.0-beta5 called with command:
/glade/p/ral/jntp/MET/METplus/METplus-4.0.0-beta5/ush/run_metplus.py
-c
./PointStat.conf -c ./defaults.conf
05/19 14:34:36.349 metplus (met_util.py:70) INFO: Log file:
/glade/scratch/zhuming/metplus4/logs/metplus.log.20210519143436
05/19 14:34:36.358 metplus.Usage (met_util.py:186) ERROR: Fatal error
occurred
Traceback (most recent call last):
File
"/glade/p/ral/jntp/MET/METplus/METplus-4.0.0-
beta5/metplus/util/met_util.py",
line 128, in run_metplus
instance=instance)
File
"/glade/p/ral/jntp/MET/METplus/METplus-4.0.0-
beta5/metplus/wrappers/usage_wrapper.py",
line 19, in __init__
self.available_processes = list(set(val for val in
met_util.LOWER_TO_WRAPPER_NAME.values()))
AttributeError: module 'metplus.util.met_util' has no attribute
'LOWER_TO_WRAPPER_NAME'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File
"/glade/p/ral/jntp/MET/METplus/METplus-4.0.0-
beta5/metplus/util/met_util.py",
line 136, in run_metplus
raise NameError("There was a problem loading "
NameError: There was a problem loading Usage wrapper.
05/19 14:34:36Z run-METplus-metplus.Usage: ERROR: Fatal error
occurred
05/19 14:34:36.359 metplus.Usage (met_util.py:187) INFO: Check the log
file
for more information:
/glade/scratch/zhuming/metplus4/logs/metplus.log.20210519143436
05/19 14:34:36.360 metplus (met_util.py:834) INFO:
/glade/scratch/zhuming/metplus4/metplus_final.conf: write metplus.conf
here
05/19 14:34:36.365 metplus (met_util.py:218) ERROR: METplus has
finished
running but had 1 error.
05/19 14:34:36Z run-METplus-metplus: ERROR: METplus has finished
running
but had 1 error.
05/19 14:34:36.365 metplus (met_util.py:219) INFO: Check the log file
for
more information:
/glade/scratch/zhuming/metplus4/logs/metplus.log.20210519143436
(NPL) cheyenne1:/glade/work/zhuming/METplus4>
On Mon, May 3, 2021 at 3:47 PM George McCabe via RT
<met_help at ucar.edu>
wrote:
> Hi Zhuming,
>
> The METplus 4.0.0-beta5 release is now available on Cheyenne. If you
are
> able to test that this version works and fixes some of the issues
you
> mentioned in this ticket before the 4.0.0 release next week, that
would be
> great! Information on how to use the new version can be found here:
>
> https://dtcenter.org/community-code/metplus/metplus-4-0-existing-
builds
> (Under NCAR > CHEYENNE > METplus-4.0.0-beta5)
>
> Here are the commands to set it up:
>
> module use /glade/p/ral/jntp/MET/METplus/modulefiles
> module load metplus/4.0.0-beta5
> ncar_pylib
>
> Please let me know if you have any issues or questions.
>
> Thanks,
> George
>
> On Thu, Apr 22, 2021 at 11:40 AM John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429 >
> >
> > Zhuming,
> >
> > This is John HG. George asked me to take a look at this question.
Since
> > you're asking about "non-standard" levels, I assume you're asking
about
> the
> > verification of the pressure levels against the ADPUPA message
type.
> >
> > In general, the logic is this...
> > For each point observation, loop through all the verification
tasks and
> see
> > if it should be used.
> > For example, let's talk about the 4 level entry for VAR3:
> >
> > FCST_VAR3_NAME = v_tr_p
> > FCST_VAR3_LEVELS = ...(0,3,*,*)...
> > OBS_VAR3_NAME = VGRD
> > OBS_VAR3_LEVELS = ...P575-625...
> >
> > The point observation will be used here if it's a "VGRD"
observation and
> > has a level value between 575 and 625. This verification task has
been
> > configured to only verify a single forecast level of data:
> v_tr_p(0,3,*,*).
> > So no vertical interpolation is done. MET interpolates
horizontally from
> > the forecast grid points to the observation lat/lon location
(using
> > user-configurable interpolation options).
> >
> > To put it simply, all obs between 575 and 625 mb are compared
directly to
> > the forecast values from v_tr_p(0,3,*,*). If the model output is
actually
> > on pressure levels, then that's a reasonable thing to do. If the
model's
> > vertical coordinate is something other than pressure levels, then
that's
> a
> > potential source for large errors.
> >
> > Note that MET does do vertical interpolation when processing the
GRIB
> > output from the Unified Post Processor tool. But all the details
of that
> > depend on how the configuration file is setup. I'll explain below.
> >
> > To date we have avoided including direct knowledge about the
vertical
> > coordinate of each modelling system in the verification code. The
goal is
> > to create verification tools applicable to many models, rather
than
> > customizing them for each. We do realize that desire for the
latter and
> are
> > exploring ways to pull data from the IODA data assimilation system
> instead,
> > which necessarily knows a lot about each model.
> >
> > So let's say you were using GRIB data, and configured it like
this:
> >
> > VAR3_NAME = VGRD
> > VAR3_LEVELS = P500-1000
> >
> > Since we haven't prefixed these options with FCST_ and OBS_, the
same
> > settings are applied to both.
> >
> > MET would read ALL forecast GRIB records from the model file with
> pressures
> > between 500 and 1000. Perhaps, that's every 100mb or perhaps every
50mb?
> It
> > reads whatever data falls in that range. For each point
observation with
> a
> > pressure level between 500 and 1000, it'll first find the actual
model
> > pressure level above/below it. It'll interpolate the forecast
> > data horizontally to the obs lat/lon location for both. And then
it'll
> > interpolate those values above/below vertically to the actual
observation
> > level, linear in the log of pressure.
> >
> > This issue is that the processing of pressure LAYERS is not
currently
> > supported for NetCDF data I believe. And the code could be
enhanced to
> > support that. But again, this only makes sense if the model's
vertical
> > coordinate is actually pressure levels and not something else.
> >
> > Hope that helps clarify.
> >
> > Thanks,
> > John
> >
> > On Thu, Apr 22, 2021 at 9:52 AM zhuming at ucar.edu via RT <
> met_help at ucar.edu
> > >
> > wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429 >
> > >
> > > Hi George,
> > >
> > > My verification result shows that there is a big error on a
> non-standard
> > > level, so my question is:
> > > Is the forecast grid match to Obs, or obs station match to
Forecast? I
> > need
> > > to do is the forecast grid interpolated to the obs location and
> vertical
> > > levels. So how to set the levels?
> > >
> > > Thanks,
> > > Zhuming
> > >
> > > FCST_VAR3_NAME = v_10m_tr
> > >
> > > FCST_VAR3_LEVELS = (0,*,*)
> > >
> > > FCST_VAR3_THRESH = >=5, <5
> > >
> > > OBS_VAR3_NAME = VGRD
> > >
> > > OBS_VAR3_LEVELS = Z10
> > >
> > > OBS_VAR3_THRESH = >=5, <5
> > >
> > > OBS_VAR3_OPTIONS = message_type = ["ADPSFC"];
> > >
> > >
> > > FCST_VAR3_NAME = v_tr_p
> > >
> > > FCST_VAR3_LEVELS =
> > >
> > >
> >
>
(0,0,*,*),(0,1,*,*),(0,2,*,*),(0,3,*,*),(0,4,*,*),(0,5,*,*),(0,6,*,*),(0,7,*,*),(0,8,*,*),(0,9,*,*)
> > >
> > > FCST_VAR3_THRESH = >=5, <5
> > >
> > > OBS_VAR3_NAME = VGRD
> > >
> > > OBS_VAR3_LEVELS =
> > >
> > >
> >
> P900-950,P825-875,P675-725,P575-625,P475-525,P375-425,P275-325,P225-
275,P175-225,P75-125
> > >
> > > OBS_VAR3_THRESH = >=5, <5
> > >
> > > OBS_VAR3_OPTIONS = message_type = ["ADPUPA"];
> > >
> > >
> > >
> > > On Thu, Apr 15, 2021 at 1:52 PM George McCabe via RT <
> met_help at ucar.edu>
> > > wrote:
> > >
> > > > Hi Zhuming,
> > > >
> > > > There was indeed a bug in the code. Howard has fixed the issue
with
> > this
> > > > pull request: https://github.com/dtcenter/MET/pull/1756
> > > > This fix will be included in the MET 10.0.0-beta5 release that
is
> > > > tentatively scheduled for next week. This beta version can be
> installed
> > > on
> > > > cheyenne so you can test that the update fixes your issue. I
will
> > follow
> > > up
> > > > and let you know when the beta release is available on
cheyenne.
> > > >
> > > > Thanks,
> > > > George
> > > >
> > > > On Wed, Apr 14, 2021 at 5:28 PM George McCabe
<mccabe at ucar.edu>
> wrote:
> > > >
> > > > > Hi Zhuming,
> > > > >
> > > > > I tried to get this to work on cheyenne with your data, but
was not
> > > > > successful. I am able to read the data with plot_data_plane
and
> > > generate
> > > > an
> > > > > image, so I think there may be a bug in the code. I have
forwarded
> > info
> > > > to
> > > > > one of our C++ developers to look into it. I will report
back when
> I
> > > know
> > > > > more.
> > > > >
> > > > > Thanks,
> > > > > George
> > > > >
> > > > > On Wed, Apr 14, 2021 at 4:13 PM zhuming at ucar.edu via RT <
> > > > met_help at ucar.edu>
> > > > > wrote:
> > > > >
> > > > >>
> > > > >> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429 >
> > > > >>
> > > > >> Hi George,
> > > > >>
> > > > >> It still can't find the correct time. I tried
> > > > >>
> > > > >> FCST_VAR1_LEVELS = (*,*,*)
> > > > >> FCST_VAR1_LEVELS = ({valid?fmt=%Y%m%d_%H%M%S},*,*)
> > > > >> FCST_VAR1_LEVELS = ({valid?fmt=%Y%m%d_%H%M},*,*)
> > > > >> FCST_VAR1_LEVELS = ({valid?fmt=%Y%m%d%H%M},*,*)
> > > > >>
> > > > >> Thanks!
> > > > >>
> > > > >> Zhuming
> > > > >>
> > > > >> On Wed, Apr 14, 2021 at 3:41 PM George McCabe via RT <
> > > met_help at ucar.edu
> > > > >
> > > > >> wrote:
> > > > >>
> > > > >> > Hi Zhuming,
> > > > >> >
> > > > >> > It looks like you are requesting the first time in the 12
hour
> > file
> > > > for
> > > > >> > each run:
> > > > >> >
> > > > >> > FCST_VAR1_LEVELS = (0,*,*)
> > > > >> > ...
> > > > >> > FCST_VAR2_LEVELS = (0,*,*)
> > > > >> >
> > > > >> > The first dimension in these files is the time offset.
You will
> > need
> > > > the
> > > > >> > correct index that corresponds to each valid time.
Fortunately,
> > MET
> > > > >> makes
> > > > >> > this easy by allowing you to specify the valid time for a
time
> > > > dimension
> > > > >> > explicitly and it will find the correct data. You can
set:
> > > > >> >
> > > > >> > FCST_VAR1_LEVELS = ({valid?fmt=%Y%m%d_%H%M%S},*,*)
> > > > >> > ...
> > > > >> > FCST_VAR2_LEVELS = ({valid?fmt=%Y%m%d_%H%M%S},*,*)
> > > > >> >
> > > > >> > And it will substitute the valid time for each run. Let
me know
> if
> > > you
> > > > >> have
> > > > >> > any questions.
> > > > >> >
> > > > >> > Thanks,
> > > > >> > George
> > > > >> >
> > > > >> > On Wed, Apr 14, 2021 at 3:18 PM zhuming at ucar.edu via RT <
> > > > >> met_help at ucar.edu
> > > > >> > >
> > > > >> > wrote:
> > > > >> >
> > > > >> > >
> > > > >> > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429
> >
> > > > >> > >
> > > > >> > > Hi George,
> > > > >> > >
> > > > >> > > Thanks for your email, I just would like to send you
for help.
> > > > >> > >
> > > > >> > > I already resolved the last problem with the WRF
forecast,
> thank
> > > you
> > > > >> for
> > > > >> > > your help.
> > > > >> > >
> > > > >> > > I also need to verify the MPAS model forecast. These 12
hour
> > > > >> > > MAPS forecasts are stored in a one file, such as
> > > > >> > > 2015071500/mpas_wrf_cf_2015071500.nc, has 12h forecasts
> > > > >> > > 2015071512/mpas_wrf_cf_2015071512.nc
> > > > >> > >
> > > > >> > > It seems MET is doing fine on the first 2 hours, but no
> matched
> > > > pairs
> > > > >> for
> > > > >> > > later hours.
> > > > >> > > I'm not sure what's wrong with it? the time is not
matched?
> > > > >> > >
> > > > >> > > see the log file
> > > > >> > >
> > > >
>
/glade/scratch/zhuming/metplus4/logs/master_metplus.log.20210414145101
> > > > >> > > config file
/gpfs/fs1/work/zhuming/METplus4/PointStat2.conf
> > > > >> > >
> > > > >> > > Thanks,
> > > > >> > > Zhuming
> > > > >> > >
> > > > >> > > On Wed, Apr 14, 2021 at 1:56 PM George McCabe via RT <
> > > > >> met_help at ucar.edu>
> > > > >> > > wrote:
> > > > >> > >
> > > > >> > > > Hi Zhuming,
> > > > >> > > >
> > > > >> > > > Just following up on this ticket. Were you able to
get this
> > > > >> resolved?
> > > > >> > Let
> > > > >> > > > me know if you need more assistance.
> > > > >> > > >
> > > > >> > > > Thanks,
> > > > >> > > > George
> > > > >> > > >
> > > > >> > > > On Thu Apr 08 10:04:44 2021, mccabe wrote:
> > > > >> > > > > Hi Zhuming,
> > > > >> > > > >
> > > > >> > > > > Typically directories that contain data for an
entire day
> > are
> > > > >> named
> > > > >> > > > > with
> > > > >> > > > > YYYYMMDD, so your filename template tag would be
> > > > >> {valid?fmt=%Y%m%d},
> > > > >> > > > > however that may not always be the case, as you are
seeing
> > > here.
> > > > >> > There
> > > > >> > > > > are
> > > > >> > > > > a couple ways to handle this.
> > > > >> > > > >
> > > > >> > > > > If you know that the directory will always be
YYYYMMDD
> with
> > 00
> > > > at
> > > > >> the
> > > > >> > > > > end,
> > > > >> > > > > you can simply put 00 in the template instead of
adding %H
> > in
> > > > the
> > > > >> > tag,
> > > > >> > > > > i.e.
> > > > >> > > > >
> > > > >> > > > > FCST_POINT_STAT_INPUT_TEMPLATE =
> > > > >> > > > >
> {valid?fmt=%Y%m%d}00/wrfout_d02_{valid?fmt=%Y-%m-%d_%H}:00:
> > > > >> 00_cf.nc
> > > > >> > > > >
> > > > >> > > > > If the HH value can vary, for instance if there are
2
> > > > directories
> > > > >> > > > > created
> > > > >> > > > > per day (i.e. 2015071400 and 2015071412), then you
can
> use
> > > the
> > > > >> > > > > truncate
> > > > >> > > > > keyword to handle it.
> > > > >> > > > >
> > > > >> > > > > If directories are created every 24 hours:
> > > > >> > > > >
> > > > >> > > > > FCST_POINT_STAT_INPUT_TEMPLATE =
> > > > >> > > > >
> > {valid?fmt=%Y%m%d%H?truncate=24H}/wrfout_d02_{valid?fmt=%Y-%m-
> > > > >> > > > > %d_%H}:00:
> > > > >> > > > > 00_cf.nc
> > > > >> > > > >
> > > > >> > > > > If directories are created every 12 hours:
> > > > >> > > > >
> > > > >> > > > > FCST_POINT_STAT_INPUT_TEMPLATE =
> > > > >> > > > >
> > {valid?fmt=%Y%m%d%H?truncate=12H}/wrfout_d02_{valid?fmt=%Y-%m-
> > > > >> > > > > %d_%H}:00:
> > > > >> > > > > 00_cf.nc
> > > > >> > > > >
> > > > >> > > > > Let me know if that doesn't work for you.
> > > > >> > > > >
> > > > >> > > > > Thanks,
> > > > >> > > > > George
> > > > >> > > > >
> > > > >> > > > > On Thu, Apr 8, 2021 at 9:46 AM zhuming at ucar.edu via
RT
> > > > >> > > > > <met_help at ucar.edu>
> > > > >> > > > > wrote:
> > > > >> > > > >
> > > > >> > > > > >
> > > > >> > > > > > <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429
> > > > >
> > > > >> > > > > >
> > > > >> > > > > > Hi George,
> > > > >> > > > > >
> > > > >> > > > > > Processing Obs data using "PB2NC" are right now.
> > > > >> > > > > >
> > > > >> > > > > > Now I'm using "PointStat" , I set VALID_BEG =
> 2015071500,
> > > > >> > VALID_END =
> > > > >> > > > > > 2015071501. It looks for the forecast data
2015071500/
> > > > >> > > > > > wrfout_d02_2015-07-15_00:00:00_cf.nc, and
2015071501/
> > > > >> > > > > > wrfout_d02_2015-07-15_01:00:00_cf.nc.
> > > > >> > > > > > But my forecast data are all put in dir of
2015071500/.
> > > > >> > > > > > So how could I set
FCST_POINT_STAT_INPUT_TEMPLATE to
> get
> > > the
> > > > >> right
> > > > >> > > > > > fcst
> > > > >> > > > > > data?
> > > > >> > > > > >
> > > > >> > > > > > FCST_POINT_STAT_INPUT_TEMPLATE =
> > > > >> > > > > >
> > {valid?fmt=%Y%m%d%H}/wrfout_d02_{valid?fmt=%Y-%m-%d_%H}:00:
> > > > >> > 00_cf.nc
> > > > >> > > > > >
> > > > >> > > > > >
> > > > >> > > > > > config file:
> > > /gpfs/fs1/work/zhuming/METplus4/PointStat1.conf
> > > > >> > > > > >
> > > > >> > > > > > log file:
> > > > >> > > > > >
> > > > >> > >
> > > >
>
/glade/scratch/zhuming/metplus4/logs/master_metplus.log.20210408085450
> > > > >> > > > > >
> > > > >> > > > > >
> > > > >> > > > > > Thanks,
> > > > >> > > > > >
> > > > >> > > > > > Zhuming
> > > > >> > > > > >
> > > > >> > > > > > On Wed, Apr 7, 2021 at 5:43 PM George McCabe via
RT
> > > > >> > > > > > <met_help at ucar.edu>
> > > > >> > > > > > wrote:
> > > > >> > > > > >
> > > > >> > > > > > > Hi Zhuming,
> > > > >> > > > > > >
> > > > >> > > > > > > Looking at the log output, it looks like the
data is
> > > stored
> > > > in
> > > > >> > the
> > > > >> > > > > > prepBUFR
> > > > >> > > > > > > files differently that you may have thought:
> > > > >> > > > > > >
> > > > >> > > > > > > DEBUG 2: PrepBufr Time Center:
> 20150715_000000
> > > > >> > > > > > > DEBUG 2: Searching Time Window:
> 20150715_030000
> > to
> > > > >> > > > > > 20150715_040000
> > > > >> > > > > > > 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60%
65% 70%
> > 75%
> > > > 80%
> > > > >> > 85%
> > > > >> > > > > > > 90%
> > > > >> > > > > > 95%
> > > > >> > > > > > > 100%
> > > > >> > > > > > >
> > > > >> > > > > > > DEBUG 2: Total Messages processed
=
> 590836
> > > > >> > > > > > > DEBUG 2: Rejected based on message type
= 0
> > > > >> > > > > > > DEBUG 2: Rejected based on station id
= 0
> > > > >> > > > > > > DEBUG 2: Rejected based on valid time
=
> 590815
> > > > >> > > > > > > DEBUG 2: Rejected based on masking grid
= 20
> > > > >> > > > > > > DEBUG 2: Rejected based on masking polygon
= 0
> > > > >> > > > > > > DEBUG 2: Rejected based on elevation
= 0
> > > > >> > > > > > > DEBUG 2: Rejected based on pb report type
= 1
> > > > >> > > > > > > DEBUG 2: Rejected based on input report type
= 0
> > > > >> > > > > > > DEBUG 2: Rejected based on instrument type
= 0
> > > > >> > > > > > > DEBUG 2: Rejected based on zero observations
= 0
> > > > >> > > > > > > DEBUG 2: Total Messages retained
= 0
> > > > >> > > > > > > DEBUG 2: Total observations retained or derived
= 0
> > > > >> > > > > > > DEBUG 1: Obs time between *20150714_210000 and
> > > > >> 20150715_030000*
> > > > >> > > > > > > WARNING:
> > > > >> > > > > > > WARNING: process_pbfile() -> No PrepBufr
messages
> > retained
> > > > >> from
> > > > >> > > > > > > file:
> > > > >> > > > > > >
/glade/collections/rda/data/ds337.0/prepnr/2015/
> > > > >> > > > > > > prepbufr.gdas.2015071500.nr
> > > > >> > > > > > > WARNING:
> > > > >> > > > > > > ERROR :
> > > > >> > > > > > > ERROR : write_netcdf_hdr_data() -> No PrepBufr
> messages
> > > > >> > retained.
> > > > >> > > > > > Nothing
> > > > >> > > > > > > to write.
> > > > >> > > > > > > ERROR :
> > > > >> > > > > > >
> > > > >> > > > > > > The prepbufr.gdas.2015071500.nr file actually
> contains
> > > data
> > > > >> from
> > > > >> > > > > > > 20150714_210000 to 20150715_030000, not
> 20150715_000000
> > to
> > > > >> > > > > > 20150715_060000.
> > > > >> > > > > > >
> > > > >> > > > > > > To get the correct file for each run, you will
have to
> > > shift
> > > > >> the
> > > > >> > > > > > > valid
> > > > >> > > > > > time
> > > > >> > > > > > > before truncating it so that it gets the
correct file
> > for
> > > > each
> > > > >> > run.
> > > > >> > > > > > Instead
> > > > >> > > > > > > of setting:
> > > > >> > > > > > >
> > > > >> > > > > > > PB2NC_INPUT_TEMPLATE =
> > > > >> > > > > > >
> > > > >> >
> >
{valid?fmt=%Y}/prepbufr.gdas.{valid?fmt=%Y%m%d%H?truncate=21600}.nr
> > > > >> > > > > > >
> > > > >> > > > > > > You will have to also shift forward 3 hours,
i.e.:
> > > > >> > > > > > >
> > > > >> > > > > > > PB2NC_INPUT_TEMPLATE =
> > > > >> > > > > > >
> > > > >> > > > > > >
> > > > >> > > > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
>
{valid?fmt=%Y}/prepbufr.gdas.{valid?fmt=%Y%m%d%H?shift=3H?truncate=21600}.nr
> > > > >> > > > > > >
> > > > >> > > > > > > This says to add 3 hours to the current valid
time
> > before
> > > > >> > > > > > > truncating to
> > > > >> > > > > > the
> > > > >> > > > > > > nearest 6 hours.
> > > > >> > > > > > >
> > > > >> > > > > > > Valid: 20150715 0Z + 3 = 3Z truncate 6H = 0Z
> > > > >> > > > > > > File: prepbufr.gdas.2015071500.nr
> > > > >> > > > > > >
> > > > >> > > > > > > Valid: 20150715 1Z + 3 = 4Z truncate 6H = 0Z
> > > > >> > > > > > > File: prepbufr.gdas.2015071500.nr
> > > > >> > > > > > >
> > > > >> > > > > > > Valid: 20150715 2Z + 3 = 5Z truncate 6H = 0Z
> > > > >> > > > > > > File: prepbufr.gdas.2015071500.nr
> > > > >> > > > > > >
> > > > >> > > > > > > Valid: 20150715 3Z + 3 = 6Z truncate 6H = 6Z
> > > > >> > > > > > > File: prepbufr.gdas.2015071506.nr
> > > > >> > > > > > >
> > > > >> > > > > > > Let me know if that doesn't work.
> > > > >> > > > > > >
> > > > >> > > > > > > Thanks,
> > > > >> > > > > > > George
> > > > >> > > > > > >
> > > > >> > > > > > >
> > > > >> > > > > > > On Wed, Apr 7, 2021 at 5:32 PM zhuming at ucar.edu
via
> RT
> > <
> > > > >> > > > > > met_help at ucar.edu>
> > > > >> > > > > > > wrote:
> > > > >> > > > > > >
> > > > >> > > > > > > >
> > > > >> > > > > > > > <URL:
> > > > >> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429
> > > > >> > >
> > > > >> > > > > > > >
> > > > >> > > > > > > > Hi George,
> > > > >> > > > > > > >
> > > > >> > > > > > > > Now makes me confused is why processed obs of
> several
> > > > times
> > > > >> are
> > > > >> > > > > > missing.
> > > > >> > > > > > > > There has files of time at 00,01,02, but no
> 03,04,05;
> > > has
> > > > >> > > > > > > > 06,07,08, but
> > > > >> > > > > > > no
> > > > >> > > > > > > > 09,10,11, has 12,13,14, no 15,16,17; has
18,19,20,
> > but
> > > > with
> > > > >> > > > > > > > 21,no
> > > > >> > > > > > 22,23.
> > > > >> > > > > > > > From yours and mines:
> > > > >> > > > > > > >
> > > > >> > > > > > > > ls -l /glade/u/home/mccabe/data/out/pb2nc1/
> > > > >> > > > > > > >
> > > > >> > > > > > > >
> > > > >> > > > > > > > -rw-r--r-- 1 mccabe ral 10141339 Apr 6 17:52
> > > > >> > pbs.2015071500.nc
> > > > >> > > > > > > >
> > > > >> > > > > > > > -rw-r--r-- 1 mccabe ral 5413907 Apr 6 17:56
> > > > >> > pbs.2015071501.nc
> > > > >> > > > > > > >
> > > > >> > > > > > > > -rw-r--r-- 1 mccabe ral 4923711 Apr 6 18:00
> > > > >> > pbs.2015071502.nc
> > > > >> > > > > > > >
> > > > >> > > > > > > > -rw-r--r-- 1 mccabe ral 5724447 Apr 6 18:14
> > > > >> > pbs.2015071506.nc
> > > > >> > > > > > > >
> > > > >> > > > > > > > -rw-r--r-- 1 mccabe ral 5280463 Apr 6 18:18
> > > > >> > pbs.2015071507.nc
> > > > >> > > > > > > >
> > > > >> > > > > > > > -rw-r--r-- 1 mccabe ral 4706119 Apr 6 18:22
> > > > >> > pbs.2015071508.nc
> > > > >> > > > > > > >
> > > > >> > > > > > > > -rw-r--r-- 1 mccabe ral 9878263 Apr 6 18:36
> > > > >> > pbs.2015071512.nc
> > > > >> > > > > > > >
> > > > >> > > > > > > > -rw-r--r-- 1 mccabe ral 5392527 Apr 6 18:40
> > > > >> > pbs.2015071513.nc
> > > > >> > > > > > > >
> > > > >> > > > > > > > -rw-r--r-- 1 mccabe ral 96 Apr 6 18:40
> > > > >> > pbs.2015071514.nc
> > > > >> > > > > > > >
> > > > >> > > > > > > > -rw-r--r-- 1 mccabe ral 96 Apr 6 18:42
> > > > >> > pbs.2015071515.nc
> > > > >> > > > > > > >
> > > > >> > > > > > > >
> > > > >> > > > > > > >
cheyenne3:/glade/scratch/zhuming/metplus4/pb2nc1>l
> > > > >> > > > > > > >
> > > > >> > > > > > > > total 96112
> > > > >> > > > > > > >
> > > > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 11550172 Apr 7
15:11
> > > > >> > > pbs.2015071500.nc
> > > > >> > > > > > > >
> > > > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 6615956 Apr 7
15:14
> > > > >> > > pbs.2015071501.nc
> > > > >> > > > > > > >
> > > > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 5984328 Apr 7
15:18
> > > > >> > > pbs.2015071502.nc
> > > > >> > > > > > > >
> > > > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 6980712 Apr 7
15:33
> > > > >> > > pbs.2015071506.nc
> > > > >> > > > > > > >
> > > > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 6455824 Apr 7
15:36
> > > > >> > > pbs.2015071507.nc
> > > > >> > > > > > > >
> > > > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 5749432 Apr 7
15:40
> > > > >> > > pbs.2015071508.nc
> > > > >> > > > > > > >
> > > > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 11284936 Apr 7
15:55
> > > > >> > > pbs.2015071512.nc
> > > > >> > > > > > > >
> > > > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 6600888 Apr 7
15:59
> > > > >> > > pbs.2015071513.nc
> > > > >> > > > > > > >
> > > > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 5868168 Apr 7
16:03
> > > > >> > > pbs.2015071514.nc
> > > > >> > > > > > > >
> > > > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 7273092 Apr 7
16:18
> > > > >> > > pbs.2015071518.nc
> > > > >> > > > > > > >
> > > > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 6656344 Apr 7
16:22
> > > > >> > > pbs.2015071519.nc
> > > > >> > > > > > > >
> > > > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 6109788 Apr 7
16:27
> > > > >> > > pbs.2015071520.nc
> > > > >> > > > > > > >
> > > > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 58808 Apr 7
16:31
> > > > >> > > pbs.2015071521.nc
> > > > >> > > > > > > >
> > > > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 11132440 Apr 7
16:43
> > > > >> > > pbs.2015071600.nc
> > > > >> > > > > > > >
> > > > >> > > > > > > >
> > > > >> > > > > > > >
> > > > >> > > > > > > > Thanks,
> > > > >> > > > > > > >
> > > > >> > > > > > > > Zhuming
> > > > >> > > > > > > >
> > > > >> > > > > > > > On Tue, Apr 6, 2021 at 6:34 PM George McCabe
via RT
> > > > >> > > > > > > > <met_help at ucar.edu
> > > > >> > > > > > >
> > > > >> > > > > > > > wrote:
> > > > >> > > > > > > >
> > > > >> > > > > > > > > Hi Zhuming,
> > > > >> > > > > > > > >
> > > > >> > > > > > > > > Thanks for the information. You will need
to use
> the
> > > > >> truncate
> > > > >> > > > > > > > > keyword
> > > > >> > > > > > > to
> > > > >> > > > > > > > > find the files, then set the valid beg/end
range
> to
> > 1
> > > > >> hour.
> > > > >> > > > > > > > > This is
> > > > >> > > > > > > what
> > > > >> > > > > > > > I
> > > > >> > > > > > > > > changed from your PB2NC1.conf config file:
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > > *PB2NC_VALID_BEGIN =
> > > > {valid?fmt=%Y%m%d_%H}PB2NC_VALID_END
> > > > >> =
> > > > >> > > > > > > > > {valid?fmt=%Y%m%d_%H?shift=1H}*
> > > > >> > > > > > > > >
> > > > >> > > > > > > > > *PB2NC_INPUT_TEMPLATE =
> > > > >> > > > > > > > >
> > > > >> > > >
> > > >
{valid?fmt=%Y}/prepbufr.gdas.{valid?fmt=%Y%m%d%H?truncate=21600}.nr*
> > > > >> > > > > > > > >
> > > > >> > > > > > > > > My config file is on cheyenne here:
> > > > >> > > > > > > > > /glade/u/home/mccabe/PB2NC1.conf
> > > > >> > > > > > > > >
> > > > >> > > > > > > > > I just started running it and the output is
being
> > > > written
> > > > >> > here:
> > > > >> > > > > > > > > /glade/u/home/mccabe/data/out/pb2nc1
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > > For PointStat, you will want to loop over
the same
> > > > times.
> > > > >> I
> > > > >> > > > > > > > > noticed
> > > > >> > > > > > you
> > > > >> > > > > > > > > specify message type in the VAR<n>_OPTIONS,
but
> you
> > do
> > > > not
> > > > >> > need
> > > > >> > > > > > > > > to do
> > > > >> > > > > > > > this
> > > > >> > > > > > > > > since there is a POINT_STAT_MESSAGE_TYPE
variable
> > that
> > > > >> > defines
> > > > >> > > > > > > > > this.
> > > > >> > > > > > > This
> > > > >> > > > > > > > > is what I changed in your PointStat1.conf
file:
> > > > >> > > > > > > > >
> > > > >> > > > > > > > > *VALID_END = 2015071600*
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > > *VALID_INCREMENT =
1HPOINT_STAT_OBS_VALID_BEG
> > > > >> > > > > > =POINT_STAT_OBS_VALID_BEG
> > > > >> > > > > > > > =*
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > > *POINT_STAT_MESSAGE_TYPE = ADPSFC*
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > > *#OBS_VAR1_OPTIONS = message_type =
> > > > >> > > > > > > > > ["ADPSFC"];#OBS_VAR2_OPTIONS =
> > > > >> > > > > > > > > message_type = ["ADPSFC"];#OBS_VAR3_OPTIONS
=
> > > > >> message_type =
> > > > >> > > > > > > > > ["ADPSFC"];#OBS_POINT_STAT_INPUT_DIR =
> > > > >> > > > > > > > > /glade/scratch/zhuming/metplus4/pb2nc1*
> > > > >> > > > > > > > >
> > > > >> > > > > > > > > *OBS_POINT_STAT_INPUT_DIR =
{OUTPUT_BASE}/pb2nc1*
> > > > >> > > > > > > > >
> > > > >> > > > > > > > > Or you can view my file in the same
directory. I
> > > usually
> > > > >> try
> > > > >> > to
> > > > >> > > > > > > > > test
> > > > >> > > > > > > > with a
> > > > >> > > > > > > > > single run time to make sure it is working
> properly,
> > > so
> > > > I
> > > > >> set
> > > > >> > > > > > VALID_END
> > > > >> > > > > > > > > back to 2015071500. I ran it and it
produced an
> > output
> > > > >> file,
> > > > >> > > > > > > > > but
> > > > >> > > > > > there
> > > > >> > > > > > > > were
> > > > >> > > > > > > > > no values besides the header. I checked the
log
> file
> > > and
> > > > >> saw
> > > > >> > > > > > > > > this:
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > > *DEBUG 2: Processing T_sfc(0,*,*) versus
TMP/Z2,
> for
> > > > >> > > > > > > > > observation type
> > > > >> > > > > > > > > ADPSFC, over region FULL, for interpolation
method
> > > > >> BILIN(4),
> > > > >> > > > > > > > > using 0
> > > > >> > > > > > > > > matched pairs.DEBUG 3: Number of matched
pairs =
> > > > 0DEBUG
> > > > >> 3:
> > > > >> > > > > > > Observations
> > > > >> > > > > > > > > processed = 393373DEBUG 3: Rejected:
station id
> > > > =
> > > > >> > > > > > > > > 0DEBUG 3:
> > > > >> > > > > > > > > Rejected: obs type = 390528DEBUG 3:
> Rejected:
> > > > valid
> > > > >> > time
> > > > >> > > > > > =
> > > > >> > > > > > > > > 0DEBUG 3: Rejected: bad obs value =
0DEBUG 3:
> > > > Rejected:
> > > > >> off
> > > > >> > > > > > > > > the
> > > > >> > > > > > grid
> > > > >> > > > > > > > =
> > > > >> > > > > > > > > 2586DEBUG 3: Rejected: topography =
0DEBUG 3:
> > > > >> Rejected:
> > > > >> > > > > > > > > level
> > > > >> > > > > > > > mismatch
> > > > >> > > > > > > > > = 259DEBUG 3: Rejected: quality marker =
0DEBUG
> 3:
> > > > >> Rejected:
> > > > >> > > > > > message
> > > > >> > > > > > > > type
> > > > >> > > > > > > > > = 0DEBUG 3: Rejected: masking region =
0DEBUG 3:
> > > > >> Rejected:
> > > > >> > bad
> > > > >> > > > > > fcst
> > > > >> > > > > > > > > value = 0DEBUG 3: Rejected: bad climo mean
=
> > 0DEBUG
> > > 3:
> > > > >> > > > > > > > > Rejected:
> > > > >> > > > > > bad
> > > > >> > > > > > > > > climo stdev = 0DEBUG 3: Rejected:
duplicates
> =
> > > > 0DEBUG
> > > > >> > > > > > > > > 2:DEBUG 2:
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > >
> > > > >> > > > > > >
> > > > >> > > > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
>
--------------------------------------------------------------------------------DEBUG
> > > > >> > > > > > > > > 2:DEBUG 2: Processing u_10m_gr(0,*,*)
versus
> > UGRD/Z10,
> > > > for
> > > > >> > > > > > observation
> > > > >> > > > > > > > type
> > > > >> > > > > > > > > ADPSFC, over region FULL, for interpolation
method
> > > > >> BILIN(4),
> > > > >> > > > > > > > > using 0
> > > > >> > > > > > > > > matched pairs.DEBUG 3: Number of matched
pairs =
> > > > 0DEBUG
> > > > >> 3:
> > > > >> > > > > > > Observations
> > > > >> > > > > > > > > processed = 393373*
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > > *DEBUG 3: Rejected: station id =
0DEBUG 3:
> > > > Rejected:
> > > > >> obs
> > > > >> > > > > > > > > type
> > > > >> > > > > > > > =
> > > > >> > > > > > > > > 382965DEBUG 3: Rejected: valid time =
0DEBUG
> 3:
> > > > >> > Rejected:
> > > > >> > > > > > > > > bad
> > > > >> > > > > > obs
> > > > >> > > > > > > > > value = 0DEBUG 3: Rejected: off the grid
=
> > > > 9240DEBUG
> > > > >> 3:
> > > > >> > > > > > Rejected:
> > > > >> > > > > > > > > topography = 0DEBUG 3: Rejected: level
> mismatch
> > > =
> > > > >> > > > > > > > > 1168DEBUG 3:
> > > > >> > > > > > > > > Rejected: quality marker = 0DEBUG 3:
Rejected:
> > > message
> > > > >> type
> > > > >> > > > > > > > > =
> > > > >> > > > > > > 0DEBUG
> > > > >> > > > > > > > 3:
> > > > >> > > > > > > > > Rejected: masking region = 0DEBUG 3:
Rejected:
> bad
> > > fcst
> > > > >> > value
> > > > >> > > > > > > > > =
> > > > >> > > > > > > 0DEBUG
> > > > >> > > > > > > > 3:
> > > > >> > > > > > > > > Rejected: bad climo mean = 0DEBUG 3:
Rejected:
> bad
> > > > climo
> > > > >> > stdev
> > > > >> > > > > > > > > =
> > > > >> > > > > > > 0DEBUG
> > > > >> > > > > > > > 3:
> > > > >> > > > > > > > > Rejected: duplicates = 0DEBUG 2:DEBUG
2:
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > >
> > > > >> > > > > > >
> > > > >> > > > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
>
--------------------------------------------------------------------------------DEBUG
> > > > >> > > > > > > > > 2:DEBUG 2: Processing v_10m_gr(0,*,*)
versus
> > VGRD/Z10,
> > > > for
> > > > >> > > > > > observation
> > > > >> > > > > > > > type
> > > > >> > > > > > > > > ADPSFC, over region FULL, for interpolation
method
> > > > >> BILIN(4),
> > > > >> > > > > > > > > using 0
> > > > >> > > > > > > > > matched pairs.DEBUG 3: Number of matched
pairs =
> > > > 0DEBUG
> > > > >> 3:
> > > > >> > > > > > > Observations
> > > > >> > > > > > > > > processed = 393373DEBUG 3: Rejected:
station id
> > > > =
> > > > >> > > > > > > > > 0DEBUG 3:
> > > > >> > > > > > > > > Rejected: obs type = 382965DEBUG 3:
> Rejected:
> > > > valid
> > > > >> > time
> > > > >> > > > > > =
> > > > >> > > > > > > > > 0DEBUG 3: Rejected: bad obs value =
0DEBUG 3:
> > > > Rejected:
> > > > >> off
> > > > >> > > > > > > > > the
> > > > >> > > > > > grid
> > > > >> > > > > > > > =
> > > > >> > > > > > > > > 9240DEBUG 3: Rejected: topography =
0DEBUG 3:
> > > > >> Rejected:
> > > > >> > > > > > > > > level
> > > > >> > > > > > > > mismatch
> > > > >> > > > > > > > > = 1168DEBUG 3: Rejected: quality marker =
0DEBUG
> 3:
> > > > >> > Rejected:
> > > > >> > > > > > message
> > > > >> > > > > > > > > type = 0DEBUG 3: Rejected: masking
region =
> > 0DEBUG
> > > > 3:
> > > > >> > > > > > > > > Rejected:
> > > > >> > > > > > bad
> > > > >> > > > > > > > > fcst value = 0DEBUG 3: Rejected: bad climo
mean
> =
> > > > >> 0DEBUG 3:
> > > > >> > > > > > Rejected:
> > > > >> > > > > > > > bad
> > > > >> > > > > > > > > climo stdev = 0*
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > > *DEBUG 3: Rejected: duplicates =
0DEBUG
> 2:DEBUG
> > > 2:
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > >
> > > > >> > > > > > >
> > > > >> > > > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
>
--------------------------------------------------------------------------------DEBUG
> > > > >> > > > > > > > > 2:DEBUG 1: Output file:
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > >
> > > > >> > > > > > >
> > > > >> > > > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
>
/glade/u/home/mccabe/data/out/point_stat1/point_stat_000000L_20150715_000000V.stat*
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > > This describes the reason why each
observation was
> > > > >> rejected.
> > > > >> > > > > > > > > Some
> > > > >> > > > > > > > > observations didn't match the message type
you
> > > > requested,
> > > > >> > some
> > > > >> > > > > > > > > were
> > > > >> > > > > > not
> > > > >> > > > > > > > > within the forecast grid, and some did not
match
> the
> > > > >> vertical
> > > > >> > > > > > > > > level.
> > > > >> > > > > > > You
> > > > >> > > > > > > > > will have to make adjustments to your
settings so
> > that
> > > > the
> > > > >> > data
> > > > >> > > > > > matches
> > > > >> > > > > > > > up
> > > > >> > > > > > > > > properly. Maybe the grid you are using in
the
> PB2NC
> > > > config
> > > > >> > file
> > > > >> > > > > > > > > does
> > > > >> > > > > > > not
> > > > >> > > > > > > > > match the forecast grid and you are missing
> values.
> > If
> > > > >> that
> > > > >> > is
> > > > >> > > > > > > > > the
> > > > >> > > > > > > case,
> > > > >> > > > > > > > > you will have to reconfigure and rerun
PB2NC to
> get
> > > > >> output on
> > > > >> > > > > > > > > the
> > > > >> > > > > > > correct
> > > > >> > > > > > > > > gr5id. Maybe the observation points don't
line up
> > with
> > > > the
> > > > >> > > > > > > > > vertical
> > > > >> > > > > > > level
> > > > >> > > > > > > > > of the model fields. You can use a range of
level
> > > values
> > > > >> for
> > > > >> > > > > > > > > the
> > > > >> > > > > > > > > observation data to allow data within that
range,
> > i.e.
> > > > >> Z0-10.
> > > > >> > > > > > > > >
> > > > >> > > > > > > > > Let me know if you are still having issues
getting
> > > this
> > > > to
> > > > >> > work
> > > > >> > > > > > > > > and I
> > > > >> > > > > > > can
> > > > >> > > > > > > > > help out.
> > > > >> > > > > > > > >
> > > > >> > > > > > > > > Thanks,
> > > > >> > > > > > > > > George
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > > On Tue, Apr 6, 2021 at 5:17 PM
zhuming at ucar.edu
> via
> > > RT
> > > > <
> > > > >> > > > > > > > met_help at ucar.edu>
> > > > >> > > > > > > > > wrote:
> > > > >> > > > > > > > >
> > > > >> > > > > > > > > >
> > > > >> > > > > > > > > > <URL:
> > > > >> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429
> > > > >> > > > > > > > > > >
> > > > >> > > > > > > > > >
> > > > >> > > > > > > > > > Hi George,
> > > > >> > > > > > > > > >
> > > > >> > > > > > > > > > Thanks for your reply!
> > > > >> > > > > > > > > >
> > > > >> > > > > > > > > > I'm still confused, I need to output
hourly obs
> > data
> > > > to
> > > > >> do
> > > > >> > > > > > > > > > hourly
> > > > >> > > > > > > > > > verification.
> > > > >> > > > > > > > > > The original GDAS Prepbufr data has 4
files one
> > day,
> > > > >> should
> > > > >> > > > > > > > > > with 6
> > > > >> > > > > > > > hours
> > > > >> > > > > > > > > > obs data in one file.
> > > > >> > > > > > > > > >
> > > > >> > > > > > > > > > After "PB2NC", I can see
pbs.2015071500.nc (obs
> > > time
> > > > is
> > > > >> > 00?
> > > > >> > > > > > > > > > or 6
> > > > >> > > > > > > times
> > > > >> > > > > > > > > of
> > > > >> > > > > > > > > > 00,01,02,03,04,05?), pbs.2015071506.nc...
> > > > >> > > > > > > > > >
> > > > >> > > > > > > > > > my config file :
> > > > >> > > > > >
cheyenne:/gpfs/fs1/work/zhuming/METplus4/PB2NC1.conf
> > > > >> > > > > > > > > > and
> > /gpfs/fs1/work/zhuming/METplus4/PointStat1.conf
> > > > >> > > > > > > > > >
> > > > >> > > > > > > > > > Thanks,
> > > > >> > > > > > > > > > Zhuming
> > > > >> > > > > > > > > >
> > > > >> > > > > > > > > > On Tue, Apr 6, 2021 at 4:23 PM George
McCabe via
> > RT
> > > <
> > > > >> > > > > > > met_help at ucar.edu
> > > > >> > > > > > > > >
> > > > >> > > > > > > > > > wrote:
> > > > >> > > > > > > > > >
> > > > >> > > > > > > > > > > Hi Zhuming,
> > > > >> > > > > > > > > > >
> > > > >> > > > > > > > > > > To run the tools hourly you would set
the
> > > > >> VALID_INCREMENT
> > > > >> > > > > > > > > > > value
> > > > >> > > > > > for
> > > > >> > > > > > > > 1H.
> > > > >> > > > > > > > > > > Here is an example that runs once per
hour
> for a
> > > day
> > > > >> from
> > > > >> > > > > > > 2015-07-15
> > > > >> > > > > > > > 0Z
> > > > >> > > > > > > > > > to
> > > > >> > > > > > > > > > > 2017-07-16 0Z:
> > > > >> > > > > > > > > > >
> > > > >> > > > > > > > > > > VALID_TIME_FMT = %Y%m%d%H
> > > > >> > > > > > > > > > >
> > > > >> > > > > > > > > > > VALID_BEG = 2015071500
> > > > >> > > > > > > > > > >
> > > > >> > > > > > > > > > > VALID_END = 2015071600
> > > > >> > > > > > > > > > >
> > > > >> > > > > > > > > > > VALID_INCREMENT = 1H
> > > > >> > > > > > > > > > >
> > > > >> > > > > > > > > > > LEAD_SEQ = 0
> > > > >> > > > > > > > > > >
> > > > >> > > > > > > > > > > Each run time will use the
> PB2NC_INPUT_TEMPLATE
> > to
> > > > >> find
> > > > >> > > > > > > > > > > files
> > > > >> > > > > > whose
> > > > >> > > > > > > > > > > filenames match the time you need.
> > > > >> > > > > > > > > > >
> > > > >> > > > > > > > > > > To read prepBUFR observations for 12
hours,
> you
> > > will
> > > > >> want
> > > > >> > > > > > > > > > > to set
> > > > >> > > > > > > > > > > PB2NC_VALID_BEGIN and PB2NC_VALID_END
to
> specify
> > > the
> > > > >> > range
> > > > >> > > > > > > > > > > around
> > > > >> > > > > > > the
> > > > >> > > > > > > > > > valid
> > > > >> > > > > > > > > > > time to process. These values depend on
what
> is
> > > > >> contained
> > > > >> > > > > > > > > > > inside
> > > > >> > > > > > > your
> > > > >> > > > > > > > > > > prepBUFR files. If you want to use data
from
> the
> > > > valid
> > > > >> > time
> > > > >> > > > > > > > > > > of
> > > > >> > > > > > the
> > > > >> > > > > > > > > > filename
> > > > >> > > > > > > > > > > backwards 12 hours, you would set:
> > > > >> > > > > > > > > > >
> > > > >> > > > > > > > > > > PB2NC_VALID_BEGIN =
> > > {valid?fmt=%Y%m%d_%H?shift=-12H
> > > > }
> > > > >> > > > > > > > > > > PB2NC_VALID_END = {valid?fmt=%Y%m%d_%H}
> > > > >> > > > > > > > > > >
> > > > >> > > > > > > > > > > For a runtime of 2015071512, this would
be
> > > > substituted
> > > > >> > as:
> > > > >> > > > > > > > > > >
> > > > >> > > > > > > > > > > valid_beg: 20150715_00
> > > > >> > > > > > > > > > > valid_end: 20150715_12
> > > > >> > > > > > > > > > >
> > > > >> > > > > > > > > > > If you want to create an output file
for each
> > hour
> > > > in
> > > > >> > PB2NC
> > > > >> > > > > > > > > > > you
> > > > >> > > > > > > would
> > > > >> > > > > > > > > > need
> > > > >> > > > > > > > > > > to make sure that the hour is specified
in the
> > > > >> > > > > > > PB2NC_OUTPUT_TEMPLATE.
> > > > >> > > > > > > > > > >
> > > > >> > > > > > > > > > > PointStat has similar config variables
to set
> > the
> > > > >> valid
> > > > >> > beg
> > > > >> > > > > > > > > > > and
> > > > >> > > > > > end
> > > > >> > > > > > > > > range
> > > > >> > > > > > > > > > > to use (POINT_STAT_OBS_VALID_BEG and
> > > > >> > > > > > > > > > > POINT_STAT_OBS_VALID_END).
> > > > >> > > > > > > > > > >
> > > > >> > > > > > > > > > > If your prepBUFR files are separated
out into
> > > > multiple
> > > > >> > > > > > > > > > > files that
> > > > >> > > > > > > > need
> > > > >> > > > > > > > > to
> > > > >> > > > > > > > > > > be read into PB2NC, you can set a file
window
> > > around
> > > > >> the
> > > > >> > > > > > > > > > > valid
> > > > >> > > > > > time
> > > > >> > > > > > > > to
> > > > >> > > > > > > > > > > allow files to be passed into the app.
More
> > > > >> information
> > > > >> > on
> > > > >> > > > > > > > > > > that
> > > > >> > > > > > > > > > > functionality is here:
> > > > >> > > > > > > > > > >
> > > > >> > > > > > > > > > >
> > > > >> > > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > >
> > > > >> > > > > > >
> > > > >> > > > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> >
>
https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#using-
> > > > >> > > > > > windows-to-find-valid-files
> > > > >> > > > > > > > > > >
> > > > >> > > > > > > > > > > If you are still having trouble getting
this
> > > > >> configured,
> > > > >> > > > > > > > > > > you
> > > > >> > > > > > could
> > > > >> > > > > > > > send
> > > > >> > > > > > > > > > me
> > > > >> > > > > > > > > > > the paths of the files you are trying
to read
> > and
> > > > what
> > > > >> > data
> > > > >> > > > > > > > > > > is
> > > > >> > > > > > > > > contained
> > > > >> > > > > > > > > > in
> > > > >> > > > > > > > > > > them so I can give you suggestions on
what to
> > set
> > > > for
> > > > >> the
> > > > >> > > > > > > > > configuration.
> > > > >> > > > > > > > > > >
> > > > >> > > > > > > > > > > Thanks,
> > > > >> > > > > > > > > > > George
> > > > >> > > > > > > > > > >
> > > > >> > > > > > > > > > >
> > > > >> > > > > > > > > > >
> > > > >> > > > > > > > > > > On Tue, Apr 6, 2021 at 2:17 PM Julie
> Prestopnik
> > > via
> > > > >> RT <
> > > > >> > > > > > > > > > met_help at ucar.edu>
> > > > >> > > > > > > > > > > wrote:
> > > > >> > > > > > > > > > >
> > > > >> > > > > > > > > > > >
> > > > >> > > > > > > > > > > > Tue Apr 06 14:16:42 2021: Request
99429 was
> > > acted
> > > > >> upon.
> > > > >> > > > > > > > > > > > Transaction: Given to mccabe (George
McCabe)
> > by
> > > > >> jpresto
> > > > >> > > > > > > > > > > > Queue: met_help
> > > > >> > > > > > > > > > > > Subject: How to set time for
hourly
> 12-24
> > > > hours
> > > > >> > data
> > > > >> > > > > > > > > > > > ?
> > > > >> > > > > > > > > > > > Owner: mccabe
> > > > >> > > > > > > > > > > > Requestors: zhuming at ucar.edu
> > > > >> > > > > > > > > > > > Status: new
> > > > >> > > > > > > > > > > > Ticket <URL:
> > > > >> > > > > > > > >
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429
> > > > >> > > > > > > > > > >
> > > > >> > > > > > > > > > > >
> > > > >> > > > > > > > > > > >
> > > > >> > > > > > > > > > > > This transaction appears to have no
content
> > > > >> > > > > > > > > > > >
> > > > >> > > > > > > > > > >
> > > > >> > > > > > > > > > >
> > > > >> > > > > > > > > > > --
> > > > >> > > > > > > > > > > George McCabe - Software Engineer III
> > > > >> > > > > > > > > > > National Center for Atmospheric
Research
> > > > >> > > > > > > > > > > Research Applications Laboratory
> > > > >> > > > > > > > > > > 303-497-2768
> > > > >> > > > > > > > > > > ---
> > > > >> > > > > > > > > > > My working day may not be your working
day.
> > Please
> > > > do
> > > > >> not
> > > > >> > > > > > > > > > > feel
> > > > >> > > > > > > > obliged
> > > > >> > > > > > > > > to
> > > > >> > > > > > > > > > > reply to this email outside of your
normal
> > working
> > > > >> hours.
> > > > >> > > > > > > > > > >
> > > > >> > > > > > > > > > >
> > > > >> > > > > > > > > >
> > > > >> > > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > > > --
> > > > >> > > > > > > > > George McCabe - Software Engineer III
> > > > >> > > > > > > > > National Center for Atmospheric Research
> > > > >> > > > > > > > > Research Applications Laboratory
> > > > >> > > > > > > > > 303-497-2768
> > > > >> > > > > > > > > ---
> > > > >> > > > > > > > > My working day may not be your working day.
Please
> > do
> > > > not
> > > > >> > feel
> > > > >> > > > > > obliged
> > > > >> > > > > > > to
> > > > >> > > > > > > > > reply to this email outside of your normal
working
> > > > hours.
> > > > >> > > > > > > > >
> > > > >> > > > > > > > >
> > > > >> > > > > > > >
> > > > >> > > > > > > >
> > > > >> > > > > > >
> > > > >> > > > > > > --
> > > > >> > > > > > > George McCabe - Software Engineer III
> > > > >> > > > > > > National Center for Atmospheric Research
> > > > >> > > > > > > Research Applications Laboratory
> > > > >> > > > > > > 303-497-2768
> > > > >> > > > > > > ---
> > > > >> > > > > > > My working day may not be your working day.
Please do
> > not
> > > > feel
> > > > >> > > > > > > obliged to
> > > > >> > > > > > > reply to this email outside of your normal
working
> > hours.
> > > > >> > > > > > >
> > > > >> > > > > > >
> > > > >> > > > > >
> > > > >> > > > > >
> > > > >> > > >
> > > > >> > > >
> > > > >> > > >
> > > > >> > > >
> > > > >> > >
> > > > >> > >
> > > > >> >
> > > > >> > --
> > > > >> > George McCabe - Software Engineer III
> > > > >> > National Center for Atmospheric Research
> > > > >> > Research Applications Laboratory
> > > > >> > 303-497-2768
> > > > >> > ---
> > > > >> > My working day may not be your working day. Please do not
feel
> > > obliged
> > > > >> to
> > > > >> > reply to this email outside of your normal working hours.
> > > > >> >
> > > > >> >
> > > > >>
> > > > >>
> > > > >
> > > > > --
> > > > > George McCabe - Software Engineer III
> > > > > National Center for Atmospheric Research
> > > > > Research Applications Laboratory
> > > > > 303-497-2768
> > > > > ---
> > > > > My working day may not be your working day. Please do not
feel
> > obliged
> > > to
> > > > > reply to this email outside of your normal working hours.
> > > > >
> > > >
> > > >
> > > > --
> > > > George McCabe - Software Engineer III
> > > > National Center for Atmospheric Research
> > > > Research Applications Laboratory
> > > > 303-497-2768
> > > > ---
> > > > My working day may not be your working day. Please do not feel
> obliged
> > to
> > > > reply to this email outside of your normal working hours.
> > > >
> > > >
> > >
> > >
> >
> >
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>
------------------------------------------------
Subject: How to set time for hourly 12-24 hours data ?
From: George McCabe
Time: Wed May 19 15:03:32 2021
Hi Zhuming,
There are a few things going on here:
1) There is a bug in the Usage wrapper that just prints the list of
possible wrappers. Thank you for finding this. I will fix this for the
next
development release.
2) The reason you are getting the Usage wrapper instead of running
PointStat is you are passing in the defaults.conf file last.
PointStat.conf has:
PROCESS_LIST = PointStat
and defaults.conf has:
PROCESS_LIST = Usage
Since defaults.conf was read after PointStat.conf, the value in
defaults.conf is used. The defaults.conf file in the METplus directory
(under parm/metplus_config) is actually read first automatically
before
reading any of the config files you pass into the command, so you do
not
ever need to include it in the command.
3) You no longer need to put '-c' before your config files on the
command
line. It doesn't hurt to have them, but they are ignored.
run_metplus.py ./PointStat.conf
4) The official 4.0.0 release is now available on cheyenne, so you can
use
that instead. You can set it up by running the following:
module use /glade/p/ral/jntp/MET/METplus/modulefiles
module load metplus/4.0.0
ncar_pylib
Let me know if you have any other issues.
Thanks,
George
On Wed, May 19, 2021 at 2:44 PM zhuming at ucar.edu via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429 >
>
> Hi George,
>
> I tried running MET new version beta5. It seems something is wrong.
> see log file
> /glade/scratch/zhuming/metplus4/logs/metplus.log.20210519143436
> config file in /glade/work/zhuming/METplus4
>
> Thanks!
> Zhuming
>
>
> (NPL) cheyenne1:/glade/work/zhuming/METplus4>run_metplus.py -c
> ./PointStat.conf -c ./defaults.conf
>
> Running METplus 4.0.0-beta5
>
> Starting METplus v4.0.0-beta5
>
> 05/19 14:34:36.341 metplus (config_metplus.py:117) INFO: Starting
METplus
> configuration setup.
>
> 05/19 14:34:36.343 metplus (config_metplus.py:247) INFO:
> /gpfs/fs1/work/zhuming/METplus4/parm/metplus_config/defaults.conf:
Parsed
> this file
>
> 05/19 14:34:36.344 metplus (config_metplus.py:247) INFO:
./PointStat.conf:
> Parsed this file
>
> 05/19 14:34:36.345 metplus (config_metplus.py:247) INFO:
./defaults.conf:
> Parsed this file
>
> 05/19 14:34:36.346 metplus (config_metplus.py:436) INFO: Adding:
> config.LOG_TIMESTAMP='20210519143436'
>
> 05/19 14:34:36.346 metplus (config_metplus.py:442) INFO: Replace
[config]
> LOG_METPLUS with
> '/glade/scratch/zhuming/metplus4/logs/metplus.log.20210519143436'
>
> 05/19 14:34:36.348 metplus (config_metplus.py:314) INFO: Setting
[dir]
> METPLUS_BASE to /glade/p/ral/jntp/MET/METplus/METplus-4.0.0-beta5
>
> 05/19 14:34:36.348 metplus (config_metplus.py:314) INFO: Setting
[dir]
> PARM_BASE to /gpfs/fs1/work/zhuming/METplus4/parm
>
> 05/19 14:34:36.348 metplus (config_metplus.py:321) INFO:
METPLUS_CONF:
> /glade/scratch/zhuming/metplus4/metplus_final.conf written here.
>
> 05/19 14:34:36.349 metplus (config_metplus.py:137) INFO: Completed
METplus
> configuration setup.
>
> 05/19 14:34:36.349 metplus (met_util.py:68) INFO: Running METplus
> v4.0.0-beta5 called with command:
> /glade/p/ral/jntp/MET/METplus/METplus-4.0.0-beta5/ush/run_metplus.py
-c
> ./PointStat.conf -c ./defaults.conf
>
> 05/19 14:34:36.349 metplus (met_util.py:70) INFO: Log file:
> /glade/scratch/zhuming/metplus4/logs/metplus.log.20210519143436
>
> 05/19 14:34:36.358 metplus.Usage (met_util.py:186) ERROR: Fatal
error
> occurred
>
> Traceback (most recent call last):
>
> File
>
> "/glade/p/ral/jntp/MET/METplus/METplus-4.0.0-
beta5/metplus/util/met_util.py",
> line 128, in run_metplus
>
> instance=instance)
>
> File
>
> "/glade/p/ral/jntp/MET/METplus/METplus-4.0.0-
beta5/metplus/wrappers/usage_wrapper.py",
> line 19, in __init__
>
> self.available_processes = list(set(val for val in
> met_util.LOWER_TO_WRAPPER_NAME.values()))
>
> AttributeError: module 'metplus.util.met_util' has no attribute
> 'LOWER_TO_WRAPPER_NAME'
>
>
> During handling of the above exception, another exception occurred:
>
>
> Traceback (most recent call last):
>
> File
>
> "/glade/p/ral/jntp/MET/METplus/METplus-4.0.0-
beta5/metplus/util/met_util.py",
> line 136, in run_metplus
>
> raise NameError("There was a problem loading "
>
> NameError: There was a problem loading Usage wrapper.
>
> 05/19 14:34:36Z run-METplus-metplus.Usage: ERROR: Fatal error
occurred
>
> 05/19 14:34:36.359 metplus.Usage (met_util.py:187) INFO: Check the
log file
> for more information:
> /glade/scratch/zhuming/metplus4/logs/metplus.log.20210519143436
>
> 05/19 14:34:36.360 metplus (met_util.py:834) INFO:
> /glade/scratch/zhuming/metplus4/metplus_final.conf: write
metplus.conf here
>
> 05/19 14:34:36.365 metplus (met_util.py:218) ERROR: METplus has
finished
> running but had 1 error.
>
> 05/19 14:34:36Z run-METplus-metplus: ERROR: METplus has finished
running
> but had 1 error.
>
> 05/19 14:34:36.365 metplus (met_util.py:219) INFO: Check the log
file for
> more information:
> /glade/scratch/zhuming/metplus4/logs/metplus.log.20210519143436
>
> (NPL) cheyenne1:/glade/work/zhuming/METplus4>
>
> On Mon, May 3, 2021 at 3:47 PM George McCabe via RT
<met_help at ucar.edu>
> wrote:
>
> > Hi Zhuming,
> >
> > The METplus 4.0.0-beta5 release is now available on Cheyenne. If
you are
> > able to test that this version works and fixes some of the issues
you
> > mentioned in this ticket before the 4.0.0 release next week, that
would
> be
> > great! Information on how to use the new version can be found
here:
> >
> > https://dtcenter.org/community-code/metplus/metplus-4-0-existing-
builds
> > (Under NCAR > CHEYENNE > METplus-4.0.0-beta5)
> >
> > Here are the commands to set it up:
> >
> > module use /glade/p/ral/jntp/MET/METplus/modulefiles
> > module load metplus/4.0.0-beta5
> > ncar_pylib
> >
> > Please let me know if you have any issues or questions.
> >
> > Thanks,
> > George
> >
> > On Thu, Apr 22, 2021 at 11:40 AM John Halley Gotway via RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429 >
> > >
> > > Zhuming,
> > >
> > > This is John HG. George asked me to take a look at this
question. Since
> > > you're asking about "non-standard" levels, I assume you're
asking about
> > the
> > > verification of the pressure levels against the ADPUPA message
type.
> > >
> > > In general, the logic is this...
> > > For each point observation, loop through all the verification
tasks and
> > see
> > > if it should be used.
> > > For example, let's talk about the 4 level entry for VAR3:
> > >
> > > FCST_VAR3_NAME = v_tr_p
> > > FCST_VAR3_LEVELS = ...(0,3,*,*)...
> > > OBS_VAR3_NAME = VGRD
> > > OBS_VAR3_LEVELS = ...P575-625...
> > >
> > > The point observation will be used here if it's a "VGRD"
observation
> and
> > > has a level value between 575 and 625. This verification task
has been
> > > configured to only verify a single forecast level of data:
> > v_tr_p(0,3,*,*).
> > > So no vertical interpolation is done. MET interpolates
horizontally
> from
> > > the forecast grid points to the observation lat/lon location
(using
> > > user-configurable interpolation options).
> > >
> > > To put it simply, all obs between 575 and 625 mb are compared
directly
> to
> > > the forecast values from v_tr_p(0,3,*,*). If the model output is
> actually
> > > on pressure levels, then that's a reasonable thing to do. If the
> model's
> > > vertical coordinate is something other than pressure levels,
then
> that's
> > a
> > > potential source for large errors.
> > >
> > > Note that MET does do vertical interpolation when processing the
GRIB
> > > output from the Unified Post Processor tool. But all the details
of
> that
> > > depend on how the configuration file is setup. I'll explain
below.
> > >
> > > To date we have avoided including direct knowledge about the
vertical
> > > coordinate of each modelling system in the verification code.
The goal
> is
> > > to create verification tools applicable to many models, rather
than
> > > customizing them for each. We do realize that desire for the
latter and
> > are
> > > exploring ways to pull data from the IODA data assimilation
system
> > instead,
> > > which necessarily knows a lot about each model.
> > >
> > > So let's say you were using GRIB data, and configured it like
this:
> > >
> > > VAR3_NAME = VGRD
> > > VAR3_LEVELS = P500-1000
> > >
> > > Since we haven't prefixed these options with FCST_ and OBS_, the
same
> > > settings are applied to both.
> > >
> > > MET would read ALL forecast GRIB records from the model file
with
> > pressures
> > > between 500 and 1000. Perhaps, that's every 100mb or perhaps
every
> 50mb?
> > It
> > > reads whatever data falls in that range. For each point
observation
> with
> > a
> > > pressure level between 500 and 1000, it'll first find the actual
model
> > > pressure level above/below it. It'll interpolate the forecast
> > > data horizontally to the obs lat/lon location for both. And then
it'll
> > > interpolate those values above/below vertically to the actual
> observation
> > > level, linear in the log of pressure.
> > >
> > > This issue is that the processing of pressure LAYERS is not
currently
> > > supported for NetCDF data I believe. And the code could be
enhanced to
> > > support that. But again, this only makes sense if the model's
vertical
> > > coordinate is actually pressure levels and not something else.
> > >
> > > Hope that helps clarify.
> > >
> > > Thanks,
> > > John
> > >
> > > On Thu, Apr 22, 2021 at 9:52 AM zhuming at ucar.edu via RT <
> > met_help at ucar.edu
> > > >
> > > wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429
>
> > > >
> > > > Hi George,
> > > >
> > > > My verification result shows that there is a big error on a
> > non-standard
> > > > level, so my question is:
> > > > Is the forecast grid match to Obs, or obs station match to
Forecast?
> I
> > > need
> > > > to do is the forecast grid interpolated to the obs location
and
> > vertical
> > > > levels. So how to set the levels?
> > > >
> > > > Thanks,
> > > > Zhuming
> > > >
> > > > FCST_VAR3_NAME = v_10m_tr
> > > >
> > > > FCST_VAR3_LEVELS = (0,*,*)
> > > >
> > > > FCST_VAR3_THRESH = >=5, <5
> > > >
> > > > OBS_VAR3_NAME = VGRD
> > > >
> > > > OBS_VAR3_LEVELS = Z10
> > > >
> > > > OBS_VAR3_THRESH = >=5, <5
> > > >
> > > > OBS_VAR3_OPTIONS = message_type = ["ADPSFC"];
> > > >
> > > >
> > > > FCST_VAR3_NAME = v_tr_p
> > > >
> > > > FCST_VAR3_LEVELS =
> > > >
> > > >
> > >
> >
>
(0,0,*,*),(0,1,*,*),(0,2,*,*),(0,3,*,*),(0,4,*,*),(0,5,*,*),(0,6,*,*),(0,7,*,*),(0,8,*,*),(0,9,*,*)
> > > >
> > > > FCST_VAR3_THRESH = >=5, <5
> > > >
> > > > OBS_VAR3_NAME = VGRD
> > > >
> > > > OBS_VAR3_LEVELS =
> > > >
> > > >
> > >
> >
> P900-950,P825-875,P675-725,P575-625,P475-525,P375-425,P275-325,P225-
275,P175-225,P75-125
> > > >
> > > > OBS_VAR3_THRESH = >=5, <5
> > > >
> > > > OBS_VAR3_OPTIONS = message_type = ["ADPUPA"];
> > > >
> > > >
> > > >
> > > > On Thu, Apr 15, 2021 at 1:52 PM George McCabe via RT <
> > met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > > Hi Zhuming,
> > > > >
> > > > > There was indeed a bug in the code. Howard has fixed the
issue with
> > > this
> > > > > pull request: https://github.com/dtcenter/MET/pull/1756
> > > > > This fix will be included in the MET 10.0.0-beta5 release
that is
> > > > > tentatively scheduled for next week. This beta version can
be
> > installed
> > > > on
> > > > > cheyenne so you can test that the update fixes your issue. I
will
> > > follow
> > > > up
> > > > > and let you know when the beta release is available on
cheyenne.
> > > > >
> > > > > Thanks,
> > > > > George
> > > > >
> > > > > On Wed, Apr 14, 2021 at 5:28 PM George McCabe
<mccabe at ucar.edu>
> > wrote:
> > > > >
> > > > > > Hi Zhuming,
> > > > > >
> > > > > > I tried to get this to work on cheyenne with your data,
but was
> not
> > > > > > successful. I am able to read the data with
plot_data_plane and
> > > > generate
> > > > > an
> > > > > > image, so I think there may be a bug in the code. I have
> forwarded
> > > info
> > > > > to
> > > > > > one of our C++ developers to look into it. I will report
back
> when
> > I
> > > > know
> > > > > > more.
> > > > > >
> > > > > > Thanks,
> > > > > > George
> > > > > >
> > > > > > On Wed, Apr 14, 2021 at 4:13 PM zhuming at ucar.edu via RT <
> > > > > met_help at ucar.edu>
> > > > > > wrote:
> > > > > >
> > > > > >>
> > > > > >> <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429 >
> > > > > >>
> > > > > >> Hi George,
> > > > > >>
> > > > > >> It still can't find the correct time. I tried
> > > > > >>
> > > > > >> FCST_VAR1_LEVELS = (*,*,*)
> > > > > >> FCST_VAR1_LEVELS = ({valid?fmt=%Y%m%d_%H%M%S},*,*)
> > > > > >> FCST_VAR1_LEVELS = ({valid?fmt=%Y%m%d_%H%M},*,*)
> > > > > >> FCST_VAR1_LEVELS = ({valid?fmt=%Y%m%d%H%M},*,*)
> > > > > >>
> > > > > >> Thanks!
> > > > > >>
> > > > > >> Zhuming
> > > > > >>
> > > > > >> On Wed, Apr 14, 2021 at 3:41 PM George McCabe via RT <
> > > > met_help at ucar.edu
> > > > > >
> > > > > >> wrote:
> > > > > >>
> > > > > >> > Hi Zhuming,
> > > > > >> >
> > > > > >> > It looks like you are requesting the first time in the
12 hour
> > > file
> > > > > for
> > > > > >> > each run:
> > > > > >> >
> > > > > >> > FCST_VAR1_LEVELS = (0,*,*)
> > > > > >> > ...
> > > > > >> > FCST_VAR2_LEVELS = (0,*,*)
> > > > > >> >
> > > > > >> > The first dimension in these files is the time offset.
You
> will
> > > need
> > > > > the
> > > > > >> > correct index that corresponds to each valid time.
> Fortunately,
> > > MET
> > > > > >> makes
> > > > > >> > this easy by allowing you to specify the valid time for
a time
> > > > > dimension
> > > > > >> > explicitly and it will find the correct data. You can
set:
> > > > > >> >
> > > > > >> > FCST_VAR1_LEVELS = ({valid?fmt=%Y%m%d_%H%M%S},*,*)
> > > > > >> > ...
> > > > > >> > FCST_VAR2_LEVELS = ({valid?fmt=%Y%m%d_%H%M%S},*,*)
> > > > > >> >
> > > > > >> > And it will substitute the valid time for each run. Let
me
> know
> > if
> > > > you
> > > > > >> have
> > > > > >> > any questions.
> > > > > >> >
> > > > > >> > Thanks,
> > > > > >> > George
> > > > > >> >
> > > > > >> > On Wed, Apr 14, 2021 at 3:18 PM zhuming at ucar.edu via RT
<
> > > > > >> met_help at ucar.edu
> > > > > >> > >
> > > > > >> > wrote:
> > > > > >> >
> > > > > >> > >
> > > > > >> > > <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429
> > >
> > > > > >> > >
> > > > > >> > > Hi George,
> > > > > >> > >
> > > > > >> > > Thanks for your email, I just would like to send you
for
> help.
> > > > > >> > >
> > > > > >> > > I already resolved the last problem with the WRF
forecast,
> > thank
> > > > you
> > > > > >> for
> > > > > >> > > your help.
> > > > > >> > >
> > > > > >> > > I also need to verify the MPAS model forecast. These
12 hour
> > > > > >> > > MAPS forecasts are stored in a one file, such as
> > > > > >> > > 2015071500/mpas_wrf_cf_2015071500.nc, has 12h
forecasts
> > > > > >> > > 2015071512/mpas_wrf_cf_2015071512.nc
> > > > > >> > >
> > > > > >> > > It seems MET is doing fine on the first 2 hours, but
no
> > matched
> > > > > pairs
> > > > > >> for
> > > > > >> > > later hours.
> > > > > >> > > I'm not sure what's wrong with it? the time is not
matched?
> > > > > >> > >
> > > > > >> > > see the log file
> > > > > >> > >
> > > > >
> >
/glade/scratch/zhuming/metplus4/logs/master_metplus.log.20210414145101
> > > > > >> > > config file
/gpfs/fs1/work/zhuming/METplus4/PointStat2.conf
> > > > > >> > >
> > > > > >> > > Thanks,
> > > > > >> > > Zhuming
> > > > > >> > >
> > > > > >> > > On Wed, Apr 14, 2021 at 1:56 PM George McCabe via RT
<
> > > > > >> met_help at ucar.edu>
> > > > > >> > > wrote:
> > > > > >> > >
> > > > > >> > > > Hi Zhuming,
> > > > > >> > > >
> > > > > >> > > > Just following up on this ticket. Were you able to
get
> this
> > > > > >> resolved?
> > > > > >> > Let
> > > > > >> > > > me know if you need more assistance.
> > > > > >> > > >
> > > > > >> > > > Thanks,
> > > > > >> > > > George
> > > > > >> > > >
> > > > > >> > > > On Thu Apr 08 10:04:44 2021, mccabe wrote:
> > > > > >> > > > > Hi Zhuming,
> > > > > >> > > > >
> > > > > >> > > > > Typically directories that contain data for an
entire
> day
> > > are
> > > > > >> named
> > > > > >> > > > > with
> > > > > >> > > > > YYYYMMDD, so your filename template tag would be
> > > > > >> {valid?fmt=%Y%m%d},
> > > > > >> > > > > however that may not always be the case, as you
are
> seeing
> > > > here.
> > > > > >> > There
> > > > > >> > > > > are
> > > > > >> > > > > a couple ways to handle this.
> > > > > >> > > > >
> > > > > >> > > > > If you know that the directory will always be
YYYYMMDD
> > with
> > > 00
> > > > > at
> > > > > >> the
> > > > > >> > > > > end,
> > > > > >> > > > > you can simply put 00 in the template instead of
adding
> %H
> > > in
> > > > > the
> > > > > >> > tag,
> > > > > >> > > > > i.e.
> > > > > >> > > > >
> > > > > >> > > > > FCST_POINT_STAT_INPUT_TEMPLATE =
> > > > > >> > > > >
> > {valid?fmt=%Y%m%d}00/wrfout_d02_{valid?fmt=%Y-%m-%d_%H}:00:
> > > > > >> 00_cf.nc
> > > > > >> > > > >
> > > > > >> > > > > If the HH value can vary, for instance if there
are 2
> > > > > directories
> > > > > >> > > > > created
> > > > > >> > > > > per day (i.e. 2015071400 and 2015071412), then
you can
> > use
> > > > the
> > > > > >> > > > > truncate
> > > > > >> > > > > keyword to handle it.
> > > > > >> > > > >
> > > > > >> > > > > If directories are created every 24 hours:
> > > > > >> > > > >
> > > > > >> > > > > FCST_POINT_STAT_INPUT_TEMPLATE =
> > > > > >> > > > >
> > > {valid?fmt=%Y%m%d%H?truncate=24H}/wrfout_d02_{valid?fmt=%Y-%m-
> > > > > >> > > > > %d_%H}:00:
> > > > > >> > > > > 00_cf.nc
> > > > > >> > > > >
> > > > > >> > > > > If directories are created every 12 hours:
> > > > > >> > > > >
> > > > > >> > > > > FCST_POINT_STAT_INPUT_TEMPLATE =
> > > > > >> > > > >
> > > {valid?fmt=%Y%m%d%H?truncate=12H}/wrfout_d02_{valid?fmt=%Y-%m-
> > > > > >> > > > > %d_%H}:00:
> > > > > >> > > > > 00_cf.nc
> > > > > >> > > > >
> > > > > >> > > > > Let me know if that doesn't work for you.
> > > > > >> > > > >
> > > > > >> > > > > Thanks,
> > > > > >> > > > > George
> > > > > >> > > > >
> > > > > >> > > > > On Thu, Apr 8, 2021 at 9:46 AM zhuming at ucar.edu
via RT
> > > > > >> > > > > <met_help at ucar.edu>
> > > > > >> > > > > wrote:
> > > > > >> > > > >
> > > > > >> > > > > >
> > > > > >> > > > > > <URL:
> > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429
> > > > > >
> > > > > >> > > > > >
> > > > > >> > > > > > Hi George,
> > > > > >> > > > > >
> > > > > >> > > > > > Processing Obs data using "PB2NC" are right
now.
> > > > > >> > > > > >
> > > > > >> > > > > > Now I'm using "PointStat" , I set VALID_BEG =
> > 2015071500,
> > > > > >> > VALID_END =
> > > > > >> > > > > > 2015071501. It looks for the forecast data
2015071500/
> > > > > >> > > > > > wrfout_d02_2015-07-15_00:00:00_cf.nc, and
2015071501/
> > > > > >> > > > > > wrfout_d02_2015-07-15_01:00:00_cf.nc.
> > > > > >> > > > > > But my forecast data are all put in dir of
> 2015071500/.
> > > > > >> > > > > > So how could I set
FCST_POINT_STAT_INPUT_TEMPLATE to
> > get
> > > > the
> > > > > >> right
> > > > > >> > > > > > fcst
> > > > > >> > > > > > data?
> > > > > >> > > > > >
> > > > > >> > > > > > FCST_POINT_STAT_INPUT_TEMPLATE =
> > > > > >> > > > > >
> > > {valid?fmt=%Y%m%d%H}/wrfout_d02_{valid?fmt=%Y-%m-%d_%H}:00:
> > > > > >> > 00_cf.nc
> > > > > >> > > > > >
> > > > > >> > > > > >
> > > > > >> > > > > > config file:
> > > > /gpfs/fs1/work/zhuming/METplus4/PointStat1.conf
> > > > > >> > > > > >
> > > > > >> > > > > > log file:
> > > > > >> > > > > >
> > > > > >> > >
> > > > >
> >
/glade/scratch/zhuming/metplus4/logs/master_metplus.log.20210408085450
> > > > > >> > > > > >
> > > > > >> > > > > >
> > > > > >> > > > > > Thanks,
> > > > > >> > > > > >
> > > > > >> > > > > > Zhuming
> > > > > >> > > > > >
> > > > > >> > > > > > On Wed, Apr 7, 2021 at 5:43 PM George McCabe
via RT
> > > > > >> > > > > > <met_help at ucar.edu>
> > > > > >> > > > > > wrote:
> > > > > >> > > > > >
> > > > > >> > > > > > > Hi Zhuming,
> > > > > >> > > > > > >
> > > > > >> > > > > > > Looking at the log output, it looks like the
data is
> > > > stored
> > > > > in
> > > > > >> > the
> > > > > >> > > > > > prepBUFR
> > > > > >> > > > > > > files differently that you may have thought:
> > > > > >> > > > > > >
> > > > > >> > > > > > > DEBUG 2: PrepBufr Time Center:
> > 20150715_000000
> > > > > >> > > > > > > DEBUG 2: Searching Time Window:
> > 20150715_030000
> > > to
> > > > > >> > > > > > 20150715_040000
> > > > > >> > > > > > > 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55%
60% 65%
> 70%
> > > 75%
> > > > > 80%
> > > > > >> > 85%
> > > > > >> > > > > > > 90%
> > > > > >> > > > > > 95%
> > > > > >> > > > > > > 100%
> > > > > >> > > > > > >
> > > > > >> > > > > > > DEBUG 2: Total Messages processed
=
> > 590836
> > > > > >> > > > > > > DEBUG 2: Rejected based on message type
= 0
> > > > > >> > > > > > > DEBUG 2: Rejected based on station id
= 0
> > > > > >> > > > > > > DEBUG 2: Rejected based on valid time
=
> > 590815
> > > > > >> > > > > > > DEBUG 2: Rejected based on masking grid
= 20
> > > > > >> > > > > > > DEBUG 2: Rejected based on masking polygon
= 0
> > > > > >> > > > > > > DEBUG 2: Rejected based on elevation
= 0
> > > > > >> > > > > > > DEBUG 2: Rejected based on pb report type
= 1
> > > > > >> > > > > > > DEBUG 2: Rejected based on input report type
= 0
> > > > > >> > > > > > > DEBUG 2: Rejected based on instrument type
= 0
> > > > > >> > > > > > > DEBUG 2: Rejected based on zero observations
= 0
> > > > > >> > > > > > > DEBUG 2: Total Messages retained
= 0
> > > > > >> > > > > > > DEBUG 2: Total observations retained or
derived = 0
> > > > > >> > > > > > > DEBUG 1: Obs time between *20150714_210000
and
> > > > > >> 20150715_030000*
> > > > > >> > > > > > > WARNING:
> > > > > >> > > > > > > WARNING: process_pbfile() -> No PrepBufr
messages
> > > retained
> > > > > >> from
> > > > > >> > > > > > > file:
> > > > > >> > > > > > >
/glade/collections/rda/data/ds337.0/prepnr/2015/
> > > > > >> > > > > > > prepbufr.gdas.2015071500.nr
> > > > > >> > > > > > > WARNING:
> > > > > >> > > > > > > ERROR :
> > > > > >> > > > > > > ERROR : write_netcdf_hdr_data() -> No
PrepBufr
> > messages
> > > > > >> > retained.
> > > > > >> > > > > > Nothing
> > > > > >> > > > > > > to write.
> > > > > >> > > > > > > ERROR :
> > > > > >> > > > > > >
> > > > > >> > > > > > > The prepbufr.gdas.2015071500.nr file actually
> > contains
> > > > data
> > > > > >> from
> > > > > >> > > > > > > 20150714_210000 to 20150715_030000, not
> > 20150715_000000
> > > to
> > > > > >> > > > > > 20150715_060000.
> > > > > >> > > > > > >
> > > > > >> > > > > > > To get the correct file for each run, you
will have
> to
> > > > shift
> > > > > >> the
> > > > > >> > > > > > > valid
> > > > > >> > > > > > time
> > > > > >> > > > > > > before truncating it so that it gets the
correct
> file
> > > for
> > > > > each
> > > > > >> > run.
> > > > > >> > > > > > Instead
> > > > > >> > > > > > > of setting:
> > > > > >> > > > > > >
> > > > > >> > > > > > > PB2NC_INPUT_TEMPLATE =
> > > > > >> > > > > > >
> > > > > >> >
> > >
{valid?fmt=%Y}/prepbufr.gdas.{valid?fmt=%Y%m%d%H?truncate=21600}.nr
> > > > > >> > > > > > >
> > > > > >> > > > > > > You will have to also shift forward 3 hours,
i.e.:
> > > > > >> > > > > > >
> > > > > >> > > > > > > PB2NC_INPUT_TEMPLATE =
> > > > > >> > > > > > >
> > > > > >> > > > > > >
> > > > > >> > > > > >
> > > > > >> > > >
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
>
{valid?fmt=%Y}/prepbufr.gdas.{valid?fmt=%Y%m%d%H?shift=3H?truncate=21600}.nr
> > > > > >> > > > > > >
> > > > > >> > > > > > > This says to add 3 hours to the current valid
time
> > > before
> > > > > >> > > > > > > truncating to
> > > > > >> > > > > > the
> > > > > >> > > > > > > nearest 6 hours.
> > > > > >> > > > > > >
> > > > > >> > > > > > > Valid: 20150715 0Z + 3 = 3Z truncate 6H =
0Z
> > > > > >> > > > > > > File: prepbufr.gdas.2015071500.nr
> > > > > >> > > > > > >
> > > > > >> > > > > > > Valid: 20150715 1Z + 3 = 4Z truncate 6H =
0Z
> > > > > >> > > > > > > File: prepbufr.gdas.2015071500.nr
> > > > > >> > > > > > >
> > > > > >> > > > > > > Valid: 20150715 2Z + 3 = 5Z truncate 6H =
0Z
> > > > > >> > > > > > > File: prepbufr.gdas.2015071500.nr
> > > > > >> > > > > > >
> > > > > >> > > > > > > Valid: 20150715 3Z + 3 = 6Z truncate 6H =
6Z
> > > > > >> > > > > > > File: prepbufr.gdas.2015071506.nr
> > > > > >> > > > > > >
> > > > > >> > > > > > > Let me know if that doesn't work.
> > > > > >> > > > > > >
> > > > > >> > > > > > > Thanks,
> > > > > >> > > > > > > George
> > > > > >> > > > > > >
> > > > > >> > > > > > >
> > > > > >> > > > > > > On Wed, Apr 7, 2021 at 5:32 PM
zhuming at ucar.edu via
> > RT
> > > <
> > > > > >> > > > > > met_help at ucar.edu>
> > > > > >> > > > > > > wrote:
> > > > > >> > > > > > >
> > > > > >> > > > > > > >
> > > > > >> > > > > > > > <URL:
> > > > > >> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429
> > > > > >> > >
> > > > > >> > > > > > > >
> > > > > >> > > > > > > > Hi George,
> > > > > >> > > > > > > >
> > > > > >> > > > > > > > Now makes me confused is why processed obs
of
> > several
> > > > > times
> > > > > >> are
> > > > > >> > > > > > missing.
> > > > > >> > > > > > > > There has files of time at 00,01,02, but no
> > 03,04,05;
> > > > has
> > > > > >> > > > > > > > 06,07,08, but
> > > > > >> > > > > > > no
> > > > > >> > > > > > > > 09,10,11, has 12,13,14, no 15,16,17; has
> 18,19,20,
> > > but
> > > > > with
> > > > > >> > > > > > > > 21,no
> > > > > >> > > > > > 22,23.
> > > > > >> > > > > > > > From yours and mines:
> > > > > >> > > > > > > >
> > > > > >> > > > > > > > ls -l /glade/u/home/mccabe/data/out/pb2nc1/
> > > > > >> > > > > > > >
> > > > > >> > > > > > > >
> > > > > >> > > > > > > > -rw-r--r-- 1 mccabe ral 10141339 Apr 6
17:52
> > > > > >> > pbs.2015071500.nc
> > > > > >> > > > > > > >
> > > > > >> > > > > > > > -rw-r--r-- 1 mccabe ral 5413907 Apr 6
17:56
> > > > > >> > pbs.2015071501.nc
> > > > > >> > > > > > > >
> > > > > >> > > > > > > > -rw-r--r-- 1 mccabe ral 4923711 Apr 6
18:00
> > > > > >> > pbs.2015071502.nc
> > > > > >> > > > > > > >
> > > > > >> > > > > > > > -rw-r--r-- 1 mccabe ral 5724447 Apr 6
18:14
> > > > > >> > pbs.2015071506.nc
> > > > > >> > > > > > > >
> > > > > >> > > > > > > > -rw-r--r-- 1 mccabe ral 5280463 Apr 6
18:18
> > > > > >> > pbs.2015071507.nc
> > > > > >> > > > > > > >
> > > > > >> > > > > > > > -rw-r--r-- 1 mccabe ral 4706119 Apr 6
18:22
> > > > > >> > pbs.2015071508.nc
> > > > > >> > > > > > > >
> > > > > >> > > > > > > > -rw-r--r-- 1 mccabe ral 9878263 Apr 6
18:36
> > > > > >> > pbs.2015071512.nc
> > > > > >> > > > > > > >
> > > > > >> > > > > > > > -rw-r--r-- 1 mccabe ral 5392527 Apr 6
18:40
> > > > > >> > pbs.2015071513.nc
> > > > > >> > > > > > > >
> > > > > >> > > > > > > > -rw-r--r-- 1 mccabe ral 96 Apr 6
18:40
> > > > > >> > pbs.2015071514.nc
> > > > > >> > > > > > > >
> > > > > >> > > > > > > > -rw-r--r-- 1 mccabe ral 96 Apr 6
18:42
> > > > > >> > pbs.2015071515.nc
> > > > > >> > > > > > > >
> > > > > >> > > > > > > >
> > > > > >> > > > > > > >
cheyenne3:/glade/scratch/zhuming/metplus4/pb2nc1>l
> > > > > >> > > > > > > >
> > > > > >> > > > > > > > total 96112
> > > > > >> > > > > > > >
> > > > > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 11550172 Apr 7
15:11
> > > > > >> > > pbs.2015071500.nc
> > > > > >> > > > > > > >
> > > > > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 6615956 Apr 7
15:14
> > > > > >> > > pbs.2015071501.nc
> > > > > >> > > > > > > >
> > > > > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 5984328 Apr 7
15:18
> > > > > >> > > pbs.2015071502.nc
> > > > > >> > > > > > > >
> > > > > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 6980712 Apr 7
15:33
> > > > > >> > > pbs.2015071506.nc
> > > > > >> > > > > > > >
> > > > > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 6455824 Apr 7
15:36
> > > > > >> > > pbs.2015071507.nc
> > > > > >> > > > > > > >
> > > > > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 5749432 Apr 7
15:40
> > > > > >> > > pbs.2015071508.nc
> > > > > >> > > > > > > >
> > > > > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 11284936 Apr 7
15:55
> > > > > >> > > pbs.2015071512.nc
> > > > > >> > > > > > > >
> > > > > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 6600888 Apr 7
15:59
> > > > > >> > > pbs.2015071513.nc
> > > > > >> > > > > > > >
> > > > > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 5868168 Apr 7
16:03
> > > > > >> > > pbs.2015071514.nc
> > > > > >> > > > > > > >
> > > > > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 7273092 Apr 7
16:18
> > > > > >> > > pbs.2015071518.nc
> > > > > >> > > > > > > >
> > > > > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 6656344 Apr 7
16:22
> > > > > >> > > pbs.2015071519.nc
> > > > > >> > > > > > > >
> > > > > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 6109788 Apr 7
16:27
> > > > > >> > > pbs.2015071520.nc
> > > > > >> > > > > > > >
> > > > > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 58808 Apr 7
16:31
> > > > > >> > > pbs.2015071521.nc
> > > > > >> > > > > > > >
> > > > > >> > > > > > > > -rw-r--r-- 1 zhuming ncar 11132440 Apr 7
16:43
> > > > > >> > > pbs.2015071600.nc
> > > > > >> > > > > > > >
> > > > > >> > > > > > > >
> > > > > >> > > > > > > >
> > > > > >> > > > > > > > Thanks,
> > > > > >> > > > > > > >
> > > > > >> > > > > > > > Zhuming
> > > > > >> > > > > > > >
> > > > > >> > > > > > > > On Tue, Apr 6, 2021 at 6:34 PM George
McCabe via
> RT
> > > > > >> > > > > > > > <met_help at ucar.edu
> > > > > >> > > > > > >
> > > > > >> > > > > > > > wrote:
> > > > > >> > > > > > > >
> > > > > >> > > > > > > > > Hi Zhuming,
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > > Thanks for the information. You will need
to use
> > the
> > > > > >> truncate
> > > > > >> > > > > > > > > keyword
> > > > > >> > > > > > > to
> > > > > >> > > > > > > > > find the files, then set the valid
beg/end range
> > to
> > > 1
> > > > > >> hour.
> > > > > >> > > > > > > > > This is
> > > > > >> > > > > > > what
> > > > > >> > > > > > > > I
> > > > > >> > > > > > > > > changed from your PB2NC1.conf config
file:
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > > *PB2NC_VALID_BEGIN =
> > > > > {valid?fmt=%Y%m%d_%H}PB2NC_VALID_END
> > > > > >> =
> > > > > >> > > > > > > > > {valid?fmt=%Y%m%d_%H?shift=1H}*
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > > *PB2NC_INPUT_TEMPLATE =
> > > > > >> > > > > > > > >
> > > > > >> > > >
> > > > >
> {valid?fmt=%Y}/prepbufr.gdas.{valid?fmt=%Y%m%d%H?truncate=21600}.nr*
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > > My config file is on cheyenne here:
> > > > > >> > > > > > > > > /glade/u/home/mccabe/PB2NC1.conf
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > > I just started running it and the output
is
> being
> > > > > written
> > > > > >> > here:
> > > > > >> > > > > > > > > /glade/u/home/mccabe/data/out/pb2nc1
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > > For PointStat, you will want to loop over
the
> same
> > > > > times.
> > > > > >> I
> > > > > >> > > > > > > > > noticed
> > > > > >> > > > > > you
> > > > > >> > > > > > > > > specify message type in the
VAR<n>_OPTIONS, but
> > you
> > > do
> > > > > not
> > > > > >> > need
> > > > > >> > > > > > > > > to do
> > > > > >> > > > > > > > this
> > > > > >> > > > > > > > > since there is a POINT_STAT_MESSAGE_TYPE
> variable
> > > that
> > > > > >> > defines
> > > > > >> > > > > > > > > this.
> > > > > >> > > > > > > This
> > > > > >> > > > > > > > > is what I changed in your PointStat1.conf
file:
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > > *VALID_END = 2015071600*
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > > *VALID_INCREMENT =
1HPOINT_STAT_OBS_VALID_BEG
> > > > > >> > > > > > =POINT_STAT_OBS_VALID_BEG
> > > > > >> > > > > > > > =*
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > > *POINT_STAT_MESSAGE_TYPE = ADPSFC*
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > > *#OBS_VAR1_OPTIONS = message_type =
> > > > > >> > > > > > > > > ["ADPSFC"];#OBS_VAR2_OPTIONS =
> > > > > >> > > > > > > > > message_type =
["ADPSFC"];#OBS_VAR3_OPTIONS =
> > > > > >> message_type =
> > > > > >> > > > > > > > > ["ADPSFC"];#OBS_POINT_STAT_INPUT_DIR =
> > > > > >> > > > > > > > > /glade/scratch/zhuming/metplus4/pb2nc1*
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > > *OBS_POINT_STAT_INPUT_DIR =
> {OUTPUT_BASE}/pb2nc1*
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > > Or you can view my file in the same
directory. I
> > > > usually
> > > > > >> try
> > > > > >> > to
> > > > > >> > > > > > > > > test
> > > > > >> > > > > > > > with a
> > > > > >> > > > > > > > > single run time to make sure it is
working
> > properly,
> > > > so
> > > > > I
> > > > > >> set
> > > > > >> > > > > > VALID_END
> > > > > >> > > > > > > > > back to 2015071500. I ran it and it
produced an
> > > output
> > > > > >> file,
> > > > > >> > > > > > > > > but
> > > > > >> > > > > > there
> > > > > >> > > > > > > > were
> > > > > >> > > > > > > > > no values besides the header. I checked
the log
> > file
> > > > and
> > > > > >> saw
> > > > > >> > > > > > > > > this:
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > > *DEBUG 2: Processing T_sfc(0,*,*) versus
TMP/Z2,
> > for
> > > > > >> > > > > > > > > observation type
> > > > > >> > > > > > > > > ADPSFC, over region FULL, for
interpolation
> method
> > > > > >> BILIN(4),
> > > > > >> > > > > > > > > using 0
> > > > > >> > > > > > > > > matched pairs.DEBUG 3: Number of matched
pairs
> =
> > > > > 0DEBUG
> > > > > >> 3:
> > > > > >> > > > > > > Observations
> > > > > >> > > > > > > > > processed = 393373DEBUG 3: Rejected:
station
> id
> > > > > =
> > > > > >> > > > > > > > > 0DEBUG 3:
> > > > > >> > > > > > > > > Rejected: obs type = 390528DEBUG
3:
> > Rejected:
> > > > > valid
> > > > > >> > time
> > > > > >> > > > > > =
> > > > > >> > > > > > > > > 0DEBUG 3: Rejected: bad obs value =
0DEBUG 3:
> > > > > Rejected:
> > > > > >> off
> > > > > >> > > > > > > > > the
> > > > > >> > > > > > grid
> > > > > >> > > > > > > > =
> > > > > >> > > > > > > > > 2586DEBUG 3: Rejected: topography =
0DEBUG
> 3:
> > > > > >> Rejected:
> > > > > >> > > > > > > > > level
> > > > > >> > > > > > > > mismatch
> > > > > >> > > > > > > > > = 259DEBUG 3: Rejected: quality marker =
0DEBUG
> > 3:
> > > > > >> Rejected:
> > > > > >> > > > > > message
> > > > > >> > > > > > > > type
> > > > > >> > > > > > > > > = 0DEBUG 3: Rejected: masking region =
0DEBUG
> 3:
> > > > > >> Rejected:
> > > > > >> > bad
> > > > > >> > > > > > fcst
> > > > > >> > > > > > > > > value = 0DEBUG 3: Rejected: bad climo
mean =
> > > 0DEBUG
> > > > 3:
> > > > > >> > > > > > > > > Rejected:
> > > > > >> > > > > > bad
> > > > > >> > > > > > > > > climo stdev = 0DEBUG 3: Rejected:
duplicates
> > =
> > > > > 0DEBUG
> > > > > >> > > > > > > > > 2:DEBUG 2:
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > >
> > > > > >> > > > > > >
> > > > > >> > > > > >
> > > > > >> > > >
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
>
--------------------------------------------------------------------------------DEBUG
> > > > > >> > > > > > > > > 2:DEBUG 2: Processing u_10m_gr(0,*,*)
versus
> > > UGRD/Z10,
> > > > > for
> > > > > >> > > > > > observation
> > > > > >> > > > > > > > type
> > > > > >> > > > > > > > > ADPSFC, over region FULL, for
interpolation
> method
> > > > > >> BILIN(4),
> > > > > >> > > > > > > > > using 0
> > > > > >> > > > > > > > > matched pairs.DEBUG 3: Number of matched
pairs
> =
> > > > > 0DEBUG
> > > > > >> 3:
> > > > > >> > > > > > > Observations
> > > > > >> > > > > > > > > processed = 393373*
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > > *DEBUG 3: Rejected: station id =
0DEBUG 3:
> > > > > Rejected:
> > > > > >> obs
> > > > > >> > > > > > > > > type
> > > > > >> > > > > > > > =
> > > > > >> > > > > > > > > 382965DEBUG 3: Rejected: valid time
=
> 0DEBUG
> > 3:
> > > > > >> > Rejected:
> > > > > >> > > > > > > > > bad
> > > > > >> > > > > > obs
> > > > > >> > > > > > > > > value = 0DEBUG 3: Rejected: off the
grid =
> > > > > 9240DEBUG
> > > > > >> 3:
> > > > > >> > > > > > Rejected:
> > > > > >> > > > > > > > > topography = 0DEBUG 3: Rejected:
level
> > mismatch
> > > > =
> > > > > >> > > > > > > > > 1168DEBUG 3:
> > > > > >> > > > > > > > > Rejected: quality marker = 0DEBUG 3:
Rejected:
> > > > message
> > > > > >> type
> > > > > >> > > > > > > > > =
> > > > > >> > > > > > > 0DEBUG
> > > > > >> > > > > > > > 3:
> > > > > >> > > > > > > > > Rejected: masking region = 0DEBUG 3:
Rejected:
> > bad
> > > > fcst
> > > > > >> > value
> > > > > >> > > > > > > > > =
> > > > > >> > > > > > > 0DEBUG
> > > > > >> > > > > > > > 3:
> > > > > >> > > > > > > > > Rejected: bad climo mean = 0DEBUG 3:
Rejected:
> > bad
> > > > > climo
> > > > > >> > stdev
> > > > > >> > > > > > > > > =
> > > > > >> > > > > > > 0DEBUG
> > > > > >> > > > > > > > 3:
> > > > > >> > > > > > > > > Rejected: duplicates = 0DEBUG
2:DEBUG 2:
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > >
> > > > > >> > > > > > >
> > > > > >> > > > > >
> > > > > >> > > >
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
>
--------------------------------------------------------------------------------DEBUG
> > > > > >> > > > > > > > > 2:DEBUG 2: Processing v_10m_gr(0,*,*)
versus
> > > VGRD/Z10,
> > > > > for
> > > > > >> > > > > > observation
> > > > > >> > > > > > > > type
> > > > > >> > > > > > > > > ADPSFC, over region FULL, for
interpolation
> method
> > > > > >> BILIN(4),
> > > > > >> > > > > > > > > using 0
> > > > > >> > > > > > > > > matched pairs.DEBUG 3: Number of matched
pairs
> =
> > > > > 0DEBUG
> > > > > >> 3:
> > > > > >> > > > > > > Observations
> > > > > >> > > > > > > > > processed = 393373DEBUG 3: Rejected:
station
> id
> > > > > =
> > > > > >> > > > > > > > > 0DEBUG 3:
> > > > > >> > > > > > > > > Rejected: obs type = 382965DEBUG
3:
> > Rejected:
> > > > > valid
> > > > > >> > time
> > > > > >> > > > > > =
> > > > > >> > > > > > > > > 0DEBUG 3: Rejected: bad obs value =
0DEBUG 3:
> > > > > Rejected:
> > > > > >> off
> > > > > >> > > > > > > > > the
> > > > > >> > > > > > grid
> > > > > >> > > > > > > > =
> > > > > >> > > > > > > > > 9240DEBUG 3: Rejected: topography =
0DEBUG
> 3:
> > > > > >> Rejected:
> > > > > >> > > > > > > > > level
> > > > > >> > > > > > > > mismatch
> > > > > >> > > > > > > > > = 1168DEBUG 3: Rejected: quality marker
=
> 0DEBUG
> > 3:
> > > > > >> > Rejected:
> > > > > >> > > > > > message
> > > > > >> > > > > > > > > type = 0DEBUG 3: Rejected: masking
region =
> > > 0DEBUG
> > > > > 3:
> > > > > >> > > > > > > > > Rejected:
> > > > > >> > > > > > bad
> > > > > >> > > > > > > > > fcst value = 0DEBUG 3: Rejected: bad
climo mean
> > =
> > > > > >> 0DEBUG 3:
> > > > > >> > > > > > Rejected:
> > > > > >> > > > > > > > bad
> > > > > >> > > > > > > > > climo stdev = 0*
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > > *DEBUG 3: Rejected: duplicates =
0DEBUG
> > 2:DEBUG
> > > > 2:
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > >
> > > > > >> > > > > > >
> > > > > >> > > > > >
> > > > > >> > > >
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
>
--------------------------------------------------------------------------------DEBUG
> > > > > >> > > > > > > > > 2:DEBUG 1: Output file:
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > >
> > > > > >> > > > > > >
> > > > > >> > > > > >
> > > > > >> > > >
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
>
/glade/u/home/mccabe/data/out/point_stat1/point_stat_000000L_20150715_000000V.stat*
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > > This describes the reason why each
observation
> was
> > > > > >> rejected.
> > > > > >> > > > > > > > > Some
> > > > > >> > > > > > > > > observations didn't match the message
type you
> > > > > requested,
> > > > > >> > some
> > > > > >> > > > > > > > > were
> > > > > >> > > > > > not
> > > > > >> > > > > > > > > within the forecast grid, and some did
not match
> > the
> > > > > >> vertical
> > > > > >> > > > > > > > > level.
> > > > > >> > > > > > > You
> > > > > >> > > > > > > > > will have to make adjustments to your
settings
> so
> > > that
> > > > > the
> > > > > >> > data
> > > > > >> > > > > > matches
> > > > > >> > > > > > > > up
> > > > > >> > > > > > > > > properly. Maybe the grid you are using in
the
> > PB2NC
> > > > > config
> > > > > >> > file
> > > > > >> > > > > > > > > does
> > > > > >> > > > > > > not
> > > > > >> > > > > > > > > match the forecast grid and you are
missing
> > values.
> > > If
> > > > > >> that
> > > > > >> > is
> > > > > >> > > > > > > > > the
> > > > > >> > > > > > > case,
> > > > > >> > > > > > > > > you will have to reconfigure and rerun
PB2NC to
> > get
> > > > > >> output on
> > > > > >> > > > > > > > > the
> > > > > >> > > > > > > correct
> > > > > >> > > > > > > > > gr5id. Maybe the observation points don't
line
> up
> > > with
> > > > > the
> > > > > >> > > > > > > > > vertical
> > > > > >> > > > > > > level
> > > > > >> > > > > > > > > of the model fields. You can use a range
of
> level
> > > > values
> > > > > >> for
> > > > > >> > > > > > > > > the
> > > > > >> > > > > > > > > observation data to allow data within
that
> range,
> > > i.e.
> > > > > >> Z0-10.
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > > Let me know if you are still having
issues
> getting
> > > > this
> > > > > to
> > > > > >> > work
> > > > > >> > > > > > > > > and I
> > > > > >> > > > > > > can
> > > > > >> > > > > > > > > help out.
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > > Thanks,
> > > > > >> > > > > > > > > George
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > > On Tue, Apr 6, 2021 at 5:17 PM
zhuming at ucar.edu
> > via
> > > > RT
> > > > > <
> > > > > >> > > > > > > > met_help at ucar.edu>
> > > > > >> > > > > > > > > wrote:
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > > >
> > > > > >> > > > > > > > > > <URL:
> > > > > >> > >
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429
> > > > > >> > > > > > > > > > >
> > > > > >> > > > > > > > > >
> > > > > >> > > > > > > > > > Hi George,
> > > > > >> > > > > > > > > >
> > > > > >> > > > > > > > > > Thanks for your reply!
> > > > > >> > > > > > > > > >
> > > > > >> > > > > > > > > > I'm still confused, I need to output
hourly
> obs
> > > data
> > > > > to
> > > > > >> do
> > > > > >> > > > > > > > > > hourly
> > > > > >> > > > > > > > > > verification.
> > > > > >> > > > > > > > > > The original GDAS Prepbufr data has 4
files
> one
> > > day,
> > > > > >> should
> > > > > >> > > > > > > > > > with 6
> > > > > >> > > > > > > > hours
> > > > > >> > > > > > > > > > obs data in one file.
> > > > > >> > > > > > > > > >
> > > > > >> > > > > > > > > > After "PB2NC", I can see
pbs.2015071500.nc
> (obs
> > > > time
> > > > > is
> > > > > >> > 00?
> > > > > >> > > > > > > > > > or 6
> > > > > >> > > > > > > times
> > > > > >> > > > > > > > > of
> > > > > >> > > > > > > > > > 00,01,02,03,04,05?),
pbs.2015071506.nc...
> > > > > >> > > > > > > > > >
> > > > > >> > > > > > > > > > my config file :
> > > > > >> > > > > >
cheyenne:/gpfs/fs1/work/zhuming/METplus4/PB2NC1.conf
> > > > > >> > > > > > > > > > and
> > > /gpfs/fs1/work/zhuming/METplus4/PointStat1.conf
> > > > > >> > > > > > > > > >
> > > > > >> > > > > > > > > > Thanks,
> > > > > >> > > > > > > > > > Zhuming
> > > > > >> > > > > > > > > >
> > > > > >> > > > > > > > > > On Tue, Apr 6, 2021 at 4:23 PM George
McCabe
> via
> > > RT
> > > > <
> > > > > >> > > > > > > met_help at ucar.edu
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > > > wrote:
> > > > > >> > > > > > > > > >
> > > > > >> > > > > > > > > > > Hi Zhuming,
> > > > > >> > > > > > > > > > >
> > > > > >> > > > > > > > > > > To run the tools hourly you would set
the
> > > > > >> VALID_INCREMENT
> > > > > >> > > > > > > > > > > value
> > > > > >> > > > > > for
> > > > > >> > > > > > > > 1H.
> > > > > >> > > > > > > > > > > Here is an example that runs once per
hour
> > for a
> > > > day
> > > > > >> from
> > > > > >> > > > > > > 2015-07-15
> > > > > >> > > > > > > > 0Z
> > > > > >> > > > > > > > > > to
> > > > > >> > > > > > > > > > > 2017-07-16 0Z:
> > > > > >> > > > > > > > > > >
> > > > > >> > > > > > > > > > > VALID_TIME_FMT = %Y%m%d%H
> > > > > >> > > > > > > > > > >
> > > > > >> > > > > > > > > > > VALID_BEG = 2015071500
> > > > > >> > > > > > > > > > >
> > > > > >> > > > > > > > > > > VALID_END = 2015071600
> > > > > >> > > > > > > > > > >
> > > > > >> > > > > > > > > > > VALID_INCREMENT = 1H
> > > > > >> > > > > > > > > > >
> > > > > >> > > > > > > > > > > LEAD_SEQ = 0
> > > > > >> > > > > > > > > > >
> > > > > >> > > > > > > > > > > Each run time will use the
> > PB2NC_INPUT_TEMPLATE
> > > to
> > > > > >> find
> > > > > >> > > > > > > > > > > files
> > > > > >> > > > > > whose
> > > > > >> > > > > > > > > > > filenames match the time you need.
> > > > > >> > > > > > > > > > >
> > > > > >> > > > > > > > > > > To read prepBUFR observations for 12
hours,
> > you
> > > > will
> > > > > >> want
> > > > > >> > > > > > > > > > > to set
> > > > > >> > > > > > > > > > > PB2NC_VALID_BEGIN and PB2NC_VALID_END
to
> > specify
> > > > the
> > > > > >> > range
> > > > > >> > > > > > > > > > > around
> > > > > >> > > > > > > the
> > > > > >> > > > > > > > > > valid
> > > > > >> > > > > > > > > > > time to process. These values depend
on what
> > is
> > > > > >> contained
> > > > > >> > > > > > > > > > > inside
> > > > > >> > > > > > > your
> > > > > >> > > > > > > > > > > prepBUFR files. If you want to use
data from
> > the
> > > > > valid
> > > > > >> > time
> > > > > >> > > > > > > > > > > of
> > > > > >> > > > > > the
> > > > > >> > > > > > > > > > filename
> > > > > >> > > > > > > > > > > backwards 12 hours, you would set:
> > > > > >> > > > > > > > > > >
> > > > > >> > > > > > > > > > > PB2NC_VALID_BEGIN =
> > > > {valid?fmt=%Y%m%d_%H?shift=-12H
> > > > > }
> > > > > >> > > > > > > > > > > PB2NC_VALID_END =
{valid?fmt=%Y%m%d_%H}
> > > > > >> > > > > > > > > > >
> > > > > >> > > > > > > > > > > For a runtime of 2015071512, this
would be
> > > > > substituted
> > > > > >> > as:
> > > > > >> > > > > > > > > > >
> > > > > >> > > > > > > > > > > valid_beg: 20150715_00
> > > > > >> > > > > > > > > > > valid_end: 20150715_12
> > > > > >> > > > > > > > > > >
> > > > > >> > > > > > > > > > > If you want to create an output file
for
> each
> > > hour
> > > > > in
> > > > > >> > PB2NC
> > > > > >> > > > > > > > > > > you
> > > > > >> > > > > > > would
> > > > > >> > > > > > > > > > need
> > > > > >> > > > > > > > > > > to make sure that the hour is
specified in
> the
> > > > > >> > > > > > > PB2NC_OUTPUT_TEMPLATE.
> > > > > >> > > > > > > > > > >
> > > > > >> > > > > > > > > > > PointStat has similar config
variables to
> set
> > > the
> > > > > >> valid
> > > > > >> > beg
> > > > > >> > > > > > > > > > > and
> > > > > >> > > > > > end
> > > > > >> > > > > > > > > range
> > > > > >> > > > > > > > > > > to use (POINT_STAT_OBS_VALID_BEG and
> > > > > >> > > > > > > > > > > POINT_STAT_OBS_VALID_END).
> > > > > >> > > > > > > > > > >
> > > > > >> > > > > > > > > > > If your prepBUFR files are separated
out
> into
> > > > > multiple
> > > > > >> > > > > > > > > > > files that
> > > > > >> > > > > > > > need
> > > > > >> > > > > > > > > to
> > > > > >> > > > > > > > > > > be read into PB2NC, you can set a
file
> window
> > > > around
> > > > > >> the
> > > > > >> > > > > > > > > > > valid
> > > > > >> > > > > > time
> > > > > >> > > > > > > > to
> > > > > >> > > > > > > > > > > allow files to be passed into the
app. More
> > > > > >> information
> > > > > >> > on
> > > > > >> > > > > > > > > > > that
> > > > > >> > > > > > > > > > > functionality is here:
> > > > > >> > > > > > > > > > >
> > > > > >> > > > > > > > > > >
> > > > > >> > > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > >
> > > > > >> > > > > > >
> > > > > >> > > > > >
> > > > > >> > > >
> > > > > >> > >
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
>
https://metplus.readthedocs.io/en/latest/Users_Guide/systemconfiguration.html#using-
> > > > > >> > > > > > windows-to-find-valid-files
> > > > > >> > > > > > > > > > >
> > > > > >> > > > > > > > > > > If you are still having trouble
getting this
> > > > > >> configured,
> > > > > >> > > > > > > > > > > you
> > > > > >> > > > > > could
> > > > > >> > > > > > > > send
> > > > > >> > > > > > > > > > me
> > > > > >> > > > > > > > > > > the paths of the files you are trying
to
> read
> > > and
> > > > > what
> > > > > >> > data
> > > > > >> > > > > > > > > > > is
> > > > > >> > > > > > > > > contained
> > > > > >> > > > > > > > > > in
> > > > > >> > > > > > > > > > > them so I can give you suggestions on
what
> to
> > > set
> > > > > for
> > > > > >> the
> > > > > >> > > > > > > > > configuration.
> > > > > >> > > > > > > > > > >
> > > > > >> > > > > > > > > > > Thanks,
> > > > > >> > > > > > > > > > > George
> > > > > >> > > > > > > > > > >
> > > > > >> > > > > > > > > > >
> > > > > >> > > > > > > > > > >
> > > > > >> > > > > > > > > > > On Tue, Apr 6, 2021 at 2:17 PM Julie
> > Prestopnik
> > > > via
> > > > > >> RT <
> > > > > >> > > > > > > > > > met_help at ucar.edu>
> > > > > >> > > > > > > > > > > wrote:
> > > > > >> > > > > > > > > > >
> > > > > >> > > > > > > > > > > >
> > > > > >> > > > > > > > > > > > Tue Apr 06 14:16:42 2021: Request
99429
> was
> > > > acted
> > > > > >> upon.
> > > > > >> > > > > > > > > > > > Transaction: Given to mccabe
(George
> McCabe)
> > > by
> > > > > >> jpresto
> > > > > >> > > > > > > > > > > > Queue: met_help
> > > > > >> > > > > > > > > > > > Subject: How to set time for
hourly
> > 12-24
> > > > > hours
> > > > > >> > data
> > > > > >> > > > > > > > > > > > ?
> > > > > >> > > > > > > > > > > > Owner: mccabe
> > > > > >> > > > > > > > > > > > Requestors: zhuming at ucar.edu
> > > > > >> > > > > > > > > > > > Status: new
> > > > > >> > > > > > > > > > > > Ticket <URL:
> > > > > >> > > > > > > > >
> > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=99429
> > > > > >> > > > > > > > > > >
> > > > > >> > > > > > > > > > > >
> > > > > >> > > > > > > > > > > >
> > > > > >> > > > > > > > > > > > This transaction appears to have no
> content
> > > > > >> > > > > > > > > > > >
> > > > > >> > > > > > > > > > >
> > > > > >> > > > > > > > > > >
> > > > > >> > > > > > > > > > > --
> > > > > >> > > > > > > > > > > George McCabe - Software Engineer III
> > > > > >> > > > > > > > > > > National Center for Atmospheric
Research
> > > > > >> > > > > > > > > > > Research Applications Laboratory
> > > > > >> > > > > > > > > > > 303-497-2768
> > > > > >> > > > > > > > > > > ---
> > > > > >> > > > > > > > > > > My working day may not be your
working day.
> > > Please
> > > > > do
> > > > > >> not
> > > > > >> > > > > > > > > > > feel
> > > > > >> > > > > > > > obliged
> > > > > >> > > > > > > > > to
> > > > > >> > > > > > > > > > > reply to this email outside of your
normal
> > > working
> > > > > >> hours.
> > > > > >> > > > > > > > > > >
> > > > > >> > > > > > > > > > >
> > > > > >> > > > > > > > > >
> > > > > >> > > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > > --
> > > > > >> > > > > > > > > George McCabe - Software Engineer III
> > > > > >> > > > > > > > > National Center for Atmospheric Research
> > > > > >> > > > > > > > > Research Applications Laboratory
> > > > > >> > > > > > > > > 303-497-2768
> > > > > >> > > > > > > > > ---
> > > > > >> > > > > > > > > My working day may not be your working
day.
> Please
> > > do
> > > > > not
> > > > > >> > feel
> > > > > >> > > > > > obliged
> > > > > >> > > > > > > to
> > > > > >> > > > > > > > > reply to this email outside of your
normal
> working
> > > > > hours.
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > > >
> > > > > >> > > > > > > >
> > > > > >> > > > > > > >
> > > > > >> > > > > > >
> > > > > >> > > > > > > --
> > > > > >> > > > > > > George McCabe - Software Engineer III
> > > > > >> > > > > > > National Center for Atmospheric Research
> > > > > >> > > > > > > Research Applications Laboratory
> > > > > >> > > > > > > 303-497-2768
> > > > > >> > > > > > > ---
> > > > > >> > > > > > > My working day may not be your working day.
Please
> do
> > > not
> > > > > feel
> > > > > >> > > > > > > obliged to
> > > > > >> > > > > > > reply to this email outside of your normal
working
> > > hours.
> > > > > >> > > > > > >
> > > > > >> > > > > > >
> > > > > >> > > > > >
> > > > > >> > > > > >
> > > > > >> > > >
> > > > > >> > > >
> > > > > >> > > >
> > > > > >> > > >
> > > > > >> > >
> > > > > >> > >
> > > > > >> >
> > > > > >> > --
> > > > > >> > George McCabe - Software Engineer III
> > > > > >> > National Center for Atmospheric Research
> > > > > >> > Research Applications Laboratory
> > > > > >> > 303-497-2768
> > > > > >> > ---
> > > > > >> > My working day may not be your working day. Please do
not feel
> > > > obliged
> > > > > >> to
> > > > > >> > reply to this email outside of your normal working
hours.
> > > > > >> >
> > > > > >> >
> > > > > >>
> > > > > >>
> > > > > >
> > > > > > --
> > > > > > George McCabe - Software Engineer III
> > > > > > National Center for Atmospheric Research
> > > > > > Research Applications Laboratory
> > > > > > 303-497-2768
> > > > > > ---
> > > > > > My working day may not be your working day. Please do not
feel
> > > obliged
> > > > to
> > > > > > reply to this email outside of your normal working hours.
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > George McCabe - Software Engineer III
> > > > > National Center for Atmospheric Research
> > > > > Research Applications Laboratory
> > > > > 303-497-2768
> > > > > ---
> > > > > My working day may not be your working day. Please do not
feel
> > obliged
> > > to
> > > > > reply to this email outside of your normal working hours.
> > > > >
> > > > >
> > > >
> > > >
> > >
> > >
> >
> > --
> > George McCabe - Software Engineer III
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > 303-497-2768
> > ---
> > My working day may not be your working day. Please do not feel
obliged to
> > reply to this email outside of your normal working hours.
> >
> >
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
More information about the Met_help
mailing list