[Met_help] [rt.rap.ucar.edu #90023] History for Missing observation file for point_stat, but no error
John Halley Gotway via RT
met_help at ucar.edu
Tue Jul 9 12:06:50 MDT 2019
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
I've run into a problem with point_stat....I have some runs where I have
missing observation files (well actually, the files exist but for some
reason point_stat is occasionally not reading them at runtime...I suspect
this is a Cheyenne problem and not a MET problem but I can't be sure), and
rather than giving an error code, point_stat just continues and exists with
a success status message.
WARNING:
WARNING: process_obs_file() -> can't open observation netCDF file:
/glade/scratch/kavulich/HRRR/2018/experiment3/run/2018062400/pb2nc/
rap.2018062401.prepbufr.tm00.nc
WARNING:
DEBUG 2:
DEBUG 2:
--------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Processing TMP/Z2 versus TMP/Z2, for observation type ADPSFC, over
region FULL, for interpolation method BILIN(4), using 0 pairs.
DEBUG 2:
DEBUG 2:
--------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Processing DPT/Z2 versus DPT/Z2, for observation type ADPSFC, over
region FULL, for interpolation method BILIN(4), using 0 pairs.
DEBUG 2:
DEBUG 2:
--------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Processing SPFH/Z2 versus SPFH/Z2, for observation type ADPSFC,
over region FULL, for interpolation method BILIN(4), using 0 pairs.
DEBUG 2:
DEBUG 2:
--------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Processing RH/Z2 versus RH/Z2, for observation type ADPSFC, over
region FULL, for interpolation method BILIN(4), using 0 pairs.
DEBUG 2:
DEBUG 2:
--------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Processing UGRD/Z10 versus UGRD/Z10, for observation type ADPSFC,
over region FULL, for interpolation method BILIN(4), using 0 pairs.
DEBUG 2:
DEBUG 2:
--------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Processing VGRD/Z10 versus VGRD/Z10, for observation type ADPSFC,
over region FULL, for interpolation method BILIN(4), using 0 pairs.
DEBUG 2:
DEBUG 2:
--------------------------------------------------------------------------------
DEBUG 2:
DEBUG 2: Processing WIND/Z10 versus WIND/Z10, for observation type ADPSFC,
over region FULL, for interpolation method BILIN(4), using 0 pairs.
DEBUG 2:
DEBUG 2:
--------------------------------------------------------------------------------
DEBUG 2:
DEBUG 1: Output file:
./point_stat_ARWref_hrconus_experiment3_F01_ADPSFC_010000L_20180624_010000V.stat
kavulich at cheyenne5:/gpfs/fs1/p/ral/jntp/DAtask/kavulich/HRRR-2018/code/rapid-refresh/HRRR/bin/MET>
echo $?
0
It appears to be related to this change from several years ago:
http://mailman.ucar.edu/pipermail/met_help/2010-October/001434.html
Is there any runtime option to modify this behavior to exit with an error
if the file is not found? Or is my only option to modify the code myself
and recompile?
Michael Kavulich, Jr.
Associate Scientist, National Center for Atmospheric Research (NCAR)
Joint Numerical Testbed (JNT), Research Applications Laboratory (RAL)
*kavulich at ucar.edu <kavulich at ucar.edu>*
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Missing observation file for point_stat, but no error
From: Minna Win
Time: Wed May 01 16:20:32 2019
Hi Michael,
It looks like you are requesting that MET raise an error when there
are missing observation files. I'm assigning this ticket to John.
Regards,
Minna
------------------------------------------------
Subject: Missing observation file for point_stat, but no error
From: Minna Win
Time: Wed May 01 16:24:25 2019
Hi Michael,
It looks like you are requesting that MET raise an error whenever
there are
missing point stat obs files. I'm assigning this ticket to John.
Regards,
Minna
---------------
Minna Win
NCAR
Research Applications Lab
Phone: 303-497-8423
Fax: 303-497-8401
On Wed, May 1, 2019 at 9:54 PM Michael Kavulich via RT
<met_help at ucar.edu>
wrote:
>
> Wed May 01 15:53:47 2019: Request 90023 was acted upon.
> Transaction: Ticket created by kavulich at ucar.edu
> Queue: met_help
> Subject: Missing observation file for point_stat, but no error
> Owner: Nobody
> Requestors: kavulich at ucar.edu
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90023 >
>
>
> I've run into a problem with point_stat....I have some runs where I
have
> missing observation files (well actually, the files exist but for
some
> reason point_stat is occasionally not reading them at runtime...I
suspect
> this is a Cheyenne problem and not a MET problem but I can't be
sure), and
> rather than giving an error code, point_stat just continues and
exists with
> a success status message.
>
> WARNING:
>
> WARNING: process_obs_file() -> can't open observation netCDF file:
> /glade/scratch/kavulich/HRRR/2018/experiment3/run/2018062400/pb2nc/
> rap.2018062401.prepbufr.tm00.nc
>
> WARNING:
>
> DEBUG 2:
>
> DEBUG 2:
>
>
--------------------------------------------------------------------------------
>
> DEBUG 2:
>
> DEBUG 2: Processing TMP/Z2 versus TMP/Z2, for observation type
ADPSFC, over
> region FULL, for interpolation method BILIN(4), using 0 pairs.
>
> DEBUG 2:
>
> DEBUG 2:
>
>
--------------------------------------------------------------------------------
>
> DEBUG 2:
>
> DEBUG 2: Processing DPT/Z2 versus DPT/Z2, for observation type
ADPSFC, over
> region FULL, for interpolation method BILIN(4), using 0 pairs.
>
> DEBUG 2:
>
> DEBUG 2:
>
>
--------------------------------------------------------------------------------
>
> DEBUG 2:
>
> DEBUG 2: Processing SPFH/Z2 versus SPFH/Z2, for observation type
ADPSFC,
> over region FULL, for interpolation method BILIN(4), using 0 pairs.
>
> DEBUG 2:
>
> DEBUG 2:
>
>
--------------------------------------------------------------------------------
>
> DEBUG 2:
>
> DEBUG 2: Processing RH/Z2 versus RH/Z2, for observation type ADPSFC,
over
> region FULL, for interpolation method BILIN(4), using 0 pairs.
>
> DEBUG 2:
>
> DEBUG 2:
>
>
--------------------------------------------------------------------------------
>
> DEBUG 2:
>
> DEBUG 2: Processing UGRD/Z10 versus UGRD/Z10, for observation type
ADPSFC,
> over region FULL, for interpolation method BILIN(4), using 0 pairs.
>
> DEBUG 2:
>
> DEBUG 2:
>
>
--------------------------------------------------------------------------------
>
> DEBUG 2:
>
> DEBUG 2: Processing VGRD/Z10 versus VGRD/Z10, for observation type
ADPSFC,
> over region FULL, for interpolation method BILIN(4), using 0 pairs.
>
> DEBUG 2:
>
> DEBUG 2:
>
>
--------------------------------------------------------------------------------
>
> DEBUG 2:
>
> DEBUG 2: Processing WIND/Z10 versus WIND/Z10, for observation type
ADPSFC,
> over region FULL, for interpolation method BILIN(4), using 0 pairs.
>
> DEBUG 2:
>
> DEBUG 2:
>
>
--------------------------------------------------------------------------------
>
> DEBUG 2:
>
> DEBUG 1: Output file:
>
>
./point_stat_ARWref_hrconus_experiment3_F01_ADPSFC_010000L_20180624_010000V.stat
>
> kavulich at cheyenne5
> :/gpfs/fs1/p/ral/jntp/DAtask/kavulich/HRRR-2018/code/rapid-
refresh/HRRR/bin/MET>
> echo $?
>
> 0
>
> It appears to be related to this change from several years ago:
> http://mailman.ucar.edu/pipermail/met_help/2010-October/001434.html
>
> Is there any runtime option to modify this behavior to exit with an
error
> if the file is not found? Or is my only option to modify the code
myself
> and recompile?
>
> Michael Kavulich, Jr.
> Associate Scientist, National Center for Atmospheric Research (NCAR)
> Joint Numerical Testbed (JNT), Research Applications Laboratory
(RAL)
> *kavulich at ucar.edu <kavulich at ucar.edu>*
>
>
------------------------------------------------
Subject: Missing observation file for point_stat, but no error
From: Michael Kavulich
Time: Wed May 01 19:06:45 2019
That is correct. I actually discovered the source of the files not
being
able to be read: I accidentally had a job trying to write the
observation
file at the same time it was being read. But my question still stands
(though not urgent): is it possible to exit with an error message if
an
observation file can not be read in point_stat?
Thanks,
Michael Kavulich, Jr.
Associate Scientist, National Center for Atmospheric Research (NCAR)
Joint Numerical Testbed (JNT), Research Applications Laboratory (RAL)
*kavulich at ucar.edu <kavulich at ucar.edu>*
On Wed, May 1, 2019 at 4:24 PM Minna Win via RT <met_help at ucar.edu>
wrote:
> Hi Michael,
>
> It looks like you are requesting that MET raise an error whenever
there are
> missing point stat obs files. I'm assigning this ticket to John.
>
> Regards,
> Minna
> ---------------
> Minna Win
> NCAR
> Research Applications Lab
> Phone: 303-497-8423
> Fax: 303-497-8401
>
>
>
> On Wed, May 1, 2019 at 9:54 PM Michael Kavulich via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > Wed May 01 15:53:47 2019: Request 90023 was acted upon.
> > Transaction: Ticket created by kavulich at ucar.edu
> > Queue: met_help
> > Subject: Missing observation file for point_stat, but no
error
> > Owner: Nobody
> > Requestors: kavulich at ucar.edu
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90023 >
> >
> >
> > I've run into a problem with point_stat....I have some runs where
I have
> > missing observation files (well actually, the files exist but for
some
> > reason point_stat is occasionally not reading them at runtime...I
suspect
> > this is a Cheyenne problem and not a MET problem but I can't be
sure),
> and
> > rather than giving an error code, point_stat just continues and
exists
> with
> > a success status message.
> >
> > WARNING:
> >
> > WARNING: process_obs_file() -> can't open observation netCDF file:
> >
/glade/scratch/kavulich/HRRR/2018/experiment3/run/2018062400/pb2nc/
> > rap.2018062401.prepbufr.tm00.nc
> >
> > WARNING:
> >
> > DEBUG 2:
> >
> > DEBUG 2:
> >
> >
>
--------------------------------------------------------------------------------
> >
> > DEBUG 2:
> >
> > DEBUG 2: Processing TMP/Z2 versus TMP/Z2, for observation type
ADPSFC,
> over
> > region FULL, for interpolation method BILIN(4), using 0 pairs.
> >
> > DEBUG 2:
> >
> > DEBUG 2:
> >
> >
>
--------------------------------------------------------------------------------
> >
> > DEBUG 2:
> >
> > DEBUG 2: Processing DPT/Z2 versus DPT/Z2, for observation type
ADPSFC,
> over
> > region FULL, for interpolation method BILIN(4), using 0 pairs.
> >
> > DEBUG 2:
> >
> > DEBUG 2:
> >
> >
>
--------------------------------------------------------------------------------
> >
> > DEBUG 2:
> >
> > DEBUG 2: Processing SPFH/Z2 versus SPFH/Z2, for observation type
ADPSFC,
> > over region FULL, for interpolation method BILIN(4), using 0
pairs.
> >
> > DEBUG 2:
> >
> > DEBUG 2:
> >
> >
>
--------------------------------------------------------------------------------
> >
> > DEBUG 2:
> >
> > DEBUG 2: Processing RH/Z2 versus RH/Z2, for observation type
ADPSFC, over
> > region FULL, for interpolation method BILIN(4), using 0 pairs.
> >
> > DEBUG 2:
> >
> > DEBUG 2:
> >
> >
>
--------------------------------------------------------------------------------
> >
> > DEBUG 2:
> >
> > DEBUG 2: Processing UGRD/Z10 versus UGRD/Z10, for observation type
> ADPSFC,
> > over region FULL, for interpolation method BILIN(4), using 0
pairs.
> >
> > DEBUG 2:
> >
> > DEBUG 2:
> >
> >
>
--------------------------------------------------------------------------------
> >
> > DEBUG 2:
> >
> > DEBUG 2: Processing VGRD/Z10 versus VGRD/Z10, for observation type
> ADPSFC,
> > over region FULL, for interpolation method BILIN(4), using 0
pairs.
> >
> > DEBUG 2:
> >
> > DEBUG 2:
> >
> >
>
--------------------------------------------------------------------------------
> >
> > DEBUG 2:
> >
> > DEBUG 2: Processing WIND/Z10 versus WIND/Z10, for observation type
> ADPSFC,
> > over region FULL, for interpolation method BILIN(4), using 0
pairs.
> >
> > DEBUG 2:
> >
> > DEBUG 2:
> >
> >
>
--------------------------------------------------------------------------------
> >
> > DEBUG 2:
> >
> > DEBUG 1: Output file:
> >
> >
>
./point_stat_ARWref_hrconus_experiment3_F01_ADPSFC_010000L_20180624_010000V.stat
> >
> > kavulich at cheyenne5
> >
> :/gpfs/fs1/p/ral/jntp/DAtask/kavulich/HRRR-2018/code/rapid-
refresh/HRRR/bin/MET>
> > echo $?
> >
> > 0
> >
> > It appears to be related to this change from several years ago:
> > http://mailman.ucar.edu/pipermail/met_help/2010-
October/001434.html
> >
> > Is there any runtime option to modify this behavior to exit with
an error
> > if the file is not found? Or is my only option to modify the code
myself
> > and recompile?
> >
> > Michael Kavulich, Jr.
> > Associate Scientist, National Center for Atmospheric Research
(NCAR)
> > Joint Numerical Testbed (JNT), Research Applications Laboratory
(RAL)
> > *kavulich at ucar.edu <kavulich at ucar.edu>*
> >
> >
>
>
------------------------------------------------
Subject: Missing observation file for point_stat, but no error
From: John Halley Gotway
Time: Thu May 02 14:44:45 2019
Hi Mike,
There is one existing configuration option that you may find useful.
And
this was added in support of Jamie and Michelle's mesoscale vx runs.
//
// The "exit_on_warning" entry in ConfigConstants may be set to true
or
false.
// If set to true and a MET tool encounters a warning, it will
immediately
exit
// with bad status after writing the warning message.
//
exit_on_warning = FALSE;
You can just set exit_on_warning = TRUE; in your Point-Stat config
file and
it'll overwrite the default value of FALSE. This will cause
point_stat to
exit for *ANY* warning message, not just a missing file. Perhaps,
this
will get the job done for your current situation?
Perhaps a better, longer term, solution would be to define exit and
warning
codes for MET. Right now the tools always just exits with a return
value
of 1. But we could define classes of warning and error conditions.
And
then enhance exit_on_warning to be specified as a boolean (current
behavior) or as a list of integers... and the code would only exit if
one
of the specified warning codes was encountered.
exit_on_warning = [ 101, 103, 195 ];
It would take some effort to define these classes of errors/warning,
but do
you think that would be a useful enhancement? If so, we should define
a
new development issue to capture it.
Thanks,
John
On Wed, May 1, 2019 at 7:07 PM Michael Kavulich via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90023 >
>
> That is correct. I actually discovered the source of the files not
being
> able to be read: I accidentally had a job trying to write the
observation
> file at the same time it was being read. But my question still
stands
> (though not urgent): is it possible to exit with an error message if
an
> observation file can not be read in point_stat?
>
> Thanks,
>
> Michael Kavulich, Jr.
> Associate Scientist, National Center for Atmospheric Research (NCAR)
> Joint Numerical Testbed (JNT), Research Applications Laboratory
(RAL)
> *kavulich at ucar.edu <kavulich at ucar.edu>*
>
>
> On Wed, May 1, 2019 at 4:24 PM Minna Win via RT <met_help at ucar.edu>
wrote:
>
> > Hi Michael,
> >
> > It looks like you are requesting that MET raise an error whenever
there
> are
> > missing point stat obs files. I'm assigning this ticket to John.
> >
> > Regards,
> > Minna
> > ---------------
> > Minna Win
> > NCAR
> > Research Applications Lab
> > Phone: 303-497-8423
> > Fax: 303-497-8401
> >
> >
> >
> > On Wed, May 1, 2019 at 9:54 PM Michael Kavulich via RT <
> met_help at ucar.edu>
> > wrote:
> >
> > >
> > > Wed May 01 15:53:47 2019: Request 90023 was acted upon.
> > > Transaction: Ticket created by kavulich at ucar.edu
> > > Queue: met_help
> > > Subject: Missing observation file for point_stat, but no
error
> > > Owner: Nobody
> > > Requestors: kavulich at ucar.edu
> > > Status: new
> > > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=90023
> >
> > >
> > >
> > > I've run into a problem with point_stat....I have some runs
where I
> have
> > > missing observation files (well actually, the files exist but
for some
> > > reason point_stat is occasionally not reading them at
runtime...I
> suspect
> > > this is a Cheyenne problem and not a MET problem but I can't be
sure),
> > and
> > > rather than giving an error code, point_stat just continues and
exists
> > with
> > > a success status message.
> > >
> > > WARNING:
> > >
> > > WARNING: process_obs_file() -> can't open observation netCDF
file:
> > >
/glade/scratch/kavulich/HRRR/2018/experiment3/run/2018062400/pb2nc/
> > > rap.2018062401.prepbufr.tm00.nc
> > >
> > > WARNING:
> > >
> > > DEBUG 2:
> > >
> > > DEBUG 2:
> > >
> > >
> >
>
--------------------------------------------------------------------------------
> > >
> > > DEBUG 2:
> > >
> > > DEBUG 2: Processing TMP/Z2 versus TMP/Z2, for observation type
ADPSFC,
> > over
> > > region FULL, for interpolation method BILIN(4), using 0 pairs.
> > >
> > > DEBUG 2:
> > >
> > > DEBUG 2:
> > >
> > >
> >
>
--------------------------------------------------------------------------------
> > >
> > > DEBUG 2:
> > >
> > > DEBUG 2: Processing DPT/Z2 versus DPT/Z2, for observation type
ADPSFC,
> > over
> > > region FULL, for interpolation method BILIN(4), using 0 pairs.
> > >
> > > DEBUG 2:
> > >
> > > DEBUG 2:
> > >
> > >
> >
>
--------------------------------------------------------------------------------
> > >
> > > DEBUG 2:
> > >
> > > DEBUG 2: Processing SPFH/Z2 versus SPFH/Z2, for observation type
> ADPSFC,
> > > over region FULL, for interpolation method BILIN(4), using 0
pairs.
> > >
> > > DEBUG 2:
> > >
> > > DEBUG 2:
> > >
> > >
> >
>
--------------------------------------------------------------------------------
> > >
> > > DEBUG 2:
> > >
> > > DEBUG 2: Processing RH/Z2 versus RH/Z2, for observation type
ADPSFC,
> over
> > > region FULL, for interpolation method BILIN(4), using 0 pairs.
> > >
> > > DEBUG 2:
> > >
> > > DEBUG 2:
> > >
> > >
> >
>
--------------------------------------------------------------------------------
> > >
> > > DEBUG 2:
> > >
> > > DEBUG 2: Processing UGRD/Z10 versus UGRD/Z10, for observation
type
> > ADPSFC,
> > > over region FULL, for interpolation method BILIN(4), using 0
pairs.
> > >
> > > DEBUG 2:
> > >
> > > DEBUG 2:
> > >
> > >
> >
>
--------------------------------------------------------------------------------
> > >
> > > DEBUG 2:
> > >
> > > DEBUG 2: Processing VGRD/Z10 versus VGRD/Z10, for observation
type
> > ADPSFC,
> > > over region FULL, for interpolation method BILIN(4), using 0
pairs.
> > >
> > > DEBUG 2:
> > >
> > > DEBUG 2:
> > >
> > >
> >
>
--------------------------------------------------------------------------------
> > >
> > > DEBUG 2:
> > >
> > > DEBUG 2: Processing WIND/Z10 versus WIND/Z10, for observation
type
> > ADPSFC,
> > > over region FULL, for interpolation method BILIN(4), using 0
pairs.
> > >
> > > DEBUG 2:
> > >
> > > DEBUG 2:
> > >
> > >
> >
>
--------------------------------------------------------------------------------
> > >
> > > DEBUG 2:
> > >
> > > DEBUG 1: Output file:
> > >
> > >
> >
>
./point_stat_ARWref_hrconus_experiment3_F01_ADPSFC_010000L_20180624_010000V.stat
> > >
> > > kavulich at cheyenne5
> > >
> >
> :/gpfs/fs1/p/ral/jntp/DAtask/kavulich/HRRR-2018/code/rapid-
refresh/HRRR/bin/MET>
> > > echo $?
> > >
> > > 0
> > >
> > > It appears to be related to this change from several years ago:
> > > http://mailman.ucar.edu/pipermail/met_help/2010-
October/001434.html
> > >
> > > Is there any runtime option to modify this behavior to exit with
an
> error
> > > if the file is not found? Or is my only option to modify the
code
> myself
> > > and recompile?
> > >
> > > Michael Kavulich, Jr.
> > > Associate Scientist, National Center for Atmospheric Research
(NCAR)
> > > Joint Numerical Testbed (JNT), Research Applications Laboratory
(RAL)
> > > *kavulich at ucar.edu <kavulich at ucar.edu>*
> > >
> > >
> >
> >
>
>
------------------------------------------------
More information about the Met_help
mailing list