[Met_help] [rt.rap.ucar.edu #95701] History for METplus Gridstat Error
George McCabe via RT
met_help at ucar.edu
Mon Jun 22 19:19:49 MDT 2020
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hello,
I am just testing the functionality of METplus GridStat on my specific data
- never used this specific wrapper before.
I get an error in the time_util.py (L280) - the else statement of an
if-else statement for whether 'valid' or 'init' was set, from what I
understand. My 'loop_by' is set to valid as I am basically comparing two
OBS files. I've tried some various settings, but ultimately get the same
error each time.
I am working on Cheyenne and set up the environment as specified on the
existing builds page. I can also send these files via email if preferred.
My command is:
master_metplus.py -c /glade/work/hertneky/airforce/gswr/config/GridStat.conf
Output logs are here:
/glade/work/hertneky/airforce/gswr/output/logs
I can also send these files via email if preferred.
Thanks,
Tracy
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: METplus Gridstat Error
From: John Halley Gotway
Time: Mon Jun 22 16:47:05 2020
Tracy,
I took a look and agree that this is a sensible thing to do. You're
just
trying to loop over valid times, and for each, just run Grid-Stat to
compare two different observation datasets.
Seeing the following in the METplus conf file makes me suspicious:
*# If unset, defaults to 0 (don't loop through forecast leads)*
*LEAD_SEQ = 0*
In your case, we do want to loop through forecast leads... namely just
the
single forecast lead time of 0 hours!
I'll assign this ticket to George for him to take a closer look.
Thanks,
John
On Mon, Jun 22, 2020 at 4:17 PM Tracy Hertneky via RT
<met_help at ucar.edu>
wrote:
>
> Mon Jun 22 16:14:24 2020: Request 95701 was acted upon.
> Transaction: Ticket created by hertneky at ucar.edu
> Queue: met_help
> Subject: METplus Gridstat Error
> Owner: Nobody
> Requestors: hertneky at ucar.edu
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95701 >
>
>
> Hello,
>
> I am just testing the functionality of METplus GridStat on my
specific data
> - never used this specific wrapper before.
>
> I get an error in the time_util.py (L280) - the else statement of an
> if-else statement for whether 'valid' or 'init' was set, from what I
> understand. My 'loop_by' is set to valid as I am basically comparing
two
> OBS files. I've tried some various settings, but ultimately get the
same
> error each time.
>
> I am working on Cheyenne and set up the environment as specified on
the
> existing builds page. I can also send these files via email if
preferred.
>
> My command is:
>
> master_metplus.py -c
> /glade/work/hertneky/airforce/gswr/config/GridStat.conf
>
>
> Output logs are here:
>
> /glade/work/hertneky/airforce/gswr/output/logs
>
>
> I can also send these files via email if preferred.
>
>
> Thanks,
>
> Tracy
>
>
------------------------------------------------
Subject: METplus Gridstat Error
From: George McCabe
Time: Mon Jun 22 17:38:40 2020
Hey Tracy,
There are a couple things wrong with your configuration, but it
shouldn't
crash when you have it set up this way. I figured out a way to fix it
on
cheyenne, but strangely it is no longer working because it says it is
unable to open the grib input file:
ERROR : check_grib() -> unable to open input file
"/glade/p/ral/jntp/AF/gswr/sample_data_20190715/MITLL.GSWR.CompositeReflectivity.190715.001328.f0000.GR2"
I can't even run wgrib2 on it. I'm not sure what happened, because I
was
just able to get it to run correctly.
Regarding the config file, the file window values are in seconds, so
the
range you provided isn't big enough to find your data (within 15
minutes),
so you should change those to:
FCST_GRID_STAT_FILE_WINDOW_END = 15M
OBS_GRID_STAT_FILE_WINDOW_END = 15M
The reason it crashed is it couldn't extract enough time information
from
the files based on the template you gave it. It doesn't know the year,
so
it doesn't get any time information. Changing your template to:
FCST_GRID_STAT_INPUT_TEMPLATE =
MITLL.GSWR.CompositeReflectivity.{valid?fmt=%y%m%d}.{valid?fmt=%H%M%S}.f0000.GR2
(lower case %y is 2 digit year)
This succeeds for the first run (20190715 00Z) but fails for the
second
runtime (20190715 01Z) because there isn't data available for that
time.
However, I'm not able to get it to work anymore because of the issue
with
the input data not being able to be read for some reason. Do you have
any
idea why that is happening?
- George
On Mon, Jun 22, 2020 at 4:47 PM John Halley Gotway via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95701 >
>
> Tracy,
>
> I took a look and agree that this is a sensible thing to do. You're
just
> trying to loop over valid times, and for each, just run Grid-Stat to
> compare two different observation datasets.
>
> Seeing the following in the METplus conf file makes me suspicious:
>
> *# If unset, defaults to 0 (don't loop through forecast leads)*
>
> *LEAD_SEQ = 0*
>
> In your case, we do want to loop through forecast leads... namely
just the
> single forecast lead time of 0 hours!
>
> I'll assign this ticket to George for him to take a closer look.
>
> Thanks,
> John
>
> On Mon, Jun 22, 2020 at 4:17 PM Tracy Hertneky via RT
<met_help at ucar.edu>
> wrote:
>
> >
> > Mon Jun 22 16:14:24 2020: Request 95701 was acted upon.
> > Transaction: Ticket created by hertneky at ucar.edu
> > Queue: met_help
> > Subject: METplus Gridstat Error
> > Owner: Nobody
> > Requestors: hertneky at ucar.edu
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95701 >
> >
> >
> > Hello,
> >
> > I am just testing the functionality of METplus GridStat on my
specific
> data
> > - never used this specific wrapper before.
> >
> > I get an error in the time_util.py (L280) - the else statement of
an
> > if-else statement for whether 'valid' or 'init' was set, from what
I
> > understand. My 'loop_by' is set to valid as I am basically
comparing two
> > OBS files. I've tried some various settings, but ultimately get
the same
> > error each time.
> >
> > I am working on Cheyenne and set up the environment as specified
on the
> > existing builds page. I can also send these files via email if
preferred.
> >
> > My command is:
> >
> > master_metplus.py -c
> > /glade/work/hertneky/airforce/gswr/config/GridStat.conf
> >
> >
> > Output logs are here:
> >
> > /glade/work/hertneky/airforce/gswr/output/logs
> >
> >
> > I can also send these files via email if preferred.
> >
> >
> > Thanks,
> >
> > Tracy
> >
> >
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: METplus Gridstat Error
From: George McCabe
Time: Mon Jun 22 18:25:46 2020
Hey Tracy,
OK, so maybe I was never able to process that file, because I don't
have
permissions. I was able to build the command correctly, so those
config
changes I mentioned should allow you to process the first run time. I
made
some changes to the code to not crash and give more useful logging
info in
the case you discovered. If you want to test out the new version of
METplus
(in development, just a testing env I set up), it is on cheyenne here:
/glade/u/home/mccabe/METplus
On Mon, Jun 22, 2020 at 5:38 PM George McCabe <mccabe at ucar.edu> wrote:
> Hey Tracy,
>
> There are a couple things wrong with your configuration, but it
shouldn't
> crash when you have it set up this way. I figured out a way to fix
it on
> cheyenne, but strangely it is no longer working because it says it
is
> unable to open the grib input file:
>
> ERROR : check_grib() -> unable to open input file
>
"/glade/p/ral/jntp/AF/gswr/sample_data_20190715/MITLL.GSWR.CompositeReflectivity.190715.001328.f0000.GR2"
>
> I can't even run wgrib2 on it. I'm not sure what happened, because I
was
> just able to get it to run correctly.
>
> Regarding the config file, the file window values are in seconds, so
the
> range you provided isn't big enough to find your data (within 15
minutes),
> so you should change those to:
>
> FCST_GRID_STAT_FILE_WINDOW_END = 15M
> OBS_GRID_STAT_FILE_WINDOW_END = 15M
>
> The reason it crashed is it couldn't extract enough time information
from
> the files based on the template you gave it. It doesn't know the
year, so
> it doesn't get any time information. Changing your template to:
>
> FCST_GRID_STAT_INPUT_TEMPLATE =
>
MITLL.GSWR.CompositeReflectivity.{valid?fmt=%y%m%d}.{valid?fmt=%H%M%S}.f0000.GR2
>
> (lower case %y is 2 digit year)
>
> This succeeds for the first run (20190715 00Z) but fails for the
second
> runtime (20190715 01Z) because there isn't data available for that
time.
>
> However, I'm not able to get it to work anymore because of the issue
with
> the input data not being able to be read for some reason. Do you
have any
> idea why that is happening?
>
> - George
>
>
> On Mon, Jun 22, 2020 at 4:47 PM John Halley Gotway via RT <
> met_help at ucar.edu> wrote:
>
>>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95701 >
>>
>> Tracy,
>>
>> I took a look and agree that this is a sensible thing to do. You're
just
>> trying to loop over valid times, and for each, just run Grid-Stat
to
>> compare two different observation datasets.
>>
>> Seeing the following in the METplus conf file makes me suspicious:
>>
>> *# If unset, defaults to 0 (don't loop through forecast leads)*
>>
>> *LEAD_SEQ = 0*
>>
>> In your case, we do want to loop through forecast leads... namely
just the
>> single forecast lead time of 0 hours!
>>
>> I'll assign this ticket to George for him to take a closer look.
>>
>> Thanks,
>> John
>>
>> On Mon, Jun 22, 2020 at 4:17 PM Tracy Hertneky via RT
<met_help at ucar.edu>
>> wrote:
>>
>> >
>> > Mon Jun 22 16:14:24 2020: Request 95701 was acted upon.
>> > Transaction: Ticket created by hertneky at ucar.edu
>> > Queue: met_help
>> > Subject: METplus Gridstat Error
>> > Owner: Nobody
>> > Requestors: hertneky at ucar.edu
>> > Status: new
>> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95701 >
>> >
>> >
>> > Hello,
>> >
>> > I am just testing the functionality of METplus GridStat on my
specific
>> data
>> > - never used this specific wrapper before.
>> >
>> > I get an error in the time_util.py (L280) - the else statement of
an
>> > if-else statement for whether 'valid' or 'init' was set, from
what I
>> > understand. My 'loop_by' is set to valid as I am basically
comparing two
>> > OBS files. I've tried some various settings, but ultimately get
the same
>> > error each time.
>> >
>> > I am working on Cheyenne and set up the environment as specified
on the
>> > existing builds page. I can also send these files via email if
>> preferred.
>> >
>> > My command is:
>> >
>> > master_metplus.py -c
>> > /glade/work/hertneky/airforce/gswr/config/GridStat.conf
>> >
>> >
>> > Output logs are here:
>> >
>> > /glade/work/hertneky/airforce/gswr/output/logs
>> >
>> >
>> > I can also send these files via email if preferred.
>> >
>> >
>> > Thanks,
>> >
>> > Tracy
>> >
>> >
>>
>>
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: METplus Gridstat Error
From: Tracy Hertneky
Time: Mon Jun 22 19:06:50 2020
Thank you George!
I figured there would be more issues with my conf in running, but I
just
couldn't figure out what that 1st error meant.
The change you made in the error message is more clear in your build
(I had
meant 15 mins, not seconds).
Thanks also for the info on %y instead of %Y, now I know! Fixed that
as
well and another issue with the OBS level and it successfully
completed.
Thanks for the help! I would say this is resolved.
Tracy
On Mon, Jun 22, 2020 at 6:26 PM George McCabe via RT
<met_help at ucar.edu>
wrote:
> Hey Tracy,
>
> OK, so maybe I was never able to process that file, because I don't
have
> permissions. I was able to build the command correctly, so those
config
> changes I mentioned should allow you to process the first run time.
I made
> some changes to the code to not crash and give more useful logging
info in
> the case you discovered. If you want to test out the new version of
METplus
> (in development, just a testing env I set up), it is on cheyenne
here:
>
> /glade/u/home/mccabe/METplus
>
> On Mon, Jun 22, 2020 at 5:38 PM George McCabe <mccabe at ucar.edu>
wrote:
>
> > Hey Tracy,
> >
> > There are a couple things wrong with your configuration, but it
shouldn't
> > crash when you have it set up this way. I figured out a way to fix
it on
> > cheyenne, but strangely it is no longer working because it says it
is
> > unable to open the grib input file:
> >
> > ERROR : check_grib() -> unable to open input file
> >
>
"/glade/p/ral/jntp/AF/gswr/sample_data_20190715/MITLL.GSWR.CompositeReflectivity.190715.001328.f0000.GR2"
> >
> > I can't even run wgrib2 on it. I'm not sure what happened, because
I was
> > just able to get it to run correctly.
> >
> > Regarding the config file, the file window values are in seconds,
so the
> > range you provided isn't big enough to find your data (within 15
> minutes),
> > so you should change those to:
> >
> > FCST_GRID_STAT_FILE_WINDOW_END = 15M
> > OBS_GRID_STAT_FILE_WINDOW_END = 15M
> >
> > The reason it crashed is it couldn't extract enough time
information from
> > the files based on the template you gave it. It doesn't know the
year, so
> > it doesn't get any time information. Changing your template to:
> >
> > FCST_GRID_STAT_INPUT_TEMPLATE =
> >
>
MITLL.GSWR.CompositeReflectivity.{valid?fmt=%y%m%d}.{valid?fmt=%H%M%S}.f0000.GR2
> >
> > (lower case %y is 2 digit year)
> >
> > This succeeds for the first run (20190715 00Z) but fails for the
second
> > runtime (20190715 01Z) because there isn't data available for that
time.
> >
> > However, I'm not able to get it to work anymore because of the
issue with
> > the input data not being able to be read for some reason. Do you
have any
> > idea why that is happening?
> >
> > - George
> >
> >
> > On Mon, Jun 22, 2020 at 4:47 PM John Halley Gotway via RT <
> > met_help at ucar.edu> wrote:
> >
> >>
> >> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95701 >
> >>
> >> Tracy,
> >>
> >> I took a look and agree that this is a sensible thing to do.
You're just
> >> trying to loop over valid times, and for each, just run Grid-Stat
to
> >> compare two different observation datasets.
> >>
> >> Seeing the following in the METplus conf file makes me
suspicious:
> >>
> >> *# If unset, defaults to 0 (don't loop through forecast leads)*
> >>
> >> *LEAD_SEQ = 0*
> >>
> >> In your case, we do want to loop through forecast leads... namely
just
> the
> >> single forecast lead time of 0 hours!
> >>
> >> I'll assign this ticket to George for him to take a closer look.
> >>
> >> Thanks,
> >> John
> >>
> >> On Mon, Jun 22, 2020 at 4:17 PM Tracy Hertneky via RT <
> met_help at ucar.edu>
> >> wrote:
> >>
> >> >
> >> > Mon Jun 22 16:14:24 2020: Request 95701 was acted upon.
> >> > Transaction: Ticket created by hertneky at ucar.edu
> >> > Queue: met_help
> >> > Subject: METplus Gridstat Error
> >> > Owner: Nobody
> >> > Requestors: hertneky at ucar.edu
> >> > Status: new
> >> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=95701
> >
> >> >
> >> >
> >> > Hello,
> >> >
> >> > I am just testing the functionality of METplus GridStat on my
specific
> >> data
> >> > - never used this specific wrapper before.
> >> >
> >> > I get an error in the time_util.py (L280) - the else statement
of an
> >> > if-else statement for whether 'valid' or 'init' was set, from
what I
> >> > understand. My 'loop_by' is set to valid as I am basically
comparing
> two
> >> > OBS files. I've tried some various settings, but ultimately get
the
> same
> >> > error each time.
> >> >
> >> > I am working on Cheyenne and set up the environment as
specified on
> the
> >> > existing builds page. I can also send these files via email if
> >> preferred.
> >> >
> >> > My command is:
> >> >
> >> > master_metplus.py -c
> >> > /glade/work/hertneky/airforce/gswr/config/GridStat.conf
> >> >
> >> >
> >> > Output logs are here:
> >> >
> >> > /glade/work/hertneky/airforce/gswr/output/logs
> >> >
> >> >
> >> > I can also send these files via email if preferred.
> >> >
> >> >
> >> > Thanks,
> >> >
> >> > Tracy
> >> >
> >> >
> >>
> >>
> >
> > --
> > George McCabe - Software Engineer III
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > 303-497-2768
> > ---
> > My working day may not be your working day. Please do not feel
obliged to
> > reply to this email outside of your normal working hours.
> >
>
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>
------------------------------------------------
More information about the Met_help
mailing list