[Met_help] [rt.rap.ucar.edu #93886] History for Running RegridDataPlane wrapper for fcst and obs file
George McCabe via RT
met_help at ucar.edu
Mon Mar 2 09:23:16 MST 2020
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hi,
I've been running the GridStat METplus wrapper to verify 3 different
simulated reflectivity-related products from 7 different hi-res models.
Since I want to fairly compare the 7 models, I'm using a standard
verification grid.
I've realized that simply allowing the GridStat wrapper to regrid the
forecast and observation files is highly inefficient because regridding
occurs with every forecast lead/valid combination, regardless of whether
the obs or fcst file has already been regridded. So now, I'm trying to add
the RegridDataPlane wrapper to my process list, but it appears that I'm not
able to regrid a fcst file and an obs file simultaneously?
I get the following error (on the command line or in stderr of a job
script) when I try to run RegridDataPlane with both :
01/22 19:05:33.039 jlog (master_metplus.py:190) CRITICAL: master_metplus
failed: getdir() takes at most 3 arguments (4 given)
Traceback (most recent call last):
File
"/usrx/local/dev/packages/met/METplus/METplus-2.1/ush/master_metplus.py",
line 187, in <module>
main()
File
"/usrx/local/dev/packages/met/METplus/METplus-2.1/ush/master_metplus.py",
line 120, in main
item + "Wrapper")(p, logger)
File
"/gpfs/dell3/usrx/local/dev/packages/met/METplus/METplus-2.1/ush/regrid_data_plane_wrapper.py",
line 39, in __init__
self.create_c_dict()
File
"/gpfs/dell3/usrx/local/dev/packages/met/METplus/METplus-2.1/ush/regrid_data_plane_wrapper.py",
line 104, in create_c_dict
'', self.logger)
TypeError: getdir() takes at most 3 arguments (4 given)
01/22 19:05:33Z run-METplus-CRITICAL: master_metplus failed: getdir()
takes at most 3 arguments (4 given)
In this attempt, I have FCST_REGRID_DATA_PLANE_RUN and
OBS_REGRID_DATA_PLANE_RUN both set to True, and I have input and output
directories set for both fcst and obs. I tried setting FCST...RUN to false,
and I didn't get that error.
Is there any way I can run RegridDataPlane for both forecast and obs in the
same process list? Or do I need to regrid the obs files prior to the
METplus call that runs RegridDataPlane for the fcst files prior to running
GridStat?
Thanks,
Logan
--
*Logan C. Dawson, Ph.D.*
NOAA/NWS/NCEP/EMC
5830 University Research Court
College Park, MD 20740
(301) 683-3944
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Running RegridDataPlane wrapper for fcst and obs file
From: George McCabe
Time: Wed Jan 22 13:12:00 2020
Hi Logan,
It looks like there is a bug in running RegridDataPlane on FCST data
in
METplus 2.1. It appears to have been fixed in the latest beta release,
but
I am noticing there are no use cases examples that run RegridDataPlane
on
forecast data. Would you be willing to take the METplus 3.0-beta2
release
and test it with your use case to make sure that it does indeed work
correctly? If so, I can provide you with the beta release. You will
need to
have a Python version that is 3.6 or higher with the cartopy and
pandas
packages installed. If that is not available, I can give you
instructions
on how to set up a conda environment to test. Let me know what you
think.
Thanks,
George
On Wed, Jan 22, 2020 at 12:58 PM Logan Dawson - NOAA Affiliate via RT
<
met_help at ucar.edu> wrote:
>
> Wed Jan 22 12:58:46 2020: Request 93886 was acted upon.
> Transaction: Ticket created by logan.dawson at noaa.gov
> Queue: met_help
> Subject: Running RegridDataPlane wrapper for fcst and obs file
> Owner: Nobody
> Requestors: logan.dawson at noaa.gov
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886 >
>
>
> Hi,
>
> I've been running the GridStat METplus wrapper to verify 3 different
> simulated reflectivity-related products from 7 different hi-res
models.
> Since I want to fairly compare the 7 models, I'm using a standard
> verification grid.
>
> I've realized that simply allowing the GridStat wrapper to regrid
the
> forecast and observation files is highly inefficient because
regridding
> occurs with every forecast lead/valid combination, regardless of
whether
> the obs or fcst file has already been regridded. So now, I'm trying
to add
> the RegridDataPlane wrapper to my process list, but it appears that
I'm not
> able to regrid a fcst file and an obs file simultaneously?
>
> I get the following error (on the command line or in stderr of a job
> script) when I try to run RegridDataPlane with both :
>
> 01/22 19:05:33.039 jlog (master_metplus.py:190) CRITICAL:
master_metplus
> failed: getdir() takes at most 3 arguments (4 given)
>
> Traceback (most recent call last):
>
> File
> "/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/master_metplus.py",
> line 187, in <module>
>
> main()
>
> File
> "/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/master_metplus.py",
> line 120, in main
>
> item + "Wrapper")(p, logger)
>
> File
>
> "/gpfs/dell3/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/regrid_data_plane_wrapper.py",
> line 39, in __init__
>
> self.create_c_dict()
>
> File
>
> "/gpfs/dell3/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/regrid_data_plane_wrapper.py",
> line 104, in create_c_dict
>
> '', self.logger)
>
> TypeError: getdir() takes at most 3 arguments (4 given)
>
> 01/22 19:05:33Z run-METplus-CRITICAL: master_metplus failed:
getdir()
> takes at most 3 arguments (4 given)
>
> In this attempt, I have FCST_REGRID_DATA_PLANE_RUN and
> OBS_REGRID_DATA_PLANE_RUN both set to True, and I have input and
output
> directories set for both fcst and obs. I tried setting FCST...RUN to
false,
> and I didn't get that error.
>
> Is there any way I can run RegridDataPlane for both forecast and obs
in the
> same process list? Or do I need to regrid the obs files prior to the
> METplus call that runs RegridDataPlane for the fcst files prior to
running
> GridStat?
>
> Thanks,
> Logan
>
> --
> *Logan C. Dawson, Ph.D.*
> NOAA/NWS/NCEP/EMC
> 5830 University Research Court
> College Park, MD 20740
> (301) 683-3944
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: Running RegridDataPlane wrapper for fcst and obs file
From: Logan Dawson - NOAA Affiliate
Time: Wed Jan 22 13:18:24 2020
Hi George,
I'd be happy to test out the beta release. There's a Python 3.6.3
installation on Mars that has pandas and cartopy installed.
Thanks,
Logan
On Wed, Jan 22, 2020 at 3:12 PM George McCabe via RT
<met_help at ucar.edu>
wrote:
> Hi Logan,
>
> It looks like there is a bug in running RegridDataPlane on FCST data
in
> METplus 2.1. It appears to have been fixed in the latest beta
release, but
> I am noticing there are no use cases examples that run
RegridDataPlane on
> forecast data. Would you be willing to take the METplus 3.0-beta2
release
> and test it with your use case to make sure that it does indeed work
> correctly? If so, I can provide you with the beta release. You will
need to
> have a Python version that is 3.6 or higher with the cartopy and
pandas
> packages installed. If that is not available, I can give you
instructions
> on how to set up a conda environment to test. Let me know what you
think.
>
> Thanks,
> George
>
> On Wed, Jan 22, 2020 at 12:58 PM Logan Dawson - NOAA Affiliate via
RT <
> met_help at ucar.edu> wrote:
>
> >
> > Wed Jan 22 12:58:46 2020: Request 93886 was acted upon.
> > Transaction: Ticket created by logan.dawson at noaa.gov
> > Queue: met_help
> > Subject: Running RegridDataPlane wrapper for fcst and obs
file
> > Owner: Nobody
> > Requestors: logan.dawson at noaa.gov
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886 >
> >
> >
> > Hi,
> >
> > I've been running the GridStat METplus wrapper to verify 3
different
> > simulated reflectivity-related products from 7 different hi-res
models.
> > Since I want to fairly compare the 7 models, I'm using a standard
> > verification grid.
> >
> > I've realized that simply allowing the GridStat wrapper to regrid
the
> > forecast and observation files is highly inefficient because
regridding
> > occurs with every forecast lead/valid combination, regardless of
whether
> > the obs or fcst file has already been regridded. So now, I'm
trying to
> add
> > the RegridDataPlane wrapper to my process list, but it appears
that I'm
> not
> > able to regrid a fcst file and an obs file simultaneously?
> >
> > I get the following error (on the command line or in stderr of a
job
> > script) when I try to run RegridDataPlane with both :
> >
> > 01/22 19:05:33.039 jlog (master_metplus.py:190) CRITICAL:
master_metplus
> > failed: getdir() takes at most 3 arguments (4 given)
> >
> > Traceback (most recent call last):
> >
> > File
> > "/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/master_metplus.py",
> > line 187, in <module>
> >
> > main()
> >
> > File
> > "/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/master_metplus.py",
> > line 120, in main
> >
> > item + "Wrapper")(p, logger)
> >
> > File
> >
> >
> "/gpfs/dell3/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/regrid_data_plane_wrapper.py",
> > line 39, in __init__
> >
> > self.create_c_dict()
> >
> > File
> >
> >
> "/gpfs/dell3/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/regrid_data_plane_wrapper.py",
> > line 104, in create_c_dict
> >
> > '', self.logger)
> >
> > TypeError: getdir() takes at most 3 arguments (4 given)
> >
> > 01/22 19:05:33Z run-METplus-CRITICAL: master_metplus failed:
getdir()
> > takes at most 3 arguments (4 given)
> >
> > In this attempt, I have FCST_REGRID_DATA_PLANE_RUN and
> > OBS_REGRID_DATA_PLANE_RUN both set to True, and I have input and
output
> > directories set for both fcst and obs. I tried setting FCST...RUN
to
> false,
> > and I didn't get that error.
> >
> > Is there any way I can run RegridDataPlane for both forecast and
obs in
> the
> > same process list? Or do I need to regrid the obs files prior to
the
> > METplus call that runs RegridDataPlane for the fcst files prior to
> running
> > GridStat?
> >
> > Thanks,
> > Logan
> >
> > --
> > *Logan C. Dawson, Ph.D.*
> > NOAA/NWS/NCEP/EMC
> > 5830 University Research Court
> > College Park, MD 20740
> > (301) 683-3944
> >
> >
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>
--
*Logan C. Dawson, Ph.D.*
NOAA/NWS/NCEP/EMC
5830 University Research Court
College Park, MD 20740
(301) 683-3944
------------------------------------------------
Subject: Running RegridDataPlane wrapper for fcst and obs file
From: George McCabe
Time: Wed Jan 22 13:45:23 2020
Hi Logan,
Perfect, thank you!
The release can be found here:
https://github.com/NCAR/METplus/releases/tag/v3.0-beta2
You will have to download one of the source code tarballs and extract
the
files locally. There will likely be a few configuration variable name
changes involved in migrating your config files from METplus 2 to 3.
You
can either run master_metplus.py with all of the config files you use
passed in and it will report errors regarding the variables you need
to
change. You can also run the script validate_config.py the same way as
the
master script (passing in all configs with -c) and it will prompt you
to
confirm automatically replacing config variables with the updated
equivalent. Please let me know if you have any issues with this, or if
anything is unclear in this process. That will help refine the process
for
other users when the 3.0 release is available.
Thanks,
George
On Wed, Jan 22, 2020 at 1:18 PM Logan Dawson - NOAA Affiliate via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886 >
>
> Hi George,
>
> I'd be happy to test out the beta release. There's a Python 3.6.3
> installation on Mars that has pandas and cartopy installed.
>
> Thanks,
> Logan
>
> On Wed, Jan 22, 2020 at 3:12 PM George McCabe via RT
<met_help at ucar.edu>
> wrote:
>
> > Hi Logan,
> >
> > It looks like there is a bug in running RegridDataPlane on FCST
data in
> > METplus 2.1. It appears to have been fixed in the latest beta
release,
> but
> > I am noticing there are no use cases examples that run
RegridDataPlane on
> > forecast data. Would you be willing to take the METplus 3.0-beta2
release
> > and test it with your use case to make sure that it does indeed
work
> > correctly? If so, I can provide you with the beta release. You
will need
> to
> > have a Python version that is 3.6 or higher with the cartopy and
pandas
> > packages installed. If that is not available, I can give you
instructions
> > on how to set up a conda environment to test. Let me know what you
think.
> >
> > Thanks,
> > George
> >
> > On Wed, Jan 22, 2020 at 12:58 PM Logan Dawson - NOAA Affiliate via
RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > Wed Jan 22 12:58:46 2020: Request 93886 was acted upon.
> > > Transaction: Ticket created by logan.dawson at noaa.gov
> > > Queue: met_help
> > > Subject: Running RegridDataPlane wrapper for fcst and obs
file
> > > Owner: Nobody
> > > Requestors: logan.dawson at noaa.gov
> > > Status: new
> > > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886
> >
> > >
> > >
> > > Hi,
> > >
> > > I've been running the GridStat METplus wrapper to verify 3
different
> > > simulated reflectivity-related products from 7 different hi-res
models.
> > > Since I want to fairly compare the 7 models, I'm using a
standard
> > > verification grid.
> > >
> > > I've realized that simply allowing the GridStat wrapper to
regrid the
> > > forecast and observation files is highly inefficient because
regridding
> > > occurs with every forecast lead/valid combination, regardless of
> whether
> > > the obs or fcst file has already been regridded. So now, I'm
trying to
> > add
> > > the RegridDataPlane wrapper to my process list, but it appears
that I'm
> > not
> > > able to regrid a fcst file and an obs file simultaneously?
> > >
> > > I get the following error (on the command line or in stderr of a
job
> > > script) when I try to run RegridDataPlane with both :
> > >
> > > 01/22 19:05:33.039 jlog (master_metplus.py:190) CRITICAL:
> master_metplus
> > > failed: getdir() takes at most 3 arguments (4 given)
> > >
> > > Traceback (most recent call last):
> > >
> > > File
> > >
> "/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/master_metplus.py",
> > > line 187, in <module>
> > >
> > > main()
> > >
> > > File
> > >
> "/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/master_metplus.py",
> > > line 120, in main
> > >
> > > item + "Wrapper")(p, logger)
> > >
> > > File
> > >
> > >
> >
> "/gpfs/dell3/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/regrid_data_plane_wrapper.py",
> > > line 39, in __init__
> > >
> > > self.create_c_dict()
> > >
> > > File
> > >
> > >
> >
> "/gpfs/dell3/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/regrid_data_plane_wrapper.py",
> > > line 104, in create_c_dict
> > >
> > > '', self.logger)
> > >
> > > TypeError: getdir() takes at most 3 arguments (4 given)
> > >
> > > 01/22 19:05:33Z run-METplus-CRITICAL: master_metplus failed:
getdir()
> > > takes at most 3 arguments (4 given)
> > >
> > > In this attempt, I have FCST_REGRID_DATA_PLANE_RUN and
> > > OBS_REGRID_DATA_PLANE_RUN both set to True, and I have input and
output
> > > directories set for both fcst and obs. I tried setting
FCST...RUN to
> > false,
> > > and I didn't get that error.
> > >
> > > Is there any way I can run RegridDataPlane for both forecast and
obs in
> > the
> > > same process list? Or do I need to regrid the obs files prior to
the
> > > METplus call that runs RegridDataPlane for the fcst files prior
to
> > running
> > > GridStat?
> > >
> > > Thanks,
> > > Logan
> > >
> > > --
> > > *Logan C. Dawson, Ph.D.*
> > > NOAA/NWS/NCEP/EMC
> > > 5830 University Research Court
> > > College Park, MD 20740
> > > (301) 683-3944
> > >
> > >
> >
> > --
> > George McCabe - Software Engineer III
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > 303-497-2768
> > ---
> > My working day may not be your working day. Please do not feel
obliged to
> > reply to this email outside of your normal working hours.
> >
> >
>
> --
> *Logan C. Dawson, Ph.D.*
> NOAA/NWS/NCEP/EMC
> 5830 University Research Court
> College Park, MD 20740
> (301) 683-3944
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: Running RegridDataPlane wrapper for fcst and obs file
From: Logan Dawson - NOAA Affiliate
Time: Wed Jan 22 15:27:31 2020
Hi George,
Thanks for providing the location of the beta release and for letting
me
know about the validate_config.py script. I was able to get the beta
and
run the validation script. It was super simple, and I like the
functionality. I need about 6 or 7 config changes, and it was nice to
be
able to bypass manually making each change.
Also, I can confirm that running RegridDataPlane on forecast and
observation files is working as expected in this 3.0 beta release.
Thanks so much for the quick help on this issue! This is really going
to
help me speed up my verification jobs!
Thanks,
Logan
On Wed, Jan 22, 2020 at 3:45 PM George McCabe via RT
<met_help at ucar.edu>
wrote:
> Hi Logan,
>
> Perfect, thank you!
>
> The release can be found here:
> https://github.com/NCAR/METplus/releases/tag/v3.0-beta2
>
> You will have to download one of the source code tarballs and
extract the
> files locally. There will likely be a few configuration variable
name
> changes involved in migrating your config files from METplus 2 to 3.
You
> can either run master_metplus.py with all of the config files you
use
> passed in and it will report errors regarding the variables you need
to
> change. You can also run the script validate_config.py the same way
as the
> master script (passing in all configs with -c) and it will prompt
you to
> confirm automatically replacing config variables with the updated
> equivalent. Please let me know if you have any issues with this, or
if
> anything is unclear in this process. That will help refine the
process for
> other users when the 3.0 release is available.
>
> Thanks,
> George
>
> On Wed, Jan 22, 2020 at 1:18 PM Logan Dawson - NOAA Affiliate via RT
<
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886 >
> >
> > Hi George,
> >
> > I'd be happy to test out the beta release. There's a Python 3.6.3
> > installation on Mars that has pandas and cartopy installed.
> >
> > Thanks,
> > Logan
> >
> > On Wed, Jan 22, 2020 at 3:12 PM George McCabe via RT
<met_help at ucar.edu>
> > wrote:
> >
> > > Hi Logan,
> > >
> > > It looks like there is a bug in running RegridDataPlane on FCST
data in
> > > METplus 2.1. It appears to have been fixed in the latest beta
release,
> > but
> > > I am noticing there are no use cases examples that run
RegridDataPlane
> on
> > > forecast data. Would you be willing to take the METplus 3.0-
beta2
> release
> > > and test it with your use case to make sure that it does indeed
work
> > > correctly? If so, I can provide you with the beta release. You
will
> need
> > to
> > > have a Python version that is 3.6 or higher with the cartopy and
pandas
> > > packages installed. If that is not available, I can give you
> instructions
> > > on how to set up a conda environment to test. Let me know what
you
> think.
> > >
> > > Thanks,
> > > George
> > >
> > > On Wed, Jan 22, 2020 at 12:58 PM Logan Dawson - NOAA Affiliate
via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > Wed Jan 22 12:58:46 2020: Request 93886 was acted upon.
> > > > Transaction: Ticket created by logan.dawson at noaa.gov
> > > > Queue: met_help
> > > > Subject: Running RegridDataPlane wrapper for fcst and obs
file
> > > > Owner: Nobody
> > > > Requestors: logan.dawson at noaa.gov
> > > > Status: new
> > > > Ticket <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886
> > >
> > > >
> > > >
> > > > Hi,
> > > >
> > > > I've been running the GridStat METplus wrapper to verify 3
different
> > > > simulated reflectivity-related products from 7 different hi-
res
> models.
> > > > Since I want to fairly compare the 7 models, I'm using a
standard
> > > > verification grid.
> > > >
> > > > I've realized that simply allowing the GridStat wrapper to
regrid the
> > > > forecast and observation files is highly inefficient because
> regridding
> > > > occurs with every forecast lead/valid combination, regardless
of
> > whether
> > > > the obs or fcst file has already been regridded. So now, I'm
trying
> to
> > > add
> > > > the RegridDataPlane wrapper to my process list, but it appears
that
> I'm
> > > not
> > > > able to regrid a fcst file and an obs file simultaneously?
> > > >
> > > > I get the following error (on the command line or in stderr of
a job
> > > > script) when I try to run RegridDataPlane with both :
> > > >
> > > > 01/22 19:05:33.039 jlog (master_metplus.py:190) CRITICAL:
> > master_metplus
> > > > failed: getdir() takes at most 3 arguments (4 given)
> > > >
> > > > Traceback (most recent call last):
> > > >
> > > > File
> > > >
> > "/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/master_metplus.py",
> > > > line 187, in <module>
> > > >
> > > > main()
> > > >
> > > > File
> > > >
> > "/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/master_metplus.py",
> > > > line 120, in main
> > > >
> > > > item + "Wrapper")(p, logger)
> > > >
> > > > File
> > > >
> > > >
> > >
> >
> "/gpfs/dell3/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/regrid_data_plane_wrapper.py",
> > > > line 39, in __init__
> > > >
> > > > self.create_c_dict()
> > > >
> > > > File
> > > >
> > > >
> > >
> >
> "/gpfs/dell3/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/regrid_data_plane_wrapper.py",
> > > > line 104, in create_c_dict
> > > >
> > > > '', self.logger)
> > > >
> > > > TypeError: getdir() takes at most 3 arguments (4 given)
> > > >
> > > > 01/22 19:05:33Z run-METplus-CRITICAL: master_metplus failed:
> getdir()
> > > > takes at most 3 arguments (4 given)
> > > >
> > > > In this attempt, I have FCST_REGRID_DATA_PLANE_RUN and
> > > > OBS_REGRID_DATA_PLANE_RUN both set to True, and I have input
and
> output
> > > > directories set for both fcst and obs. I tried setting
FCST...RUN to
> > > false,
> > > > and I didn't get that error.
> > > >
> > > > Is there any way I can run RegridDataPlane for both forecast
and obs
> in
> > > the
> > > > same process list? Or do I need to regrid the obs files prior
to the
> > > > METplus call that runs RegridDataPlane for the fcst files
prior to
> > > running
> > > > GridStat?
> > > >
> > > > Thanks,
> > > > Logan
> > > >
> > > > --
> > > > *Logan C. Dawson, Ph.D.*
> > > > NOAA/NWS/NCEP/EMC
> > > > 5830 University Research Court
> > > > College Park, MD 20740
> > > > (301) 683-3944
> > > >
> > > >
> > >
> > > --
> > > George McCabe - Software Engineer III
> > > National Center for Atmospheric Research
> > > Research Applications Laboratory
> > > 303-497-2768
> > > ---
> > > My working day may not be your working day. Please do not feel
obliged
> to
> > > reply to this email outside of your normal working hours.
> > >
> > >
> >
> > --
> > *Logan C. Dawson, Ph.D.*
> > NOAA/NWS/NCEP/EMC
> > 5830 University Research Court
> > College Park, MD 20740
> > (301) 683-3944
> >
> >
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>
--
*Logan C. Dawson, Ph.D.*
NOAA/NWS/NCEP/EMC
5830 University Research Court
College Park, MD 20740
(301) 683-3944
------------------------------------------------
Subject: Running RegridDataPlane wrapper for fcst and obs file
From: George McCabe
Time: Wed Jan 22 16:22:49 2020
Great news to hear! I'm glad you were able to get the new version up
and
running quickly and it allowed you to set up your use case. I will
close
this ticket, but please feel free to submit another ticket if you have
any
other issues.
- George
On Wed, Jan 22, 2020 at 3:34 PM Logan Dawson - NOAA Affiliate via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886 >
>
> Hi George,
>
> Thanks for providing the location of the beta release and for
letting me
> know about the validate_config.py script. I was able to get the beta
and
> run the validation script. It was super simple, and I like the
> functionality. I need about 6 or 7 config changes, and it was nice
to be
> able to bypass manually making each change.
>
> Also, I can confirm that running RegridDataPlane on forecast and
> observation files is working as expected in this 3.0 beta release.
>
> Thanks so much for the quick help on this issue! This is really
going to
> help me speed up my verification jobs!
>
> Thanks,
> Logan
>
>
>
> On Wed, Jan 22, 2020 at 3:45 PM George McCabe via RT
<met_help at ucar.edu>
> wrote:
>
> > Hi Logan,
> >
> > Perfect, thank you!
> >
> > The release can be found here:
> > https://github.com/NCAR/METplus/releases/tag/v3.0-beta2
> >
> > You will have to download one of the source code tarballs and
extract the
> > files locally. There will likely be a few configuration variable
name
> > changes involved in migrating your config files from METplus 2 to
3. You
> > can either run master_metplus.py with all of the config files you
use
> > passed in and it will report errors regarding the variables you
need to
> > change. You can also run the script validate_config.py the same
way as
> the
> > master script (passing in all configs with -c) and it will prompt
you to
> > confirm automatically replacing config variables with the updated
> > equivalent. Please let me know if you have any issues with this,
or if
> > anything is unclear in this process. That will help refine the
process
> for
> > other users when the 3.0 release is available.
> >
> > Thanks,
> > George
> >
> > On Wed, Jan 22, 2020 at 1:18 PM Logan Dawson - NOAA Affiliate via
RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886 >
> > >
> > > Hi George,
> > >
> > > I'd be happy to test out the beta release. There's a Python
3.6.3
> > > installation on Mars that has pandas and cartopy installed.
> > >
> > > Thanks,
> > > Logan
> > >
> > > On Wed, Jan 22, 2020 at 3:12 PM George McCabe via RT <
> met_help at ucar.edu>
> > > wrote:
> > >
> > > > Hi Logan,
> > > >
> > > > It looks like there is a bug in running RegridDataPlane on
FCST data
> in
> > > > METplus 2.1. It appears to have been fixed in the latest beta
> release,
> > > but
> > > > I am noticing there are no use cases examples that run
> RegridDataPlane
> > on
> > > > forecast data. Would you be willing to take the METplus 3.0-
beta2
> > release
> > > > and test it with your use case to make sure that it does
indeed work
> > > > correctly? If so, I can provide you with the beta release. You
will
> > need
> > > to
> > > > have a Python version that is 3.6 or higher with the cartopy
and
> pandas
> > > > packages installed. If that is not available, I can give you
> > instructions
> > > > on how to set up a conda environment to test. Let me know what
you
> > think.
> > > >
> > > > Thanks,
> > > > George
> > > >
> > > > On Wed, Jan 22, 2020 at 12:58 PM Logan Dawson - NOAA Affiliate
via
> RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > Wed Jan 22 12:58:46 2020: Request 93886 was acted upon.
> > > > > Transaction: Ticket created by logan.dawson at noaa.gov
> > > > > Queue: met_help
> > > > > Subject: Running RegridDataPlane wrapper for fcst and
obs file
> > > > > Owner: Nobody
> > > > > Requestors: logan.dawson at noaa.gov
> > > > > Status: new
> > > > > Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886
> > > >
> > > > >
> > > > >
> > > > > Hi,
> > > > >
> > > > > I've been running the GridStat METplus wrapper to verify 3
> different
> > > > > simulated reflectivity-related products from 7 different hi-
res
> > models.
> > > > > Since I want to fairly compare the 7 models, I'm using a
standard
> > > > > verification grid.
> > > > >
> > > > > I've realized that simply allowing the GridStat wrapper to
regrid
> the
> > > > > forecast and observation files is highly inefficient because
> > regridding
> > > > > occurs with every forecast lead/valid combination,
regardless of
> > > whether
> > > > > the obs or fcst file has already been regridded. So now, I'm
trying
> > to
> > > > add
> > > > > the RegridDataPlane wrapper to my process list, but it
appears that
> > I'm
> > > > not
> > > > > able to regrid a fcst file and an obs file simultaneously?
> > > > >
> > > > > I get the following error (on the command line or in stderr
of a
> job
> > > > > script) when I try to run RegridDataPlane with both :
> > > > >
> > > > > 01/22 19:05:33.039 jlog (master_metplus.py:190) CRITICAL:
> > > master_metplus
> > > > > failed: getdir() takes at most 3 arguments (4 given)
> > > > >
> > > > > Traceback (most recent call last):
> > > > >
> > > > > File
> > > > >
> > >
> "/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/master_metplus.py",
> > > > > line 187, in <module>
> > > > >
> > > > > main()
> > > > >
> > > > > File
> > > > >
> > >
> "/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/master_metplus.py",
> > > > > line 120, in main
> > > > >
> > > > > item + "Wrapper")(p, logger)
> > > > >
> > > > > File
> > > > >
> > > > >
> > > >
> > >
> >
> "/gpfs/dell3/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/regrid_data_plane_wrapper.py",
> > > > > line 39, in __init__
> > > > >
> > > > > self.create_c_dict()
> > > > >
> > > > > File
> > > > >
> > > > >
> > > >
> > >
> >
> "/gpfs/dell3/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/regrid_data_plane_wrapper.py",
> > > > > line 104, in create_c_dict
> > > > >
> > > > > '', self.logger)
> > > > >
> > > > > TypeError: getdir() takes at most 3 arguments (4 given)
> > > > >
> > > > > 01/22 19:05:33Z run-METplus-CRITICAL: master_metplus
failed:
> > getdir()
> > > > > takes at most 3 arguments (4 given)
> > > > >
> > > > > In this attempt, I have FCST_REGRID_DATA_PLANE_RUN and
> > > > > OBS_REGRID_DATA_PLANE_RUN both set to True, and I have input
and
> > output
> > > > > directories set for both fcst and obs. I tried setting
FCST...RUN
> to
> > > > false,
> > > > > and I didn't get that error.
> > > > >
> > > > > Is there any way I can run RegridDataPlane for both forecast
and
> obs
> > in
> > > > the
> > > > > same process list? Or do I need to regrid the obs files
prior to
> the
> > > > > METplus call that runs RegridDataPlane for the fcst files
prior to
> > > > running
> > > > > GridStat?
> > > > >
> > > > > Thanks,
> > > > > Logan
> > > > >
> > > > > --
> > > > > *Logan C. Dawson, Ph.D.*
> > > > > NOAA/NWS/NCEP/EMC
> > > > > 5830 University Research Court
> > > > > College Park, MD 20740
> > > > > (301) 683-3944
> > > > >
> > > > >
> > > >
> > > > --
> > > > George McCabe - Software Engineer III
> > > > National Center for Atmospheric Research
> > > > Research Applications Laboratory
> > > > 303-497-2768
> > > > ---
> > > > My working day may not be your working day. Please do not feel
> obliged
> > to
> > > > reply to this email outside of your normal working hours.
> > > >
> > > >
> > >
> > > --
> > > *Logan C. Dawson, Ph.D.*
> > > NOAA/NWS/NCEP/EMC
> > > 5830 University Research Court
> > > College Park, MD 20740
> > > (301) 683-3944
> > >
> > >
> >
> > --
> > George McCabe - Software Engineer III
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > 303-497-2768
> > ---
> > My working day may not be your working day. Please do not feel
obliged to
> > reply to this email outside of your normal working hours.
> >
> >
>
> --
> *Logan C. Dawson, Ph.D.*
> NOAA/NWS/NCEP/EMC
> 5830 University Research Court
> College Park, MD 20740
> (301) 683-3944
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: Running RegridDataPlane wrapper for fcst and obs file
From: Logan Dawson - NOAA Affiliate
Time: Wed Jan 22 16:28:24 2020
Hi George,
One quick follow-up that you're probably already aware of... I just
wanted
to let you know that the validate_config.py script doesn't appear to
be
catching deprecated configs associated with the StatAnalysis wrapper.
The script okayed my conf file associated with the StatAnalysis job,
but
I'm catching a few config items that appear to be deprecated or new
ones
that need to be defined. I can work through them and give you a list
tomorrow (or on the METplus call on Monday).
I'm able to roll back to METplus 2.1 to run the StatAnalysi job I need
to
run in the interim.
Thanks,
Logan
On Wed, Jan 22, 2020 at 6:22 PM George McCabe via RT
<met_help at ucar.edu>
wrote:
> Great news to hear! I'm glad you were able to get the new version up
and
> running quickly and it allowed you to set up your use case. I will
close
> this ticket, but please feel free to submit another ticket if you
have any
> other issues.
>
> - George
>
> On Wed, Jan 22, 2020 at 3:34 PM Logan Dawson - NOAA Affiliate via RT
<
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886 >
> >
> > Hi George,
> >
> > Thanks for providing the location of the beta release and for
letting me
> > know about the validate_config.py script. I was able to get the
beta and
> > run the validation script. It was super simple, and I like the
> > functionality. I need about 6 or 7 config changes, and it was nice
to be
> > able to bypass manually making each change.
> >
> > Also, I can confirm that running RegridDataPlane on forecast and
> > observation files is working as expected in this 3.0 beta release.
> >
> > Thanks so much for the quick help on this issue! This is really
going to
> > help me speed up my verification jobs!
> >
> > Thanks,
> > Logan
> >
> >
> >
> > On Wed, Jan 22, 2020 at 3:45 PM George McCabe via RT
<met_help at ucar.edu>
> > wrote:
> >
> > > Hi Logan,
> > >
> > > Perfect, thank you!
> > >
> > > The release can be found here:
> > > https://github.com/NCAR/METplus/releases/tag/v3.0-beta2
> > >
> > > You will have to download one of the source code tarballs and
extract
> the
> > > files locally. There will likely be a few configuration variable
name
> > > changes involved in migrating your config files from METplus 2
to 3.
> You
> > > can either run master_metplus.py with all of the config files
you use
> > > passed in and it will report errors regarding the variables you
need to
> > > change. You can also run the script validate_config.py the same
way as
> > the
> > > master script (passing in all configs with -c) and it will
prompt you
> to
> > > confirm automatically replacing config variables with the
updated
> > > equivalent. Please let me know if you have any issues with this,
or if
> > > anything is unclear in this process. That will help refine the
process
> > for
> > > other users when the 3.0 release is available.
> > >
> > > Thanks,
> > > George
> > >
> > > On Wed, Jan 22, 2020 at 1:18 PM Logan Dawson - NOAA Affiliate
via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886
>
> > > >
> > > > Hi George,
> > > >
> > > > I'd be happy to test out the beta release. There's a Python
3.6.3
> > > > installation on Mars that has pandas and cartopy installed.
> > > >
> > > > Thanks,
> > > > Logan
> > > >
> > > > On Wed, Jan 22, 2020 at 3:12 PM George McCabe via RT <
> > met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > > Hi Logan,
> > > > >
> > > > > It looks like there is a bug in running RegridDataPlane on
FCST
> data
> > in
> > > > > METplus 2.1. It appears to have been fixed in the latest
beta
> > release,
> > > > but
> > > > > I am noticing there are no use cases examples that run
> > RegridDataPlane
> > > on
> > > > > forecast data. Would you be willing to take the METplus 3.0-
beta2
> > > release
> > > > > and test it with your use case to make sure that it does
indeed
> work
> > > > > correctly? If so, I can provide you with the beta release.
You will
> > > need
> > > > to
> > > > > have a Python version that is 3.6 or higher with the cartopy
and
> > pandas
> > > > > packages installed. If that is not available, I can give you
> > > instructions
> > > > > on how to set up a conda environment to test. Let me know
what you
> > > think.
> > > > >
> > > > > Thanks,
> > > > > George
> > > > >
> > > > > On Wed, Jan 22, 2020 at 12:58 PM Logan Dawson - NOAA
Affiliate via
> > RT <
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > > >
> > > > > > Wed Jan 22 12:58:46 2020: Request 93886 was acted upon.
> > > > > > Transaction: Ticket created by logan.dawson at noaa.gov
> > > > > > Queue: met_help
> > > > > > Subject: Running RegridDataPlane wrapper for fcst and
obs
> file
> > > > > > Owner: Nobody
> > > > > > Requestors: logan.dawson at noaa.gov
> > > > > > Status: new
> > > > > > Ticket <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886
> > > > >
> > > > > >
> > > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I've been running the GridStat METplus wrapper to verify 3
> > different
> > > > > > simulated reflectivity-related products from 7 different
hi-res
> > > models.
> > > > > > Since I want to fairly compare the 7 models, I'm using a
standard
> > > > > > verification grid.
> > > > > >
> > > > > > I've realized that simply allowing the GridStat wrapper to
regrid
> > the
> > > > > > forecast and observation files is highly inefficient
because
> > > regridding
> > > > > > occurs with every forecast lead/valid combination,
regardless of
> > > > whether
> > > > > > the obs or fcst file has already been regridded. So now,
I'm
> trying
> > > to
> > > > > add
> > > > > > the RegridDataPlane wrapper to my process list, but it
appears
> that
> > > I'm
> > > > > not
> > > > > > able to regrid a fcst file and an obs file simultaneously?
> > > > > >
> > > > > > I get the following error (on the command line or in
stderr of a
> > job
> > > > > > script) when I try to run RegridDataPlane with both :
> > > > > >
> > > > > > 01/22 19:05:33.039 jlog (master_metplus.py:190) CRITICAL:
> > > > master_metplus
> > > > > > failed: getdir() takes at most 3 arguments (4 given)
> > > > > >
> > > > > > Traceback (most recent call last):
> > > > > >
> > > > > > File
> > > > > >
> > > >
> > "/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/master_metplus.py",
> > > > > > line 187, in <module>
> > > > > >
> > > > > > main()
> > > > > >
> > > > > > File
> > > > > >
> > > >
> > "/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/master_metplus.py",
> > > > > > line 120, in main
> > > > > >
> > > > > > item + "Wrapper")(p, logger)
> > > > > >
> > > > > > File
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> "/gpfs/dell3/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/regrid_data_plane_wrapper.py",
> > > > > > line 39, in __init__
> > > > > >
> > > > > > self.create_c_dict()
> > > > > >
> > > > > > File
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> "/gpfs/dell3/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/regrid_data_plane_wrapper.py",
> > > > > > line 104, in create_c_dict
> > > > > >
> > > > > > '', self.logger)
> > > > > >
> > > > > > TypeError: getdir() takes at most 3 arguments (4 given)
> > > > > >
> > > > > > 01/22 19:05:33Z run-METplus-CRITICAL: master_metplus
failed:
> > > getdir()
> > > > > > takes at most 3 arguments (4 given)
> > > > > >
> > > > > > In this attempt, I have FCST_REGRID_DATA_PLANE_RUN and
> > > > > > OBS_REGRID_DATA_PLANE_RUN both set to True, and I have
input and
> > > output
> > > > > > directories set for both fcst and obs. I tried setting
FCST...RUN
> > to
> > > > > false,
> > > > > > and I didn't get that error.
> > > > > >
> > > > > > Is there any way I can run RegridDataPlane for both
forecast and
> > obs
> > > in
> > > > > the
> > > > > > same process list? Or do I need to regrid the obs files
prior to
> > the
> > > > > > METplus call that runs RegridDataPlane for the fcst files
prior
> to
> > > > > running
> > > > > > GridStat?
> > > > > >
> > > > > > Thanks,
> > > > > > Logan
> > > > > >
> > > > > > --
> > > > > > *Logan C. Dawson, Ph.D.*
> > > > > > NOAA/NWS/NCEP/EMC
> > > > > > 5830 University Research Court
> > > > > > College Park, MD 20740
> > > > > > (301) 683-3944
> > > > > >
> > > > > >
> > > > >
> > > > > --
> > > > > George McCabe - Software Engineer III
> > > > > National Center for Atmospheric Research
> > > > > Research Applications Laboratory
> > > > > 303-497-2768
> > > > > ---
> > > > > My working day may not be your working day. Please do not
feel
> > obliged
> > > to
> > > > > reply to this email outside of your normal working hours.
> > > > >
> > > > >
> > > >
> > > > --
> > > > *Logan C. Dawson, Ph.D.*
> > > > NOAA/NWS/NCEP/EMC
> > > > 5830 University Research Court
> > > > College Park, MD 20740
> > > > (301) 683-3944
> > > >
> > > >
> > >
> > > --
> > > George McCabe - Software Engineer III
> > > National Center for Atmospheric Research
> > > Research Applications Laboratory
> > > 303-497-2768
> > > ---
> > > My working day may not be your working day. Please do not feel
obliged
> to
> > > reply to this email outside of your normal working hours.
> > >
> > >
> >
> > --
> > *Logan C. Dawson, Ph.D.*
> > NOAA/NWS/NCEP/EMC
> > 5830 University Research Court
> > College Park, MD 20740
> > (301) 683-3944
> >
> >
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>
--
*Logan C. Dawson, Ph.D.*
NOAA/NWS/NCEP/EMC
5830 University Research Court
College Park, MD 20740
(301) 683-3944
------------------------------------------------
Subject: Running RegridDataPlane wrapper for fcst and obs file
From: George McCabe
Time: Wed Jan 22 16:37:20 2020
Hi Logan,
I was not aware of that the StatAnalysis wrapper had deprecated
configs
that are not on the list. If you wouldn't mind putting together a list
that
would be very helpful. Thanks for bringing this to my attention.
On Wed, Jan 22, 2020 at 4:28 PM Logan Dawson - NOAA Affiliate via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886 >
>
> Hi George,
>
> One quick follow-up that you're probably already aware of... I just
wanted
> to let you know that the validate_config.py script doesn't appear to
be
> catching deprecated configs associated with the StatAnalysis
wrapper.
>
> The script okayed my conf file associated with the StatAnalysis job,
but
> I'm catching a few config items that appear to be deprecated or new
ones
> that need to be defined. I can work through them and give you a list
> tomorrow (or on the METplus call on Monday).
>
> I'm able to roll back to METplus 2.1 to run the StatAnalysi job I
need to
> run in the interim.
>
> Thanks,
> Logan
>
>
>
> On Wed, Jan 22, 2020 at 6:22 PM George McCabe via RT
<met_help at ucar.edu>
> wrote:
>
> > Great news to hear! I'm glad you were able to get the new version
up and
> > running quickly and it allowed you to set up your use case. I will
close
> > this ticket, but please feel free to submit another ticket if you
have
> any
> > other issues.
> >
> > - George
> >
> > On Wed, Jan 22, 2020 at 3:34 PM Logan Dawson - NOAA Affiliate via
RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886 >
> > >
> > > Hi George,
> > >
> > > Thanks for providing the location of the beta release and for
letting
> me
> > > know about the validate_config.py script. I was able to get the
beta
> and
> > > run the validation script. It was super simple, and I like the
> > > functionality. I need about 6 or 7 config changes, and it was
nice to
> be
> > > able to bypass manually making each change.
> > >
> > > Also, I can confirm that running RegridDataPlane on forecast and
> > > observation files is working as expected in this 3.0 beta
release.
> > >
> > > Thanks so much for the quick help on this issue! This is really
going
> to
> > > help me speed up my verification jobs!
> > >
> > > Thanks,
> > > Logan
> > >
> > >
> > >
> > > On Wed, Jan 22, 2020 at 3:45 PM George McCabe via RT <
> met_help at ucar.edu>
> > > wrote:
> > >
> > > > Hi Logan,
> > > >
> > > > Perfect, thank you!
> > > >
> > > > The release can be found here:
> > > > https://github.com/NCAR/METplus/releases/tag/v3.0-beta2
> > > >
> > > > You will have to download one of the source code tarballs and
extract
> > the
> > > > files locally. There will likely be a few configuration
variable name
> > > > changes involved in migrating your config files from METplus 2
to 3.
> > You
> > > > can either run master_metplus.py with all of the config files
you use
> > > > passed in and it will report errors regarding the variables
you need
> to
> > > > change. You can also run the script validate_config.py the
same way
> as
> > > the
> > > > master script (passing in all configs with -c) and it will
prompt you
> > to
> > > > confirm automatically replacing config variables with the
updated
> > > > equivalent. Please let me know if you have any issues with
this, or
> if
> > > > anything is unclear in this process. That will help refine the
> process
> > > for
> > > > other users when the 3.0 release is available.
> > > >
> > > > Thanks,
> > > > George
> > > >
> > > > On Wed, Jan 22, 2020 at 1:18 PM Logan Dawson - NOAA Affiliate
via RT
> <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886 >
> > > > >
> > > > > Hi George,
> > > > >
> > > > > I'd be happy to test out the beta release. There's a Python
3.6.3
> > > > > installation on Mars that has pandas and cartopy installed.
> > > > >
> > > > > Thanks,
> > > > > Logan
> > > > >
> > > > > On Wed, Jan 22, 2020 at 3:12 PM George McCabe via RT <
> > > met_help at ucar.edu>
> > > > > wrote:
> > > > >
> > > > > > Hi Logan,
> > > > > >
> > > > > > It looks like there is a bug in running RegridDataPlane on
FCST
> > data
> > > in
> > > > > > METplus 2.1. It appears to have been fixed in the latest
beta
> > > release,
> > > > > but
> > > > > > I am noticing there are no use cases examples that run
> > > RegridDataPlane
> > > > on
> > > > > > forecast data. Would you be willing to take the METplus
3.0-beta2
> > > > release
> > > > > > and test it with your use case to make sure that it does
indeed
> > work
> > > > > > correctly? If so, I can provide you with the beta release.
You
> will
> > > > need
> > > > > to
> > > > > > have a Python version that is 3.6 or higher with the
cartopy and
> > > pandas
> > > > > > packages installed. If that is not available, I can give
you
> > > > instructions
> > > > > > on how to set up a conda environment to test. Let me know
what
> you
> > > > think.
> > > > > >
> > > > > > Thanks,
> > > > > > George
> > > > > >
> > > > > > On Wed, Jan 22, 2020 at 12:58 PM Logan Dawson - NOAA
Affiliate
> via
> > > RT <
> > > > > > met_help at ucar.edu> wrote:
> > > > > >
> > > > > > >
> > > > > > > Wed Jan 22 12:58:46 2020: Request 93886 was acted upon.
> > > > > > > Transaction: Ticket created by logan.dawson at noaa.gov
> > > > > > > Queue: met_help
> > > > > > > Subject: Running RegridDataPlane wrapper for fcst
and obs
> > file
> > > > > > > Owner: Nobody
> > > > > > > Requestors: logan.dawson at noaa.gov
> > > > > > > Status: new
> > > > > > > Ticket <URL:
> > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886
> > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Hi,
> > > > > > >
> > > > > > > I've been running the GridStat METplus wrapper to verify
3
> > > different
> > > > > > > simulated reflectivity-related products from 7 different
hi-res
> > > > models.
> > > > > > > Since I want to fairly compare the 7 models, I'm using a
> standard
> > > > > > > verification grid.
> > > > > > >
> > > > > > > I've realized that simply allowing the GridStat wrapper
to
> regrid
> > > the
> > > > > > > forecast and observation files is highly inefficient
because
> > > > regridding
> > > > > > > occurs with every forecast lead/valid combination,
regardless
> of
> > > > > whether
> > > > > > > the obs or fcst file has already been regridded. So now,
I'm
> > trying
> > > > to
> > > > > > add
> > > > > > > the RegridDataPlane wrapper to my process list, but it
appears
> > that
> > > > I'm
> > > > > > not
> > > > > > > able to regrid a fcst file and an obs file
simultaneously?
> > > > > > >
> > > > > > > I get the following error (on the command line or in
stderr of
> a
> > > job
> > > > > > > script) when I try to run RegridDataPlane with both :
> > > > > > >
> > > > > > > 01/22 19:05:33.039 jlog (master_metplus.py:190)
CRITICAL:
> > > > > master_metplus
> > > > > > > failed: getdir() takes at most 3 arguments (4 given)
> > > > > > >
> > > > > > > Traceback (most recent call last):
> > > > > > >
> > > > > > > File
> > > > > > >
> > > > >
> > >
> "/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/master_metplus.py",
> > > > > > > line 187, in <module>
> > > > > > >
> > > > > > > main()
> > > > > > >
> > > > > > > File
> > > > > > >
> > > > >
> > >
> "/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/master_metplus.py",
> > > > > > > line 120, in main
> > > > > > >
> > > > > > > item + "Wrapper")(p, logger)
> > > > > > >
> > > > > > > File
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> "/gpfs/dell3/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/regrid_data_plane_wrapper.py",
> > > > > > > line 39, in __init__
> > > > > > >
> > > > > > > self.create_c_dict()
> > > > > > >
> > > > > > > File
> > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> "/gpfs/dell3/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/regrid_data_plane_wrapper.py",
> > > > > > > line 104, in create_c_dict
> > > > > > >
> > > > > > > '', self.logger)
> > > > > > >
> > > > > > > TypeError: getdir() takes at most 3 arguments (4 given)
> > > > > > >
> > > > > > > 01/22 19:05:33Z run-METplus-CRITICAL: master_metplus
failed:
> > > > getdir()
> > > > > > > takes at most 3 arguments (4 given)
> > > > > > >
> > > > > > > In this attempt, I have FCST_REGRID_DATA_PLANE_RUN and
> > > > > > > OBS_REGRID_DATA_PLANE_RUN both set to True, and I have
input
> and
> > > > output
> > > > > > > directories set for both fcst and obs. I tried setting
> FCST...RUN
> > > to
> > > > > > false,
> > > > > > > and I didn't get that error.
> > > > > > >
> > > > > > > Is there any way I can run RegridDataPlane for both
forecast
> and
> > > obs
> > > > in
> > > > > > the
> > > > > > > same process list? Or do I need to regrid the obs files
prior
> to
> > > the
> > > > > > > METplus call that runs RegridDataPlane for the fcst
files prior
> > to
> > > > > > running
> > > > > > > GridStat?
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Logan
> > > > > > >
> > > > > > > --
> > > > > > > *Logan C. Dawson, Ph.D.*
> > > > > > > NOAA/NWS/NCEP/EMC
> > > > > > > 5830 University Research Court
> > > > > > > College Park, MD 20740
> > > > > > > (301) 683-3944
> > > > > > >
> > > > > > >
> > > > > >
> > > > > > --
> > > > > > George McCabe - Software Engineer III
> > > > > > National Center for Atmospheric Research
> > > > > > Research Applications Laboratory
> > > > > > 303-497-2768
> > > > > > ---
> > > > > > My working day may not be your working day. Please do not
feel
> > > obliged
> > > > to
> > > > > > reply to this email outside of your normal working hours.
> > > > > >
> > > > > >
> > > > >
> > > > > --
> > > > > *Logan C. Dawson, Ph.D.*
> > > > > NOAA/NWS/NCEP/EMC
> > > > > 5830 University Research Court
> > > > > College Park, MD 20740
> > > > > (301) 683-3944
> > > > >
> > > > >
> > > >
> > > > --
> > > > George McCabe - Software Engineer III
> > > > National Center for Atmospheric Research
> > > > Research Applications Laboratory
> > > > 303-497-2768
> > > > ---
> > > > My working day may not be your working day. Please do not feel
> obliged
> > to
> > > > reply to this email outside of your normal working hours.
> > > >
> > > >
> > >
> > > --
> > > *Logan C. Dawson, Ph.D.*
> > > NOAA/NWS/NCEP/EMC
> > > 5830 University Research Court
> > > College Park, MD 20740
> > > (301) 683-3944
> > >
> > >
> >
> > --
> > George McCabe - Software Engineer III
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > 303-497-2768
> > ---
> > My working day may not be your working day. Please do not feel
obliged to
> > reply to this email outside of your normal working hours.
> >
> >
>
> --
> *Logan C. Dawson, Ph.D.*
> NOAA/NWS/NCEP/EMC
> 5830 University Research Court
> College Park, MD 20740
> (301) 683-3944
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: Running RegridDataPlane wrapper for fcst and obs file
From: Logan Dawson - NOAA Affiliate
Time: Thu Jan 23 09:49:58 2020
Hi George,
I talked with Mallory this morning, and she has a list of deprecated
and
new configs for the StatAnalysis wrapper. She said she's shared that
list
with you. It will definitely be more complete than anything I could
work
up.
Also, I'm running into a new MET error at the grid_stat step. In my
GridStatConfig file, I have output_prefix =
"${MODEL}_${FCST_VAR}_vs_${OBTYPE}";, and the error I get is:
replace_env() -> unable to get value for environment variable
"FCST_VAR"
This MET config setting previously worked in v2.1 with only
FCST_VAR1_NAME
defined in my metplus_final conf file. I also tried using
${FCST_VAR_NAME}
in the GridStatConfig or explicitly defining FCST_VAR in one of my
metplus
conf files. I'm still getting the same error though. Is there
something I'm
missing?
Thanks,
Logan
On Wed, Jan 22, 2020 at 11:37 PM George McCabe via RT
<met_help at ucar.edu>
wrote:
> Hi Logan,
>
> I was not aware of that the StatAnalysis wrapper had deprecated
configs
> that are not on the list. If you wouldn't mind putting together a
list that
> would be very helpful. Thanks for bringing this to my attention.
>
> On Wed, Jan 22, 2020 at 4:28 PM Logan Dawson - NOAA Affiliate via RT
<
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886 >
> >
> > Hi George,
> >
> > One quick follow-up that you're probably already aware of... I
just
> wanted
> > to let you know that the validate_config.py script doesn't appear
to be
> > catching deprecated configs associated with the StatAnalysis
wrapper.
> >
> > The script okayed my conf file associated with the StatAnalysis
job, but
> > I'm catching a few config items that appear to be deprecated or
new ones
> > that need to be defined. I can work through them and give you a
list
> > tomorrow (or on the METplus call on Monday).
> >
> > I'm able to roll back to METplus 2.1 to run the StatAnalysi job I
need to
> > run in the interim.
> >
> > Thanks,
> > Logan
> >
> >
> >
> > On Wed, Jan 22, 2020 at 6:22 PM George McCabe via RT
<met_help at ucar.edu>
> > wrote:
> >
> > > Great news to hear! I'm glad you were able to get the new
version up
> and
> > > running quickly and it allowed you to set up your use case. I
will
> close
> > > this ticket, but please feel free to submit another ticket if
you have
> > any
> > > other issues.
> > >
> > > - George
> > >
> > > On Wed, Jan 22, 2020 at 3:34 PM Logan Dawson - NOAA Affiliate
via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886
>
> > > >
> > > > Hi George,
> > > >
> > > > Thanks for providing the location of the beta release and for
letting
> > me
> > > > know about the validate_config.py script. I was able to get
the beta
> > and
> > > > run the validation script. It was super simple, and I like the
> > > > functionality. I need about 6 or 7 config changes, and it was
nice to
> > be
> > > > able to bypass manually making each change.
> > > >
> > > > Also, I can confirm that running RegridDataPlane on forecast
and
> > > > observation files is working as expected in this 3.0 beta
release.
> > > >
> > > > Thanks so much for the quick help on this issue! This is
really going
> > to
> > > > help me speed up my verification jobs!
> > > >
> > > > Thanks,
> > > > Logan
> > > >
> > > >
> > > >
> > > > On Wed, Jan 22, 2020 at 3:45 PM George McCabe via RT <
> > met_help at ucar.edu>
> > > > wrote:
> > > >
> > > > > Hi Logan,
> > > > >
> > > > > Perfect, thank you!
> > > > >
> > > > > The release can be found here:
> > > > > https://github.com/NCAR/METplus/releases/tag/v3.0-beta2
> > > > >
> > > > > You will have to download one of the source code tarballs
and
> extract
> > > the
> > > > > files locally. There will likely be a few configuration
variable
> name
> > > > > changes involved in migrating your config files from METplus
2 to
> 3.
> > > You
> > > > > can either run master_metplus.py with all of the config
files you
> use
> > > > > passed in and it will report errors regarding the variables
you
> need
> > to
> > > > > change. You can also run the script validate_config.py the
same way
> > as
> > > > the
> > > > > master script (passing in all configs with -c) and it will
prompt
> you
> > > to
> > > > > confirm automatically replacing config variables with the
updated
> > > > > equivalent. Please let me know if you have any issues with
this, or
> > if
> > > > > anything is unclear in this process. That will help refine
the
> > process
> > > > for
> > > > > other users when the 3.0 release is available.
> > > > >
> > > > > Thanks,
> > > > > George
> > > > >
> > > > > On Wed, Jan 22, 2020 at 1:18 PM Logan Dawson - NOAA
Affiliate via
> RT
> > <
> > > > > met_help at ucar.edu> wrote:
> > > > >
> > > > > >
> > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886 >
> > > > > >
> > > > > > Hi George,
> > > > > >
> > > > > > I'd be happy to test out the beta release. There's a
Python 3.6.3
> > > > > > installation on Mars that has pandas and cartopy
installed.
> > > > > >
> > > > > > Thanks,
> > > > > > Logan
> > > > > >
> > > > > > On Wed, Jan 22, 2020 at 3:12 PM George McCabe via RT <
> > > > met_help at ucar.edu>
> > > > > > wrote:
> > > > > >
> > > > > > > Hi Logan,
> > > > > > >
> > > > > > > It looks like there is a bug in running RegridDataPlane
on FCST
> > > data
> > > > in
> > > > > > > METplus 2.1. It appears to have been fixed in the latest
beta
> > > > release,
> > > > > > but
> > > > > > > I am noticing there are no use cases examples that run
> > > > RegridDataPlane
> > > > > on
> > > > > > > forecast data. Would you be willing to take the METplus
> 3.0-beta2
> > > > > release
> > > > > > > and test it with your use case to make sure that it does
indeed
> > > work
> > > > > > > correctly? If so, I can provide you with the beta
release. You
> > will
> > > > > need
> > > > > > to
> > > > > > > have a Python version that is 3.6 or higher with the
cartopy
> and
> > > > pandas
> > > > > > > packages installed. If that is not available, I can give
you
> > > > > instructions
> > > > > > > on how to set up a conda environment to test. Let me
know what
> > you
> > > > > think.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > George
> > > > > > >
> > > > > > > On Wed, Jan 22, 2020 at 12:58 PM Logan Dawson - NOAA
Affiliate
> > via
> > > > RT <
> > > > > > > met_help at ucar.edu> wrote:
> > > > > > >
> > > > > > > >
> > > > > > > > Wed Jan 22 12:58:46 2020: Request 93886 was acted
upon.
> > > > > > > > Transaction: Ticket created by logan.dawson at noaa.gov
> > > > > > > > Queue: met_help
> > > > > > > > Subject: Running RegridDataPlane wrapper for fcst
and
> obs
> > > file
> > > > > > > > Owner: Nobody
> > > > > > > > Requestors: logan.dawson at noaa.gov
> > > > > > > > Status: new
> > > > > > > > Ticket <URL:
> > > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886
> > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Hi,
> > > > > > > >
> > > > > > > > I've been running the GridStat METplus wrapper to
verify 3
> > > > different
> > > > > > > > simulated reflectivity-related products from 7
different
> hi-res
> > > > > models.
> > > > > > > > Since I want to fairly compare the 7 models, I'm using
a
> > standard
> > > > > > > > verification grid.
> > > > > > > >
> > > > > > > > I've realized that simply allowing the GridStat
wrapper to
> > regrid
> > > > the
> > > > > > > > forecast and observation files is highly inefficient
because
> > > > > regridding
> > > > > > > > occurs with every forecast lead/valid combination,
regardless
> > of
> > > > > > whether
> > > > > > > > the obs or fcst file has already been regridded. So
now, I'm
> > > trying
> > > > > to
> > > > > > > add
> > > > > > > > the RegridDataPlane wrapper to my process list, but it
> appears
> > > that
> > > > > I'm
> > > > > > > not
> > > > > > > > able to regrid a fcst file and an obs file
simultaneously?
> > > > > > > >
> > > > > > > > I get the following error (on the command line or in
stderr
> of
> > a
> > > > job
> > > > > > > > script) when I try to run RegridDataPlane with both :
> > > > > > > >
> > > > > > > > 01/22 19:05:33.039 jlog (master_metplus.py:190)
CRITICAL:
> > > > > > master_metplus
> > > > > > > > failed: getdir() takes at most 3 arguments (4 given)
> > > > > > > >
> > > > > > > > Traceback (most recent call last):
> > > > > > > >
> > > > > > > > File
> > > > > > > >
> > > > > >
> > > >
> > "/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/master_metplus.py",
> > > > > > > > line 187, in <module>
> > > > > > > >
> > > > > > > > main()
> > > > > > > >
> > > > > > > > File
> > > > > > > >
> > > > > >
> > > >
> > "/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/master_metplus.py",
> > > > > > > > line 120, in main
> > > > > > > >
> > > > > > > > item + "Wrapper")(p, logger)
> > > > > > > >
> > > > > > > > File
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> "/gpfs/dell3/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/regrid_data_plane_wrapper.py",
> > > > > > > > line 39, in __init__
> > > > > > > >
> > > > > > > > self.create_c_dict()
> > > > > > > >
> > > > > > > > File
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> "/gpfs/dell3/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/regrid_data_plane_wrapper.py",
> > > > > > > > line 104, in create_c_dict
> > > > > > > >
> > > > > > > > '', self.logger)
> > > > > > > >
> > > > > > > > TypeError: getdir() takes at most 3 arguments (4
given)
> > > > > > > >
> > > > > > > > 01/22 19:05:33Z run-METplus-CRITICAL: master_metplus
> failed:
> > > > > getdir()
> > > > > > > > takes at most 3 arguments (4 given)
> > > > > > > >
> > > > > > > > In this attempt, I have FCST_REGRID_DATA_PLANE_RUN and
> > > > > > > > OBS_REGRID_DATA_PLANE_RUN both set to True, and I have
input
> > and
> > > > > output
> > > > > > > > directories set for both fcst and obs. I tried setting
> > FCST...RUN
> > > > to
> > > > > > > false,
> > > > > > > > and I didn't get that error.
> > > > > > > >
> > > > > > > > Is there any way I can run RegridDataPlane for both
forecast
> > and
> > > > obs
> > > > > in
> > > > > > > the
> > > > > > > > same process list? Or do I need to regrid the obs
files prior
> > to
> > > > the
> > > > > > > > METplus call that runs RegridDataPlane for the fcst
files
> prior
> > > to
> > > > > > > running
> > > > > > > > GridStat?
> > > > > > > >
> > > > > > > > Thanks,
> > > > > > > > Logan
> > > > > > > >
> > > > > > > > --
> > > > > > > > *Logan C. Dawson, Ph.D.*
> > > > > > > > NOAA/NWS/NCEP/EMC
> > > > > > > > 5830 University Research Court
> > > > > > > > College Park, MD 20740
> > > > > > > > (301) 683-3944
> > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > George McCabe - Software Engineer III
> > > > > > > National Center for Atmospheric Research
> > > > > > > Research Applications Laboratory
> > > > > > > 303-497-2768
> > > > > > > ---
> > > > > > > My working day may not be your working day. Please do
not feel
> > > > obliged
> > > > > to
> > > > > > > reply to this email outside of your normal working
hours.
> > > > > > >
> > > > > > >
> > > > > >
> > > > > > --
> > > > > > *Logan C. Dawson, Ph.D.*
> > > > > > NOAA/NWS/NCEP/EMC
> > > > > > 5830 University Research Court
> > > > > > College Park, MD 20740
> > > > > > (301) 683-3944
> > > > > >
> > > > > >
> > > > >
> > > > > --
> > > > > George McCabe - Software Engineer III
> > > > > National Center for Atmospheric Research
> > > > > Research Applications Laboratory
> > > > > 303-497-2768
> > > > > ---
> > > > > My working day may not be your working day. Please do not
feel
> > obliged
> > > to
> > > > > reply to this email outside of your normal working hours.
> > > > >
> > > > >
> > > >
> > > > --
> > > > *Logan C. Dawson, Ph.D.*
> > > > NOAA/NWS/NCEP/EMC
> > > > 5830 University Research Court
> > > > College Park, MD 20740
> > > > (301) 683-3944
> > > >
> > > >
> > >
> > > --
> > > George McCabe - Software Engineer III
> > > National Center for Atmospheric Research
> > > Research Applications Laboratory
> > > 303-497-2768
> > > ---
> > > My working day may not be your working day. Please do not feel
obliged
> to
> > > reply to this email outside of your normal working hours.
> > >
> > >
> >
> > --
> > *Logan C. Dawson, Ph.D.*
> > NOAA/NWS/NCEP/EMC
> > 5830 University Research Court
> > College Park, MD 20740
> > (301) 683-3944
> >
> >
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>
--
*Logan C. Dawson, Ph.D.*
NOAA/NWS/NCEP/EMC
5830 University Research Court
College Park, MD 20740
(301) 683-3944
------------------------------------------------
Subject: Running RegridDataPlane wrapper for fcst and obs file
From: Logan Dawson - NOAA Affiliate
Time: Thu Jan 23 13:24:23 2020
Hi George,
So I figured it out. I worked with Mallory on debugging a similar
issue
with the StatAnalysis MET config file. From there, I saw where you
committed changes to how the output prefix is defined for the GridStat
MET
config. I now have GRID_STAT_OUTPUT_PREFIX defined in my METplus
config
section, and grid_stat is running as expected.
If there's a way to have validate_config.py flag a warning telling
users
they might need to set the new GRID_STAT_OUTPUT_PREFIX config, that
would
be a really helpful addition. Maybe just flag it if other GRID_STAT_X
configs are parsed and the new output prefix isn't defined?
Thanks,
Logan
On Thu, Jan 23, 2020 at 4:49 PM Logan Dawson - NOAA Affiliate <
logan.dawson at noaa.gov> wrote:
> Hi George,
>
> I talked with Mallory this morning, and she has a list of deprecated
and
> new configs for the StatAnalysis wrapper. She said she's shared that
list
> with you. It will definitely be more complete than anything I could
work
> up.
>
> Also, I'm running into a new MET error at the grid_stat step. In my
> GridStatConfig file, I have output_prefix =
> "${MODEL}_${FCST_VAR}_vs_${OBTYPE}";, and the error I get is:
>
> replace_env() -> unable to get value for environment variable
"FCST_VAR"
>
> This MET config setting previously worked in v2.1 with only
FCST_VAR1_NAME
> defined in my metplus_final conf file. I also tried using
${FCST_VAR_NAME}
> in the GridStatConfig or explicitly defining FCST_VAR in one of my
metplus
> conf files. I'm still getting the same error though. Is there
something I'm
> missing?
>
> Thanks,
> Logan
>
> On Wed, Jan 22, 2020 at 11:37 PM George McCabe via RT
<met_help at ucar.edu>
> wrote:
>
>> Hi Logan,
>>
>> I was not aware of that the StatAnalysis wrapper had deprecated
configs
>> that are not on the list. If you wouldn't mind putting together a
list
>> that
>> would be very helpful. Thanks for bringing this to my attention.
>>
>> On Wed, Jan 22, 2020 at 4:28 PM Logan Dawson - NOAA Affiliate via
RT <
>> met_help at ucar.edu> wrote:
>>
>> >
>> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886 >
>> >
>> > Hi George,
>> >
>> > One quick follow-up that you're probably already aware of... I
just
>> wanted
>> > to let you know that the validate_config.py script doesn't appear
to be
>> > catching deprecated configs associated with the StatAnalysis
wrapper.
>> >
>> > The script okayed my conf file associated with the StatAnalysis
job, but
>> > I'm catching a few config items that appear to be deprecated or
new ones
>> > that need to be defined. I can work through them and give you a
list
>> > tomorrow (or on the METplus call on Monday).
>> >
>> > I'm able to roll back to METplus 2.1 to run the StatAnalysi job I
need
>> to
>> > run in the interim.
>> >
>> > Thanks,
>> > Logan
>> >
>> >
>> >
>> > On Wed, Jan 22, 2020 at 6:22 PM George McCabe via RT
<met_help at ucar.edu
>> >
>> > wrote:
>> >
>> > > Great news to hear! I'm glad you were able to get the new
version up
>> and
>> > > running quickly and it allowed you to set up your use case. I
will
>> close
>> > > this ticket, but please feel free to submit another ticket if
you have
>> > any
>> > > other issues.
>> > >
>> > > - George
>> > >
>> > > On Wed, Jan 22, 2020 at 3:34 PM Logan Dawson - NOAA Affiliate
via RT <
>> > > met_help at ucar.edu> wrote:
>> > >
>> > > >
>> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886
>
>> > > >
>> > > > Hi George,
>> > > >
>> > > > Thanks for providing the location of the beta release and for
>> letting
>> > me
>> > > > know about the validate_config.py script. I was able to get
the beta
>> > and
>> > > > run the validation script. It was super simple, and I like
the
>> > > > functionality. I need about 6 or 7 config changes, and it was
nice
>> to
>> > be
>> > > > able to bypass manually making each change.
>> > > >
>> > > > Also, I can confirm that running RegridDataPlane on forecast
and
>> > > > observation files is working as expected in this 3.0 beta
release.
>> > > >
>> > > > Thanks so much for the quick help on this issue! This is
really
>> going
>> > to
>> > > > help me speed up my verification jobs!
>> > > >
>> > > > Thanks,
>> > > > Logan
>> > > >
>> > > >
>> > > >
>> > > > On Wed, Jan 22, 2020 at 3:45 PM George McCabe via RT <
>> > met_help at ucar.edu>
>> > > > wrote:
>> > > >
>> > > > > Hi Logan,
>> > > > >
>> > > > > Perfect, thank you!
>> > > > >
>> > > > > The release can be found here:
>> > > > > https://github.com/NCAR/METplus/releases/tag/v3.0-beta2
>> > > > >
>> > > > > You will have to download one of the source code tarballs
and
>> extract
>> > > the
>> > > > > files locally. There will likely be a few configuration
variable
>> name
>> > > > > changes involved in migrating your config files from
METplus 2 to
>> 3.
>> > > You
>> > > > > can either run master_metplus.py with all of the config
files you
>> use
>> > > > > passed in and it will report errors regarding the variables
you
>> need
>> > to
>> > > > > change. You can also run the script validate_config.py the
same
>> way
>> > as
>> > > > the
>> > > > > master script (passing in all configs with -c) and it will
prompt
>> you
>> > > to
>> > > > > confirm automatically replacing config variables with the
updated
>> > > > > equivalent. Please let me know if you have any issues with
this,
>> or
>> > if
>> > > > > anything is unclear in this process. That will help refine
the
>> > process
>> > > > for
>> > > > > other users when the 3.0 release is available.
>> > > > >
>> > > > > Thanks,
>> > > > > George
>> > > > >
>> > > > > On Wed, Jan 22, 2020 at 1:18 PM Logan Dawson - NOAA
Affiliate via
>> RT
>> > <
>> > > > > met_help at ucar.edu> wrote:
>> > > > >
>> > > > > >
>> > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886 >
>> > > > > >
>> > > > > > Hi George,
>> > > > > >
>> > > > > > I'd be happy to test out the beta release. There's a
Python
>> 3.6.3
>> > > > > > installation on Mars that has pandas and cartopy
installed.
>> > > > > >
>> > > > > > Thanks,
>> > > > > > Logan
>> > > > > >
>> > > > > > On Wed, Jan 22, 2020 at 3:12 PM George McCabe via RT <
>> > > > met_help at ucar.edu>
>> > > > > > wrote:
>> > > > > >
>> > > > > > > Hi Logan,
>> > > > > > >
>> > > > > > > It looks like there is a bug in running RegridDataPlane
on
>> FCST
>> > > data
>> > > > in
>> > > > > > > METplus 2.1. It appears to have been fixed in the
latest beta
>> > > > release,
>> > > > > > but
>> > > > > > > I am noticing there are no use cases examples that run
>> > > > RegridDataPlane
>> > > > > on
>> > > > > > > forecast data. Would you be willing to take the METplus
>> 3.0-beta2
>> > > > > release
>> > > > > > > and test it with your use case to make sure that it
does
>> indeed
>> > > work
>> > > > > > > correctly? If so, I can provide you with the beta
release. You
>> > will
>> > > > > need
>> > > > > > to
>> > > > > > > have a Python version that is 3.6 or higher with the
cartopy
>> and
>> > > > pandas
>> > > > > > > packages installed. If that is not available, I can
give you
>> > > > > instructions
>> > > > > > > on how to set up a conda environment to test. Let me
know what
>> > you
>> > > > > think.
>> > > > > > >
>> > > > > > > Thanks,
>> > > > > > > George
>> > > > > > >
>> > > > > > > On Wed, Jan 22, 2020 at 12:58 PM Logan Dawson - NOAA
Affiliate
>> > via
>> > > > RT <
>> > > > > > > met_help at ucar.edu> wrote:
>> > > > > > >
>> > > > > > > >
>> > > > > > > > Wed Jan 22 12:58:46 2020: Request 93886 was acted
upon.
>> > > > > > > > Transaction: Ticket created by logan.dawson at noaa.gov
>> > > > > > > > Queue: met_help
>> > > > > > > > Subject: Running RegridDataPlane wrapper for
fcst and
>> obs
>> > > file
>> > > > > > > > Owner: Nobody
>> > > > > > > > Requestors: logan.dawson at noaa.gov
>> > > > > > > > Status: new
>> > > > > > > > Ticket <URL:
>> > > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886
>> > > > > > >
>> > > > > > > >
>> > > > > > > >
>> > > > > > > > Hi,
>> > > > > > > >
>> > > > > > > > I've been running the GridStat METplus wrapper to
verify 3
>> > > > different
>> > > > > > > > simulated reflectivity-related products from 7
different
>> hi-res
>> > > > > models.
>> > > > > > > > Since I want to fairly compare the 7 models, I'm
using a
>> > standard
>> > > > > > > > verification grid.
>> > > > > > > >
>> > > > > > > > I've realized that simply allowing the GridStat
wrapper to
>> > regrid
>> > > > the
>> > > > > > > > forecast and observation files is highly inefficient
because
>> > > > > regridding
>> > > > > > > > occurs with every forecast lead/valid combination,
>> regardless
>> > of
>> > > > > > whether
>> > > > > > > > the obs or fcst file has already been regridded. So
now, I'm
>> > > trying
>> > > > > to
>> > > > > > > add
>> > > > > > > > the RegridDataPlane wrapper to my process list, but
it
>> appears
>> > > that
>> > > > > I'm
>> > > > > > > not
>> > > > > > > > able to regrid a fcst file and an obs file
simultaneously?
>> > > > > > > >
>> > > > > > > > I get the following error (on the command line or in
stderr
>> of
>> > a
>> > > > job
>> > > > > > > > script) when I try to run RegridDataPlane with both :
>> > > > > > > >
>> > > > > > > > 01/22 19:05:33.039 jlog (master_metplus.py:190)
CRITICAL:
>> > > > > > master_metplus
>> > > > > > > > failed: getdir() takes at most 3 arguments (4 given)
>> > > > > > > >
>> > > > > > > > Traceback (most recent call last):
>> > > > > > > >
>> > > > > > > > File
>> > > > > > > >
>> > > > > >
>> > > >
>> >
>> "/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/master_metplus.py",
>> > > > > > > > line 187, in <module>
>> > > > > > > >
>> > > > > > > > main()
>> > > > > > > >
>> > > > > > > > File
>> > > > > > > >
>> > > > > >
>> > > >
>> >
>> "/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/master_metplus.py",
>> > > > > > > > line 120, in main
>> > > > > > > >
>> > > > > > > > item + "Wrapper")(p, logger)
>> > > > > > > >
>> > > > > > > > File
>> > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> "/gpfs/dell3/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/regrid_data_plane_wrapper.py",
>> > > > > > > > line 39, in __init__
>> > > > > > > >
>> > > > > > > > self.create_c_dict()
>> > > > > > > >
>> > > > > > > > File
>> > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > >
>> > > > >
>> > > >
>> > >
>> >
>> "/gpfs/dell3/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/regrid_data_plane_wrapper.py",
>> > > > > > > > line 104, in create_c_dict
>> > > > > > > >
>> > > > > > > > '', self.logger)
>> > > > > > > >
>> > > > > > > > TypeError: getdir() takes at most 3 arguments (4
given)
>> > > > > > > >
>> > > > > > > > 01/22 19:05:33Z run-METplus-CRITICAL: master_metplus
>> failed:
>> > > > > getdir()
>> > > > > > > > takes at most 3 arguments (4 given)
>> > > > > > > >
>> > > > > > > > In this attempt, I have FCST_REGRID_DATA_PLANE_RUN
and
>> > > > > > > > OBS_REGRID_DATA_PLANE_RUN both set to True, and I
have input
>> > and
>> > > > > output
>> > > > > > > > directories set for both fcst and obs. I tried
setting
>> > FCST...RUN
>> > > > to
>> > > > > > > false,
>> > > > > > > > and I didn't get that error.
>> > > > > > > >
>> > > > > > > > Is there any way I can run RegridDataPlane for both
forecast
>> > and
>> > > > obs
>> > > > > in
>> > > > > > > the
>> > > > > > > > same process list? Or do I need to regrid the obs
files
>> prior
>> > to
>> > > > the
>> > > > > > > > METplus call that runs RegridDataPlane for the fcst
files
>> prior
>> > > to
>> > > > > > > running
>> > > > > > > > GridStat?
>> > > > > > > >
>> > > > > > > > Thanks,
>> > > > > > > > Logan
>> > > > > > > >
>> > > > > > > > --
>> > > > > > > > *Logan C. Dawson, Ph.D.*
>> > > > > > > > NOAA/NWS/NCEP/EMC
>> > > > > > > > 5830 University Research Court
>> > > > > > > > College Park, MD 20740
>> > > > > > > > (301) 683-3944
>> > > > > > > >
>> > > > > > > >
>> > > > > > >
>> > > > > > > --
>> > > > > > > George McCabe - Software Engineer III
>> > > > > > > National Center for Atmospheric Research
>> > > > > > > Research Applications Laboratory
>> > > > > > > 303-497-2768
>> > > > > > > ---
>> > > > > > > My working day may not be your working day. Please do
not feel
>> > > > obliged
>> > > > > to
>> > > > > > > reply to this email outside of your normal working
hours.
>> > > > > > >
>> > > > > > >
>> > > > > >
>> > > > > > --
>> > > > > > *Logan C. Dawson, Ph.D.*
>> > > > > > NOAA/NWS/NCEP/EMC
>> > > > > > 5830 University Research Court
>> > > > > > College Park, MD 20740
>> > > > > > (301) 683-3944
>> > > > > >
>> > > > > >
>> > > > >
>> > > > > --
>> > > > > George McCabe - Software Engineer III
>> > > > > National Center for Atmospheric Research
>> > > > > Research Applications Laboratory
>> > > > > 303-497-2768
>> > > > > ---
>> > > > > My working day may not be your working day. Please do not
feel
>> > obliged
>> > > to
>> > > > > reply to this email outside of your normal working hours.
>> > > > >
>> > > > >
>> > > >
>> > > > --
>> > > > *Logan C. Dawson, Ph.D.*
>> > > > NOAA/NWS/NCEP/EMC
>> > > > 5830 University Research Court
>> > > > College Park, MD 20740
>> > > > (301) 683-3944
>> > > >
>> > > >
>> > >
>> > > --
>> > > George McCabe - Software Engineer III
>> > > National Center for Atmospheric Research
>> > > Research Applications Laboratory
>> > > 303-497-2768
>> > > ---
>> > > My working day may not be your working day. Please do not feel
>> obliged to
>> > > reply to this email outside of your normal working hours.
>> > >
>> > >
>> >
>> > --
>> > *Logan C. Dawson, Ph.D.*
>> > NOAA/NWS/NCEP/EMC
>> > 5830 University Research Court
>> > College Park, MD 20740
>> > (301) 683-3944
>> >
>> >
>>
>> --
>> George McCabe - Software Engineer III
>> National Center for Atmospheric Research
>> Research Applications Laboratory
>> 303-497-2768
>> ---
>> My working day may not be your working day. Please do not feel
obliged to
>> reply to this email outside of your normal working hours.
>>
>>
>
> --
> *Logan C. Dawson, Ph.D.*
> NOAA/NWS/NCEP/EMC
> 5830 University Research Court
> College Park, MD 20740
> (301) 683-3944
>
--
*Logan C. Dawson, Ph.D.*
NOAA/NWS/NCEP/EMC
5830 University Research Court
College Park, MD 20740
(301) 683-3944
------------------------------------------------
Subject: Running RegridDataPlane wrapper for fcst and obs file
From: George McCabe
Time: Fri Jan 24 07:08:46 2020
Hi Logan,
Thank you for bringing this to my attention. If you have DEBUG logging
turned on, the log file should mention that you are using a default
value
for GRID_STAT_OUTPUT_PREFIX. However, I didn't think to validate
changes in
the MET config files. The validation script could parse the METplus
config
files for any _CONFIG_FILE variable that is set, then open that file
to
check for any deprecated environment variables that are used, like
FCST_VAR. If output_prefix is set using one of the deprecated
variables, it
could recommend replacing it with the OUTPUT_PREFIX environment
variable.
What do you think?
On Thu, Jan 23, 2020 at 1:24 PM Logan Dawson - NOAA Affiliate via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886 >
>
> Hi George,
>
> So I figured it out. I worked with Mallory on debugging a similar
issue
> with the StatAnalysis MET config file. From there, I saw where you
> committed changes to how the output prefix is defined for the
GridStat MET
> config. I now have GRID_STAT_OUTPUT_PREFIX defined in my METplus
config
> section, and grid_stat is running as expected.
>
> If there's a way to have validate_config.py flag a warning telling
users
> they might need to set the new GRID_STAT_OUTPUT_PREFIX config, that
would
> be a really helpful addition. Maybe just flag it if other
GRID_STAT_X
> configs are parsed and the new output prefix isn't defined?
>
> Thanks,
> Logan
>
> On Thu, Jan 23, 2020 at 4:49 PM Logan Dawson - NOAA Affiliate <
> logan.dawson at noaa.gov> wrote:
>
> > Hi George,
> >
> > I talked with Mallory this morning, and she has a list of
deprecated and
> > new configs for the StatAnalysis wrapper. She said she's shared
that list
> > with you. It will definitely be more complete than anything I
could work
> > up.
> >
> > Also, I'm running into a new MET error at the grid_stat step. In
my
> > GridStatConfig file, I have output_prefix =
> > "${MODEL}_${FCST_VAR}_vs_${OBTYPE}";, and the error I get is:
> >
> > replace_env() -> unable to get value for environment variable
"FCST_VAR"
> >
> > This MET config setting previously worked in v2.1 with only
> FCST_VAR1_NAME
> > defined in my metplus_final conf file. I also tried using
> ${FCST_VAR_NAME}
> > in the GridStatConfig or explicitly defining FCST_VAR in one of my
> metplus
> > conf files. I'm still getting the same error though. Is there
something
> I'm
> > missing?
> >
> > Thanks,
> > Logan
> >
> > On Wed, Jan 22, 2020 at 11:37 PM George McCabe via RT
<met_help at ucar.edu
> >
> > wrote:
> >
> >> Hi Logan,
> >>
> >> I was not aware of that the StatAnalysis wrapper had deprecated
configs
> >> that are not on the list. If you wouldn't mind putting together a
list
> >> that
> >> would be very helpful. Thanks for bringing this to my attention.
> >>
> >> On Wed, Jan 22, 2020 at 4:28 PM Logan Dawson - NOAA Affiliate via
RT <
> >> met_help at ucar.edu> wrote:
> >>
> >> >
> >> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886 >
> >> >
> >> > Hi George,
> >> >
> >> > One quick follow-up that you're probably already aware of... I
just
> >> wanted
> >> > to let you know that the validate_config.py script doesn't
appear to
> be
> >> > catching deprecated configs associated with the StatAnalysis
wrapper.
> >> >
> >> > The script okayed my conf file associated with the StatAnalysis
job,
> but
> >> > I'm catching a few config items that appear to be deprecated or
new
> ones
> >> > that need to be defined. I can work through them and give you a
list
> >> > tomorrow (or on the METplus call on Monday).
> >> >
> >> > I'm able to roll back to METplus 2.1 to run the StatAnalysi job
I need
> >> to
> >> > run in the interim.
> >> >
> >> > Thanks,
> >> > Logan
> >> >
> >> >
> >> >
> >> > On Wed, Jan 22, 2020 at 6:22 PM George McCabe via RT <
> met_help at ucar.edu
> >> >
> >> > wrote:
> >> >
> >> > > Great news to hear! I'm glad you were able to get the new
version up
> >> and
> >> > > running quickly and it allowed you to set up your use case. I
will
> >> close
> >> > > this ticket, but please feel free to submit another ticket if
you
> have
> >> > any
> >> > > other issues.
> >> > >
> >> > > - George
> >> > >
> >> > > On Wed, Jan 22, 2020 at 3:34 PM Logan Dawson - NOAA Affiliate
via
> RT <
> >> > > met_help at ucar.edu> wrote:
> >> > >
> >> > > >
> >> > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886 >
> >> > > >
> >> > > > Hi George,
> >> > > >
> >> > > > Thanks for providing the location of the beta release and
for
> >> letting
> >> > me
> >> > > > know about the validate_config.py script. I was able to get
the
> beta
> >> > and
> >> > > > run the validation script. It was super simple, and I like
the
> >> > > > functionality. I need about 6 or 7 config changes, and it
was nice
> >> to
> >> > be
> >> > > > able to bypass manually making each change.
> >> > > >
> >> > > > Also, I can confirm that running RegridDataPlane on
forecast and
> >> > > > observation files is working as expected in this 3.0 beta
release.
> >> > > >
> >> > > > Thanks so much for the quick help on this issue! This is
really
> >> going
> >> > to
> >> > > > help me speed up my verification jobs!
> >> > > >
> >> > > > Thanks,
> >> > > > Logan
> >> > > >
> >> > > >
> >> > > >
> >> > > > On Wed, Jan 22, 2020 at 3:45 PM George McCabe via RT <
> >> > met_help at ucar.edu>
> >> > > > wrote:
> >> > > >
> >> > > > > Hi Logan,
> >> > > > >
> >> > > > > Perfect, thank you!
> >> > > > >
> >> > > > > The release can be found here:
> >> > > > > https://github.com/NCAR/METplus/releases/tag/v3.0-beta2
> >> > > > >
> >> > > > > You will have to download one of the source code tarballs
and
> >> extract
> >> > > the
> >> > > > > files locally. There will likely be a few configuration
variable
> >> name
> >> > > > > changes involved in migrating your config files from
METplus 2
> to
> >> 3.
> >> > > You
> >> > > > > can either run master_metplus.py with all of the config
files
> you
> >> use
> >> > > > > passed in and it will report errors regarding the
variables you
> >> need
> >> > to
> >> > > > > change. You can also run the script validate_config.py
the same
> >> way
> >> > as
> >> > > > the
> >> > > > > master script (passing in all configs with -c) and it
will
> prompt
> >> you
> >> > > to
> >> > > > > confirm automatically replacing config variables with the
> updated
> >> > > > > equivalent. Please let me know if you have any issues
with this,
> >> or
> >> > if
> >> > > > > anything is unclear in this process. That will help
refine the
> >> > process
> >> > > > for
> >> > > > > other users when the 3.0 release is available.
> >> > > > >
> >> > > > > Thanks,
> >> > > > > George
> >> > > > >
> >> > > > > On Wed, Jan 22, 2020 at 1:18 PM Logan Dawson - NOAA
Affiliate
> via
> >> RT
> >> > <
> >> > > > > met_help at ucar.edu> wrote:
> >> > > > >
> >> > > > > >
> >> > > > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886
> >
> >> > > > > >
> >> > > > > > Hi George,
> >> > > > > >
> >> > > > > > I'd be happy to test out the beta release. There's a
Python
> >> 3.6.3
> >> > > > > > installation on Mars that has pandas and cartopy
installed.
> >> > > > > >
> >> > > > > > Thanks,
> >> > > > > > Logan
> >> > > > > >
> >> > > > > > On Wed, Jan 22, 2020 at 3:12 PM George McCabe via RT <
> >> > > > met_help at ucar.edu>
> >> > > > > > wrote:
> >> > > > > >
> >> > > > > > > Hi Logan,
> >> > > > > > >
> >> > > > > > > It looks like there is a bug in running
RegridDataPlane on
> >> FCST
> >> > > data
> >> > > > in
> >> > > > > > > METplus 2.1. It appears to have been fixed in the
latest
> beta
> >> > > > release,
> >> > > > > > but
> >> > > > > > > I am noticing there are no use cases examples that
run
> >> > > > RegridDataPlane
> >> > > > > on
> >> > > > > > > forecast data. Would you be willing to take the
METplus
> >> 3.0-beta2
> >> > > > > release
> >> > > > > > > and test it with your use case to make sure that it
does
> >> indeed
> >> > > work
> >> > > > > > > correctly? If so, I can provide you with the beta
release.
> You
> >> > will
> >> > > > > need
> >> > > > > > to
> >> > > > > > > have a Python version that is 3.6 or higher with the
cartopy
> >> and
> >> > > > pandas
> >> > > > > > > packages installed. If that is not available, I can
give you
> >> > > > > instructions
> >> > > > > > > on how to set up a conda environment to test. Let me
know
> what
> >> > you
> >> > > > > think.
> >> > > > > > >
> >> > > > > > > Thanks,
> >> > > > > > > George
> >> > > > > > >
> >> > > > > > > On Wed, Jan 22, 2020 at 12:58 PM Logan Dawson - NOAA
> Affiliate
> >> > via
> >> > > > RT <
> >> > > > > > > met_help at ucar.edu> wrote:
> >> > > > > > >
> >> > > > > > > >
> >> > > > > > > > Wed Jan 22 12:58:46 2020: Request 93886 was acted
upon.
> >> > > > > > > > Transaction: Ticket created by
logan.dawson at noaa.gov
> >> > > > > > > > Queue: met_help
> >> > > > > > > > Subject: Running RegridDataPlane wrapper for
fcst and
> >> obs
> >> > > file
> >> > > > > > > > Owner: Nobody
> >> > > > > > > > Requestors: logan.dawson at noaa.gov
> >> > > > > > > > Status: new
> >> > > > > > > > Ticket <URL:
> >> > > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886
> >> > > > > > >
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > > > Hi,
> >> > > > > > > >
> >> > > > > > > > I've been running the GridStat METplus wrapper to
verify 3
> >> > > > different
> >> > > > > > > > simulated reflectivity-related products from 7
different
> >> hi-res
> >> > > > > models.
> >> > > > > > > > Since I want to fairly compare the 7 models, I'm
using a
> >> > standard
> >> > > > > > > > verification grid.
> >> > > > > > > >
> >> > > > > > > > I've realized that simply allowing the GridStat
wrapper to
> >> > regrid
> >> > > > the
> >> > > > > > > > forecast and observation files is highly
inefficient
> because
> >> > > > > regridding
> >> > > > > > > > occurs with every forecast lead/valid combination,
> >> regardless
> >> > of
> >> > > > > > whether
> >> > > > > > > > the obs or fcst file has already been regridded. So
now,
> I'm
> >> > > trying
> >> > > > > to
> >> > > > > > > add
> >> > > > > > > > the RegridDataPlane wrapper to my process list, but
it
> >> appears
> >> > > that
> >> > > > > I'm
> >> > > > > > > not
> >> > > > > > > > able to regrid a fcst file and an obs file
simultaneously?
> >> > > > > > > >
> >> > > > > > > > I get the following error (on the command line or
in
> stderr
> >> of
> >> > a
> >> > > > job
> >> > > > > > > > script) when I try to run RegridDataPlane with both
:
> >> > > > > > > >
> >> > > > > > > > 01/22 19:05:33.039 jlog (master_metplus.py:190)
CRITICAL:
> >> > > > > > master_metplus
> >> > > > > > > > failed: getdir() takes at most 3 arguments (4
given)
> >> > > > > > > >
> >> > > > > > > > Traceback (most recent call last):
> >> > > > > > > >
> >> > > > > > > > File
> >> > > > > > > >
> >> > > > > >
> >> > > >
> >> >
> >>
> "/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/master_metplus.py",
> >> > > > > > > > line 187, in <module>
> >> > > > > > > >
> >> > > > > > > > main()
> >> > > > > > > >
> >> > > > > > > > File
> >> > > > > > > >
> >> > > > > >
> >> > > >
> >> >
> >>
> "/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/master_metplus.py",
> >> > > > > > > > line 120, in main
> >> > > > > > > >
> >> > > > > > > > item + "Wrapper")(p, logger)
> >> > > > > > > >
> >> > > > > > > > File
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> "/gpfs/dell3/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/regrid_data_plane_wrapper.py",
> >> > > > > > > > line 39, in __init__
> >> > > > > > > >
> >> > > > > > > > self.create_c_dict()
> >> > > > > > > >
> >> > > > > > > > File
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> "/gpfs/dell3/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/regrid_data_plane_wrapper.py",
> >> > > > > > > > line 104, in create_c_dict
> >> > > > > > > >
> >> > > > > > > > '', self.logger)
> >> > > > > > > >
> >> > > > > > > > TypeError: getdir() takes at most 3 arguments (4
given)
> >> > > > > > > >
> >> > > > > > > > 01/22 19:05:33Z run-METplus-CRITICAL:
master_metplus
> >> failed:
> >> > > > > getdir()
> >> > > > > > > > takes at most 3 arguments (4 given)
> >> > > > > > > >
> >> > > > > > > > In this attempt, I have FCST_REGRID_DATA_PLANE_RUN
and
> >> > > > > > > > OBS_REGRID_DATA_PLANE_RUN both set to True, and I
have
> input
> >> > and
> >> > > > > output
> >> > > > > > > > directories set for both fcst and obs. I tried
setting
> >> > FCST...RUN
> >> > > > to
> >> > > > > > > false,
> >> > > > > > > > and I didn't get that error.
> >> > > > > > > >
> >> > > > > > > > Is there any way I can run RegridDataPlane for both
> forecast
> >> > and
> >> > > > obs
> >> > > > > in
> >> > > > > > > the
> >> > > > > > > > same process list? Or do I need to regrid the obs
files
> >> prior
> >> > to
> >> > > > the
> >> > > > > > > > METplus call that runs RegridDataPlane for the fcst
files
> >> prior
> >> > > to
> >> > > > > > > running
> >> > > > > > > > GridStat?
> >> > > > > > > >
> >> > > > > > > > Thanks,
> >> > > > > > > > Logan
> >> > > > > > > >
> >> > > > > > > > --
> >> > > > > > > > *Logan C. Dawson, Ph.D.*
> >> > > > > > > > NOAA/NWS/NCEP/EMC
> >> > > > > > > > 5830 University Research Court
> >> > > > > > > > College Park, MD 20740
> >> > > > > > > > (301) 683-3944
> >> > > > > > > >
> >> > > > > > > >
> >> > > > > > >
> >> > > > > > > --
> >> > > > > > > George McCabe - Software Engineer III
> >> > > > > > > National Center for Atmospheric Research
> >> > > > > > > Research Applications Laboratory
> >> > > > > > > 303-497-2768
> >> > > > > > > ---
> >> > > > > > > My working day may not be your working day. Please do
not
> feel
> >> > > > obliged
> >> > > > > to
> >> > > > > > > reply to this email outside of your normal working
hours.
> >> > > > > > >
> >> > > > > > >
> >> > > > > >
> >> > > > > > --
> >> > > > > > *Logan C. Dawson, Ph.D.*
> >> > > > > > NOAA/NWS/NCEP/EMC
> >> > > > > > 5830 University Research Court
> >> > > > > > College Park, MD 20740
> >> > > > > > (301) 683-3944
> >> > > > > >
> >> > > > > >
> >> > > > >
> >> > > > > --
> >> > > > > George McCabe - Software Engineer III
> >> > > > > National Center for Atmospheric Research
> >> > > > > Research Applications Laboratory
> >> > > > > 303-497-2768
> >> > > > > ---
> >> > > > > My working day may not be your working day. Please do not
feel
> >> > obliged
> >> > > to
> >> > > > > reply to this email outside of your normal working hours.
> >> > > > >
> >> > > > >
> >> > > >
> >> > > > --
> >> > > > *Logan C. Dawson, Ph.D.*
> >> > > > NOAA/NWS/NCEP/EMC
> >> > > > 5830 University Research Court
> >> > > > College Park, MD 20740
> >> > > > (301) 683-3944
> >> > > >
> >> > > >
> >> > >
> >> > > --
> >> > > George McCabe - Software Engineer III
> >> > > National Center for Atmospheric Research
> >> > > Research Applications Laboratory
> >> > > 303-497-2768
> >> > > ---
> >> > > My working day may not be your working day. Please do not
feel
> >> obliged to
> >> > > reply to this email outside of your normal working hours.
> >> > >
> >> > >
> >> >
> >> > --
> >> > *Logan C. Dawson, Ph.D.*
> >> > NOAA/NWS/NCEP/EMC
> >> > 5830 University Research Court
> >> > College Park, MD 20740
> >> > (301) 683-3944
> >> >
> >> >
> >>
> >> --
> >> George McCabe - Software Engineer III
> >> National Center for Atmospheric Research
> >> Research Applications Laboratory
> >> 303-497-2768
> >> ---
> >> My working day may not be your working day. Please do not feel
obliged
> to
> >> reply to this email outside of your normal working hours.
> >>
> >>
> >
> > --
> > *Logan C. Dawson, Ph.D.*
> > NOAA/NWS/NCEP/EMC
> > 5830 University Research Court
> > College Park, MD 20740
> > (301) 683-3944
> >
>
>
> --
> *Logan C. Dawson, Ph.D.*
> NOAA/NWS/NCEP/EMC
> 5830 University Research Court
> College Park, MD 20740
> (301) 683-3944
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: Running RegridDataPlane wrapper for fcst and obs file
From: Logan Dawson - NOAA Affiliate
Time: Fri Jan 24 09:50:40 2020
Hi George,
That extra level of validation would be perfect!
I'm certain I just missed the log file saying GRID_STAT_OUTPUT_PREFIX
was
set to a default value. Since some of those auto-default configs don't
really factor into how the METplus job runs, I don't always pay too
close
attention to them.
Thanks,
Logan
On Fri, Jan 24, 2020 at 2:08 PM George McCabe via RT
<met_help at ucar.edu>
wrote:
> Hi Logan,
>
> Thank you for bringing this to my attention. If you have DEBUG
logging
> turned on, the log file should mention that you are using a default
value
> for GRID_STAT_OUTPUT_PREFIX. However, I didn't think to validate
changes in
> the MET config files. The validation script could parse the METplus
config
> files for any _CONFIG_FILE variable that is set, then open that file
to
> check for any deprecated environment variables that are used, like
> FCST_VAR. If output_prefix is set using one of the deprecated
variables, it
> could recommend replacing it with the OUTPUT_PREFIX environment
variable.
> What do you think?
>
> On Thu, Jan 23, 2020 at 1:24 PM Logan Dawson - NOAA Affiliate via RT
<
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886 >
> >
> > Hi George,
> >
> > So I figured it out. I worked with Mallory on debugging a similar
issue
> > with the StatAnalysis MET config file. From there, I saw where you
> > committed changes to how the output prefix is defined for the
GridStat
> MET
> > config. I now have GRID_STAT_OUTPUT_PREFIX defined in my METplus
config
> > section, and grid_stat is running as expected.
> >
> > If there's a way to have validate_config.py flag a warning telling
users
> > they might need to set the new GRID_STAT_OUTPUT_PREFIX config,
that would
> > be a really helpful addition. Maybe just flag it if other
GRID_STAT_X
> > configs are parsed and the new output prefix isn't defined?
> >
> > Thanks,
> > Logan
> >
> > On Thu, Jan 23, 2020 at 4:49 PM Logan Dawson - NOAA Affiliate <
> > logan.dawson at noaa.gov> wrote:
> >
> > > Hi George,
> > >
> > > I talked with Mallory this morning, and she has a list of
deprecated
> and
> > > new configs for the StatAnalysis wrapper. She said she's shared
that
> list
> > > with you. It will definitely be more complete than anything I
could
> work
> > > up.
> > >
> > > Also, I'm running into a new MET error at the grid_stat step. In
my
> > > GridStatConfig file, I have output_prefix =
> > > "${MODEL}_${FCST_VAR}_vs_${OBTYPE}";, and the error I get is:
> > >
> > > replace_env() -> unable to get value for environment variable
> "FCST_VAR"
> > >
> > > This MET config setting previously worked in v2.1 with only
> > FCST_VAR1_NAME
> > > defined in my metplus_final conf file. I also tried using
> > ${FCST_VAR_NAME}
> > > in the GridStatConfig or explicitly defining FCST_VAR in one of
my
> > metplus
> > > conf files. I'm still getting the same error though. Is there
something
> > I'm
> > > missing?
> > >
> > > Thanks,
> > > Logan
> > >
> > > On Wed, Jan 22, 2020 at 11:37 PM George McCabe via RT <
> met_help at ucar.edu
> > >
> > > wrote:
> > >
> > >> Hi Logan,
> > >>
> > >> I was not aware of that the StatAnalysis wrapper had deprecated
> configs
> > >> that are not on the list. If you wouldn't mind putting together
a list
> > >> that
> > >> would be very helpful. Thanks for bringing this to my
attention.
> > >>
> > >> On Wed, Jan 22, 2020 at 4:28 PM Logan Dawson - NOAA Affiliate
via RT <
> > >> met_help at ucar.edu> wrote:
> > >>
> > >> >
> > >> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886
>
> > >> >
> > >> > Hi George,
> > >> >
> > >> > One quick follow-up that you're probably already aware of...
I just
> > >> wanted
> > >> > to let you know that the validate_config.py script doesn't
appear to
> > be
> > >> > catching deprecated configs associated with the StatAnalysis
> wrapper.
> > >> >
> > >> > The script okayed my conf file associated with the
StatAnalysis job,
> > but
> > >> > I'm catching a few config items that appear to be deprecated
or new
> > ones
> > >> > that need to be defined. I can work through them and give you
a list
> > >> > tomorrow (or on the METplus call on Monday).
> > >> >
> > >> > I'm able to roll back to METplus 2.1 to run the StatAnalysi
job I
> need
> > >> to
> > >> > run in the interim.
> > >> >
> > >> > Thanks,
> > >> > Logan
> > >> >
> > >> >
> > >> >
> > >> > On Wed, Jan 22, 2020 at 6:22 PM George McCabe via RT <
> > met_help at ucar.edu
> > >> >
> > >> > wrote:
> > >> >
> > >> > > Great news to hear! I'm glad you were able to get the new
version
> up
> > >> and
> > >> > > running quickly and it allowed you to set up your use case.
I will
> > >> close
> > >> > > this ticket, but please feel free to submit another ticket
if you
> > have
> > >> > any
> > >> > > other issues.
> > >> > >
> > >> > > - George
> > >> > >
> > >> > > On Wed, Jan 22, 2020 at 3:34 PM Logan Dawson - NOAA
Affiliate via
> > RT <
> > >> > > met_help at ucar.edu> wrote:
> > >> > >
> > >> > > >
> > >> > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886 >
> > >> > > >
> > >> > > > Hi George,
> > >> > > >
> > >> > > > Thanks for providing the location of the beta release and
for
> > >> letting
> > >> > me
> > >> > > > know about the validate_config.py script. I was able to
get the
> > beta
> > >> > and
> > >> > > > run the validation script. It was super simple, and I
like the
> > >> > > > functionality. I need about 6 or 7 config changes, and it
was
> nice
> > >> to
> > >> > be
> > >> > > > able to bypass manually making each change.
> > >> > > >
> > >> > > > Also, I can confirm that running RegridDataPlane on
forecast and
> > >> > > > observation files is working as expected in this 3.0 beta
> release.
> > >> > > >
> > >> > > > Thanks so much for the quick help on this issue! This is
really
> > >> going
> > >> > to
> > >> > > > help me speed up my verification jobs!
> > >> > > >
> > >> > > > Thanks,
> > >> > > > Logan
> > >> > > >
> > >> > > >
> > >> > > >
> > >> > > > On Wed, Jan 22, 2020 at 3:45 PM George McCabe via RT <
> > >> > met_help at ucar.edu>
> > >> > > > wrote:
> > >> > > >
> > >> > > > > Hi Logan,
> > >> > > > >
> > >> > > > > Perfect, thank you!
> > >> > > > >
> > >> > > > > The release can be found here:
> > >> > > > > https://github.com/NCAR/METplus/releases/tag/v3.0-beta2
> > >> > > > >
> > >> > > > > You will have to download one of the source code
tarballs and
> > >> extract
> > >> > > the
> > >> > > > > files locally. There will likely be a few configuration
> variable
> > >> name
> > >> > > > > changes involved in migrating your config files from
METplus 2
> > to
> > >> 3.
> > >> > > You
> > >> > > > > can either run master_metplus.py with all of the config
files
> > you
> > >> use
> > >> > > > > passed in and it will report errors regarding the
variables
> you
> > >> need
> > >> > to
> > >> > > > > change. You can also run the script validate_config.py
the
> same
> > >> way
> > >> > as
> > >> > > > the
> > >> > > > > master script (passing in all configs with -c) and it
will
> > prompt
> > >> you
> > >> > > to
> > >> > > > > confirm automatically replacing config variables with
the
> > updated
> > >> > > > > equivalent. Please let me know if you have any issues
with
> this,
> > >> or
> > >> > if
> > >> > > > > anything is unclear in this process. That will help
refine the
> > >> > process
> > >> > > > for
> > >> > > > > other users when the 3.0 release is available.
> > >> > > > >
> > >> > > > > Thanks,
> > >> > > > > George
> > >> > > > >
> > >> > > > > On Wed, Jan 22, 2020 at 1:18 PM Logan Dawson - NOAA
Affiliate
> > via
> > >> RT
> > >> > <
> > >> > > > > met_help at ucar.edu> wrote:
> > >> > > > >
> > >> > > > > >
> > >> > > > > > <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886
> > >
> > >> > > > > >
> > >> > > > > > Hi George,
> > >> > > > > >
> > >> > > > > > I'd be happy to test out the beta release. There's a
Python
> > >> 3.6.3
> > >> > > > > > installation on Mars that has pandas and cartopy
installed.
> > >> > > > > >
> > >> > > > > > Thanks,
> > >> > > > > > Logan
> > >> > > > > >
> > >> > > > > > On Wed, Jan 22, 2020 at 3:12 PM George McCabe via RT
<
> > >> > > > met_help at ucar.edu>
> > >> > > > > > wrote:
> > >> > > > > >
> > >> > > > > > > Hi Logan,
> > >> > > > > > >
> > >> > > > > > > It looks like there is a bug in running
RegridDataPlane on
> > >> FCST
> > >> > > data
> > >> > > > in
> > >> > > > > > > METplus 2.1. It appears to have been fixed in the
latest
> > beta
> > >> > > > release,
> > >> > > > > > but
> > >> > > > > > > I am noticing there are no use cases examples that
run
> > >> > > > RegridDataPlane
> > >> > > > > on
> > >> > > > > > > forecast data. Would you be willing to take the
METplus
> > >> 3.0-beta2
> > >> > > > > release
> > >> > > > > > > and test it with your use case to make sure that it
does
> > >> indeed
> > >> > > work
> > >> > > > > > > correctly? If so, I can provide you with the beta
release.
> > You
> > >> > will
> > >> > > > > need
> > >> > > > > > to
> > >> > > > > > > have a Python version that is 3.6 or higher with
the
> cartopy
> > >> and
> > >> > > > pandas
> > >> > > > > > > packages installed. If that is not available, I can
give
> you
> > >> > > > > instructions
> > >> > > > > > > on how to set up a conda environment to test. Let
me know
> > what
> > >> > you
> > >> > > > > think.
> > >> > > > > > >
> > >> > > > > > > Thanks,
> > >> > > > > > > George
> > >> > > > > > >
> > >> > > > > > > On Wed, Jan 22, 2020 at 12:58 PM Logan Dawson -
NOAA
> > Affiliate
> > >> > via
> > >> > > > RT <
> > >> > > > > > > met_help at ucar.edu> wrote:
> > >> > > > > > >
> > >> > > > > > > >
> > >> > > > > > > > Wed Jan 22 12:58:46 2020: Request 93886 was acted
upon.
> > >> > > > > > > > Transaction: Ticket created by
logan.dawson at noaa.gov
> > >> > > > > > > > Queue: met_help
> > >> > > > > > > > Subject: Running RegridDataPlane wrapper for
fcst
> and
> > >> obs
> > >> > > file
> > >> > > > > > > > Owner: Nobody
> > >> > > > > > > > Requestors: logan.dawson at noaa.gov
> > >> > > > > > > > Status: new
> > >> > > > > > > > Ticket <URL:
> > >> > > > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886
> > >> > > > > > >
> > >> > > > > > > >
> > >> > > > > > > >
> > >> > > > > > > > Hi,
> > >> > > > > > > >
> > >> > > > > > > > I've been running the GridStat METplus wrapper to
> verify 3
> > >> > > > different
> > >> > > > > > > > simulated reflectivity-related products from 7
different
> > >> hi-res
> > >> > > > > models.
> > >> > > > > > > > Since I want to fairly compare the 7 models, I'm
using a
> > >> > standard
> > >> > > > > > > > verification grid.
> > >> > > > > > > >
> > >> > > > > > > > I've realized that simply allowing the GridStat
wrapper
> to
> > >> > regrid
> > >> > > > the
> > >> > > > > > > > forecast and observation files is highly
inefficient
> > because
> > >> > > > > regridding
> > >> > > > > > > > occurs with every forecast lead/valid
combination,
> > >> regardless
> > >> > of
> > >> > > > > > whether
> > >> > > > > > > > the obs or fcst file has already been regridded.
So now,
> > I'm
> > >> > > trying
> > >> > > > > to
> > >> > > > > > > add
> > >> > > > > > > > the RegridDataPlane wrapper to my process list,
but it
> > >> appears
> > >> > > that
> > >> > > > > I'm
> > >> > > > > > > not
> > >> > > > > > > > able to regrid a fcst file and an obs file
> simultaneously?
> > >> > > > > > > >
> > >> > > > > > > > I get the following error (on the command line or
in
> > stderr
> > >> of
> > >> > a
> > >> > > > job
> > >> > > > > > > > script) when I try to run RegridDataPlane with
both :
> > >> > > > > > > >
> > >> > > > > > > > 01/22 19:05:33.039 jlog (master_metplus.py:190)
> CRITICAL:
> > >> > > > > > master_metplus
> > >> > > > > > > > failed: getdir() takes at most 3 arguments (4
given)
> > >> > > > > > > >
> > >> > > > > > > > Traceback (most recent call last):
> > >> > > > > > > >
> > >> > > > > > > > File
> > >> > > > > > > >
> > >> > > > > >
> > >> > > >
> > >> >
> > >>
> > "/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/master_metplus.py",
> > >> > > > > > > > line 187, in <module>
> > >> > > > > > > >
> > >> > > > > > > > main()
> > >> > > > > > > >
> > >> > > > > > > > File
> > >> > > > > > > >
> > >> > > > > >
> > >> > > >
> > >> >
> > >>
> > "/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/master_metplus.py",
> > >> > > > > > > > line 120, in main
> > >> > > > > > > >
> > >> > > > > > > > item + "Wrapper")(p, logger)
> > >> > > > > > > >
> > >> > > > > > > > File
> > >> > > > > > > >
> > >> > > > > > > >
> > >> > > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> "/gpfs/dell3/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/regrid_data_plane_wrapper.py",
> > >> > > > > > > > line 39, in __init__
> > >> > > > > > > >
> > >> > > > > > > > self.create_c_dict()
> > >> > > > > > > >
> > >> > > > > > > > File
> > >> > > > > > > >
> > >> > > > > > > >
> > >> > > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> >
> "/gpfs/dell3/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/regrid_data_plane_wrapper.py",
> > >> > > > > > > > line 104, in create_c_dict
> > >> > > > > > > >
> > >> > > > > > > > '', self.logger)
> > >> > > > > > > >
> > >> > > > > > > > TypeError: getdir() takes at most 3 arguments (4
given)
> > >> > > > > > > >
> > >> > > > > > > > 01/22 19:05:33Z run-METplus-CRITICAL:
master_metplus
> > >> failed:
> > >> > > > > getdir()
> > >> > > > > > > > takes at most 3 arguments (4 given)
> > >> > > > > > > >
> > >> > > > > > > > In this attempt, I have
FCST_REGRID_DATA_PLANE_RUN and
> > >> > > > > > > > OBS_REGRID_DATA_PLANE_RUN both set to True, and I
have
> > input
> > >> > and
> > >> > > > > output
> > >> > > > > > > > directories set for both fcst and obs. I tried
setting
> > >> > FCST...RUN
> > >> > > > to
> > >> > > > > > > false,
> > >> > > > > > > > and I didn't get that error.
> > >> > > > > > > >
> > >> > > > > > > > Is there any way I can run RegridDataPlane for
both
> > forecast
> > >> > and
> > >> > > > obs
> > >> > > > > in
> > >> > > > > > > the
> > >> > > > > > > > same process list? Or do I need to regrid the obs
files
> > >> prior
> > >> > to
> > >> > > > the
> > >> > > > > > > > METplus call that runs RegridDataPlane for the
fcst
> files
> > >> prior
> > >> > > to
> > >> > > > > > > running
> > >> > > > > > > > GridStat?
> > >> > > > > > > >
> > >> > > > > > > > Thanks,
> > >> > > > > > > > Logan
> > >> > > > > > > >
> > >> > > > > > > > --
> > >> > > > > > > > *Logan C. Dawson, Ph.D.*
> > >> > > > > > > > NOAA/NWS/NCEP/EMC
> > >> > > > > > > > 5830 University Research Court
> > >> > > > > > > > College Park, MD 20740
> > >> > > > > > > > (301) 683-3944
> > >> > > > > > > >
> > >> > > > > > > >
> > >> > > > > > >
> > >> > > > > > > --
> > >> > > > > > > George McCabe - Software Engineer III
> > >> > > > > > > National Center for Atmospheric Research
> > >> > > > > > > Research Applications Laboratory
> > >> > > > > > > 303-497-2768
> > >> > > > > > > ---
> > >> > > > > > > My working day may not be your working day. Please
do not
> > feel
> > >> > > > obliged
> > >> > > > > to
> > >> > > > > > > reply to this email outside of your normal working
hours.
> > >> > > > > > >
> > >> > > > > > >
> > >> > > > > >
> > >> > > > > > --
> > >> > > > > > *Logan C. Dawson, Ph.D.*
> > >> > > > > > NOAA/NWS/NCEP/EMC
> > >> > > > > > 5830 University Research Court
> > >> > > > > > College Park, MD 20740
> > >> > > > > > (301) 683-3944
> > >> > > > > >
> > >> > > > > >
> > >> > > > >
> > >> > > > > --
> > >> > > > > George McCabe - Software Engineer III
> > >> > > > > National Center for Atmospheric Research
> > >> > > > > Research Applications Laboratory
> > >> > > > > 303-497-2768
> > >> > > > > ---
> > >> > > > > My working day may not be your working day. Please do
not feel
> > >> > obliged
> > >> > > to
> > >> > > > > reply to this email outside of your normal working
hours.
> > >> > > > >
> > >> > > > >
> > >> > > >
> > >> > > > --
> > >> > > > *Logan C. Dawson, Ph.D.*
> > >> > > > NOAA/NWS/NCEP/EMC
> > >> > > > 5830 University Research Court
> > >> > > > College Park, MD 20740
> > >> > > > (301) 683-3944
> > >> > > >
> > >> > > >
> > >> > >
> > >> > > --
> > >> > > George McCabe - Software Engineer III
> > >> > > National Center for Atmospheric Research
> > >> > > Research Applications Laboratory
> > >> > > 303-497-2768
> > >> > > ---
> > >> > > My working day may not be your working day. Please do not
feel
> > >> obliged to
> > >> > > reply to this email outside of your normal working hours.
> > >> > >
> > >> > >
> > >> >
> > >> > --
> > >> > *Logan C. Dawson, Ph.D.*
> > >> > NOAA/NWS/NCEP/EMC
> > >> > 5830 University Research Court
> > >> > College Park, MD 20740
> > >> > (301) 683-3944
> > >> >
> > >> >
> > >>
> > >> --
> > >> George McCabe - Software Engineer III
> > >> National Center for Atmospheric Research
> > >> Research Applications Laboratory
> > >> 303-497-2768
> > >> ---
> > >> My working day may not be your working day. Please do not feel
obliged
> > to
> > >> reply to this email outside of your normal working hours.
> > >>
> > >>
> > >
> > > --
> > > *Logan C. Dawson, Ph.D.*
> > > NOAA/NWS/NCEP/EMC
> > > 5830 University Research Court
> > > College Park, MD 20740
> > > (301) 683-3944
> > >
> >
> >
> > --
> > *Logan C. Dawson, Ph.D.*
> > NOAA/NWS/NCEP/EMC
> > 5830 University Research Court
> > College Park, MD 20740
> > (301) 683-3944
> >
> >
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>
--
*Logan C. Dawson, Ph.D.*
NOAA/NWS/NCEP/EMC
5830 University Research Court
College Park, MD 20740
(301) 683-3944
------------------------------------------------
Subject: Running RegridDataPlane wrapper for fcst and obs file
From: George McCabe
Time: Fri Jan 24 12:33:14 2020
Hi Logan,
Yea, I know there are a lot of default values that get output in the
logs
so it can clutter up useful info. In general it is a good idea to
review
that list if you have errors to make sure you don't have a typo in one
of
your config variable names.
I added checks in MET config files and recommended replacements. I
also
added the deprecated items from StatAnalysis/MakePlots and added
functionality to handle items that have numbers in them, i.e.
MODEL1_OBS_NAME. Attached are two files you can replace in the
METplus/ush
directory to test these changes. Let me know if the new additions are
useful for updating your config files or if you have any issues.
Thanks,
George
On Fri, Jan 24, 2020 at 9:51 AM Logan Dawson - NOAA Affiliate via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886 >
>
> Hi George,
>
> That extra level of validation would be perfect!
>
> I'm certain I just missed the log file saying
GRID_STAT_OUTPUT_PREFIX was
> set to a default value. Since some of those auto-default configs
don't
> really factor into how the METplus job runs, I don't always pay too
close
> attention to them.
>
> Thanks,
> Logan
>
> On Fri, Jan 24, 2020 at 2:08 PM George McCabe via RT
<met_help at ucar.edu>
> wrote:
>
> > Hi Logan,
> >
> > Thank you for bringing this to my attention. If you have DEBUG
logging
> > turned on, the log file should mention that you are using a
default value
> > for GRID_STAT_OUTPUT_PREFIX. However, I didn't think to validate
changes
> in
> > the MET config files. The validation script could parse the
METplus
> config
> > files for any _CONFIG_FILE variable that is set, then open that
file to
> > check for any deprecated environment variables that are used, like
> > FCST_VAR. If output_prefix is set using one of the deprecated
variables,
> it
> > could recommend replacing it with the OUTPUT_PREFIX environment
variable.
> > What do you think?
> >
> > On Thu, Jan 23, 2020 at 1:24 PM Logan Dawson - NOAA Affiliate via
RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886 >
> > >
> > > Hi George,
> > >
> > > So I figured it out. I worked with Mallory on debugging a
similar issue
> > > with the StatAnalysis MET config file. From there, I saw where
you
> > > committed changes to how the output prefix is defined for the
GridStat
> > MET
> > > config. I now have GRID_STAT_OUTPUT_PREFIX defined in my METplus
config
> > > section, and grid_stat is running as expected.
> > >
> > > If there's a way to have validate_config.py flag a warning
telling
> users
> > > they might need to set the new GRID_STAT_OUTPUT_PREFIX config,
that
> would
> > > be a really helpful addition. Maybe just flag it if other
GRID_STAT_X
> > > configs are parsed and the new output prefix isn't defined?
> > >
> > > Thanks,
> > > Logan
> > >
> > > On Thu, Jan 23, 2020 at 4:49 PM Logan Dawson - NOAA Affiliate <
> > > logan.dawson at noaa.gov> wrote:
> > >
> > > > Hi George,
> > > >
> > > > I talked with Mallory this morning, and she has a list of
deprecated
> > and
> > > > new configs for the StatAnalysis wrapper. She said she's
shared that
> > list
> > > > with you. It will definitely be more complete than anything I
could
> > work
> > > > up.
> > > >
> > > > Also, I'm running into a new MET error at the grid_stat step.
In my
> > > > GridStatConfig file, I have output_prefix =
> > > > "${MODEL}_${FCST_VAR}_vs_${OBTYPE}";, and the error I get is:
> > > >
> > > > replace_env() -> unable to get value for environment variable
> > "FCST_VAR"
> > > >
> > > > This MET config setting previously worked in v2.1 with only
> > > FCST_VAR1_NAME
> > > > defined in my metplus_final conf file. I also tried using
> > > ${FCST_VAR_NAME}
> > > > in the GridStatConfig or explicitly defining FCST_VAR in one
of my
> > > metplus
> > > > conf files. I'm still getting the same error though. Is there
> something
> > > I'm
> > > > missing?
> > > >
> > > > Thanks,
> > > > Logan
> > > >
> > > > On Wed, Jan 22, 2020 at 11:37 PM George McCabe via RT <
> > met_help at ucar.edu
> > > >
> > > > wrote:
> > > >
> > > >> Hi Logan,
> > > >>
> > > >> I was not aware of that the StatAnalysis wrapper had
deprecated
> > configs
> > > >> that are not on the list. If you wouldn't mind putting
together a
> list
> > > >> that
> > > >> would be very helpful. Thanks for bringing this to my
attention.
> > > >>
> > > >> On Wed, Jan 22, 2020 at 4:28 PM Logan Dawson - NOAA Affiliate
via
> RT <
> > > >> met_help at ucar.edu> wrote:
> > > >>
> > > >> >
> > > >> > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886 >
> > > >> >
> > > >> > Hi George,
> > > >> >
> > > >> > One quick follow-up that you're probably already aware
of... I
> just
> > > >> wanted
> > > >> > to let you know that the validate_config.py script doesn't
appear
> to
> > > be
> > > >> > catching deprecated configs associated with the
StatAnalysis
> > wrapper.
> > > >> >
> > > >> > The script okayed my conf file associated with the
StatAnalysis
> job,
> > > but
> > > >> > I'm catching a few config items that appear to be
deprecated or
> new
> > > ones
> > > >> > that need to be defined. I can work through them and give
you a
> list
> > > >> > tomorrow (or on the METplus call on Monday).
> > > >> >
> > > >> > I'm able to roll back to METplus 2.1 to run the StatAnalysi
job I
> > need
> > > >> to
> > > >> > run in the interim.
> > > >> >
> > > >> > Thanks,
> > > >> > Logan
> > > >> >
> > > >> >
> > > >> >
> > > >> > On Wed, Jan 22, 2020 at 6:22 PM George McCabe via RT <
> > > met_help at ucar.edu
> > > >> >
> > > >> > wrote:
> > > >> >
> > > >> > > Great news to hear! I'm glad you were able to get the new
> version
> > up
> > > >> and
> > > >> > > running quickly and it allowed you to set up your use
case. I
> will
> > > >> close
> > > >> > > this ticket, but please feel free to submit another
ticket if
> you
> > > have
> > > >> > any
> > > >> > > other issues.
> > > >> > >
> > > >> > > - George
> > > >> > >
> > > >> > > On Wed, Jan 22, 2020 at 3:34 PM Logan Dawson - NOAA
Affiliate
> via
> > > RT <
> > > >> > > met_help at ucar.edu> wrote:
> > > >> > >
> > > >> > > >
> > > >> > > > <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886
> >
> > > >> > > >
> > > >> > > > Hi George,
> > > >> > > >
> > > >> > > > Thanks for providing the location of the beta release
and for
> > > >> letting
> > > >> > me
> > > >> > > > know about the validate_config.py script. I was able to
get
> the
> > > beta
> > > >> > and
> > > >> > > > run the validation script. It was super simple, and I
like the
> > > >> > > > functionality. I need about 6 or 7 config changes, and
it was
> > nice
> > > >> to
> > > >> > be
> > > >> > > > able to bypass manually making each change.
> > > >> > > >
> > > >> > > > Also, I can confirm that running RegridDataPlane on
forecast
> and
> > > >> > > > observation files is working as expected in this 3.0
beta
> > release.
> > > >> > > >
> > > >> > > > Thanks so much for the quick help on this issue! This
is
> really
> > > >> going
> > > >> > to
> > > >> > > > help me speed up my verification jobs!
> > > >> > > >
> > > >> > > > Thanks,
> > > >> > > > Logan
> > > >> > > >
> > > >> > > >
> > > >> > > >
> > > >> > > > On Wed, Jan 22, 2020 at 3:45 PM George McCabe via RT <
> > > >> > met_help at ucar.edu>
> > > >> > > > wrote:
> > > >> > > >
> > > >> > > > > Hi Logan,
> > > >> > > > >
> > > >> > > > > Perfect, thank you!
> > > >> > > > >
> > > >> > > > > The release can be found here:
> > > >> > > > > https://github.com/NCAR/METplus/releases/tag/v3.0-
beta2
> > > >> > > > >
> > > >> > > > > You will have to download one of the source code
tarballs
> and
> > > >> extract
> > > >> > > the
> > > >> > > > > files locally. There will likely be a few
configuration
> > variable
> > > >> name
> > > >> > > > > changes involved in migrating your config files from
> METplus 2
> > > to
> > > >> 3.
> > > >> > > You
> > > >> > > > > can either run master_metplus.py with all of the
config
> files
> > > you
> > > >> use
> > > >> > > > > passed in and it will report errors regarding the
variables
> > you
> > > >> need
> > > >> > to
> > > >> > > > > change. You can also run the script
validate_config.py the
> > same
> > > >> way
> > > >> > as
> > > >> > > > the
> > > >> > > > > master script (passing in all configs with -c) and it
will
> > > prompt
> > > >> you
> > > >> > > to
> > > >> > > > > confirm automatically replacing config variables with
the
> > > updated
> > > >> > > > > equivalent. Please let me know if you have any issues
with
> > this,
> > > >> or
> > > >> > if
> > > >> > > > > anything is unclear in this process. That will help
refine
> the
> > > >> > process
> > > >> > > > for
> > > >> > > > > other users when the 3.0 release is available.
> > > >> > > > >
> > > >> > > > > Thanks,
> > > >> > > > > George
> > > >> > > > >
> > > >> > > > > On Wed, Jan 22, 2020 at 1:18 PM Logan Dawson - NOAA
> Affiliate
> > > via
> > > >> RT
> > > >> > <
> > > >> > > > > met_help at ucar.edu> wrote:
> > > >> > > > >
> > > >> > > > > >
> > > >> > > > > > <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886
> > > >
> > > >> > > > > >
> > > >> > > > > > Hi George,
> > > >> > > > > >
> > > >> > > > > > I'd be happy to test out the beta release. There's
a
> Python
> > > >> 3.6.3
> > > >> > > > > > installation on Mars that has pandas and cartopy
> installed.
> > > >> > > > > >
> > > >> > > > > > Thanks,
> > > >> > > > > > Logan
> > > >> > > > > >
> > > >> > > > > > On Wed, Jan 22, 2020 at 3:12 PM George McCabe via
RT <
> > > >> > > > met_help at ucar.edu>
> > > >> > > > > > wrote:
> > > >> > > > > >
> > > >> > > > > > > Hi Logan,
> > > >> > > > > > >
> > > >> > > > > > > It looks like there is a bug in running
RegridDataPlane
> on
> > > >> FCST
> > > >> > > data
> > > >> > > > in
> > > >> > > > > > > METplus 2.1. It appears to have been fixed in the
latest
> > > beta
> > > >> > > > release,
> > > >> > > > > > but
> > > >> > > > > > > I am noticing there are no use cases examples
that run
> > > >> > > > RegridDataPlane
> > > >> > > > > on
> > > >> > > > > > > forecast data. Would you be willing to take the
METplus
> > > >> 3.0-beta2
> > > >> > > > > release
> > > >> > > > > > > and test it with your use case to make sure that
it does
> > > >> indeed
> > > >> > > work
> > > >> > > > > > > correctly? If so, I can provide you with the beta
> release.
> > > You
> > > >> > will
> > > >> > > > > need
> > > >> > > > > > to
> > > >> > > > > > > have a Python version that is 3.6 or higher with
the
> > cartopy
> > > >> and
> > > >> > > > pandas
> > > >> > > > > > > packages installed. If that is not available, I
can give
> > you
> > > >> > > > > instructions
> > > >> > > > > > > on how to set up a conda environment to test. Let
me
> know
> > > what
> > > >> > you
> > > >> > > > > think.
> > > >> > > > > > >
> > > >> > > > > > > Thanks,
> > > >> > > > > > > George
> > > >> > > > > > >
> > > >> > > > > > > On Wed, Jan 22, 2020 at 12:58 PM Logan Dawson -
NOAA
> > > Affiliate
> > > >> > via
> > > >> > > > RT <
> > > >> > > > > > > met_help at ucar.edu> wrote:
> > > >> > > > > > >
> > > >> > > > > > > >
> > > >> > > > > > > > Wed Jan 22 12:58:46 2020: Request 93886 was
acted
> upon.
> > > >> > > > > > > > Transaction: Ticket created by
logan.dawson at noaa.gov
> > > >> > > > > > > > Queue: met_help
> > > >> > > > > > > > Subject: Running RegridDataPlane wrapper
for fcst
> > and
> > > >> obs
> > > >> > > file
> > > >> > > > > > > > Owner: Nobody
> > > >> > > > > > > > Requestors: logan.dawson at noaa.gov
> > > >> > > > > > > > Status: new
> > > >> > > > > > > > Ticket <URL:
> > > >> > > > >
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886
> > > >> > > > > > >
> > > >> > > > > > > >
> > > >> > > > > > > >
> > > >> > > > > > > > Hi,
> > > >> > > > > > > >
> > > >> > > > > > > > I've been running the GridStat METplus wrapper
to
> > verify 3
> > > >> > > > different
> > > >> > > > > > > > simulated reflectivity-related products from 7
> different
> > > >> hi-res
> > > >> > > > > models.
> > > >> > > > > > > > Since I want to fairly compare the 7 models,
I'm
> using a
> > > >> > standard
> > > >> > > > > > > > verification grid.
> > > >> > > > > > > >
> > > >> > > > > > > > I've realized that simply allowing the GridStat
> wrapper
> > to
> > > >> > regrid
> > > >> > > > the
> > > >> > > > > > > > forecast and observation files is highly
inefficient
> > > because
> > > >> > > > > regridding
> > > >> > > > > > > > occurs with every forecast lead/valid
combination,
> > > >> regardless
> > > >> > of
> > > >> > > > > > whether
> > > >> > > > > > > > the obs or fcst file has already been
regridded. So
> now,
> > > I'm
> > > >> > > trying
> > > >> > > > > to
> > > >> > > > > > > add
> > > >> > > > > > > > the RegridDataPlane wrapper to my process list,
but it
> > > >> appears
> > > >> > > that
> > > >> > > > > I'm
> > > >> > > > > > > not
> > > >> > > > > > > > able to regrid a fcst file and an obs file
> > simultaneously?
> > > >> > > > > > > >
> > > >> > > > > > > > I get the following error (on the command line
or in
> > > stderr
> > > >> of
> > > >> > a
> > > >> > > > job
> > > >> > > > > > > > script) when I try to run RegridDataPlane with
both :
> > > >> > > > > > > >
> > > >> > > > > > > > 01/22 19:05:33.039 jlog (master_metplus.py:190)
> > CRITICAL:
> > > >> > > > > > master_metplus
> > > >> > > > > > > > failed: getdir() takes at most 3 arguments (4
given)
> > > >> > > > > > > >
> > > >> > > > > > > > Traceback (most recent call last):
> > > >> > > > > > > >
> > > >> > > > > > > > File
> > > >> > > > > > > >
> > > >> > > > > >
> > > >> > > >
> > > >> >
> > > >>
> > >
> "/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/master_metplus.py",
> > > >> > > > > > > > line 187, in <module>
> > > >> > > > > > > >
> > > >> > > > > > > > main()
> > > >> > > > > > > >
> > > >> > > > > > > > File
> > > >> > > > > > > >
> > > >> > > > > >
> > > >> > > >
> > > >> >
> > > >>
> > >
> "/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/master_metplus.py",
> > > >> > > > > > > > line 120, in main
> > > >> > > > > > > >
> > > >> > > > > > > > item + "Wrapper")(p, logger)
> > > >> > > > > > > >
> > > >> > > > > > > > File
> > > >> > > > > > > >
> > > >> > > > > > > >
> > > >> > > > > > >
> > > >> > > > > >
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> "/gpfs/dell3/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/regrid_data_plane_wrapper.py",
> > > >> > > > > > > > line 39, in __init__
> > > >> > > > > > > >
> > > >> > > > > > > > self.create_c_dict()
> > > >> > > > > > > >
> > > >> > > > > > > > File
> > > >> > > > > > > >
> > > >> > > > > > > >
> > > >> > > > > > >
> > > >> > > > > >
> > > >> > > > >
> > > >> > > >
> > > >> > >
> > > >> >
> > > >>
> > >
> >
> "/gpfs/dell3/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/regrid_data_plane_wrapper.py",
> > > >> > > > > > > > line 104, in create_c_dict
> > > >> > > > > > > >
> > > >> > > > > > > > '', self.logger)
> > > >> > > > > > > >
> > > >> > > > > > > > TypeError: getdir() takes at most 3 arguments
(4
> given)
> > > >> > > > > > > >
> > > >> > > > > > > > 01/22 19:05:33Z run-METplus-CRITICAL:
master_metplus
> > > >> failed:
> > > >> > > > > getdir()
> > > >> > > > > > > > takes at most 3 arguments (4 given)
> > > >> > > > > > > >
> > > >> > > > > > > > In this attempt, I have
FCST_REGRID_DATA_PLANE_RUN and
> > > >> > > > > > > > OBS_REGRID_DATA_PLANE_RUN both set to True, and
I have
> > > input
> > > >> > and
> > > >> > > > > output
> > > >> > > > > > > > directories set for both fcst and obs. I tried
setting
> > > >> > FCST...RUN
> > > >> > > > to
> > > >> > > > > > > false,
> > > >> > > > > > > > and I didn't get that error.
> > > >> > > > > > > >
> > > >> > > > > > > > Is there any way I can run RegridDataPlane for
both
> > > forecast
> > > >> > and
> > > >> > > > obs
> > > >> > > > > in
> > > >> > > > > > > the
> > > >> > > > > > > > same process list? Or do I need to regrid the
obs
> files
> > > >> prior
> > > >> > to
> > > >> > > > the
> > > >> > > > > > > > METplus call that runs RegridDataPlane for the
fcst
> > files
> > > >> prior
> > > >> > > to
> > > >> > > > > > > running
> > > >> > > > > > > > GridStat?
> > > >> > > > > > > >
> > > >> > > > > > > > Thanks,
> > > >> > > > > > > > Logan
> > > >> > > > > > > >
> > > >> > > > > > > > --
> > > >> > > > > > > > *Logan C. Dawson, Ph.D.*
> > > >> > > > > > > > NOAA/NWS/NCEP/EMC
> > > >> > > > > > > > 5830 University Research Court
> > > >> > > > > > > > College Park, MD 20740
> > > >> > > > > > > > (301) 683-3944
> > > >> > > > > > > >
> > > >> > > > > > > >
> > > >> > > > > > >
> > > >> > > > > > > --
> > > >> > > > > > > George McCabe - Software Engineer III
> > > >> > > > > > > National Center for Atmospheric Research
> > > >> > > > > > > Research Applications Laboratory
> > > >> > > > > > > 303-497-2768
> > > >> > > > > > > ---
> > > >> > > > > > > My working day may not be your working day.
Please do
> not
> > > feel
> > > >> > > > obliged
> > > >> > > > > to
> > > >> > > > > > > reply to this email outside of your normal
working
> hours.
> > > >> > > > > > >
> > > >> > > > > > >
> > > >> > > > > >
> > > >> > > > > > --
> > > >> > > > > > *Logan C. Dawson, Ph.D.*
> > > >> > > > > > NOAA/NWS/NCEP/EMC
> > > >> > > > > > 5830 University Research Court
> > > >> > > > > > College Park, MD 20740
> > > >> > > > > > (301) 683-3944
> > > >> > > > > >
> > > >> > > > > >
> > > >> > > > >
> > > >> > > > > --
> > > >> > > > > George McCabe - Software Engineer III
> > > >> > > > > National Center for Atmospheric Research
> > > >> > > > > Research Applications Laboratory
> > > >> > > > > 303-497-2768
> > > >> > > > > ---
> > > >> > > > > My working day may not be your working day. Please do
not
> feel
> > > >> > obliged
> > > >> > > to
> > > >> > > > > reply to this email outside of your normal working
hours.
> > > >> > > > >
> > > >> > > > >
> > > >> > > >
> > > >> > > > --
> > > >> > > > *Logan C. Dawson, Ph.D.*
> > > >> > > > NOAA/NWS/NCEP/EMC
> > > >> > > > 5830 University Research Court
> > > >> > > > College Park, MD 20740
> > > >> > > > (301) 683-3944
> > > >> > > >
> > > >> > > >
> > > >> > >
> > > >> > > --
> > > >> > > George McCabe - Software Engineer III
> > > >> > > National Center for Atmospheric Research
> > > >> > > Research Applications Laboratory
> > > >> > > 303-497-2768
> > > >> > > ---
> > > >> > > My working day may not be your working day. Please do not
feel
> > > >> obliged to
> > > >> > > reply to this email outside of your normal working hours.
> > > >> > >
> > > >> > >
> > > >> >
> > > >> > --
> > > >> > *Logan C. Dawson, Ph.D.*
> > > >> > NOAA/NWS/NCEP/EMC
> > > >> > 5830 University Research Court
> > > >> > College Park, MD 20740
> > > >> > (301) 683-3944
> > > >> >
> > > >> >
> > > >>
> > > >> --
> > > >> George McCabe - Software Engineer III
> > > >> National Center for Atmospheric Research
> > > >> Research Applications Laboratory
> > > >> 303-497-2768
> > > >> ---
> > > >> My working day may not be your working day. Please do not
feel
> obliged
> > > to
> > > >> reply to this email outside of your normal working hours.
> > > >>
> > > >>
> > > >
> > > > --
> > > > *Logan C. Dawson, Ph.D.*
> > > > NOAA/NWS/NCEP/EMC
> > > > 5830 University Research Court
> > > > College Park, MD 20740
> > > > (301) 683-3944
> > > >
> > >
> > >
> > > --
> > > *Logan C. Dawson, Ph.D.*
> > > NOAA/NWS/NCEP/EMC
> > > 5830 University Research Court
> > > College Park, MD 20740
> > > (301) 683-3944
> > >
> > >
> >
> > --
> > George McCabe - Software Engineer III
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > 303-497-2768
> > ---
> > My working day may not be your working day. Please do not feel
obliged to
> > reply to this email outside of your normal working hours.
> >
> >
>
> --
> *Logan C. Dawson, Ph.D.*
> NOAA/NWS/NCEP/EMC
> 5830 University Research Court
> College Park, MD 20740
> (301) 683-3944
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: Running RegridDataPlane wrapper for fcst and obs file
From: George McCabe
Time: Mon Mar 02 09:23:08 2020
Hi Logan,
FYI, I added logic to validate_config.py to list a recommended value
to set for *_OUTPUT_PREFIX in the METplus config files based on the
value that was set in your MET config file. It is not completely
automated, but it at least gives you a suggestion instead of just
causing that information to be lost. I am going to close this ticket,
but feel free to reply if you have any other issues regarding this.
Thanks,
George
On Fri Jan 24 12:33:14 2020, mccabe wrote:
> Hi Logan,
>
> Yea, I know there are a lot of default values that get output in the
> logs
> so it can clutter up useful info. In general it is a good idea to
> review
> that list if you have errors to make sure you don't have a typo in
one
> of
> your config variable names.
>
> I added checks in MET config files and recommended replacements. I
> also
> added the deprecated items from StatAnalysis/MakePlots and added
> functionality to handle items that have numbers in them, i.e.
> MODEL1_OBS_NAME. Attached are two files you can replace in the
> METplus/ush
> directory to test these changes. Let me know if the new additions
are
> useful for updating your config files or if you have any issues.
>
> Thanks,
> George
>
> On Fri, Jan 24, 2020 at 9:51 AM Logan Dawson - NOAA Affiliate via RT
<
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886 >
> >
> > Hi George,
> >
> > That extra level of validation would be perfect!
> >
> > I'm certain I just missed the log file saying
GRID_STAT_OUTPUT_PREFIX
> > was
> > set to a default value. Since some of those auto-default configs
> > don't
> > really factor into how the METplus job runs, I don't always pay
too
> > close
> > attention to them.
> >
> > Thanks,
> > Logan
> >
> > On Fri, Jan 24, 2020 at 2:08 PM George McCabe via RT
> > <met_help at ucar.edu>
> > wrote:
> >
> > > Hi Logan,
> > >
> > > Thank you for bringing this to my attention. If you have DEBUG
> > > logging
> > > turned on, the log file should mention that you are using a
default
> > > value
> > > for GRID_STAT_OUTPUT_PREFIX. However, I didn't think to validate
> > > changes
> > in
> > > the MET config files. The validation script could parse the
METplus
> > config
> > > files for any _CONFIG_FILE variable that is set, then open that
> > > file to
> > > check for any deprecated environment variables that are used,
like
> > > FCST_VAR. If output_prefix is set using one of the deprecated
> > > variables,
> > it
> > > could recommend replacing it with the OUTPUT_PREFIX environment
> > > variable.
> > > What do you think?
> > >
> > > On Thu, Jan 23, 2020 at 1:24 PM Logan Dawson - NOAA Affiliate
via
> > > RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886
>
> > > >
> > > > Hi George,
> > > >
> > > > So I figured it out. I worked with Mallory on debugging a
similar
> > > > issue
> > > > with the StatAnalysis MET config file. From there, I saw where
> > > > you
> > > > committed changes to how the output prefix is defined for the
> > > > GridStat
> > > MET
> > > > config. I now have GRID_STAT_OUTPUT_PREFIX defined in my
METplus
> > > > config
> > > > section, and grid_stat is running as expected.
> > > >
> > > > If there's a way to have validate_config.py flag a warning
> > > > telling
> > users
> > > > they might need to set the new GRID_STAT_OUTPUT_PREFIX config,
> > > > that
> > would
> > > > be a really helpful addition. Maybe just flag it if other
> > > > GRID_STAT_X
> > > > configs are parsed and the new output prefix isn't defined?
> > > >
> > > > Thanks,
> > > > Logan
> > > >
> > > > On Thu, Jan 23, 2020 at 4:49 PM Logan Dawson - NOAA Affiliate
<
> > > > logan.dawson at noaa.gov> wrote:
> > > >
> > > > > Hi George,
> > > > >
> > > > > I talked with Mallory this morning, and she has a list of
> > > > > deprecated
> > > and
> > > > > new configs for the StatAnalysis wrapper. She said she's
shared
> > > > > that
> > > list
> > > > > with you. It will definitely be more complete than anything
I
> > > > > could
> > > work
> > > > > up.
> > > > >
> > > > > Also, I'm running into a new MET error at the grid_stat
step.
> > > > > In my
> > > > > GridStatConfig file, I have output_prefix =
> > > > > "${MODEL}_${FCST_VAR}_vs_${OBTYPE}";, and the error I get
is:
> > > > >
> > > > > replace_env() -> unable to get value for environment
variable
> > > "FCST_VAR"
> > > > >
> > > > > This MET config setting previously worked in v2.1 with only
> > > > FCST_VAR1_NAME
> > > > > defined in my metplus_final conf file. I also tried using
> > > > ${FCST_VAR_NAME}
> > > > > in the GridStatConfig or explicitly defining FCST_VAR in one
of
> > > > > my
> > > > metplus
> > > > > conf files. I'm still getting the same error though. Is
there
> > something
> > > > I'm
> > > > > missing?
> > > > >
> > > > > Thanks,
> > > > > Logan
> > > > >
> > > > > On Wed, Jan 22, 2020 at 11:37 PM George McCabe via RT <
> > > met_help at ucar.edu
> > > > >
> > > > > wrote:
> > > > >
> > > > >> Hi Logan,
> > > > >>
> > > > >> I was not aware of that the StatAnalysis wrapper had
> > > > >> deprecated
> > > configs
> > > > >> that are not on the list. If you wouldn't mind putting
> > > > >> together a
> > list
> > > > >> that
> > > > >> would be very helpful. Thanks for bringing this to my
> > > > >> attention.
> > > > >>
> > > > >> On Wed, Jan 22, 2020 at 4:28 PM Logan Dawson - NOAA
Affiliate
> > > > >> via
> > RT <
> > > > >> met_help at ucar.edu> wrote:
> > > > >>
> > > > >> >
> > > > >> > <URL:
> > > > >> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886 >
> > > > >> >
> > > > >> > Hi George,
> > > > >> >
> > > > >> > One quick follow-up that you're probably already aware
of...
> > > > >> > I
> > just
> > > > >> wanted
> > > > >> > to let you know that the validate_config.py script
doesn't
> > > > >> > appear
> > to
> > > > be
> > > > >> > catching deprecated configs associated with the
StatAnalysis
> > > wrapper.
> > > > >> >
> > > > >> > The script okayed my conf file associated with the
> > > > >> > StatAnalysis
> > job,
> > > > but
> > > > >> > I'm catching a few config items that appear to be
deprecated
> > > > >> > or
> > new
> > > > ones
> > > > >> > that need to be defined. I can work through them and give
> > > > >> > you a
> > list
> > > > >> > tomorrow (or on the METplus call on Monday).
> > > > >> >
> > > > >> > I'm able to roll back to METplus 2.1 to run the
StatAnalysi
> > > > >> > job I
> > > need
> > > > >> to
> > > > >> > run in the interim.
> > > > >> >
> > > > >> > Thanks,
> > > > >> > Logan
> > > > >> >
> > > > >> >
> > > > >> >
> > > > >> > On Wed, Jan 22, 2020 at 6:22 PM George McCabe via RT <
> > > > met_help at ucar.edu
> > > > >> >
> > > > >> > wrote:
> > > > >> >
> > > > >> > > Great news to hear! I'm glad you were able to get the
new
> > version
> > > up
> > > > >> and
> > > > >> > > running quickly and it allowed you to set up your use
> > > > >> > > case. I
> > will
> > > > >> close
> > > > >> > > this ticket, but please feel free to submit another
ticket
> > > > >> > > if
> > you
> > > > have
> > > > >> > any
> > > > >> > > other issues.
> > > > >> > >
> > > > >> > > - George
> > > > >> > >
> > > > >> > > On Wed, Jan 22, 2020 at 3:34 PM Logan Dawson - NOAA
> > > > >> > > Affiliate
> > via
> > > > RT <
> > > > >> > > met_help at ucar.edu> wrote:
> > > > >> > >
> > > > >> > > >
> > > > >> > > > <URL:
> > > > >> > > >
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886
> > >
> > > > >> > > >
> > > > >> > > > Hi George,
> > > > >> > > >
> > > > >> > > > Thanks for providing the location of the beta release
> > > > >> > > > and for
> > > > >> letting
> > > > >> > me
> > > > >> > > > know about the validate_config.py script. I was able
to
> > > > >> > > > get
> > the
> > > > beta
> > > > >> > and
> > > > >> > > > run the validation script. It was super simple, and I
> > > > >> > > > like the
> > > > >> > > > functionality. I need about 6 or 7 config changes,
and
> > > > >> > > > it was
> > > nice
> > > > >> to
> > > > >> > be
> > > > >> > > > able to bypass manually making each change.
> > > > >> > > >
> > > > >> > > > Also, I can confirm that running RegridDataPlane on
> > > > >> > > > forecast
> > and
> > > > >> > > > observation files is working as expected in this 3.0
> > > > >> > > > beta
> > > release.
> > > > >> > > >
> > > > >> > > > Thanks so much for the quick help on this issue! This
is
> > really
> > > > >> going
> > > > >> > to
> > > > >> > > > help me speed up my verification jobs!
> > > > >> > > >
> > > > >> > > > Thanks,
> > > > >> > > > Logan
> > > > >> > > >
> > > > >> > > >
> > > > >> > > >
> > > > >> > > > On Wed, Jan 22, 2020 at 3:45 PM George McCabe via RT
<
> > > > >> > met_help at ucar.edu>
> > > > >> > > > wrote:
> > > > >> > > >
> > > > >> > > > > Hi Logan,
> > > > >> > > > >
> > > > >> > > > > Perfect, thank you!
> > > > >> > > > >
> > > > >> > > > > The release can be found here:
> > > > >> > > > > https://github.com/NCAR/METplus/releases/tag/v3.0-
> > > > >> > > > > beta2
> > > > >> > > > >
> > > > >> > > > > You will have to download one of the source code
> > > > >> > > > > tarballs
> > and
> > > > >> extract
> > > > >> > > the
> > > > >> > > > > files locally. There will likely be a few
> > > > >> > > > > configuration
> > > variable
> > > > >> name
> > > > >> > > > > changes involved in migrating your config files
from
> > METplus 2
> > > > to
> > > > >> 3.
> > > > >> > > You
> > > > >> > > > > can either run master_metplus.py with all of the
> > > > >> > > > > config
> > files
> > > > you
> > > > >> use
> > > > >> > > > > passed in and it will report errors regarding the
> > > > >> > > > > variables
> > > you
> > > > >> need
> > > > >> > to
> > > > >> > > > > change. You can also run the script
validate_config.py
> > > > >> > > > > the
> > > same
> > > > >> way
> > > > >> > as
> > > > >> > > > the
> > > > >> > > > > master script (passing in all configs with -c) and
it
> > > > >> > > > > will
> > > > prompt
> > > > >> you
> > > > >> > > to
> > > > >> > > > > confirm automatically replacing config variables
with
> > > > >> > > > > the
> > > > updated
> > > > >> > > > > equivalent. Please let me know if you have any
issues
> > > > >> > > > > with
> > > this,
> > > > >> or
> > > > >> > if
> > > > >> > > > > anything is unclear in this process. That will help
> > > > >> > > > > refine
> > the
> > > > >> > process
> > > > >> > > > for
> > > > >> > > > > other users when the 3.0 release is available.
> > > > >> > > > >
> > > > >> > > > > Thanks,
> > > > >> > > > > George
> > > > >> > > > >
> > > > >> > > > > On Wed, Jan 22, 2020 at 1:18 PM Logan Dawson - NOAA
> > Affiliate
> > > > via
> > > > >> RT
> > > > >> > <
> > > > >> > > > > met_help at ucar.edu> wrote:
> > > > >> > > > >
> > > > >> > > > > >
> > > > >> > > > > > <URL:
> > > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886
> > > > >
> > > > >> > > > > >
> > > > >> > > > > > Hi George,
> > > > >> > > > > >
> > > > >> > > > > > I'd be happy to test out the beta release.
There's a
> > Python
> > > > >> 3.6.3
> > > > >> > > > > > installation on Mars that has pandas and cartopy
> > installed.
> > > > >> > > > > >
> > > > >> > > > > > Thanks,
> > > > >> > > > > > Logan
> > > > >> > > > > >
> > > > >> > > > > > On Wed, Jan 22, 2020 at 3:12 PM George McCabe via
RT
> > > > >> > > > > > <
> > > > >> > > > met_help at ucar.edu>
> > > > >> > > > > > wrote:
> > > > >> > > > > >
> > > > >> > > > > > > Hi Logan,
> > > > >> > > > > > >
> > > > >> > > > > > > It looks like there is a bug in running
> > > > >> > > > > > > RegridDataPlane
> > on
> > > > >> FCST
> > > > >> > > data
> > > > >> > > > in
> > > > >> > > > > > > METplus 2.1. It appears to have been fixed in
the
> > > > >> > > > > > > latest
> > > > beta
> > > > >> > > > release,
> > > > >> > > > > > but
> > > > >> > > > > > > I am noticing there are no use cases examples
that
> > > > >> > > > > > > run
> > > > >> > > > RegridDataPlane
> > > > >> > > > > on
> > > > >> > > > > > > forecast data. Would you be willing to take the
> > > > >> > > > > > > METplus
> > > > >> 3.0-beta2
> > > > >> > > > > release
> > > > >> > > > > > > and test it with your use case to make sure
that
> > > > >> > > > > > > it does
> > > > >> indeed
> > > > >> > > work
> > > > >> > > > > > > correctly? If so, I can provide you with the
beta
> > release.
> > > > You
> > > > >> > will
> > > > >> > > > > need
> > > > >> > > > > > to
> > > > >> > > > > > > have a Python version that is 3.6 or higher
with
> > > > >> > > > > > > the
> > > cartopy
> > > > >> and
> > > > >> > > > pandas
> > > > >> > > > > > > packages installed. If that is not available, I
> > > > >> > > > > > > can give
> > > you
> > > > >> > > > > instructions
> > > > >> > > > > > > on how to set up a conda environment to test.
Let
> > > > >> > > > > > > me
> > know
> > > > what
> > > > >> > you
> > > > >> > > > > think.
> > > > >> > > > > > >
> > > > >> > > > > > > Thanks,
> > > > >> > > > > > > George
> > > > >> > > > > > >
> > > > >> > > > > > > On Wed, Jan 22, 2020 at 12:58 PM Logan Dawson -
> > > > >> > > > > > > NOAA
> > > > Affiliate
> > > > >> > via
> > > > >> > > > RT <
> > > > >> > > > > > > met_help at ucar.edu> wrote:
> > > > >> > > > > > >
> > > > >> > > > > > > >
> > > > >> > > > > > > > Wed Jan 22 12:58:46 2020: Request 93886 was
> > > > >> > > > > > > > acted
> > upon.
> > > > >> > > > > > > > Transaction: Ticket created by
> > > > >> > > > > > > > logan.dawson at noaa.gov
> > > > >> > > > > > > > Queue: met_help
> > > > >> > > > > > > > Subject: Running RegridDataPlane wrapper
> > > > >> > > > > > > > for fcst
> > > and
> > > > >> obs
> > > > >> > > file
> > > > >> > > > > > > > Owner: Nobody
> > > > >> > > > > > > > Requestors: logan.dawson at noaa.gov
> > > > >> > > > > > > > Status: new
> > > > >> > > > > > > > Ticket <URL:
> > > > >> > > > >
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886
> > > > >> > > > > > >
> > > > >> > > > > > > >
> > > > >> > > > > > > >
> > > > >> > > > > > > > Hi,
> > > > >> > > > > > > >
> > > > >> > > > > > > > I've been running the GridStat METplus
wrapper
> > > > >> > > > > > > > to
> > > verify 3
> > > > >> > > > different
> > > > >> > > > > > > > simulated reflectivity-related products from
7
> > different
> > > > >> hi-res
> > > > >> > > > > models.
> > > > >> > > > > > > > Since I want to fairly compare the 7 models,
I'm
> > using a
> > > > >> > standard
> > > > >> > > > > > > > verification grid.
> > > > >> > > > > > > >
> > > > >> > > > > > > > I've realized that simply allowing the
GridStat
> > wrapper
> > > to
> > > > >> > regrid
> > > > >> > > > the
> > > > >> > > > > > > > forecast and observation files is highly
> > > > >> > > > > > > > inefficient
> > > > because
> > > > >> > > > > regridding
> > > > >> > > > > > > > occurs with every forecast lead/valid
> > > > >> > > > > > > > combination,
> > > > >> regardless
> > > > >> > of
> > > > >> > > > > > whether
> > > > >> > > > > > > > the obs or fcst file has already been
regridded.
> > > > >> > > > > > > > So
> > now,
> > > > I'm
> > > > >> > > trying
> > > > >> > > > > to
> > > > >> > > > > > > add
> > > > >> > > > > > > > the RegridDataPlane wrapper to my process
list,
> > > > >> > > > > > > > but it
> > > > >> appears
> > > > >> > > that
> > > > >> > > > > I'm
> > > > >> > > > > > > not
> > > > >> > > > > > > > able to regrid a fcst file and an obs file
> > > simultaneously?
> > > > >> > > > > > > >
> > > > >> > > > > > > > I get the following error (on the command
line
> > > > >> > > > > > > > or in
> > > > stderr
> > > > >> of
> > > > >> > a
> > > > >> > > > job
> > > > >> > > > > > > > script) when I try to run RegridDataPlane
with
> > > > >> > > > > > > > both :
> > > > >> > > > > > > >
> > > > >> > > > > > > > 01/22 19:05:33.039 jlog
(master_metplus.py:190)
> > > CRITICAL:
> > > > >> > > > > > master_metplus
> > > > >> > > > > > > > failed: getdir() takes at most 3 arguments (4
> > > > >> > > > > > > > given)
> > > > >> > > > > > > >
> > > > >> > > > > > > > Traceback (most recent call last):
> > > > >> > > > > > > >
> > > > >> > > > > > > > File
> > > > >> > > > > > > >
> > > > >> > > > > >
> > > > >> > > >
> > > > >> >
> > > > >>
> > > >
> > "/usrx/local/dev/packages/met/METplus/METplus-
> > 2.1/ush/master_metplus.py",
> > > > >> > > > > > > > line 187, in <module>
> > > > >> > > > > > > >
> > > > >> > > > > > > > main()
> > > > >> > > > > > > >
> > > > >> > > > > > > > File
> > > > >> > > > > > > >
> > > > >> > > > > >
> > > > >> > > >
> > > > >> >
> > > > >>
> > > >
> > "/usrx/local/dev/packages/met/METplus/METplus-
> > 2.1/ush/master_metplus.py",
> > > > >> > > > > > > > line 120, in main
> > > > >> > > > > > > >
> > > > >> > > > > > > > item + "Wrapper")(p, logger)
> > > > >> > > > > > > >
> > > > >> > > > > > > > File
> > > > >> > > > > > > >
> > > > >> > > > > > > >
> > > > >> > > > > > >
> > > > >> > > > > >
> > > > >> > > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> > "/gpfs/dell3/usrx/local/dev/packages/met/METplus/METplus-
> > 2.1/ush/regrid_data_plane_wrapper.py",
> > > > >> > > > > > > > line 39, in __init__
> > > > >> > > > > > > >
> > > > >> > > > > > > > self.create_c_dict()
> > > > >> > > > > > > >
> > > > >> > > > > > > > File
> > > > >> > > > > > > >
> > > > >> > > > > > > >
> > > > >> > > > > > >
> > > > >> > > > > >
> > > > >> > > > >
> > > > >> > > >
> > > > >> > >
> > > > >> >
> > > > >>
> > > >
> > >
> > "/gpfs/dell3/usrx/local/dev/packages/met/METplus/METplus-
> > 2.1/ush/regrid_data_plane_wrapper.py",
> > > > >> > > > > > > > line 104, in create_c_dict
> > > > >> > > > > > > >
> > > > >> > > > > > > > '', self.logger)
> > > > >> > > > > > > >
> > > > >> > > > > > > > TypeError: getdir() takes at most 3 arguments
(4
> > given)
> > > > >> > > > > > > >
> > > > >> > > > > > > > 01/22 19:05:33Z run-METplus-CRITICAL:
> > > > >> > > > > > > > master_metplus
> > > > >> failed:
> > > > >> > > > > getdir()
> > > > >> > > > > > > > takes at most 3 arguments (4 given)
> > > > >> > > > > > > >
> > > > >> > > > > > > > In this attempt, I have
> > > > >> > > > > > > > FCST_REGRID_DATA_PLANE_RUN and
> > > > >> > > > > > > > OBS_REGRID_DATA_PLANE_RUN both set to True,
and
> > > > >> > > > > > > > I have
> > > > input
> > > > >> > and
> > > > >> > > > > output
> > > > >> > > > > > > > directories set for both fcst and obs. I
tried
> > > > >> > > > > > > > setting
> > > > >> > FCST...RUN
> > > > >> > > > to
> > > > >> > > > > > > false,
> > > > >> > > > > > > > and I didn't get that error.
> > > > >> > > > > > > >
> > > > >> > > > > > > > Is there any way I can run RegridDataPlane
for
> > > > >> > > > > > > > both
> > > > forecast
> > > > >> > and
> > > > >> > > > obs
> > > > >> > > > > in
> > > > >> > > > > > > the
> > > > >> > > > > > > > same process list? Or do I need to regrid the
> > > > >> > > > > > > > obs
> > files
> > > > >> prior
> > > > >> > to
> > > > >> > > > the
> > > > >> > > > > > > > METplus call that runs RegridDataPlane for
the
> > > > >> > > > > > > > fcst
> > > files
> > > > >> prior
> > > > >> > > to
> > > > >> > > > > > > running
> > > > >> > > > > > > > GridStat?
> > > > >> > > > > > > >
> > > > >> > > > > > > > Thanks,
> > > > >> > > > > > > > Logan
> > > > >> > > > > > > >
> > > > >> > > > > > > > --
> > > > >> > > > > > > > *Logan C. Dawson, Ph.D.*
> > > > >> > > > > > > > NOAA/NWS/NCEP/EMC
> > > > >> > > > > > > > 5830 University Research Court
> > > > >> > > > > > > > College Park, MD 20740
> > > > >> > > > > > > > (301) 683-3944
> > > > >> > > > > > > >
> > > > >> > > > > > > >
> > > > >> > > > > > >
> > > > >> > > > > > > --
> > > > >> > > > > > > George McCabe - Software Engineer III
> > > > >> > > > > > > National Center for Atmospheric Research
> > > > >> > > > > > > Research Applications Laboratory
> > > > >> > > > > > > 303-497-2768
> > > > >> > > > > > > ---
> > > > >> > > > > > > My working day may not be your working day.
Please
> > > > >> > > > > > > do
> > not
> > > > feel
> > > > >> > > > obliged
> > > > >> > > > > to
> > > > >> > > > > > > reply to this email outside of your normal
working
> > hours.
> > > > >> > > > > > >
> > > > >> > > > > > >
> > > > >> > > > > >
> > > > >> > > > > > --
> > > > >> > > > > > *Logan C. Dawson, Ph.D.*
> > > > >> > > > > > NOAA/NWS/NCEP/EMC
> > > > >> > > > > > 5830 University Research Court
> > > > >> > > > > > College Park, MD 20740
> > > > >> > > > > > (301) 683-3944
> > > > >> > > > > >
> > > > >> > > > > >
> > > > >> > > > >
> > > > >> > > > > --
> > > > >> > > > > George McCabe - Software Engineer III
> > > > >> > > > > National Center for Atmospheric Research
> > > > >> > > > > Research Applications Laboratory
> > > > >> > > > > 303-497-2768
> > > > >> > > > > ---
> > > > >> > > > > My working day may not be your working day. Please
do
> > > > >> > > > > not
> > feel
> > > > >> > obliged
> > > > >> > > to
> > > > >> > > > > reply to this email outside of your normal working
> > > > >> > > > > hours.
> > > > >> > > > >
> > > > >> > > > >
> > > > >> > > >
> > > > >> > > > --
> > > > >> > > > *Logan C. Dawson, Ph.D.*
> > > > >> > > > NOAA/NWS/NCEP/EMC
> > > > >> > > > 5830 University Research Court
> > > > >> > > > College Park, MD 20740
> > > > >> > > > (301) 683-3944
> > > > >> > > >
> > > > >> > > >
> > > > >> > >
> > > > >> > > --
> > > > >> > > George McCabe - Software Engineer III
> > > > >> > > National Center for Atmospheric Research
> > > > >> > > Research Applications Laboratory
> > > > >> > > 303-497-2768
> > > > >> > > ---
> > > > >> > > My working day may not be your working day. Please do
not
> > > > >> > > feel
> > > > >> obliged to
> > > > >> > > reply to this email outside of your normal working
hours.
> > > > >> > >
> > > > >> > >
> > > > >> >
> > > > >> > --
> > > > >> > *Logan C. Dawson, Ph.D.*
> > > > >> > NOAA/NWS/NCEP/EMC
> > > > >> > 5830 University Research Court
> > > > >> > College Park, MD 20740
> > > > >> > (301) 683-3944
> > > > >> >
> > > > >> >
> > > > >>
> > > > >> --
> > > > >> George McCabe - Software Engineer III
> > > > >> National Center for Atmospheric Research
> > > > >> Research Applications Laboratory
> > > > >> 303-497-2768
> > > > >> ---
> > > > >> My working day may not be your working day. Please do not
feel
> > obliged
> > > > to
> > > > >> reply to this email outside of your normal working hours.
> > > > >>
> > > > >>
> > > > >
> > > > > --
> > > > > *Logan C. Dawson, Ph.D.*
> > > > > NOAA/NWS/NCEP/EMC
> > > > > 5830 University Research Court
> > > > > College Park, MD 20740
> > > > > (301) 683-3944
> > > > >
> > > >
> > > >
> > > > --
> > > > *Logan C. Dawson, Ph.D.*
> > > > NOAA/NWS/NCEP/EMC
> > > > 5830 University Research Court
> > > > College Park, MD 20740
> > > > (301) 683-3944
> > > >
> > > >
> > >
> > > --
> > > George McCabe - Software Engineer III
> > > National Center for Atmospheric Research
> > > Research Applications Laboratory
> > > 303-497-2768
> > > ---
> > > My working day may not be your working day. Please do not feel
> > > obliged to
> > > reply to this email outside of your normal working hours.
> > >
> > >
> >
> > --
> > *Logan C. Dawson, Ph.D.*
> > NOAA/NWS/NCEP/EMC
> > 5830 University Research Court
> > College Park, MD 20740
> > (301) 683-3944
> >
> >
------------------------------------------------
More information about the Met_help
mailing list