[Met_help] [rt.rap.ucar.edu #93886] History for Running RegridDataPlane wrapper for fcst and obs file
George McCabe via RT
met_help at ucar.edu
Wed Jan 22 16:27:58 MST 2020
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hi,
I've been running the GridStat METplus wrapper to verify 3 different
simulated reflectivity-related products from 7 different hi-res models.
Since I want to fairly compare the 7 models, I'm using a standard
verification grid.
I've realized that simply allowing the GridStat wrapper to regrid the
forecast and observation files is highly inefficient because regridding
occurs with every forecast lead/valid combination, regardless of whether
the obs or fcst file has already been regridded. So now, I'm trying to add
the RegridDataPlane wrapper to my process list, but it appears that I'm not
able to regrid a fcst file and an obs file simultaneously?
I get the following error (on the command line or in stderr of a job
script) when I try to run RegridDataPlane with both :
01/22 19:05:33.039 jlog (master_metplus.py:190) CRITICAL: master_metplus
failed: getdir() takes at most 3 arguments (4 given)
Traceback (most recent call last):
File
"/usrx/local/dev/packages/met/METplus/METplus-2.1/ush/master_metplus.py",
line 187, in <module>
main()
File
"/usrx/local/dev/packages/met/METplus/METplus-2.1/ush/master_metplus.py",
line 120, in main
item + "Wrapper")(p, logger)
File
"/gpfs/dell3/usrx/local/dev/packages/met/METplus/METplus-2.1/ush/regrid_data_plane_wrapper.py",
line 39, in __init__
self.create_c_dict()
File
"/gpfs/dell3/usrx/local/dev/packages/met/METplus/METplus-2.1/ush/regrid_data_plane_wrapper.py",
line 104, in create_c_dict
'', self.logger)
TypeError: getdir() takes at most 3 arguments (4 given)
01/22 19:05:33Z run-METplus-CRITICAL: master_metplus failed: getdir()
takes at most 3 arguments (4 given)
In this attempt, I have FCST_REGRID_DATA_PLANE_RUN and
OBS_REGRID_DATA_PLANE_RUN both set to True, and I have input and output
directories set for both fcst and obs. I tried setting FCST...RUN to false,
and I didn't get that error.
Is there any way I can run RegridDataPlane for both forecast and obs in the
same process list? Or do I need to regrid the obs files prior to the
METplus call that runs RegridDataPlane for the fcst files prior to running
GridStat?
Thanks,
Logan
--
*Logan C. Dawson, Ph.D.*
NOAA/NWS/NCEP/EMC
5830 University Research Court
College Park, MD 20740
(301) 683-3944
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Running RegridDataPlane wrapper for fcst and obs file
From: George McCabe
Time: Wed Jan 22 13:12:00 2020
Hi Logan,
It looks like there is a bug in running RegridDataPlane on FCST data
in
METplus 2.1. It appears to have been fixed in the latest beta release,
but
I am noticing there are no use cases examples that run RegridDataPlane
on
forecast data. Would you be willing to take the METplus 3.0-beta2
release
and test it with your use case to make sure that it does indeed work
correctly? If so, I can provide you with the beta release. You will
need to
have a Python version that is 3.6 or higher with the cartopy and
pandas
packages installed. If that is not available, I can give you
instructions
on how to set up a conda environment to test. Let me know what you
think.
Thanks,
George
On Wed, Jan 22, 2020 at 12:58 PM Logan Dawson - NOAA Affiliate via RT
<
met_help at ucar.edu> wrote:
>
> Wed Jan 22 12:58:46 2020: Request 93886 was acted upon.
> Transaction: Ticket created by logan.dawson at noaa.gov
> Queue: met_help
> Subject: Running RegridDataPlane wrapper for fcst and obs file
> Owner: Nobody
> Requestors: logan.dawson at noaa.gov
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886 >
>
>
> Hi,
>
> I've been running the GridStat METplus wrapper to verify 3 different
> simulated reflectivity-related products from 7 different hi-res
models.
> Since I want to fairly compare the 7 models, I'm using a standard
> verification grid.
>
> I've realized that simply allowing the GridStat wrapper to regrid
the
> forecast and observation files is highly inefficient because
regridding
> occurs with every forecast lead/valid combination, regardless of
whether
> the obs or fcst file has already been regridded. So now, I'm trying
to add
> the RegridDataPlane wrapper to my process list, but it appears that
I'm not
> able to regrid a fcst file and an obs file simultaneously?
>
> I get the following error (on the command line or in stderr of a job
> script) when I try to run RegridDataPlane with both :
>
> 01/22 19:05:33.039 jlog (master_metplus.py:190) CRITICAL:
master_metplus
> failed: getdir() takes at most 3 arguments (4 given)
>
> Traceback (most recent call last):
>
> File
> "/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/master_metplus.py",
> line 187, in <module>
>
> main()
>
> File
> "/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/master_metplus.py",
> line 120, in main
>
> item + "Wrapper")(p, logger)
>
> File
>
> "/gpfs/dell3/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/regrid_data_plane_wrapper.py",
> line 39, in __init__
>
> self.create_c_dict()
>
> File
>
> "/gpfs/dell3/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/regrid_data_plane_wrapper.py",
> line 104, in create_c_dict
>
> '', self.logger)
>
> TypeError: getdir() takes at most 3 arguments (4 given)
>
> 01/22 19:05:33Z run-METplus-CRITICAL: master_metplus failed:
getdir()
> takes at most 3 arguments (4 given)
>
> In this attempt, I have FCST_REGRID_DATA_PLANE_RUN and
> OBS_REGRID_DATA_PLANE_RUN both set to True, and I have input and
output
> directories set for both fcst and obs. I tried setting FCST...RUN to
false,
> and I didn't get that error.
>
> Is there any way I can run RegridDataPlane for both forecast and obs
in the
> same process list? Or do I need to regrid the obs files prior to the
> METplus call that runs RegridDataPlane for the fcst files prior to
running
> GridStat?
>
> Thanks,
> Logan
>
> --
> *Logan C. Dawson, Ph.D.*
> NOAA/NWS/NCEP/EMC
> 5830 University Research Court
> College Park, MD 20740
> (301) 683-3944
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: Running RegridDataPlane wrapper for fcst and obs file
From: Logan Dawson - NOAA Affiliate
Time: Wed Jan 22 13:18:24 2020
Hi George,
I'd be happy to test out the beta release. There's a Python 3.6.3
installation on Mars that has pandas and cartopy installed.
Thanks,
Logan
On Wed, Jan 22, 2020 at 3:12 PM George McCabe via RT
<met_help at ucar.edu>
wrote:
> Hi Logan,
>
> It looks like there is a bug in running RegridDataPlane on FCST data
in
> METplus 2.1. It appears to have been fixed in the latest beta
release, but
> I am noticing there are no use cases examples that run
RegridDataPlane on
> forecast data. Would you be willing to take the METplus 3.0-beta2
release
> and test it with your use case to make sure that it does indeed work
> correctly? If so, I can provide you with the beta release. You will
need to
> have a Python version that is 3.6 or higher with the cartopy and
pandas
> packages installed. If that is not available, I can give you
instructions
> on how to set up a conda environment to test. Let me know what you
think.
>
> Thanks,
> George
>
> On Wed, Jan 22, 2020 at 12:58 PM Logan Dawson - NOAA Affiliate via
RT <
> met_help at ucar.edu> wrote:
>
> >
> > Wed Jan 22 12:58:46 2020: Request 93886 was acted upon.
> > Transaction: Ticket created by logan.dawson at noaa.gov
> > Queue: met_help
> > Subject: Running RegridDataPlane wrapper for fcst and obs
file
> > Owner: Nobody
> > Requestors: logan.dawson at noaa.gov
> > Status: new
> > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886 >
> >
> >
> > Hi,
> >
> > I've been running the GridStat METplus wrapper to verify 3
different
> > simulated reflectivity-related products from 7 different hi-res
models.
> > Since I want to fairly compare the 7 models, I'm using a standard
> > verification grid.
> >
> > I've realized that simply allowing the GridStat wrapper to regrid
the
> > forecast and observation files is highly inefficient because
regridding
> > occurs with every forecast lead/valid combination, regardless of
whether
> > the obs or fcst file has already been regridded. So now, I'm
trying to
> add
> > the RegridDataPlane wrapper to my process list, but it appears
that I'm
> not
> > able to regrid a fcst file and an obs file simultaneously?
> >
> > I get the following error (on the command line or in stderr of a
job
> > script) when I try to run RegridDataPlane with both :
> >
> > 01/22 19:05:33.039 jlog (master_metplus.py:190) CRITICAL:
master_metplus
> > failed: getdir() takes at most 3 arguments (4 given)
> >
> > Traceback (most recent call last):
> >
> > File
> > "/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/master_metplus.py",
> > line 187, in <module>
> >
> > main()
> >
> > File
> > "/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/master_metplus.py",
> > line 120, in main
> >
> > item + "Wrapper")(p, logger)
> >
> > File
> >
> >
> "/gpfs/dell3/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/regrid_data_plane_wrapper.py",
> > line 39, in __init__
> >
> > self.create_c_dict()
> >
> > File
> >
> >
> "/gpfs/dell3/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/regrid_data_plane_wrapper.py",
> > line 104, in create_c_dict
> >
> > '', self.logger)
> >
> > TypeError: getdir() takes at most 3 arguments (4 given)
> >
> > 01/22 19:05:33Z run-METplus-CRITICAL: master_metplus failed:
getdir()
> > takes at most 3 arguments (4 given)
> >
> > In this attempt, I have FCST_REGRID_DATA_PLANE_RUN and
> > OBS_REGRID_DATA_PLANE_RUN both set to True, and I have input and
output
> > directories set for both fcst and obs. I tried setting FCST...RUN
to
> false,
> > and I didn't get that error.
> >
> > Is there any way I can run RegridDataPlane for both forecast and
obs in
> the
> > same process list? Or do I need to regrid the obs files prior to
the
> > METplus call that runs RegridDataPlane for the fcst files prior to
> running
> > GridStat?
> >
> > Thanks,
> > Logan
> >
> > --
> > *Logan C. Dawson, Ph.D.*
> > NOAA/NWS/NCEP/EMC
> > 5830 University Research Court
> > College Park, MD 20740
> > (301) 683-3944
> >
> >
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>
--
*Logan C. Dawson, Ph.D.*
NOAA/NWS/NCEP/EMC
5830 University Research Court
College Park, MD 20740
(301) 683-3944
------------------------------------------------
Subject: Running RegridDataPlane wrapper for fcst and obs file
From: George McCabe
Time: Wed Jan 22 13:45:23 2020
Hi Logan,
Perfect, thank you!
The release can be found here:
https://github.com/NCAR/METplus/releases/tag/v3.0-beta2
You will have to download one of the source code tarballs and extract
the
files locally. There will likely be a few configuration variable name
changes involved in migrating your config files from METplus 2 to 3.
You
can either run master_metplus.py with all of the config files you use
passed in and it will report errors regarding the variables you need
to
change. You can also run the script validate_config.py the same way as
the
master script (passing in all configs with -c) and it will prompt you
to
confirm automatically replacing config variables with the updated
equivalent. Please let me know if you have any issues with this, or if
anything is unclear in this process. That will help refine the process
for
other users when the 3.0 release is available.
Thanks,
George
On Wed, Jan 22, 2020 at 1:18 PM Logan Dawson - NOAA Affiliate via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886 >
>
> Hi George,
>
> I'd be happy to test out the beta release. There's a Python 3.6.3
> installation on Mars that has pandas and cartopy installed.
>
> Thanks,
> Logan
>
> On Wed, Jan 22, 2020 at 3:12 PM George McCabe via RT
<met_help at ucar.edu>
> wrote:
>
> > Hi Logan,
> >
> > It looks like there is a bug in running RegridDataPlane on FCST
data in
> > METplus 2.1. It appears to have been fixed in the latest beta
release,
> but
> > I am noticing there are no use cases examples that run
RegridDataPlane on
> > forecast data. Would you be willing to take the METplus 3.0-beta2
release
> > and test it with your use case to make sure that it does indeed
work
> > correctly? If so, I can provide you with the beta release. You
will need
> to
> > have a Python version that is 3.6 or higher with the cartopy and
pandas
> > packages installed. If that is not available, I can give you
instructions
> > on how to set up a conda environment to test. Let me know what you
think.
> >
> > Thanks,
> > George
> >
> > On Wed, Jan 22, 2020 at 12:58 PM Logan Dawson - NOAA Affiliate via
RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > Wed Jan 22 12:58:46 2020: Request 93886 was acted upon.
> > > Transaction: Ticket created by logan.dawson at noaa.gov
> > > Queue: met_help
> > > Subject: Running RegridDataPlane wrapper for fcst and obs
file
> > > Owner: Nobody
> > > Requestors: logan.dawson at noaa.gov
> > > Status: new
> > > Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886
> >
> > >
> > >
> > > Hi,
> > >
> > > I've been running the GridStat METplus wrapper to verify 3
different
> > > simulated reflectivity-related products from 7 different hi-res
models.
> > > Since I want to fairly compare the 7 models, I'm using a
standard
> > > verification grid.
> > >
> > > I've realized that simply allowing the GridStat wrapper to
regrid the
> > > forecast and observation files is highly inefficient because
regridding
> > > occurs with every forecast lead/valid combination, regardless of
> whether
> > > the obs or fcst file has already been regridded. So now, I'm
trying to
> > add
> > > the RegridDataPlane wrapper to my process list, but it appears
that I'm
> > not
> > > able to regrid a fcst file and an obs file simultaneously?
> > >
> > > I get the following error (on the command line or in stderr of a
job
> > > script) when I try to run RegridDataPlane with both :
> > >
> > > 01/22 19:05:33.039 jlog (master_metplus.py:190) CRITICAL:
> master_metplus
> > > failed: getdir() takes at most 3 arguments (4 given)
> > >
> > > Traceback (most recent call last):
> > >
> > > File
> > >
> "/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/master_metplus.py",
> > > line 187, in <module>
> > >
> > > main()
> > >
> > > File
> > >
> "/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/master_metplus.py",
> > > line 120, in main
> > >
> > > item + "Wrapper")(p, logger)
> > >
> > > File
> > >
> > >
> >
> "/gpfs/dell3/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/regrid_data_plane_wrapper.py",
> > > line 39, in __init__
> > >
> > > self.create_c_dict()
> > >
> > > File
> > >
> > >
> >
> "/gpfs/dell3/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/regrid_data_plane_wrapper.py",
> > > line 104, in create_c_dict
> > >
> > > '', self.logger)
> > >
> > > TypeError: getdir() takes at most 3 arguments (4 given)
> > >
> > > 01/22 19:05:33Z run-METplus-CRITICAL: master_metplus failed:
getdir()
> > > takes at most 3 arguments (4 given)
> > >
> > > In this attempt, I have FCST_REGRID_DATA_PLANE_RUN and
> > > OBS_REGRID_DATA_PLANE_RUN both set to True, and I have input and
output
> > > directories set for both fcst and obs. I tried setting
FCST...RUN to
> > false,
> > > and I didn't get that error.
> > >
> > > Is there any way I can run RegridDataPlane for both forecast and
obs in
> > the
> > > same process list? Or do I need to regrid the obs files prior to
the
> > > METplus call that runs RegridDataPlane for the fcst files prior
to
> > running
> > > GridStat?
> > >
> > > Thanks,
> > > Logan
> > >
> > > --
> > > *Logan C. Dawson, Ph.D.*
> > > NOAA/NWS/NCEP/EMC
> > > 5830 University Research Court
> > > College Park, MD 20740
> > > (301) 683-3944
> > >
> > >
> >
> > --
> > George McCabe - Software Engineer III
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > 303-497-2768
> > ---
> > My working day may not be your working day. Please do not feel
obliged to
> > reply to this email outside of your normal working hours.
> >
> >
>
> --
> *Logan C. Dawson, Ph.D.*
> NOAA/NWS/NCEP/EMC
> 5830 University Research Court
> College Park, MD 20740
> (301) 683-3944
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
Subject: Running RegridDataPlane wrapper for fcst and obs file
From: Logan Dawson - NOAA Affiliate
Time: Wed Jan 22 15:27:31 2020
Hi George,
Thanks for providing the location of the beta release and for letting
me
know about the validate_config.py script. I was able to get the beta
and
run the validation script. It was super simple, and I like the
functionality. I need about 6 or 7 config changes, and it was nice to
be
able to bypass manually making each change.
Also, I can confirm that running RegridDataPlane on forecast and
observation files is working as expected in this 3.0 beta release.
Thanks so much for the quick help on this issue! This is really going
to
help me speed up my verification jobs!
Thanks,
Logan
On Wed, Jan 22, 2020 at 3:45 PM George McCabe via RT
<met_help at ucar.edu>
wrote:
> Hi Logan,
>
> Perfect, thank you!
>
> The release can be found here:
> https://github.com/NCAR/METplus/releases/tag/v3.0-beta2
>
> You will have to download one of the source code tarballs and
extract the
> files locally. There will likely be a few configuration variable
name
> changes involved in migrating your config files from METplus 2 to 3.
You
> can either run master_metplus.py with all of the config files you
use
> passed in and it will report errors regarding the variables you need
to
> change. You can also run the script validate_config.py the same way
as the
> master script (passing in all configs with -c) and it will prompt
you to
> confirm automatically replacing config variables with the updated
> equivalent. Please let me know if you have any issues with this, or
if
> anything is unclear in this process. That will help refine the
process for
> other users when the 3.0 release is available.
>
> Thanks,
> George
>
> On Wed, Jan 22, 2020 at 1:18 PM Logan Dawson - NOAA Affiliate via RT
<
> met_help at ucar.edu> wrote:
>
> >
> > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886 >
> >
> > Hi George,
> >
> > I'd be happy to test out the beta release. There's a Python 3.6.3
> > installation on Mars that has pandas and cartopy installed.
> >
> > Thanks,
> > Logan
> >
> > On Wed, Jan 22, 2020 at 3:12 PM George McCabe via RT
<met_help at ucar.edu>
> > wrote:
> >
> > > Hi Logan,
> > >
> > > It looks like there is a bug in running RegridDataPlane on FCST
data in
> > > METplus 2.1. It appears to have been fixed in the latest beta
release,
> > but
> > > I am noticing there are no use cases examples that run
RegridDataPlane
> on
> > > forecast data. Would you be willing to take the METplus 3.0-
beta2
> release
> > > and test it with your use case to make sure that it does indeed
work
> > > correctly? If so, I can provide you with the beta release. You
will
> need
> > to
> > > have a Python version that is 3.6 or higher with the cartopy and
pandas
> > > packages installed. If that is not available, I can give you
> instructions
> > > on how to set up a conda environment to test. Let me know what
you
> think.
> > >
> > > Thanks,
> > > George
> > >
> > > On Wed, Jan 22, 2020 at 12:58 PM Logan Dawson - NOAA Affiliate
via RT <
> > > met_help at ucar.edu> wrote:
> > >
> > > >
> > > > Wed Jan 22 12:58:46 2020: Request 93886 was acted upon.
> > > > Transaction: Ticket created by logan.dawson at noaa.gov
> > > > Queue: met_help
> > > > Subject: Running RegridDataPlane wrapper for fcst and obs
file
> > > > Owner: Nobody
> > > > Requestors: logan.dawson at noaa.gov
> > > > Status: new
> > > > Ticket <URL:
> https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886
> > >
> > > >
> > > >
> > > > Hi,
> > > >
> > > > I've been running the GridStat METplus wrapper to verify 3
different
> > > > simulated reflectivity-related products from 7 different hi-
res
> models.
> > > > Since I want to fairly compare the 7 models, I'm using a
standard
> > > > verification grid.
> > > >
> > > > I've realized that simply allowing the GridStat wrapper to
regrid the
> > > > forecast and observation files is highly inefficient because
> regridding
> > > > occurs with every forecast lead/valid combination, regardless
of
> > whether
> > > > the obs or fcst file has already been regridded. So now, I'm
trying
> to
> > > add
> > > > the RegridDataPlane wrapper to my process list, but it appears
that
> I'm
> > > not
> > > > able to regrid a fcst file and an obs file simultaneously?
> > > >
> > > > I get the following error (on the command line or in stderr of
a job
> > > > script) when I try to run RegridDataPlane with both :
> > > >
> > > > 01/22 19:05:33.039 jlog (master_metplus.py:190) CRITICAL:
> > master_metplus
> > > > failed: getdir() takes at most 3 arguments (4 given)
> > > >
> > > > Traceback (most recent call last):
> > > >
> > > > File
> > > >
> > "/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/master_metplus.py",
> > > > line 187, in <module>
> > > >
> > > > main()
> > > >
> > > > File
> > > >
> > "/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/master_metplus.py",
> > > > line 120, in main
> > > >
> > > > item + "Wrapper")(p, logger)
> > > >
> > > > File
> > > >
> > > >
> > >
> >
> "/gpfs/dell3/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/regrid_data_plane_wrapper.py",
> > > > line 39, in __init__
> > > >
> > > > self.create_c_dict()
> > > >
> > > > File
> > > >
> > > >
> > >
> >
> "/gpfs/dell3/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/regrid_data_plane_wrapper.py",
> > > > line 104, in create_c_dict
> > > >
> > > > '', self.logger)
> > > >
> > > > TypeError: getdir() takes at most 3 arguments (4 given)
> > > >
> > > > 01/22 19:05:33Z run-METplus-CRITICAL: master_metplus failed:
> getdir()
> > > > takes at most 3 arguments (4 given)
> > > >
> > > > In this attempt, I have FCST_REGRID_DATA_PLANE_RUN and
> > > > OBS_REGRID_DATA_PLANE_RUN both set to True, and I have input
and
> output
> > > > directories set for both fcst and obs. I tried setting
FCST...RUN to
> > > false,
> > > > and I didn't get that error.
> > > >
> > > > Is there any way I can run RegridDataPlane for both forecast
and obs
> in
> > > the
> > > > same process list? Or do I need to regrid the obs files prior
to the
> > > > METplus call that runs RegridDataPlane for the fcst files
prior to
> > > running
> > > > GridStat?
> > > >
> > > > Thanks,
> > > > Logan
> > > >
> > > > --
> > > > *Logan C. Dawson, Ph.D.*
> > > > NOAA/NWS/NCEP/EMC
> > > > 5830 University Research Court
> > > > College Park, MD 20740
> > > > (301) 683-3944
> > > >
> > > >
> > >
> > > --
> > > George McCabe - Software Engineer III
> > > National Center for Atmospheric Research
> > > Research Applications Laboratory
> > > 303-497-2768
> > > ---
> > > My working day may not be your working day. Please do not feel
obliged
> to
> > > reply to this email outside of your normal working hours.
> > >
> > >
> >
> > --
> > *Logan C. Dawson, Ph.D.*
> > NOAA/NWS/NCEP/EMC
> > 5830 University Research Court
> > College Park, MD 20740
> > (301) 683-3944
> >
> >
>
> --
> George McCabe - Software Engineer III
> National Center for Atmospheric Research
> Research Applications Laboratory
> 303-497-2768
> ---
> My working day may not be your working day. Please do not feel
obliged to
> reply to this email outside of your normal working hours.
>
>
--
*Logan C. Dawson, Ph.D.*
NOAA/NWS/NCEP/EMC
5830 University Research Court
College Park, MD 20740
(301) 683-3944
------------------------------------------------
Subject: Running RegridDataPlane wrapper for fcst and obs file
From: George McCabe
Time: Wed Jan 22 16:22:49 2020
Great news to hear! I'm glad you were able to get the new version up
and
running quickly and it allowed you to set up your use case. I will
close
this ticket, but please feel free to submit another ticket if you have
any
other issues.
- George
On Wed, Jan 22, 2020 at 3:34 PM Logan Dawson - NOAA Affiliate via RT <
met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886 >
>
> Hi George,
>
> Thanks for providing the location of the beta release and for
letting me
> know about the validate_config.py script. I was able to get the beta
and
> run the validation script. It was super simple, and I like the
> functionality. I need about 6 or 7 config changes, and it was nice
to be
> able to bypass manually making each change.
>
> Also, I can confirm that running RegridDataPlane on forecast and
> observation files is working as expected in this 3.0 beta release.
>
> Thanks so much for the quick help on this issue! This is really
going to
> help me speed up my verification jobs!
>
> Thanks,
> Logan
>
>
>
> On Wed, Jan 22, 2020 at 3:45 PM George McCabe via RT
<met_help at ucar.edu>
> wrote:
>
> > Hi Logan,
> >
> > Perfect, thank you!
> >
> > The release can be found here:
> > https://github.com/NCAR/METplus/releases/tag/v3.0-beta2
> >
> > You will have to download one of the source code tarballs and
extract the
> > files locally. There will likely be a few configuration variable
name
> > changes involved in migrating your config files from METplus 2 to
3. You
> > can either run master_metplus.py with all of the config files you
use
> > passed in and it will report errors regarding the variables you
need to
> > change. You can also run the script validate_config.py the same
way as
> the
> > master script (passing in all configs with -c) and it will prompt
you to
> > confirm automatically replacing config variables with the updated
> > equivalent. Please let me know if you have any issues with this,
or if
> > anything is unclear in this process. That will help refine the
process
> for
> > other users when the 3.0 release is available.
> >
> > Thanks,
> > George
> >
> > On Wed, Jan 22, 2020 at 1:18 PM Logan Dawson - NOAA Affiliate via
RT <
> > met_help at ucar.edu> wrote:
> >
> > >
> > > <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886 >
> > >
> > > Hi George,
> > >
> > > I'd be happy to test out the beta release. There's a Python
3.6.3
> > > installation on Mars that has pandas and cartopy installed.
> > >
> > > Thanks,
> > > Logan
> > >
> > > On Wed, Jan 22, 2020 at 3:12 PM George McCabe via RT <
> met_help at ucar.edu>
> > > wrote:
> > >
> > > > Hi Logan,
> > > >
> > > > It looks like there is a bug in running RegridDataPlane on
FCST data
> in
> > > > METplus 2.1. It appears to have been fixed in the latest beta
> release,
> > > but
> > > > I am noticing there are no use cases examples that run
> RegridDataPlane
> > on
> > > > forecast data. Would you be willing to take the METplus 3.0-
beta2
> > release
> > > > and test it with your use case to make sure that it does
indeed work
> > > > correctly? If so, I can provide you with the beta release. You
will
> > need
> > > to
> > > > have a Python version that is 3.6 or higher with the cartopy
and
> pandas
> > > > packages installed. If that is not available, I can give you
> > instructions
> > > > on how to set up a conda environment to test. Let me know what
you
> > think.
> > > >
> > > > Thanks,
> > > > George
> > > >
> > > > On Wed, Jan 22, 2020 at 12:58 PM Logan Dawson - NOAA Affiliate
via
> RT <
> > > > met_help at ucar.edu> wrote:
> > > >
> > > > >
> > > > > Wed Jan 22 12:58:46 2020: Request 93886 was acted upon.
> > > > > Transaction: Ticket created by logan.dawson at noaa.gov
> > > > > Queue: met_help
> > > > > Subject: Running RegridDataPlane wrapper for fcst and
obs file
> > > > > Owner: Nobody
> > > > > Requestors: logan.dawson at noaa.gov
> > > > > Status: new
> > > > > Ticket <URL:
> > https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=93886
> > > >
> > > > >
> > > > >
> > > > > Hi,
> > > > >
> > > > > I've been running the GridStat METplus wrapper to verify 3
> different
> > > > > simulated reflectivity-related products from 7 different hi-
res
> > models.
> > > > > Since I want to fairly compare the 7 models, I'm using a
standard
> > > > > verification grid.
> > > > >
> > > > > I've realized that simply allowing the GridStat wrapper to
regrid
> the
> > > > > forecast and observation files is highly inefficient because
> > regridding
> > > > > occurs with every forecast lead/valid combination,
regardless of
> > > whether
> > > > > the obs or fcst file has already been regridded. So now, I'm
trying
> > to
> > > > add
> > > > > the RegridDataPlane wrapper to my process list, but it
appears that
> > I'm
> > > > not
> > > > > able to regrid a fcst file and an obs file simultaneously?
> > > > >
> > > > > I get the following error (on the command line or in stderr
of a
> job
> > > > > script) when I try to run RegridDataPlane with both :
> > > > >
> > > > > 01/22 19:05:33.039 jlog (master_metplus.py:190) CRITICAL:
> > > master_metplus
> > > > > failed: getdir() takes at most 3 arguments (4 given)
> > > > >
> > > > > Traceback (most recent call last):
> > > > >
> > > > > File
> > > > >
> > >
> "/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/master_metplus.py",
> > > > > line 187, in <module>
> > > > >
> > > > > main()
> > > > >
> > > > > File
> > > > >
> > >
> "/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/master_metplus.py",
> > > > > line 120, in main
> > > > >
> > > > > item + "Wrapper")(p, logger)
> > > > >
> > > > > File
> > > > >
> > > > >
> > > >
> > >
> >
> "/gpfs/dell3/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/regrid_data_plane_wrapper.py",
> > > > > line 39, in __init__
> > > > >
> > > > > self.create_c_dict()
> > > > >
> > > > > File
> > > > >
> > > > >
> > > >
> > >
> >
> "/gpfs/dell3/usrx/local/dev/packages/met/METplus/METplus-
2.1/ush/regrid_data_plane_wrapper.py",
> > > > > line 104, in create_c_dict
> > > > >
> > > > > '', self.logger)
> > > > >
> > > > > TypeError: getdir() takes at most 3 arguments (4 given)
> > > > >
> > > > > 01/22 19:05:33Z run-METplus-CRITICAL: master_metplus
failed:
> > getdir()
> > > > > takes at most 3 arguments (4 given)
> > > > >
> > > > > In this attempt, I have FCST_REGRID_DATA_PLANE_RUN and
> > > > > OBS_REGRID_DATA_PLANE_RUN both set to True, and I have input
and
> > output
> > > > > directories set for both fcst and obs. I tried setting
FCST...RUN
> to
> > > > false,
> > > > > and I didn't get that error.
> > > > >
> > > > > Is there any way I can run RegridDataPlane for both forecast
and
> obs
> > in
> > > > the
> > > > > same process list? Or do I need to regrid the obs files
prior to
> the
> > > > > METplus call that runs RegridDataPlane for the fcst files
prior to
> > > > running
> > > > > GridStat?
> > > > >
> > > > > Thanks,
> > > > > Logan
> > > > >
> > > > > --
> > > > > *Logan C. Dawson, Ph.D.*
> > > > > NOAA/NWS/NCEP/EMC
> > > > > 5830 University Research Court
> > > > > College Park, MD 20740
> > > > > (301) 683-3944
> > > > >
> > > > >
> > > >
> > > > --
> > > > George McCabe - Software Engineer III
> > > > National Center for Atmospheric Research
> > > > Research Applications Laboratory
> > > > 303-497-2768
> > > > ---
> > > > My working day may not be your working day. Please do not feel
> obliged
> > to
> > > > reply to this email outside of your normal working hours.
> > > >
> > > >
> > >
> > > --
> > > *Logan C. Dawson, Ph.D.*
> > > NOAA/NWS/NCEP/EMC
> > > 5830 University Research Court
> > > College Park, MD 20740
> > > (301) 683-3944
> > >
> > >
> >
> > --
> > George McCabe - Software Engineer III
> > National Center for Atmospheric Research
> > Research Applications Laboratory
> > 303-497-2768
> > ---
> > My working day may not be your working day. Please do not feel
obliged to
> > reply to this email outside of your normal working hours.
> >
> >
>
> --
> *Logan C. Dawson, Ph.D.*
> NOAA/NWS/NCEP/EMC
> 5830 University Research Court
> College Park, MD 20740
> (301) 683-3944
>
>
--
George McCabe - Software Engineer III
National Center for Atmospheric Research
Research Applications Laboratory
303-497-2768
---
My working day may not be your working day. Please do not feel obliged
to
reply to this email outside of your normal working hours.
------------------------------------------------
More information about the Met_help
mailing list