[Met_help] [rt.rap.ucar.edu #100025] History for Help to get python embedding working with grid_stat
John Halley Gotway via RT
met_help at ucar.edu
Tue Jun 1 08:30:46 MDT 2021
----------------------------------------------------------------
Initial Request
----------------------------------------------------------------
Hi metplus support team,
I am struggling to get a test case using python embedding to work correctly. The crux of the problem is copied from the error log (please see attached tarball of experiment files):
DEBUG 3: Running user's python script (/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5fcst2xr).
WARNING:
WARNING: python_dataplane() -> an error occurred importing module "/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5fcst2xr"
WARNING:
ERROR :
ERROR : Trouble reading data from forecast file "PYTHON_NUMPY"
ERROR :
There seems to be an issue passing the numpy array to grid_stat, but I don’t see many more clues regarding what is going wrong. My environment appears to be correct by eyeball. When I call the embedded scripts in a stand-along manor as in “GridStat_py-embed_test/logs/.all_commands.20210527141605”, all the output looks correct in these log files:
GridStat_py-embed_test/read_era5pl2xr.log
GridStat_py-embed_test/read_g5fcst2xr.log
I should mention I am using METplus-4.0 and MET 10.0.0 as indicated in my log files. I’m not sure what else to try to diagnose the problem. Do you have any ideas I can try?
Thanks,
Scott
----------------------------------------------------------------
Complete Ticket History
----------------------------------------------------------------
Subject: Help to get python embedding working with grid_stat
From: John Halley Gotway
Time: Thu May 27 16:06:36 2021
Hi Scott,
I see that you're having trouble with python embedding with the Grid-
Stat
tool. Thanks for letting us know what software versions you're using
and
sending sample files to demonstrate.
Whenever getting going with using new gridded datasets in MET,
including
python embedding, I always recommend starting with the plot_data_plane
tool. That'll read the input data and create a plot of it, which tells
you
whether there are any issues with the python script or the orientation
or
location of the data. Based on what you sent, please trying the
following:
(1) First run the python script on the command line to make sure it
runs
without error:
*python3
/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5fcst2xr.py
H 20191221_2100 20191223_1200 P500*
(2) Assuming that worked, try running through plot_data_plane:
*/usr/local/other/met/10.0.0/bin/plot_data_plane PYTHON_NUMPY plot.ps
<http://plot.ps>
'name="/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5fcst2xr.py
H 20191221_2100 20191223_1200 P500";'*
Since grid_stat errored out, I'm going to assume that this errors out
as
well.
If so, please try setting the MET_PYTHON_EXE environment variable to
the
full path of python3 that you used in (1).
*export MET_PYTHON_EXE=`which python3`*
And then retry (2).
Does that get you any further? Here's more information about the
MET_PYTHON_EXE variable:
https://met.readthedocs.io/en/latest/Users_Guide/appendixF.html?highlight=MET_PYTHON_EXE#met-
python-exe
Thanks,
John Halley Gotway
On Thu, May 27, 2021 at 1:51 PM Rabenhorst, Scott D. (GSFC-
610.1)[SCIENCE
SYSTEMS AND APPLICATIONS INC] via RT <met_help at ucar.edu> wrote:
>
> Thu May 27 13:51:19 2021: Request 100025 was acted upon.
> Transaction: Ticket created by scott.d.rabenhorst at nasa.gov
> Queue: met_help
> Subject: Help to get python embedding working with grid_stat
> Owner: Nobody
> Requestors: scott.d.rabenhorst at nasa.gov
> Status: new
> Ticket <URL:
https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=100025 >
>
>
> Hi metplus support team,
>
> I am struggling to get a test case using python embedding to work
> correctly. The crux of the problem is copied from the error log
(please see
> attached tarball of experiment files):
>
> DEBUG 3: Running user's python script
>
(/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5fcst2xr).
> WARNING:
> WARNING: python_dataplane() -> an error occurred importing module
>
"/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5fcst2xr"
> WARNING:
> ERROR :
> ERROR : Trouble reading data from forecast file "PYTHON_NUMPY"
> ERROR :
>
> There seems to be an issue passing the numpy array to grid_stat, but
I
> don’t see many more clues regarding what is going wrong. My
environment
> appears to be correct by eyeball. When I call the embedded scripts
in a
> stand-along manor as in
> “GridStat_py-embed_test/logs/.all_commands.20210527141605”, all the
output
> looks correct in these log files:
>
> GridStat_py-embed_test/read_era5pl2xr.log
> GridStat_py-embed_test/read_g5fcst2xr.log
>
> I should mention I am using METplus-4.0 and MET 10.0.0 as indicated
in my
> log files. I’m not sure what else to try to diagnose the problem. Do
you
> have any ideas I can try?
>
> Thanks,
> Scott
>
>
------------------------------------------------
Subject: Re: [EXTERNAL] Re: [rt.rap.ucar.edu #100025] Help to get python embedding working with grid_stat
From: Rabenhorst, Scott D. (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC]
Time: Fri May 28 14:55:26 2021
Hi John,
Thanks for your quick response. I tried your suggestions below setting
MET_PYTHON_EXE to an environment variable with the full path to the
python I have loaded from a system module. However, I do not know if
MET used this when it was built since it is very difficult to match
Anaconda's internal compilers with that provided on my HPC system - I
know MET is picky things must match. I fear MET was build with some
bare bones version just to accommodate the same compiler. Anyway, then
I ran:
/usr/local/other/met/10.0.0/bin/plot_data_plane PYTHON_NUMPY
/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/metplus_testing_v2/PlotDataPlane_py-
embed_test/plot_data_plane/py_embed_fcst.ps
'name="/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_era5pl2xr.py
H 20191223_1200 P500";' -title "Python Embedding FCST" -v 5
DEBUG 1: Opening data file: PYTHON_NUMPY
DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created new
Met2dDataFile object of type "FileType_Python_Numpy".
DEBUG 4: VarInfoFactory::new_var_info() -> created new VarInfo object
of type "FileType_Python_Numpy".
DEBUG 3: Calling /usr/local/other/python/GEOSpyD/2019.10_py3.7/2020-
01-15/bin/python3 to run user's python script
(/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_era5pl2xr).
Python Script:
/usr/local/other/met/10.0.0/share/met/wrappers/write_pickle_dataplane.py
User Command:
['/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_era5pl2xr',
'H', '20191223_1200', 'P500']
Write Pickle: /tmp/tmp_met_pickle_8524_0
Parsed program arguments:
Namespace(level=500.0, timestamp=datetime.datetime(2019, 12, 23, 12,
0), varname='H')
file:
/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/era5/pres_levels_daily/Y2019/M12/era5_pl-
allvar.20191223_12z.nc4
varname: h (H)
time index: 0 (2019-12-23 12:00:00)
level index: 21 (500.0)
Number missing values in original array: 0
Number missing values in final64 array: 0
met_data obj type: <class 'numpy.ndarray'>
met_data dtype: dtype('float64')
met_data.shape: (721, 1440)
data min/median/max: array([4735.84472656, 5541.72607422,
5959.75878906])
attrs =
{'accum': '00',
'grid': {'Nlat': 721,
'Nlon': 1440,
'delta_lat': 0.25,
'delta_lon': 0.25,
'lat_ll': -90.0,
'lon_ll': -180.0,
'name': 'ERA5_pl',
'type': 'LatLon'},
'init': '20191223_120000',
'lead': '0',
'level': 'P500',
'long_name': 'Geopotential height',
'name': 'h',
'units': 'm',
'valid': '20191223_120000'}
{'attrs': {'valid': '20191223_120000', 'init': '20191223_120000',
'lead': '0', 'accum': '00', 'name': 'h', 'long_name': 'Geopotential
height', 'level': 'P500', 'units': 'm', 'grid': {'type': 'LatLon',
'name': 'ERA5_pl', 'lat_ll': -90.0, 'lon_ll': -180.0, 'delta_lat':
0.25, 'delta_lon': 0.25, 'Nlat': 721, 'Nlon': 1440}}, 'met_data':
array([[5160.1484375 , 5160.1484375 , 5160.1484375 , ..., 5160.1484375
,
5160.1484375 , 5160.1484375 ],
[5156.78320312, 5156.78320312, 5156.7578125 , ...,
5156.80908203,
5156.80908203, 5156.80908203],
[5153.34179688, 5153.31640625, 5153.29101562, ...,
5153.39257812,
5153.3671875 , 5153.3671875 ],
...,
[5225.94580078, 5225.89453125, 5225.84375 , ...,
5226.12402344,
5226.07324219, 5225.99658203],
[5226.02197266, 5225.99658203, 5225.97119141, ...,
5226.12402344,
5226.09863281, 5226.07324219],
[5225.91992188, 5225.91992188, 5225.91992188, ...,
5225.91992188,
5225.91992188, 5225.91992188]])}
DEBUG 3: Reading temporary pickle file: /tmp/tmp_met_pickle_8524_0
WARNING:
WARNING: pickle_dataplane() -> an error occurred importing module
"read_pickle_dataplane"
WARNING:
ERROR :
ERROR : plot_data_plane -> trouble getting field
"name="/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_era5pl2xr.py
H 20191223_1200 P500";" from file "PYTHON_NUMPY"
ERROR :
Does this point to an issue with the python used to build MET 10.0.0?
Or does this look like something else? My environment python
definitely has numpy and my script works fine with that, so I'm
guessing this is related to the version of python MET was built with?
Assuming this is the case, what do you recommend regarding
downloading/getting/building python, or does MET 10.0.0 play nicely
with Anaconda now? Any suggestions would be much appreciated.
Thanks,
Scott
On 5/27/21, 6:06 PM, "John Halley Gotway via RT" <met_help at ucar.edu>
wrote:
Hi Scott,
I see that you're having trouble with python embedding with the
Grid-Stat
tool. Thanks for letting us know what software versions you're
using and
sending sample files to demonstrate.
Whenever getting going with using new gridded datasets in MET,
including
python embedding, I always recommend starting with the
plot_data_plane
tool. That'll read the input data and create a plot of it, which
tells you
whether there are any issues with the python script or the
orientation or
location of the data. Based on what you sent, please trying the
following:
(1) First run the python script on the command line to make sure
it runs
without error:
*python3
/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5fcst2xr.py
H 20191221_2100 20191223_1200 P500*
(2) Assuming that worked, try running through plot_data_plane:
*/usr/local/other/met/10.0.0/bin/plot_data_plane PYTHON_NUMPY
plot.ps
<https://gcc02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fplot.ps%2F&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C48438e0edbcb4b76b32e08d9215bb896%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637577500083517744%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=rdFW5OcdXpFVt3E1zs3g4EvHc2QIePHbdWgRimQ%2BlWw%3D&reserved=0>
'name="/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5fcst2xr.py
H 20191221_2100 20191223_1200 P500";'*
Since grid_stat errored out, I'm going to assume that this errors
out as
well.
If so, please try setting the MET_PYTHON_EXE environment variable
to the
full path of python3 that you used in (1).
*export MET_PYTHON_EXE=`which python3`*
And then retry (2).
Does that get you any further? Here's more information about the
MET_PYTHON_EXE variable:
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmet.readthedocs.io%2Fen%2Flatest%2FUsers_Guide%2FappendixF.html%3Fhighlight%3DMET_PYTHON_EXE%23met-
python-
exe&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C48438e0edbcb4b76b32e08d9215bb896%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637577500083517744%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=GJPoPqijfS9qxFzdyjYCAE%2BnoL2%2FUyIQHgLINaAKTmc%3D&reserved=0
Thanks,
John Halley Gotway
On Thu, May 27, 2021 at 1:51 PM Rabenhorst, Scott D. (GSFC-
610.1)[SCIENCE
SYSTEMS AND APPLICATIONS INC] via RT <met_help at ucar.edu> wrote:
>
> Thu May 27 13:51:19 2021: Request 100025 was acted upon.
> Transaction: Ticket created by scott.d.rabenhorst at nasa.gov
> Queue: met_help
> Subject: Help to get python embedding working with
grid_stat
> Owner: Nobody
> Requestors: scott.d.rabenhorst at nasa.gov
> Status: new
> Ticket <URL:
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100025&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C48438e0edbcb4b76b32e08d9215bb896%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637577500083527706%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=4YmW1bJIDVaql0KqFLEiVQeJEeaFjzzOGb56h1mBJ2U%3D&reserved=0
>
>
>
> Hi metplus support team,
>
> I am struggling to get a test case using python embedding to
work
> correctly. The crux of the problem is copied from the error log
(please see
> attached tarball of experiment files):
>
> DEBUG 3: Running user's python script
>
(/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5fcst2xr).
> WARNING:
> WARNING: python_dataplane() -> an error occurred importing
module
>
"/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5fcst2xr"
> WARNING:
> ERROR :
> ERROR : Trouble reading data from forecast file "PYTHON_NUMPY"
> ERROR :
>
> There seems to be an issue passing the numpy array to grid_stat,
but I
> don’t see many more clues regarding what is going wrong. My
environment
> appears to be correct by eyeball. When I call the embedded
scripts in a
> stand-along manor as in
> “GridStat_py-embed_test/logs/.all_commands.20210527141605”, all
the output
> looks correct in these log files:
>
> GridStat_py-embed_test/read_era5pl2xr.log
> GridStat_py-embed_test/read_g5fcst2xr.log
>
> I should mention I am using METplus-4.0 and MET 10.0.0 as
indicated in my
> log files. I’m not sure what else to try to diagnose the
problem. Do you
> have any ideas I can try?
>
> Thanks,
> Scott
>
>
------------------------------------------------
Subject: Help to get python embedding working with grid_stat
From: John Halley Gotway
Time: Fri May 28 15:11:41 2021
Scott,
I am confused. You state that you're running MET version 10.0.0.
However,
the log message you sent is puzzling:
WARNING: pickle_dataplane() -> an error occurred importing module
"read_pickle_dataplane"
All references to "pickle" have been removed from MET version 10.0.0.
We
have replaced the pickling logic with writing temp files in NetCDF
format.
So there literally should be no more instances of the word "pickle" in
the
MET codebase (other than in comments).
Please check the actual version of the plot_data_plane tool you're
running:
/usr/local/other/met/10.0.0/bin/plot_data_plane --version
Also please make sure that you do not have the MET_BASE environment
variable set, which could be complicating the debugging, if so.
John
On Fri, May 28, 2021 at 2:55 PM Rabenhorst, Scott D. (GSFC-
610.1)[SCIENCE
SYSTEMS AND APPLICATIONS INC] via RT <met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=100025 >
>
> Hi John,
>
> Thanks for your quick response. I tried your suggestions below
setting
> MET_PYTHON_EXE to an environment variable with the full path to the
python
> I have loaded from a system module. However, I do not know if MET
used this
> when it was built since it is very difficult to match Anaconda's
internal
> compilers with that provided on my HPC system - I know MET is picky
things
> must match. I fear MET was build with some bare bones version just
to
> accommodate the same compiler. Anyway, then I ran:
>
>
> /usr/local/other/met/10.0.0/bin/plot_data_plane PYTHON_NUMPY
>
/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/metplus_testing_v2/PlotDataPlane_py-
embed_test/plot_data_plane/
> py_embed_fcst.ps
>
'name="/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_era5pl2xr.py
> H 20191223_1200 P500";' -title "Python Embedding FCST" -v 5
>
> DEBUG 1: Opening data file: PYTHON_NUMPY
> DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created new
> Met2dDataFile object of type "FileType_Python_Numpy".
> DEBUG 4: VarInfoFactory::new_var_info() -> created new VarInfo
object of
> type "FileType_Python_Numpy".
> DEBUG 3: Calling
> /usr/local/other/python/GEOSpyD/2019.10_py3.7/2020-01-15/bin/python3
to run
> user's python script
>
(/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_era5pl2xr).
> Python Script:
>
/usr/local/other/met/10.0.0/share/met/wrappers/write_pickle_dataplane.py
> User Command:
>
['/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_era5pl2xr',
> 'H', '20191223_1200', 'P500']
> Write Pickle: /tmp/tmp_met_pickle_8524_0
>
> Parsed program arguments:
> Namespace(level=500.0, timestamp=datetime.datetime(2019, 12, 23, 12,
0),
> varname='H')
>
> file:
>
/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/era5/pres_levels_daily/Y2019/M12/era5_pl-
allvar.20191223_12z.nc4
> varname: h (H)
> time index: 0 (2019-12-23 12:00:00)
> level index: 21 (500.0)
> Number missing values in original array: 0
> Number missing values in final64 array: 0
> met_data obj type: <class 'numpy.ndarray'>
> met_data dtype: dtype('float64')
> met_data.shape: (721, 1440)
> data min/median/max: array([4735.84472656, 5541.72607422,
5959.75878906])
>
> attrs =
> {'accum': '00',
> 'grid': {'Nlat': 721,
> 'Nlon': 1440,
> 'delta_lat': 0.25,
> 'delta_lon': 0.25,
> 'lat_ll': -90.0,
> 'lon_ll': -180.0,
> 'name': 'ERA5_pl',
> 'type': 'LatLon'},
> 'init': '20191223_120000',
> 'lead': '0',
> 'level': 'P500',
> 'long_name': 'Geopotential height',
> 'name': 'h',
> 'units': 'm',
> 'valid': '20191223_120000'}
>
> {'attrs': {'valid': '20191223_120000', 'init': '20191223_120000',
'lead':
> '0', 'accum': '00', 'name': 'h', 'long_name': 'Geopotential height',
> 'level': 'P500', 'units': 'm', 'grid': {'type': 'LatLon', 'name':
> 'ERA5_pl', 'lat_ll': -90.0, 'lon_ll': -180.0, 'delta_lat': 0.25,
> 'delta_lon': 0.25, 'Nlat': 721, 'Nlon': 1440}}, 'met_data':
> array([[5160.1484375 , 5160.1484375 , 5160.1484375 , ...,
5160.1484375 ,
> 5160.1484375 , 5160.1484375 ],
> [5156.78320312, 5156.78320312, 5156.7578125 , ...,
5156.80908203,
> 5156.80908203, 5156.80908203],
> [5153.34179688, 5153.31640625, 5153.29101562, ...,
5153.39257812,
> 5153.3671875 , 5153.3671875 ],
> ...,
> [5225.94580078, 5225.89453125, 5225.84375 , ...,
5226.12402344,
> 5226.07324219, 5225.99658203],
> [5226.02197266, 5225.99658203, 5225.97119141, ...,
5226.12402344,
> 5226.09863281, 5226.07324219],
> [5225.91992188, 5225.91992188, 5225.91992188, ...,
5225.91992188,
> 5225.91992188, 5225.91992188]])}
> DEBUG 3: Reading temporary pickle file: /tmp/tmp_met_pickle_8524_0
> WARNING:
> WARNING: pickle_dataplane() -> an error occurred importing module
> "read_pickle_dataplane"
> WARNING:
> ERROR :
> ERROR : plot_data_plane -> trouble getting field
>
"name="/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_era5pl2xr.py
> H 20191223_1200 P500";" from file "PYTHON_NUMPY"
> ERROR :
>
>
>
>
>
> Does this point to an issue with the python used to build MET
10.0.0? Or
> does this look like something else? My environment python definitely
has
> numpy and my script works fine with that, so I'm guessing this is
related
> to the version of python MET was built with?
>
> Assuming this is the case, what do you recommend regarding
> downloading/getting/building python, or does MET 10.0.0 play nicely
with
> Anaconda now? Any suggestions would be much appreciated.
>
> Thanks,
> Scott
>
>
>
>
>
>
>
>
> On 5/27/21, 6:06 PM, "John Halley Gotway via RT"
<met_help at ucar.edu>
> wrote:
>
> Hi Scott,
>
> I see that you're having trouble with python embedding with the
> Grid-Stat
> tool. Thanks for letting us know what software versions you're
using
> and
> sending sample files to demonstrate.
>
> Whenever getting going with using new gridded datasets in MET,
> including
> python embedding, I always recommend starting with the
plot_data_plane
> tool. That'll read the input data and create a plot of it, which
tells
> you
> whether there are any issues with the python script or the
orientation
> or
> location of the data. Based on what you sent, please trying the
> following:
>
> (1) First run the python script on the command line to make sure
it
> runs
> without error:
>
> *python3
>
>
/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5fcst2xr.py
> H 20191221_2100 20191223_1200 P500*
>
>
> (2) Assuming that worked, try running through plot_data_plane:
>
>
> */usr/local/other/met/10.0.0/bin/plot_data_plane PYTHON_NUMPY
plot.ps
> <
>
https://gcc02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fplot.ps%2F&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C48438e0edbcb4b76b32e08d9215bb896%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637577500083517744%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=rdFW5OcdXpFVt3E1zs3g4EvHc2QIePHbdWgRimQ%2BlWw%3D&reserved=0
> >
>
>
'name="/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5fcst2xr.py
> H 20191221_2100 20191223_1200 P500";'*
>
>
> Since grid_stat errored out, I'm going to assume that this
errors out
> as
> well.
>
> If so, please try setting the MET_PYTHON_EXE environment
variable to
> the
> full path of python3 that you used in (1).
> *export MET_PYTHON_EXE=`which python3`*
>
> And then retry (2).
>
> Does that get you any further? Here's more information about the
> MET_PYTHON_EXE variable:
>
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmet.readthedocs.io%2Fen%2Flatest%2FUsers_Guide%2FappendixF.html%3Fhighlight%3DMET_PYTHON_EXE%23met-
python-
exe&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C48438e0edbcb4b76b32e08d9215bb896%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637577500083517744%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=GJPoPqijfS9qxFzdyjYCAE%2BnoL2%2FUyIQHgLINaAKTmc%3D&reserved=0
>
> Thanks,
> John Halley Gotway
>
>
> On Thu, May 27, 2021 at 1:51 PM Rabenhorst, Scott D.
> (GSFC-610.1)[SCIENCE
> SYSTEMS AND APPLICATIONS INC] via RT <met_help at ucar.edu> wrote:
>
> >
> > Thu May 27 13:51:19 2021: Request 100025 was acted upon.
> > Transaction: Ticket created by scott.d.rabenhorst at nasa.gov
> > Queue: met_help
> > Subject: Help to get python embedding working with
grid_stat
> > Owner: Nobody
> > Requestors: scott.d.rabenhorst at nasa.gov
> > Status: new
> > Ticket <URL:
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100025&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7C48438e0edbcb4b76b32e08d9215bb896%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637577500083527706%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=4YmW1bJIDVaql0KqFLEiVQeJEeaFjzzOGb56h1mBJ2U%3D&reserved=0
> >
> >
> >
> > Hi metplus support team,
> >
> > I am struggling to get a test case using python embedding to
work
> > correctly. The crux of the problem is copied from the error
log
> (please see
> > attached tarball of experiment files):
> >
> > DEBUG 3: Running user's python script
> >
>
(/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5fcst2xr).
> > WARNING:
> > WARNING: python_dataplane() -> an error occurred importing
module
> >
>
"/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5fcst2xr"
> > WARNING:
> > ERROR :
> > ERROR : Trouble reading data from forecast file
"PYTHON_NUMPY"
> > ERROR :
> >
> > There seems to be an issue passing the numpy array to
grid_stat, but
> I
> > don’t see many more clues regarding what is going wrong. My
> environment
> > appears to be correct by eyeball. When I call the embedded
scripts
> in a
> > stand-along manor as in
> > “GridStat_py-embed_test/logs/.all_commands.20210527141605”,
all the
> output
> > looks correct in these log files:
> >
> > GridStat_py-embed_test/read_era5pl2xr.log
> > GridStat_py-embed_test/read_g5fcst2xr.log
> >
> > I should mention I am using METplus-4.0 and MET 10.0.0 as
indicated
> in my
> > log files. I’m not sure what else to try to diagnose the
problem. Do
> you
> > have any ideas I can try?
> >
> > Thanks,
> > Scott
> >
> >
>
>
>
>
>
------------------------------------------------
Subject: Re: [EXTERNAL] Re: [rt.rap.ucar.edu #100025] Help to get python embedding working with grid_stat
From: Rabenhorst, Scott D. (GSFC-610.1)[SCIENCE SYSTEMS AND APPLICATIONS INC]
Time: Fri May 28 15:27:25 2021
Hi John,
I'm trying to figure this out too since I didn't build MET 10.0.0
myself - it is one of our HPC modules. It looks like it was met-
10.0.0.0-beta2 build around DEC 17th of last year. There is a
subdirectory ./share/met/wrappers/ containing 6 python scripts one of
which is "write_pickle_dataplane.py". I'm guessing this beta version
may not be compatible with metplus-4.0??
I've build met before so I can do it again, however, I have always had
issues with Anaconda vs MET as mentioned before. How do you all do it?
Do you grab python source from somewhere and build it as a dependency
for MET? I'm interested in knowing a little more about how to
get/build my python so MET is happy.
Thanks,
Scott
On 5/28/21, 5:11 PM, "John Halley Gotway via RT" <met_help at ucar.edu>
wrote:
Scott,
I am confused. You state that you're running MET version 10.0.0.
However,
the log message you sent is puzzling:
WARNING: pickle_dataplane() -> an error occurred importing module
"read_pickle_dataplane"
All references to "pickle" have been removed from MET version
10.0.0. We
have replaced the pickling logic with writing temp files in NetCDF
format.
So there literally should be no more instances of the word
"pickle" in the
MET codebase (other than in comments).
Please check the actual version of the plot_data_plane tool you're
running:
/usr/local/other/met/10.0.0/bin/plot_data_plane --version
Also please make sure that you do not have the MET_BASE
environment
variable set, which could be complicating the debugging, if so.
John
On Fri, May 28, 2021 at 2:55 PM Rabenhorst, Scott D. (GSFC-
610.1)[SCIENCE
SYSTEMS AND APPLICATIONS INC] via RT <met_help at ucar.edu> wrote:
>
> <URL:
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100025&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7Cc68ec462e70c4d8f13c508d9221d3657%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637578331183212296%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=6qymQ9JzlO%2FSpghqvxF7cMt%2Bqp8%2BjCfb1NU1fMKYy2s%3D&reserved=0
>
>
> Hi John,
>
> Thanks for your quick response. I tried your suggestions below
setting
> MET_PYTHON_EXE to an environment variable with the full path to
the python
> I have loaded from a system module. However, I do not know if
MET used this
> when it was built since it is very difficult to match Anaconda's
internal
> compilers with that provided on my HPC system - I know MET is
picky things
> must match. I fear MET was build with some bare bones version
just to
> accommodate the same compiler. Anyway, then I ran:
>
>
> /usr/local/other/met/10.0.0/bin/plot_data_plane PYTHON_NUMPY
>
/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/metplus_testing_v2/PlotDataPlane_py-
embed_test/plot_data_plane/
> py_embed_fcst.ps
>
'name="/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_era5pl2xr.py
> H 20191223_1200 P500";' -title "Python Embedding FCST" -v 5
>
> DEBUG 1: Opening data file: PYTHON_NUMPY
> DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() -> created
new
> Met2dDataFile object of type "FileType_Python_Numpy".
> DEBUG 4: VarInfoFactory::new_var_info() -> created new VarInfo
object of
> type "FileType_Python_Numpy".
> DEBUG 3: Calling
> /usr/local/other/python/GEOSpyD/2019.10_py3.7/2020-01-
15/bin/python3 to run
> user's python script
>
(/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_era5pl2xr).
> Python Script:
>
/usr/local/other/met/10.0.0/share/met/wrappers/write_pickle_dataplane.py
> User Command:
>
['/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_era5pl2xr',
> 'H', '20191223_1200', 'P500']
> Write Pickle: /tmp/tmp_met_pickle_8524_0
>
> Parsed program arguments:
> Namespace(level=500.0, timestamp=datetime.datetime(2019, 12, 23,
12, 0),
> varname='H')
>
> file:
>
/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/era5/pres_levels_daily/Y2019/M12/era5_pl-
allvar.20191223_12z.nc4
> varname: h (H)
> time index: 0 (2019-12-23 12:00:00)
> level index: 21 (500.0)
> Number missing values in original array: 0
> Number missing values in final64 array: 0
> met_data obj type: <class 'numpy.ndarray'>
> met_data dtype: dtype('float64')
> met_data.shape: (721, 1440)
> data min/median/max: array([4735.84472656, 5541.72607422,
5959.75878906])
>
> attrs =
> {'accum': '00',
> 'grid': {'Nlat': 721,
> 'Nlon': 1440,
> 'delta_lat': 0.25,
> 'delta_lon': 0.25,
> 'lat_ll': -90.0,
> 'lon_ll': -180.0,
> 'name': 'ERA5_pl',
> 'type': 'LatLon'},
> 'init': '20191223_120000',
> 'lead': '0',
> 'level': 'P500',
> 'long_name': 'Geopotential height',
> 'name': 'h',
> 'units': 'm',
> 'valid': '20191223_120000'}
>
> {'attrs': {'valid': '20191223_120000', 'init':
'20191223_120000', 'lead':
> '0', 'accum': '00', 'name': 'h', 'long_name': 'Geopotential
height',
> 'level': 'P500', 'units': 'm', 'grid': {'type': 'LatLon',
'name':
> 'ERA5_pl', 'lat_ll': -90.0, 'lon_ll': -180.0, 'delta_lat': 0.25,
> 'delta_lon': 0.25, 'Nlat': 721, 'Nlon': 1440}}, 'met_data':
> array([[5160.1484375 , 5160.1484375 , 5160.1484375 , ...,
5160.1484375 ,
> 5160.1484375 , 5160.1484375 ],
> [5156.78320312, 5156.78320312, 5156.7578125 , ...,
5156.80908203,
> 5156.80908203, 5156.80908203],
> [5153.34179688, 5153.31640625, 5153.29101562, ...,
5153.39257812,
> 5153.3671875 , 5153.3671875 ],
> ...,
> [5225.94580078, 5225.89453125, 5225.84375 , ...,
5226.12402344,
> 5226.07324219, 5225.99658203],
> [5226.02197266, 5225.99658203, 5225.97119141, ...,
5226.12402344,
> 5226.09863281, 5226.07324219],
> [5225.91992188, 5225.91992188, 5225.91992188, ...,
5225.91992188,
> 5225.91992188, 5225.91992188]])}
> DEBUG 3: Reading temporary pickle file:
/tmp/tmp_met_pickle_8524_0
> WARNING:
> WARNING: pickle_dataplane() -> an error occurred importing
module
> "read_pickle_dataplane"
> WARNING:
> ERROR :
> ERROR : plot_data_plane -> trouble getting field
>
"name="/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_era5pl2xr.py
> H 20191223_1200 P500";" from file "PYTHON_NUMPY"
> ERROR :
>
>
>
>
>
> Does this point to an issue with the python used to build MET
10.0.0? Or
> does this look like something else? My environment python
definitely has
> numpy and my script works fine with that, so I'm guessing this
is related
> to the version of python MET was built with?
>
> Assuming this is the case, what do you recommend regarding
> downloading/getting/building python, or does MET 10.0.0 play
nicely with
> Anaconda now? Any suggestions would be much appreciated.
>
> Thanks,
> Scott
>
>
>
>
>
>
>
>
> On 5/27/21, 6:06 PM, "John Halley Gotway via RT"
<met_help at ucar.edu>
> wrote:
>
> Hi Scott,
>
> I see that you're having trouble with python embedding with
the
> Grid-Stat
> tool. Thanks for letting us know what software versions
you're using
> and
> sending sample files to demonstrate.
>
> Whenever getting going with using new gridded datasets in
MET,
> including
> python embedding, I always recommend starting with the
plot_data_plane
> tool. That'll read the input data and create a plot of it,
which tells
> you
> whether there are any issues with the python script or the
orientation
> or
> location of the data. Based on what you sent, please trying
the
> following:
>
> (1) First run the python script on the command line to make
sure it
> runs
> without error:
>
> *python3
>
>
/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5fcst2xr.py
> H 20191221_2100 20191223_1200 P500*
>
>
> (2) Assuming that worked, try running through
plot_data_plane:
>
>
> */usr/local/other/met/10.0.0/bin/plot_data_plane
PYTHON_NUMPY plot.ps
> <
>
https://gcc02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fplot.ps%2F&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7Cc68ec462e70c4d8f13c508d9221d3657%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637578331183212296%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=vvCWf6iJ%2FjfeclJkKJELJynWCoZgo95B%2FCnRWrZ%2FBtc%3D&reserved=0
> >
>
>
'name="/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5fcst2xr.py
> H 20191221_2100 20191223_1200 P500";'*
>
>
> Since grid_stat errored out, I'm going to assume that this
errors out
> as
> well.
>
> If so, please try setting the MET_PYTHON_EXE environment
variable to
> the
> full path of python3 that you used in (1).
> *export MET_PYTHON_EXE=`which python3`*
>
> And then retry (2).
>
> Does that get you any further? Here's more information about
the
> MET_PYTHON_EXE variable:
>
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmet.readthedocs.io%2Fen%2Flatest%2FUsers_Guide%2FappendixF.html%3Fhighlight%3DMET_PYTHON_EXE%23met-
python-
exe&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7Cc68ec462e70c4d8f13c508d9221d3657%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637578331183212296%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Kopgt5XyGGuUmTM5XnJF0goMZWBzG3r9l2WPEuWkPEA%3D&reserved=0
>
> Thanks,
> John Halley Gotway
>
>
> On Thu, May 27, 2021 at 1:51 PM Rabenhorst, Scott D.
> (GSFC-610.1)[SCIENCE
> SYSTEMS AND APPLICATIONS INC] via RT <met_help at ucar.edu>
wrote:
>
> >
> > Thu May 27 13:51:19 2021: Request 100025 was acted upon.
> > Transaction: Ticket created by scott.d.rabenhorst at nasa.gov
> > Queue: met_help
> > Subject: Help to get python embedding working with
grid_stat
> > Owner: Nobody
> > Requestors: scott.d.rabenhorst at nasa.gov
> > Status: new
> > Ticket <URL:
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100025&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7Cc68ec462e70c4d8f13c508d9221d3657%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637578331183222253%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=1fNpv2qLOF5bqW%2B4MSzBMXP4VHJGskdG2%2BnCuc32Ct4%3D&reserved=0
> >
> >
> >
> > Hi metplus support team,
> >
> > I am struggling to get a test case using python embedding
to work
> > correctly. The crux of the problem is copied from the
error log
> (please see
> > attached tarball of experiment files):
> >
> > DEBUG 3: Running user's python script
> >
>
(/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5fcst2xr).
> > WARNING:
> > WARNING: python_dataplane() -> an error occurred importing
module
> >
>
"/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5fcst2xr"
> > WARNING:
> > ERROR :
> > ERROR : Trouble reading data from forecast file
"PYTHON_NUMPY"
> > ERROR :
> >
> > There seems to be an issue passing the numpy array to
grid_stat, but
> I
> > don’t see many more clues regarding what is going wrong.
My
> environment
> > appears to be correct by eyeball. When I call the embedded
scripts
> in a
> > stand-along manor as in
> > “GridStat_py-
embed_test/logs/.all_commands.20210527141605”, all the
> output
> > looks correct in these log files:
> >
> > GridStat_py-embed_test/read_era5pl2xr.log
> > GridStat_py-embed_test/read_g5fcst2xr.log
> >
> > I should mention I am using METplus-4.0 and MET 10.0.0 as
indicated
> in my
> > log files. I’m not sure what else to try to diagnose the
problem. Do
> you
> > have any ideas I can try?
> >
> > Thanks,
> > Scott
> >
> >
>
>
>
>
>
------------------------------------------------
Subject: Help to get python embedding working with grid_stat
From: John Halley Gotway
Time: Fri May 28 16:00:00 2021
Scott,
We created 5 different development versions on our way toward the
official
MET version 10.0.0 release. Those development releases (named beta1,
2, 3,
4, and 5) are only intended for testing. Stripping out the pickle
logic
happened after the beta2 release. Before debugging the python
functionality
any further, I'd recommend compiling/running with the official MET
version
10.0.0 release.
Julie Prestopnik is my colleague who's compiled many versions of MET
in
many places. She uses the compile_MET_all.sh script mentioned on the
Downloads page:
https://dtcenter.org/community-code/model-evaluation-tools-
met/download
And that script actually does live in the MET repo
<https://github.com/dtcenter/MET/blob/main_v10.0/scripts/installation/compile_MET_all.sh>.
The MET_PYTHON_CC and MET_PYTHON_LD environment variables at
compilation
time control the version of python that MET uses at runtime. So those
should be set well prior to compiling and you need to use the
"--enable-python" option when running the "configure" command to
enable
compiling python support.
Please see appendix F of the user's guide for more info:
https://met.readthedocs.io/en/latest/Users_Guide/appendixF.html
Thanks,
John
On Fri, May 28, 2021 at 3:27 PM Rabenhorst, Scott D. (GSFC-
610.1)[SCIENCE
SYSTEMS AND APPLICATIONS INC] via RT <met_help at ucar.edu> wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=100025 >
>
> Hi John,
>
> I'm trying to figure this out too since I didn't build MET 10.0.0
myself -
> it is one of our HPC modules. It looks like it was met-10.0.0.0-
beta2 build
> around DEC 17th of last year. There is a subdirectory
./share/met/wrappers/
> containing 6 python scripts one of which is
"write_pickle_dataplane.py".
> I'm guessing this beta version may not be compatible with metplus-
4.0??
>
> I've build met before so I can do it again, however, I have always
had
> issues with Anaconda vs MET as mentioned before. How do you all do
it? Do
> you grab python source from somewhere and build it as a dependency
for MET?
> I'm interested in knowing a little more about how to get/build my
python so
> MET is happy.
>
> Thanks,
> Scott
>
> On 5/28/21, 5:11 PM, "John Halley Gotway via RT"
<met_help at ucar.edu>
> wrote:
>
> Scott,
>
> I am confused. You state that you're running MET version 10.0.0.
> However,
> the log message you sent is puzzling:
>
> WARNING: pickle_dataplane() -> an error occurred importing
module
> "read_pickle_dataplane"
>
> All references to "pickle" have been removed from MET version
10.0.0.
> We
> have replaced the pickling logic with writing temp files in
NetCDF
> format.
> So there literally should be no more instances of the word
"pickle" in
> the
> MET codebase (other than in comments).
>
> Please check the actual version of the plot_data_plane tool
you're
> running:
>
> /usr/local/other/met/10.0.0/bin/plot_data_plane --version
>
> Also please make sure that you do not have the MET_BASE
environment
> variable set, which could be complicating the debugging, if so.
>
> John
>
>
> On Fri, May 28, 2021 at 2:55 PM Rabenhorst, Scott D.
> (GSFC-610.1)[SCIENCE
> SYSTEMS AND APPLICATIONS INC] via RT <met_help at ucar.edu> wrote:
>
> >
> > <URL:
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100025&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7Cc68ec462e70c4d8f13c508d9221d3657%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637578331183212296%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=6qymQ9JzlO%2FSpghqvxF7cMt%2Bqp8%2BjCfb1NU1fMKYy2s%3D&reserved=0
> >
> >
> > Hi John,
> >
> > Thanks for your quick response. I tried your suggestions below
> setting
> > MET_PYTHON_EXE to an environment variable with the full path
to the
> python
> > I have loaded from a system module. However, I do not know if
MET
> used this
> > when it was built since it is very difficult to match
Anaconda's
> internal
> > compilers with that provided on my HPC system - I know MET is
picky
> things
> > must match. I fear MET was build with some bare bones version
just to
> > accommodate the same compiler. Anyway, then I ran:
> >
> >
> > /usr/local/other/met/10.0.0/bin/plot_data_plane PYTHON_NUMPY
> >
>
/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/metplus_testing_v2/PlotDataPlane_py-
embed_test/plot_data_plane/
> > py_embed_fcst.ps
> >
>
'name="/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_era5pl2xr.py
> > H 20191223_1200 P500";' -title "Python Embedding FCST" -v 5
> >
> > DEBUG 1: Opening data file: PYTHON_NUMPY
> > DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() ->
created new
> > Met2dDataFile object of type "FileType_Python_Numpy".
> > DEBUG 4: VarInfoFactory::new_var_info() -> created new VarInfo
> object of
> > type "FileType_Python_Numpy".
> > DEBUG 3: Calling
> > /usr/local/other/python/GEOSpyD/2019.10_py3.7/2020-01-
15/bin/python3
> to run
> > user's python script
> >
>
(/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_era5pl2xr).
> > Python Script:
> >
>
/usr/local/other/met/10.0.0/share/met/wrappers/write_pickle_dataplane.py
> > User Command:
> >
>
['/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_era5pl2xr',
> > 'H', '20191223_1200', 'P500']
> > Write Pickle: /tmp/tmp_met_pickle_8524_0
> >
> > Parsed program arguments:
> > Namespace(level=500.0, timestamp=datetime.datetime(2019, 12,
23, 12,
> 0),
> > varname='H')
> >
> > file:
> >
>
/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/era5/pres_levels_daily/Y2019/M12/era5_pl-
allvar.20191223_12z.nc4
> > varname: h (H)
> > time index: 0 (2019-12-23 12:00:00)
> > level index: 21 (500.0)
> > Number missing values in original array: 0
> > Number missing values in final64 array: 0
> > met_data obj type: <class 'numpy.ndarray'>
> > met_data dtype: dtype('float64')
> > met_data.shape: (721, 1440)
> > data min/median/max: array([4735.84472656, 5541.72607422,
> 5959.75878906])
> >
> > attrs =
> > {'accum': '00',
> > 'grid': {'Nlat': 721,
> > 'Nlon': 1440,
> > 'delta_lat': 0.25,
> > 'delta_lon': 0.25,
> > 'lat_ll': -90.0,
> > 'lon_ll': -180.0,
> > 'name': 'ERA5_pl',
> > 'type': 'LatLon'},
> > 'init': '20191223_120000',
> > 'lead': '0',
> > 'level': 'P500',
> > 'long_name': 'Geopotential height',
> > 'name': 'h',
> > 'units': 'm',
> > 'valid': '20191223_120000'}
> >
> > {'attrs': {'valid': '20191223_120000', 'init':
'20191223_120000',
> 'lead':
> > '0', 'accum': '00', 'name': 'h', 'long_name': 'Geopotential
height',
> > 'level': 'P500', 'units': 'm', 'grid': {'type': 'LatLon',
'name':
> > 'ERA5_pl', 'lat_ll': -90.0, 'lon_ll': -180.0, 'delta_lat':
0.25,
> > 'delta_lon': 0.25, 'Nlat': 721, 'Nlon': 1440}}, 'met_data':
> > array([[5160.1484375 , 5160.1484375 , 5160.1484375 , ...,
> 5160.1484375 ,
> > 5160.1484375 , 5160.1484375 ],
> > [5156.78320312, 5156.78320312, 5156.7578125 , ...,
> 5156.80908203,
> > 5156.80908203, 5156.80908203],
> > [5153.34179688, 5153.31640625, 5153.29101562, ...,
> 5153.39257812,
> > 5153.3671875 , 5153.3671875 ],
> > ...,
> > [5225.94580078, 5225.89453125, 5225.84375 , ...,
> 5226.12402344,
> > 5226.07324219, 5225.99658203],
> > [5226.02197266, 5225.99658203, 5225.97119141, ...,
> 5226.12402344,
> > 5226.09863281, 5226.07324219],
> > [5225.91992188, 5225.91992188, 5225.91992188, ...,
> 5225.91992188,
> > 5225.91992188, 5225.91992188]])}
> > DEBUG 3: Reading temporary pickle file:
/tmp/tmp_met_pickle_8524_0
> > WARNING:
> > WARNING: pickle_dataplane() -> an error occurred importing
module
> > "read_pickle_dataplane"
> > WARNING:
> > ERROR :
> > ERROR : plot_data_plane -> trouble getting field
> >
>
"name="/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_era5pl2xr.py
> > H 20191223_1200 P500";" from file "PYTHON_NUMPY"
> > ERROR :
> >
> >
> >
> >
> >
> > Does this point to an issue with the python used to build MET
> 10.0.0? Or
> > does this look like something else? My environment python
definitely
> has
> > numpy and my script works fine with that, so I'm guessing this
is
> related
> > to the version of python MET was built with?
> >
> > Assuming this is the case, what do you recommend regarding
> > downloading/getting/building python, or does MET 10.0.0 play
nicely
> with
> > Anaconda now? Any suggestions would be much appreciated.
> >
> > Thanks,
> > Scott
> >
> >
> >
> >
> >
> >
> >
> >
> > On 5/27/21, 6:06 PM, "John Halley Gotway via RT"
<met_help at ucar.edu>
> > wrote:
> >
> > Hi Scott,
> >
> > I see that you're having trouble with python embedding
with the
> > Grid-Stat
> > tool. Thanks for letting us know what software versions
you're
> using
> > and
> > sending sample files to demonstrate.
> >
> > Whenever getting going with using new gridded datasets in
MET,
> > including
> > python embedding, I always recommend starting with the
> plot_data_plane
> > tool. That'll read the input data and create a plot of it,
which
> tells
> > you
> > whether there are any issues with the python script or the
> orientation
> > or
> > location of the data. Based on what you sent, please
trying the
> > following:
> >
> > (1) First run the python script on the command line to
make sure
> it
> > runs
> > without error:
> >
> > *python3
> >
> >
>
/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5fcst2xr.py
> > H 20191221_2100 20191223_1200 P500*
> >
> >
> > (2) Assuming that worked, try running through
plot_data_plane:
> >
> >
> > */usr/local/other/met/10.0.0/bin/plot_data_plane
PYTHON_NUMPY
> plot.ps
> > <
> >
>
https://gcc02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fplot.ps%2F&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7Cc68ec462e70c4d8f13c508d9221d3657%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637578331183212296%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=vvCWf6iJ%2FjfeclJkKJELJynWCoZgo95B%2FCnRWrZ%2FBtc%3D&reserved=0
> > >
> >
> >
>
'name="/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5fcst2xr.py
> > H 20191221_2100 20191223_1200 P500";'*
> >
> >
> > Since grid_stat errored out, I'm going to assume that this
> errors out
> > as
> > well.
> >
> > If so, please try setting the MET_PYTHON_EXE environment
> variable to
> > the
> > full path of python3 that you used in (1).
> > *export MET_PYTHON_EXE=`which python3`*
> >
> > And then retry (2).
> >
> > Does that get you any further? Here's more information
about the
> > MET_PYTHON_EXE variable:
> >
> >
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmet.readthedocs.io%2Fen%2Flatest%2FUsers_Guide%2FappendixF.html%3Fhighlight%3DMET_PYTHON_EXE%23met-
python-
exe&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7Cc68ec462e70c4d8f13c508d9221d3657%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637578331183212296%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Kopgt5XyGGuUmTM5XnJF0goMZWBzG3r9l2WPEuWkPEA%3D&reserved=0
> >
> > Thanks,
> > John Halley Gotway
> >
> >
> > On Thu, May 27, 2021 at 1:51 PM Rabenhorst, Scott D.
> > (GSFC-610.1)[SCIENCE
> > SYSTEMS AND APPLICATIONS INC] via RT <met_help at ucar.edu>
wrote:
> >
> > >
> > > Thu May 27 13:51:19 2021: Request 100025 was acted upon.
> > > Transaction: Ticket created by
scott.d.rabenhorst at nasa.gov
> > > Queue: met_help
> > > Subject: Help to get python embedding working with
> grid_stat
> > > Owner: Nobody
> > > Requestors: scott.d.rabenhorst at nasa.gov
> > > Status: new
> > > Ticket <URL:
> >
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100025&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7Cc68ec462e70c4d8f13c508d9221d3657%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637578331183222253%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=1fNpv2qLOF5bqW%2B4MSzBMXP4VHJGskdG2%2BnCuc32Ct4%3D&reserved=0
> > >
> > >
> > >
> > > Hi metplus support team,
> > >
> > > I am struggling to get a test case using python
embedding to
> work
> > > correctly. The crux of the problem is copied from the
error log
> > (please see
> > > attached tarball of experiment files):
> > >
> > > DEBUG 3: Running user's python script
> > >
> >
>
(/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5fcst2xr).
> > > WARNING:
> > > WARNING: python_dataplane() -> an error occurred
importing
> module
> > >
> >
>
"/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5fcst2xr"
> > > WARNING:
> > > ERROR :
> > > ERROR : Trouble reading data from forecast file
"PYTHON_NUMPY"
> > > ERROR :
> > >
> > > There seems to be an issue passing the numpy array to
> grid_stat, but
> > I
> > > don’t see many more clues regarding what is going wrong.
My
> > environment
> > > appears to be correct by eyeball. When I call the
embedded
> scripts
> > in a
> > > stand-along manor as in
> > > “GridStat_py-
embed_test/logs/.all_commands.20210527141605”,
> all the
> > output
> > > looks correct in these log files:
> > >
> > > GridStat_py-embed_test/read_era5pl2xr.log
> > > GridStat_py-embed_test/read_g5fcst2xr.log
> > >
> > > I should mention I am using METplus-4.0 and MET 10.0.0
as
> indicated
> > in my
> > > log files. I’m not sure what else to try to diagnose the
> problem. Do
> > you
> > > have any ideas I can try?
> > >
> > > Thanks,
> > > Scott
> > >
> > >
> >
> >
> >
> >
> >
>
>
>
>
>
------------------------------------------------
Subject: Re: [EXTERNAL] Re: [rt.rap.ucar.edu #100025] Help to get python embedding working with grid_stat
From: Scott Rabenhorst
Time: Tue Jun 01 05:54:55 2021
Hi John,
Thanks so much for your help. It was only when you mentioned all
references to "pickling" were remove that I thought to look deeper
into
the system module named "met/10.0.0" and discovered this was not
actually the final release version. After building the correct version
myself, my script works. Please feel free to close this ticket. Thanks
again for pointing me in the right direction.
Thanks,
Scott
On 5/28/21 6:00 PM, John Halley Gotway via RT wrote:
> Scott,
>
> We created 5 different development versions on our way toward the
official
> MET version 10.0.0 release. Those development releases (named beta1,
2, 3,
> 4, and 5) are only intended for testing. Stripping out the pickle
logic
> happened after the beta2 release. Before debugging the python
functionality
> any further, I'd recommend compiling/running with the official MET
version
> 10.0.0 release.
>
> Julie Prestopnik is my colleague who's compiled many versions of MET
in
> many places. She uses the compile_MET_all.sh script mentioned on the
> Downloads page:
> https://dtcenter.org/community-code/model-evaluation-tools-
met/download
>
> And that script actually does live in the MET repo
>
<https://github.com/dtcenter/MET/blob/main_v10.0/scripts/installation/compile_MET_all.sh>.
> The MET_PYTHON_CC and MET_PYTHON_LD environment variables at
compilation
> time control the version of python that MET uses at runtime. So
those
> should be set well prior to compiling and you need to use the
> "--enable-python" option when running the "configure" command to
enable
> compiling python support.
>
> Please see appendix F of the user's guide for more info:
> https://met.readthedocs.io/en/latest/Users_Guide/appendixF.html
>
> Thanks,
> John
>
> On Fri, May 28, 2021 at 3:27 PM Rabenhorst, Scott D. (GSFC-
610.1)[SCIENCE
> SYSTEMS AND APPLICATIONS INC] via RT <met_help at ucar.edu> wrote:
>
>> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=100025 >
>>
>> Hi John,
>>
>> I'm trying to figure this out too since I didn't build MET 10.0.0
myself -
>> it is one of our HPC modules. It looks like it was met-10.0.0.0-
beta2 build
>> around DEC 17th of last year. There is a subdirectory
./share/met/wrappers/
>> containing 6 python scripts one of which is
"write_pickle_dataplane.py".
>> I'm guessing this beta version may not be compatible with metplus-
4.0??
>>
>> I've build met before so I can do it again, however, I have always
had
>> issues with Anaconda vs MET as mentioned before. How do you all do
it? Do
>> you grab python source from somewhere and build it as a dependency
for MET?
>> I'm interested in knowing a little more about how to get/build my
python so
>> MET is happy.
>>
>> Thanks,
>> Scott
>>
>> On 5/28/21, 5:11 PM, "John Halley Gotway via RT"
<met_help at ucar.edu>
>> wrote:
>>
>> Scott,
>>
>> I am confused. You state that you're running MET version
10.0.0.
>> However,
>> the log message you sent is puzzling:
>>
>> WARNING: pickle_dataplane() -> an error occurred importing
module
>> "read_pickle_dataplane"
>>
>> All references to "pickle" have been removed from MET version
10.0.0.
>> We
>> have replaced the pickling logic with writing temp files in
NetCDF
>> format.
>> So there literally should be no more instances of the word
"pickle" in
>> the
>> MET codebase (other than in comments).
>>
>> Please check the actual version of the plot_data_plane tool
you're
>> running:
>>
>> /usr/local/other/met/10.0.0/bin/plot_data_plane --version
>>
>> Also please make sure that you do not have the MET_BASE
environment
>> variable set, which could be complicating the debugging, if
so.
>>
>> John
>>
>>
>> On Fri, May 28, 2021 at 2:55 PM Rabenhorst, Scott D.
>> (GSFC-610.1)[SCIENCE
>> SYSTEMS AND APPLICATIONS INC] via RT <met_help at ucar.edu>
wrote:
>>
>> >
>> > <URL:
>>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100025&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7Cc68ec462e70c4d8f13c508d9221d3657%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637578331183212296%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=6qymQ9JzlO%2FSpghqvxF7cMt%2Bqp8%2BjCfb1NU1fMKYy2s%3D&reserved=0
>> >
>> > Hi John,
>> >
>> > Thanks for your quick response. I tried your suggestions
below
>> setting
>> > MET_PYTHON_EXE to an environment variable with the full path
to the
>> python
>> > I have loaded from a system module. However, I do not know
if MET
>> used this
>> > when it was built since it is very difficult to match
Anaconda's
>> internal
>> > compilers with that provided on my HPC system - I know MET
is picky
>> things
>> > must match. I fear MET was build with some bare bones
version just to
>> > accommodate the same compiler. Anyway, then I ran:
>> >
>> >
>> > /usr/local/other/met/10.0.0/bin/plot_data_plane PYTHON_NUMPY
>> >
>>
/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/metplus_testing_v2/PlotDataPlane_py-
embed_test/plot_data_plane/
>> > py_embed_fcst.ps
>> >
>>
'name="/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_era5pl2xr.py
>> > H 20191223_1200 P500";' -title "Python Embedding FCST" -v 5
>> >
>> > DEBUG 1: Opening data file: PYTHON_NUMPY
>> > DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() ->
created new
>> > Met2dDataFile object of type "FileType_Python_Numpy".
>> > DEBUG 4: VarInfoFactory::new_var_info() -> created new
VarInfo
>> object of
>> > type "FileType_Python_Numpy".
>> > DEBUG 3: Calling
>> > /usr/local/other/python/GEOSpyD/2019.10_py3.7/2020-01-
15/bin/python3
>> to run
>> > user's python script
>> >
>>
(/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_era5pl2xr).
>> > Python Script:
>> >
>>
/usr/local/other/met/10.0.0/share/met/wrappers/write_pickle_dataplane.py
>> > User Command:
>> >
>>
['/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_era5pl2xr',
>> > 'H', '20191223_1200', 'P500']
>> > Write Pickle: /tmp/tmp_met_pickle_8524_0
>> >
>> > Parsed program arguments:
>> > Namespace(level=500.0, timestamp=datetime.datetime(2019, 12,
23, 12,
>> 0),
>> > varname='H')
>> >
>> > file:
>> >
>>
/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/era5/pres_levels_daily/Y2019/M12/era5_pl-
allvar.20191223_12z.nc4
>> > varname: h (H)
>> > time index: 0 (2019-12-23 12:00:00)
>> > level index: 21 (500.0)
>> > Number missing values in original array: 0
>> > Number missing values in final64 array: 0
>> > met_data obj type: <class 'numpy.ndarray'>
>> > met_data dtype: dtype('float64')
>> > met_data.shape: (721, 1440)
>> > data min/median/max: array([4735.84472656, 5541.72607422,
>> 5959.75878906])
>> >
>> > attrs =
>> > {'accum': '00',
>> > 'grid': {'Nlat': 721,
>> > 'Nlon': 1440,
>> > 'delta_lat': 0.25,
>> > 'delta_lon': 0.25,
>> > 'lat_ll': -90.0,
>> > 'lon_ll': -180.0,
>> > 'name': 'ERA5_pl',
>> > 'type': 'LatLon'},
>> > 'init': '20191223_120000',
>> > 'lead': '0',
>> > 'level': 'P500',
>> > 'long_name': 'Geopotential height',
>> > 'name': 'h',
>> > 'units': 'm',
>> > 'valid': '20191223_120000'}
>> >
>> > {'attrs': {'valid': '20191223_120000', 'init':
'20191223_120000',
>> 'lead':
>> > '0', 'accum': '00', 'name': 'h', 'long_name': 'Geopotential
height',
>> > 'level': 'P500', 'units': 'm', 'grid': {'type': 'LatLon',
'name':
>> > 'ERA5_pl', 'lat_ll': -90.0, 'lon_ll': -180.0, 'delta_lat':
0.25,
>> > 'delta_lon': 0.25, 'Nlat': 721, 'Nlon': 1440}}, 'met_data':
>> > array([[5160.1484375 , 5160.1484375 , 5160.1484375 , ...,
>> 5160.1484375 ,
>> > 5160.1484375 , 5160.1484375 ],
>> > [5156.78320312, 5156.78320312, 5156.7578125 , ...,
>> 5156.80908203,
>> > 5156.80908203, 5156.80908203],
>> > [5153.34179688, 5153.31640625, 5153.29101562, ...,
>> 5153.39257812,
>> > 5153.3671875 , 5153.3671875 ],
>> > ...,
>> > [5225.94580078, 5225.89453125, 5225.84375 , ...,
>> 5226.12402344,
>> > 5226.07324219, 5225.99658203],
>> > [5226.02197266, 5225.99658203, 5225.97119141, ...,
>> 5226.12402344,
>> > 5226.09863281, 5226.07324219],
>> > [5225.91992188, 5225.91992188, 5225.91992188, ...,
>> 5225.91992188,
>> > 5225.91992188, 5225.91992188]])}
>> > DEBUG 3: Reading temporary pickle file:
/tmp/tmp_met_pickle_8524_0
>> > WARNING:
>> > WARNING: pickle_dataplane() -> an error occurred importing
module
>> > "read_pickle_dataplane"
>> > WARNING:
>> > ERROR :
>> > ERROR : plot_data_plane -> trouble getting field
>> >
>>
"name="/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_era5pl2xr.py
>> > H 20191223_1200 P500";" from file "PYTHON_NUMPY"
>> > ERROR :
>> >
>> >
>> >
>> >
>> >
>> > Does this point to an issue with the python used to build
MET
>> 10.0.0? Or
>> > does this look like something else? My environment python
definitely
>> has
>> > numpy and my script works fine with that, so I'm guessing
this is
>> related
>> > to the version of python MET was built with?
>> >
>> > Assuming this is the case, what do you recommend regarding
>> > downloading/getting/building python, or does MET 10.0.0 play
nicely
>> with
>> > Anaconda now? Any suggestions would be much appreciated.
>> >
>> > Thanks,
>> > Scott
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> > On 5/27/21, 6:06 PM, "John Halley Gotway via RT"
<met_help at ucar.edu>
>> > wrote:
>> >
>> > Hi Scott,
>> >
>> > I see that you're having trouble with python embedding
with the
>> > Grid-Stat
>> > tool. Thanks for letting us know what software versions
you're
>> using
>> > and
>> > sending sample files to demonstrate.
>> >
>> > Whenever getting going with using new gridded datasets
in MET,
>> > including
>> > python embedding, I always recommend starting with the
>> plot_data_plane
>> > tool. That'll read the input data and create a plot of
it, which
>> tells
>> > you
>> > whether there are any issues with the python script or
the
>> orientation
>> > or
>> > location of the data. Based on what you sent, please
trying the
>> > following:
>> >
>> > (1) First run the python script on the command line to
make sure
>> it
>> > runs
>> > without error:
>> >
>> > *python3
>> >
>> >
>>
/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5fcst2xr.py
>> > H 20191221_2100 20191223_1200 P500*
>> >
>> >
>> > (2) Assuming that worked, try running through
plot_data_plane:
>> >
>> >
>> > */usr/local/other/met/10.0.0/bin/plot_data_plane
PYTHON_NUMPY
>> plot.ps
>> > <
>> >
>>
https://gcc02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fplot.ps%2F&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7Cc68ec462e70c4d8f13c508d9221d3657%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637578331183212296%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=vvCWf6iJ%2FjfeclJkKJELJynWCoZgo95B%2FCnRWrZ%2FBtc%3D&reserved=0
>> > >
>> >
>> >
>>
'name="/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5fcst2xr.py
>> > H 20191221_2100 20191223_1200 P500";'*
>> >
>> >
>> > Since grid_stat errored out, I'm going to assume that
this
>> errors out
>> > as
>> > well.
>> >
>> > If so, please try setting the MET_PYTHON_EXE environment
>> variable to
>> > the
>> > full path of python3 that you used in (1).
>> > *export MET_PYTHON_EXE=`which python3`*
>> >
>> > And then retry (2).
>> >
>> > Does that get you any further? Here's more information
about the
>> > MET_PYTHON_EXE variable:
>> >
>> >
>>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmet.readthedocs.io%2Fen%2Flatest%2FUsers_Guide%2FappendixF.html%3Fhighlight%3DMET_PYTHON_EXE%23met-
python-
exe&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7Cc68ec462e70c4d8f13c508d9221d3657%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637578331183212296%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Kopgt5XyGGuUmTM5XnJF0goMZWBzG3r9l2WPEuWkPEA%3D&reserved=0
>> >
>> > Thanks,
>> > John Halley Gotway
>> >
>> >
>> > On Thu, May 27, 2021 at 1:51 PM Rabenhorst, Scott D.
>> > (GSFC-610.1)[SCIENCE
>> > SYSTEMS AND APPLICATIONS INC] via RT <met_help at ucar.edu>
wrote:
>> >
>> > >
>> > > Thu May 27 13:51:19 2021: Request 100025 was acted
upon.
>> > > Transaction: Ticket created by
scott.d.rabenhorst at nasa.gov
>> > > Queue: met_help
>> > > Subject: Help to get python embedding working
with
>> grid_stat
>> > > Owner: Nobody
>> > > Requestors: scott.d.rabenhorst at nasa.gov
>> > > Status: new
>> > > Ticket <URL:
>> >
>>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100025&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7Cc68ec462e70c4d8f13c508d9221d3657%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637578331183222253%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=1fNpv2qLOF5bqW%2B4MSzBMXP4VHJGskdG2%2BnCuc32Ct4%3D&reserved=0
>> > >
>> > >
>> > >
>> > > Hi metplus support team,
>> > >
>> > > I am struggling to get a test case using python
embedding to
>> work
>> > > correctly. The crux of the problem is copied from the
error log
>> > (please see
>> > > attached tarball of experiment files):
>> > >
>> > > DEBUG 3: Running user's python script
>> > >
>> >
>>
(/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5fcst2xr).
>> > > WARNING:
>> > > WARNING: python_dataplane() -> an error occurred
importing
>> module
>> > >
>> >
>>
"/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5fcst2xr"
>> > > WARNING:
>> > > ERROR :
>> > > ERROR : Trouble reading data from forecast file
"PYTHON_NUMPY"
>> > > ERROR :
>> > >
>> > > There seems to be an issue passing the numpy array to
>> grid_stat, but
>> > I
>> > > don’t see many more clues regarding what is going
wrong. My
>> > environment
>> > > appears to be correct by eyeball. When I call the
embedded
>> scripts
>> > in a
>> > > stand-along manor as in
>> > > “GridStat_py-
embed_test/logs/.all_commands.20210527141605”,
>> all the
>> > output
>> > > looks correct in these log files:
>> > >
>> > > GridStat_py-embed_test/read_era5pl2xr.log
>> > > GridStat_py-embed_test/read_g5fcst2xr.log
>> > >
>> > > I should mention I am using METplus-4.0 and MET 10.0.0
as
>> indicated
>> > in my
>> > > log files. I’m not sure what else to try to diagnose
the
>> problem. Do
>> > you
>> > > have any ideas I can try?
>> > >
>> > > Thanks,
>> > > Scott
>> > >
>> > >
>> >
>> >
>> >
>> >
>> >
>>
>>
>>
>>
>>
------------------------------------------------
Subject: Help to get python embedding working with grid_stat
From: John Halley Gotway
Time: Tue Jun 01 08:30:40 2021
Excellent. Thanks for confirming!
John
On Tue, Jun 1, 2021 at 5:55 AM Scott Rabenhorst via RT
<met_help at ucar.edu>
wrote:
>
> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=100025 >
>
> Hi John,
>
> Thanks so much for your help. It was only when you mentioned all
> references to "pickling" were remove that I thought to look deeper
into
> the system module named "met/10.0.0" and discovered this was not
> actually the final release version. After building the correct
version
> myself, my script works. Please feel free to close this ticket.
Thanks
> again for pointing me in the right direction.
>
> Thanks,
> Scott
>
>
> On 5/28/21 6:00 PM, John Halley Gotway via RT wrote:
> > Scott,
> >
> > We created 5 different development versions on our way toward the
> official
> > MET version 10.0.0 release. Those development releases (named
beta1, 2,
> 3,
> > 4, and 5) are only intended for testing. Stripping out the pickle
logic
> > happened after the beta2 release. Before debugging the python
> functionality
> > any further, I'd recommend compiling/running with the official MET
> version
> > 10.0.0 release.
> >
> > Julie Prestopnik is my colleague who's compiled many versions of
MET in
> > many places. She uses the compile_MET_all.sh script mentioned on
the
> > Downloads page:
> > https://dtcenter.org/community-code/model-evaluation-tools-
met/download
> >
> > And that script actually does live in the MET repo
> > <
>
https://github.com/dtcenter/MET/blob/main_v10.0/scripts/installation/compile_MET_all.sh
> >.
> > The MET_PYTHON_CC and MET_PYTHON_LD environment variables at
compilation
> > time control the version of python that MET uses at runtime. So
those
> > should be set well prior to compiling and you need to use the
> > "--enable-python" option when running the "configure" command to
enable
> > compiling python support.
> >
> > Please see appendix F of the user's guide for more info:
> > https://met.readthedocs.io/en/latest/Users_Guide/appendixF.html
> >
> > Thanks,
> > John
> >
> > On Fri, May 28, 2021 at 3:27 PM Rabenhorst, Scott D. (GSFC-
610.1)[SCIENCE
> > SYSTEMS AND APPLICATIONS INC] via RT <met_help at ucar.edu> wrote:
> >
> >> <URL: https://rt.rap.ucar.edu/rt/Ticket/Display.html?id=100025 >
> >>
> >> Hi John,
> >>
> >> I'm trying to figure this out too since I didn't build MET 10.0.0
> myself -
> >> it is one of our HPC modules. It looks like it was met-10.0.0.0-
beta2
> build
> >> around DEC 17th of last year. There is a subdirectory
> ./share/met/wrappers/
> >> containing 6 python scripts one of which is
"write_pickle_dataplane.py".
> >> I'm guessing this beta version may not be compatible with
metplus-4.0??
> >>
> >> I've build met before so I can do it again, however, I have
always had
> >> issues with Anaconda vs MET as mentioned before. How do you all
do it?
> Do
> >> you grab python source from somewhere and build it as a
dependency for
> MET?
> >> I'm interested in knowing a little more about how to get/build my
> python so
> >> MET is happy.
> >>
> >> Thanks,
> >> Scott
> >>
> >> On 5/28/21, 5:11 PM, "John Halley Gotway via RT"
<met_help at ucar.edu>
> >> wrote:
> >>
> >> Scott,
> >>
> >> I am confused. You state that you're running MET version
10.0.0.
> >> However,
> >> the log message you sent is puzzling:
> >>
> >> WARNING: pickle_dataplane() -> an error occurred importing
module
> >> "read_pickle_dataplane"
> >>
> >> All references to "pickle" have been removed from MET
version
> 10.0.0.
> >> We
> >> have replaced the pickling logic with writing temp files in
NetCDF
> >> format.
> >> So there literally should be no more instances of the word
> "pickle" in
> >> the
> >> MET codebase (other than in comments).
> >>
> >> Please check the actual version of the plot_data_plane tool
you're
> >> running:
> >>
> >> /usr/local/other/met/10.0.0/bin/plot_data_plane --version
> >>
> >> Also please make sure that you do not have the MET_BASE
environment
> >> variable set, which could be complicating the debugging, if
so.
> >>
> >> John
> >>
> >>
> >> On Fri, May 28, 2021 at 2:55 PM Rabenhorst, Scott D.
> >> (GSFC-610.1)[SCIENCE
> >> SYSTEMS AND APPLICATIONS INC] via RT <met_help at ucar.edu>
wrote:
> >>
> >> >
> >> > <URL:
> >>
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100025&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7Cc68ec462e70c4d8f13c508d9221d3657%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637578331183212296%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=6qymQ9JzlO%2FSpghqvxF7cMt%2Bqp8%2BjCfb1NU1fMKYy2s%3D&reserved=0
> >> >
> >> > Hi John,
> >> >
> >> > Thanks for your quick response. I tried your suggestions
below
> >> setting
> >> > MET_PYTHON_EXE to an environment variable with the full
path to
> the
> >> python
> >> > I have loaded from a system module. However, I do not know
if MET
> >> used this
> >> > when it was built since it is very difficult to match
Anaconda's
> >> internal
> >> > compilers with that provided on my HPC system - I know MET
is
> picky
> >> things
> >> > must match. I fear MET was build with some bare bones
version
> just to
> >> > accommodate the same compiler. Anyway, then I ran:
> >> >
> >> >
> >> > /usr/local/other/met/10.0.0/bin/plot_data_plane
PYTHON_NUMPY
> >> >
> >>
>
/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/metplus_testing_v2/PlotDataPlane_py-
embed_test/plot_data_plane/
> >> > py_embed_fcst.ps
> >> >
> >>
>
'name="/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_era5pl2xr.py
> >> > H 20191223_1200 P500";' -title "Python Embedding FCST" -v
5
> >> >
> >> > DEBUG 1: Opening data file: PYTHON_NUMPY
> >> > DEBUG 4: Met2dDataFileFactory::new_met_2d_data_file() ->
created
> new
> >> > Met2dDataFile object of type "FileType_Python_Numpy".
> >> > DEBUG 4: VarInfoFactory::new_var_info() -> created new
VarInfo
> >> object of
> >> > type "FileType_Python_Numpy".
> >> > DEBUG 3: Calling
> >> >
> /usr/local/other/python/GEOSpyD/2019.10_py3.7/2020-01-15/bin/python3
> >> to run
> >> > user's python script
> >> >
> >>
>
(/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_era5pl2xr).
> >> > Python Script:
> >> >
> >>
/usr/local/other/met/10.0.0/share/met/wrappers/write_pickle_dataplane.py
> >> > User Command:
> >> >
> >>
>
['/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_era5pl2xr',
> >> > 'H', '20191223_1200', 'P500']
> >> > Write Pickle: /tmp/tmp_met_pickle_8524_0
> >> >
> >> > Parsed program arguments:
> >> > Namespace(level=500.0, timestamp=datetime.datetime(2019,
12, 23,
> 12,
> >> 0),
> >> > varname='H')
> >> >
> >> > file:
> >> >
> >>
>
/discover/nobackup/projects/gmao/g6dev/sdrabenh/validation_data/era5/pres_levels_daily/Y2019/M12/era5_pl-
allvar.20191223_12z.nc4
> >> > varname: h (H)
> >> > time index: 0 (2019-12-23 12:00:00)
> >> > level index: 21 (500.0)
> >> > Number missing values in original array: 0
> >> > Number missing values in final64 array: 0
> >> > met_data obj type: <class 'numpy.ndarray'>
> >> > met_data dtype: dtype('float64')
> >> > met_data.shape: (721, 1440)
> >> > data min/median/max: array([4735.84472656, 5541.72607422,
> >> 5959.75878906])
> >> >
> >> > attrs =
> >> > {'accum': '00',
> >> > 'grid': {'Nlat': 721,
> >> > 'Nlon': 1440,
> >> > 'delta_lat': 0.25,
> >> > 'delta_lon': 0.25,
> >> > 'lat_ll': -90.0,
> >> > 'lon_ll': -180.0,
> >> > 'name': 'ERA5_pl',
> >> > 'type': 'LatLon'},
> >> > 'init': '20191223_120000',
> >> > 'lead': '0',
> >> > 'level': 'P500',
> >> > 'long_name': 'Geopotential height',
> >> > 'name': 'h',
> >> > 'units': 'm',
> >> > 'valid': '20191223_120000'}
> >> >
> >> > {'attrs': {'valid': '20191223_120000', 'init':
'20191223_120000',
> >> 'lead':
> >> > '0', 'accum': '00', 'name': 'h', 'long_name':
'Geopotential
> height',
> >> > 'level': 'P500', 'units': 'm', 'grid': {'type': 'LatLon',
'name':
> >> > 'ERA5_pl', 'lat_ll': -90.0, 'lon_ll': -180.0, 'delta_lat':
0.25,
> >> > 'delta_lon': 0.25, 'Nlat': 721, 'Nlon': 1440}},
'met_data':
> >> > array([[5160.1484375 , 5160.1484375 , 5160.1484375 , ...,
> >> 5160.1484375 ,
> >> > 5160.1484375 , 5160.1484375 ],
> >> > [5156.78320312, 5156.78320312, 5156.7578125 , ...,
> >> 5156.80908203,
> >> > 5156.80908203, 5156.80908203],
> >> > [5153.34179688, 5153.31640625, 5153.29101562, ...,
> >> 5153.39257812,
> >> > 5153.3671875 , 5153.3671875 ],
> >> > ...,
> >> > [5225.94580078, 5225.89453125, 5225.84375 , ...,
> >> 5226.12402344,
> >> > 5226.07324219, 5225.99658203],
> >> > [5226.02197266, 5225.99658203, 5225.97119141, ...,
> >> 5226.12402344,
> >> > 5226.09863281, 5226.07324219],
> >> > [5225.91992188, 5225.91992188, 5225.91992188, ...,
> >> 5225.91992188,
> >> > 5225.91992188, 5225.91992188]])}
> >> > DEBUG 3: Reading temporary pickle file:
> /tmp/tmp_met_pickle_8524_0
> >> > WARNING:
> >> > WARNING: pickle_dataplane() -> an error occurred importing
module
> >> > "read_pickle_dataplane"
> >> > WARNING:
> >> > ERROR :
> >> > ERROR : plot_data_plane -> trouble getting field
> >> >
> >>
>
"name="/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_era5pl2xr.py
> >> > H 20191223_1200 P500";" from file "PYTHON_NUMPY"
> >> > ERROR :
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > Does this point to an issue with the python used to build
MET
> >> 10.0.0? Or
> >> > does this look like something else? My environment python
> definitely
> >> has
> >> > numpy and my script works fine with that, so I'm guessing
this is
> >> related
> >> > to the version of python MET was built with?
> >> >
> >> > Assuming this is the case, what do you recommend regarding
> >> > downloading/getting/building python, or does MET 10.0.0
play
> nicely
> >> with
> >> > Anaconda now? Any suggestions would be much appreciated.
> >> >
> >> > Thanks,
> >> > Scott
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > On 5/27/21, 6:06 PM, "John Halley Gotway via RT" <
> met_help at ucar.edu>
> >> > wrote:
> >> >
> >> > Hi Scott,
> >> >
> >> > I see that you're having trouble with python embedding
with
> the
> >> > Grid-Stat
> >> > tool. Thanks for letting us know what software
versions
> you're
> >> using
> >> > and
> >> > sending sample files to demonstrate.
> >> >
> >> > Whenever getting going with using new gridded datasets
in
> MET,
> >> > including
> >> > python embedding, I always recommend starting with the
> >> plot_data_plane
> >> > tool. That'll read the input data and create a plot of
it,
> which
> >> tells
> >> > you
> >> > whether there are any issues with the python script or
the
> >> orientation
> >> > or
> >> > location of the data. Based on what you sent, please
trying
> the
> >> > following:
> >> >
> >> > (1) First run the python script on the command line to
make
> sure
> >> it
> >> > runs
> >> > without error:
> >> >
> >> > *python3
> >> >
> >> >
> >>
>
/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5fcst2xr.py
> >> > H 20191221_2100 20191223_1200 P500*
> >> >
> >> >
> >> > (2) Assuming that worked, try running through
> plot_data_plane:
> >> >
> >> >
> >> > */usr/local/other/met/10.0.0/bin/plot_data_plane
PYTHON_NUMPY
> >> plot.ps
> >> > <
> >> >
> >>
>
https://gcc02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fplot.ps%2F&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7Cc68ec462e70c4d8f13c508d9221d3657%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637578331183212296%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=vvCWf6iJ%2FjfeclJkKJELJynWCoZgo95B%2FCnRWrZ%2FBtc%3D&reserved=0
> >> > >
> >> >
> >> >
> >>
>
'name="/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5fcst2xr.py
> >> > H 20191221_2100 20191223_1200 P500";'*
> >> >
> >> >
> >> > Since grid_stat errored out, I'm going to assume that
this
> >> errors out
> >> > as
> >> > well.
> >> >
> >> > If so, please try setting the MET_PYTHON_EXE
environment
> >> variable to
> >> > the
> >> > full path of python3 that you used in (1).
> >> > *export MET_PYTHON_EXE=`which python3`*
> >> >
> >> > And then retry (2).
> >> >
> >> > Does that get you any further? Here's more information
about
> the
> >> > MET_PYTHON_EXE variable:
> >> >
> >> >
> >>
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmet.readthedocs.io%2Fen%2Flatest%2FUsers_Guide%2FappendixF.html%3Fhighlight%3DMET_PYTHON_EXE%23met-
python-
exe&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7Cc68ec462e70c4d8f13c508d9221d3657%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637578331183212296%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Kopgt5XyGGuUmTM5XnJF0goMZWBzG3r9l2WPEuWkPEA%3D&reserved=0
> >> >
> >> > Thanks,
> >> > John Halley Gotway
> >> >
> >> >
> >> > On Thu, May 27, 2021 at 1:51 PM Rabenhorst, Scott D.
> >> > (GSFC-610.1)[SCIENCE
> >> > SYSTEMS AND APPLICATIONS INC] via RT
<met_help at ucar.edu>
> wrote:
> >> >
> >> > >
> >> > > Thu May 27 13:51:19 2021: Request 100025 was acted
upon.
> >> > > Transaction: Ticket created by
scott.d.rabenhorst at nasa.gov
> >> > > Queue: met_help
> >> > > Subject: Help to get python embedding working
with
> >> grid_stat
> >> > > Owner: Nobody
> >> > > Requestors: scott.d.rabenhorst at nasa.gov
> >> > > Status: new
> >> > > Ticket <URL:
> >> >
> >>
>
https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frt.rap.ucar.edu%2Frt%2FTicket%2FDisplay.html%3Fid%3D100025&data=04%7C01%7Cscott.d.rabenhorst%40nasa.gov%7Cc68ec462e70c4d8f13c508d9221d3657%7C7005d45845be48ae8140d43da96dd17b%7C0%7C0%7C637578331183222253%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=1fNpv2qLOF5bqW%2B4MSzBMXP4VHJGskdG2%2BnCuc32Ct4%3D&reserved=0
> >> > >
> >> > >
> >> > >
> >> > > Hi metplus support team,
> >> > >
> >> > > I am struggling to get a test case using python
embedding
> to
> >> work
> >> > > correctly. The crux of the problem is copied from
the
> error log
> >> > (please see
> >> > > attached tarball of experiment files):
> >> > >
> >> > > DEBUG 3: Running user's python script
> >> > >
> >> >
> >>
>
(/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5fcst2xr).
> >> > > WARNING:
> >> > > WARNING: python_dataplane() -> an error occurred
importing
> >> module
> >> > >
> >> >
> >>
>
"/gpfsm/dnb78s1/projects/p18/sdrabenh/validation_data/metplus_testing_v2/scripts/read_g5fcst2xr"
> >> > > WARNING:
> >> > > ERROR :
> >> > > ERROR : Trouble reading data from forecast file
> "PYTHON_NUMPY"
> >> > > ERROR :
> >> > >
> >> > > There seems to be an issue passing the numpy array
to
> >> grid_stat, but
> >> > I
> >> > > don’t see many more clues regarding what is going
wrong. My
> >> > environment
> >> > > appears to be correct by eyeball. When I call the
embedded
> >> scripts
> >> > in a
> >> > > stand-along manor as in
> >> > > “GridStat_py-
embed_test/logs/.all_commands.20210527141605”,
> >> all the
> >> > output
> >> > > looks correct in these log files:
> >> > >
> >> > > GridStat_py-embed_test/read_era5pl2xr.log
> >> > > GridStat_py-embed_test/read_g5fcst2xr.log
> >> > >
> >> > > I should mention I am using METplus-4.0 and MET
10.0.0 as
> >> indicated
> >> > in my
> >> > > log files. I’m not sure what else to try to diagnose
the
> >> problem. Do
> >> > you
> >> > > have any ideas I can try?
> >> > >
> >> > > Thanks,
> >> > > Scott
> >> > >
> >> > >
> >> >
> >> >
> >> >
> >> >
> >> >
> >>
> >>
> >>
> >>
> >>
>
>
------------------------------------------------
More information about the Met_help
mailing list